AxoSyslog is a scalable, open source security data processor by Axoflow.
AxoSyslog started as a fork, branched after syslog-ng™ v4.7.1 with the following focus:
Cloud native:
Provides containers, Helm charts, Kubernetes integration, ARM support
Parsing and transformation tailored for security data (FilterX, app-parser, app-transform)
Performance (eBPF support, alternative memory allocator, and so on),
AxoSyslog (created by the original creators of syslog-ng):
is a drop in replacement for syslog-ng,
keeps using the same license and development practices.
If you want to try running syslog-ng in a container, or use it as a log collector in Kubernetes, try AxoSyslog! See Install AxoSyslog to get started.
Feedback
Any feedback is greatly appreciated, especially on what else this document should cover. General comments, errors found in the text, and any suggestions about how to improve the documentation are also welcome as GitHub issues, or on the Axoflow Discord server.
Support
In case you need help with any of the AxoSyslog projects, or directly with syslog-ng, you have several ways to contact us:
In compliance with the above license, syslog-ng is used only to refer to the binary of the application, or in filenames, pathnames, and similar technical terms. For all other references, we use the AxoSyslog name. Syslog-ng is a trademark of One Identity.
New files and modifications compared to the syslog-ng Open Source Edition documentation are under the copyright of Axoflow, and licensed under Apache License 2.0, except for using the Axoflow and AxoSyslog trademarks.
The Axoflow and AxoSyslog trademarks and logos are trademarks of Axoflow Inc. All other trademarks are property of their respective owners.
This documentation is maintained by Axoflow, and continually updated for new releases.
Acknowledgements
We would like to express our gratitude to everyone involved with the project, including the syslog-ng developers, advocates, and documentation maintainers who worked at BalaBit or are still working at One Identity, or who helped the project as external contributors, users, or as part of the community.
1 - What's new
This page is a changelog that collects the major changes and additions to this documentation. (If you want to know the details about why we have separate documentation for AxoSyslog and how it relates to the syslog-ng documentation, read our syslog-ng documentation and similarities with AxoSyslog Core blog post.)
AxoSyslog is a scalable, open source security data processor by Axoflow.
AxoSyslog started as a fork, branched after syslog-ng™ v4.7.1 with the following focus:
Cloud native:
Provides containers, Helm charts, Kubernetes integration, ARM support
Parsing and transformation tailored for security data (FilterX, app-parser, app-transform)
Performance (eBPF support, alternative memory allocator, and so on),
AxoSyslog (created by the original creators of syslog-ng):
is a drop in replacement for syslog-ng,
keeps using the same license and development practices.
The AxoSyslog application is a flexible and highly scalable data processing and system logging application that is ideal for creating centralized and trusted logging and observability solutions and telemetry pipelines, supporting on-premises, cloud-native, and hybrid environments.
Among others, AxoSyslog allows you the following.
Secure and reliable data transfer
The AxoSyslog application enables you to send log messages and other observability data from your hosts to remote servers using the latest protocol standards. You can collect and store your data centrally on dedicated solutions, like log servers or SIEMs. Transfer your log messages and observability data using traditional or modern, reliable protocols (like OpenTelemetry) to ensure that no messages are lost.
Disk-based message buffering
To minimize the risk of losing important log messages, the AxoSyslog application can store messages on the local hard disk if the central log server or the network connection becomes unavailable. The AxoSyslog application automatically sends the stored messages to the server when the connection is reestablished, in the same order the messages were received. The disk buffer is persistent – no messages are lost even if AxoSyslog is restarted.
Secure logging using TLS
Log messages may contain sensitive information that should not be accessed by third parties. Therefore, AxoSyslog supports the Transport Layer Security (TLS) protocol to encrypt the communication. TLS also allows you to authenticate your clients and the logserver using X.509 certificates.
Flexible data extraction and processing
Most log messages are inherently unstructured, which makes them difficult to process. To overcome this problem, AxoSyslog comes with a huge set of built-in parsers, which you can combine to build very complex things.
Filter and classify
The AxoSyslog application can sort the incoming log messages based on their content and various parameters like the source host, application, and priority. You can create directories, files, and database tables dynamically using macros. Complex filtering using logical operators offers almost unlimited flexibility to forward only the important data to the selected destinations.
Parse and rewrite
The AxoSyslog application can segment the incoming data to named fields or columns, and also modify the values of these fields. You can process JSON messages, key-value pairs, and more.
The data that your organization has to process, store, and review increases daily, so many organizations use big data solutions for their logs. To accommodate this huge amount of data, AxoSyslog natively supports storing log messages and observability data in destinations like Elasticsearch/OpenSearch/OpenObserve, and cloud storage solutions like Google BigQuery, Google Pub/Sub and Amazon S3.
Storing your log messages in a database allows you to easily search and query the messages and interoperate with log analyzing applications. The AxoSyslog application supports the following databases: MongoDB, MSSQL, MySQL, Oracle, PostgreSQL, and SQLite.
AxoSyslog also allows you to extract the information you need from your data, and directly send it to your Graphite, Redis, or Riemann monitoring system.
syslog protocol standards
AxoSyslog not only supports legacy BSD syslog (RFC3164) and the enhanced RFC5424 protocols but also JavaScript Object Notation (JSON) and journald message formats.
Heterogeneous on-premises and cloud-native environments
The AxoSyslog application is the ideal choice to collect logs and observability data in massively heterogeneous environments that use several different operating systems. You can also run it in containerized environments.
IPv4 and IPv6 support
The AxoSyslog application can operate in both IPv4 and IPv6 network environments, and can receive and send messages to both types of networks.
What AxoSyslog is not
The AxoSyslog application is not log analysis software. It can filter log messages and select only the ones matching certain criteria. It can even convert the messages and restructure them to a predefined format, or parse the messages and segment them into different fields. But AxoSyslog cannot interpret and analyze the meaning behind the messages, or recognize patterns in the occurrence of different messages.
Why do you need AxoSyslog
Log messages contain information about the events happening on the hosts. Monitoring system events is essential for security and system health monitoring reasons.
The original syslog protocol separates messages based on the priority of the message and the facility sending the message. These two parameters alone are often inadequate to consistently classify messages, as many applications might use the same facility, and the facility itself is not even included in the log message. To make things worse, many log messages contain unimportant information. The AxoSyslog application helps you to select only the really interesting messages, and forward them to a central server.
Company policies or other regulations often require log messages to be archived. Storing the important messages in a central location greatly simplifies this process.
Who uses AxoSyslog
The AxoSyslog application is used worldwide by companies and institutions who collect and manage the logs of several hosts, and want to store them in a centralized, organized way. Using AxoSyslog is particularly advantageous for:
Internet Service Providers
Financial institutions and companies requiring policy compliance
Server, web, and application hosting companies
Datacenters
Wide area network (WAN) operators
Server farm administrators.
2.1 - Supported platforms
The AxoSyslog application supports x86_64 and ARM architechtures on Linux and macOS operating systems.
This chapter provides a very brief introduction into configuring the AxoSyslog application. For details on the format of the configuration file and how to configure sources, destinations, and other features, refer to the subsequent chapters.
To configure AxoSyslog as a relay that receives log messages from client hosts and forwards them to a central logserver, see Configuring AxoSyslog on server hosts.
To configure AxoSyslog on a client host, complete the following steps.
Steps:
Install the AxoSyslog application on the host. For details installing AxoSyslog on specific operating systems, see Install AxoSyslog.
Configure the local sources to collect the log messages of the host. Starting with version 3.2, AxoSyslog automatically collects the log messages that use the native system logging method of the platform, for example, messages from /dev/log on Linux, or /dev/klog on FreeBSD. For a complete list of messages that are collected automatically, see system: Collect the system-specific log messages of a platform.
To configure AxoSyslog, edit the syslog-ng.conf file with any regular text editor application. The location of the configuration file depends on the platform you are running AxoSyslog, and how you have installed AxoSyslog it.
Native packages of a platform (like the ones downloaded from Linux repositories) typically place the configuration file under the /etc/syslog-ng/ directory.
Kubernetes: If you’re running AxoSyslog in Kubernetes and have installed it with helm, usually you configure AxoSyslog by editing a values.yaml file, and redeploying AxoSyslog. Often the syslog-ng.conf part is under the config.raw section in the values.yaml file. For details, see Parameters of the AxoSyslog Helm chart.
Add sources to collect the messages from your log files. File sources look like this:
Note
Many applications send log messages to logfiles by default (for example, the Roundcube webmail client, or the ProFTPD FTP server), but can be configured to send them to syslog instead. If possible, it is recommended to reconfigure the application that way.
Note
The default configuration file of AxoSyslog collects platform-specific log messages and the internal log messages of AxoSyslog.
source s_local { system(); internal();};
Create a network destination that points directly to the AxoSyslog server, or to a local relay. The network destination greatly depends on the protocol that your log server or relay accepts messages. Many systems still use the legacy BSD-syslog protocol (RFC3162) over the unreliable UDP transport:
Create a log statement connecting the local sources to the file destination.
Note
The default configuration of AxoSyslog has only one log statement:
log { source(s_local); destination(d_local);};
Set filters, macros and other features and options (for example, TLS encryption) as necessary.
Example: The default configuration file of AxoSyslog
The following is the default configuration file of AxoSyslog4.8.1. It collects local log messages and the log messages of AxoSyslog and saves them in the /var/log/messages file.
To configure AxoSyslog on a server host, complete the following steps.
Steps:
Install the AxoSyslog application on the host. For details installing AxoSyslog on specific operating systems, see Install AxoSyslog.
Starting with version 3.2, AxoSyslog automatically collects the log messages that use the native system logging method of the platform, for example, messages from /dev/log on Linux, or /dev/klog on FreeBSD. For a complete list of messages that are collected automatically, see system: Collect the system-specific log messages of a platform.
To configure AxoSyslog, edit the syslog-ng.conf file with any regular text editor application. The location of the configuration file depends on the platform you are running AxoSyslog, and how you have installed AxoSyslog it.
Native packages of a platform (like the ones downloaded from Linux repositories) typically place the configuration file under the /etc/syslog-ng/ directory.
Kubernetes: If you’re running AxoSyslog in Kubernetes and have installed it with helm, usually you configure AxoSyslog by editing a values.yaml file, and redeploying AxoSyslog. Often the syslog-ng.conf part is under the config.raw section in the values.yaml file. For details, see Parameters of the AxoSyslog Helm chart.
Configure the network sources that collect the log messages sent by the clients and relays. How the network sources should be configured depends also on the capabilities of your client hosts: many older networking devices support only the legacy BSD-syslog protocol (RFC3164) using UDP transport:
Note
Starting with AxoSyslog version 3.2, the syslog() source driver can handle both BSD-syslog (RFC 3164) and IETF-syslog (RFC 5424-26) messages.
Create local destinations that will store the log messages, for example, file- or program destinations. The default configuration of AxoSyslog places the collected messages into the /var/log/messages file:
Set filters, options (for example, TLS encryption) and other advanced features as necessary.
Note
By default, the AxoSyslog server treats the relayed messages as if they were created by the relay host, not the host that originally sent them to the relay. In order to use the original hostname on the AxoSyslog server, use the keep-hostname(yes) option both on the AxoSyslog relay and the AxoSyslog server. This option can be set individually for every source if needed.
If you are relaying log messages and want to resolve IP addresses to hostnames, configure the first relay to do the name resolution.
Example: A simple configuration for servers
The following is a simple configuration file for AxoSyslog that collects incoming log messages and stores them in a text file.
This section describes how to configure AxoSyslog as a relay.
3.3.1 - Configuring AxoSyslog on relay hosts
To configure AxoSyslog on a relay host, complete the following steps:
Install the AxoSyslog application on the host. For details on installing AxoSyslog on specific operating systems, see Install AxoSyslog.
Configure the network sources that collect the log messages sent by the clients.
Create a network destination that points to the AxoSyslog server.
Create a log statement connecting the network sources to the AxoSyslog server.
Configure the local sources that collect the log messages of the relay host.
Create a log statement connecting the local sources to the AxoSyslog server.
Enable the keep-hostname() and disable the chain-hostnames() options. (For details on how these options work, see Global options.)
Note
It is recommended to use these options on your AxoSyslog server as well.
Set filters and options (for example, TLS encryption) as necessary.
Note
By default, the AxoSyslog server treats the relayed messages as if they were created by the relay host, not the host that originally sent them to the relay. In order to use the original hostname on the AxoSyslog server, use the keep-hostname(yes) option both on the AxoSyslog relay and the AxoSyslog server. This option can be set individually for every source if needed.
If you are relaying log messages and want to resolve IP addresses to hostnames, configure the first relay to do the name resolution.
Example: A simple configuration for relays
The following is a simple configuration file that collects local and incoming log messages and forwards them to a logserver using the IETF-syslog protocol.
Depending on your exact needs about relaying log messages, there are many scenarios and AxoSyslog options that influence how the log message will look like on the logserver. Some of the most common cases are summarized in the following example:
Consider the following example: client-host > relay > server, where the IP address of client-host is 192.168.1.2. The client-host device sends a syslog message to relay. Depending on the settings of relay, the following can happen.
By default, the keep-hostname() option is disabled, so relay writes the IP address of the sender host (in this case, 192.168.1.2) to the HOST field of the syslog message, discarding any IP address or hostname that was originally in the message.
If the keep-hostname() option is enabled on relay, but name resolution is disabled (the use-dns() option is set to no), relay uses the HOST field of the message as-is, which is probably 192.168.1.2.
To resolve the 192.168.1.2 IP address to a hostname on relay using a DNS server, use the keep-hostname(no) and use-dns(yes) options. If the DNS server is properly configured and reverse DNS lookup is available for the 192.168.1.2 address, AxoSyslog will rewrite the HOST field of the log message to client-host.
Note
It is also possible to resolve IP addresses locally, without relying on the DNS server. For details on local name resolution, see Resolving hostnames locally.
The above points apply to the AxoSyslog server (server) as well, so if relay is configured properly, use the keep-hostname(yes) option on server to retain the proper HOST field. Setting keep-hostname(no) on server would result in AxoSyslog rewriting the HOST field to the address of the host that sent the message to server, which is relay in this case.
If you cannot or do not want to resolve the 192.168.1.2 IP address on relay, but want to store your log messages on server using the IP address of the original host (that is, client-host), you can enable the spoof-source() option on relay. However, spoof-source() works only under the following conditions:
The syslog-ng binary has been compiled with the --enable-spoof-source option.
The log messages are sent using the highly unreliable UDP transport protocol. (Extremely unrecommended.)
3.4 - Sending Kubernetes logs to OpenSearch
The following tutorial shows you how to install AxoSyslog on Kubernetes, deploy OpenSearch to the same cluster, and send Kubernetes logs to OpenSearch.
Prerequisites
You need:
A Kubernetes cluster. We used minikube with docker driver and Helm. We used a Ubuntu 22.04 (amd64) machine, but it should work on any system that can run minikube (2 CPUs, 2GB of free memory, 20GB of free disk space).
The OpenSearch service needs a large mmap count setting, so set it to at least 262144, for example:
If you don’t already have an application that generates logs deployed to the Kubernetes cluster, install kube-logging/log-generator to generate sample logs. Complete the following steps.
"kube-logging" has been added to your repositories
Update your repositories.
helm repo update
The output should look like:
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "kube-logging" chart repository
Update Complete. ⎈Happy Helming!⎈
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "opensearch" chart repository
Update Complete. ⎈Happy Helming!⎈
Install OpenSearch. This step can take a few minutes.
NAME: opensearch-dashboards-1684695728
LAST DEPLOYED: Sun May 21 21:02:09 2023NAMESPACE: default
STATUS: deployed
REVISION: 1TEST SUITE: None
NOTES:
1. Get the application URL by running these commands:
exportPOD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=opensearch-dashboards,app.kubernetes.io/instance=opensearch-dashboards-1684695728" -o jsonpath="{.items[0].metadata.name}")exportCONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")echo"Visit http://127.0.0.1:8080 to use your application" kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT
Now you should have 5 pods. Check that they exist:
kubectl get pods
The output should look like:
NAME READY STATUS RESTARTS AGE
log-generator-1681984863-5946c559b9-ftrrn 1/1 Running 0 3m39s
opensearch-cluster-master-0 1/1 Running 0 81s
opensearch-cluster-master-1 1/1 Running 0 81s
opensearch-cluster-master-2 1/1 Running 0 81s
opensearch-dashboards-1681999620-59f64f98f7-bjwwh 1/1 Running 0 44s
Forward the 5601 port of the OpenSearch Dashboards service (replace the name of the pod with your pod).
Forwarding from 127.0.0.1:8080 -> 5601Forwarding from [::1]:8080 -> 5601
Log in to the dashboard at http://localhost:8080 with admin/admin. You will soon create an Index Pattern here, but first you have to send some logs from syslog-ng.
Create a YAML file (called axoflow-demo.yaml in the examples) to configure the collector.
config:sources:kubernetes:# Collect kubernetes logsenabled:truedestinations:# Send logs to OpenSearchopensearch:- address:"opensearch-cluster-master"index:"test-axoflow-index"user:"admin"password:"admin"tls:# Do not validate the server's TLS certificate.peerVerify:false# Send the syslog fields + the metadata from .k8s.* in JSON formattemplate:"$(format-json --scope rfc5424 --exclude DATE --key ISODATE @timestamp=${ISODATE} k8s=$(format-json .k8s.* --shift-levels 2 --exclude .k8s.log))"
Check how the syslog-ng.conf file looks with your custom values:
Open OpenSearch dashboard at http://localhost:8080/app/management/opensearch-dashboards/.
Create an Index Pattern called test-axoflow-index: http://localhost:8080/app/management/opensearch-dashboards/indexPatterns. At Step 2, set the Time field to @timestamp.
Now you can see your logs on the Discover view at http://localhost:8080/app/discover. Opening the detailed view for a log entry shows you the fields sent to OpenSearch.
3.5 - Managing and checking the syslog-ng service on Linux
This section describes how to start, stop and check the status of AxoSyslog service on Linux.
Starting AxoSyslog
To start AxoSyslog, execute the following command as root. For example:
systemctl start syslog-ng
If the service starts successfully, no output will be displayed.
Job for syslog-ng.service failed because the control process exited with error code. See `systemctl status syslog-ng.service` and `journalctl -xe`for details.
The internal logs of AxoSyslog contains informal, warning and error messages.
By default, AxoSyslog log messages (generated on the internal() source) are written to /var/log/messages.
Check the internal logs of AxoSyslog for any issue.
Message processing
The AxoSyslog application collects statistics about the number of processed messages on the different sources and destinations.
Note
When using syslog-ng-ctl stats, consider that while the output is generally consistent, there is no explicit ordering behind the command. Consequently, Axoflow does not recommend creating parsers that depend on a fix output order.
If needed, you can sort the output with an external application, for example, | sort.
Central statistics
To check the central statistics, execute the following command to see the number of received and queued (sent) messages by AxoSyslog.
Edit the unit file as needed for your environment.
We recommend using the default mount points:
Purpose
On the host
In the container
Disk-buffer and persist files
/var/lib/syslog-ng
/var/lib/syslog-ng
syslog-ng configuration file
/opt/axosyslog/etc
/etc/syslog-ng
Output log files
/opt/axosyslog/var/log
/var/log
(Optional) Create an override.conf file to set custom environment values. This can be useful if you don’t want to modify /etc/containers/systemd/axosyslog.container. Run:
systemctl edit axosyslog
Later you can edit this file by running the previous command again.
Create the /opt/axosyslog/etc/syslog-ng.conf configuration file based on the following template.
With the following sample configuration file AxoSyslog collects the local system logs and logs received from the network into the /var/log/messages file.
You can customize the configuration file according to your needs. For a few pointers, see Configuring AxoSyslog on server hosts and the rest of this guide.
Run the following commands to reload the systemd configuration and launch the axosyslog service. Though the systemctl commands are run as root, the container will run as the specified user if set appropriately in the unit file.
If there aren’t any errors, these commands don’t have any output.
Run the following command to verify that the service was properly started:
journalctl -b -u axosyslog | tail -100
The output should be similar to:
Feb 12 09:04:40 <your-hostname> systemd[1]: Starting AxoSyslog Container...
Feb 12 09:04:40 <your-hostname> podman[2783]: 2024-02-12 09:04:40.454665314 -0500 EST m=+0.167732500 system refresh
Feb 12 09:04:40 <your-hostname> axosyslog[2783]: Trying to pull ghcr.io/axoflow/axosyslog:latest...
Feb 12 09:04:40 <your-hostname> axosyslog[2783]: Pulling image //ghcr.io/axoflow/axosyslog:latest inside systemd: setting pull timeout to 5m0s
Feb 12 09:04:41 <your-hostname> axosyslog[2783]: Getting image source signatures
Feb 12 09:04:41 <your-hostname> axosyslog[2783]: Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Feb 12 09:04:41 <your-hostname> axosyslog[2783]: Copying blob sha256:619be1103602d98e1963557998c954c892b3872986c27365e9f651f5bc27cab8
Feb 12 09:04:41 <your-hostname> axosyslog[2783]: Copying blob sha256:b061f41886afb563aff2a5f731f3286ba54ea6f657ed3e282f5339a12a64c5ef
Feb 12 09:04:41 <your-hostname> axosyslog[2783]: Copying blob sha256:1b8d965a650c6a05227bd5c549930c9898071e8e7abb26886d4169a99762de0a
Feb 12 09:04:41 <your-hostname> axosyslog[2783]: Copying blob sha256:b5b0ce6ebef193c4f909379188cfb59443e8a1809816fbb476074908b170b4d1
Feb 12 09:04:50 <your-hostname> axosyslog[2783]: Copying config sha256:c379d94ef2c5ec348dfb3a93eed9a19aed667c396008db85edc354c8f4f8cb6a
Feb 12 09:04:50 <your-hostname> axosyslog[2783]: Writing manifest to image destination
Feb 12 09:04:50 <your-hostname> podman[2783]: 2024-02-12 09:04:50.422390687 -0500 EST m=+10.135457863 container create 477c9f011684f767aae138a0f88602ff30a8c95a46d616bb3b95318ec3a4b79f (image=ghcr.io/axoflow/axosyslog:latest, name=AxoSyslog, org.opencontainers.image.documentation=https://axoflow.com/docs/axosyslog/docs/, org.opencontainers.image.url=https://axoflow.io/, org.opencontainers.image.source=https://github.com/axoflow/axosyslog, org.opencontainers.image.authors=Axoflow, org.opencontainers.image.title=AxoSyslog, org.opencontainers.image.vendor=Axoflow, PODMAN_SYSTEMD_UNIT=axosyslog.service, org.opencontainers.image.description=A cloud-native distribution of syslog-ng by Axoflow, maintainer=axoflow.io, org.opencontainers.image.licenses=GPL-3.0-only)Feb 12 09:04:50 <your-hostname> podman[2783]: 2024-02-12 09:04:50.402626446 -0500 EST m=+10.115693622 image pull c379d94ef2c5ec348dfb3a93eed9a19aed667c396008db85edc354c8f4f8cb6a ghcr.io/axoflow/axosyslog:latest
Feb 12 09:04:50 <your-hostname> podman[2783]: 2024-02-12 09:04:50.489925509 -0500 EST m=+10.202992695 container init 477c9f011684f767aae138a0f88602ff30a8c95a46d616bb3b95318ec3a4b79f (image=ghcr.io/axoflow/axosyslog:latest, name=AxoSyslog, org.opencontainers.image.authors=Axoflow, org.opencontainers.image.licenses=GPL-3.0-only, org.opencontainers.image.vendor=Axoflow, maintainer=axoflow.io, PODMAN_SYSTEMD_UNIT=axosyslog.service, org.opencontainers.image.url=https://axoflow.io/, org.opencontainers.image.documentation=https://axoflow.com/docs/axosyslog/docs/, org.opencontainers.image.title=AxoSyslog, org.opencontainers.image.description=A cloud-native distribution of syslog-ng by Axoflow, org.opencontainers.image.source=https://github.com/axoflow/axosyslog)Feb 12 09:04:50 <your-hostname> systemd[1]: Started AxoSyslog Container.
Feb 12 09:04:50 <your-hostname> podman[2783]: 2024-02-12 09:04:50.500050669 -0500 EST m=+10.213117845 container start 477c9f011684f767aae138a0f88602ff30a8c95a46d616bb3b95318ec3a4b79f (image=ghcr.io/axoflow/axosyslog:latest, name=AxoSyslog, PODMAN_SYSTEMD_UNIT=axosyslog.service, org.opencontainers.image.source=https://github.com/axoflow/axosyslog, org.opencontainers.image.authors=Axoflow, org.opencontainers.image.description=A cloud-native distribution of syslog-ng by Axoflow, org.opencontainers.image.documentation=https://axoflow.com/docs/axosyslog/docs/, org.opencontainers.image.licenses=GPL-3.0-only, org.opencontainers.image.vendor=Axoflow, org.opencontainers.image.title=AxoSyslog, maintainer=axoflow.io, org.opencontainers.image.url=https://axoflow.io/)Feb 12 09:04:50 <your-hostname> axosyslog[2783]: 477c9f011684f767aae138a0f88602ff30a8c95a46d616bb3b95318ec3a4b79f
Feb 12 09:04:50 <your-hostname> AxoSyslog[2821]: [2024-02-12T14:04:50.806054] syslog-ng starting up;version='4.6.0'
Send a test message to the service (requires nc to be installed):
echo'<5> localhost test: this is a test message'| nc localhost 514
Check that the test message has arrived into the log file:
cat /opt/axosyslog/var/log/messages
The output should be similar to:
Feb 19 15:49:12 localhost test: this is a test message
Customize the configuration
To customize the configuration, edit the /opt/axosyslog/etc/syslog-ng.conf file on the host, then reload the service.
Managing the AxoSyslog systemd service
You can reload syslog-ng running in the container via systemctl. The following command reloads the syslog-ng.conf file, without stopping/starting syslog-ng itself.
sudo systemctl reload axosyslog
You can access syslog-ng-ctl from the host, for example by running:
If you use syslog-ng-ctl regularly, you can create the /opt/axosyslog/bin/syslog-ng-ctl file with the following content, make it executable, and add it to your path. That way running syslog-ng-ctl <command> will execute the command in the AxoSyslog container.
The traditional method of starting a service at boot (systemctl enable) is not supported for container services. To automatically start the AxoSyslog service, make sure that the following line is included in the unit file. (It is included in the sample template.)
[Install]WantedBy=default.target
4.2 - Install AxoSyslog on Debian/Ubuntu
You can install AxoSyslog 4.8 and newer on your Debian-based system from Axoflow’s APT repository. AxoSyslog is a drop in replacement for the syslog-ng Debian package, all the AxoSyslog binaries and configuration files are stored at the same place on your system.
The following x86-64 distributions are supported:
Distribution
sources.list component
Debian 12
debian-bookworm
Debian 11
debian-bullseye
Debian Unstable
debian-sid
Debian Testing
debian-testing
Ubuntu 24.04
ubuntu-noble
Ubuntu 23.10
ubuntu-mantic
Ubuntu 23.04
ubuntu-lunar
Ubuntu 22.04
ubuntu-jammy
Ubuntu 20.04
ubuntu-focal
Which package to install?
AxoSyslog supports many features. Some of these, like specific sources and destinations require additional packages that you need only if you’re actually using the specific destination. Therefore, AxoSyslog has a number of modules that you can install as a separate package if you need the particular feature. For example, to use the gRPC-based destinations (like loki() or opentelemetry()), install the axosyslog-grpc-* package. For HTTP-based destinations like elasticsearch-http() or sumologic-http(), you need the axosyslog-http-* package.
Usually, you install the base package axosyslog, and the packages of specific modules that you want to use. We also provide debuginfo packages for every module, but you only need these in certain troubleshooting scenarios.
Steps
To install AxoSyslog from the APT repository, complete the following steps.
Run the following commands to add the APT repository of your distribution (for example, Ubuntu 24.04) to the APT sources list:
echo"deb [signed-by=/usr/share/keyrings/axoflow-code-signing-pub.gpg] https://pkg.axoflow.io/apt stable ubuntu-noble"| tee --append /etc/apt/sources.list.d/axoflow.list
apt update
Note
Nightly builds are also available:
echo"deb [signed-by=/usr/share/keyrings/axoflow-code-signing-pub.gpg] https://pkg.axoflow.io/apt nightly ubuntu-noble"| tee --append /etc/apt/sources.list.d/axoflow.list
Install the AxoSyslog package.
apt install axosyslog
4.3 - Install AxoSyslog on RHEL/Fedora/AlmaLinux
You can install AxoSyslog 4.8 and newer on your RPM-based system using Axoflow’s RPM packages. AxoSyslog is a drop in replacement for the syslog-ng RPM package, all the AxoSyslog binaries and configuration files are stored at the same place on your system.
The following x86-64 distributions are supported:
Red Hat Enterprise Linux (RHEL) 9 / AlmaLinux 9
Red Hat Enterprise Linux (RHEL) 8 / AlmaLinux 8
Fedora 40
Fedora 39
We are working on providing an RPM repository for the upcoming releases.
(The packages for AlmaLinux probably work for Rocky Linux as well, but we haven’t tested it.)
Which package to install?
AxoSyslog supports many features. Some of these, like specific sources and destinations require additional packages that you need only if you’re actually using the specific destination. Therefore, AxoSyslog has a number of modules that you can install as a separate package if you need the particular feature. For example, to use the gRPC-based destinations (like loki() or opentelemetry()), install the axosyslog-grpc-* package. For HTTP-based destinations like elasticsearch-http() or sumologic-http(), you need the axosyslog-http-* package.
Usually, you install the base package axosyslog-<version-number>.<distro>.x86_64.rpm, and the packages of specific modules that you want to use. We also provide debuginfo packages for every module, but you only need these in certain troubleshooting scenarios.
Steps
To install AxoSyslog on RedHat Enterprise Linux 9 or AlmaLinux 9, complete the following steps. The instructions for AlmaLinux probably work for Rocky Linux 9 as well, but we haven’t tested it.
Run the following commands to enable the EPEL repositories for your distribution. This is needed to install some dependencies of AxoSyslog. (For RHEL 8 and compatible distributions, use these instructions.)
Download and extract the release tarball for your distribution, for example, on Almalinux 9:
wget https://github.com/axoflow/axosyslog/releases/download/axosyslog-4.8.1/rpm-almalinux-9.tar.gz
tar -xvzf rpm-almalinux-9.tar.gz
cd rpm-almalinux-9/
Install AxoSyslog:
sudo yum install ./axosyslog-*
Install other packages for the modules you want to use as needed. For example, to use the gRPC-based destinations (like loki() or opentelemetry()), install the axosyslog-grpc-* package. For HTTP-based destinations like elasticsearch-http() or sumologic-http(), you need the axosyslog-http-* package.
(Optional) If you don’t want to run other log collectors on the host, you can delete the existing one (which is rsyslog by default):
sudo yum remove rsyslog.x86_64
Install AxoSyslog on Fedora
Download the release tarball for your distribution, for example, on Fedora 40:
wget https://github.com/axoflow/axosyslog/releases/download/axosyslog-4.8.1/rpm-fedora-40.tar.gz
tar -xvzf rpm-fedora-40.tar.gz
cd rpm-fedora-40/
Install AxoSyslog:
sudo yum install ./axosyslog-4.8.1.2*
Install other packages for the modules you want to use as needed. For example, to use the gRPC-based destinations (like loki() or opentelemetry()), install the axosyslog-grpc-* package. For HTTP-based destinations like elasticsearch-http() or sumologic-http(), you need the axosyslog-http-* package.
(Optional) If you don’t want to run other log collectors on the host, you can delete the existing one (which is rsyslog by default):
sudo yum remove rsyslog.x86_64
Using AxoSyslog
After you’ve installed AxoSyslog, you can configure it just like syslog-ng, using the same configurations files (/etc/syslog-ng/syslog-ng.conf by default). For details, see the Quick-start guide.
Getting help
If you run into any issues while installing or configuring AxoSyslog, or you have any questions, you can find us on our Discord server.
The chart provides parameters that make it easy to deploy AxoSyslog for the following use cases:
As a collector, to collect local logs using the kubernetes() source, and forward them to another syslog server, to an opensearch() node, or to another AxoSyslog node.
to receive RFC3164 and RFC5424 formatted syslog messages from any sender, or syslog-ng-otlp messages from another AxoSyslog node, and then
store them locally, or forward them to remote destinations.
These two use cases are independent from each other and can be configured separately. For other use cases, for example, to use other sources and destinations, you can use the config.raw parameter of the collector or the server. For the list of configurable parameters and their default values, see Parameters of the AxoSyslog Helm chart.
Install
To install the axosyslog chart, complete the following steps.
Release "axosyslog-1713953907" has been upgraded. Happy Helming!
...
Tip: You can retrieve the non-default values of a deployment by running helm get values <name-of-your-axosyslog-deployment>
For the collector use case, configure the destination where the logs are forwarded. For example, the following values file sends the logs in JSON format to the localhost:514 address via TCP:
The generated log messages (like 2024-05-02T10:56:31.000000+00:00 localhost prg00000[1234]: seq: 0000000065, thread: 0000, runid: 1714647391, stamp: 2024-05-02T10:56:31 PADDPADDPADDPADD) should show up in the configured destinations, for example, in the file destination:
kubectl exec axosyslog-1714389625-syslog-0 -- less /var/log/syslog
How to use disk-buffers in containers and Kubernetes
When you are running AxoSyslog in a container or in Kubernetes, and you want to use disk-buffers, there are some additional things to configure.
Make sure to mount the disk-buffer files and the persist file (by default, both are stored in /var/lib/syslog-ng) in a way they are not lost when the pod or container is restarted.
In Kubernetes, add a persistent volume to your pod and store the disk buffer files (/var/lib/syslog-ng) there.
In a container, mount the disk-buffer directory from the host, or store it on a local volume.
Use a reliable disk-buffer only if your storage is fast enough. For example, a low-speed persistent volume in Kubernetes can cause a significant performance degradation for AxoSyslog.
Use the latest available version of AxoSyslog, as many related improvements and performance improvements (for example, disk-buffer related metrics) are only supported in recent versions.
If you are using syslog-ng without disk-buffering configured, syslog-ng stores everything in memory, which results in great performance. If you enable disk-buffering, the performance decreases. Make sure to size your observability pipeline appropriately.
Uninstall
Tip: List all installed releases using helm list.
To uninstall a chart release, run:
helm delete <name-of-the-release-to-delete>
4.4.1 - Parameters of the AxoSyslog Helm chart
The following table lists the configurable parameters of the AxoSyslog collector chart and their default values. For details on installing the chart, see Install AxoSyslog with Helm.
Collector parameters
When you deploy AxoSyslog as a collector (which is a DaemonSet), it collects and forwards local logs to a destination. You can use the following parameters to configure the collector. The parameters for specific destinations are shown in subsequent sections.
Parameter
Description
Default
collector.enabled
Deploy AxoSyslog as a collector to collect and forward local logs
true
collector.config.destinations
The configurations of destinations that can be configured using chart values: syslog, opensearch, and syslogNgOtlp. For destinations and options not available as chart values, you can use the collector.config.raw option.
""
collector.config.raw
A complete syslog-ng configuration. If this parameter is set, all other parameters in the collector.config section are ignored. You can use this to set parameters that are not available as chart values. For details on how to create a configuration for syslog-ng, see the AxoSyslog Core documentation.
""
collector.config.rewrites.set
A list of name-value pairs to set for the collected log messages. Uses the set rewrite rule.
{}
collector.config.sources.kubernetes.enabled
Collect pod logs using the kubernetes() source. If disabled, the chart doesn’t configure any source. For the list of available sources, see the Sources chapter
true
collector.config.sources.kubernetes.prefix
Set JSON prefix for logs collected from the Kubernetes cluster
""
collector.config.sources.kubernetes.keyDelimiter
Set JSON key delimiter for logs collected from the Kubernetes cluster
""
collector.stats.level
Specifies the level of statistics AxoSyslog collects about the processed messages. For details, see (level()).
2
The following example uses the collector.config.raw parameter to configure a custom destination:
A directory containing a set of trusted CA certificates in PEM format. The name of the files must be the 32-bit hash of the subject’s name. AxoSyslog verifies the certificate of the server using these CA certificates.
The CA certificate in PEM format to use when verifying the certificate of the server.
"/path/to/CAFile.pem"
collector.config.destinations.opensearch.tls.Cert
Name of a file containing an X.509 certificate or a certificate chain in PEM format. AxoSyslog authenticates with this certificate on the server, with the private key set in the collector.config.destinations.opensearch.tls.Key field. If the file contains a certificate chain, the file must begin with the certificate of the host, followed by the CA certificate that signed the certificate of the host, and any other signing CAs in order.
"/path/to/Cert.pem"
collector.config.destinations.opensearch.tls.Key
Name of a file containing an unencrypted private key in PEM format. AxoSyslog authenticates with this key and the certificate set in the collector.config.destinations.opensearch.tls.Cert field.
If true, AxoSyslog verifies the certificate of the server with the CA certificates set in collector.config.destinations.opensearch.tls.CAFile and collector.config.destinations.opensearch.tls.CADir.
The maximum number of unavailable pods during a rolling update
1
collector.nodeSelector
Node labels for pod assignment
{}
collector.resources
Resource requests and limits
{}
collector.tolerations
Tolerations for pod assignment
[]
collector.secretMounts
Mount additional secrets as volumes
[]
collector.securityContext
Security context for the pod
{}
Syslog server parameters
When you deploy AxoSyslog as a server (which is a StatefulSet), it receives incoming data from the network and routes it to a local or remote destination. collects and forwards local logs to a destination. You can use the following parameters to configure the syslog server. The parameters for specific destinations are shown in subsequent sections.
Parameter
Description
Default
syslog.enabled
Deploy AxoSyslog as a collector to collect and forward local logs
true
syslog.bufferStorage.enabled
Configures a storage using PersistentVolumes to use as disk-buffer.
false
syslog.bufferStorage.storageClass
The class of the storage to use, for example, standard.
standard
syslog.bufferStorage.size
The maximum size of the storage to use as disk-buffer, for example, 10Gi.
10Gi
syslog.logFileStorage.enabled
Configures a storage using PersistentVolumes to store the log files.
false
syslog.logFileStorage.storageClass
The class of the storage to use, for example, standard.
standard
syslog.logFileStorage.size
The maximum size of the storage to use as for log storage, for example, 10Gi.
500Gi
syslog.config.raw
A complete syslog-ng configuration. If this parameter is set, all other parameters in the syslog.config section are ignored. You can use this to set parameters that are not available as chart values. For details on how to create a configuration for syslog-ng, see the AxoSyslog Core documentation.
""
syslog.config.stats.level
Specifies the detail of statistics AxoSyslog collects about the processed messages. For details, see level().
2
syslog.config.rewrites.set
A list of name-value pairs to set for the collected log messages. Uses the set rewrite rule.
{}
syslog.config.sources
The configurations of the sources that can be configured using chart values: syslog and syslogNgOtlp.
syslog and syslogNgOtlp are enabled by default. See the individual sources for details. For sources not available as chart values, you can use the collector.config.raw option.
The file, syslog, opensearch destinations are enabled by default. For destinations not available as chart values, you can use the collector.config.raw option.
Syslog source
You can use the syslog source to receive RFC3164 or RFC5424 formatted syslog messages on the following ports:
1514: RFC3164-formatted traffic over TCP and UDP (NodePort 30514)
1601: RFC5424-formatted traffic over TCP (NodePort 30601)
6514: RFC5424-formatted traffic over TLS (NodePort 30614)
If needed, you can open additional ports using the service.extraPorts option.
Set to yes to request a certificate from the peers. In this case, you must also set the CA directory or the CA file.
no
syslog.config.sources.syslog.tls.CAFile
A file containing trusted CA certificates. For details, see TLS options.
""
syslog.config.sources.syslog.tls.CADir
The directory for the trusted CA files. For details, see TLS options.
""
syslog.config.sources.syslog.tls.Cert
The certificate file to show to the peer. For details, see TLS options.
""
syslog.config.sources.syslog.tls.Key
The private key file for the certificate. For details, see TLS options.
""
syslogNgOtlp source
Initializes a syslog-ng-otlp() to receive messages from another AxoSyslog node that sends telemetry data using the syslog-ng-otlp() destination driver.
Parameter
Description
Default
syslog.config.sources.syslogNgOtlp.enabled
Enable receiving syslog-ng-otlp() messages.
true
syslog.config.sources.syslogNgOtlp.port
The port where messages are received.
4317
File destination
To write the collected logs into files, configure the syslog.logFileStorage and the syslog.config.destinations.file options.
The template used to format the log messages. Can include macros.
""
syslog.config.destinations.file.extraOptionsRaw
Other options of the file() destination. If the directories used in syslog.destinations.file.path do not exist, set extraOptionsRaw: "create-dirs(yes)"
Name of the OpenSearch index that stores the messages.
"test-axoflow-index"
syslog.config.destinations.opensearch.user
The username to use for authentication on the OpenSearch server, if not authenticating with a certificate.
"admin"
syslog.config.destinations.opensearch.password
The password to use for authentication on the OpenSearch server.
"admin"
syslog.config.destinations.opensearch.template
A template to format the messages.
"$(format-json --scope rfc5424 --exclude DATE --key ISODATE @timestamp=${ISODATE})"
syslog.config.destinations.opensearch.tls.CAFile
The CA certificate in PEM format to use when verifying the certificate of the server.
""
syslog.config.destinations.opensearch.tls.CADir
A directory containing a set of trusted CA certificates in PEM format. The name of the files must be the 32-bit hash of the subject’s name. AxoSyslog verifies the certificate of the server using these CA certificates.
""
syslog.config.destinations.opensearch.tls.Cert
Name of a file containing an X.509 certificate or a certificate chain in PEM format. AxoSyslog authenticates with this certificate on the server, with the private key set in the syslog.config.destinations.opensearch.tls.Key field. If the file contains a certificate chain, the file must begin with the certificate of the host, followed by the CA certificate that signed the certificate of the host, and any other signing CAs in order.
""
syslog.config.destinations.opensearch.tls.Key
Name of a file containing an unencrypted private key in PEM format. AxoSyslog authenticates with this key and the certificate set in the syslog.config.destinations.opensearch.tls.Cert field.
If true, AxoSyslog verifies the certificate of the server with the CA certificates set in syslog.config.destinations.opensearch.tls.CAFile and syslog.config.destinations.opensearch.tls.CADir.
The transport protocol to use. Possible values: tcp, udp
tcp
For example:
syslog:enabled:truebufferStorage:enabled:truestorageClass:standardsize:10Giconfig:sources:syslog:enabled:truedestinations:syslog:enabled:truetransport:tcpaddress:192.168.77.133port:12345# convert incoming data to JSON#template: "$(format-json .*)\n"# use standard syslog logfile#template: "$ISODATE $HOST $MSGHDR$MSG\n"extraOptionsRaw:"time-reopen(10)"
syslogNgOtlp destination
Send data using the syslog-ng-otlp() destination driver to another AxoSyslog node.
You can also use it as a base image in your Dockerfile:
FROM ghcr.io/axoflow/axosyslog:latest
If you want to test a development version, you can use the nightly builds:
podman pull ghcr.io/axoflow/axosyslog:nightly
Note: These named packages are automatically updated when a new package is released. To install a specific version, run podman pull ghcr.io/axoflow/axosyslog:<version-number>, for example:
podman pull ghcr.io/axoflow/axosyslog:4.8.1
Customize the configuration
The AxoSyslog container image stores the configuration file at /etc/syslog-ng/syslog-ng.conf. By default, AxoSyslog collects the local system logs and logs received from the network into the /var/log/messages and /var/log/messages-kv.log files using this configuration file from the syslog-ng repository.
To customize the configuration, create your own configuration file and override the file in the container image with it, for example:
podman run --rm --volume <path-to-your/syslog-ng.conf>:/etc/syslog-ng/syslog-ng.conf ghcr.io/axoflow/axosyslog:latest
How to use disk-buffers in containers and Kubernetes
When you are running AxoSyslog in a container or in Kubernetes, and you want to use disk-buffers, there are some additional things to configure.
Make sure to mount the disk-buffer files and the persist file (by default, both are stored in /var/lib/syslog-ng) in a way they are not lost when the pod or container is restarted.
In Kubernetes, add a persistent volume to your pod and store the disk buffer files (/var/lib/syslog-ng) there.
In a container, mount the disk-buffer directory from the host, or store it on a local volume.
Use a reliable disk-buffer only if your storage is fast enough. For example, a low-speed persistent volume in Kubernetes can cause a significant performance degradation for AxoSyslog.
Use the latest available version of AxoSyslog, as many related improvements and performance improvements (for example, disk-buffer related metrics) are only supported in recent versions.
If you are using syslog-ng without disk-buffering configured, syslog-ng stores everything in memory, which results in great performance. If you enable disk-buffering, the performance decreases. Make sure to size your observability pipeline appropriately.
Expose port to receive incoming traffic
To receive incoming network in a container, you must expose the port from the container where you want to receive the traffic to the host that’s running the container. Typically, this is only needed if you are running AxoSyslog as a relay or a server/aggregator.
By default, the AxoSyslog container images expose the ports commonly used to receive syslog traffic:
514/udp, typically used for RFC3164 (BSD-syslog) formatted traffic.
601/tcp, typically used for RFC5424 (IETF-syslog) formatted traffic.
6514/tcp, typically used for RFC5424 (IETF-syslog) formatted traffic over TLS.
To expose a specific port, use the --expose option when starting the container. Make sure to include the IP address of the host to make the port externally accessible.
For example, if you are receiving OpenTelemetry messages using the opentelemetry() source, expose the 4317 port:
podman run --rm --expose 127.0.0.1:4317:4317/tcp --volume <path-to-your/syslog-ng.conf>:/etc/syslog-ng/syslog-ng.conf ghcr.io/axoflow/axosyslog:latest
4.6 - Install AxoSyslog with Docker
AxoSyslog provides cloud-ready images. These images differ from the upstream syslog-ng images, because:
They’re based on Alpine Linux, instead of Debian testing for reliability and smaller size (thus smaller attack surface).
They incorporate cloud-native features and settings, such as the Kubernetes source.
They incorporate container-level optimizations for better performance and improved security. For example, they use an alternative malloc library.
They support the ARM architecture.
The AxoSyslog images support the following architectures:
You can also use it as a base image in your Dockerfile:
FROM ghcr.io/axoflow/axosyslog:latest
If you want to test a development version, you can use the nightly builds:
docker pull ghcr.io/axoflow/axosyslog:nightly
Note: These named packages are automatically updated when a new package is released. To install a specific version, run docker pull ghcr.io/axoflow/axosyslog:<version-number>, for example:
docker pull ghcr.io/axoflow/axosyslog:4.8.1
Customize the configuration
The AxoSyslog container image stores the configuration file at /etc/syslog-ng/syslog-ng.conf. By default, AxoSyslog collects the local system logs and logs received from the network into the /var/log/messages and /var/log/messages-kv.log files using this configuration file from the syslog-ng repository.
To customize the configuration, create your own configuration file and override the file in the container image with it, for example:
docker run --rm --volume <path-to-your/syslog-ng.conf>:/etc/syslog-ng/syslog-ng.conf ghcr.io/axoflow/axosyslog:latest
How to use disk-buffers in containers and Kubernetes
When you are running AxoSyslog in a container or in Kubernetes, and you want to use disk-buffers, there are some additional things to configure.
Make sure to mount the disk-buffer files and the persist file (by default, both are stored in /var/lib/syslog-ng) in a way they are not lost when the pod or container is restarted.
In Kubernetes, add a persistent volume to your pod and store the disk buffer files (/var/lib/syslog-ng) there.
In a container, mount the disk-buffer directory from the host, or store it on a local volume.
Use a reliable disk-buffer only if your storage is fast enough. For example, a low-speed persistent volume in Kubernetes can cause a significant performance degradation for AxoSyslog.
Use the latest available version of AxoSyslog, as many related improvements and performance improvements (for example, disk-buffer related metrics) are only supported in recent versions.
If you are using syslog-ng without disk-buffering configured, syslog-ng stores everything in memory, which results in great performance. If you enable disk-buffering, the performance decreases. Make sure to size your observability pipeline appropriately.
Expose port to receive incoming traffic
To receive incoming network in a container, you must expose the port from the container where you want to receive the traffic to the host that’s running the container. Typically, this is only needed if you are running AxoSyslog as a relay or a server/aggregator.
By default, the AxoSyslog container images expose the ports commonly used to receive syslog traffic:
514/udp, typically used for RFC3164 (BSD-syslog) formatted traffic.
601/tcp, typically used for RFC5424 (IETF-syslog) formatted traffic.
6514/tcp, typically used for RFC5424 (IETF-syslog) formatted traffic over TLS.
To expose a specific port, use the --expose option when starting the container. Make sure to include the IP address of the host to make the port externally accessible.
For example, if you are receiving OpenTelemetry messages using the opentelemetry() source, expose the 4317 port:
docker run --rm --expose 127.0.0.1:4317:4317/tcp --volume <path-to-your/syslog-ng.conf>:/etc/syslog-ng/syslog-ng.conf ghcr.io/axoflow/axosyslog:latest
5 - The concepts of AxoSyslog
This chapter discusses the technical concepts of AxoSyslog.
5.1 - The philosophy of AxoSyslog
Typically, AxoSyslog is used to manage log messages and implement centralized logging, where the aim is to collect the log messages of several devices on a single, central log server. The different devices — called AxoSyslog clients — all run AxoSyslog, and collect the log messages from the various applications, files, and other sources. The clients send all important log messages to the remote AxoSyslog server, which sorts and stores them.
5.2 - Logging with AxoSyslog
The AxoSyslog application reads incoming messages and forwards them to the selected destinations. The AxoSyslog application can receive messages from files, remote hosts, and other sources.
Log messages enter AxoSyslog in one of the defined sources, and are sent to one or more destinations.
Sources and destinations are independent objects, log paths define what AxoSyslog does with a message, connecting the sources to the destinations. A log path consists of one or more sources and one or more destinations: messages arriving from a source are sent to every destination listed in the log path. A log path defined in AxoSyslog is called a log statement.
Optionally, log paths can include filters. Filters are rules that select only certain messages, for example, selecting only messages sent by a specific application. If a log path includes filters, AxoSyslog sends only the messages satisfying the filter rules to the destinations set in the log path.
Other optional elements that can appear in log statements are parsers and rewriting rules. Parsers segment messages into different fields to help processing the messages, while rewrite rules modify the messages by adding, replacing, or removing parts of the messages.
5.2.1 - The route of a log message in AxoSyslog
Purpose:
The following procedure illustrates the route of a log message from its source on the AxoSyslog client to its final destination on the central AxoSyslog server.
Steps:
A device or application sends a log message to a source on the AxoSyslog client. For example, an Apache web server running on Linux enters a message into the /var/log/apache file.
The AxoSyslog client running on the web server reads the message from its /var/log/apache source.
The AxoSyslog client processes the first log statement that includes the /var/log/apache source.
The AxoSyslog client performs optional operations (message filtering, parsing, and rewriting) on the message, for example, it compares the message to the filters of the log statement (if any). If the message complies with all filter rules, AxoSyslog sends the message to the destinations set in the log statement, for example, to the remote AxoSyslog server.
Warning
Message filtering, parsing, and rewriting is performed in the order that the operations appear in the log statement.
The AxoSyslog client processes the next log statement that includes the /var/log/apache source, repeating Steps 3-4.
The message sent by the AxoSyslog client arrives from a source set in the AxoSyslog server.
The AxoSyslog server reads the message from its source and processes the first log statement that includes that source.
The AxoSyslog server performs optional operations (message filtering, parsing, and rewriting) on the message, for example, it compares the message to the filters of the log statement (if any). If the message complies with all filter rules, AxoSyslog sends the message to the destinations set in the log statement.
Warning
Message filtering, parsing, and rewriting is performed in the order that the operations appear in the log statement.
The AxoSyslog server processes the next log statement, repeating Steps 7-9.
Note
The AxoSyslog application can stop reading messages from its sources if the destinations cannot process the sent messages. This feature is called flow-control and is detailed in Managing incoming and outgoing messages with flow-control.
5.3 - Modes of operation
The AxoSyslog application has three typical operation scenarios: Client, Server, and Relay.
5.3.1 - Client mode
In client mode, AxoSyslog collects the local logs generated by the host and forwards them through a network connection to the central AxoSyslog server or to a relay. Clients often also log the messages locally into files.
5.3.2 - Relay mode
In relay mode, AxoSyslog receives logs through the network from AxoSyslog clients and forwards them to the central AxoSyslog server using a network connection. Relays also log the messages from the relay host into a local file, or forward these messages to the central AxoSyslog server.
Example relay use cases
The relay collects log messages through the network and after processing, but without writing them on the disk for storage, forwards them to one or more remote destinations.
You can use a relay for many different use cases as described in the examples below.
UDP-only source devices
Most network devices send log messages over UDP. However, UDP does not guarantee that all packets are delivered, which makes UDP unreliable.
To ensure at least a best effort level of reliability, Axoflow recommends that you deploy a relay on the network, close to the source devices. With the most reliable hops between the source and the relay, you can minimize the risk of losing UDP packets. Once the packet arrives at the relay, AxoSyslog ensures that the messages are delivered to the central server in a reliable manner.
Too many source devices
Depending on the hardware and configuration, an average instance can usually handle the following number of concurrent connections:
If the maximum message rate is lower than 200,000 messages per second:
maximum ca. 5,000 TCP connections
maximum ca. 1,000 TLS connections
If the message rate is higher than 200,000 messages per second, contact Axoflow.
If you have more source devices, you must deploy a relay machine at least per 5,000 sources and batch up all the logs into a single TCP connection that connects the relay to the server. If TLS is used, deploy relays per 1,000 source devices.
Collecting logs from remote sites (especially over public WAN)
If you need to collect log messages from geographically remote sites or over public WAN, Axoflow recommends that you install at least a relay node per each remote site. The relay can be the last outgoing hop for all the messages of the remote site, which has several benefits:
Maintenance: You only need to change the configuration of the relay if you want to re-route the logs of some or all sources of the remote site. Also you do not need to change each source’s configuration one by one.
Security: If you trust your internal network, it is not necessary to hold encrypted connections within the LAN of the remote site as the messages can get to the relay without encryption. Messages must be sent in an encrypted way over the public WAN, and it is enough to hold only a single TCP/TLS connection between the sites, that is, between the remote relay and the central server. This eliminates the wasting of resources as holding several TLS connections directly from the clients is more costly than holding a single connection from the relay.
Reliability: You can set up a main disk-buffer on the relay. The main disk-buffer is only responsible for buffering all the logs of the remote site if the central AxoSyslog server is temporarily unavailable. It is easier to maintain this single main disk-buffer instead of setting disk-buffers on individual client machines.
Separation, distribution, and balancing of message processing tasks
Most Linux applications have their own human readable, but difficult to handle, log messages. Without parsing and normalization it is difficult to alert and report on these log messages. Many AxoSyslog users use the message parsing tools of AxoSyslog to normalize their different log messages. Just like normalization, filtering can also be resource-heavy, depending on what the filtering is based on. In this case, it might be inefficient to perform all the message processing tasks on the server as it can result in decreased overall performance.
It is a typical setup to deploy relays in front of the central server operating as a receiver front-end. Most resource-heavy tasks, for example, parsing, filtering, and so on, are performed on this receiver layer. As all resource-heavy tasks are performed on the relay, the central server behind it only needs to get the messages from the relay and write them into the final text-based or tamper-proof (logstore) format. Since you can run several relays, you can balance the resource-heavy tasks between more relays, and a single server behind the relays can still be fast enough to write all the messages on the disk.
Acting as a relay also depends on the functionality. A relay does not have to be a dedicated relay machine at all. For log collection, it can be one of the clients with a relay configuration. Note that in a robust log collection infrastructure, the relays have their own purpose, and Axoflow recommends running dedicated relay machines.
You can run several parallel relays to ensure horizontal redundancy. For example, if each of the relays has the same configuration, when one relay goes down another relay can take over the processing. Distribution of the logs can be done by the built-in client-side failover functionality and also by a general load balancer. The load balancer is also used to serve N+1 redundant relay deployments. In this case, switching from one relay to another relay is done when there is an outage but also for real load balancing purposes.
What AxoSyslog relays are not good for
The purpose of the relay is to buffer the logs for short term, for example, a few minutes or a few hours long outages (depending on the log volume). It is not designed to buffer logs generated by the sources during a very long server or connection outage, for example, up to a few days long.
If you expect extended outages, Axoflow recommends that you deploy servers instead of relays. There are many deployments where long term storage and archiving are performed on the central AxoSyslog server, but relays also do short-term log storage.
5.3.3 - Server mode
In server mode, AxoSyslog acts as a central log-collecting server. It receives messages from AxoSyslog clients and relays over the network, and stores them locally in files, or passes them to other applications, for example, log analyzers.
5.4 - Global objects
The AxoSyslog application uses the following objects:
Source driver: A communication method used to receive log messages. For example, AxoSyslog can receive messages from a remote host via TCP/IP, or read the messages of a local application from a file. For details on source drivers, see source: Read, receive, and collect log messages.
Source: A named collection of configured source drivers.
Destination driver: A communication method used to send log messages. For example, AxoSyslog can send messages to a remote host via TCP/IP, or write the messages into a file or database. For details on destination drivers, see destination: Forward, send, and store log messages.
Destination: A named collection of configured destination drivers.
Macro: An identifier that refers to a part of the log message. For example, the ${HOST} macro returns the name of the host that sent the message. Macros are often used in templates and filenames. For details, see Customize message format using macros and templates.
Rewrite rule: A rule modifies a part of the message, for example, replaces a string, or sets a field to a specified value. For details, see Modifying messages using rewrite rules.
Log paths: A combination of sources, destinations, and other objects like filters, parsers, and rewrite rules. The AxoSyslog application sends messages arriving from the sources of the log paths to the defined destinations, and performs filtering, parsing, and rewriting of the messages. Log paths are also called log statements. Log statements can include other (embedded) log statements and junctions to create complex log paths. For details, see log: Filter and route log messages using log paths, flags, and filters.
Template: A template is a set of macros that can be used to restructure log messages or automatically generate file names. For example, a template can add the hostname and the date to the beginning of every log message. For details, see Customize message format using macros and templates.
Option: Options set global parameters of syslog-ng, like the parameters of name resolution and timezone handling. For details, see Global options.
The AxoSyslog application receives the timezone and daylight saving information from the operating system it is installed on. If the operating system handles daylight saving correctly, so does syslog-ng.
The AxoSyslog application supports messages originating from different timezones. The original syslog protocol (RFC3164) does not include timezone information, but syslog-ng provides a solution by extending the syslog protocol to include the timezone in the log messages. The AxoSyslog application also enables administrators to supply timezone information for legacy devices which do not support the protocol extension.
5.5.1 - Assigning timezone to the message
When AxoSyslog receives a message, it assigns timezone information to the message using the following algorithm.
The sender application or host specifies the timezone of the messages. If the incoming message includes a timezone it is associated with the message. Otherwise, the local timezone is assumed.
Specify the time-zone() parameter for the source driver that reads the message. This timezone will be associated with the messages only if no timezone is specified within the message itself. Each source defaults to the value of the recv-time-zone() global option. It is not possible to override only the timezone information of the incoming message, but setting the keep-timestamp() option to no allows AxoSyslog to replace the full timestamp (timezone included) with the time the message was received.
Note
When processing a message that does not contain timezone information, the AxoSyslog application will use the timezone and daylight-saving that was effective when the timestamp was generated. For example, the current time is 2011-03-11 (March 11, 2011) in the EU/Budapest timezone. When daylight-saving is active (summertime), the offset is +02:00. When daylight-saving is inactive (wintertime) the timezone offset is +01:00. If the timestamp of an incoming message is 2011-01-01, the timezone associated with the message will be +01:00, but the timestamp will be converted, because 2011-01-01 meant winter time when daylight saving is not active but the current timezone is +02:00.
Specify the timezone in the destination driver using the time-zone() parameter. Each destination driver might have an associated timezone value: syslog-ng converts message timestamps to this timezone before sending the message to its destination (file or network socket). Each destination defaults to the value of the send-time-zone() global option.
Note
A message can be sent to multiple destination zones. The AxoSyslog application converts the timezone information properly for every individual destination zone.
Warning
If AxoSyslog sends the message is to the destination using the legacy-syslog protocol (RFC3164) which does not support timezone information in its timestamps, the timezone information cannot be encapsulated into the sent timestamp, so AxoSyslog will convert the hour:min values based on the explicitly specified timezone.
If the timezone is not specified, local timezone is used.
When macro expansions are used in the destination filenames, the local timezone is used. (Also, if the timestamp of the received message does not contain the year of the message, AxoSyslog uses the local year.)
Note
You can modify the timezone of the message using timezone-specific rewrite rules. For details, see Rewrite the timezone of a message.
5.5.2 - A note on timezones and timestamps
If the clients run syslog-ng, then use the ISO timestamp, because it includes timezone information. That way you do not need to adjust the recv-time-zone() parameter of syslog-ng.
If you want to output timestamps in Unix (POSIX) time format, use the S_UNIXTIME and R_UNIXTIME macros. You do not need to change any of the timezone related parameters, because the timestamp information of incoming messages is converted to Unix time internally, and Unix time is a timezone-independent time representation. (Actually, Unix time measures the number of seconds elapsed since midnight of Coordinated Universal Time (UTC) January 1, 1970, but does not count leap seconds.)
5.6 - Product licensing
Starting with version 3.2, the AxoSyslog application is licensed under a combined LGPL+GPL license. The core of AxoSyslog is licensed under the GNU Lesser General Public License Version 2.1 license, while the rest of the codebase is licensed under the GNU General Public License Version 2 license.
Note
Practically, the code stored under the lib directory of the source code package is under LGPL, the rest is GPL.
Multiple AxoSyslog servers can be run in fail-over mode. The AxoSyslog application does not include any internal support for this, as clustering support must be implemented on the operating system level. A tool that can be used to create UNIX clusters is Heartbeat (for details, see this page).
Starting with AxoSyslog version 3.2, AxoSyslog clients can be configured to send the log messages to failover servers in case the primary syslog server becomes unaccessible. For details on configuring failover servers, see the description of the failover-servers() destination option in destination: Forward, send, and store log messages.
5.8 - The structure of a log message
The following sections describe the structure of log messages. Currently there are two standard syslog message formats:
This section describes the format of a syslog message, according to the legacy-syslog or BSD-syslog protocol. A syslog message consists of the following parts:
The total message cannot be longer than 1024 bytes.
The following is a sample syslog message
<133>Feb 25 14:09:07 webserver syslogd: restart
The message corresponds to the following format:
<priority>timestamp hostname application: message
The different parts of the message are explained in the following sections.
Note
The AxoSyslog application supports longer messages as well. For details, see the log-msg-size() option in Global options reference. However, it is not recommended to enable messages larger than the packet size when using UDP destinations.
The PRI part of the syslog message (known as Priority value) represents the Facility and Severity of the message. Facility represents the part of the system sending the message, while Severity marks its importance.
PRI formula
The Priority value is calculated using the following formula:
<PRI> =( <facility> * 8) + <severity>
That is, you first multiply the Facility number by 8, and then add the numerical value of the Severity to the multiplied sum.
Example: the correlation between facility value, severity value, and the Priority value in the PRI message part
The following example illustrates a sample syslog message with a sample PRI field (that is, Priority value):
<133> Feb 25 14:09:07 webserver syslogd: restart
In this example, <133> represents the PRI field (Priority value). The syslog message’s Facility value is 16, and the Severity value is 5.
Substituting the numerical values into the <PRI> = ( <facility> * 8) + <severity> formula, the results match the Priority value in our example:
The HEADER message part contains a timestamp and the hostname (without the domain name) or the IP address of the device. The timestamp field is the local time in the Mmm dd hh:mm:ss format, where:
Mmm is the English abbreviation of the month: Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec.
dd is the day of the month on two digits. If the day of the month is less than 10, the first digit is replaced with a space. (for example, Aug 7.)
hh:mm:ss is the local time. The hour (hh) is represented in a 24-hour format. Valid entries are between 00 and 23, inclusive. The minute (mm) and second (ss) entries are between 00 and 59 inclusive.
Note
The AxoSyslog application supports other timestamp formats as well, like ISO, or the PIX extended format. For details, see the ts-format() option in Global options reference.
The MSG part contains the name of the program or process that generated the message, and the text of the message itself. The MSG part is usually in the following format: program[pid]: message text.
5.8.2 - IETF-syslog messages
This section describes the format of a syslog message, according to the IETF-syslog protocol. A syslog message consists of the following parts:
The message was created on 11 October 2003 at 10:14:15pm UTC, 3 milliseconds into the next second.
The message originated from a host that identifies itself as “mymachine.example.com”.
The APP-NAME is “su” and the PROCID is unknown.
The MSGID is “ID47”.
The MSG is “‘su root’ failed for lonvick…”, encoded in UTF-8.
In this example, the encoding is defined by the BOM:
The byte order mark (BOM) is a Unicode character used to signal the byte-order of the message text.
There is no STRUCTURED-DATA present in the message, this is indicated by “-” in the STRUCTURED-DATA field.
The HEADER part of the message must be in plain ASCII format, the parameter values of the STRUCTURED-DATA part must be in UTF-8, while the MSG part should be in UTF-8. The different parts of the message are explained in the following sections.
The PRI message part
The PRI part of the syslog message (known as Priority value) represents the Facility and Severity of the message. Facility represents the part of the system sending the message, while severity marks its importance. The Priority value is calculated by first multiplying the Facility number by 8 and then adding the numerical value of the Severity. The possible facility and severity values are presented below.
Note
Facility codes may slightly vary between different platforms. The AxoSyslog application accepts facility codes as numerical values as well.
Numerical Code
Facility
0
kernel messages
1
user-level messages
2
mail system
3
system daemons
4
security/authorization messages
5
messages generated internally by syslogd
6
line printer subsystem
7
network news subsystem
8
UUCP subsystem
9
clock daemon
10
security/authorization messages
11
FTP daemon
12
NTP subsystem
13
log audit
14
log alert
15
clock daemon
16-23
locally used facilities (local0-local7)
The following table lists the severity values.
Numerical Code
Severity
0
Emergency: system is unusable
1
Alert: action must be taken immediately
2
Critical: critical conditions
3
Error: error conditions
4
Warning: warning conditions
5
Notice: normal but significant condition
6
Informational: informational messages
7
Debug: debug-level messages
syslog Message Severities
The HEADER message part
The HEADER part contains the following elements:
VERSION: Version number of the syslog protocol standard. Currently this can only be 1.
ISOTIMESTAMP: The time when the message was generated in the ISO 8601 compatible standard timestamp format (yyyy-mm-ddThh:mm:ss+-ZONE), for example: 2006-06-13T15:58:00.123+01:00.
HOSTNAME: The machine that originally sent the message.
APPLICATION: The device or application that generated the message
PID: The process name or process ID of the syslog application that sent the message. It is not necessarily the process ID of the application that generated the message.
MESSAGEID: The ID number of the message.
Note
The AxoSyslog application supports other timestamp formats as well, like ISO, or the PIX extended format. The timestamp used in the IETF-syslog protocol is derived from RFC3339, which is based on ISO8601. For details, see the ts-format() option in Global options reference.
The AxoSyslog application will truncate the following fields:
If APP-NAME is longer than 48 characters it will be truncated to 48 characters.
If PROC-ID is longer than 128 characters it will be truncated to 128 characters.
If MSGID is longer than 32 characters it will be truncated to 32 characters.
If HOSTNAME is longer than 255 characters it will be truncated to 255 characters.
The STRUCTURED-DATA message part
The STRUCTURED-DATA message part may contain meta- information about the syslog message, or application-specific information such as traffic counters or IP addresses. STRUCTURED-DATA consists of data blocks enclosed in brackets ([]). Every block includes the ID of the block, and one or more name=value pairs. The AxoSyslog application automatically parses the STRUCTURED-DATA part of syslog messages, which can be referenced in macros (for details, see Macros of AxoSyslog). An example STRUCTURED-DATA block looks like:
The MSG part contains the text of the message itself. The encoding of the text must be UTF-8 if the BOM
The byte order mark (BOM) is a Unicode character used to signal the byte-order of the message text.
character is present in the message. If the message does not contain the BOM character, the encoding is treated as unknown. Usually messages arriving from legacy sources do not include the BOM character. CRLF characters will not be removed from the message.
5.8.3 - Enterprise-wide message model (EWMM)
The following section describes the structure of log messages using the Enterprise-wide message model or EWMM message format.
The Enterprise-wide message model or EWMM allows you to deliver structured messages from the initial receiving AxoSyslog component right up to the central log server, through any number of hops. It does not matter if you parse the messages on the client, on a relay, or on the central server, their structured results will be available where you store the messages. Optionally, you can also forward the original raw message as the first AxoSyslog component in your infrastructure has received it, which is important if you want to forward a message for example, to a SIEM system. To make use of the enterprise-wide message model, you have to use the syslog-ng() destination on the sender side, and the default-network-drivers() source on the receiver side.
The following is a sample log message in EWMM format.
<13>1 2018-05-13T13:27:50.993+00:00 my-host @syslog-ng - - -
{"MESSAGE":"<34>Oct 11 22:14:15 mymachine su: 'su root' failed for username on
/dev/pts/8","HOST_FROM":"my-host","HOST":"my-host","FILE_NAME":"/tmp/in","._TAGS":".source.s_file"}
The message has the following parts:
The header of the complies with the RFC5424 message format, where the PROGRAM field is set to @syslog-ng, and the SDATA field is empty.
The MESSAGE part is in JSON format, and contains the actual message, as well as any name-value pairs that AxoSyslog has attached to or extracted from the message. The ${._TAGS} field contains the identifier of the AxoSyslog source that has originally received the message on the first AxoSyslog node.
When the AxoSyslog application receives a message, it automatically parses the message. The AxoSyslog application can automatically parse log messages that conform to the RFC3164 (BSD or legacy-syslog) or the RFC5424 (IETF-syslog) message formats. If AxoSyslog cannot parse a message, it results in an error.
Note
In case you need to relay messages that cannot be parsed without any modifications or changes, use the flags(no-parse) option in the source definition, and a template containing only the ${MESSAGE} macro in the destination definition.
To parse non-syslog messages, for example, JSON, CSV, or other messages, you can use the built-in parsers of AxoSyslog. For details, see parser: Parse and segment structured messages.
A parsed syslog message has the following parts:
Timestamps
Two timestamps are associated with every message: one is the timestamp contained within the message (that is, when the sender sent the message), the other is the time when AxoSyslog has actually received the message.
Severity
The severity of the message.
Facility
The facility that sent the message.
Tags
Custom text labels added to the message that are mainly used for filtering. None of the current message transport protocols adds tags to the log messages. Tags can be added to the log message only within AxoSyslog. The AxoSyslog application automatically adds the id of the source as a tag to the incoming messages. Other tags can be added to the message by the pattern database, or using the tags() option of the source.
IP address of the sender
The IP address of the host that sent the message. Note that the IP address of the sender is a hard macro and cannot be modified within AxoSyslog but the associated hostname can be modified, for example, using rewrite rules.
Hard macros
Hard macros contain data that is directly derived from the log message, for example, the ${MONTH} macro derives its value from the timestamp. The most important consideration with hard macros is that they are read-only, meaning they cannot be modified using rewrite rules or other means.
Soft macros
Soft macros (sometimes also called name-value pairs) are either built-in macros automatically generated from the log message (for example, ${HOST}), or custom user-created macros generated by using the AxoSyslog pattern database or a CSV-parser. The SDATA fields of RFC5424-formatted log messages become soft macros as well. In contrast with hard macros, soft macros are writable and can be modified within AxoSyslog, for example, using rewrite rules.
Note
It is also possible to set the value of built-in soft macros using parsers, for example, to set the ${HOST} macro from the message using a column of a CSV-parser.
The data extracted from the log messages using named pattern parsers in the pattern database are also soft macros.
Internally, AxoSyslog represents every message as UTF-8. The maximal length of the log messages is limited by the log-msg-size() option: if a message is longer than this value, AxoSyslog truncates the message at the location it reaches the log-msg-size() value, and discards the rest of the message.
When encoding is set in a source (using the encoding() option) and the message is longer (in bytes) than log-msg-size() in UTF-8 representation, AxoSyslog splits the message at an undefined location (because the conversion between different encodings is not trivial).
5.10 - Structuring macros, metadata, and other value-pairs
Available in AxoSyslog 3.3 and later.
The AxoSyslog application allows you to select and construct name-value pairs from any information already available about the log message, or extracted from the message itself. You can directly use this structured information, for example, in the following places:
Prior to version 4.0, AxoSyslog handled every data as strings, and allowed you to convert the strings into other types of data that only certain destinations data formats supported. For example, SQL, MongoDB, JSON, or AMQP support data types like numbers or dates. The AxoSyslog application allows you to specify the data type in templates (this is also called type-hinting or type-casting). If the destination driver supports data types, AxoSyslog converts the incoming data to the specified data type. For example, this allows you to store integer numbers as numbers in MongoDB, instead of strings.
Starting with AxoSyslog 4.0, each name-value pair is a (name, type, value) triplet, and several components of AxoSyslog have typing support, for example, json-parser() and the $(format-json) template function. For details, see the list of supported data types.
Using explicit type-hinting
You can explicitly type-cast a AxoSyslog template to a specific type. To use type-hinting, enclose the macro or template containing the data with the type: <datatype>("<macro>"), for example: int("$PID"). See the Type-hinting examples and the list of supported data types for details.
Warning
Hazard of data loss! If AxoSyslog cannot convert the data into the specified type, an error occurs, and AxoSyslog drops the message by default. To change how AxoSyslog handles data-conversion errors, see Global options.
The following example stores the MESSAGE, PID, DATE, and PROGRAM fields of a log message in a MongoDB database. The DATE and PID parts are stored as numbers instead of strings.
The following example formats the MESSAGE field as a JSON list.
$(format-json message=list($MESSAGE))"
Data types in AxoSyslog
The AxoSyslog application currently supports the following data-types.
boolean: Converts the data to a boolean value. Anything that begins with a t or 1 is converted to true, anything that begins with an f or 0 is converted to false.
datetime: Use it only with UNIX timestamps, anything else will likely result in an error. This means that currently you can use only the $UNIXTIME macro for this purpose.
double: A floating-point number.
json: A JSON snippet. (Available in AxoSyslog 4.0 and later.)
integer: A 32-bit or 64-bit integer, determined by the destination. For example, mongodb uses int32 if the number is less than MAXINT32 and int64 otherwise.
string: The data as a string.
Components that support data types
In AxoSyslog 4.0 and later, the following AxoSyslog components that support data types. Other components treat every data as strings.
Comparisons in filter expressions: the previously numeric operators are type-aware. The exact comparison depends on the types associated with the values you compare. For details, see Comparing macro values in filters.
When using the json-parser(), AxoSyslog converts all elements of the JSON object to name-value pairs. Any type information originally present in the incoming JSON object is retained, and automatically propagated to other AxoSyslog components (for example, a destination) if they support types.
Elements without a type are treated as strings.
JSON lists (arrays) are converted to AxoSyslog lists, so you can manipulate them using the $(list-*) template functions.
You can set the type of the field. Where you can use of templates in set() and groupset(), you can use type-casting, and the type information is properly promoted. For details, see Specifying data types in value-pairs.
db-parser(): The db-parser() rules can associate types with values using the "type" attribute, for example:
<valuename="foobar"type="integer">$PID</value>
The integer is a type-cast that associates $foobar with an integer type. db-parser()’s internal parsers (for example, @NUMBER@) automatically associate type information to the parsed name-value pair.
add-contextual-data(): Name-value pairs that are populated using add-contextual-data() propagate type information, similarly to db-parser().
SQL type support: The sql() driver supports types, so that columns with specific types are stored as those types.
Template type support: You can cast templates explicitly to a specific type. Templates also propagate type information from macros, template functions, and values in the template string.
python() typing: All Python components (sources, destinations, parsers, and template functions) support all data types, except json().
On-disk serialized formats (that is, disk buffer): Version 4.0 and newer are compatible with messages serialized with an earlier version, and the format is compatible for downgrades as well. This means that even if a newer version of AxoSyslog serialized a message, older versions and associated tools are able to read it (but drop the type information of course).
5.10.2 - value-pairs()
Type:
parameter list of the value-pairs() option
Default:
empty string
Description: The value-pairs() option allows you to select specific information about a message easily using predefined macro groups. The selected information is represented as name-value pairs and can be used formatted to JSON format, or directly used in a mongodb() destination.
Example: Using the value-pairs() option
The following example selects every available information about the log message, except for the date-related macros (R_* and S_*), selects the .SDATA.meta.sequenceId macro, and defines a new value-pair called MSGHDR that contains the program name and PID of the application that sent the log message.
Note
Every macro is included in the selection only once, but redundant information may appear if multiple macros include the same information (for example, including several date-related macros in the selection).
value-pairs() parameters
The value-pairs() option has the following parameters. The parameters are evaluated in the following order:
Space-separated list of macros to remove from the selection created using the scope() option.
Default:
empty string
Description: This option removes the specified macros from the selection. Use it to remove unneeded macros selected using the scope() parameter. For example, the following example removes the SDATA macros from the selection.
The name of the macro to remove can include wildcards (*, ?). Regular expressions are not supported.
key()
Type:
Space-separated list of macros to be included in the selection.
Default:
empty string
Description: This option selects the specified macros. The selected macros will be included as MACRONAME = MACROVALUE, that is using key("HOST") will result in HOST = $HOST. You can use wildcards (*, ?) to select multiple macros. For example:
Description: If this option is specified, AxoSyslog does not include value-pairs with empty values in the output. For example: $(format-json --scope none --omit-empty-values) or
Description: This option defines a new name-value pair to be included in the message. The value part can include macros, templates, and template functions as well. For example:
<pattern-to-select-names>, <list of transformations>
Default:
empty string
Description: This option allows you to manipulate and modify the name of the value-pairs. You can define transformations, which are are applied to the selected name-value pairs. The first parameter of the rekey() option is a glob pattern that selects the name-value pairs to modify. If you omit the pattern, the transformations are applied to every key of the scope. For details on globs, see glob.
Replaces a substring at the beginning of the key with another string. Only prefixes can be replaced. For example, replace-prefix(".class", ".patterndb") changes the beginning tag .class to .patterndb
This option was called replace() in AxoSyslog version 3.4.
lower
Convert all keys to lowercase. Only supports US ASCII.
shift("<number>")
Cuts the specified number of characters from the beginning of the name.
shift-levels("<number>")
Similar to –shift, but instead of cutting characters, it cuts dot-delimited “levels” in the name (including the initial dot). For example, --shift-levels 2 deletes the prefix up to the second dot in the name of the key: .iptables.SRC becomes SRC
upper
Convert all keys to uppercase. Only supports US ASCII.
Example: Using the rekey() option
The following sample selects every value-pair that begins with .cee., deletes this prefix by cutting 4 characters from the names, and adds a new prefix (events.).
Space-separated list of macro groups to be included in the selection.
Default:
empty string
Description: This option selects predefined groups of macros. The following groups are available:
nv-pairs: Every soft macro (name-value pair) associated with the message, except the ones that start with a dot (.) character. Macros starting with a dot character are generated within AxoSyslog and are not originally part of the message, therefore are not included in this group.
dot-nv-pairs: Every soft macro (name-value pair) associated with the message which starts with a dot (.) character. For example, .classifier.rule_id and .sdata.*. Macros starting with a dot character are generated within AxoSyslog and are not originally part of the message.
all-nv-pairs: Include every soft macro (name-value pair). Equivalent to using both nv-pairs and dot-nv-pairs.
rfc3164: The macros that correspond to the RFC3164 (legacy or BSD-syslog) message format: $FACILITY, $PRIORITY, $HOST, $PROGRAM, $PID, $MESSAGE, and $DATE.
rfc5424: The macros that correspond to the RFC5424 (IETF-syslog) message format: $FACILITY, $PRIORITY, $HOST, $PROGRAM, $PID, $MESSAGE, $MSGID, $R_DATE, and the metadata from the structured-data (SDATA) part of RFC5424-formatted messages, that is, every macro that starts with .SDATA..
The rfc5424 group also has the following alias: syslog-proto. Note that the value of $R_DATE will be listed under the DATE key.
The rfc5424 group does not contain any metadata about the message, only information that was present in the original message. To include the most commonly used metadata (for example, the $SOURCEIP macro), use the selected-macros group instead.
all-macros: Include every hard macro. This group is mainly useful for debugging, as it contains redundant information (for example, the date-related macros include the date-related information several times in various formats).
selected-macros: Include the macros of the rfc3164 groups, and the most commonly used metadata about the log message: the $TAGS, $SOURCEIP, and $SEQNUM macros.
sdata: The metadata from the structured-data (SDATA) part of RFC5424-formatted messages, that is, every macro that starts with .SDATA.
everything: Include every hard and soft macros. This group is mainly useful for debugging, as it contains redundant information (for example, the date-related macros include the date-related information several times in various formats).
none: Reset previously added scopes, for example, to delete automatically-added name-value pairs. The following example deletes every value-pair from the scope, and adds only the ones starting with iptables: $(format-welf --scope none .iptables.*)
For example:
value-pairs( scope(rfc3164 selected-macros))
5.11 - Things to consider when forwarding messages between AxoSyslog hosts
When you send your log messages from a AxoSyslog client through the network to a AxoSyslog server, you can use different protocols and options. Every combination has its advantages and disadvantages. The most important thing is to use matching protocols and options, so the server handles the incoming log messages properly.
In AxoSyslog you can change many aspects of the network communication. First of all, there is the structure of the messages itself. Currently, AxoSyslog supports two standard syslog protocols: the BSD (RFC3164) and the syslog (RFC5424) message format.
These RFCs describe the format and the structure of the log message, and add a (lightweight) framing around the messages. You can set this framing/structure by selecting the appropriate driver in AxoSyslog. There are two drivers you can use: the network() driver and the syslog() driver. The syslog() driver is for the syslog (RFC5424) protocol and the network() driver is for the BSD (RFC3164) protocol.
The tcp() and udp() drivers are now deprecated, they are essentially equivalent with the network(transport(tcp)) and network(transport(udp)) drivers.
In addition to selecting the driver to use, both drivers allow you to use different transport-layer protocols: TCP and UDP, and optionally also higher-level transport protocols: TLS (over TCP. To complicate things a bit more, you can configure the network() driver (corresponding to the BSD (RFC3164) protocol) to send the messages in the syslog (RFC5424) format (but without the framing used in RFC5424) using the flag(syslog-protocol) option.
Because some combination of drivers and options are invalid, you can use the following drivers and options as sources and as destinations:
If you use the same driver and options in the destination of your AxoSyslog client and the source of your AxoSyslog server, everything should work as expected. Unfortunately there are some other combinations, that seem to work, but result in losing parts of the messages. The following table show the combinations:
Source \ Destination
syslog/tcp
syslog/udp
syslog/tls
network/tcp
network/udp
network/tls
network/tcp/flag
network/udp/flag
network/tls/flag
syslog/tcp
✔
-
-
!
-
-
!
-
-
syslog/udp
-
✔
-
-
!
-
-
!
-
syslog/tls
-
-
✔
-
-
!
-
-
!
network/tcp
-
-
-
✔
-
-
✔?
-
-
network/udp
-
✔?
-
-
✔
-
-
✔?
-
network/tls
-
-
-
-
-
✔
-
-
✔?
network/tcp/flag
!
-
-
!
-
-
✔
-
-
network/udp/flag
-
!
-
-
!
-
-
✔
-
network/tls/flag
-
-
!
-
-
!
-
-
✔
Source-destination driver combinations
- This method does not work. The logs will not get to the server.
✔ This method works.
! This method has some visible drawbacks. The logs go through, but some of the values are missing/misplaced/and so on.
✔? This method seems to work, but it is not recommended because this can change in a future release.
6 - The configuration file
6.1 - Location of the configuration file
To configure AxoSyslog, edit the syslog-ng.conf file with any regular text editor application. The location of the configuration file depends on the platform you are running AxoSyslog, and how you have installed AxoSyslog it.
Native packages of a platform (like the ones downloaded from Linux repositories) typically place the configuration file under the /etc/syslog-ng/ directory.
Kubernetes: If you’re running AxoSyslog in Kubernetes and have installed it with helm, usually you configure AxoSyslog by editing a values.yaml file, and redeploying AxoSyslog. Often the syslog-ng.conf part is under the config.raw section in the values.yaml file. For details, see Parameters of the AxoSyslog Helm chart.
6.2 - The configuration syntax in detail
Every syslog-ng.conf configuration file must begin with a line containing version information. For syslog-ng version 4.8, this line looks like:
@version: 4.8.1
If the configuration file does not contain the version information, syslog-ng assumes that the file is for version 2.x. In this case it interprets the configuration and sends warnings about the parts of the configuration that should be updated. Version 3.0 and later can operate with configuration files of version 2.x, but the default values of certain parameters have changed since 3.0.
@version: current sets the configuration version to the currently installed version.
Example: A simple configuration file
The following is a very simple configuration file for syslog-ng: it collects the internal messages of syslog-ng and the messages from /dev/log into the /var/log/messages_syslog-ng.log file.
The syslog-ng’s config file format was written by programmers for programmers to be understood by programmers. That may not have been the stated intent, but it is how things turned out. The syntax is exactly that of C, all the way down to braces and statement terminators.
The main body of the configuration file consists of object definitions: sources, destinations, logpaths define which log message are received and where they are sent. All identifiers, option names and attributes, and any other strings used in the syslog-ng.conf configuration file are case sensitive. Object definitions (also called statements) have the following syntax:
Type of the object: One of source, destination, log, filter, parser, rewrite rule, or template.
Identifier of the object: A unique name identifying the object. When using a reserved word as an identifier, enclose the identifier in quotation marks.
All identifiers, attributes, and any other strings used in the syslog-ng.conf configuration file are case sensitive.
Use identifiers that refer to the type of the object they identify. For example, prefix source objects with s_, destinations with d_, and so on.
Repeating a definition of an object (that is, defining the same object with the same id more than once) is not allowed, unless you use the @define allow-config-dups 1 definition in the configuration file.
Parameters: The parameters of the object, enclosed in braces {parameters}.
Semicolon: Object definitions end with a semicolon (;).
For example, the following line defines a source and calls it s_internal.
source s_internal { internal();};
The object can be later referenced in other statements using its ID, for example, the previous source is used as a parameter of the following log statement:
log { source(s_internal); destination(d_file);};
The parameters and options within a statement are similar to function calls of the C programming language: the name of the option followed by a list of its parameters enclosed within brackets and terminated with a semicolon.
For example, the file() driver in the following source statement has three options: the filename (/var/log/apache/access.log), follow-freq(), and flags(). The follow-freq() option also has a parameter, while the flags() option has two parameters.
Objects may have required and optional parameters. Required parameters are positional, meaning that they must be specified in a defined order. Optional parameters can be specified in any order using the option(value) format. If a parameter (optional or required) is not specified, its default value is used. The parameters and their default values are listed in the reference section of the particular object.
Example: Using required and optional parameters
The unix-stream() source driver has a single required argument: the name of the socket to listen on. Optional parameters follow the socket name in any order, so the following source definitions have the same effect:
Some options are global options, or can be set globally, for example, whether AxoSyslog should use DNS resolution to resolve IP addresses. Global options are detailed in Global options.
options { use-dns(no);};
Objects can be used before definition.
Objects can be defined inline as well. This is useful if you use the object only once (for example, a filter). For details, see Defining configuration objects inline.
To add comments to the configuration file, start a line with # and write your comments. These lines are ignored by syslog-ng.
# Comment: This is a stream sourcesource s_demo_stream { unix-stream("<path-to-socket>" max-connections(10) group(log));};
Note
Before activating a new configuration, check that your configuration file is syntactically correct using the syslog-ng --syntax-only command.
To activate the configuration, reload the configuration using the /etc/init.d/syslog-ng reload command.
6.3 - Notes about the configuration syntax
When you are editing the syslog-ng.conf configuration file, note the following points:
The configuration file can contain a maximum of 6665 source / destination / log elements.
When writing the names of options and parameters (or other reserved words), the hyphen (-) and underscore (_) characters are equivalent, for example, max-connections(10) and max_connections(10) are both correct.
Numbers can be prefixed with + or - to indicate positive or negative values. Numbers beginning with zero (0) or 0x are treated as octal or hexadecimal numbers, respectively.
Starting with AxoSyslog version 3.5, you can use suffixes for kilo-, mega-, and gigabytes. Use the Kb, Mb, or Gb suffixes for the base-10 version, and Kib, Mib, or Gib for the base-2 version. That is, 2MB means 2000000, while 2MiB means 2097152. For example, to set the log-msg-size() option to 2000000 bytes, use log-msg-size(2Mb).
You can use commas (,) to separate options or other parameters for readability, AxoSyslog completely ignores them. The following declarations are equivalent:
When enclosing object IDs (for example, the name of a destination) between double-quotes ("mydestination"), the ID can include whitespace as well, for example:
Starting with AxoSyslog 3.4, you can define configuration objects inline, where they are actually used, without having to define them in a separate placement. This is useful if you need an object only once, for example, a filter or a rewrite rule. Every object can be defined inline: sources, destinations, filters, parsers, rewrite rules, and so on.
To define an object inline, use braces instead of parentheses. That is, instead of <object-type> (<object-id>);, you use <object-type> {<object-definition>};
Example: Using inline definitions
The following two configuration examples are equivalent. The first one uses traditional statements, while the second uses inline definitions.
Starting with AxoSyslog 3.4, every configuration object is a log expression. Every configuration object is essentially a configuration block, and can include multiple objects. To reference the block, only the top-level object must be referenced. That way you can use embedded log statements, junctions and in-line object definitions within source, destination, filter, rewrite and parser definitions. For example, a source can include a rewrite rule to modify the messages received by the source, and that combination can be used as a simple source in a log statement. This feature allows you to preprocess the log messages very close to the source itself.
To embed multiple objects into a configuration object, use the following syntax. Note that you must enclose the configuration block between braces instead of parenthesis.
For example, to process a log file in a specific way, you can define the required processing rules (parsers and rewrite expressions) and combine them in a single object:
The s_apache source uses a file source (the error log of an Apache webserver) and references a specific parser to process the messages of the error log. The log statement references only the s_apache source, and any other object in the log statement can already use the results of the p_apache_parserparser.
Note
You must start the object definition with a channel even if you will use a junction, for example:
If you want to embed configuration objects into sources or destinations, always use channels, otherwise the source or destination will not behave as expected. For example, the following configuration is good:
You can define global variables in the configuration file. Global variables are actually name-value pairs. When syslog-ng processes the configuration file during startup, it automatically replaces name with value. To define a global variable, use the following syntax:
@define name "value"
The value can be any string, but special characters must be escaped (for details, see Regular expressions). To use the variable, insert the name of the variable enclosed between backticks (`, similarly to using variables in Linux or UNIX shells) anywhere in the configuration file. If backticks are meant literally, repeat the backticks to escape them. For example:
``not-substituted-value``
The value of the global variable can be also specified using the following methods:
Without any quotes, as long as the value does not contain any spaces or special characters. In other words, it contains only the following characters: a-zA-Z0-9_..
Between apostrophes, in case the value does not contain apostrophes.
Between double quotes, in which case special characters must be escaped using backslashes (\\).
Note
The environmental variables of the host are automatically imported and can be used as global variables.
In AxoSyslog 3.24 and later, the location of the configuration file is available as the syslog-ng-sysconfdir variable.
Example: Using global variables
For example, if an application is creating multiple log files in a directory, you can store the path in a global variable, and use it in your source definitions.
Starting with AxoSyslog version 4.2, you can specify a configuration identifier in the syslog-ng.conf file, for example:
@config-id: cfg-20230404-13-g02b0850fc
This can be useful in managed environments, where AxoSyslog instances and their configuration are automatically deployed or generated.
To show the configuration ID, run syslog-ng-ctl config --id
This returns the ID of the currently active configuration, and the SHA256 hash of the configuration (the hash of the output of the syslog-ng-ctl config --preprocessed command). The output is similar to:
To increase its flexibility and simplify the development of additional modules, the AxoSyslog application is modular. The majority of AxoSyslog’s functionality is in separate modules. As a result, it is also possible to fine-tune the resource requirements of AxoSyslog (for example, by loading only the modules that are actually used in the configuration, or simply omitting modules that are not used but require large amount of memory).
Each module contains one or more plugins that add some functionality to AxoSyslog (for example, a destination or a source driver).
To display the list of available modules, run the syslog-ng --version command.
To display the description of the available modules, run the syslog-ng --module-registry command.
To customize which modules AxoSyslog automatically loads when AxoSyslog starts, use the --default-modules command-line option of AxoSyslog.
To request loading a module from the AxoSyslog configuration file, see Loading modules.
For details on the command-line parameters of AxoSyslog mentioned in the previous list, see the AxoSyslog man page at The syslog-ng manual page.
6.8.1 - Loading modules
The AxoSyslog application loads every available module during startup.
To load a module that is not loaded automatically, include the following statement in the AxoSyslog configuration file:
@module <module-name>
Note the following points about the @module statement:
The @module statement is a top-level statement, that is, it cannot be nested into any other statement. It is usually used immediately after the @version statement.
Every @module statement loads a single module: loading multiple modules requires a separate @module statement for every module.
In the configuration file, the @module statement of a module must be earlier than the module is used.
Note
To disable loading every module automatically, set the autoload-compiled-modules global variable to 0 in your configuration file:
@define autoload-compiled-modules 0
Note that in this case you have to explicitly load the modules you want to use.
Use the @requires statement to ensure that the specified module is loaded
To ensure that a module is loaded, include the following statement in the AxoSyslog configuration file or the external files included in the configuration file:
@requires <module-name>
Note
If you include the @requires statement in the:
AxoSyslog configuration file, AxoSyslog attempts to load the required module. If it fails to load the module, AxoSyslog stops and an error message is displayed.
external files included in the configuration file, AxoSyslog attempts to load the required module. If it fails to load the module, only the external file is not processed.
Note that this is not true for modules marked as mandatory. You can make a dependency module mandatory by defining an error message after the @requires <module-name> statement, for example:
Example
@requires http "The http() driver is required for elasticsearch-http(). Install syslog-ng-mod-http to continue."
6.8.2 - Listing configuration options
Starting with AxoSyslog 3.25, you can use the syslog-ng-cfg-db.py utility to list the available options of configuration objects. For example, you can list all the options that can be set in the file source, and so on.
The syslog-ng-cfg-db.py utility has the following options:
The following command lists the contexts that the utility supports.
syslog-ng-cfg-db.py
Note
Currently, sources and destinations are supported.
The following command lists the available drivers of a context:
syslog-ng-cfg-db.py -c <source|destination>
The following command lists the available options of a specific driver and specifies the context and the driver:
Note
The script caches the list of the options, so if you want to rebuild the database, you have to use the -r option.
6.8.3 - Visualize the configuration
Starting with AxoSyslog 3.25, you can visualize the configuration of a running AxoSyslog instance using the syslog-ng-ctl --export-config-graph command. The command walks through the effective configuration, and exports it as a graph into a JSON structure.
The resulting JSON file can be converted into DOT file format that visualization tools (for example, Graphviz) can use. The package includes a Python script to convert the exported JSON file into DOT format: <syslog-ng-installation-directory>/contrib/scripts/config-graph-json-to-dot.py
You can convert the DOT file into PNG or PDF format using external tools.
6.9 - Managing complex configurations
The following sections describe some methods that can be useful to simplify the management of large-scale AxoSyslog installations.
6.9.1 - Including configuration files
The AxoSyslog application supports including external files in its configuration file, so parts of its configuration can be managed separately. To include the contents of a file in the AxoSyslog configuration, use the following syntax:
@include "<filename>"
This imports the entire file into the configuration of AxoSyslog, at the location of the include statement. The <filename> can be one of the following:
A filename, optionally with full path. The filename (not the path) can include UNIX-style wildcard characters (*, ?). When using wildcard characters, AxoSyslog will include every matching file. For details on using wildcard characters, see Types and options of regular expressions.
A directory. When including a directory, AxoSyslog will try to include every file from the directory, except files beginning with a ~ (tilde) or a . (dot) character. Including a directory is not recursive. The files are included in alphabetic order, first files beginning with uppercase characters, then files beginning with lowercase characters. For example, if the directory contains the a.conf, B. conf, c.conf, D.conf files, they will be included in the following order: B.conf, D. conf, a.conf, c.conf.
When including configuration files, consider the following points:
The default path where AxoSyslog looks for the file depends on where AxoSyslog is installed. The syslog-ng --version command displays this path as Include-Path.
Defining an object twice is not allowed, unless you use the @define allow-config-dups 1 definition in the configuration file. If an object is defined twice (for example, the original configuration file and the file imported into this configuration file both define the same option, source, or other object), then the object that is defined later in the configuration file will be effective. For example, if you set a global option at the beginning of the configuration file, and later include a file that defines the same option with a different value, then the option defined in the imported file will be used.
Files can be embedded into each other: the included files can contain include statements as well, up to a maximum depth of 15 levels.
You cannot include complete configuration files into each other, only configuration snippets can be included. This means that the included file cannot have a @version statement.
Include statements can only be used at top level of the configuration file. For example, the following is correct:
@version: 4.8.1
@include "example.conf"
But the following is not:
source s_example { @include "example.conf"};
Warning
The AxoSyslog application will not start if it cannot find a file that is to be included in its configuration. Always double-check the filenames, paths, and access rights when including configuration files, and use the --syntax-only command-line option to check your configuration.
6.9.2 - Reusing configuration blocks
To create a reusable configuration snippet and reuse parts of a configuration file, you have to define the block (for example, a source) once, and reference it later. Any AxoSyslog object can be a block. Use the following syntax to define a block:
block type name(){<contents of the block>};
Type must be one of the following: destination, filter, log, options, parser, rewrite, root, source. The root blocks can be used in the “root” context of the configuration file, that is, outside any other statements.
Note that options can be used in blocks only in version 3.22 and later.
Blocks may be nested into each other, so for example, a block can be built from other blocks. Blocks are somewhat similar to C++ templates.
The type and name combination of each block must be unique, that is, two blocks can have the same name if their type is different.
To use a block in your configuration file, you have to do two things:
Include the file defining the block in the syslog-ng.conf file — or a file already included into syslog-ng.conf. Version 3.7 and newer automatically includes the *.conf files from the <directory-where-syslog-ng-is-installed>/scl/*/ directories.
Reference the name of the block in your configuration file. This will insert the block into your configuration. For example, to use a block called myblock, include the following line in your configuration:
myblock()
Blocks may have parameters, but even if they do not, the reference must include opening and closing parentheses like in the previous example.
The contents of the block will be inserted into the configuration when AxoSyslog is started or reloaded.
Example: Reusing configuration blocks
Suppose you are running an application on your hosts that logs into the /opt/var/myapplication.log file. Create a file (for example, myblocks.conf) that stores a source describing this file and how it should be read:
To define a block that defines more than one object, use root as the type of the block, and reference the block from the main part of the AxoSyslog configuration file.
Example: Defining blocks with multiple elements
The following example defines a source, a destination, and a log path to connect them.
Since the block is inserted into the AxoSyslog configuration when AxoSyslog is started, the block can be generated dynamically using an external script if needed. This is useful when you are running AxoSyslog on different hosts and you want to keep the main configuration identical.
If you want to reuse more than a single configuration object, for example, a logpath and the definitions of its sources and destinations, use the include feature to reuse the entire snippet. For details, see Including configuration files.
Mandatory parameters
You can express in block definitons that a parameter is mandatory by defining it with empty brackets (). In this case, the parameter must be overridden in the reference block. Failing to do so will result in an error message and initialization failure.
To make a parameter expand into nothing (for example, because it has no default value, like hook-commands() or tls()), insert a pair of double quote marks inside the empty brackets: ("")
Example: Mandatory parameters
The following example defines a TCP source that can receive the following parameters: the port where AxoSyslog listens (localport), and optionally source flags (flags).
Because localport is defined with empty brackets (), it is a mandatory parameter. However, the flags parameter is not mandatory, because it is defined with an empty double quote bracket pair (""). If you do not enter a specific value when referencing this parameter, the value will be an empty string. This means that in this case
my_tcp_source(localport(8080))
will be expanded to:
network(port(8080) transport(tcp) flags());
Passing arguments to configuration blocks
Configuration blocks can receive arguments as well. The parameters the block can receive must be specified when the block is defined, using the following syntax:
block type block_name(argument1(<default-value-of-the-argument>) argument2(<default-value-of-the-argument>) argument3())
If an argument does not have a default value, use an empty double quote bracket pair ("") after the name of the argument. To refer the value of the argument in the block, use the name of the argument between backticks, for example:
`argument1`
Example: Passing arguments to blocks
The following sample defines a file source block, which can receive the name of the file as a parameter. If no parameter is set, it reads messages from the /var/log/messages file.
If you reference the block with more arguments then specified in its definition, you can use these additional arguments as a single argument-list within the block. That way, you can use a variable number of optional arguments in your block. This can be useful when passing arguments to a template, or optional arguments to an underlying driver.
The three dots (…) at the end of the argument list refer to any additional parameters. It tells AxoSyslog that this macro accepts __VARARGS__, therefore any name-value pair can be passed without validation. To reference this argument-list, insert __VARARGS__ to the place in the block where you want to insert the argument-list. Note that you can use this only once in a block.
The following definition extends the logfile block from the previous example, and passes the optional arguments (follow-freq(1) flags(no-parse)) to the file() source.
6.9.3 - Generating configuration blocks from a script
Purpose:
The AxoSyslog application can automatically execute scripts when it is started, and can include the output of such script in the configuration file. To create and use a script that generates a part of the AxoSyslog configuration file (actually, a configuration block), complete the following steps. The steps include examples for collecting Apache access log files (access.log) from subdirectories, but you can create any script that creates a valid AxoSyslog configuration snippet.
Steps:
Navigate to the directory where you have installed AxoSyslog (for example, /opt/syslog-ng/share/include/scl/), and create a new directory, for example, apache-access-logs. The name of the directory will be used in the AxoSyslog configuration file as well, so use a descriptive name.
Create a file called plugin.conf in this new directory.
Edit the plugin.conf file and add the following line:
Replace <directory-name> with the name of the directory (for example, apache-access-logs), and <my-script> with the filename of your script (for example, apache-access-logs.sh). You can reference the script in your AxoSyslog configuration file as a configuration block using the value name option.
The context option determines the type of the configuration snippet that the script generates, and must be one of the following: destination, filter, log, parser, rewrite, root, source. The root blocks can be used in the “root” context of the configuration file, that is, outside any other statements. In the example, context(source) means that the output of the script will be used within a source statement.
You can pass parameters to the script. In the script these parameters are available as environment variables, and have the confgen_ prefix. For example, passing the --myparameter parameter becomes available in the script as the confgen_myparameter environment variable.
Write a script that generates the output you need, and formats it to a configuration snippet that AxoSyslog can use. The filename of the script must match with the filename used in plugin.conf, for example, apache-access-logs.sh.
The following example checks the /var/log/apache2/ directory and its subdirectories, and creates a source driver for every directory that contains an access.log file.
#!/bin/bashfor i in `find /var/log/apache2/ -type d`;doecho"file(\"$i/access.log\" flags(no-parse) program-override(\"apache2\"));";done;
The script generates an output similar to this one, where service* is the actual name of a subdirectory:
Include the plugin.conf file in the syslog-ng.conf file — or a file already included into syslog-ng.conf. Version 3.7 and newer automatically includes the *.conf files from the <directory-where-syslog-ng-is-installed>/scl/*/ directories. For details on including configuration files, see Including configuration files.
Add the block you defined in the plugin.conf file to your AxoSyslog configuration file. You can reference the block using the value of the name option from the plugin.conf file, followed by parentheses, for example, apache-access-logs(). Make sure to use the block in the appropriate context of the configuration file, for example, within a source statement if the value of the context option in the plugin.conf file is source.
Instead of writing Python code into your AxoSyslog configuration file, you can store the Python code for your Python object in an external file. That way, it is easier to write, maintain, and debug the code. You can store the Python code in any directory in your system, but make sure to include it in your Python path.
When referencing a Python class from an external file in the class() option of a Python block in the AxoSyslog configuration file, the class name must include the name of the Python file containing the class, without the path and the .py extension. For example, if the MyDestination class is available in the /etc/syslog-ng/etc/pythonexample.py file, use class("pythonexample.MyDestination"):
Starting with 3.26, AxoSyslog assigns a persist name to Python sources and destinations. The persist name is generated from the class name. If you want to use the same Python class multiple times in your AxoSyslog configuration, add a unique persist-name() to each source or destination, otherwise AxoSyslog will not start. For example:
Alternatively, you can include the following line in the Python package: @staticmethod generate_persist_name. For example:
from syslogng import LogSource
class PyNetworSource(LogSource):
@staticmethod
def generate_persist_name(options):
return options["port"] def run(self):
pass
def request_exit(self):
pass
If you store the Python code in a separate Python file and only include it in the AxoSyslog configuration file, make sure that the PYTHONPATH environment variable includes the path to the Python file, and export the PYTHON_PATH environment variable. For example, if you start AxoSyslog manually from a terminal and you store your Python files in the /opt/syslog-ng/etc directory, use the following command: export PYTHONPATH=/opt/syslog-ng/etc.
In production, when AxoSyslog starts on boot, you must configure your startup script to include the Python path. The exact method depends on your operating system. For recent Red Hat Enterprise Linux, Fedora, and CentOS distributions that use systemd, the systemctl command sources the /etc/sysconfig/syslog-ng file before starting AxoSyslog. (On openSUSE and SLES, /etc/sysconfig/syslog file.) Append the following line to the end of this file: PYTHONPATH="<path-to-your-python-file>", for example, PYTHONPATH="/opt/syslog-ng/etc".
To help debugging and troubleshooting your Python code, you can send log messages to the internal() source of AxoSyslog. For details, see Logging from your Python code.
To debug and troubleshoot your Python code, AxoSyslog allows you to use the logger() method to send log messages to the internal() source of AxoSyslog. That way the diagnostic messages of your Python code are treated the same way as other such log messages of AxoSyslog. This has the following benefits:
The logger() method respects the log level settings of AxoSyslog. You can write error, warning, info, debug, and trace level messages.
You can follow what your Python code is doing even if AxoSyslog is running as a daemon in the background.
Logging to the internal() source is available in AxoSyslog version 3.20 and later.
To send log messages to the internal() source from Python
Add the following import to your Python code:
import syslogng
Create a logger object:
logger= syslogng.Logger()
Use the logger object in your Python code, for example:
logger.info("This is a sample log message send from the Python code.")
You can use the following log levels: logger.error, logger.warning, logger.info, logger.debug, logger.trace
Make sure that your AxoSyslog configuration includes the internal() source, for example:
If the message received by the source does not have a proper syslog header, you can use the default-facility() and default-priority() options to set the facility and priority of the messages. Note that these values are applied only to messages that do not set these parameters in their header.
Define a source only once. The same source can be used in several log paths. Duplicating sources causes AxoSyslog to open the source (TCP/IP port, file, and so on) more than once, which might cause problems. For example, include the /dev/log file source only in one source statement, and use this statement in more than one log path if needed.
Warning
Sources and destinations are initialized only when they are used in a log statement. For example, AxoSyslog starts listening on a port or starts polling a file only if the source is used in a log statement. For details on creating log statements, see log: Filter and route log messages using log paths, flags, and filters.
To collect log messages on a specific platform, it is important to know how the native syslogd communicates on that platform. The following table summarizes the operation methods of syslogd on some of the tested platforms:
Communication methods used between the applications and syslogd
Platform
Method
Linux
A SOCK_DGRAM unix socket named /dev/log. Newer distributions that use systemd collect log messages into a journal file.
BSD flavors
A SOCK_DGRAM unix socket named /var/run/log.
Solaris (2.5 or below)
An SVR4 style STREAMS device named /dev/log.
Solaris (2.6 or above)
In addition to the STREAMS device used in earlier versions, 2.6 uses a new multithreaded IPC method called door. By default the door used by syslogd is /etc/.syslog_door.
HP-UX 11 or later
HP-UX uses a named pipe called /dev/log that is padded to 2048 bytes, for example, source s_hp-ux {pipe ("/dev/log" pad-size(2048)}.
AIX 5.2 and 5.3
A SOCK_STREAM or SOCK_DGRAM unix socket called /dev/log.
Each possible communication mechanism has a corresponding source driver in syslog-ng. For example, to open a unix socket with SOCK_DGRAM style communication use the driver unix-dgram. The same socket using the SOCK_STREAM style — as used under Linux — is called unix-stream.
Example: Source statement on a Linux based operating system
The following source statement collects the following log messages:
internal(): Messages generated by syslog-ng.
network(transport(“udp”)): Messages arriving to the 514/UDP port of any interface of the host.
unix-dgram("/dev/log");: Messages arriving to the /dev/log socket.
The logging module is stored in the <prefix><module> name-value pair, for example: .radarr.module => ImportListSyncService.
You can modify the prefix with the prefix() option.
This driver is actually a reusable configuration snippet. For details on using or writing such configuration snippets, see Reusing configuration blocks. You can find the source of this configuration snippet on GitHub.
7.3 - Collect native macOS system logs
Starting with version 4.6.0, AxoSyslog can collect logs on macOS using its native OSLog framework using the darwin-oslog() and darwin-oslog-stream() source drivers.
darwin-oslog(): This source builds on the native OSLog framework, and replaces the earlier file-source based solution.
This source is based on the native OSLog Framework to read logs from the local store of the unified logging system on darwin OSes. The AxoSyslog system() source automatically uses this new source on darwin platforms if the darwinosl plugin is available. This plugin is available only on macOS 10.15 Catalina and above, the first version that has the OSLog API.
Note that the persistent OSLog store usually keeps about 7 days of logs on disk.
The darwin-oslog() source has the following options:
Description: By default, AxoSyslog continues to read the logs from the last remembered position after a restart. If this option is set to yes, it will always start reading from the end or beginning of the available log list (depending on the setting of the go-reverse() option).
fetch-delay()
Type:
integer
Default:
10000
Description: Controls the time AxoSyslog waits between reading and sending log messages. This is a fraction of a second, where wait_time = 1 second / n, so n=1 means that only about 1 log is sent in each second, and n=1000000 means only 1 microsecond is the delay between read/write attempts. The maximal value of this parameter is 1000000. Note that increasing the value of this parameter (thus lowering delay time) can increase log feed performance, but at the same time could increase system load.
fetch-retry-delay()
Type:
integer
Default:
1
Description: Controls how many seconds AxoSyslog waits before trying to check for new logs if there were no more logs to read the last time.
go-reverse()
Type:
boolean
Default:
no
Description: Set it to yes to process the logs in a reverse order (from latest to oldest).
log-fetch-limit()
Type:
integer
Default:
0 (no limit)
Warning
This option is currently disabled because of an OSLog API bug.
Description: The maximum number of messages fetched from a source during a single poll loop. The destination queues might fill up before flow-control could stop reading if log-fetch-limit() is too high.
max-bookmark-distance()
Type:
integer
Default:
0 (no limit) [seconds]
Description: The maximum distance in seconds that a bookmark can point backwards. That is, if AxoSyslog was stopped for 10 minutes and max-bookmark-distance() is set to 60, then AxoSyslog will start reading the logs from 60 seconds before the startup, losing 9 minutes of logs.
read-old-records()
Type:
boolean
Default:
no
Description: If set to yes, AxoSyslog starts reading logs from the oldest available log when it’s first started on a system, or if there are no bookmarks for some reason
darwin-oslog-stream()
This source is a wrapper around the OS command line log stream command that provides a live log stream feed. Unlike the darwin-oslog() source, the live stream can contain non-persistent log events as well. This might result in a large number of log events every second.
This source has only one option.
params()
Type:
string
Default:
--type log --type trace --level info --level debug
Description: A string that can contain all the possible params the macOS log tool can accept. The source uses the “–style” internally (defaults to ndjson), so use templates or rewrite rules to format the final output. Use the def-osl-stream-params string to reference the default values when extending them with your own.
For a full reference, see the output of the log --help stream and man log commands.
7.4 - default-network-drivers: Receive and parse common syslog messages
The default-network-drivers() source is a special source that uses multiple source drivers to receive and parse several different types of syslog messages from the network. Available in version 3.16 and later.
To use the default-network-drivers() source, the scl.conf file must be included in your AxoSyslog configuration:
@include "scl.conf"
Also, make sure that your SELinux, AppArmor, and firewall settings permit AxoSyslog to access the ports where you want to receive messages, and that no other application is using these ports. By default, the default-network-drivers() source accepts messages on the following ports:
514, both TCP and UDP, for RFC3164 (BSD-syslog) formatted traffic
601 TCP, for RFC5424 (IETF-syslog) formatted traffic
6514 TCP, for TLS-encrypted traffic
In addition to receiving messages on different ports and in different formats, this source tries to parse the messages automatically. If successful, it sets the ${.app.name} name-value pair to the name of the application that sent the log message. Currently it uses the following procedures.
Warning
If you do not configure the TLS keys to show to the clients, AxoSyslog cannot accept encrypted connections. The application starts and listens on TCP:6514, and can receive messages on other ports, but will display a warning messages about missing keys.
Parsing RFC3164-formatted messages
For RFC3164-formatted messages (that is, messages received on the ports set in options udp-port() and tcp-port() which default to port 514), AxoSyslog attempts to use the following parsers. If a parser cannot parse the message, it passes the original message to the next parser.
The Enterprise-wide message model or EWMM allows you to deliver structured messages from the initial receiving AxoSyslog component right up to the central log server, through any number of hops. It does not matter if you parse the messages on the client, on a relay, or on the central server, their structured results will be available where you store the messages. Optionally, you can also forward the original raw message as the first AxoSyslog component in your infrastructure has received it, which is important if you want to forward a message for example, to a SIEM system. To make use of the enterprise-wide message model, you have to use the syslog-ng() destination on the sender side, and the default-network-drivers() source on the receiver side.
Otherwise, apply the application adapters if the message was sent from an application that already has a specific parser in AxoSyslog (for example, Splunk Common Information Model (CIM), iptables, or sudo).
Parsing RFC5424-formatted messages
For RFC5424-formatted messages (that is, messages received on the ports set in options rfc5424-tls-port() and rfc5424-tcp-port(), which default to port 601 and 6514), AxoSyslog parses the message according to RFC5424, then attempts apply the application adapters if the message was sent from an application that already has a specific parser in AxoSyslog (for example, Splunk Common Information Model (CIM), iptables, or sudo).
Example: Using the default-network-drivers() driver
The following example uses only the default settings.
source s_network { default-network-drivers();};
The following example can receive TLS-encrypted connections on the default port (port 6514).
The systemd-journal() driver has the following options.
ca-dir()
Accepted values:
Directory name
Default:
none
Description: The name of a directory that contains a set of trusted CA certificates in PEM format. The CA certificate files have to be named after the 32-bit hash of the subject’s name. This naming can be created using the c_rehash utility in openssl. For an example, see Configuring TLS on the AxoSyslog clients. The AxoSyslog application uses the CA certificates in this directory to validate the certificate of the peer.
This option can be used together with the optional ca-file() option.
ca-file()
Accepted values:
File name
Default:
empty
Description: Optional. The name of a file that contains a set of trusted CA certificates in PEM format. The AxoSyslog application uses the CA certificates in this file to validate the certificate of the peer.
Example format in configuration:
ca-file("/etc/pki/tls/certs/ca-bundle.crt")
Note
The ca-file() option can be used together with the ca-dir() option, and it is relevant when peer-verify() is set to other than no or optional-untrusted.
Description: Specifies the log parsing options of the source.
assume-utf8: The assume-utf8 flag assumes that the incoming messages are UTF-8 encoded, but does not verify the encoding. If you explicitly want to validate the UTF-8 encoding of the incoming message, use the validate-utf8 flag.
empty-lines: Use the empty-lines flag to keep the empty lines of the messages. By default, AxoSyslog removes empty lines automatically.
expect-hostname: If the expect-hostname flag is enabled, AxoSyslog will assume that the log message contains a hostname and parse the message accordingly. This is the default behavior for TCP sources. Note that pipe sources use the no-hostname flag by default.
guess-timezone: Attempt to guess the timezone of the message if this information is not available in the message. Works when the incoming message stream is close to real time, and the timezone information is missing from the timestamp.
kernel: The kernel flag makes the source default to the LOG_KERN | LOG_NOTICE priority if not specified otherwise.
no-header: The no-header flag triggers AxoSyslog to parse only the PRI field of incoming messages, and put the rest of the message contents into $MSG.
Its functionality is similar to that of the no-parse flag, except the no-header flag does not skip the PRI field.
Note
Essentially, the no-header flag signals AxoSyslog that the syslog header is not present (or does not adhere to the conventions / RFCs), so the entire message (except from the PRI field) is put into $MSG.
Example: using the no-header flag with the syslog-parser() parser
The following example illustrates using the no-header flag with the syslog-parser() parser:
no-hostname: Enable the no-hostname flag if the log message does not include the hostname of the sender host. That way AxoSyslog assumes that the first part of the message header is ${PROGRAM} instead of ${HOST}. For example:
no-multi-line: The no-multi-line flag disables line-breaking in the messages: the entire message is converted to a single line. Note that this happens only if the underlying transport method actually supports multi-line messages. Currently the file() and pipe() drivers support multi-line messages.
no-parse: By default, AxoSyslog parses incoming messages as syslog messages. The no-parse flag completely disables syslog message parsing and processes the complete line as the message part of a syslog message. The AxoSyslog application will generate a new syslog header (timestamp, host, and so on) automatically and put the entire incoming message into the MESSAGE part of the syslog message (available using the ${MESSAGE} macro). This flag is useful for parsing messages not complying to the syslog format.
If you are using the flags(no-parse) option, then syslog message parsing is completely disabled, and the entire incoming message is treated as the ${MESSAGE} part of a syslog message. In this case, AxoSyslog generates a new syslog header (timestamp, host, and so on) automatically. Note that even though flags(no-parse) disables message parsing, some flags can still be used, for example, the no-multi-line flag.
dont-store-legacy-msghdr: By default, AxoSyslog stores the original incoming header of the log message. This is useful if the original format of a non-syslog-compliant message must be retained (AxoSyslog automatically corrects minor header errors, for example, adds a whitespace before msg in the following message: Jan 22 10:06:11 host program:msg). If you do not want to store the original header of the message, enable the dont-store-legacy-msghdr flag.
sanitize-utf8: When using the sanitize-utf8 flag, AxoSyslog converts non-UTF-8 input to an escaped form, which is valid UTF-8.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
store-raw-message: Save the original message as received from the client in the ${RAWMSG} macro. You can forward this raw message in its original form to another AxoSyslog node using the syslog-ng() destination, or to a SIEM system, ensuring that the SIEM can process it. Available only in 3.16 and later.
syslog-protocol: The syslog-protocol flag specifies that incoming messages are expected to be formatted according to the new IETF syslog protocol standard (RFC5424), but without the frame header. Note that this flag is not needed for the syslog driver, which handles only messages that have a frame header.
validate-utf8: The validate-utf8 flag enables encoding-verification for messages.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
For RFC5424-formatted messages, if the BOM character is missing, but the message is otherwise UTF-8 compliant, AxoSyslog automatically adds the BOM character to the message.
The byte order mark (BOM) is a Unicode character used to signal the byte-order of the message text.
log-iw-size()
Type:
number
Default:
100
Description: The size of the initial window, this value is used during flow-control. Its value cannot be lower than 100, unless the dynamic-window-size() option is enabled. For details on flow-control, see Managing incoming and outgoing messages with flow-control.
log-msg-size()
Type:
number (bytes)
Default:
Use the global log-msg-size() option, which defaults to 65536 (64 KiB).
Description: Maximum length of an incoming message in bytes. This length includes the entire message (the data structure and individual fields). The maximal value that can be set is 268435456 bytes (256 MiB).
For messages using the IETF-syslog message format (RFC5424), the maximal size of the value of an SDATA field is 64 KiB.
Note
In most cases, log-msg-size() does not need to be set higher than 10 MiB.
Uses the value of the global option if not specified.
max-connections()
Type:
number
Default:
10
Description: Specifies the maximum number of simultaneous connections.
Note that the total number of connections the default-network-drivers() source can use is 3*max-connections(), because this value applies to the network(tcp), syslog(tcp), and syslog(tls) connections individually.
rfc5424-tcp-port()
Type:
number
Default:
601
Description: The TCP port number where the default-network-drivers() source receives RFC5424-formatted (IETF-syslog) messages.
rfc5424-tls-port()
Type:
number
Default:
6514
Description: The TCP port number where the default-network-drivers() source receives RFC5424-formatted (IETF-syslog), TLS-encrypted messages.
Warning
To receive messages using a TLS-encrypted connection, you must set the tls(key-file() cert-file()) options of the default-network-drivers() source. For example:
Description: The TCP port number where the default-network-drivers() source receives RFC3164-formatted (BSD-syslog) messages.
tls()
Type:
tls options
Default:
n/a
Description: This option sets various options related to TLS encryption, for example, key/certificate files and trusted CA locations. TLS can be used only with tcp-based transport protocols. For details, see TLS options.
udp-port()
Type:
number
Default:
514
Description: The UDP port number where the default-network-drivers() source receives RFC3164-formatted (BSD-syslog) messages.
7.5 - internal: Collect internal messages
All messages generated internally by AxoSyslog use this special source. To collect warnings, errors and notices from AxoSyslog itself, include this source in one of your source statements.
internal()
The AxoSyslog application will issue a warning upon startup if none of the defined log paths reference this driver.
Example: Using the internal() driver
source s_local { internal();};
The AxoSyslog application sends the following message types from the internal() source:
info: Priority value: info (6), Facility value: syslog (5)
7.5.1 - internal() source options
The internal() driver has the following options:
host-override()
Type:
string
Default:
Description: Replaces the ${HOST} part of the message with the parameter string.
log-iw-size()
Type:
number
Default:
100
Description: The size of the initial window, this value is used during flow-control. Its value cannot be lower than 100, unless the dynamic-window-size() option is enabled. For details on flow-control, see Managing incoming and outgoing messages with flow-control.
normalize-hostnames()
Accepted values:
yes, no
Default:
no
Description: If enabled (normalize-hostnames(yes)), AxoSyslog converts the hostnames to lowercase.
program-override()
Type:
string
Default:
Description: Replaces the ${PROGRAM} part of the message with the parameter string. For example, to mark every message coming from the kernel, include the program-override("kernel") option in the source containing /proc/kmsg.
tags()
Type:
string
Default:
Description: Label the messages received from the source with custom tags. Tags must be unique, and enclosed between double quotes. When adding multiple tags, separate them with comma, for example, tags("dmz", "router"). This option is available only in version 3.1 and later.
use-fqdn()
Type:
yes or no
Default:
no
Description: Add Fully Qualified Domain Name instead of short hostname. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.
The AxoSyslog application notices if a file is renamed or replaced with a new file, so it can correctly follow the file even if logrotation is used. When AxoSyslog is restarted, it records the position of the last sent log message in the /opt/syslog-ng/var/syslog-ng.persist/var/lib/syslog-ng/syslog-ng.persist file, and continues to send messages from this position after the restart.
Note
If the message does not have a proper syslog header, AxoSyslog treats messages received from files as sent by the kern facility. Use the default-facility() and default-priority() options in the source definition to assign a different facility if needed.
7.6.1 - Notes on reading kernel messages
Note the following points when reading kernel messages on various platforms.
The kernel usually sends log messages to a special file (/dev/kmsg on BSDs, /proc/kmsg on Linux). The file() driver reads log messages from such files. The AxoSyslog application can periodically check the file for new log messages if the follow-freq() option is set.
On Linux, the klogd daemon can be used in addition to AxoSyslog to read kernel messages and forward them to syslog-ng. klogd used to preprocess kernel messages to resolve symbols and so on, but as this is deprecated by ksymoops there is really no point in running both klogd and AxoSyslog in parallel. Also note that running two processes reading /proc/kmsg at the same time might result in dead-locks.
When using AxoSyslog to read messages from the /proc/kmsg file, AxoSyslog automatically disables the follow-freq() parameter to avoid blocking the file.
To read the kernel messages on HP-UX platforms, use the following options in the source statement:
Use the global check-hostname() option, which defaults to no.
Checks that the hostname contains valid characters. Uses the value of the global option if not specified.
default-facility()
Type:
facility string
Default:
kern
Description: This parameter assigns a facility value to the messages received from the file source if the message does not specify one.
default-priority()
Type:
priority string
Default:
Description: This parameter assigns an emergency level to the messages received from the file source if the message does not specify one. For example, default-priority(warning).
encoding()
Type:
string
Default:
Description: Specifies the character set (encoding, for example, UTF-8) of messages using the legacy BSD-syslog protocol. To list the available character sets on a host, execute the iconv -l command. For details on how encoding affects the size of the message, see Message size and encoding.
Description: Specifies the log parsing options of the source.
assume-utf8: The assume-utf8 flag assumes that the incoming messages are UTF-8 encoded, but does not verify the encoding. If you explicitly want to validate the UTF-8 encoding of the incoming message, use the validate-utf8 flag.
empty-lines: Use the empty-lines flag to keep the empty lines of the messages. By default, AxoSyslog removes empty lines automatically.
expect-hostname: If the expect-hostname flag is enabled, AxoSyslog will assume that the log message contains a hostname and parse the message accordingly. This is the default behavior for TCP sources. Note that pipe sources use the no-hostname flag by default.
guess-timezone: Attempt to guess the timezone of the message if this information is not available in the message. Works when the incoming message stream is close to real time, and the timezone information is missing from the timestamp.
kernel: The kernel flag makes the source default to the LOG_KERN | LOG_NOTICE priority if not specified otherwise.
no-header: The no-header flag triggers AxoSyslog to parse only the PRI field of incoming messages, and put the rest of the message contents into $MSG.
Its functionality is similar to that of the no-parse flag, except the no-header flag does not skip the PRI field.
Note
Essentially, the no-header flag signals AxoSyslog that the syslog header is not present (or does not adhere to the conventions / RFCs), so the entire message (except from the PRI field) is put into $MSG.
Example: using the no-header flag with the syslog-parser() parser
The following example illustrates using the no-header flag with the syslog-parser() parser:
no-hostname: Enable the no-hostname flag if the log message does not include the hostname of the sender host. That way AxoSyslog assumes that the first part of the message header is ${PROGRAM} instead of ${HOST}. For example:
no-multi-line: The no-multi-line flag disables line-breaking in the messages: the entire message is converted to a single line. Note that this happens only if the underlying transport method actually supports multi-line messages. Currently the file() and pipe() drivers support multi-line messages.
no-parse: By default, AxoSyslog parses incoming messages as syslog messages. The no-parse flag completely disables syslog message parsing and processes the complete line as the message part of a syslog message. The AxoSyslog application will generate a new syslog header (timestamp, host, and so on) automatically and put the entire incoming message into the MESSAGE part of the syslog message (available using the ${MESSAGE} macro). This flag is useful for parsing messages not complying to the syslog format.
If you are using the flags(no-parse) option, then syslog message parsing is completely disabled, and the entire incoming message is treated as the ${MESSAGE} part of a syslog message. In this case, AxoSyslog generates a new syslog header (timestamp, host, and so on) automatically. Note that even though flags(no-parse) disables message parsing, some flags can still be used, for example, the no-multi-line flag.
dont-store-legacy-msghdr: By default, AxoSyslog stores the original incoming header of the log message. This is useful if the original format of a non-syslog-compliant message must be retained (AxoSyslog automatically corrects minor header errors, for example, adds a whitespace before msg in the following message: Jan 22 10:06:11 host program:msg). If you do not want to store the original header of the message, enable the dont-store-legacy-msghdr flag.
sanitize-utf8: When using the sanitize-utf8 flag, AxoSyslog converts non-UTF-8 input to an escaped form, which is valid UTF-8.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
store-raw-message: Save the original message as received from the client in the ${RAWMSG} macro. You can forward this raw message in its original form to another AxoSyslog node using the syslog-ng() destination, or to a SIEM system, ensuring that the SIEM can process it. Available only in 3.16 and later.
syslog-protocol: The syslog-protocol flag specifies that incoming messages are expected to be formatted according to the new IETF syslog protocol standard (RFC5424), but without the frame header. Note that this flag is not needed for the syslog driver, which handles only messages that have a frame header.
validate-utf8: The validate-utf8 flag enables encoding-verification for messages.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
For RFC5424-formatted messages, if the BOM character is missing, but the message is otherwise UTF-8 compliant, AxoSyslog automatically adds the BOM character to the message.
The byte order mark (BOM) is a Unicode character used to signal the byte-order of the message text.
follow-freq()
Type:
number
Default:
1
Description: Indicates that the source should be checked periodically. This is useful for files which always indicate readability, even though no new lines were appended. If this value is higher than zero, AxoSyslog will not attempt to use poll() on the file, but checks whether the file changed every time the follow-freq() interval (in seconds) has elapsed. Floating-point numbers (for example, 1.5) can be used as well.
hook-commands()
Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.
Note
The AxoSyslog application must be able to start and restart the external program, and have the necessary permissions to do so. For example, if your host is running AppArmor or SELinux, you might have to modify your AppArmor or SELinux configuration to enable AxoSyslog to execute external applications.
Using hook-commands() when AxoSyslog starts or stops
To execute an external program when AxoSyslog starts or stops, use the following options:
startup()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog starts.
shutdown()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog stops.
Using the hook-commands() when AxoSyslog reloads
To execute an external program when the AxoSyslog configuration is initiated or torn down, for example, on startup/shutdown or during a AxoSyslog reload, use the following options:
setup()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is initiated, for example, on startup or during a AxoSyslog reload.
teardown()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is stopped or torn down, for example, on shutdown or during a AxoSyslog reload.
Example: Using the hook-commands() with a network source
In the following example, the hook-commands() is used with the network() driver and it opens an iptables port automatically as AxoSyslog is started/stopped.
The assumption in this example is that the LOGCHAIN chain is part of a larger ruleset that routes traffic to it. Whenever the AxoSyslog created rule is there, packets can flow, otherwise the port is closed.
Description: Specifies whether AxoSyslog should accept the timestamp received from the sending application or client. If disabled, the time of reception will be used instead. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.
Warning
To use the S_ macros, the keep-timestamp() option must be enabled (this is the default behavior of AxoSyslog).
log-fetch-limit()
Type:
number
Default:
100
Description: The maximum number of messages fetched from a source during a single poll loop. The destination queues might fill up before flow-control could stop reading if log-fetch-limit() is too high.
log-iw-size()
Type:
number
Default:
10000
Description: The size of the initial window, this value is used during flow control. Make sure that log-iw-size() is larger than the value of log-fetch-limit().
log-msg-size()
Type:
number (bytes)
Default:
Use the global log-msg-size() option, which defaults to 65536 (64 KiB).
Description: Maximum length of an incoming message in bytes. This length includes the entire message (the data structure and individual fields). The maximal value that can be set is 268435456 bytes (256 MiB).
For messages using the IETF-syslog message format (RFC5424), the maximal size of the value of an SDATA field is 64 KiB.
Note
In most cases, log-msg-size() does not need to be set higher than 10 MiB.
Uses the value of the global option if not specified.
log-prefix() (DEPRECATED)
Type:
string
Default:
Description: A string added to the beginning of every log message. It can be used to add an arbitrary string to any log source, though it is most commonly used for adding kernel: to the kernel messages on Linux.
Note
This option is deprecated. Use program-override instead.
multi-line-garbage()
Type:
regular expression
Default:
empty string
Description: Use the multi-line-garbage() option when processing multi-line messages that contain unneeded parts between the messages. Specify a string or regular expression that matches the beginning of the unneeded message parts. If the multi-line-garbage() option is set, AxoSyslog ignores the lines between the line matching the multi-line-garbage() and the next line matching multi-line-prefix(). See also the multi-line-prefix() option.
When receiving multi-line messages from a source when the multi-line-garbage() option is set, but no matching line is received between two lines that match multi-line-prefix(), AxoSyslog will continue to process the incoming lines as a single message until a line matching multi-line-garbage() is received.
To use the multi-line-garbage() option, set the multi-line-mode() option to prefix-garbage.
Warning
If the multi-line-garbage() option is set, AxoSyslog discards lines between the line matching the multi-line-garbage() and the next line matching multi-line-prefix().
Description: Use the multi-line-mode() option when processing multi-line messages. The AxoSyslog application provides the following methods to process multi-line messages:
indented: The indented mode can process messages where each line that belongs to the previous line is indented by whitespace, and the message continues until the first non-indented line. For example, the Linux kernel (starting with version 3.5) uses this format for /dev/log, as well as several applications, like Apache Tomcat.
prefix-garbage: The prefix-garbage mode uses a string or regular expression (set in multi-line-prefix()) that matches the beginning of the log messages, ignores newline characters from the source until a line matches the regular expression again, and treats the lines between the matching lines as a single message. For details on using multi-line-mode(prefix-garbage), see the multi-line-prefix() and multi-line-garbage() options.
prefix-suffix: The prefix-suffix mode uses a string or regular expression (set in multi-line-prefix()) that matches the beginning of the log messages, ignores newline characters from the source until a line matches the regular expression set in multi-line-suffix(), and treats the lines between multi-line-prefix() and multi-line-suffix() as a single message. Any other lines between the end of the message and the beginning of a new message (that is, a line that matches the multi-line-prefix() expression) are discarded. For details on using multi-line-mode(prefix-suffix), see the multi-line-prefix() and multi-line-suffix() options.
The prefix-suffix mode is similar to the prefix-garbage mode, but it appends the garbage part to the message instead of discarding it.
smart: The smart mode recognizes multi-line data backtraces even if they span multiple lines in the input. The backtraces are converted to a single log message for easier analysis. Backtraces for the following programming languages are recognized : Python, Java, JavaScript, PHP, Go, Ruby, and Dart.
smart mode is available in AxoSyslog version 4.2 and newer.
The regular expressions to recognize these programming languages are specified in an external file called /usr/share/syslog-ng/smart-multi-line.fsm (installation path depends on configure arguments), in a format that is described in that file.
Note
To format multi-line messages to your individual needs, consider the following:
To make multi-line messages more readable when written to a file, use a template in the destination and instead of the ${MESSAGE} macro, use the following: $(indent-multi-line ${MESSAGE}). This expression inserts a tab after every newline character (except when a tab is already present), indenting every line of the message after the first. For example:
To actually convert the lines of multi-line messages to single line (by replacing the newline characters with whitespaces), use the flags(no-multi-line) option in the source.
multi-line-prefix()
Type:
regular expression starting with the ^ character
Default:
empty string
Description: Use the multi-line-prefix() option to process multi-line messages, that is, log messages that contain newline characters (for example, Tomcat logs). Specify a string or regular expression that matches the beginning of the log messages (always start with the ^ character). Use as simple regular expressions as possible, because complex regular expressions can severely reduce the rate of processing multi-line messages. If the multi-line-prefix() option is set, AxoSyslog ignores newline characters from the source until a line matches the regular expression again, and treats the lines between the matching lines as a single message. See also the multi-line-garbage() option.
Note
To format multi-line messages to your individual needs, consider the following:
To make multi-line messages more readable when written to a file, use a template in the destination and instead of the ${MESSAGE} macro, use the following: $(indent-multi-line ${MESSAGE}). This expression inserts a tab after every newline character (except when a tab is already present), indenting every line of the message after the first. For example:
To actually convert the lines of multi-line messages to single line (by replacing the newline characters with whitespaces), use the flags(no-multi-line) option in the source.
Example: Processing Tomcat logs
The log messages of the Apache Tomcat server are a typical example for multi-line log messages. The messages start with the date and time of the query in the YYYY.MM.DD HH:MM:SS format, as you can see in the following example.
2010.06.09. 12:07:39 org.apache.catalina.startup.Catalina start
SEVERE: Catalina.start:
LifecycleException: service.getName(): "Catalina"; Protocol handler start failed: java.net.BindException: Address already in use null:8080
at org.apache.catalina.connector.Connector.start(Connector.java:1138) at org.apache.catalina.core.StandardService.start(StandardService.java:531) at org.apache.catalina.core.StandardServer.start(StandardServer.java:710) at org.apache.catalina.startup.Catalina.start(Catalina.java:583) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:177) 2010.06.09. 12:07:39 org.apache.catalina.startup.Catalina start
INFO: Server startup in 1206 ms
2010.06.09. 12:45:08 org.apache.coyote.http11.Http11Protocol pause
INFO: Pausing Coyote HTTP/1.1 on http-8080
2010.06.09. 12:45:09 org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
To process these messages, specify a regular expression matching the timestamp of the messages in the multi-line-prefix() option. Such an expression is the following:
Note that flags(no-parse) is needed to prevent AxoSyslog trying to interpret the date in the message.
multi-line-suffix()
Type:
regular expression
Default:
empty string
Description: Use the multi-line-suffix() option when processing multi-line messages. Specify a string or regular expression that matches the end of the multi-line message.
To use the multi-line-suffix() option, set the multi-line-mode() option to prefix-suffix. See also the multi-line-prefix() option.
pad-size()
Type:
number
Default:
0
Description: Specifies input padding. Some operating systems (such as HP-UX) pad all messages to block boundary. This option can be used to specify the block size. The AxoSyslog application will pad reads from the associated device to the number of bytes set in pad-size(). Mostly used on HP-UX where /dev/log is a named pipe and every write is padded to 2048 bytes. If pad-size() was given and the incoming message does not fit into pad-size(), AxoSyslog will not read anymore from this pipe and displays the following error message:
Padding was set, and couldn't read enough bytes
program-override()
Type:
string
Default:
Description: Replaces the ${PROGRAM} part of the message with the parameter string. For example, to mark every message coming from the kernel, include the program-override("kernel") option in the source containing /proc/kmsg.
tags()
Type:
string
Default:
Description: Label the messages received from the source with custom tags. Tags must be unique, and enclosed between double quotes. When adding multiple tags, separate them with comma, for example, tags("dmz", "router"). This option is available only in version 3.1 and later.
time-zone()
Type:
name of the timezone, or the timezone offset
Default:
Description: The default timezone for messages read from the source. Applies only if no timezone is specified within the message itself.
The timezone can be specified by using the name, for example, time-zone("Europe/Budapest")), or as the timezone offset in +/-HH:MM format, for example, +01:00). On Linux and UNIX platforms, the valid timezone names are listed under the /usr/share/zoneinfo directory.
7.7 - wildcard-file: Collect messages from multiple text files
The wildcard-file() source collects log messages from multiple plain-text files from multiple directories. The wildcard-file() source is available in AxoSyslog version 3.10 and later.
The AxoSyslog application notices if a file is renamed or replaced with a new file, so it can correctly follow the file even if logrotation is used. When AxoSyslog is restarted, it records the position of the last sent log message in the persist file, and continues to send messages from this position after the restart. The location of the persist file depends on how you installed AxoSyslog, typically it is /var/lib/syslog-ng/syslog-ng.persist or /opt/syslog-ng/var/syslog-ng.persist.
You can use the * and ? wildcard characters in the filename (the filename-pattern() option), but not in the path (the base-dir() option).
If you use multiple wildcard-file() sources in your configuration, make sure that the files and folders that match the wildcards do not overlap. That is, every file and folder should belong to only one file source. Monitoring a file from multiple wildcard sources can lead to data loss.
When using wildcards, AxoSyslog monitors every matching file (up to the limit set in the max-files() option), and can receive new log messages from any of the files. However, monitoring (polling) many files (that is, more than ten) has a significant overhead and may affect performance. On Linux this overhead is not so significant, because AxoSyslog uses the inotify feature of the kernel. Set the max-files() option at least to the number of files you want to monitor. If the wildcard-file source matches more files than the value of the max-files() option, it is random which files will AxoSyslog actually monitor. The default value of max-files() is 100.
If the message does not have a proper syslog header, AxoSyslog treats messages received from files as sent by the user facility. Use the default-facility() and default-priority() options in the source definition to assign a different facility if needed.
For every message that AxoSyslog reads from the source files, the path and name of the file is available in the ${FILE_NAME} macro.
Required parameters: base-dir(), filename-pattern(). For the list of available optional parameters, see wildcard-file() source options.
Example: Using the wildcard-file() driver
The following example monitors every file with the .log extension in the /var/log directory for log messages.
The wildcard-file() driver has the following options:
base-dir()
Type:
path without filename
Default:
Description: The path to the directory that contains the log files to monitor, for example, base-dir("/var/log"). To monitor also the subdirectories of the base directory, use the recursive(yes) option. For details, see recursive().
Warning
If you use multiple wildcard-file() sources in your configuration, make sure that the files and folders that match the wildcards do not overlap. That is, every file and folder should belong to only one file source. Monitoring a file from multiple wildcard sources can lead to data loss.
Use the global check-hostname() option, which defaults to no.
Checks that the hostname contains valid characters. Uses the value of the global option if not specified.
default-facility()
Type:
facility string
Default:
kern
Description: This parameter assigns a facility value to the messages received from the file source if the message does not specify one.
default-priority()
Type:
priority string
Default:
Description: This parameter assigns an emergency level to the messages received from the file source if the message does not specify one. For example, default-priority(warning).
encoding()
Type:
string
Default:
Description: Specifies the character set (encoding, for example, UTF-8) of messages using the legacy BSD-syslog protocol. To list the available character sets on a host, execute the iconv -l command. For details on how encoding affects the size of the message, see Message size and encoding.
filename-pattern()
Type:
filename without path
Default:
Description: The filename to read messages from, without the path. You can use the * and ? wildcard characters, without regular expression and character range support. You cannot use the * and ? literally in the pattern.
For example, filename-pattern("*.log") matches the syslog.log and auth.log files, but does not match the access_log file. The filename-pattern("*log") pattern matches all three.
* matches an arbitrary string, including an empty string
? matches an arbitrary character
Warning
If you use multiple wildcard-file() sources in your configuration, make sure that the files and folders that match the wildcards do not overlap. That is, every file and folder should belong to only one file source. Monitoring a file from multiple wildcard sources can lead to data loss.
Description: Specifies the log parsing options of the source.
assume-utf8: The assume-utf8 flag assumes that the incoming messages are UTF-8 encoded, but does not verify the encoding. If you explicitly want to validate the UTF-8 encoding of the incoming message, use the validate-utf8 flag.
empty-lines: Use the empty-lines flag to keep the empty lines of the messages. By default, AxoSyslog removes empty lines automatically.
expect-hostname: If the expect-hostname flag is enabled, AxoSyslog will assume that the log message contains a hostname and parse the message accordingly. This is the default behavior for TCP sources. Note that pipe sources use the no-hostname flag by default.
guess-timezone: Attempt to guess the timezone of the message if this information is not available in the message. Works when the incoming message stream is close to real time, and the timezone information is missing from the timestamp.
kernel: The kernel flag makes the source default to the LOG_KERN | LOG_NOTICE priority if not specified otherwise.
no-header: The no-header flag triggers AxoSyslog to parse only the PRI field of incoming messages, and put the rest of the message contents into $MSG.
Its functionality is similar to that of the no-parse flag, except the no-header flag does not skip the PRI field.
Note
Essentially, the no-header flag signals AxoSyslog that the syslog header is not present (or does not adhere to the conventions / RFCs), so the entire message (except from the PRI field) is put into $MSG.
Example: using the no-header flag with the syslog-parser() parser
The following example illustrates using the no-header flag with the syslog-parser() parser:
no-hostname: Enable the no-hostname flag if the log message does not include the hostname of the sender host. That way AxoSyslog assumes that the first part of the message header is ${PROGRAM} instead of ${HOST}. For example:
no-multi-line: The no-multi-line flag disables line-breaking in the messages: the entire message is converted to a single line. Note that this happens only if the underlying transport method actually supports multi-line messages. Currently the file() and pipe() drivers support multi-line messages.
no-parse: By default, AxoSyslog parses incoming messages as syslog messages. The no-parse flag completely disables syslog message parsing and processes the complete line as the message part of a syslog message. The AxoSyslog application will generate a new syslog header (timestamp, host, and so on) automatically and put the entire incoming message into the MESSAGE part of the syslog message (available using the ${MESSAGE} macro). This flag is useful for parsing messages not complying to the syslog format.
If you are using the flags(no-parse) option, then syslog message parsing is completely disabled, and the entire incoming message is treated as the ${MESSAGE} part of a syslog message. In this case, AxoSyslog generates a new syslog header (timestamp, host, and so on) automatically. Note that even though flags(no-parse) disables message parsing, some flags can still be used, for example, the no-multi-line flag.
dont-store-legacy-msghdr: By default, AxoSyslog stores the original incoming header of the log message. This is useful if the original format of a non-syslog-compliant message must be retained (AxoSyslog automatically corrects minor header errors, for example, adds a whitespace before msg in the following message: Jan 22 10:06:11 host program:msg). If you do not want to store the original header of the message, enable the dont-store-legacy-msghdr flag.
sanitize-utf8: When using the sanitize-utf8 flag, AxoSyslog converts non-UTF-8 input to an escaped form, which is valid UTF-8.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
store-raw-message: Save the original message as received from the client in the ${RAWMSG} macro. You can forward this raw message in its original form to another AxoSyslog node using the syslog-ng() destination, or to a SIEM system, ensuring that the SIEM can process it. Available only in 3.16 and later.
syslog-protocol: The syslog-protocol flag specifies that incoming messages are expected to be formatted according to the new IETF syslog protocol standard (RFC5424), but without the frame header. Note that this flag is not needed for the syslog driver, which handles only messages that have a frame header.
validate-utf8: The validate-utf8 flag enables encoding-verification for messages.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
For RFC5424-formatted messages, if the BOM character is missing, but the message is otherwise UTF-8 compliant, AxoSyslog automatically adds the BOM character to the message.
The byte order mark (BOM) is a Unicode character used to signal the byte-order of the message text.
follow-freq()
Type:
number
Default:
1
Description: Indicates that the source should be checked periodically. This is useful for files which always indicate readability, even though no new lines were appended. If this value is higher than zero, AxoSyslog will not attempt to use poll() on the file, but checks whether the file changed every time the follow-freq() interval (in seconds) has elapsed. Floating-point numbers (for example, 1.5) can be used as well.
hook-commands()
Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.
Note
The AxoSyslog application must be able to start and restart the external program, and have the necessary permissions to do so. For example, if your host is running AppArmor or SELinux, you might have to modify your AppArmor or SELinux configuration to enable AxoSyslog to execute external applications.
Using hook-commands() when AxoSyslog starts or stops
To execute an external program when AxoSyslog starts or stops, use the following options:
startup()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog starts.
shutdown()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog stops.
Using the hook-commands() when AxoSyslog reloads
To execute an external program when the AxoSyslog configuration is initiated or torn down, for example, on startup/shutdown or during a AxoSyslog reload, use the following options:
setup()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is initiated, for example, on startup or during a AxoSyslog reload.
teardown()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is stopped or torn down, for example, on shutdown or during a AxoSyslog reload.
Example: Using the hook-commands() with a network source
In the following example, the hook-commands() is used with the network() driver and it opens an iptables port automatically as AxoSyslog is started/stopped.
The assumption in this example is that the LOGCHAIN chain is part of a larger ruleset that routes traffic to it. Whenever the AxoSyslog created rule is there, packets can flow, otherwise the port is closed.
Description: Specifies whether AxoSyslog should accept the timestamp received from the sending application or client. If disabled, the time of reception will be used instead. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.
Warning
To use the S_ macros, the keep-timestamp() option must be enabled (this is the default behavior of AxoSyslog).
log-fetch-limit()
Type:
number
Default:
100
Description: The maximum number of messages fetched from a source during a single poll loop. The destination queues might fill up before flow-control could stop reading if log-fetch-limit() is too high.
log-iw-size()
Type:
number
Default:
10000
Description: The size of the initial window, this value is used during flow control. Make sure that log-iw-size() is larger than the value of log-fetch-limit().
When using wildcards in the filenames, AxoSyslog attempts to read log-fetch-limit() number of messages from each file. For optimal performance, make sure that log-iw-size() is greater than log-fetch-limit()*max-files(). Note that to avoid performance problems, if log-iw-size()/max-files() is smaller than 100, AxoSyslog automatically sets log-iw-size() to max-files()*100.
Example: Initial window size of file sources
If log-fetch-limit() is 100, and your wildcard file source has 200 files, then log-iw-size() should be at least 20000.
log-msg-size()
Type:
number (bytes)
Default:
Use the global log-msg-size() option, which defaults to 65536 (64 KiB).
Description: Maximum length of an incoming message in bytes. This length includes the entire message (the data structure and individual fields). The maximal value that can be set is 268435456 bytes (256 MiB).
For messages using the IETF-syslog message format (RFC5424), the maximal size of the value of an SDATA field is 64 KiB.
Note
In most cases, log-msg-size() does not need to be set higher than 10 MiB.
Uses the value of the global option if not specified.
log-prefix() (DEPRECATED)
Type:
string
Default:
Description: A string added to the beginning of every log message. It can be used to add an arbitrary string to any log source, though it is most commonly used for adding kernel: to the kernel messages on Linux.
Note
This option is deprecated. Use program-override instead.
max-files()
Type:
integer
Default:
100
Description: Limits the number of files that the wildcard-file source monitors.
When using wildcards, AxoSyslog monitors every matching file (up to the limit set in the max-files() option), and can receive new log messages from any of the files. However, monitoring (polling) many files (that is, more than ten) has a significant overhead and may affect performance. On Linux this overhead is not so significant, because AxoSyslog uses the inotify feature of the kernel. Set the max-files() option at least to the number of files you want to monitor. If the wildcard-file source matches more files than the value of the max-files() option, it is random which files will AxoSyslog actually monitor. The default value of max-files() is 100.
monitor-method()
Type:
`auto
Default:
auto
Description: If the platform supports inotify, AxoSyslog uses it automatically to detect changes to the source files. If inotify is not available, AxoSyslog polls the files as set in the follow-freq() option. To force AxoSyslog poll the files even if inotify is available, set this option to poll.
multi-line-garbage()
Type:
regular expression
Default:
empty string
Description: Use the multi-line-garbage() option when processing multi-line messages that contain unneeded parts between the messages. Specify a string or regular expression that matches the beginning of the unneeded message parts. If the multi-line-garbage() option is set, AxoSyslog ignores the lines between the line matching the multi-line-garbage() and the next line matching multi-line-prefix(). See also the multi-line-prefix() option.
When receiving multi-line messages from a source when the multi-line-garbage() option is set, but no matching line is received between two lines that match multi-line-prefix(), AxoSyslog will continue to process the incoming lines as a single message until a line matching multi-line-garbage() is received.
To use the multi-line-garbage() option, set the multi-line-mode() option to prefix-garbage.
Warning
If the multi-line-garbage() option is set, AxoSyslog discards lines between the line matching the multi-line-garbage() and the next line matching multi-line-prefix().
Description: Use the multi-line-mode() option when processing multi-line messages. The AxoSyslog application provides the following methods to process multi-line messages:
indented: The indented mode can process messages where each line that belongs to the previous line is indented by whitespace, and the message continues until the first non-indented line. For example, the Linux kernel (starting with version 3.5) uses this format for /dev/log, as well as several applications, like Apache Tomcat.
prefix-garbage: The prefix-garbage mode uses a string or regular expression (set in multi-line-prefix()) that matches the beginning of the log messages, ignores newline characters from the source until a line matches the regular expression again, and treats the lines between the matching lines as a single message. For details on using multi-line-mode(prefix-garbage), see the multi-line-prefix() and multi-line-garbage() options.
prefix-suffix: The prefix-suffix mode uses a string or regular expression (set in multi-line-prefix()) that matches the beginning of the log messages, ignores newline characters from the source until a line matches the regular expression set in multi-line-suffix(), and treats the lines between multi-line-prefix() and multi-line-suffix() as a single message. Any other lines between the end of the message and the beginning of a new message (that is, a line that matches the multi-line-prefix() expression) are discarded. For details on using multi-line-mode(prefix-suffix), see the multi-line-prefix() and multi-line-suffix() options.
The prefix-suffix mode is similar to the prefix-garbage mode, but it appends the garbage part to the message instead of discarding it.
smart: The smart mode recognizes multi-line data backtraces even if they span multiple lines in the input. The backtraces are converted to a single log message for easier analysis. Backtraces for the following programming languages are recognized : Python, Java, JavaScript, PHP, Go, Ruby, and Dart.
smart mode is available in AxoSyslog version 4.2 and newer.
The regular expressions to recognize these programming languages are specified in an external file called /usr/share/syslog-ng/smart-multi-line.fsm (installation path depends on configure arguments), in a format that is described in that file.
Note
To format multi-line messages to your individual needs, consider the following:
To make multi-line messages more readable when written to a file, use a template in the destination and instead of the ${MESSAGE} macro, use the following: $(indent-multi-line ${MESSAGE}). This expression inserts a tab after every newline character (except when a tab is already present), indenting every line of the message after the first. For example:
To actually convert the lines of multi-line messages to single line (by replacing the newline characters with whitespaces), use the flags(no-multi-line) option in the source.
multi-line-prefix()
Type:
regular expression starting with the ^ character
Default:
empty string
Description: Use the multi-line-prefix() option to process multi-line messages, that is, log messages that contain newline characters (for example, Tomcat logs). Specify a string or regular expression that matches the beginning of the log messages (always start with the ^ character). Use as simple regular expressions as possible, because complex regular expressions can severely reduce the rate of processing multi-line messages. If the multi-line-prefix() option is set, AxoSyslog ignores newline characters from the source until a line matches the regular expression again, and treats the lines between the matching lines as a single message. See also the multi-line-garbage() option.
Note
To format multi-line messages to your individual needs, consider the following:
To make multi-line messages more readable when written to a file, use a template in the destination and instead of the ${MESSAGE} macro, use the following: $(indent-multi-line ${MESSAGE}). This expression inserts a tab after every newline character (except when a tab is already present), indenting every line of the message after the first. For example:
To actually convert the lines of multi-line messages to single line (by replacing the newline characters with whitespaces), use the flags(no-multi-line) option in the source.
Example: Processing Tomcat logs
The log messages of the Apache Tomcat server are a typical example for multi-line log messages. The messages start with the date and time of the query in the YYYY.MM.DD HH:MM:SS format, as you can see in the following example.
2010.06.09. 12:07:39 org.apache.catalina.startup.Catalina start
SEVERE: Catalina.start:
LifecycleException: service.getName(): "Catalina"; Protocol handler start failed: java.net.BindException: Address already in use null:8080
at org.apache.catalina.connector.Connector.start(Connector.java:1138) at org.apache.catalina.core.StandardService.start(StandardService.java:531) at org.apache.catalina.core.StandardServer.start(StandardServer.java:710) at org.apache.catalina.startup.Catalina.start(Catalina.java:583) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:177) 2010.06.09. 12:07:39 org.apache.catalina.startup.Catalina start
INFO: Server startup in 1206 ms
2010.06.09. 12:45:08 org.apache.coyote.http11.Http11Protocol pause
INFO: Pausing Coyote HTTP/1.1 on http-8080
2010.06.09. 12:45:09 org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
To process these messages, specify a regular expression matching the timestamp of the messages in the multi-line-prefix() option. Such an expression is the following:
Note that flags(no-parse) is needed to prevent AxoSyslog trying to interpret the date in the message.
multi-line-suffix()
Type:
regular expression
Default:
empty string
Description: Use the multi-line-suffix() option when processing multi-line messages. Specify a string or regular expression that matches the end of the multi-line message.
To use the multi-line-suffix() option, set the multi-line-mode() option to prefix-suffix. See also the multi-line-prefix() option.
pad-size()
Type:
number
Default:
0
Description: Specifies input padding. Some operating systems (such as HP-UX) pad all messages to block boundary. This option can be used to specify the block size. The AxoSyslog application will pad reads from the associated device to the number of bytes set in pad-size(). Mostly used on HP-UX where /dev/log is a named pipe and every write is padded to 2048 bytes. If pad-size() was given and the incoming message does not fit into pad-size(), AxoSyslog will not read anymore from this pipe and displays the following error message:
Padding was set, and couldn't read enough bytes
program-override()
Type:
string
Default:
Description: Replaces the ${PROGRAM} part of the message with the parameter string. For example, to mark every message coming from the kernel, include the program-override("kernel") option in the source containing /proc/kmsg.
recursive()
Type:
yes or no
Default:
no
Description: When enabled, AxoSyslog monitors every subdirectory of the path set in the base-dir() option, and reads log messages from files with matching filenames. The recursive option can be used together with wildcards in the filename.
Warning
If you use multiple wildcard-file() sources in your configuration, make sure that the files and folders that match the wildcards do not overlap. That is, every file and folder should belong to only one file source. Monitoring a file from multiple wildcard sources can lead to data loss.
Example: Monitoring multiple directories
The following example reads files having the .log extension from the /var/log/ directory and its subdirectories, including for example, the /var/log/apt/history.log file.
Description: Label the messages received from the source with custom tags. Tags must be unique, and enclosed between double quotes. When adding multiple tags, separate them with comma, for example, tags("dmz", "router"). This option is available only in version 3.1 and later.
time-zone()
Type:
name of the timezone, or the timezone offset
Default:
Description: The default timezone for messages read from the source. Applies only if no timezone is specified within the message itself.
The timezone can be specified by using the name, for example, time-zone("Europe/Budapest")), or as the timezone offset in +/-HH:MM format, for example, +01:00). On Linux and UNIX platforms, the valid timezone names are listed under the /usr/share/zoneinfo directory.
7.8 - Hypr Audit Trail and Hypr App Audit Trail
Starting with version 4.2.0, AxoSyslog can fetch events from the Hypr REST API using the hypr-audit-trail() and hypr-app-audit-trail() source drivers.
hypr-audit-trail(): is a source driver that pulls messages from the Hypr API, associated to any RP Application ID.
hypr-app-audit-trail(): is a source driver that pulls messages from the Hypr API, but only those associated to a specific RP Application ID.
Hypr Audit Trail
The hypr-audit-trail() source queries the Hypr API for the list of potential applications at startup, then monitors the audit trail for each of the detected applications.
Note: Applications that are registered after syslog-ng is started are not recognized.
To start following those audit trails, you must restart syslog-ng.
source s_hypr { hypr-audit-trail( url('https://<custom domain>.hypr.com') bearer-token('<base64 encoded bearer token>') page-size(<number of results to return in a single page>) initial-hours(<number of hours to search backward on initial fetch>) application-skip-list('HYPRDefaultApplication', 'HYPRDefaultWorkstationApplication') log-level('INFO') flags(<optional flags passed to the source>) ignore-persistence(<yes/no>));};
Available options:
url(): custom URL for Hypr API access ('https://<custom domain>.hypr.com')
bearer-token(): base64 encoded authentication token from Hypr
page-size(): number of results to return in a single page (optional - defaults to 100)
initial-hours(): number of hours to search backward on initial fetch (optional - defaults to 4)
application-skip-list(): list of rpAppIds not to retrieve from Hypr (optional - defaults to 'HYPRDefaultApplication', 'HYPRDefaultWorkstationApplication')
log-level(): logging level, possible values: "DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL" (optional - defaults to "INFO")
flags(): flags passed to the source, can be used for example to disable message parsing with flags(no-parse) (optional - defaults to empty)
ignore-persistence(): ignores the saved value in the persist file, and starts querying from the current time (optional - defaults to no)
Hypr App Audit Trail
The hypr-app-audit-trail() monitors the audit trail for one specific RP Application ID. This driver requires the rp-app-id() parameter in order to operate.
Available options:
url(): custom URL for Hypr API access ('https://<custom domain>.hypr.com')
bearer-token(): base64 encoded authentication token from Hypr
rp-app-id(): the RP Application ID for the application to monitor
page-size(): number of results to return in a single page (optional - defaults to 100)
initial-hours(): number of hours to search backward on initial fetch (optional - defaults to 4)
log-level(): logging level, possible values: "DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL" (optional - defaults to "INFO")
flags(): flags passed to the source, can be used for example to disable message parsing with flags(no-parse) (optional - defaults to empty)
ignore-persistence(): ignores the saved value in the persist file, and starts querying from the current time (optional - defaults to no)
Acknowledgements
This documentation page is based on the README file of the hypr-audit-trail() source, written by Dan Elder.
7.9 - Jellyfin logs
Starting with version 4.7.0, you can use the jellyfin() source to read Jellyfin logs from its log file output.
This driver is actually a reusable configuration snippet. For details on using or writing such configuration snippets, see Reusing configuration blocks. You can find the source of this configuration snippet on GitHub.
7.10 - linux-audit: Collect messages from Linux audit logs
It reads and automatically parses the Linux audit logs. You can override the file name using the filename() parameter and the prefix for the created name-value pairs using the prefix() parameter. Any additional parameters are passed to the file source.
Note
Most recent Linux distributions enable Security-Enhanced Linux (SELinux) or AppArmor as a security measure. If enabled, these technologies might disable access to the Linux Audit log file by default. Consult their manuals to enable Linux Audit log access for AxoSyslog.
Description: The log file of linux-audit. The AxoSyslog application reads the Linux audit logs from this file.
prefix()
Synopsis:
prefix()
Default:
.auditd.
Description: Insert a prefix before the name part of the parsed name-value pairs to help further processing. For example:
To insert the my-parsed-data. prefix, use the prefix(my-parsed-data.) option.
To refer to a particular data that has a prefix, use the prefix in the name of the macro, for example, ${my-parsed-data.name}.
If you forward the parsed messages using the IETF-syslog protocol, you can insert all the parsed data into the SDATA part of the message using the prefix(.SDATA.my-parsed-data.) option.
Names starting with a dot (for example, .example) are reserved for use by AxoSyslog. Note that if you use an empty prefix (prefix("")) or one starting with a dot, AxoSyslog might replace the original value of an existing macro (note that only soft macros can be overwritten, see Hard versus soft macros for details). To avoid such problems, use a prefix when naming the parsed values, for example, prefix(my-parsed-data.)
7.11 - kubernetes: Collect and parse the Kubernetes CRI (Container Runtime Interface) format
The kubernetes() source collects container logs managed by the Kubelet. It reads plain-text and JSON-formatted container logs (as described in the Container Runtime Interface (CRI) design proposal), for example, from the /var/log/containers or /var/log/pods files, and enriches them with various metadata retrieved from the Kubernetes API.
The kubernetes() source is available in AxoSyslog version 3.37 and later.
By default, it reads the /var/log/containers folder and extracts:
the log content, and
Kubernetes metadata, for example, namespace, pod, and container information.
The Kubernetes-related metadata is available in name-value pairs with the .k8s. prefix. The following table shows the retrieved metadata and their source:
syslog-ng name-value pair
source
.k8s.namespace_name
Container log file name.
.k8s.pod_name
Container log file name.
.k8s.pod_uuid
Container log file name or python kubernetes.client.CoreV1Api.
.k8s.container_name
Container log file name or python kubernetes.client.CoreV1Api.
.k8s.container_id
Container log file name.
.k8s.container_image
python kubernetes.client.CoreV1Api.
.k8s.container_hash
python kubernetes.client.CoreV1Api.
.k8s.docker_id
python kubernetes.client.CoreV1Api.
.k8s.labels.*
python kubernetes.client.CoreV1Api.
.k8s.annotations.*
python kubernetes.client.CoreV1Api.
Declaration
kubernetes( base-dir("<pathname>"));
7.11.1 - kubernetes() source options
The kubernetes() source has the following options:
base-dir()
Type:
path without filename
Default:
/var/log/containers
Description: The path to the directory that contains the log files, for example, base-dir("/var/log/pods").
cluster-name()
Type:
string
Default:
k8s
Description: The name of the Kubernetes cluster.
key-delimiter()
Type:
character
Default:
.
Description: The delimiter character to use when parsing flattened keys. Supports Only single characters.
prefix()
Synopsis:
prefix()
Description: Insert a prefix before the name part of the parsed name-value pairs to help further processing. For example:
To insert the my-parsed-data. prefix, use the prefix(my-parsed-data.) option.
To refer to a particular data that has a prefix, use the prefix in the name of the macro, for example, ${my-parsed-data.name}.
If you forward the parsed messages using the IETF-syslog protocol, you can insert all the parsed data into the SDATA part of the message using the prefix(.SDATA.my-parsed-data.) option.
Names starting with a dot (for example, .example) are reserved for use by AxoSyslog. If you use such a macro name as the name of a parsed value, it will attempt to replace the original value of the macro (note that only soft macros can be overwritten, see Hard versus soft macros for details). To avoid such problems, use a prefix when naming the parsed values, for example, prefix(my-parsed-data.)
The prefix() option is optional and its default value is ".k8s.".
7.12 - mbox: Convert local email messages to log messages
Using the mbox() driver, AxoSyslog can read email messages from local mbox files, and convert them to multiline log messages.
This driver has only one required option, the filename of the mbox file. To use the mbox() driver, the scl.conf file must be included in your AxoSyslog configuration:
@include "scl.conf"
The mbox() driver is actually a reusable configuration snippet configured to read log messages using the file() driver. For details on using or writing such configuration snippets, see Reusing configuration blocks. You can find the source of the configuration snippet on GitHub.
Example: Using the mbox() driver
The following example reads the emails of the root user on the AxoSyslog host.
Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.
Note
The AxoSyslog application must be able to start and restart the external program, and have the necessary permissions to do so. For example, if your host is running AppArmor or SELinux, you might have to modify your AppArmor or SELinux configuration to enable AxoSyslog to execute external applications.
Using hook-commands() when AxoSyslog starts or stops
To execute an external program when AxoSyslog starts or stops, use the following options:
startup()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog starts.
shutdown()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog stops.
Using the hook-commands() when AxoSyslog reloads
To execute an external program when the AxoSyslog configuration is initiated or torn down, for example, on startup/shutdown or during a AxoSyslog reload, use the following options:
setup()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is initiated, for example, on startup or during a AxoSyslog reload.
teardown()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is stopped or torn down, for example, on shutdown or during a AxoSyslog reload.
Example: Using the hook-commands() with a network source
In the following example, the hook-commands() is used with the network() driver and it opens an iptables port automatically as AxoSyslog is started/stopped.
The assumption in this example is that the LOGCHAIN chain is part of a larger ruleset that routes traffic to it. Whenever the AxoSyslog created rule is there, packets can flow, otherwise the port is closed.
Note
The rest of this chapter and its sections build on your familiarity with the MQTT protocol, the concept of client and broker entities, and how these entities function within an MQTT system.
Starting with AxoSyslog version 4.7, mqtt() source automatically sets the ${MQTT_TOPIC} name-value pair for the messages it receives. This is useful when the name of the topic contains MQTT wildcards ($, +, #). For example:
Example: Using the mqtt() source in your configuration
The following example illustrates an mqtt() source configured to fetch messages from the MQTT broker running on localhost:4444 using the test/test topic, and send them to the localhost:4445 address.
Using the current implementation of the mqtt() source has the following prerequisites:
Installing the eclipse-paho-mqtt-c library.
Note
The default package manager for some Linux operating systems contains the eclipse-paho-mqtt-c library, but depending on your OS, you may have to install the library manually. For more information about how you can download and install the eclipse-paho-mqtt-c library, see Eclipse Paho on the Eclipse Foundation website.
Having a broker entity in a functional MQTT system.
Note
In your configuration, you will specify the broker entity of your MQTT system in the address() option of your mqtt() source.
7.13.2 - Limitations to using the mqtt() source
Using the mqtt() source of AxoSyslog has the following limitations:
You can only use the mqtt() source with AxoSyslog version 3.35 or higher.
You cannot use the mqtt() source without installing the the eclipse-paho-mqtt-c library.
For more information about how you can download and install the eclipse-paho-mqtt-c library, see Eclipse Paho on the Eclipse Foundation website.
The current implementation of the mqtt() source supports versions 3.1 and 3.1.1 of the MQTT protocol
7.13.3 - Options of the mqtt() source
The mqtt() source has the following options.
Required options: address(), fallback-topic(), and topic().
address()
Type:
string
Default:
tcp://localhost:1883
Required:
yes
Description: Specifies the hostname or IP address, and the port number of the MQTT broker to which AxoSyslog will send the log messages.
Syntax: <protocol type>://<host>:<port>
client-id()
Type:
string
Default:
syslog-ng-source-{topic option}
Required:
no
Description: The client-id() is used to identify the client to the MQTT server, which stores session data for each client. The session data can contains information regarding which message has been sent, received. It is not possible to set the client-id() to an empty string. To always start a new session see the cleansession() option.
cleansession()
Type:
yes or no
Default:
no
Description: This option instruments the MQTT broker to clean the session data when connecting. The session data contains information about which message was processed.
http-proxy()
Type:
URL
Default:
N/A
Description: Specifies HTTP/HTTPS proxy for WebSocket connections.
keep-alive()
Type:
positive integer number (in seconds)
Default:
60
Description: Specifies the number of seconds that AxoSyslog keeps the connection between the broker and clients open in case there is no message traffic. When keep-alive() number of seconds pass, the connection is terminated, and you have to reconnect.
On the MQTT side, the keep alive function provides a workaround method to access connections that are still open.
password()
Type:
string
Default:
N/A
Description: The password used to authenticate on the MQTT broker.
persist-name()
Type:
string
Default:
N/A
Description: If you receive the following error message during AxoSyslog startup, set the persist-name() option of the duplicate drivers:
Error checking the uniqueness of the persist names, please override it with persist-name option. Shutting down.
This error happens if you use identical drivers in multiple sources, for example, if you configure two file sources to read from the same file. In this case, set the persist-name() of the drivers to a custom string, for example, persist-name("example-persist-name1").
qos()
Type:
number
Default:
`0`
Possible values:
`0` - at most once (the fastest option)
`1` - at least once (a much slower option than `0`)
`2` - exactly once (the slowest option)
Description: The Quality of Service (QoS) level in MQTT messaging is an agreement between sender and receiver on the guarantee of delivering a message.
tls()
Type:
tls options
Default:
n/a
Description: This option sets various options related to TLS encryption, for example, key/certificate files and trusted CA locations. TLS can be used only with tcp-based transport protocols. For details, see TLS options.
The following options are relevant for the mqtt()tls() block: ca-dir(), ca-file(), cert-file(), cipher-suite(), key-file(), peer-verify(), ssl-version(), use-system-cert-store().
topic()
Type:
string or template
Default:
N/A
Description: Required option. Specifies the MQTT topic.
username()
Type:
string
Default:
N/A
Description: The username used to authenticate on the MQTT broker.
7.14 - network: Collect messages using the RFC3164 protocol (network() driver)
The network() source driver can receive syslog messages conforming to RFC3164 from the network using the TCP, TLS, and UDP networking protocols.
UDP is a simple datagram oriented protocol, which provides “best effort service” to transfer messages between hosts. It may lose messages, and no attempt is made to retransmit lost messages. The BSD-syslog protocol traditionally uses UDP.
Use UDP only if you have no other choice.
TCP provides connection-oriented service: the client and the server establish a connection, each message is acknowledged, and lost packets are resent. TCP can detect lost connections, and messages are lost, only if the TCP connection breaks. When a TCP connection is broken, messages that the client has sent but were not yet received on the server are lost.
The AxoSyslog application supports TLS (Transport Layer Security, also known as SSL) over TCP. For details, see Encrypting log messages with TLS.
Declaration:
network([options]);
By default, the network() driver binds to 0.0.0.0, meaning that it listens on every available IPV4 interface on the TCP/514 port. To limit accepted connections to only one interface, use the localip() parameter. To listen on IPv6 addresses, use the ip-protocol(6) option.
Example: Using the network() driver
Using only the default settings: listen on every available IPV4 interface on the TCP/514 port.
source s_network { network();};
UDP source listening on 192.168.1.1 (the default port for UDP is 514):
A TCP source listening for messages using the IETF-syslog message format. Note that for transferring IETF-syslog messages, generally you are recommended to use the syslog() driver on both the client and the server, as it uses both the IETF-syslog message format and the protocol. For details, see syslog: Collect messages using the IETF-syslog protocol.
Description: The name of a directory that contains a set of trusted CA certificates in PEM format. The CA certificate files have to be named after the 32-bit hash of the subject’s name. This naming can be created using the c_rehash utility in openssl. For an example, see Configuring TLS on the AxoSyslog clients. The AxoSyslog application uses the CA certificates in this directory to validate the certificate of the peer.
This option can be used together with the optional ca-file() option.
ca-file()
Accepted values:
File name
Default:
empty
Description: Optional. The name of a file that contains a set of trusted CA certificates in PEM format. The AxoSyslog application uses the CA certificates in this file to validate the certificate of the peer.
Example format in configuration:
ca-file("/etc/pki/tls/certs/ca-bundle.crt")
Note
The ca-file() option can be used together with the ca-dir() option, and it is relevant when peer-verify() is set to other than no or optional-untrusted.
check-hostname()
Type:
boolean (yes or no)
Default:
Use the global check-hostname() option, which defaults to no.
Checks that the hostname contains valid characters. Uses the value of the global option if not specified.
By default, the kernel chooses the receive socket for a specific UDP randomly based on the source IP/port of the sender. You can customize this algorithm using the Extended Berkeley Packet Filter (eBPF) plugin. The ebpf() option changes the kernel’s SO_REUSEPORT algorithm so that all messages are randomly placed into one of the UDP sockets. The decision which UDP socket buffer a datagram is placed is made for every datagram, and not once for every stream. This means that messages are perfectly load-balanced across your set of UDP sockets. While this resolves the imbalance between the sockets and results in perfect load balancing, you will lose ordering between messages from the same sender, which is the price to pay for increased throughput.
Description: Specifies the character set (encoding, for example, UTF-8) of messages using the legacy BSD-syslog protocol. To list the available character sets on a host, execute the iconv -l command. For details on how encoding affects the size of the message, see Message size and encoding.
Description: Specifies the log parsing options of the source.
assume-utf8: The assume-utf8 flag assumes that the incoming messages are UTF-8 encoded, but does not verify the encoding. If you explicitly want to validate the UTF-8 encoding of the incoming message, use the validate-utf8 flag.
empty-lines: Use the empty-lines flag to keep the empty lines of the messages. By default, AxoSyslog removes empty lines automatically.
expect-hostname: If the expect-hostname flag is enabled, AxoSyslog will assume that the log message contains a hostname and parse the message accordingly. This is the default behavior for TCP sources. Note that pipe sources use the no-hostname flag by default.
guess-timezone: Attempt to guess the timezone of the message if this information is not available in the message. Works when the incoming message stream is close to real time, and the timezone information is missing from the timestamp.
kernel: The kernel flag makes the source default to the LOG_KERN | LOG_NOTICE priority if not specified otherwise.
no-header: The no-header flag triggers AxoSyslog to parse only the PRI field of incoming messages, and put the rest of the message contents into $MSG.
Its functionality is similar to that of the no-parse flag, except the no-header flag does not skip the PRI field.
Note
Essentially, the no-header flag signals AxoSyslog that the syslog header is not present (or does not adhere to the conventions / RFCs), so the entire message (except from the PRI field) is put into $MSG.
Example: using the no-header flag with the syslog-parser() parser
The following example illustrates using the no-header flag with the syslog-parser() parser:
no-hostname: Enable the no-hostname flag if the log message does not include the hostname of the sender host. That way AxoSyslog assumes that the first part of the message header is ${PROGRAM} instead of ${HOST}. For example:
no-multi-line: The no-multi-line flag disables line-breaking in the messages: the entire message is converted to a single line. Note that this happens only if the underlying transport method actually supports multi-line messages. Currently the file() and pipe() drivers support multi-line messages.
no-parse: By default, AxoSyslog parses incoming messages as syslog messages. The no-parse flag completely disables syslog message parsing and processes the complete line as the message part of a syslog message. The AxoSyslog application will generate a new syslog header (timestamp, host, and so on) automatically and put the entire incoming message into the MESSAGE part of the syslog message (available using the ${MESSAGE} macro). This flag is useful for parsing messages not complying to the syslog format.
If you are using the flags(no-parse) option, then syslog message parsing is completely disabled, and the entire incoming message is treated as the ${MESSAGE} part of a syslog message. In this case, AxoSyslog generates a new syslog header (timestamp, host, and so on) automatically. Note that even though flags(no-parse) disables message parsing, some flags can still be used, for example, the no-multi-line flag.
dont-store-legacy-msghdr: By default, AxoSyslog stores the original incoming header of the log message. This is useful if the original format of a non-syslog-compliant message must be retained (AxoSyslog automatically corrects minor header errors, for example, adds a whitespace before msg in the following message: Jan 22 10:06:11 host program:msg). If you do not want to store the original header of the message, enable the dont-store-legacy-msghdr flag.
sanitize-utf8: When using the sanitize-utf8 flag, AxoSyslog converts non-UTF-8 input to an escaped form, which is valid UTF-8.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
store-raw-message: Save the original message as received from the client in the ${RAWMSG} macro. You can forward this raw message in its original form to another AxoSyslog node using the syslog-ng() destination, or to a SIEM system, ensuring that the SIEM can process it. Available only in 3.16 and later.
syslog-protocol: The syslog-protocol flag specifies that incoming messages are expected to be formatted according to the new IETF syslog protocol standard (RFC5424), but without the frame header. Note that this flag is not needed for the syslog driver, which handles only messages that have a frame header.
validate-utf8: The validate-utf8 flag enables encoding-verification for messages.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
For RFC5424-formatted messages, if the BOM character is missing, but the message is otherwise UTF-8 compliant, AxoSyslog automatically adds the BOM character to the message.
The byte order mark (BOM) is a Unicode character used to signal the byte-order of the message text.
threaded: The threaded flag enables multithreading for the source. For details on multithreading, see Multithreading and scaling.
Note
The syslog source uses multiple threads only if the source uses the tls or tcp transport protocols.
hook-commands()
Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.
Note
The AxoSyslog application must be able to start and restart the external program, and have the necessary permissions to do so. For example, if your host is running AppArmor or SELinux, you might have to modify your AppArmor or SELinux configuration to enable AxoSyslog to execute external applications.
Using hook-commands() when AxoSyslog starts or stops
To execute an external program when AxoSyslog starts or stops, use the following options:
startup()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog starts.
shutdown()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog stops.
Using the hook-commands() when AxoSyslog reloads
To execute an external program when the AxoSyslog configuration is initiated or torn down, for example, on startup/shutdown or during a AxoSyslog reload, use the following options:
setup()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is initiated, for example, on startup or during a AxoSyslog reload.
teardown()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is stopped or torn down, for example, on shutdown or during a AxoSyslog reload.
Example: Using the hook-commands() with a network source
In the following example, the hook-commands() is used with the network() driver and it opens an iptables port automatically as AxoSyslog is started/stopped.
The assumption in this example is that the LOGCHAIN chain is part of a larger ruleset that routes traffic to it. Whenever the AxoSyslog created rule is there, packets can flow, otherwise the port is closed.
Description: Replaces the ${HOST} part of the message with the parameter string.
interface()
Type:
string
Default:
None
Description: Bind to the specified interface instead of an IP address. Available in 3.19 and later.
ip() or localip()
Type:
string
Default:
0.0.0.0
Description: The IP address to bind to. By default, AxoSyslog listens on every available interface. Note that this is not the address where messages are accepted from.
If you specify a multicast bind address and use the udp transport, AxoSyslog automatically joins the necessary multicast group. TCP does not support multicasting.
ip-protocol()
Type:
number
Default:
4
Description: Determines the internet protocol version of the given driver (network() or syslog()). The possible values are 4 and 6, corresponding to IPv4 and IPv6. The default value is ip-protocol(4).
Note that listening on a port using IPv6 automatically means that you are also listening on that port using IPv4. That is, if you want to have receive messages on an IP-address/port pair using both IPv4 and IPv6, create a source that uses the ip-protocol(6). You cannot have two sources with the same IP-address/port pair, but with different ip-protocol() settings (it causes an Address already in use error).
For example, the following source receives messages on TCP, using the network() driver, on every available interface of the host on both IPv4 and IPv6.
Description: Specifies the Type-of-Service value of outgoing packets.
ip-ttl()
Type:
number
Default:
0
Description: Specifies the Time-To-Live value of outgoing packets.
keep-alive()
Type:
yes or no
Default:
yes
Description: Specifies whether connections to sources should be closed when AxoSyslog is forced to reload its configuration (upon the receipt of a SIGHUP signal). Note that this applies to the server (source) side of the AxoSyslog connections, client-side (destination) connections are always reopened after receiving a HUP signal unless the keep-alive option is enabled for the destination.
keep-hostname()
Type:
yes or no
Default:
no
Description: Enable or disable hostname rewriting.
If enabled (keep-hostname(yes)), AxoSyslog assumes that the incoming log message was sent by the host specified in the HOST field of the message.
If disabled (keep-hostname(no)), AxoSyslog rewrites the HOST field of the message, either to the IP address (if the use-dns() parameter is set to no), or to the hostname (if the use-dns() parameter is set to yes and the IP address can be resolved to a hostname) of the host sending the message to AxoSyslog. For details on using name resolution in AxoSyslog, see Using name resolution in syslog-ng.
Note
If the log message does not contain a hostname in its HOST field, AxoSyslog automatically adds a hostname to the message.
For messages received from the network, this hostname is the address of the host that sent the message (this means the address of the last hop if the message was transferred via a relay).
For messages received from the local host, AxoSyslog adds the name of the host.
This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.
Note
When relaying messages, enable this option on the AxoSyslog server and also on every relay, otherwise AxoSyslog will treat incoming messages as if they were sent by the last relay.
keep-timestamp()
Type:
yes or no
Default:
yes
Description: Specifies whether AxoSyslog should accept the timestamp received from the sending application or client. If disabled, the time of reception will be used instead. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.
Warning
To use the S_ macros, the keep-timestamp() option must be enabled (this is the default behavior of AxoSyslog).
listen-backlog()
Type:
integer
Default:
256
Description: Available only for stream based transports (unix-stream, tcp, tls). In TCP, connections are treated incomplete until the three-way handshake is completed between the server and the client. Incomplete connection requests wait on the TCP port for the listener to accept the request. The listen-backlog() option sets the maximum number of incomplete connection requests. For example:
Description: The maximum number of messages fetched from a source during a single poll loop. The destination queues might fill up before flow-control could stop reading if log-fetch-limit() is too high.
log-iw-size()
Type:
number
Default:
100
Description: The size of the initial window, this value is used during flow-control. Its value cannot be lower than 100, unless the dynamic-window-size() option is enabled. For details on flow-control, see Managing incoming and outgoing messages with flow-control.
If the max-connections() option is set, the log-iw-size() will be divided by the number of connections, otherwise log-iw-size() is divided by 10 (the default value of the max-connections() option). The resulting number is the initial window size of each connection. For optimal performance when receiving messages from AxoSyslog clients, make sure that the window size is larger than the flush-lines() option set in the destination of your clients.
Example: Initial window size of a connection
If log-iw-size(1000) and max-connections(10), then each connection will have an initial window size of 100.
log-msg-size()
Type:
number (bytes)
Default:
Use the global log-msg-size() option, which defaults to 65536 (64 KiB).
Description: Maximum length of an incoming message in bytes. This length includes the entire message (the data structure and individual fields). The maximal value that can be set is 268435456 bytes (256 MiB).
For messages using the IETF-syslog message format (RFC5424), the maximal size of the value of an SDATA field is 64 KiB.
Note
In most cases, log-msg-size() does not need to be set higher than 10 MiB.
Uses the value of the global option if not specified.
max-connections()
Type:
number
Default:
10
Description: Specifies the maximum number of simultaneous connections.
pad-size()
Type:
number
Default:
0
Description: Specifies input padding. Some operating systems (such as HP-UX) pad all messages to block boundary. This option can be used to specify the block size. The AxoSyslog application will pad reads from the associated device to the number of bytes set in pad-size(). Mostly used on HP-UX where /dev/log is a named pipe and every write is padded to 2048 bytes. If pad-size() was given and the incoming message does not fit into pad-size(), AxoSyslog will not read anymore from this pipe and displays the following error message:
Padding was set, and couldn't read enough bytes
port() or localport()
Type:
number
Default:
In case of TCP transport: 514
In case of UDP transport: 514
Description: The port number to bind to.
program-override()
Type:
string
Default:
Description: Replaces the ${PROGRAM} part of the message with the parameter string. For example, to mark every message coming from the kernel, include the program-override("kernel") option in the source containing /proc/kmsg.
so-broadcast()
Type:
yes or no
Default:
no
Description: This option controls the SO_BROADCAST socket option required to make AxoSyslog send messages to a broadcast address. For details, see the socket(7) manual page.
so-keepalive()
Type:
yes or no
Default:
no
Description: Enables keep-alive messages, keeping the socket open. This only effects TCP and UNIX-stream sockets. For details, see the socket(7) manual page.
so-rcvbuf()
Type:
number
Default:
0
Description: Specifies the size of the socket receive buffer in bytes. For details, see the socket(7) manual page.
Warning
When receiving messages using the UDP protocol, increase the size of the UDP receive buffer on the receiver host (that is, the AxoSyslog server or relay receiving the messages). Note that on certain platforms, for example, on Red Hat Enterprise Linux 5, even low message load (~200 messages per second) can result in message loss, unless the so-rcvbuf() option of the source is increased. In this cases, you will need to increase the net.core.rmem_max parameter of the host (for example, to 1024000), but do not modify net.core.rmem_default parameter.
As a general rule, increase the so-rcvbuf() so that the buffer size in kilobytes is higher than the rate of incoming messages per second. For example, to receive 2000 messages per second, set the so-rcvbuf() at least to 2 097 152 bytes.
so-reuseport()
Type:
yes or no
Default:
no
Description: Enables SO_REUSEPORT on systems that support it. When enabled, the kernel allows multiple UDP sockets to be bound to the same port, and the kernel load-balances incoming UDP datagrams to the sockets. The sockets are distributed based on the hash of (srcip, dstip, srcport, dstport), so the same listener should be receiving packets from the same endpoint. For example:
Enables keep-alive messages, keeping the socket open. This only effects TCP and UNIX-stream sockets. For details, see the socket(7) manual page.
so-sndbuf()
Type:
number
Default:
0
Description: Specifies the size of the socket send buffer in bytes. For details, see the socket(7) manual page.
tags()
Type:
string
Default:
Description: Label the messages received from the source with custom tags. Tags must be unique, and enclosed between double quotes. When adding multiple tags, separate them with comma, for example, tags("dmz", "router"). This option is available only in version 3.1 and later.
time-zone()
Type:
name of the timezone, or the timezone offset
Default:
Description: The default timezone for messages read from the source. Applies only if no timezone is specified within the message itself.
The timezone can be specified by using the name, for example, time-zone("Europe/Budapest")), or as the timezone offset in +/-HH:MM format, for example, +01:00). On Linux and UNIX platforms, the valid timezone names are listed under the /usr/share/zoneinfo directory.
Description: Specifies the protocol used to receive messages from the source.
For detailed information about how AxoSyslog supports the proxied-tcp, the proxied-tls, and the proxied-tls-passthrough parameters, see Proxy Protocol support.
text-with-nuls: Allows embedded NUL characters in the message from a TCP source, that is, AxoSyslog will not delimiter the incoming messages on NUL characters, only on newline characters (contrary to tcp transport, which splits the incoming log on newline characters and NUL characters).
Note
The AxoSyslog application does not support embedded NUL characters everywhere, so it is recommended that you also use flags(no-multi-line) that causes NUL characters to be replaced by space.
Warning
When receiving messages using the UDP protocol, increase the size of the UDP receive buffer on the receiver host (that is, the AxoSyslog server or relay receiving the messages). Note that on certain platforms, for example, on Red Hat Enterprise Linux 5, even low message load (~200 messages per second) can result in message loss, unless the so-rcvbuf() option of the source is increased. In this cases, you will need to increase the net.core.rmem_max parameter of the host (for example, to 1024000), but do not modify net.core.rmem_default parameter.
As a general rule, increase the so-rcvbuf() so that the buffer size in kilobytes is higher than the rate of incoming messages per second. For example, to receive 2000 messages per second, set the so-rcvbuf() at least to 2 097 152 bytes.
trim-large-messages()
Type:
`yes
Default:
Use the global trim-large-messages() option, which defaults to no.
Description: Determines what AxoSyslog does with incoming log messages that are received using the IETF-syslog protocol using the syslog() driver, and are longer than the value of log-msg-size(). Other drivers ignore the trim-large-messages() option.
If set to no, AxoSyslog drops the incoming log message.
If set to yes, AxoSyslog trims the incoming log message to the size set in log-msg-size(), and adds the trimmed tag to the message. The rest of the message is dropped. You can use the tag to filter on such messages.
filter f_trimmed { tags("trimmed");};
If AxoSyslog trims a log message, it sends a debug-level log message to its internal() source.
As a result of trimming, a parser could fail to parse the trimmed message. For example, a trimmed JSON or XML message will not be valid JSON or XML.
Available in AxoSyslog version 3.21 and later.
Uses the value of the global option if not specified.
tls()
Type:
tls options
Default:
n/a
Description: This option sets various options related to TLS encryption, for example, key/certificate files and trusted CA locations. TLS can be used only with tcp-based transport protocols. For details, see TLS options.
use-dns()
Type:
yes, no, persist_only
Default:
yes
Description: Enable or disable DNS usage. The persist_only option attempts to resolve hostnames locally from file (for example, from /etc/hosts). The AxoSyslog application blocks on DNS queries, so enabling DNS may lead to a Denial of Service attack. To prevent DoS, protect your AxoSyslog network endpoint with firewall rules, and make sure that all hosts which may get to AxoSyslog are resolvable. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.
Note
This option has no effect if the keep-hostname() option is enabled (keep-hostname(yes)) and the message contains a hostname.
use-fqdn()
Type:
yes or no
Default:
no
Description: Use this option to add a Fully Qualified Domain Name (FQDN) instead of a short hostname. You can specify this option either globally or per-source. The local setting of the source overrides the global option if available.
Note
Set use-fqdn() to yes if you want to use the custom-domain() global option.
Note
This option has no effect if the keep-hostname() option is enabled (keep-hostname(yes)) and the message contains a hostname.
7.14.2 - Proxy Protocol support
If you connect load balancers to your AxoSyslog application, AxoSyslog identifies every connection that is connected to the load balancers identically by default, regardless of the source IP or the source protocol. Essentially, the load balancer masks the source IP unless you enable Proxy Protocol support for your proxy TLS transport() to inject information about the original connection into the forwarded TCP session.
For further details about the working mechanism behind the Proxy Protocol support on AxoSyslog and the configuration details, see the following sections:
7.14.2.1 - The working mechanism behind the Proxy Protocol
This section describes how AxoSyslog supports the Proxy Protocol.
The working mechanism behind the Proxy Protocol
When using the Proxy Protocol during load balancing, AxoSyslog detects the information behind connections connected to the load balancer, then parses the injected information and adds the following macros to every message the comes through the connection later on:
PROXY_SRCIP (the source IP of the proxy)
PROXY_SRCPORT (the source port of the proxy)
PROXY_DSTIP (the destination IP of the proxy)
PROXY_DSTPORT (the destination port of the proxy)
Note
Consider the following about macros and headers:
When the proxy protocol header is PROXY UNKNOWN, no additional macros are added.
When AxoSyslog cannot parse a proxy protocol header, the connection is closed:
Note
Originally, the driver supported version 1 of the PROXY protocol (TCP4, TCP6, and TLS connections). PROXY protocol v2 support is available in AxoSyslog version 4.1 and later.
7.14.2.2 - Proxy Protocol: configuration and output examples
This section provides information about enabling Proxy Protocol support in your network() source options, and an example configuration and output to illustrate how the Proxy Protocol method works in AxoSyslog.
Enabling Proxy Protocol support for your network() source options
Unless you enable Proxy Protocol support for your network() source, AxoSyslog identifies every connection that is connected to the load balancers identically by default, regardless of the source IP or the source protocol.
proxied-tls can be used in complex MITM (man in the middle) configurations, where the proxy header is sent encrypted within the same TLS session as the proxied messages.
When you enable Proxy Protocol support for your network() source, you can use the following configuration example with your AxoSyslog application.
Configuration
The following code sample illustrates how you can use the Proxy Protocol in your AxoSyslog configuration (using the transport() parameter set to proxied-tls-passthrough).
With this configuration, the Proxy Protocol method will perform injecting the information of the original connection into the forwarded TCP session, based on the working mechanism described in The working mechanism behind the Proxy Protocol.
The following example illustrates how the parsed macros will appear in the output.
Example: Output for the PROXY TCP4 192.168.1.1 10.10.0.1 1111 2222 input header
With the PROXY TCP4 192.168.1.1 10.10.0.1 1111 2222 input header, the output looks like this:
Note that the macros that AxoSyslog adds to the message appear in the output.
7.15 - nodejs: Receive JSON messages from nodejs applications
Using the nodejs() driver, AxoSyslog can receive application logs directly from nodejs applications that use the widespread Winston logging API. The AxoSyslog application automatically adds the .nodejs.winston. prefix to the name of the fields the extracted from the message.
To use the nodejs() driver, the scl.conf file must be included in your AxoSyslog configuration:
@include "scl.conf"
The nodejs() driver is actually a reusable configuration snippet configured to receive log messages using the network() driver, and process its JSON contents. For details on using or writing such configuration snippets, see Reusing configuration blocks. You can find the source of the nodejs configuration snippet on GitHub.
Example: Using the nodejs() driver
The following example uses the default settings of the driver, listening for messages on port 9003 of every IP address of the AxoSyslog host.
@include "scl.conf"source apps { nodejs();};
The following example listens only on IP address 192.168.1.1, port 9999.
Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.
Note
The AxoSyslog application must be able to start and restart the external program, and have the necessary permissions to do so. For example, if your host is running AppArmor or SELinux, you might have to modify your AppArmor or SELinux configuration to enable AxoSyslog to execute external applications.
Using hook-commands() when AxoSyslog starts or stops
To execute an external program when AxoSyslog starts or stops, use the following options:
startup()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog starts.
shutdown()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog stops.
Using the hook-commands() when AxoSyslog reloads
To execute an external program when the AxoSyslog configuration is initiated or torn down, for example, on startup/shutdown or during a AxoSyslog reload, use the following options:
setup()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is initiated, for example, on startup or during a AxoSyslog reload.
teardown()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is stopped or torn down, for example, on shutdown or during a AxoSyslog reload.
Example: Using the hook-commands() with a network source
In the following example, the hook-commands() is used with the network() driver and it opens an iptables port automatically as AxoSyslog is started/stopped.
The assumption in this example is that the LOGCHAIN chain is part of a larger ruleset that routes traffic to it. Whenever the AxoSyslog created rule is there, packets can flow, otherwise the port is closed.
Description: The IP address to bind to. By default, AxoSyslog listens on every available interface. Note that this is not the address where messages are accepted from.
If you specify a multicast bind address and use the udp transport, AxoSyslog automatically joins the necessary multicast group. TCP does not support multicasting.
port() or localport()
Type:
number
Default:
9003
Description: The port number to bind to.
7.16 - osquery: Collect and parse osquery result logs
The osquery application allows you to ask questions about your machine using an SQL-like language. For example, you can query running processes, logged in users, installed packages and syslog messages as well. You can make queries on demand, and also schedule them to run regularly.
The osquery() source of AxoSyslog allows you read the results of periodical osquery queries (from the /var/log/osquery/osqueryd.results.log file) and automatically parse the messages (if you want to use AxoSyslog to send log messages to osquery, read this blogpost). For example, you can:
Create filters from the fields of the messages.
Limit which fields to store, or create additional fields (combine multiple fields into one field, and so on).
Send the messages to a central location, for example, to Elasticsearch, directly from AxoSyslog.
The AxoSyslog application automatically adds the .osquery. prefix to the name of the fields the extracted from the message.
The osquery() source is available in AxoSyslog version 3.10 and later.
Prerequisites:
To use the osquery() driver, the scl.conf file must be included in your AxoSyslog configuration:
@include "scl.conf"
AxoSyslog must be compiled with JSON-support enabled.
The osquery() driver is actually a reusable configuration snippet configured to read the osquery log file using the file() driver, and process its JSON contents. For details on using or writing such configuration snippets, see Reusing configuration blocks. You can find the source of this configuration snippet on GitHub.
Example: Using the osquery() driver with the default settings
The following AxoSyslog configuration sample uses the default settings of the driver, reading osquery result logs from the /var/log/osquery/osqueryd.results.log file, and writes the log messages generated from the traps into a file.
Example: Using the osquery() driver with custom configuration
The following AxoSyslog configuration sample reads osquery result logs from the /tmp/osquery_input.log file, and writes the log messages generated from the traps into a file. Using the format-json template, the outgoing message will be a well-formed JSON message.
Description: The log file of osquery that stores the results of periodic queries. The AxoSyslog application reads the messages from this file.
hook-commands()
Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.
Note
The AxoSyslog application must be able to start and restart the external program, and have the necessary permissions to do so. For example, if your host is running AppArmor or SELinux, you might have to modify your AppArmor or SELinux configuration to enable AxoSyslog to execute external applications.
Using hook-commands() when AxoSyslog starts or stops
To execute an external program when AxoSyslog starts or stops, use the following options:
startup()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog starts.
shutdown()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog stops.
Using the hook-commands() when AxoSyslog reloads
To execute an external program when the AxoSyslog configuration is initiated or torn down, for example, on startup/shutdown or during a AxoSyslog reload, use the following options:
setup()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is initiated, for example, on startup or during a AxoSyslog reload.
teardown()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is stopped or torn down, for example, on shutdown or during a AxoSyslog reload.
Example: Using the hook-commands() with a network source
In the following example, the hook-commands() is used with the network() driver and it opens an iptables port automatically as AxoSyslog is started/stopped.
The assumption in this example is that the LOGCHAIN chain is part of a larger ruleset that routes traffic to it. Whenever the AxoSyslog created rule is there, packets can flow, otherwise the port is closed.
Description: Insert a prefix before the name part of the parsed name-value pairs to help further processing. For example:
To insert the my-parsed-data. prefix, use the prefix(my-parsed-data.) option.
To refer to a particular data that has a prefix, use the prefix in the name of the macro, for example, ${my-parsed-data.name}.
If you forward the parsed messages using the IETF-syslog protocol, you can insert all the parsed data into the SDATA part of the message using the prefix(.SDATA.my-parsed-data.) option.
Names starting with a dot (for example, .example) are reserved for use by AxoSyslog. If you use such a macro name as the name of a parsed value, it will attempt to replace the original value of the macro (note that only soft macros can be overwritten, see Hard versus soft macros for details). To avoid such problems, use a prefix when naming the parsed values, for example, prefix(my-parsed-data.)
Default value:
.osquery. option.
7.17 - Receive logs, metrics, and traces from OpenTelemetry
The following example receives OpenTelemetry data and forwards it to an OpenTelemetry receiver. Note that by default, AxoSyslog doesn’t parse the fields of the incoming messages into name-value pairs, but are only available for forwarding using the opentelemetry() destination. To parse the fields into name-value pairs, use the opentelemetry() parser.
Application Layer Transport Security (ALTS) is a simple to use authentication, only available within Google’s infrastructure. It accepts the target-service-account() option, where you can list service accounts to match against when authenticating the server.
tls(peer-verify()) is not available for the opentelemetry() and loki() destination.
The gRPC-based drivers (opentelemetry() and loki()) have a different tls() block implementation from the network() or http() drivers. Most features are the same.
channel-args()
Type:
arrow list
Default:
-
Description: The channel-args() option is available in gRPC-based drivers. It accepts name-value pairs and sets channel arguments defined in the GRPC Core library documentation. For example:
Description: Configures the maximal number of in-flight gRPC requests per worker. Setting this value in the range of 10s or 100s is recommended when there are a high number of clients sending simultaneously. Ideally, workers() * concurrent-requests() should be greater than or equal to the number of clients, but this can increase the memory usage.
keep-hostname()
The opentelemetry() source ignores this option and uses the address of the OTLP peer as the HOST.
log-fetch-limit()
Type:
number
Default:
100
Description: The maximum number of messages fetched from a source during a single poll loop. The destination queues might fill up before flow-control could stop reading if log-fetch-limit() is too high.
port()
The port number to receive incoming connections. Default value: 4317
workers()
Type:
integer
Default:
1
Description: Specifies the number of worker threads (at least 1) that AxoSyslog uses to process the messages of the source. Increasing the number of worker threads can drastically improve the performance of the source.
7.18 - pacct: Collect process accounting logs on Linux
Starting with version 3.2, AxoSyslog can collect process accounting logs on Linux systems.Process accounting is the method of recording and summarizing commands executed on Linux, for example, the commands being run, the user who executed the command, CPU time used by the process, exit code, and so on. When process accounting (also called pacct) is enabled on a system, the kernel writes accounting records to the /var/log/account/pacct file (might vary between different Linux distributions).
To use the pacct() driver, the following conditions must be met:
The AxoSyslog application must be compiled with the --enable-pacct option. Execute the syslog-ng -V command to check if your binary supports process accounting.
The pacctformat plugin must be loaded. By default, AxoSyslog automatically loads the available modules.
The scl.conf file must be included in your AxoSyslog configuration:
@include "scl.conf"
Process accounting must be running on the host. You can enable it with the accton command.
The pacct() driver parses the fields of the accounting logs and transforms them into name-value pairs. The fields are defined in the manual page of the accounting log file (man acct), AxoSyslog prepends every field with the .pacct. prefix. For example, the ac_uid field that contains the id of the user who started the process will be available under the $.pacct.ac_uid name. These can be used as macros in templates, in filters to select specific messages, and so on.
To use the pacct() driver, use the following syntax.
The pacct() driver is actually a reusable configuration snippet configured to handle Linux accounting logs. For details on using or writing such configuration snippets, see Reusing configuration blocks. You can find the source of the pacct configuration snippet on GitHub.
7.18.1 - pacct() options
The pacct() driver has the following options:
file()
Type:
filename with path
Default:
/var/log/account/pacct
Description: The file where the process accounting logs are stored — AxoSyslog reads accounting messages from this file.
follow-freq()
Type:
number
Default:
1
Description: Indicates that the source should be checked periodically. This is useful for files which always indicate readability, even though no new lines were appended. If this value is higher than zero, AxoSyslog will not attempt to use poll() on the file, but checks whether the file changed every time the follow-freq() interval (in seconds) has elapsed. Floating-point numbers (for example, 1.5) can be used as well.
hook-commands()
Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.
Note
The AxoSyslog application must be able to start and restart the external program, and have the necessary permissions to do so. For example, if your host is running AppArmor or SELinux, you might have to modify your AppArmor or SELinux configuration to enable AxoSyslog to execute external applications.
Using hook-commands() when AxoSyslog starts or stops
To execute an external program when AxoSyslog starts or stops, use the following options:
startup()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog starts.
shutdown()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog stops.
Using the hook-commands() when AxoSyslog reloads
To execute an external program when the AxoSyslog configuration is initiated or torn down, for example, on startup/shutdown or during a AxoSyslog reload, use the following options:
setup()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is initiated, for example, on startup or during a AxoSyslog reload.
teardown()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is stopped or torn down, for example, on shutdown or during a AxoSyslog reload.
Example: Using the hook-commands() with a network source
In the following example, the hook-commands() is used with the network() driver and it opens an iptables port automatically as AxoSyslog is started/stopped.
The assumption in this example is that the LOGCHAIN chain is part of a larger ruleset that routes traffic to it. Whenever the AxoSyslog created rule is there, packets can flow, otherwise the port is closed.
Starting with version 4.6.0, AxoSyslog can collect logs of the Pi-hole FTL (Faster Than Light) application.
source s_pihole_ftl { pihole-ftl();};
By default, the source reads the /var/log/pihole/FTL.log file. If the root directory of your Pi-hole installation is different, specify the directory where the FTL.log file is with the dir() option. You can find the root log directory by selecting Tools > Pi-hole diagnosis in the Pi-hole application. Otherwise, the pihole-ftl() source has the same parameters as the file() source.
The pihole-ftl() driver is actually a reusable configuration snippet. For details on using or writing such configuration snippets, see Reusing configuration blocks. You can find the source of this configuration snippet on GitHub.
7.20 - pipe: Collect messages from named pipes
The pipe driver opens a named pipe with the specified name and listens for messages. It is used as the native message delivery protocol on HP-UX.
The pipe driver has a single required parameter, specifying the filename of the pipe to open. For the list of available optional parameters, see pipe() source options.
Declaration:
pipe(filename);
Note
As of version 3.0.2, pipes are created automatically. In earlier versions, you had to create the pipe using the mkfifo(1) command.
Pipe is very similar to the file() driver, but there are a few differences, for example, pipe() opens its argument in read-write mode, therefore it is not recommended to be used on special files like /proc/kmsg.
Warning
It is not recommended to use pipe() on anything else than real pipes.
By default, AxoSyslog uses the flags(no-hostname) option for pipes, meaning that AxoSyslog assumes that the log messages received from the pipe do not contain the hostname field. If your messages do contain the hostname field, use flags(expect-hostname).
Description: Specifies the log parsing options of the source.
assume-utf8: The assume-utf8 flag assumes that the incoming messages are UTF-8 encoded, but does not verify the encoding. If you explicitly want to validate the UTF-8 encoding of the incoming message, use the validate-utf8 flag.
empty-lines: Use the empty-lines flag to keep the empty lines of the messages. By default, AxoSyslog removes empty lines automatically.
expect-hostname: If the expect-hostname flag is enabled, AxoSyslog will assume that the log message contains a hostname and parse the message accordingly. This is the default behavior for TCP sources. Note that pipe sources use the no-hostname flag by default.
guess-timezone: Attempt to guess the timezone of the message if this information is not available in the message. Works when the incoming message stream is close to real time, and the timezone information is missing from the timestamp.
kernel: The kernel flag makes the source default to the LOG_KERN | LOG_NOTICE priority if not specified otherwise.
no-header: The no-header flag triggers AxoSyslog to parse only the PRI field of incoming messages, and put the rest of the message contents into $MSG.
Its functionality is similar to that of the no-parse flag, except the no-header flag does not skip the PRI field.
Note
Essentially, the no-header flag signals AxoSyslog that the syslog header is not present (or does not adhere to the conventions / RFCs), so the entire message (except from the PRI field) is put into $MSG.
Example: using the no-header flag with the syslog-parser() parser
The following example illustrates using the no-header flag with the syslog-parser() parser:
no-hostname: Enable the no-hostname flag if the log message does not include the hostname of the sender host. That way AxoSyslog assumes that the first part of the message header is ${PROGRAM} instead of ${HOST}. For example:
no-multi-line: The no-multi-line flag disables line-breaking in the messages: the entire message is converted to a single line. Note that this happens only if the underlying transport method actually supports multi-line messages. Currently the file() and pipe() drivers support multi-line messages.
no-parse: By default, AxoSyslog parses incoming messages as syslog messages. The no-parse flag completely disables syslog message parsing and processes the complete line as the message part of a syslog message. The AxoSyslog application will generate a new syslog header (timestamp, host, and so on) automatically and put the entire incoming message into the MESSAGE part of the syslog message (available using the ${MESSAGE} macro). This flag is useful for parsing messages not complying to the syslog format.
If you are using the flags(no-parse) option, then syslog message parsing is completely disabled, and the entire incoming message is treated as the ${MESSAGE} part of a syslog message. In this case, AxoSyslog generates a new syslog header (timestamp, host, and so on) automatically. Note that even though flags(no-parse) disables message parsing, some flags can still be used, for example, the no-multi-line flag.
dont-store-legacy-msghdr: By default, AxoSyslog stores the original incoming header of the log message. This is useful if the original format of a non-syslog-compliant message must be retained (AxoSyslog automatically corrects minor header errors, for example, adds a whitespace before msg in the following message: Jan 22 10:06:11 host program:msg). If you do not want to store the original header of the message, enable the dont-store-legacy-msghdr flag.
sanitize-utf8: When using the sanitize-utf8 flag, AxoSyslog converts non-UTF-8 input to an escaped form, which is valid UTF-8.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
store-raw-message: Save the original message as received from the client in the ${RAWMSG} macro. You can forward this raw message in its original form to another AxoSyslog node using the syslog-ng() destination, or to a SIEM system, ensuring that the SIEM can process it. Available only in 3.16 and later.
syslog-protocol: The syslog-protocol flag specifies that incoming messages are expected to be formatted according to the new IETF syslog protocol standard (RFC5424), but without the frame header. Note that this flag is not needed for the syslog driver, which handles only messages that have a frame header.
validate-utf8: The validate-utf8 flag enables encoding-verification for messages.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
For RFC5424-formatted messages, if the BOM character is missing, but the message is otherwise UTF-8 compliant, AxoSyslog automatically adds the BOM character to the message.
The byte order mark (BOM) is a Unicode character used to signal the byte-order of the message text.
follow-freq()
Type:
number
Default:
1
Description: Indicates that the source should be checked periodically. This is useful for files which always indicate readability, even though no new lines were appended. If this value is higher than zero, AxoSyslog will not attempt to use poll() on the file, but checks whether the file changed every time the follow-freq() interval (in seconds) has elapsed. Floating-point numbers (for example, 1.5) can be used as well.
hook-commands()
Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.
Note
The AxoSyslog application must be able to start and restart the external program, and have the necessary permissions to do so. For example, if your host is running AppArmor or SELinux, you might have to modify your AppArmor or SELinux configuration to enable AxoSyslog to execute external applications.
Using hook-commands() when AxoSyslog starts or stops
To execute an external program when AxoSyslog starts or stops, use the following options:
startup()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog starts.
shutdown()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog stops.
Using the hook-commands() when AxoSyslog reloads
To execute an external program when the AxoSyslog configuration is initiated or torn down, for example, on startup/shutdown or during a AxoSyslog reload, use the following options:
setup()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is initiated, for example, on startup or during a AxoSyslog reload.
teardown()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is stopped or torn down, for example, on shutdown or during a AxoSyslog reload.
Example: Using the hook-commands() with a network source
In the following example, the hook-commands() is used with the network() driver and it opens an iptables port automatically as AxoSyslog is started/stopped.
The assumption in this example is that the LOGCHAIN chain is part of a larger ruleset that routes traffic to it. Whenever the AxoSyslog created rule is there, packets can flow, otherwise the port is closed.
Description: Specifies whether AxoSyslog should accept the timestamp received from the sending application or client. If disabled, the time of reception will be used instead. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.
Warning
To use the S_ macros, the keep-timestamp() option must be enabled (this is the default behavior of AxoSyslog).
log-fetch-limit()
Type:
number
Default:
100
Description: The maximum number of messages fetched from a source during a single poll loop. The destination queues might fill up before flow-control could stop reading if log-fetch-limit() is too high.
log-iw-size()
Type:
number
Default:
100
Description: The size of the initial window, this value is used during flow-control. Its value cannot be lower than 100, unless the dynamic-window-size() option is enabled. For details on flow-control, see Managing incoming and outgoing messages with flow-control.
log-msg-size()
Type:
number (bytes)
Default:
Use the global log-msg-size() option, which defaults to 65536 (64 KiB).
Description: Maximum length of an incoming message in bytes. This length includes the entire message (the data structure and individual fields). The maximal value that can be set is 268435456 bytes (256 MiB).
For messages using the IETF-syslog message format (RFC5424), the maximal size of the value of an SDATA field is 64 KiB.
Note
In most cases, log-msg-size() does not need to be set higher than 10 MiB.
Uses the value of the global option if not specified.
log-prefix() (DEPRECATED)
Type:
string
Default:
Description: A string added to the beginning of every log message. It can be used to add an arbitrary string to any log source, though it is most commonly used for adding kernel: to the kernel messages on Linux.
Note
This option is deprecated. Use program-override instead.
multi-line-garbage()
Type:
regular expression
Default:
empty string
Description: Use the multi-line-garbage() option when processing multi-line messages that contain unneeded parts between the messages. Specify a string or regular expression that matches the beginning of the unneeded message parts. If the multi-line-garbage() option is set, AxoSyslog ignores the lines between the line matching the multi-line-garbage() and the next line matching multi-line-prefix(). See also the multi-line-prefix() option.
When receiving multi-line messages from a source when the multi-line-garbage() option is set, but no matching line is received between two lines that match multi-line-prefix(), AxoSyslog will continue to process the incoming lines as a single message until a line matching multi-line-garbage() is received.
To use the multi-line-garbage() option, set the multi-line-mode() option to prefix-garbage.
Warning
If the multi-line-garbage() option is set, AxoSyslog discards lines between the line matching the multi-line-garbage() and the next line matching multi-line-prefix().
Description: Use the multi-line-mode() option when processing multi-line messages. The AxoSyslog application provides the following methods to process multi-line messages:
indented: The indented mode can process messages where each line that belongs to the previous line is indented by whitespace, and the message continues until the first non-indented line. For example, the Linux kernel (starting with version 3.5) uses this format for /dev/log, as well as several applications, like Apache Tomcat.
prefix-garbage: The prefix-garbage mode uses a string or regular expression (set in multi-line-prefix()) that matches the beginning of the log messages, ignores newline characters from the source until a line matches the regular expression again, and treats the lines between the matching lines as a single message. For details on using multi-line-mode(prefix-garbage), see the multi-line-prefix() and multi-line-garbage() options.
prefix-suffix: The prefix-suffix mode uses a string or regular expression (set in multi-line-prefix()) that matches the beginning of the log messages, ignores newline characters from the source until a line matches the regular expression set in multi-line-suffix(), and treats the lines between multi-line-prefix() and multi-line-suffix() as a single message. Any other lines between the end of the message and the beginning of a new message (that is, a line that matches the multi-line-prefix() expression) are discarded. For details on using multi-line-mode(prefix-suffix), see the multi-line-prefix() and multi-line-suffix() options.
The prefix-suffix mode is similar to the prefix-garbage mode, but it appends the garbage part to the message instead of discarding it.
smart: The smart mode recognizes multi-line data backtraces even if they span multiple lines in the input. The backtraces are converted to a single log message for easier analysis. Backtraces for the following programming languages are recognized : Python, Java, JavaScript, PHP, Go, Ruby, and Dart.
smart mode is available in AxoSyslog version 4.2 and newer.
The regular expressions to recognize these programming languages are specified in an external file called /usr/share/syslog-ng/smart-multi-line.fsm (installation path depends on configure arguments), in a format that is described in that file.
Note
To format multi-line messages to your individual needs, consider the following:
To make multi-line messages more readable when written to a file, use a template in the destination and instead of the ${MESSAGE} macro, use the following: $(indent-multi-line ${MESSAGE}). This expression inserts a tab after every newline character (except when a tab is already present), indenting every line of the message after the first. For example:
To actually convert the lines of multi-line messages to single line (by replacing the newline characters with whitespaces), use the flags(no-multi-line) option in the source.
multi-line-prefix()
Type:
regular expression starting with the ^ character
Default:
empty string
Description: Use the multi-line-prefix() option to process multi-line messages, that is, log messages that contain newline characters (for example, Tomcat logs). Specify a string or regular expression that matches the beginning of the log messages (always start with the ^ character). Use as simple regular expressions as possible, because complex regular expressions can severely reduce the rate of processing multi-line messages. If the multi-line-prefix() option is set, AxoSyslog ignores newline characters from the source until a line matches the regular expression again, and treats the lines between the matching lines as a single message. See also the multi-line-garbage() option.
Note
To format multi-line messages to your individual needs, consider the following:
To make multi-line messages more readable when written to a file, use a template in the destination and instead of the ${MESSAGE} macro, use the following: $(indent-multi-line ${MESSAGE}). This expression inserts a tab after every newline character (except when a tab is already present), indenting every line of the message after the first. For example:
To actually convert the lines of multi-line messages to single line (by replacing the newline characters with whitespaces), use the flags(no-multi-line) option in the source.
Example: Processing Tomcat logs
The log messages of the Apache Tomcat server are a typical example for multi-line log messages. The messages start with the date and time of the query in the YYYY.MM.DD HH:MM:SS format, as you can see in the following example.
2010.06.09. 12:07:39 org.apache.catalina.startup.Catalina start
SEVERE: Catalina.start:
LifecycleException: service.getName(): "Catalina"; Protocol handler start failed: java.net.BindException: Address already in use null:8080
at org.apache.catalina.connector.Connector.start(Connector.java:1138) at org.apache.catalina.core.StandardService.start(StandardService.java:531) at org.apache.catalina.core.StandardServer.start(StandardServer.java:710) at org.apache.catalina.startup.Catalina.start(Catalina.java:583) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:177) 2010.06.09. 12:07:39 org.apache.catalina.startup.Catalina start
INFO: Server startup in 1206 ms
2010.06.09. 12:45:08 org.apache.coyote.http11.Http11Protocol pause
INFO: Pausing Coyote HTTP/1.1 on http-8080
2010.06.09. 12:45:09 org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
To process these messages, specify a regular expression matching the timestamp of the messages in the multi-line-prefix() option. Such an expression is the following:
Note that flags(no-parse) is needed to prevent AxoSyslog trying to interpret the date in the message.
multi-line-suffix()
Type:
regular expression
Default:
empty string
Description: Use the multi-line-suffix() option when processing multi-line messages. Specify a string or regular expression that matches the end of the multi-line message.
To use the multi-line-suffix() option, set the multi-line-mode() option to prefix-suffix. See also the multi-line-prefix() option.
optional()
Type:
yes or no
Default:
Description: Instruct AxoSyslog to ignore the error if a specific source cannot be initialized. No other attempts to initialize the source will be made until the configuration is reloaded. This option currently applies to the pipe(), unix-dgram, and unix-stream drivers.
pad-size()
Type:
number
Default:
0
Description: Specifies input padding. Some operating systems (such as HP-UX) pad all messages to block boundary. This option can be used to specify the block size. The AxoSyslog application will pad reads from the associated device to the number of bytes set in pad-size(). Mostly used on HP-UX where /dev/log is a named pipe and every write is padded to 2048 bytes. If pad-size() was given and the incoming message does not fit into pad-size(), AxoSyslog will not read anymore from this pipe and displays the following error message:
Padding was set, and couldn't read enough bytes
program-override()
Type:
string
Default:
Description: Replaces the ${PROGRAM} part of the message with the parameter string. For example, to mark every message coming from the kernel, include the program-override("kernel") option in the source containing /proc/kmsg.
tags()
Type:
string
Default:
Description: Label the messages received from the source with custom tags. Tags must be unique, and enclosed between double quotes. When adding multiple tags, separate them with comma, for example, tags("dmz", "router"). This option is available only in version 3.1 and later.
time-zone()
Type:
name of the timezone, or the timezone offset
Default:
Description: The default timezone for messages read from the source. Applies only if no timezone is specified within the message itself.
The timezone can be specified by using the name, for example, time-zone("Europe/Budapest")), or as the timezone offset in +/-HH:MM format, for example, +01:00). On Linux and UNIX platforms, the valid timezone names are listed under the /usr/share/zoneinfo directory.
7.21 - program: Receive messages from external applications
The program driver starts an external application and reads messages from the standard output (stdout) of the application. It is mainly useful to receive log messages from daemons that accept incoming messages and convert them to log messages.
The program driver has a single required parameter, specifying the name of the application to start.
Description: Specifies the log parsing options of the source.
assume-utf8: The assume-utf8 flag assumes that the incoming messages are UTF-8 encoded, but does not verify the encoding. If you explicitly want to validate the UTF-8 encoding of the incoming message, use the validate-utf8 flag.
empty-lines: Use the empty-lines flag to keep the empty lines of the messages. By default, AxoSyslog removes empty lines automatically.
expect-hostname: If the expect-hostname flag is enabled, AxoSyslog will assume that the log message contains a hostname and parse the message accordingly. This is the default behavior for TCP sources. Note that pipe sources use the no-hostname flag by default.
guess-timezone: Attempt to guess the timezone of the message if this information is not available in the message. Works when the incoming message stream is close to real time, and the timezone information is missing from the timestamp.
kernel: The kernel flag makes the source default to the LOG_KERN | LOG_NOTICE priority if not specified otherwise.
no-header: The no-header flag triggers AxoSyslog to parse only the PRI field of incoming messages, and put the rest of the message contents into $MSG.
Its functionality is similar to that of the no-parse flag, except the no-header flag does not skip the PRI field.
Note
Essentially, the no-header flag signals AxoSyslog that the syslog header is not present (or does not adhere to the conventions / RFCs), so the entire message (except from the PRI field) is put into $MSG.
Example: using the no-header flag with the syslog-parser() parser
The following example illustrates using the no-header flag with the syslog-parser() parser:
no-hostname: Enable the no-hostname flag if the log message does not include the hostname of the sender host. That way AxoSyslog assumes that the first part of the message header is ${PROGRAM} instead of ${HOST}. For example:
no-multi-line: The no-multi-line flag disables line-breaking in the messages: the entire message is converted to a single line. Note that this happens only if the underlying transport method actually supports multi-line messages. Currently the file() and pipe() drivers support multi-line messages.
no-parse: By default, AxoSyslog parses incoming messages as syslog messages. The no-parse flag completely disables syslog message parsing and processes the complete line as the message part of a syslog message. The AxoSyslog application will generate a new syslog header (timestamp, host, and so on) automatically and put the entire incoming message into the MESSAGE part of the syslog message (available using the ${MESSAGE} macro). This flag is useful for parsing messages not complying to the syslog format.
If you are using the flags(no-parse) option, then syslog message parsing is completely disabled, and the entire incoming message is treated as the ${MESSAGE} part of a syslog message. In this case, AxoSyslog generates a new syslog header (timestamp, host, and so on) automatically. Note that even though flags(no-parse) disables message parsing, some flags can still be used, for example, the no-multi-line flag.
dont-store-legacy-msghdr: By default, AxoSyslog stores the original incoming header of the log message. This is useful if the original format of a non-syslog-compliant message must be retained (AxoSyslog automatically corrects minor header errors, for example, adds a whitespace before msg in the following message: Jan 22 10:06:11 host program:msg). If you do not want to store the original header of the message, enable the dont-store-legacy-msghdr flag.
sanitize-utf8: When using the sanitize-utf8 flag, AxoSyslog converts non-UTF-8 input to an escaped form, which is valid UTF-8.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
store-raw-message: Save the original message as received from the client in the ${RAWMSG} macro. You can forward this raw message in its original form to another AxoSyslog node using the syslog-ng() destination, or to a SIEM system, ensuring that the SIEM can process it. Available only in 3.16 and later.
syslog-protocol: The syslog-protocol flag specifies that incoming messages are expected to be formatted according to the new IETF syslog protocol standard (RFC5424), but without the frame header. Note that this flag is not needed for the syslog driver, which handles only messages that have a frame header.
validate-utf8: The validate-utf8 flag enables encoding-verification for messages.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
For RFC5424-formatted messages, if the BOM character is missing, but the message is otherwise UTF-8 compliant, AxoSyslog automatically adds the BOM character to the message.
The byte order mark (BOM) is a Unicode character used to signal the byte-order of the message text.
hook-commands()
Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.
Note
The AxoSyslog application must be able to start and restart the external program, and have the necessary permissions to do so. For example, if your host is running AppArmor or SELinux, you might have to modify your AppArmor or SELinux configuration to enable AxoSyslog to execute external applications.
Using hook-commands() when AxoSyslog starts or stops
To execute an external program when AxoSyslog starts or stops, use the following options:
startup()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog starts.
shutdown()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog stops.
Using the hook-commands() when AxoSyslog reloads
To execute an external program when the AxoSyslog configuration is initiated or torn down, for example, on startup/shutdown or during a AxoSyslog reload, use the following options:
setup()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is initiated, for example, on startup or during a AxoSyslog reload.
teardown()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is stopped or torn down, for example, on shutdown or during a AxoSyslog reload.
Example: Using the hook-commands() with a network source
In the following example, the hook-commands() is used with the network() driver and it opens an iptables port automatically as AxoSyslog is started/stopped.
The assumption in this example is that the LOGCHAIN chain is part of a larger ruleset that routes traffic to it. Whenever the AxoSyslog created rule is there, packets can flow, otherwise the port is closed.
Description: Specifies whether AxoSyslog should accept the timestamp received from the sending application or client. If disabled, the time of reception will be used instead. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.
Warning
To use the S_ macros, the keep-timestamp() option must be enabled (this is the default behavior of AxoSyslog).
log-fetch-limit()
Type:
number
Default:
100
Description: The maximum number of messages fetched from a source during a single poll loop. The destination queues might fill up before flow-control could stop reading if log-fetch-limit() is too high.
inherit-environment()
Type:
`yes
Default:
yes
Description: By default, when program() starts an external application or script, it inherits the entire environment of the parent process (that is, AxoSyslog). Use inherit-environment(no) to prevent this.
log-iw-size()
Type:
number
Default:
100
Description: The size of the initial window, this value is used during flow-control. Its value cannot be lower than 100, unless the dynamic-window-size() option is enabled. For details on flow-control, see Managing incoming and outgoing messages with flow-control.
log-msg-size()
Type:
number (bytes)
Default:
Use the global log-msg-size() option, which defaults to 65536 (64 KiB).
Description: Maximum length of an incoming message in bytes. This length includes the entire message (the data structure and individual fields). The maximal value that can be set is 268435456 bytes (256 MiB).
For messages using the IETF-syslog message format (RFC5424), the maximal size of the value of an SDATA field is 64 KiB.
Note
In most cases, log-msg-size() does not need to be set higher than 10 MiB.
Uses the value of the global option if not specified.
log-prefix() (DEPRECATED)
Type:
string
Default:
Description: A string added to the beginning of every log message. It can be used to add an arbitrary string to any log source, though it is most commonly used for adding kernel: to the kernel messages on Linux.
Note
This option is deprecated. Use program-override instead.
optional()
Type:
yes or no
Default:
Description: Instruct AxoSyslog to ignore the error if a specific source cannot be initialized. No other attempts to initialize the source will be made until the configuration is reloaded. This option currently applies to the pipe(), unix-dgram, and unix-stream drivers.
pad-size()
Type:
number
Default:
0
Description: Specifies input padding. Some operating systems (such as HP-UX) pad all messages to block boundary. This option can be used to specify the block size. The AxoSyslog application will pad reads from the associated device to the number of bytes set in pad-size(). Mostly used on HP-UX where /dev/log is a named pipe and every write is padded to 2048 bytes. If pad-size() was given and the incoming message does not fit into pad-size(), AxoSyslog will not read anymore from this pipe and displays the following error message:
Padding was set, and couldn't read enough bytes
program()
Type:
filename with path
Default:
Description: The name of the application to start and read messages from.
program-override()
Type:
string
Default:
Description: Replaces the ${PROGRAM} part of the message with the parameter string. For example, to mark every message coming from the kernel, include the program-override("kernel") option in the source containing /proc/kmsg.
tags()
Type:
string
Default:
Description: Label the messages received from the source with custom tags. Tags must be unique, and enclosed between double quotes. When adding multiple tags, separate them with comma, for example, tags("dmz", "router"). This option is available only in version 3.1 and later.
time-zone()
Type:
name of the timezone, or the timezone offset
Default:
Description: The default timezone for messages read from the source. Applies only if no timezone is specified within the message itself.
The timezone can be specified by using the name, for example, time-zone("Europe/Budapest")), or as the timezone offset in +/-HH:MM format, for example, +01:00). On Linux and UNIX platforms, the valid timezone names are listed under the /usr/share/zoneinfo directory.
The Python source allows you to write your own source in Python. You can import external Python modules to receive or fetch the messages. Since many services have a Python library, the Python source makes integrating AxoSyslog very easy and quick.
You can write two different type of sources in Python:
Server-style sources that receives messages. Write server-style sources if you want to use an event-loop based, nonblocking server framework in Python, or if you want to implement a custom loop.
Fetcher-style sources that actively fetch messages. In general, write fetcher-style sources (for example, when using simple blocking APIs), unless you explicitly need a server-style source.
The following points apply to using Python blocks in AxoSyslog in general:
Python parsers and template functions are available in AxoSyslog version 3.10 and later.
Python destinations and sources are available in AxoSyslog version 3.18 and later.
Supported Python versions: 2.7 and 3.4+ (if you are using pre-built binaries, check the dependencies of the package to find out which Python version it was compiled with).
The Python block must be a top-level block in the AxoSyslog configuration file.
If you store the Python code in a separate Python file and only include it in the AxoSyslog configuration file, make sure that the PYTHONPATH environment variable includes the path to the Python file, and export the PYTHON_PATH environment variable. For example, if you start AxoSyslog manually from a terminal and you store your Python files in the /opt/syslog-ng/etc directory, use the following command: export PYTHONPATH=/opt/syslog-ng/etc.
In production, when AxoSyslog starts on boot, you must configure your startup script to include the Python path. The exact method depends on your operating system. For recent Red Hat Enterprise Linux, Fedora, and CentOS distributions that use systemd, the systemctl command sources the /etc/sysconfig/syslog-ng file before starting AxoSyslog. (On openSUSE and SLES, /etc/sysconfig/syslog file.) Append the following line to the end of this file: PYTHONPATH="<path-to-your-python-file>", for example, PYTHONPATH="/opt/syslog-ng/etc".
The Python object is initiated every time when AxoSyslog is started or reloaded.
Warning
If you reload AxoSyslog, existing Python objects are destroyed, therefore the context and state information of Python blocks is lost. Log rotation and updating the configuration of AxoSyslog typically involves a reload.
The Python block can contain multiple Python functions.
Using Python code in AxoSyslog can significantly decrease the performance of AxoSyslog, especially if the Python code is slow. In general, the features of AxoSyslog are implemented in C, and are faster than implementations of the same or similar features in Python.
Validate and lint the Python code before using it. The AxoSyslog application does not do any of this.
Python error messages are available in the internal() source of AxoSyslog.
You can access the name-value pairs of AxoSyslog directly through a message object or a dictionary.
To help debugging and troubleshooting your Python code, you can send log messages to the internal() source of AxoSyslog. For details, see Logging from your Python code.
Note
Starting with 3.26, AxoSyslog assigns a persist name to Python sources and destinations. The persist name is generated from the class name. If you want to use the same Python class multiple times in your AxoSyslog configuration, add a unique persist-name() to each source or destination, otherwise AxoSyslog will not start. For example:
Alternatively, you can include the following line in the Python package: @staticmethod generate_persist_name. For example:
from syslogng import LogSource
class PyNetworSource(LogSource):
@staticmethod
def generate_persist_name(options):
return options["port"] def run(self):
pass
def request_exit(self):
pass
Declaration:
Python sources consist of two parts. The first is a AxoSyslog source object that you define in your AxoSyslog configuration and use in the log path. This object references a Python class, which is the second part of the Python source. The Python class receives or fetches the log messages, and can do virtually anything that you can code in Python. You can either embed the Python class into your AxoSyslog configuration file, or store it in an external Python file.
source <name_of_the_python_source>{ python( class("<name_of_the_python_class_executed_by_the_source>") options("option1""value1",
"option2""value2"));}; python { from syslogng import LogSource
from syslogng import LogMessage
class <name_of_the_python_class_executed_by_the_source>(LogSource):
def init(self, options): # optional print("init") print(options) self.exit = False
return True
def deinit(self): # optional print("deinit") def run(self): # mandatory print("run")while not self.exit:
# Must create a messagemsg= LogMessage("this is a log message") self.post_message(msg) def request_exit(self): # mandatory print("exit") self.exit = True
};
Methods of the python() source
Server-style Python sources must be inherited from the syslogng.LogSource class, and must implement at least the run and request_exit methods. Multiple inheritance is allowed, but only for pure Python super classes.
You can implement your own event loop, or integrate the event loop of an external framework or library, for example, KafkaConsumer, Flask, Twisted engine, and so on.
To post messages, call LogSource::post_message() method in the run method.
The AxoSyslog application initializes Python objects every time when it is started or reloaded. The init method is executed as part of the initialization. You can perform any initialization steps that are necessary for your source to work.
Warning
If you reload AxoSyslog, existing Python objects are destroyed, therefore the context and state information of Python blocks is lost. Log rotation and updating the configuration of AxoSyslog typically involves a reload.
When this method returns with False, AxoSyslog does not start. It can be used to check options and return False when they prevent the successful start of the source.
options: This optional argument contains the contents of the options() parameter of the AxoSyslog configuration object as a Python dictionary.
run(self) method (mandatory)
Use the run method to implement an event loop, or start a server framework or library. Create LogMessage instances in this method, and pass them to the log paths by calling LogSource::post_message().
Currently, run stops permanently if an unhandled exception happens.
The AxoSyslog application calls this method when AxoSyslog is shut down or restarted. The request_exit method must shut down the event loop or framework, so the run method can return gracefully. If you use blocking operations within the run() method, use request_exit() to interrupt those operations and set an exit flag, otherwise AxoSyslog is not able to stop. Note that AxoSyslog calls the request_exit method from a thread different from the source thread.
close_batch(self)
Closes the current source-side batch. Source-side batching helps AxoSyslog to effectively process a larger chunk of messages, instead of processing messages each message. For example, when feeding a destination queue and instead of taking a lock on the queue for every message (causing contention), we only take it once per batch.
The native drivers built into AxoSyslog typically close batches once every mainloop iteration, allowing a single iteration to process multiple messages. For instance, when receiving multiple messages in a single TCP datagram, all of those messages
can be processed as a part of the same batch.
In Python-based log sources, a batch will automatically be closed after every message posted via post_message(), except if self.auto_close_batches is set to False during initialization. In case self.auto_close_batches is set to False, the driver has to call close_batch() explicitly, preferably at a natural boundary between incoming batches of messages. A good example is when we retrieve several messages via the same HTTP REST call, then the right time to close the batch would be after the last message in the response is posted.
The deinit(self) method (optional)
This method is executed when AxoSyslog is stopped or reloaded. This method does not return a value.
Warning
If you reload AxoSyslog, existing Python objects are destroyed, therefore the context and state information of Python blocks is lost. Log rotation and updating the configuration of AxoSyslog typically involves a reload.
set_transport_name(self, name)
Set the transport name used to retrieve messages This function can be called to customize the ${TRANSPORT} name-value pair.
7.22.1 - Python LogMessage API
The LogMessage API allows you to create LogMessage objects in Python sources, parse syslog messages, and set the various fields of the log message.
LogMessage() method: Create log message objects
You can use the LogMessage() method to create a structured log message instance. For example:
from syslogng import LogMessage
msg= LogMessage()# Initialize an empty message with default values (recvd timestamp, rcptid, hostid, ...)msg= LogMessage("string or bytes-like object")# Initialize a message and set its ${MESSAGE} field to the specified argument
You can also explicitly set the different values of the log message. For example:
msg["MESSAGE"]="message" msg["HOST"]="hostname"
You can set certain special field (timestamp, priority) by using specific methods.
Note the following points when creating a log message:
When setting the hostname, AxoSyslog takes the following hostname-related options of the configuration into account: chain-hostnames(), keep-hostname(), use-dns(), and use-fqdn().
Python sources ignore the log-msg-size() option.
The AxoSyslog application accepts only one message from every LogSource::post_message() or fetch() call, batching is currently not supported. If your Python code accepts batches of messages, you must pass them to AxoSyslog one-by-one. Similarly, if you need to split messages in the source, you must do so in your Python code, and pass the messages separately.
Do not reuse or store LogMessage objects after posting (calling post_message()) or returning the message from fetch().
parse() method: Parse syslog messages
The parse() method allows you to parse incoming messages as syslog messages. By default, the parse() method attempts to parse the message as an IETF-syslog (RFC5424) log message. If that fails, it parses the log message as a BSD-syslog (RFC3164) log message. Note that AxoSyslog takes the parsing-related options of the configuration into account: flags(), keep-hostname(), recv-time-zone().
If keep-hostname() is set to no, AxoSyslog ignores the hostname set in the message, and uses the IP address of the AxoSyslog host as the hostname (to use the hostname instead of the IP address, set the use-dns() or use-fqdn() options in the Python source).
msg_ietf= LogMessage.parse('<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] An application event log entry', self.parse_options)msg_bsd= LogMessage.parse('<34>Oct 11 22:14:15 mymachine su: \'su root\' failed for lonvick on /dev/pts/8', self.parse_options)
set_pri() method
You can set the priority of the message with the set_pri() method.
msg.set_pri(165)
set_timestamp() method
You can use the set_timestamp() method to set the date and time of the log message.
timestamp= datetime.fromisoformat("2018-09-11T14:49:02.100+02:00") msg.set_timestamp(timestamp)# datetime object, includes timezone information
In Python 2, timezone information cannot be attached to the datetime instance without using an external library. The AxoSyslog represents naive datetime objects in UTC.
In Python 3, naive and timezone-aware datetime objects are both supported.
7.22.2 - python() and python-fetcher() source options
The python() and python-fetcher() drivers have the following options.
class()
Type:
string
Default:
N/A
Description: The name of the Python class that implements the source, for example:
python( class("MyPythonSource"));
If you want to store the Python code in an external Python file, the class() option must include the name of the Python file containing the class, without the path and the .py extension, for example:
Description: If the fetch method of a python-fetcher() source returns with the LogFetcher.FETCH_NO_DATA constant, the source waits fetch-no-data-delay() seconds before calling the fetch method again. If you want to call the fetch method sooner, set the fetch-no-data-delay() option to the number of seconds to wait before calling the fetch method.
Description: Specifies the log parsing options of the source.
assume-utf8: The assume-utf8 flag assumes that the incoming messages are UTF-8 encoded, but does not verify the encoding. If you explicitly want to validate the UTF-8 encoding of the incoming message, use the validate-utf8 flag.
empty-lines: Use the empty-lines flag to keep the empty lines of the messages. By default, AxoSyslog removes empty lines automatically.
expect-hostname: If the expect-hostname flag is enabled, AxoSyslog will assume that the log message contains a hostname and parse the message accordingly. This is the default behavior for TCP sources. Note that pipe sources use the no-hostname flag by default.
guess-timezone: Attempt to guess the timezone of the message if this information is not available in the message. Works when the incoming message stream is close to real time, and the timezone information is missing from the timestamp.
kernel: The kernel flag makes the source default to the LOG_KERN | LOG_NOTICE priority if not specified otherwise.
no-header: The no-header flag triggers AxoSyslog to parse only the PRI field of incoming messages, and put the rest of the message contents into $MSG.
Its functionality is similar to that of the no-parse flag, except the no-header flag does not skip the PRI field.
Note
Essentially, the no-header flag signals AxoSyslog that the syslog header is not present (or does not adhere to the conventions / RFCs), so the entire message (except from the PRI field) is put into $MSG.
Example: using the no-header flag with the syslog-parser() parser
The following example illustrates using the no-header flag with the syslog-parser() parser:
no-hostname: Enable the no-hostname flag if the log message does not include the hostname of the sender host. That way AxoSyslog assumes that the first part of the message header is ${PROGRAM} instead of ${HOST}. For example:
no-multi-line: The no-multi-line flag disables line-breaking in the messages: the entire message is converted to a single line. Note that this happens only if the underlying transport method actually supports multi-line messages. Currently the file() and pipe() drivers support multi-line messages.
no-parse: By default, AxoSyslog parses incoming messages as syslog messages. The no-parse flag completely disables syslog message parsing and processes the complete line as the message part of a syslog message. The AxoSyslog application will generate a new syslog header (timestamp, host, and so on) automatically and put the entire incoming message into the MESSAGE part of the syslog message (available using the ${MESSAGE} macro). This flag is useful for parsing messages not complying to the syslog format.
If you are using the flags(no-parse) option, then syslog message parsing is completely disabled, and the entire incoming message is treated as the ${MESSAGE} part of a syslog message. In this case, AxoSyslog generates a new syslog header (timestamp, host, and so on) automatically. Note that even though flags(no-parse) disables message parsing, some flags can still be used, for example, the no-multi-line flag.
dont-store-legacy-msghdr: By default, AxoSyslog stores the original incoming header of the log message. This is useful if the original format of a non-syslog-compliant message must be retained (AxoSyslog automatically corrects minor header errors, for example, adds a whitespace before msg in the following message: Jan 22 10:06:11 host program:msg). If you do not want to store the original header of the message, enable the dont-store-legacy-msghdr flag.
sanitize-utf8: When using the sanitize-utf8 flag, AxoSyslog converts non-UTF-8 input to an escaped form, which is valid UTF-8.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
store-raw-message: Save the original message as received from the client in the ${RAWMSG} macro. You can forward this raw message in its original form to another AxoSyslog node using the syslog-ng() destination, or to a SIEM system, ensuring that the SIEM can process it. Available only in 3.16 and later.
syslog-protocol: The syslog-protocol flag specifies that incoming messages are expected to be formatted according to the new IETF syslog protocol standard (RFC5424), but without the frame header. Note that this flag is not needed for the syslog driver, which handles only messages that have a frame header.
validate-utf8: The validate-utf8 flag enables encoding-verification for messages.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
For RFC5424-formatted messages, if the BOM character is missing, but the message is otherwise UTF-8 compliant, AxoSyslog automatically adds the BOM character to the message.
The byte order mark (BOM) is a Unicode character used to signal the byte-order of the message text.
For the python() and python-fetcher() sources you can also set the check-hostname flag, which is equivalent with the check-hostname() global option, but only applies to this source.
The flags and the hostname-related options (for example, use-dns) set in the configuration file influence the behavior of the LogMessage.parse() method of the Python source. They have no effect if you set the message or the hostname directly, without using LogMessage.parse().
keep-hostname()
Type:
yes or no
Default:
no
Description: Enable or disable hostname rewriting.
If enabled (keep-hostname(yes)), AxoSyslog assumes that the incoming log message was sent by the host specified in the HOST field of the message.
If disabled (keep-hostname(no)), AxoSyslog rewrites the HOST field of the message, either to the IP address (if the use-dns() parameter is set to no), or to the hostname (if the use-dns() parameter is set to yes and the IP address can be resolved to a hostname) of the host sending the message to AxoSyslog. For details on using name resolution in AxoSyslog, see Using name resolution in syslog-ng.
Note
If the log message does not contain a hostname in its HOST field, AxoSyslog automatically adds a hostname to the message.
For messages received from the network, this hostname is the address of the host that sent the message (this means the address of the last hop if the message was transferred via a relay).
For messages received from the local host, AxoSyslog adds the name of the host.
This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.
Note
When relaying messages, enable this option on the AxoSyslog server and also on every relay, otherwise AxoSyslog will treat incoming messages as if they were sent by the last relay.
log-iw-size()
Type:
number
Default:
100
Description: The size of the initial window, this value is used during flow-control. Its value cannot be lower than 100, unless the dynamic-window-size() option is enabled. For details on flow-control, see Managing incoming and outgoing messages with flow-control.
loaders()
Type:
list of python modules
Default:
empty list
Description: The AxoSyslog application imports Python modules specified in this option, before importing the code of the Python class. This option has effect only when the Python class is provided in an external Python file. This option has no effect when the Python class is provided within the AxoSyslog configuration file (in a python{} block). You can use the loaders() option to modify the import mechanism that imports Python class. For example, that way you can use hy in your Python class.
python(class(usermodule.HyParser) loaders(hy))
options()
Type:
string
Default:
N/A
Description: This option allows you to pass custom values from the configuration file to the Python code. Enclose both the option names and their values in double-quotes. The Python code will receive these values during initialization as the options dictionary. For example, you can use this to set the IP address of the server from the configuration file, so it is not hard-coded in the Python object.
For example, you can refer to the value of the host field in the Python code as options["host"]. Note that the Python code receives the values as strings, so you might have to cast them to the type required, for example: int(options["port"])
Note
From version 3.27, AxoSyslog supports the arrow syntax for declaring custom Java and Python options. You can alternatively declare them using a similar syntax:
Description: If you receive the following error message during AxoSyslog startup, set the persist-name() option of the duplicate drivers:
Error checking the uniqueness of the persist names, please override it with persist-name option. Shutting down.
This error happens if you use identical drivers in multiple sources, for example, if you configure two file sources to read from the same file. In this case, set the persist-name() of the drivers to a custom string, for example, persist-name("example-persist-name1").
Note
Starting with 3.26, AxoSyslog assigns a persist name to Python sources and destinations. The persist name is generated from the class name. If you want to use the same Python class multiple times in your AxoSyslog configuration, add a unique persist-name() to each source or destination, otherwise AxoSyslog will not start. For example:
Alternatively, you can include the following line in the Python package: @staticmethod generate_persist_name. For example:
from syslogng import LogSource
class PyNetworSource(LogSource):
@staticmethod
def generate_persist_name(options):
return options["port"] def run(self):
pass
def request_exit(self):
pass
tags()
Type:
string
Default:
Description: Label the messages received from the source with custom tags. Tags must be unique, and enclosed between double quotes. When adding multiple tags, separate them with comma, for example, tags("dmz", "router"). This option is available only in version 3.1 and later.
time-reopen()
Accepted values:
number [seconds]
Default:
1
Description: The time to wait in seconds before a dead connection is reestablished.
time-zone()
Type:
name of the timezone, or the timezone offset
Default:
Description: The default timezone for messages read from the source. Applies only if no timezone is specified within the message itself.
The timezone can be specified by using the name, for example, time-zone("Europe/Budapest")), or as the timezone offset in +/-HH:MM format, for example, +01:00). On Linux and UNIX platforms, the valid timezone names are listed under the /usr/share/zoneinfo directory.
Warning
This option is available only when using Python 3.
The Python source allows you to write your own source in Python. You can import external Python modules to receive or fetch the messages. Since many services have a Python library, the Python source makes integrating AxoSyslog very easy and quick.
You can write two different type of sources in Python:
Server-style sources that receives messages. Write server-style sources if you want to use an event-loop based, nonblocking server framework in Python, or if you want to implement a custom loop.
Fetcher-style sources that actively fetch messages. In general, write fetcher-style sources (for example, when using simple blocking APIs), unless you explicitly need a server-style source.
The following points apply to using Python blocks in AxoSyslog in general:
Python parsers and template functions are available in AxoSyslog version 3.10 and later.
Python destinations and sources are available in AxoSyslog version 3.18 and later.
Supported Python versions: 2.7 and 3.4+ (if you are using pre-built binaries, check the dependencies of the package to find out which Python version it was compiled with).
The Python block must be a top-level block in the AxoSyslog configuration file.
If you store the Python code in a separate Python file and only include it in the AxoSyslog configuration file, make sure that the PYTHONPATH environment variable includes the path to the Python file, and export the PYTHON_PATH environment variable. For example, if you start AxoSyslog manually from a terminal and you store your Python files in the /opt/syslog-ng/etc directory, use the following command: export PYTHONPATH=/opt/syslog-ng/etc.
In production, when AxoSyslog starts on boot, you must configure your startup script to include the Python path. The exact method depends on your operating system. For recent Red Hat Enterprise Linux, Fedora, and CentOS distributions that use systemd, the systemctl command sources the /etc/sysconfig/syslog-ng file before starting AxoSyslog. (On openSUSE and SLES, /etc/sysconfig/syslog file.) Append the following line to the end of this file: PYTHONPATH="<path-to-your-python-file>", for example, PYTHONPATH="/opt/syslog-ng/etc".
The Python object is initiated every time when AxoSyslog is started or reloaded.
Warning
If you reload AxoSyslog, existing Python objects are destroyed, therefore the context and state information of Python blocks is lost. Log rotation and updating the configuration of AxoSyslog typically involves a reload.
The Python block can contain multiple Python functions.
Using Python code in AxoSyslog can significantly decrease the performance of AxoSyslog, especially if the Python code is slow. In general, the features of AxoSyslog are implemented in C, and are faster than implementations of the same or similar features in Python.
Validate and lint the Python code before using it. The AxoSyslog application does not do any of this.
Python error messages are available in the internal() source of AxoSyslog.
You can access the name-value pairs of AxoSyslog directly through a message object or a dictionary.
To help debugging and troubleshooting your Python code, you can send log messages to the internal() source of AxoSyslog. For details, see Logging from your Python code.
Declaration:
Python sources consist of two parts. The first is a AxoSyslog source object that you define in your AxoSyslog configuration and use in the log path. This object references a Python class, which is the second part of the Python source. The Python class receives or fetches the log messages, and can do virtually anything that you can code in Python. You can either embed the Python class into your AxoSyslog configuration file, or store it in an external Python file.
source <name_of_the_python_source>{ python-fetcher( class("<name_of_the_python_class_executed_by_the_source>"));}; python { from syslogng import LogFetcher
from syslogng import LogMessage
class <name_of_the_python_class_executed_by_the_source>(LogFetcher):
def init(self, options): # optional print("init") print(options)return True
def deinit(self): # optional print("deinit") def open(self): # optional print("open")return True
def fetch(self): # mandatory print("fetch")# return LogFetcher.FETCH_ERROR,# return LogFetcher.FETCH_NOT_CONNECTED,# return LogFetcher.FETCH_TRY_AGAIN,# return LogFetcher.FETCH_NO_DATA,return LogFetcher.FETCH_SUCCESS, msg
def request_exit(self):
print("request_exit")# If your fetching method is blocking, do something to break it# For example, if it reads a socket: socket.shutdown() def close(self): # optional print("close")};
Methods of the python-fetcher() source
Fetcher-style Python sources must be inherited from the syslogng.LogFetcher class, and must implement at least the fetch method. Multiple inheritance is allowed, but only for pure Python super classes.
For fetcher-style Python sources, AxoSyslog handles the event loop and the scheduling automatically. You can use simple blocking server/client libraries to receive or fetch logs.
You can retrieve messages using the fetch() method.
init(self, options) method (optional)
The AxoSyslog application initializes Python objects every time when it is started or reloaded. The init method is executed as part of the initialization. You can perform any initialization steps that are necessary for your source to work.
Warning
If you reload AxoSyslog, existing Python objects are destroyed, therefore the context and state information of Python blocks is lost. Log rotation and updating the configuration of AxoSyslog typically involves a reload.
When this method returns with False, AxoSyslog does not start. It can be used to check options and return False when they prevent the successful start of the source.
options: This optional argument contains the contents of the options() parameter of the AxoSyslog configuration object as a Python dictionary.
open(self) method (optional)
The open(self) method opens the resources required for the source, for example, it initiates a connection to the target service. It is called after init() when AxoSyslog is started or reloaded. If fetch() returns with an error, AxoSyslog calls the close() and open() methods before trying to fetch a new message.
If open() fails, it should return the False value. In this case, AxoSyslog retries it every time-reopen() seconds. By default, this is 1 second for Python sources and destinations, the value of time-reopen() is not inherited from the global option. For details, see Error handling in the python() destination.
fetch(self) method (mandatory)
Use the fetch method to fetch messages and pass them to the log paths.
The fetch method must return one of the following values:
LogFetcher.FETCH_ERROR: Fetching new messages failed, AxoSyslog calls the close and open methods.
LogFetcher.FETCH_NO_DATA: There was not any data available. The source waits before calling the fetch method again. The wait time is equal to time-reopen() by default, but you can override it by setting the fetch-no-data-delay() option in the source.
LogFetcher.FETCH_NOT_CONNECTED: Could not access the source, AxoSyslog calls the open method.
LogFetcher.FETCH_SUCCESS, msg: Post the message returned as the second argument.
LogFetcher.FETCH_TRY_AGAIN: The fetcher could not provide a message this time, but will make the source call the fetch method as soon as possible.
request_exit(self) method (optional)
If you use blocking operations within the fetch() method, use request_exit() to interrupt those operations (for example, to shut down a socket), otherwise AxoSyslog is not able to stop. Note that AxoSyslog calls the request_exit method from a thread different from the source thread.
close(self) method (optional)
Close the connection to the target service. Usually it is called right before deinit() when stopping or reloading AxoSyslog. It is also called when fecth() fails.
close_batch(self)
Closes the current source-side batch. Source-side batching helps AxoSyslog to effectively process a larger chunk of messages, instead of processing messages each message. For example, when feeding a destination queue and instead of taking a lock on the queue for every message (causing contention), we only take it once per batch.
The native drivers built into AxoSyslog typically close batches once every mainloop iteration, allowing a single iteration to process multiple messages. For instance, when receiving multiple messages in a single TCP datagram, all of those messages
can be processed as a part of the same batch.
In Python-based log sources, a batch will automatically be closed after every message posted via post_message(), except if self.auto_close_batches is set to False during initialization. In case self.auto_close_batches is set to False, the driver has to call close_batch() explicitly, preferably at a natural boundary between incoming batches of messages. A good example is when we retrieve several messages via the same HTTP REST call, then the right time to close the batch would be after the last message in the response is posted.
The deinit(self) method (optional)
This method is executed when AxoSyslog is stopped or reloaded. This method does not return a value.
Warning
If you reload AxoSyslog, existing Python objects are destroyed, therefore the context and state information of Python blocks is lost. Log rotation and updating the configuration of AxoSyslog typically involves a reload.
To configure the source, you only have to specify the root log directory of qBittorrent in the dir() parameter. You can find the root log directory by selecting Tools > Preferences > Behavior > Log file > Save path in the qBittorrent application. Otherwise, the qbittorrent() source has the same parameters as the file() source.
The qbittorrent() driver is actually a reusable configuration snippet. For details on using or writing such configuration snippets, see Reusing configuration blocks. You can find the source of this configuration snippet on GitHub.
7.25 - snmptrap: Read Net-SNMP traps
Using the snmptrap() source, you can read and parse the SNMP traps of the Net-SNMP’s snmptrapd application. AxoSyslog can read these traps from a log file, and extract their content into name-value pairs, making it easy to forward them as a structured log message (for example, in JSON format). The AxoSyslog application automatically adds the .snmp. prefix to the name of the fields the extracted from the message.
The snmptrap() source is available in AxoSyslog version 3.10 and later.
Limitations:
The snmptrap() source has only the options listed in snmptrap() source options. Other options commonly available in other source drivers are not supported.
In addition to traps, the log of snmptrapd may contain other messages (for example, daemon start/stop information, debug logs) as well. Currently AxoSyslog discards these messages.
The AxoSyslog application cannot resolve OIDs, you have to configure snmptrapd to do so. Note that because of a bug, if snmptrapd does not escape String values in the VarBindList if it can resolve an OID to a symbolic name. As a result, AxoSyslog cannot process traps that contain the = in the value of the string. To overcome this problem, disable resolving OIDs in snmptrapd. For details, see the documentation of snmptrapd.
The colon (:) character is commonly used in SNMP traps. However, this character cannot be used in the name of AxoSyslog macros (name-value pairs). Therefore, the AxoSyslog application automatically replaces all consecutive : characters with a single underscore (_) character. For example, you can reference the value of the NET-SNMP-EXAMPLES-MIB::netSnmpExampleString key using the ${NET-SNMP-EXAMPLES-MIB_netSnmpExampleString} macro.
Note that this affects only name-value pairs (macros). The generated message always contains the original name of the key.
Prerequisites:
Configure snmptrapd to log into a file.
If you use SMIv1 traps, include the following format string in the configuration file of snmptrapd:
Beacause of an snmptrapd bug, if you specify the filename in the configuration file with logOption, you must also specify another output as a command line argument (-Lf, -Ls). Otherwise, snmptrapd will not apply the the trap format.
To use the snmptrap() driver, the scl.conf file must be included in your AxoSyslog configuration:
@include "scl.conf"
Example: Using the snmptrap() driver
A sample snmptrapd configuration:
authCommunity log,execute,net public
format1 %.4y-%.2m-%.2l %.2h:%.2j:%.2k %B [%b]: %N\n\t%W Trap (%q) Uptime: %#T\n%v\n outputOption s
Starting snmptrapd: snmptrapd -A -Lf /var/log/snmptrapd.log
Sending a sample V2 trap message: snmptrap -v2c -c public 127.0.0.1 666 NET-SNMP-EXAMPLES-MIB::netSnmpExampleHeartbeatNotification netSnmpExampleHeartbeatRate i 60 netSnmpExampleString s "string". From this trap, AxoSyslog receives the following input:
The following AxoSyslog configuration sample uses the default settings of the driver, reading SNMP traps from the /var/log/snmptrapd.log file, and writes the log messages generated from the traps into a file.
The snmptrap() driver has the following options. Only the filename() option is required, the others are optional.
filename()
Type:
path
Default:
Description: The log file of snmptrapd. The AxoSyslog application reads the traps from this file.
In addition to traps, the log of snmptrapd may contain other messages (for example, daemon start/stop information, debug logs) as well. Currently AxoSyslog discards these messages.
hook-commands()
Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.
Note
The AxoSyslog application must be able to start and restart the external program, and have the necessary permissions to do so. For example, if your host is running AppArmor or SELinux, you might have to modify your AppArmor or SELinux configuration to enable AxoSyslog to execute external applications.
Using hook-commands() when AxoSyslog starts or stops
To execute an external program when AxoSyslog starts or stops, use the following options:
startup()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog starts.
shutdown()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog stops.
Using the hook-commands() when AxoSyslog reloads
To execute an external program when the AxoSyslog configuration is initiated or torn down, for example, on startup/shutdown or during a AxoSyslog reload, use the following options:
setup()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is initiated, for example, on startup or during a AxoSyslog reload.
teardown()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is stopped or torn down, for example, on shutdown or during a AxoSyslog reload.
Example: Using the hook-commands() with a network source
In the following example, the hook-commands() is used with the network() driver and it opens an iptables port automatically as AxoSyslog is started/stopped.
The assumption in this example is that the LOGCHAIN chain is part of a larger ruleset that routes traffic to it. Whenever the AxoSyslog created rule is there, packets can flow, otherwise the port is closed.
Description: If you receive the following error message during AxoSyslog startup, set the persist-name() option of the duplicate drivers:
Error checking the uniqueness of the persist names, please override it with persist-name option. Shutting down.
This error happens if you use identical drivers in multiple sources, for example, if you configure two file sources to read from the same file. In this case, set the persist-name() of the drivers to a custom string, for example, persist-name("example-persist-name1").
prefix()
Synopsis:
prefix()
Description: Insert a prefix before the name part of the parsed name-value pairs to help further processing. For example:
To insert the my-parsed-data. prefix, use the prefix(my-parsed-data.) option.
To refer to a particular data that has a prefix, use the prefix in the name of the macro, for example, ${my-parsed-data.name}.
If you forward the parsed messages using the IETF-syslog protocol, you can insert all the parsed data into the SDATA part of the message using the prefix(.SDATA.my-parsed-data.) option.
Names starting with a dot (for example, .example) are reserved for use by AxoSyslog. If you use such a macro name as the name of a parsed value, it will attempt to replace the original value of the macro (note that only soft macros can be overwritten, see Hard versus soft macros for details). To avoid such problems, use a prefix when naming the parsed values, for example, prefix(my-parsed-data.)
Default value: .snmp. option.
set-message-macro()
Type:
`yes
Default:
yes
Description: The snmptrap() source automatically parses the traps into name-value pairs, so you can handle the content of the trap as a structured message. Consequently, you might not even need the ${MESSAGE} part of the log message. If set-message-macro() is set to no, AxoSyslog leaves the ${MESSAGE} part empty. If set-message-macro() is set to yes, AxoSyslog generates a regular log message from the trap.
7.26 - sun-streams: Collect messages on Sun Solaris
Solaris uses its STREAMS framework to send messages to the syslogd process. Solaris 2.5.1 and above uses an IPC called door in addition to STREAMS, to confirm the delivery of a message. The AxoSyslog application supports the IPC mechanism via the door() option (see below).
Note
The sun-streams() driver must be enabled when the syslog-ng application is compiled (see ./configure --help).
The sun-streams() driver has a single required argument specifying the STREAMS device to open, and the door() option. For the list of available optional parameters, see sun-streams() source options.
Note
Starting with version 3.7, the AxoSyslogsystem() driver automatically extracts the msgid from the message (if available), and stores it in the .solaris.msgid macro. To extract the msgid from the message without using the system()driver, use the extract-solaris-msgid() parser. You can find the exact source of this parser in the AxoSyslog GitHub repository.
Description: Specifies the log parsing options of the source.
assume-utf8: The assume-utf8 flag assumes that the incoming messages are UTF-8 encoded, but does not verify the encoding. If you explicitly want to validate the UTF-8 encoding of the incoming message, use the validate-utf8 flag.
empty-lines: Use the empty-lines flag to keep the empty lines of the messages. By default, AxoSyslog removes empty lines automatically.
expect-hostname: If the expect-hostname flag is enabled, AxoSyslog will assume that the log message contains a hostname and parse the message accordingly. This is the default behavior for TCP sources. Note that pipe sources use the no-hostname flag by default.
guess-timezone: Attempt to guess the timezone of the message if this information is not available in the message. Works when the incoming message stream is close to real time, and the timezone information is missing from the timestamp.
kernel: The kernel flag makes the source default to the LOG_KERN | LOG_NOTICE priority if not specified otherwise.
no-header: The no-header flag triggers AxoSyslog to parse only the PRI field of incoming messages, and put the rest of the message contents into $MSG.
Its functionality is similar to that of the no-parse flag, except the no-header flag does not skip the PRI field.
Note
Essentially, the no-header flag signals AxoSyslog that the syslog header is not present (or does not adhere to the conventions / RFCs), so the entire message (except from the PRI field) is put into $MSG.
Example: using the no-header flag with the syslog-parser() parser
The following example illustrates using the no-header flag with the syslog-parser() parser:
no-hostname: Enable the no-hostname flag if the log message does not include the hostname of the sender host. That way AxoSyslog assumes that the first part of the message header is ${PROGRAM} instead of ${HOST}. For example:
no-multi-line: The no-multi-line flag disables line-breaking in the messages: the entire message is converted to a single line. Note that this happens only if the underlying transport method actually supports multi-line messages. Currently the file() and pipe() drivers support multi-line messages.
no-parse: By default, AxoSyslog parses incoming messages as syslog messages. The no-parse flag completely disables syslog message parsing and processes the complete line as the message part of a syslog message. The AxoSyslog application will generate a new syslog header (timestamp, host, and so on) automatically and put the entire incoming message into the MESSAGE part of the syslog message (available using the ${MESSAGE} macro). This flag is useful for parsing messages not complying to the syslog format.
If you are using the flags(no-parse) option, then syslog message parsing is completely disabled, and the entire incoming message is treated as the ${MESSAGE} part of a syslog message. In this case, AxoSyslog generates a new syslog header (timestamp, host, and so on) automatically. Note that even though flags(no-parse) disables message parsing, some flags can still be used, for example, the no-multi-line flag.
dont-store-legacy-msghdr: By default, AxoSyslog stores the original incoming header of the log message. This is useful if the original format of a non-syslog-compliant message must be retained (AxoSyslog automatically corrects minor header errors, for example, adds a whitespace before msg in the following message: Jan 22 10:06:11 host program:msg). If you do not want to store the original header of the message, enable the dont-store-legacy-msghdr flag.
sanitize-utf8: When using the sanitize-utf8 flag, AxoSyslog converts non-UTF-8 input to an escaped form, which is valid UTF-8.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
store-raw-message: Save the original message as received from the client in the ${RAWMSG} macro. You can forward this raw message in its original form to another AxoSyslog node using the syslog-ng() destination, or to a SIEM system, ensuring that the SIEM can process it. Available only in 3.16 and later.
syslog-protocol: The syslog-protocol flag specifies that incoming messages are expected to be formatted according to the new IETF syslog protocol standard (RFC5424), but without the frame header. Note that this flag is not needed for the syslog driver, which handles only messages that have a frame header.
validate-utf8: The validate-utf8 flag enables encoding-verification for messages.
Prior to version 4.6, this flag worked only when parsing RFC3164 messages. Starting with version 4.6, it works also for RFC5424 and raw messages.
For RFC5424-formatted messages, if the BOM character is missing, but the message is otherwise UTF-8 compliant, AxoSyslog automatically adds the BOM character to the message.
The byte order mark (BOM) is a Unicode character used to signal the byte-order of the message text.
follow-freq()
Type:
number
Default:
1
Description: Indicates that the source should be checked periodically. This is useful for files which always indicate readability, even though no new lines were appended. If this value is higher than zero, AxoSyslog will not attempt to use poll() on the file, but checks whether the file changed every time the follow-freq() interval (in seconds) has elapsed. Floating-point numbers (for example, 1.5) can be used as well.
hook-commands()
Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.
Note
The AxoSyslog application must be able to start and restart the external program, and have the necessary permissions to do so. For example, if your host is running AppArmor or SELinux, you might have to modify your AppArmor or SELinux configuration to enable AxoSyslog to execute external applications.
Using hook-commands() when AxoSyslog starts or stops
To execute an external program when AxoSyslog starts or stops, use the following options:
startup()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog starts.
shutdown()
Type:
string
Default:
N/A
Description: Defines the external program that is executed as AxoSyslog stops.
Using the hook-commands() when AxoSyslog reloads
To execute an external program when the AxoSyslog configuration is initiated or torn down, for example, on startup/shutdown or during a AxoSyslog reload, use the following options:
setup()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is initiated, for example, on startup or during a AxoSyslog reload.
teardown()
Type:
string
Default:
N/A
Description: Defines an external program that is executed when the AxoSyslog configuration is stopped or torn down, for example, on shutdown or during a AxoSyslog reload.
Example: Using the hook-commands() with a network source
In the following example, the hook-commands() is used with the network() driver and it opens an iptables port automatically as AxoSyslog is started/stopped.
The assumption in this example is that the LOGCHAIN chain is part of a larger ruleset that routes traffic to it. Whenever the AxoSyslog created rule is there, packets can flow, otherwise the port is closed.
Description: Specifies whether AxoSyslog should accept the timestamp received from the sending application or client. If disabled, the time of reception will be used instead. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.
Warning
To use the S_ macros, the keep-timestamp() option must be enabled (this is the default behavior of AxoSyslog).
log-fetch-limit()
Type:
number
Default:
100
Description: The maximum number of messages fetched from a source during a single poll loop. The destination queues might fill up before flow-control could stop reading if log-fetch-limit() is too high.
log-iw-size()
Type:
number
Default:
100
Description: The size of the initial window, this value is used during flow-control. Its value cannot be lower than 100, unless the dynamic-window-size() option is enabled. For details on flow-control, see Managing incoming and outgoing messages with flow-control.
log-msg-size()
Type:
number (bytes)
Default:
Use the global log-msg-size() option, which defaults to 65536 (64 KiB).
Description: Maximum length of an incoming message in bytes. This length includes the entire message (the data structure and individual fields). The maximal value that can be set is 268435456 bytes (256 MiB).
For messages using the IETF-syslog message format (RFC5424), the maximal size of the value of an SDATA field is 64 KiB.
Note
In most cases, log-msg-size() does not need to be set higher than 10 MiB.
Uses the value of the global option if not specified.
log-prefix() (DEPRECATED)
Type:
string
Default:
Description: A string added to the beginning of every log message. It can be used to add an arbitrary string to any log source, though it is most commonly used for adding kernel: to the kernel messages on Linux.
Note
This option is deprecated. Use program-override instead.
optional()
Type:
yes or no
Default:
Description: Instruct AxoSyslog to ignore the error if a specific source cannot be initialized. No other attempts to initialize the source will be made until the configuration is reloaded. This option currently applies to the pipe(), unix-dgram, and unix-stream drivers.
pad-size()
Type:
number
Default:
0
Description: Specifies input padding. Some operating systems (such as HP-UX) pad all messages to block boundary. This option can be used to specify the block size. The AxoSyslog application will pad reads from the associated device to the number of bytes set in pad-size(). Mostly used on HP-UX where /dev/log is a named pipe and every write is padded to 2048 bytes. If pad-size() was given and the incoming message does not fit into pad-size(), AxoSyslog will not read anymore from this pipe and displays the following error message:
Padding was set, and couldn't read enough bytes
program-override()
Type:
string
Default:
Description: Replaces the ${PROGRAM} part of the message with the parameter string. For example, to mark every message coming from the kernel, include the program-override("kernel") option in the source containing /proc/kmsg.
tags()
Type:
string
Default:
Description: Label the messages received from the source with custom tags. Tags must be unique, and enclosed between double quotes. When adding multiple tags, separate them with comma, for example, tags("dmz", "router"). This option is available only in version 3.1 and later.
time-zone()
Type:
name of the timezone, or the timezone offset
Default:
Description: The default timezone for messages read from the source. Applies only if no timezone is specified within the message itself.
The timezone can be specified by using the name, for example, time-zone("Europe/Budapest")), or as the timezone offset in +/-HH:MM format, for example, +01:00). On Linux and UNIX platforms, the valid timezone names are listed under the /usr/share/zoneinfo directory.
7.27 - syslog: Collect messages using the IETF-syslog protocol
The syslog() driver can receive messages from the network using the standard IETF-syslog protocol (as described in RFC5424-26). UDP, TCP, and TLS-encrypted TCP can all be used to transport the messages.
Note
The syslog() driver can also receive BSD-syslog-formatted messages (described in RFC 3164, see BSD-syslog or legacy-syslog messages) if they are sent using the IETF-syslog protocol.
In AxoSyslog versions 3.1 and earlier, the syslog() driver could handle only messages in the IETF-syslog (RFC 5424-26) format.
For the list of available optional parameters, see
