This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

systemd-journal: Collect messages from the systemd-journal system log storage

1: systemd-journal() source options

The systemd-journal() source is used on various Linux distributions, such as RHEL (from RHEL7) and CentOS. The systemd-journal() source driver can read the structured name-value format of the journald system service, making it easier to reach the custom fields in the message. By default, AxoSyslog adds the .journald. prefix to the name of every parsed value. For a list and description of name-value pairs that journald provides, see the documentation of journald for your platform (for example, man systemd.journal-fields).

The systemd-journal() source driver is designed to read only local messages through the systemd-journal API. It is not possible to set the location of the journal files, or the directories.

Note The log-msg-size() option is not applicable for this source. Use the max-field-size() option instead.

Note

This source will not handle the following cases:

Corrupted journal file
Incorrect journal configuration
Any other journald-related bugs

Note If you are using RHEL-7, the default source in the configuration is systemd-journal() instead of unix-dgram("/dev/log") and file("/proc/kmsg"). If you are using unix-dgram("/dev/log") or unix-stream("/dev/log") in your configuration as a source, AxoSyslog will revert to using systemd-journal() instead.

Warning Only one systemd-journal() source can be configured in the configuration file. If there is more than one systemd-journal() source configured, AxoSyslog will not start.

Declaration:

   systemd-journal(options);

If you want to use multiple systemd-journal() sources in your configuration, the sources must use unique systemd namespaces. For details, see the namespace() option.

Example: Send all fields through syslog protocol

To send all fields through the syslog protocol, enter the prefix in the following format: “.SDATA.<name>”.

   @version: 4.26
    
    source s_journald {
        systemd-journal(prefix(".SDATA.journald."));
    };
    
    destination d_network {
        syslog("server.host");
    };
    
    log {
        source(s_journald);
        destination(d_network);
    };

Example: Filter for a specific field

   @version: 4.26
    
    source s_journald {
        systemd-journal(prefix(".SDATA.journald."));
    };
    
    filter f_uid {"${.SDATA.journald._UID}" eq "1000"};
    
    destination d_network {
        syslog("server.host");
    };
    
    log {
        source(s_journald);
        filter(f_uid);
        destination(d_network);
    };

Example: Send all fields in value-pairs

   @version: 4.26
    
    source s_local {
        systemd-journal(prefix("journald."));
    };
    
    destination d_network {
        network("server.host" template("$(format_json --scope rfc5424 --key journald.*)\n"));
    };
    
    log {
        source(s_local);
        destination(d_network);
    };

The journal contains credential information about the process that sent the log message. The AxoSyslog application makes this information available in the following macros:

Journald fields as macros

Journald field	AxoSyslog predefined macro
MESSAGE	$MESSAGE
_HOSTNAME	$HOST
_PID	$PID
_COMM or SYSLOG_IDENTIFIER	$PROGRAM If both _COMM and SYSLOG_IDENTIFIER exists, AxoSyslog uses SYSLOG_IDENTIFIER
SYSLOG_FACILITY	$FACILITY_NUM
PRIORITY	$LEVEL_NUM

1 - systemd-journal() source options

The systemd-journal() driver has the following options:

chain-hostnames()


Accepted values:	`yes`, `no`
Default:	`no`

Description: Enable or disable the chained hostname format. For details, see the chain-hostnames() global option.

default-facility()


Type:	facility string
Default:	local0

Description: The default facility value if the SYSLOG_FACILITY entry does not exist.

default-level()


Type:	string
Default:	notice

Description: The default level value if the PRIORITY entry does not exist.

default-priority()


Type:	priority string
Default:	notice

Description: This parameter assigns a severity to the messages received from the source if the message does not specify one. For example, default-priority(warning).

default-severity()


Type:	severity string
Default:	`notice`

Description: Assigns a default severity to messages received from this source when the message doesn’t specify one. Accepts the standard syslog severity keywords (for example, emerg, alert, crit, err, warning, notice, info, debug). Combined with default-facility() to compute the message priority.

dns-cache()


Accepted values:	`yes`, `no`
Default:	`no`

Description: Enable or disable DNS cache usage.

hook-commands()

Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.

Note The AxoSyslog application must be able to start and restart the external program, and have the necessary permissions to do so. For example, if your host is running AppArmor or SELinux, you might have to modify your AppArmor or SELinux configuration to enable AxoSyslog to execute external applications.

Using `hook-commands()` when AxoSyslog starts or stops

To execute an external program when AxoSyslog starts or stops, use the following options:

`startup()`

Type:	string
Default:	N/A

Description: Defines the external program that is executed as AxoSyslog starts.

`shutdown()`

Type:	string
Default:	N/A

Description: Defines the external program that is executed as AxoSyslog stops.

Using the hook-commands() when AxoSyslog reloads

To execute an external program when the AxoSyslog configuration is initiated or torn down, for example, on startup/shutdown or during a AxoSyslog reload, use the following options:

`setup()`

Type:	string
Default:	N/A

Description: Defines an external program that is executed when the AxoSyslog configuration is initiated, for example, on startup or during a AxoSyslog reload.

`teardown()`

Type:	string
Default:	N/A

Description: Defines an external program that is executed when the AxoSyslog configuration is stopped or torn down, for example, on shutdown or during a AxoSyslog reload.

Example: Using hook-commands() with a network source

In the following example, the hook-commands() is used with the network() driver and it opens an iptables port automatically as AxoSyslog is started/stopped.

The assumption in this example is that the LOGCHAIN chain is part of a larger ruleset that routes traffic to it. Whenever the AxoSyslog created rule is there, packets can flow, otherwise the port is closed.

source {
    network(transport(udp)
    hook-commands(
          startup("iptables -I LOGCHAIN 1 -p udp --dport 514 -j ACCEPT")
          shutdown("iptables -D LOGCHAIN 1")
        )
     );
};

host-override()


Type:	string
Default:

Description: Replaces the ${HOST} part of the message with the parameter string.

internal()


Accepted values:	`yes`, `no`
Default:	`no`

Description: Marks this pipeline element as internal. Elements marked as internal() are treated as an implementation detail, so for example statistics of the given pipe are available only on higher stats level. This option is mainly useful for developers or when writing SCL blocks and integrations.

keep-hostname()


Type:	yes or no
Default:	no

Description: Enable or disable hostname rewriting.

If enabled (keep-hostname(yes)), AxoSyslog will retain the hostname information read from the systemd journal messages.
If disabled (keep-hostname(no)), AxoSyslog will use the hostname that has been set up for the operating system instance that AxoSyslog is running on. To query or set this value, use the hostnamectl command.

This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.

keep-timestamp()


Type:	yes or no
Default:	yes

Description: Specifies whether AxoSyslog should accept the timestamp received from the sending application or client. If disabled, the time of reception will be used instead. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.

Warning To use the S_ macros, the keep-timestamp() option must be enabled (this is the default behavior of AxoSyslog).

log-fetch-limit()


Type:	number
Default:	100

Description: The maximum number of messages fetched from a source during a single poll loop. The destination queues might fill up before flow-control could stop reading if log-fetch-limit() is too high.

log-iw-size()


Type:	number
Default:	100

Description: Specifies the source window size - the maximum number of in-flight messages permitted by the source before flow control is enforced. This only applies when flow-control is enabled.

CAUTION:

If you change the value of log-iw-size() and keep-alive() is enabled, the change will affect only new connections, the log-iw-size() of kept-alive connections will not change. To apply the new log-iw-size() value to every connection, restart the syslog-ng service. A simple configuration reload is NOT sufficient.

If the source is receiving data using the UDP protocol, always restart the syslog-ng service after changing the value of log-iw-size() for the changes to take effect.

Note that when using disk-buffer(), the messages stored on disk are not included in the window size calculation. For details about the effects of this parameter, see Managing incoming and outgoing messages with flow-control.

log-prefix() (DEPRECATED)


Type:	string
Default:

This option is deprecated. Use program-override instead.

Description: A string added to the beginning of every log message. It can be used to add an arbitrary string to any log source, though it is most commonly used for adding kernel: to the kernel messages on Linux.

long-hostnames()

Obsolete alias for chain-hostnames().

-->

match-boot()


Type:	`yes`, `no`
Default:	no

Available in AxoSyslog 4.1 and later.

Description: If set to yes, AxoSyslog fetches only journal messages that relate to the current boot, and to ignores messages generated in previous boots.

matches()


Type:	arrow list
Default:

Available in AxoSyslog 4.1 and later.

Description: Specifies one or more filters to apply on the journal fields, similarly how you can use journalctl. For example:

matches(
    "_COMM" => "systemd"
    )

max-field-size()


Type:	number (characters)
Default:	65536

Description: The maximum length of a field’s value.

namespace()


Type:	string
Default:	“*”

Description: The namespace() option works exactly the same way as the respective option of the Journalctl command line tool.

The following modes of operation are available:

If you do not specify the namespace() option in your configuration, or if you specify an empty string, the systemd-journal() source reads and displays log data from all namespaces.
If you specify the namespace() option as namespace("*"), the systemd-journal() source reads and displays log data from all namespaces, interleaved.
If namespace(<specific-namespace>) is specified, the systemd-journal() source only reads and displays log data from the specified namespace.
If the namespace identifier is prefixed with "+" when you specify your namespace() option, the systemd-journal() source only reads and displays log data from the specified namespace and the default namespace, interleaved.

Starting with AxoSyslog 4.4, you can use multiple systemd-journal() sources in your configuration. In this case, each source must use unique systemd namespaces.

Syntax: namespace(string)

Example: configuration examples for using the namespace() option

The following configuration example uses the default value for the namespace() option:

   source s_journal
    { 
      systemd-journal(namespace("*"));
    };

The following configuration example uses a prefixed namespace identifier in the namespace() option:

   source s_journal
    { 
      systemd-journal(namespace("+foobar"));
    };

Note Namespace support was introduced to the Journalctl command line tool in Systemd version 2.45. The AxoSyslog application supports the namespace() option from version 3.29. For further information about namespaces on the Systemd side, see Journal Namespaces.

normalize-hostnames()


Accepted values:	`yes`, `no`
Default:	`no`

Description: If enabled (normalize-hostnames(yes)), AxoSyslog converts the hostnames to lowercase.

Note

This setting applies only to hostnames resolved from DNS. It has no effect if the <code>keep-hostname()</code> option is enabled, and the message contains a hostname.

persist-name()


Type:	string
Default:	N/A

Description: If you receive the following error message during AxoSyslog startup, set the persist-name() option of the duplicate drivers:

Error checking the uniqueness of the persist names, please override it with persist-name option. Shutting down.

This error happens if you use identical drivers in multiple sources, for example, if you configure two file sources to read from the same file. In this case, set the persist-name() of the drivers to a custom string, for example, persist-name("example-persist-name1").

prefix()


Type:	string
Default:	.journald.

Description: If this option is set, every non-built-in mapped names get a prefix (for example: ".SDATA.journald."). By default, AxoSyslog adds the .journald. prefix to every value.

program-override()


Type:	string
Default:

Description: Replaces the ${PROGRAM} part of the message with the parameter string. For example, to mark every message coming from the kernel, include the program-override("kernel") option in the source containing /proc/kmsg.

read-old-records()


Type:	`yes` or `no`
Default:	yes

Description: If set to yes, AxoSyslog will start reading the records from the beginning of the file/journal, if the file/journal has not been read yet. If set to no, AxoSyslog will read only the new records. If the source has a state in the persist file, this option will have no effect.

tags()


Type:	string
Default:

Description: Label the messages received from the source with custom tags. Tags must be unique, and enclosed between double quotes. When adding multiple tags, separate them with comma, for example, tags("dmz", "router"). This option is available only in version 3.1 and later.

time-zone()


Type:	name of the timezone, or the timezone offset
Default:

Description: The default timezone for messages read from the source. Applies only if no timezone is specified within the message itself.

The timezone can be specified by using the name, for example, time-zone("Europe/Budapest")), or as the timezone offset in +/-HH:MM format, for example, +01:00). On Linux and UNIX platforms, the valid timezone names are listed under the /usr/share/zoneinfo directory.

use-dns()


Type:	`yes`, `no`, `persist_only`
Default:	`yes`

Description: Enable or disable DNS usage. The persist_only option attempts to resolve hostnames locally from a file (for example, from /etc/hosts) set using the dns-cache-hosts() global option. The AxoSyslog application blocks on DNS queries, so enabling DNS may lead to a Denial of Service attack. To prevent DoS, protect your AxoSyslog network endpoint with firewall rules, and make sure that all hosts which may get to AxoSyslog are resolvable. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.

For details on configuring local DNS resolution, see Using name resolution in syslog-ng.

Note

This option has no effect if the <code>keep-hostname()</code> option is enabled (<code>keep-hostname(yes)</code>) and the message contains a hostname.

use-fqdn()


Type:	yes or no
Default:	no

Description: Use this option to add a Fully Qualified Domain Name (FQDN) instead of a short hostname. You can specify this option either globally or per-source. The local setting of the source overrides the global option if available.

Note Set use-fqdn() to yes if you want to use the custom-domain() global option.

Note This option has no effect if the keep-hostname() option is enabled (keep-hostname(yes)) and the message contains a hostname.

use-syslogng-pid()


Type:	`yes`, `no`
Default:	`no`

Description: When enabled, AxoSyslog overrides the PID from the original message with the PID of the syslog-ng process.

systemd-journal: Collect messages from the systemd-journal system log storage

Declaration:

Example: Send all fields through syslog protocol

Example: Filter for a specific field

Example: Send all fields in value-pairs

Journald fields as macros

1 - systemd-journal() source options

chain-hostnames()

default-facility()

default-level()

default-priority()

default-severity()

dns-cache()

hook-commands()

Using hook-commands() when AxoSyslog starts or stops

startup()

shutdown()

Using the hook-commands() when AxoSyslog reloads

setup()

teardown()

Example: Using hook-commands() with a network source

host-override()

internal()

keep-hostname()

keep-timestamp()

log-fetch-limit()

log-iw-size()

log-prefix() (DEPRECATED)

long-hostnames()

match-boot()

matches()

max-field-size()

namespace()

Example: configuration examples for using the namespace() option

normalize-hostnames()

persist-name()

prefix()

program-override()

read-old-records()

tags()

time-zone()

use-dns()

use-fqdn()

use-syslogng-pid()

Using `hook-commands()` when AxoSyslog starts or stops

`startup()`

`shutdown()`

`setup()`

`teardown()`