This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Linux audit parser

The Linux audit parser can parse the log messages of the Linux Audit subsystem (auditd). The AxoSyslog application can separate these log messages to name-value pairs. For details on using value-pairs in AxoSyslog see Structuring macros, metadata, and other value-pairs. The following is a sample log message of auditd:

   type=SYSCALL msg=audit(1441988805.991:239): arch=c000003e syscall=59 success=yes exit=0 a0=7fe49a6d0e98 a1=7fe49a6d0e40 a2=7fe49a6d0e80 a3=2 items=2 ppid=3652 pid=3660 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=5 comm="dumpe2fs" exe="/sbin/dumpe2fs" key=(null)
    type=EXECVE msg=audit(1441988805.991:239): argc=3 a0="dumpe2fs" a1="-h" a2="/dev/sda1"
    type=CWD msg=audit(1441988805.991:239):  cwd="/"
    type=PATH msg=audit(1441988805.991:239): item=0 name="/sbin/dumpe2fs" inode=137078 dev=08:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
    type=PATH msg=audit(1441988805.991:239): item=1 name="/lib64/ld-linux-x86-64.so.2" inode=5243184 dev=08:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
    type=PROCTITLE msg=audit(1441988805.991:239): proctitle=64756D7065326673002D68002F6465762F73646131

Certain fields of the audit log can be encoded in hexadecimal format, for example, the arch field, or the a<number> fields in the previous example. The AxoSyslog application automatically decodes these fields (for example, the c000003e value becomes x86_64).

The AxoSyslog application extracts every field into name-value pairs. It automatically decodes the following fields:

  • name

  • proctitle

  • path

  • dir

  • comm

  • ocomm

  • data

  • old

  • new

To parse the log messages of the Linux Audit subsystem, define a parser that has the linux-audit-parser() option. By default, the parser will process the ${MESSAGE} part of the log message. To process other parts of a log message, use the template() option. You can also define the parser inline in the log path.

Declaration:

   parser parser_name {
        linux-audit-parser(
            prefix()
            template()
        );
    };

Example: Using the linux-audit-parser() parser

In the following example, the source is a log file created by auditd. Since the audit log format is not a syslog format, the syslog parser is disabled, so that AxoSyslog does not parse the message: flags(no-parse). The parser inserts “.auditd.” prefix before all extracted name-value pairs. The destination is a file, that uses the format-json template function. Every name-value pair that begins with a dot (".") character will be written to the file (dot-nv-pairs). The log line connects the source, the destination, and the parser.

   source s_auditd {
        file(/var/log/audit/audit.log flags(no-parse));
    };
    
    destination d_json {
        file(
            "/tmp/test.json"
            template("$(format-json .auditd.*)\n")
        );
    };
    
    parser p_auditd {
        linux-audit-parser (prefix(".auditd."));
    };
    
    log {
        source(s_auditd);
        parser(p_auditd);
        destination(d_json);
    };

You can also define the parser inline in the log path.

   source s_auditd {
        file(/var/log/audit/audit.log);
    };
    
    destination d_json {
        file(
            "/tmp/test.json"
            template("$(format-json .auditd.*)\n")
        );
    };
    
    log {
        source(s_auditd);
        parser {
            linux-audit-parser (prefix(".auditd."));
        };
        destination(d_json);
    };

1 - Options of linux-audit-parser() parsers

The linux-audit-parser() has the following options. Because linux-audit-parser() is a specialized key-value parser, it accepts the same key-value options as Options of key=value parsers.

<!-- This file is under the copyright of Axoflow, and licensed under Apache License 2.0, except for using the Axoflow and AxoSyslog trademarks. -->

allow-pair-separator-in-value()
Accepted values: yes, no
Default: yes

Description: Permits the pair separator character inside the value of a key-value pair, so that key1=value with spaces, key2=other value parses into two pairs even though the value of key1 contains spaces.

AxoSyslog always parses key-value pairs this way, so setting this option has no effect. AxoSyslog only accepts it so that older configurations remain valid.

<!-- DISCLAIMER: This file is based on the syslog-ng Open Source Edition documentation https://github.com/balabit/syslog-ng-ose-guides/commit/2f4a52ee61d1ea9ad27cb4f3168b95408fddfdf2 and is used under the terms of The syslog-ng Open Source Edition Documentation License. The file has been modified by Axoflow. -->

extract-stray-words-into()
Synopsis: extract-stray-words-into(")

Description: Specifies the name-value pair where AxoSyslog stores any stray words that appear before or between the parsed key-value pairs (mainly when the pair-separator() option is also set). If multiple stray words appear in a message, then AxoSyslog stores them as a comma-separated list. Note that the prefix() option does not affect the name-value pair storing the stray words. Default value:N/A

Example: Extracting stray words in key-value pairs

For example, consider the following message:

   VSYS=public; Slot=5/1; protocol=17; source-ip=10.116.214.221; source-port=50989; destination-ip=172.16.236.16; destination-port=162;time=2016/02/18 16:00:07; interzone-emtn_s1_vpn-enodeb_om; inbound; policy=370;

This is a list of key-value pairs, where the value separator is = and the pair separator is ;. However, before the last key-value pair (policy=370), there are two stray words: interzone-emtn_s1_vpn-enodeb_om inbound. If you want to store or process these, specify a name-value pair to store them in the extract-stray-words-into() option, for example, extract-stray-words-into("my-stray-words"). The value of ${my-stray-words} for this message will be interzone-emtn_s1_vpn-enodeb_om, inbound

internal()
Accepted values: yes, no
Default: no

Description: Marks this pipeline element as internal. Elements marked as internal() are treated as an implementation detail, so for example statistics of the given pipe are available only on higher stats level. This option is mainly useful for developers or when writing SCL blocks and integrations.

<!-- DISCLAIMER: This file is based on the syslog-ng Open Source Edition documentation https://github.com/balabit/syslog-ng-ose-guides/commit/2f4a52ee61d1ea9ad27cb4f3168b95408fddfdf2 and is used under the terms of The syslog-ng Open Source Edition Documentation License. The file has been modified by Axoflow. -->

pair-separator()
Synopsis: pair-separator(")

Description: Specifies the character or string that separates the key-value pairs from each other. Default value: , .

For example, to parse key1=value1;key2=value2 pairs, use kv-parser(pair-separator(";")); .

<!-- DISCLAIMER: This file is based on the syslog-ng Open Source Edition documentation https://github.com/balabit/syslog-ng-ose-guides/commit/2f4a52ee61d1ea9ad27cb4f3168b95408fddfdf2 and is used under the terms of The syslog-ng Open Source Edition Documentation License. The file has been modified by Axoflow. -->

prefix()
Synopsis: prefix()

Description: Insert a prefix before the name part of the parsed name-value pairs to help further processing. For example:

  • To insert the my-parsed-data. prefix, use the prefix(my-parsed-data.) option.

  • To refer to a particular data that has a prefix, use the prefix in the name of the macro, for example, ${my-parsed-data.name}.

  • If you forward the parsed messages using the IETF-syslog protocol, you can insert all the parsed data into the SDATA part of the message using the prefix(.SDATA.my-parsed-data.) option.

Names starting with a dot (for example, .example) are reserved for use by AxoSyslog. If you use such a macro name as the name of a parsed value, it will attempt to replace the original value of the macro (note that only soft macros can be overwritten, see Hard versus soft macros for details). To avoid such problems, use a prefix when naming the parsed values, for example, prefix(my-parsed-data.)

By default, linux-audit-parser() uses the .auditd. prefix. To modify it, use the following format:

   parser {
        linux-audit-parser(prefix("myprefix."));
    };

<!-- DISCLAIMER: This file is based on the syslog-ng Open Source Edition documentation https://github.com/balabit/syslog-ng-ose-guides/commit/2f4a52ee61d1ea9ad27cb4f3168b95408fddfdf2 and is used under the terms of The syslog-ng Open Source Edition Documentation License. The file has been modified by Axoflow. -->

template()
Synopsis: template("${<macroname>}")

Description: The macro that contains the part of the message that the parser will process. It can also be a macro created by a previous parser of the log path. By default, the parser processes the entire message (${MESSAGE}).

<!-- DISCLAIMER: This file is based on the syslog-ng Open Source Edition documentation https://github.com/balabit/syslog-ng-ose-guides/commit/2f4a52ee61d1ea9ad27cb4f3168b95408fddfdf2 and is used under the terms of The syslog-ng Open Source Edition Documentation License. The file has been modified by Axoflow. -->

value-separator()
Synopsis: value-separator("")

Description: Specifies the character that separates the keys from the values. Default value: =.

For example, to parse key:value pairs, use kv-parser(value-separator(":"));.