Parsing syslog messages

By default, AxoSyslog parses every message using the syslog-parser as a syslog message, and fills the macros with values of the message. The syslog-parser does not discard messages: the message cannot be parsed as a syslog message, the entire message (including its header) is stored in the ${MSG} macro. If you do not want to parse the message as a syslog message, use the flags(no-parse) option of the source.

You can also use the syslog-parser to explicitly parse a message, or a part of a message as a syslog message (for example, after rewriting the beginning of a message that does not comply with the syslog standards).

Example: Using junctions

For example, suppose that you have a single network source that receives log messages from different devices, and some devices send messages that are not RFC-compliant (some routers are notorious for that). To solve this problem in earlier versions of AxoSyslog, you had to create two different network sources using different IP addresses or ports: one that received the RFC-compliant messages, and one that received the improperly formatted messages (for example, using the flags(no-parse) option). Using junctions this becomes much more simple: you can use a single network source to receive every message, then use a junction and two channels. The first channel processes the RFC-compliant messages, the second everything else. At the end, every message is stored in a single file. The filters used in the example can be host() filters (if you have a list of the IP addresses of the devices sending non-compliant messages), but that depends on your environment.

   log {
        source {
            syslog(
                ip(10.1.2.3)
                transport("tcp")
                flags(no-parse)
            );
        };
        junction {
            channel {
                filter(f_compliant_hosts);
                parser {
                    syslog-parser();
                };
            };
            channel {
                filter(f_noncompliant_hosts);
            };
        };
        destination {
            file("/var/log/messages");
        };
    };

Since every channel receives every message that reaches the junction, use the flags(final) option in the channels to avoid the unnecessary processing the messages multiple times:

   log {
        source {
            syslog(
                ip(10.1.2.3)
                transport("tcp")
                flags(no-parse)
            );
        };
        junction {
            channel {
                filter(f_compliant_hosts);
                parser {
                    syslog-parser();
                };
                flags(final);
            };
            channel {
                filter(f_noncompliant_hosts);
                flags(final);
            };
        };
        destination {
            file("/var/log/messages");
        };
    };

Note that by default, the syslog-parser attempts to parse the message as an RFC3164-formatted (BSD-syslog) message. To parse the message as an RFC5424-formatted message, use the flags(syslog-protocol) option in the parser.

   syslog-parser(flags(syslog-protocol));

Parsing errors

AxoSyslog 4.7 and newer automatically adds the following tags if it encounters errors when parsing syslog messages.

• message.utf8_sanitized
• message.parse_error
• syslog.missing_pri
• syslog.missing_timestamp
• syslog.invalid_hostname
• syslog.unexpected_framing
• syslog.rfc3164_missing_header
• syslog.rfc5424_unquoted_sdata_value