# Parsing syslog messages

By default, AxoSyslog parses every message using the `syslog-parser` as a syslog message, and fills the macros with values of the message. The `syslog-parser` does not discard messages: the message cannot be parsed as a syslog message, the entire message (including its header) is stored in the `${MSG}` macro. If you do not want to parse the message as a syslog message, use the `flags(no-parse)` option of the source.

You can also use the `syslog-parser` to explicitly parse a message, or a part of a message as a syslog message (for example, after rewriting the beginning of a message that does not comply with the syslog standards).

## Example: Using junctions

For example, suppose that you have a single network source that receives log messages from different devices, and some devices send messages that are not RFC-compliant (some routers are notorious for that). To solve this problem in earlier versions of AxoSyslog, you had to create two different network sources using different IP addresses or ports: one that received the RFC-compliant messages, and one that received the improperly formatted messages (for example, using the `flags(no-parse)` option). Using junctions this becomes much more simple: you can use a single network source to receive every message, then use a junction and two channels. The first channel processes the RFC-compliant messages, the second everything else. At the end, every message is stored in a single file. The filters used in the example can be `host()` filters (if you have a list of the IP addresses of the devices sending non-compliant messages), but that depends on your environment.
```
 
    log {
        source {
            syslog(
                ip(10.1.2.3)
                transport("tcp")
                flags(no-parse)
            );
        };
        junction {
            channel {
                filter(f_compliant_hosts);
                parser {
                    syslog-parser();
                };
            };
            channel {
                filter(f_noncompliant_hosts);
            };
        };
        destination {
            file("/var/log/messages");
        };
    };
    
```

Since every channel receives every message that reaches the junction, use the `flags(final)` option in the channels to avoid the unnecessary processing the messages multiple times:
```
 
    log {
        source {
            syslog(
                ip(10.1.2.3)
                transport("tcp")
                flags(no-parse)
            );
        };
        junction {
            channel {
                filter(f_compliant_hosts);
                parser {
                    syslog-parser();
                };
                flags(final);
            };
            channel {
                filter(f_noncompliant_hosts);
                flags(final);
            };
        };
        destination {
            file("/var/log/messages");
        };
    };
    
```

Note AxoSyslog has several parsers that you can use to parse non-compliant messages. You can even [write a custom parser in Python](../../docs/axosyslog-core/chapter-parsers/python-parser/index.md). For details, see [parser: Parse and segment structured messages](../../docs/axosyslog-core/chapter-parsers/index.md). 

Note that by default, the `syslog-parser` attempts to parse the message as an RFC3164-formatted (BSD-syslog) message. To parse the message as an RFC5424-formatted message, use the `flags(syslog-protocol)` option in the parser.
```
 
       syslog-parser(flags(syslog-protocol));
    
```

## Parsing errors

AxoSyslog 4.7 and newer automatically adds the following tags if it encounters errors when parsing syslog messages.

  * `message.utf8_sanitized`: The message is not valid UTF-8.
  * `syslog.missing_timestamp`: The message has no timestamp.
  * `syslog.invalid_hostname`: The hostname field doesn’t seem to be valid, for example, it contains invalid characters. For details on the valid characters, see the [`check-hostname()` global option](../../docs/axosyslog-core/chapter-global-options/reference-options/index.md#global-option-check-hostname).
  * `syslog.missing_pri`: The priority (PRI) field is missing from the message.
  * `syslog.unexpected_framing`: An octet count was found in front of the message, suggested invalid framing.
  * `syslog.rfc3164_missing_header`: The date and the host are missing from an RFC3164-formatted message - practically that’s the entire header of RFC3164-formatted messages.
  * `syslog.rfc5424_unquoted_sdata_value`: An RFC5424 message contains an incorrectly quoted SDATA field.
  * `message.parse_error`: Some other parsing error occurred.



* * *

[Options of syslog-parser() parsers](../../docs/axosyslog-core/chapter-parsers/parser-syslog/parser-syslog-options/index.md)

Last modified April 19, 2024: [[4.7] Documents parser error tags (446cd3e)](<https://github.com/axoflow/axosyslog-core-docs/commit/446cd3e735c7eed2ac8a6340a66af76345f1438c>)