Message representation in AxoSylog
When the AxoSyslog application receives a message, it automatically parses the message. The AxoSyslog application can automatically parse log messages that conform to the RFC3164 (BSD or legacy-syslog) or the RFC5424 (IETF-syslog) message formats. If AxoSyslog cannot parse a message, it results in an error.
Note
In case you need to relay messages that cannot be parsed without any modifications or changes, use the flags(no-parse) option in the source definition, and a template containing only the ${MESSAGE} macro in the destination definition.
To parse non-syslog messages, for example, JSON, CSV, or other messages, you can use the built-in parsers of AxoSyslog. For details, see parser: Parse and segment structured messages.
A parsed syslog message has the following parts:
-
TimestampsTwo timestamps are associated with every message: one is the timestamp contained within the message (that is, when the sender sent the message), the other is the time when AxoSyslog has actually received the message.
-
SeverityThe severity of the message.
-
FacilityThe facility that sent the message.
-
TagsCustom text labels added to the message that are mainly used for filtering. None of the current message transport protocols adds tags to the log messages. Tags can be added to the log message only within AxoSyslog. The AxoSyslog application automatically adds the id of the source as a tag to the incoming messages. Other tags can be added to the message by the pattern database, or using the
tags()option of the source. -
IP address of the senderThe IP address of the host that sent the message. Note that the IP address of the sender is a hard macro and cannot be modified within AxoSyslog but the associated hostname can be modified, for example, using rewrite rules.
-
Hard macrosHard macros contain data that is directly derived from the log message, for example, the ${MONTH} macro derives its value from the timestamp. The most important consideration with hard macros is that they are read-only, meaning they cannot be modified using rewrite rules or other means.
-
Soft macrosSoft macros (sometimes also called name-value pairs) are either built-in macros automatically generated from the log message (for example,
${HOST}), or custom user-created macros generated by using the AxoSyslog pattern database or a CSV-parser. The SDATA fields of RFC5424-formatted log messages become soft macros as well. In contrast with hard macros, soft macros are writable and can be modified within AxoSyslog, for example, using rewrite rules.Note
It is also possible to set the value of built-in soft macros using parsers, for example, to set the
${HOST}macro from the message using a column of a CSV-parser.The data extracted from the log messages using named pattern parsers in the pattern database are also soft macros.Note
For the list of hard and soft macros, see Hard versus soft macros.
Message size and encoding
Internally, AxoSyslog represents every message as UTF-8. The maximal length of the log messages is limited by the log-msg-size() option: if a message is longer than this value, AxoSyslog truncates the message at the location it reaches the log-msg-size() value, and discards the rest of the message.
When encoding is set in a source (using the encoding() option) and the message is longer (in bytes) than log-msg-size() in UTF-8 representation, AxoSyslog splits the message at an undefined location (because the conversion between different encodings is not trivial).