This post shows you how Axoflow products (including our syslog-ng™ fork AxoSyslog) flag and display formatting errors in syslog messages.

Syslog sources still generate a significant amount (~30-50%) of the events that organizations send into their SIEM and analytics tools. Although syslog has been around for decades (and the standards long since codified), shockingly few applications and network devices send properly formatted log messages.

For example, they may fail to include the hostname of the sending device, have incorrect timestamps, or have formatting inconsistencies. Timezone discrepancies – or outright missing timestamps – are also common. Such errors can cause different kinds of problems:

• Log messages are often routed to different destinations based on the sender hostname. Missing or invalid hostnames mean that the message is not attributed to the right host, and often doesn’t arrive at its intended destination.
• Incorrect timestamp or timezone can hamper investigations during an incident, resulting in potentially critical data failing to show up (or extraneous data appearing) in queries for a particular period.
• They can even lead to memory leaks or resource overload in the processing software (or at a minimum unusable monitoring dashboards) when a sequence number or other rapidly varying field is mistakenly parsed as the hostname, program name, or other expected “low cardinality” field.
• Overall, they decrease the quality of security data you’re sending to your SIEM tools, which increases false positives, requires secondary data processing to clean, and increases query time – all of which ends up costing firms a lot more.

Logging engineers typically fix such problems and correct formatting errors manually using parsing and rewriting rules. AxoSyslog uses heuristics to fix several of these errors automatically. Starting with version 4.7, it also adds tags to messages that don’t have proper formatting. This allows your team to find erroneous messages more easily, fix the formatting of the offending data source, and/or notify the support team of the sending product.

AxoSyslog currently adds the following tags:

• message.utf8_sanitized: The message is not valid UTF-8.
• syslog.missing_timestamp: The message has no timestamp.
• syslog.invalid_hostname: The hostname field doesn’t seem to be valid, for example, it contains invalid characters.
• syslog.missing_pri: The priority (PRI) field is missing from the message.
• syslog.unexpected_framing: An octet count was found in front of the message, suggested invalid framing.
• syslog.rfc3164_missing_header: The date and the host are missing from the message – practically that’s the entire header of RFC3164-formatted messages.
• syslog.rfc5424_unquoted_sdata_value: The message contains an incorrectly quoted RFC5424 SDATA field.
• message.parse_error: Some other parsing error occurred.

You can see these tags by setting the stats(level()) option of AxoSyslog to 3, and running syslog-ng-ctl query list.

How Axoflow Helps

In addition to the heuristic fixes and tagging that AxoSyslog already performs, our products help you to detect, troubleshoot, and in many cases, automatically fix such issues.

AxoRouter recognizes several commonly used networking devices and automatically fixes errors specific to that device, for example, by extracting the relevant information from the body of the message.

The Axoflow Console allows you to quickly drill down to find log flows with issues, and to tap into the log flow and see samples of the specific messages that are processed, along with the related parsing information, like tags.

For a quick demo watch this video about using Log tapping to find rogue devices and parsing errors on the Axoflow Console.

Summary

Syslog errors significantly degrade the quality of the data feeding into your SIEM and analytics tools, leading to increased costs, missed incidents, and wasted effort on manual fixes.

AxoSyslog can automatically detect and tag common syslog message errors, enabling your team to quickly identify and address data issues at the source. With features like heuristic error correction and detailed tagging, AxoSyslog helps ensure that your log messages are correctly formatted, reducing false positives and improving the efficiency of your security operations.

Moreover, Axoflow’s suite of tools, including AxoRouter and the Axoflow Console, provides powerful capabilities for diagnosing and fixing log data issues. These tools automate much of the tedious work involved in log management, whether you are dealing with improperly formatted messages, missing data, or parsing errors.

Axoflow’s comprehensive approach helps you maintain the integrity and reliability of your security data, allowing for more accurate and timely incident response. For more information on how Axoflow can help improve your syslog management, watch our demo videos!

Legal notice: syslog-ng is a trademark of One Identity LLC.