Did you know that most network appliances send improperly formatted log messages? Missing hostnames, invalid timestamps, formatting errors, timezone confusion – it’s all there in your logs, severely degrading the data quality of the security data sent to your SIEM (for example, Splunk or QRadar).

So how is it still working? Typically, your logging engineers fix these errors manually, writing parsing and rewriting rules (yes, usually regular expressions), or something similar that your standard low-level log-forwarding agent supports. But in a world of thousands of different log source formats and protocols, it’s a huge effort to fine-tune all your logging flows to fix the incoming messages and properly format them for your consumers, like your SIEM or other 3rd party tools. Therefore, switching to a new destination and having to reformat everything can easily become a huge and tedious task. In this post, we’d like to introduce you to AxoRouter (now in Early Access), specifically built to address these problems.

What AxoRouter is

AxoRouter powers Axoflow platform, our security data curation pipeline. AxoRouter a containerized solution (currently in Early Access) to collect all kinds of telemetry and security data. It has all the low-level functions you would expect of log-forwarding agents and aggregators, but it is much more. AxoRouter automatically identifies your log sources, and fixes common errors in the incoming data: for example,

• correcting missing hostnames,
• invalid timestamps,
• formatting errors, and so on.

Your engineers won’t have to spend time creating and maintaining rules or trying to fix processing bottlenecks.

AxoRouter has a range of zero-maintenance connectors for various networking and security products (for example, switches, firewalls, and web gateways), so it can classify the incoming data (by recognizing the product that is sending it), and apply various data curation and enrichment steps to reduce noise and improve data quality.

Before sending your data to its destination, AxoRouter automatically converts the data into a format that best suits the destination to optimize ingestion speed and data quality. For example, when sending data to Splunk, setting the proper sourcetype and index is essential.

In addition to curating and formatting your data, AxoRouter also collects detailed metrics about the processed data. These real-time metrics give you insight into the status of the telemetry pipeline and its components, providing end-to-end observability into your data flows.

How does AxoRouter fit into your infrastructure?

AxoRouter is versatile and can be deployed in various configurations to fit seamlessly into your infrastructure.

• It can sit as an aggregator right before your destinations, transforming, curating, and optimizing your data.
• It can be as close as possible to your appliances or other log sources as a collector. 
• If you need both the collector and the aggregator, they are interconnected with the high-performance OTLP protocol to reliably bridge large distances.

Let’s see how AxoRouter can help in these very different scenarios.

AxoRouter as the ultimate collector

There are two different methods for ingesting data into your pipeline: push and pull.

For a long time, most of the tools relied on the push method, and sent their logs somewhere immediately when they were generated: fire and forget was their motto. That means they wrote the logs into a file, or sent them out through the network. This was a convenient way to extract information without interrupting the “business logic” of the appliance. Many carrier-grade devices still only guarantee high throughput when using this mode. The reason is simple: they don’t want to spend resources on using a complex protocol, instead they rely on the most simple one: UDP. 

Modern log sources (like service logs) usually come from a message queue or a custom API via a pull method. As they are unique for the particular service, it is a bit trickier to collect them, because custom protocols can have a large footprint, making the collector agent heavier and more complex. AxoRouter can operate as a standalone collector for such systems. To keep it simple yet high performance, such components are part of AxoRouter, but can be separately enabled or disabled as needed.

AxoRouter is an excellent choice for receiving syslog traffic over UDP or TCP, but of course it supports other protocols as well.

When you are receiving traffic without guarantees, some of the data is inevitably lost because of packet loss, network or service outages, performance issues, and other causes. AxoRouter is optimized to handle incoming UDP and TCP syslog traffic, and either deliver it directly to its destination, or transfer it to an aggregator over a secure and reliable channel.

AxoRouter as an aggregator

AxoRouter is the heart and engine of the Axoflow security data curation pipeline, aggregating and transforming the data. When used as an aggregator, AxoRouter:

• Receives the incoming data from your appliances and other log collector agents.
• Fixes the common errors in the incoming data (for example, corrects missing hostnames, invalid timestamps, formatting errors, and so on). For details and examples, read the Fix the Syslog Mess: keep invalid syslog data from wrecking your SIEM blog post.
  
• Classifies, curates, and enriches the data to reduce noise and improve data quality.
• Filters and routes data to its intended destinations.
• Maps and converts data to a format optimized for the specific destination.

Sends metrics to Axoflow about the processed data, giving you insight into your pipeline.

Benefits of using AxoRouter

The benefits for your SIEM users and your SOC teams include the following.

• Instant reduction in the data volume sent to your SIEM, which leads to:
  • lower SIEM costs (for SIEMs billed based on ingested volume)
  • less data your team and SIEM have to analyze,
  • fewer false positives,
  • quicker queries.
• Reduced infrastructure footprint that has:
  • lower complexity, 
  • lower resource requirements, and is 
  • easier and cheaper to maintain.
• Metric-based insights and alerts that help you:
  • Identify bottlenecks (latency, buffering, …)
  • Identify data loss (UDP, TCP, etc…)

Zero-maintenance data connectors

Since Axoflow maintains and updates the classification patterns, you don’t need to write, test, and maintain your own patterns, which is difficult and time-consuming, and often the resulting patterns are not optimal and become resource hogs or bottlenecks. Our solution is faster and more efficient than other common solutions that are based on complex and thus fragile regular expressions.

If you’re interested in seeing Axoflow and AxoRouter in action request a demo.

AxoRouter and the Axoflow Platform

Although some of its features are focused on Splunk, AxoRouter supports sending data to multiple different destinations, including multiple SIEM solutions. The Axoflow Platform provides a complete, UI-based configuration management for your AxoRouter instances and absolute insight into your telemetry pipeline and your dataflows, based on the metrics it collects from AxoRouter and other log collectors like SC4S and syslog-ng (including its commercial version).

As you can see, with Axoflow you can modernize and improve your logging infrastructure. If you’re interested in seeing Axoflow and AxoRouter in action, request a demo.

Trademark attribution

syslog-ng™ is the trademark of One Identity LLC