AxoRouter Opens Windows

Axoflow’s mission has long been to help enterprises unlock the power of security data. This means simplifying the process of getting high quality data into SIEMs and to optimize security analytics and threat detection efforts. Of course, much of the landscape of many enterprises consists of large numbers of Windows machines – both servers and user workstations (or “endpoints”). While much of the telemetry from endpoints is now collected via EDR software and services, much of the server complement still uses traditional log collection techniques.

As with much of the OS, Windows log collection and viewing is proprietary. At a system level, the OS offers three main categories for logs: Application, System, and Security. These logs are stored in a proprietary format, and viewed with the included “Event Viewer”. Of course, over time many enterprises desired the ability to centrally collect these logs, and offload them to an external analytics engine or SIEM.

Main methods for collecting Windows logs

Two primary methods exist to centrally collect Windows logs

• Windows Event Forwarding/Windows Event Collector (WEF/WEC). This capability offers a centrally-configurable means to ship logs to a central collection point. It can be configured in either a push or pull setup, and can be centrally managed with a Group Policy Object.
• Agent-based collection. There are several options here, all made by third-party entities (both open-source and proprietary). Examples include WinLogBeats, the Splunk Universal Forwarder, and the OpenTelemetry (OTEL) Collector.

We will limit our discussion here to the traditional event log collection and offer two opinionated solutions for event log collection (and flat-file collection in the case of OTEL as well). The goal is to automate as much as possible, and require as little input as necessary to securely send event logs to analytics platforms at scale.

Axoflow can be configured very quickly either to act as a Windows Event Collector server, or as an OTEL listener for the OTEL Collector agent.

Windows Event Forwarding/Collection (WEF/WEC)

Windows Event Forwarding and Collection was created to meet the needs of centralized event log collection, with the goal of remaining primarily in the Windows world. In this scheme, individual windows servers or endpoints forward their logs (via WEF) to a central collection server (WEC). However, folks quickly realized that, once centralized on a Windows server, they wanted to forward all of these logs to an analytics tool of choice (which in many cases was not Windows-based). A market therefore emerged that sought to create a plug-compatible WEC service that was not itself based on Windows. OpenWEC is one such implementation that allows centralized, non-Windows collection of event logs using Windows Event Forwarding.

WEC itself is quite flexible; it has a very comprehensive XML-based configuration that allows the administrator to configure items such as:

• which event IDs are sent,
• noisy event suppression,
• batching, and many other parameters.

It adheres to the best practices of limiting what is sent over the wire, as close to the data source as possible. However, along with this flexibility comes complexity; substantial compute and disk resources can be needed to support a large-scale WEC implementation.

We will review the use of WEF/WEC for centralized Windows Event Log collection in the next blog in the series.

The OpenTelemetry Collector for Windows

The OpenTelemetry (OTEL) Collector is a significant leap forward in standardizing a single agent that is able to collect from many different locations, and operate a service in your infrastructure. OTEL is compatible with the newest of application architectures, and it can enhance the data collected with information such as cloud region/availability zones, pod information in a Kubernetes setting, etc. In the Windows case, the Otel Collector can offer a very simple way to send Windows Event logs to central locations. The OTEL Collector can also operate in “gateway” mode, in which it can also serve as the central aggregator. The Collector can also send to any other service that can accept OTEL traffic directly (such as AxoRouter).

Using the OpenTelemetry Collector for log collection is in its infancy, and much of the capability (especially for log collection – vs. metrics and traces) is still being developed. However, it is quickly gaining traction and fulfilling a need for open-source, agent-based Windows Event Log collection.

In many cases, the selection of agent or centralized Windows Event Collection is governed by what is currently in place, as there are many legacy Windows collection installations out there.

Receiving Windows logs with Axoflow

As mentioned before, AxoRouter can be configured very quickly for use with either Windows Event Collector server functionality, or as an OTEL listener for the OTEL Collector agent. Specific configurations for each method focus largely on the Windows side, as once the data is sent to AxoRouter, the processing of the datastream delivered to each selected destination is largely automatic. Here is an example of an AxoRouter instance configured for use for 3 main classes of data: OpenTelemetry, syslog, and Windows WEC:

In this blog series we will briefly review what is needed on the Windows side to connect to these services (connectors), and what the output looks like when it traverses a flow, which is described below. We will focus first on the OpenTelemetry (OTLP) connector (first in the list above), and the steps required to configure the pipeline. We’ll cover the WEC use case in a later blog post.

Configure OTEL Collector for Windows Data Collection

AxoRouter comes configured out of the box for use with OTEL, using the standard OTLP (OpenTelemetry Protocol) port 4317 by default. OpenTelemetry on Windows is quite simple to configure, and can be operated “by hand” on the command line for testing, and as a service in production. The OTEL Collector binary executable is configured at runtime with an attendant YAML configuration file, which in the simplest sense consists of three main sections:

• The receivers stanza. These “source” data from respective locations on the local machine.
• The exporters stanza. This instructs the collector where to send the data streams.
• Finally, the services stanza containing one or more pipelines. These connect the receivers to the exporters, and facilitate the actual data transfer. This is similar in concept and structure to how syslog-ng configures sources, destinations, and log paths. 

An example of this file is shown below, which highlights a simple configuration consisting of the three main event logs (application, system, and security) and a flat file (windowsupdate.log). Production use would likely include more options, particularly to configure certificates for secure data transfer.  A common option often used in large enterprises when configuring Windows receivers (which is included below) is

suppress_rendering_info: true

This suppresses the RenderingInfo block, which mostly contains message descriptions that can be looked up based on the Windows EventID (and therefore need not be sent with every message).

receivers:
  windowseventlog/application:
    channel: application
    raw: true
    suppress_rendering_info: true
  windowseventlog/system:
    channel: system
    raw: true
    suppress_rendering_info: true
  windowseventlog/security:
    channel: security
    raw: true
    suppress_rendering_info: true

  filelog/windowsupdatelog:
    include: ['c:\\windows\\windowsupdate.log']
    start_at: beginning

exporters:
  otlp/windowsupdatelog:
    endpoint: 10.37.129.12:4317
    tls:
      insecure: true
    headers:
      stream-name: windowsupdatelog

  otlp/application:
    endpoint: 10.37.129.12:4317
    tls:
      insecure: true
    headers:
      stream-name: application

  otlp/system:
    endpoint: 10.37.129.12:4317
    tls:
      insecure: true
    headers:
      stream-name: system

  otlp/security:
    endpoint: 10.37.129.12:4317
    tls:
      insecure: true
    headers:
      stream-name: security

service:
  pipelines:
    logs/windowsupdatelog:
      receivers: [filelog/windowsupdatelog]
      exporters: [otlp/windowsupdatelog]
    logs/application:
      receivers: [windowseventlog/application]
      exporters: [otlp/application]
    logs/system:
      receivers: [windowseventlog/system]
      exporters: [otlp/system]
    logs/security:
      receivers: [windowseventlog/security]
      exporters: [otlp/security]

You can start OTEL Collector on the command line (or optionally as a service) using this configuration file.

To see events in your destination analytics tool (like Splunk), configure a flow such as that shown below. These flows comprise the “here-to-there” part of the Axoflow platform, and tie the AxoRouter connectors shown above to a final destination at the end (Splunk, in this case). You can also see optional steps (such as data reduction/transformation steps) that can be included in the processing pipeline:

Once this flow is set up, run the following command on Windows to generate an arbitrary error event in the application Event Log:

eventcreate /t ERROR /id 100 /l application /d "BBBBBBBB"

Now you can see the default XML datastream in Splunk:

You’ll see that the data arrives in the native XML format; as this is the preferred raw format for the Splunk Technology Add-On (TA). However, if you’re not using the TA or otherwise would rather see the output in a more human-readable format, just enable the Transform XML to JSON processing step (see the screenshot above).

With this step, AxoRouter automatically extracts the fields of the message and sends them in a structured format to Splunk (or another supported SIEM). In Splunk, relevant fields are also added to the Interesting Fields list (on the left).

Wrap-up

We’ve covered the configuration of the OTel Collector as an agent for Windows to send logs to a central AxoRouter receiver. This is one of the many scenarios in which the OTel Collector is becoming increasingly popular. Regardless of where OTel is deployed, Axoflow automatically classifies and optimizes the received data streams for efficiency, content enrichment, and proper formatting for each selected destination.

In the next blog in the series, we will review the use of WEF/WEC for centralized Windows Event Log collection. This differs from OTel primarily in that the WEF/WEC protocol is much more Windows-centric and started out largely proprietary, and is also agentless. The setup is typically more involved on the Windows side, particularly in large enterprises (primarily due to certificates and authorization choices). Stay tuned for our next chapter on how to receive WEC via AxoRouter!