AxoSyslog 4.23–4.28: Splunk HEC, Elastic Bulk & Arrow Flight

AxoSyslog 4.28 can now receive data directly from Splunk HEC clients, Elastic Agent, and Beats, so you can filter, enrich, and route that data before it reaches your SIEM. Releases 4.23-4.28 also add an Apache Arrow Flight destination, cut memory queue usage by up to 50%, and introduce an experimental compiler for FilterX, AxoSyslog's typed language for parsing, filtering, and transforming log data. AxoSyslog remains a binary-compatible drop-in replacement for syslog-ng (see why you might choose AxoSyslog over syslog-ng).

Below, we walk through the highlights grouped by topic. The version that introduced a feature is shown in parentheses. For earlier releases, see what's new in AxoSyslog 4.18–4.22 and 4.13–4.17.

Send Splunk HEC, Elastic Agent, and Beats Data to AxoSyslog

AxoSyslog 4.28 introduces a native HTTP server framework with HTTP/1.1 keep-alive and pipelining support, and three sources built on top of it. All of them decompress gzip and deflate requests automatically based on their Content-Encoding header, and support the usual network and TLS options.

Splunk HTTP Event Collector (HEC) Source

The splunk-hec() source receives events from Splunk HTTP Event Collector clients. Forwarders and applications that already log to Splunk can send their events to AxoSyslog by changing only the HEC URL, so you can filter, enrich, and route the data before it reaches Splunk (or anything else).

The event field becomes $MESSAGE, host and time set $HOST and the timestamp, and the whole HEC envelope is available under ${.splunk.*}. If the body of a message isn't JSON, AxoSyslog parses it as syslog.

source s_hec {
  splunk-hec(
    token("a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d")
    transport(tls)
    tls(key-file("/etc/syslog-ng/hec.key") cert-file("/etc/syslog-ng/hec.crt"))
  );
};

Elasticsearch Bulk API Source

Elastic Agent, Beats, and other clients of the Elasticsearch Bulk API can send their events to AxoSyslog by pointing their Elasticsearch output at the elasticsearch-bulk() source. The document of each index and create action becomes a log message, and its action line is stored in ${.es_bulk.action}. The source answers the version and license probes of the Elastic clients, and the version() option sets the Elasticsearch version reported to them.

source s_es {
  elasticsearch-bulk(
    port(9200)
    auth-token("ApiKey <key>")
    version("8.15.0")
  );
};

Native HTTP/HTTPS Log Source

The experimental ehttp() source receives log messages over HTTP and HTTPS. Unlike the Python-based webhook() source, it's implemented natively, and it's intended to replace webhook() in the long run. Once its options are stable, it will be renamed to http().

The mode() option controls how the request body is split into messages: single, line-separated/jsonl, json (a JSON array or concatenated JSON objects), or auto, which picks json when the body parses as JSON and line-separated otherwise. You can require an Authorization header with auth-token(), customize the response with response-body(), and limit the request size with max-request-size().

source s_http {
  ehttp(
    port(8080)
    mode("json")
    auth-token("Bearer s3cr3t")
    response-body('{"status": "received"}')
    flags(no-parse)
  );
};

PROXY Protocol over UDP and OpenTelemetry Source Updates

• PROXY protocol over UDP (4.25). The syslog() and network() sources support HAProxy PROXY protocol v2 over UDP. With transport(proxied-udp), the original client address and port behind the load balancer are available as ${SOURCEIP}, ${SOURCEPORT}, ${DESTIP} and ${DESTPORT}.
• OpenTelemetry source. The opentelemetry() source gained an ip() option to set the bind address (4.23), and it sets the .tls.x509_cn, .tls.x509_o and .tls.x509_ou name-value pairs from the client certificate, the same way the network() and syslog() sources do (4.28). Its new mode() option is covered in the performance section below. If you're new to OpenTelemetry as a log transport, see why use OpenTelemetry gRPC for log data transport.

Apache Arrow Flight Destination

Version 4.26 adds the arrow-flight() destination that sends structured, columnar data to Apache Arrow Flight servers over gRPC. The path() descriptor is templatable, so a single destination can route to many tables, and the schema() option maps each column to a template with an explicit type (STRING, INT64, DOUBLE, BOOL, TIMESTAMP, or MAP(STRING, STRING)).

destination d_arrow {
  arrow-flight(
    url("grpc://flight.example.com:8815")
    path("events.${HOST}")
    schema(
      "ts"       TIMESTAMP => "$UNIXTIME"
      "host"     STRING    => "$HOST"
      "program"  STRING    => "$PROGRAM"
      "severity" INT64     => "$LEVEL_NUM"
      "msg"      STRING    => "$MSG"
    )
    batch-lines(1000)
    batch-bytes(1048576)
    batch-timeout(5000)
    workers(4)
    worker-partition-key("${HOST}")
  );
};

In 4.27, the keep-alive() option replaced the timeout() option of arrow-flight().

HTTP Delivery Improvements

Retry OpenObserve and Splunk Partial Failures

Some servers report errors in the response body instead of the HTTP status code. The response-adapter() option of the http() and openobserve() destinations (4.27) finds these errors, so the affected batch can be retried (up to the number of retries()). This closes a gap in your message delivery guarantees: without it, a partially rejected batch counts as delivered.

• response-adapter(openobserve) turns OpenObserve's partial failures, which come back as HTTP 200, into real errors.
• The splunk-hec-event() destination enables response-adapter(splunk) by default: when Splunk rejects an event of a batch, the offending message is logged and the batch is retried.

Better Batching and Compression

• batch-idle-timeout() (4.24). Complements batch-timeout(): while batch-timeout() counts from the first message of the batch, batch-idle-timeout() measures the time since the last message was added. Whichever expires first sends the batch. It's available in http() and most other batching destinations.
• force-content-compression() (4.23). By default, the http() destination sends a payload uncompressed if compression doesn't make it smaller. If your server accepts only compressed data, set force-content-compression(yes).‍‍

destination d_openobserve {
  http(
    url("https://openobserve.example.com/api/default/logs/_json")
    response-adapter(openobserve)
    batch-lines(1000)
    batch-timeout(5000)
    batch-idle-timeout(500)
  );
};

For background on how batching interacts with queues and flow control, see AxoSyslog internals: flow control, window size, queues, and batching.

Other Destination Updates

• The network-load-balancer() configuration generator supports failover (4.24): it generates the list of failover servers for each destination automatically.
• The usertty() destination gained an escaping() option (4.26).

Performance Improvements

Performance got attention in every release of this range:

• Memory queues use up to 50% less memory (4.23). Holding 1M messages in memory now takes about 8 GB instead of 15 GB of RSS.
• Experimental FilterX compiler (4.26). The new FilterX compiler compiles blocks instead of interpreting them. Enable it with the filterx-jit(yes) global option. Only a subset of expressions is compiled for now, so the gains are modest at this stage — but the foundation is in place.
• OpenTelemetry without the detour (4.28). With the new mode(filterx-dict) option, the opentelemetry() source converts incoming log records directly into resource, scope and log FilterX dictionaries, skipping the serialization and deserialization of the ${.otel_raw.*} name-value pairs. This makes processing OpenTelemetry logs in FilterX significantly faster. (For more OTLP tuning tips, see Maximizing OpenTelemetry transport performance.)
• Faster FilterX and source processing (4.24–4.27). JSON parsing in FilterX is now powered by jsmn (4.24), and a series of optimizations sped up the FilterX runtime and source-side processing.
• Coalesced writes in network() and tcp() destinations (4.27). These destinations now honor flush-lines() and write up to that many messages in a single system call over stream and TLS transports. With @config: 4.26 or newer, flush-lines() defaults to 100.
• Faster spoof-source() (4.27). Spoofed packets of the network() and syslog() destinations are now sent from a destination thread instead of the source thread, and without a per-destination lock, which improves scalability.
• Smarter parallelize() batching (4.26). parallelize() avoids batches that are too large, and its default batch-size() is now 100.
• gRPC arenas (4.25). The opentelemetry(), axosyslog-otlp(), loki(), google-pubsub(), clickhouse() and bigquery() destinations allocate memory using gRPC arenas.
• Faster reloads with disk-buffer() (4.25). disk-buffer() queues are kept alive during reload, which can speed up reloads considerably.
• Monolithic build (4.26). If you build AxoSyslog yourself, the --with-linking-mode=monolithic configure option produces a single binary with all modules statically linked, which generally improves performance.

FilterX: New Types and Functions

IP Addresses and Subnets

The subnet() and ip() types (4.25) represent an IPv4/IPv6 subnet in CIDR notation, or a single IP address. Combined with the in operator, checking whether an address belongs to a network becomes a one-liner, which is handy when you parse firewall logs with FilterX:

a = subnet("192.168.0.0/24");

"192.168.0.5" in a;
ip("192.168.0.11") in a;

a6 = subnet("DEAD:BEEF::1/64");
"DEAD:BEEF::2" in a6;

Tuples

The tuple type (4.27) is a read-only, list-like value patterned after the Python type of the same name: you initialize it once, and it stays immutable. Version 4.28 adds Python-like tuple unpacking in assignments:‍

t = ();          # empty tuple
t = ("foo",);    # singleton
t = (1, 2, 3);   # a tuple of 3 elements

(a, b, c) = (1, 2, 3);

‍Hashes, Encodings, and UTF-8

Version 4.25 added a set of functions for common data transformations:

• md5(), sha1(), sha256(), sha512() return the hex digest of a string or bytes value, and the generic digest() function accepts an algorithm name and returns the raw hash as bytes.
• base64_encode()/base64_decode(), urlencode()/urldecode() and hex_encode()/hex_decode() cover the common encodings.
• utf8_validate() checks whether a string is valid UTF-8, and utf8_sanitize() replaces invalid byte sequences with their \xNN escaped representation.

Timestamps and Timezones

A set of timezone functions (4.24) helps you deal with timestamps that lack timezone information. fix_timezone() and set_timezone() adjust a datetime, guess_timezone() infers the timezone, and get_timezone_source() tells you whether the timezone was parsed, assumed, fixed or guessed:‍

timestamp = strptime("2000-01-01T00:00:00", "%Y-%m-%dT%H:%M:%S");
if (get_timezone_source(timestamp) === "assumed") {
  timestamp = fix_timezone(timestamp, "CET");
};

‍In addition, format_isodate() (4.23) formats datetimes as ISO 8601 dates.

Identifiers and Matching

• uuid() (4.25) generates a random UUID v4, and uuid7() (4.27) generates RFC 9562 UUIDv7 identifiers that embed a millisecond-precision timestamp, so they sort by creation time. uuid4() is an alias of uuid().
• glob_match() (4.25) matches a filename against one or more glob patterns, for example: glob_match(filename, ["*.zip", "*.7z"]).

Enrichment, Formatting, and Metrics

• cache_json_file() (4.25) accepts a default_value parameter, which is used when the file is missing or cannot be loaded — instead of failing with a configuration error.
• format_kv() (4.28) gained the quote_char and always_quote options to control how values are quoted.
• update_metric() (4.28) gained a set parameter that assigns an absolute value instead of incrementing, so you can use it for gauge-like metrics:‍

update_metric("demo_gauge", set=int($MSG), labels={"host": "demo"});

Language Improvements

• Ranged switch cases (4.26). The switch statement supports ranges for integer targets, for example case 1..4:.
• move() (4.23). The move() function tells FilterX that a variable can be moved to its new location instead of being copied. It's like unset(), but more explicit, and it returns the moved value.
• Concise error messages (4.27). Instead of multiple layers of errors for a single statement, FilterX now logs one precise entry that points to the statement and explains what failed (to route failed messages instead of just logging them, see error tagging in AxoSyslog):

FILTERX ERROR; err_idx='[1/1]', expr='syslog-ng.conf:12:4|	d[string($PID)]', error='Variable is unset: "$PID"'

TLS and Security

Trust Certificates by Fingerprint

The trusted-fingerprints() TLS option (4.26) lets you trust X.509 certificates based on their fingerprints, and deprecates trusted-keys(). Two major differences: a certificate listed in trusted-fingerprints() is accepted even if the normal X.509 validation fails, and you're no longer limited to SHA-1 — any digest algorithm supported by OpenSSL works. The algorithm is the word before the first colon:‍

tls(
  ...
  trusted-fingerprints(
    "SHA256:0C:EF:34:4D:0B:74:AE:03:72:9A:4E:68:AF:90:59:A9:EF:35:1F:AA:1B:2C:3D:4E:5F:60:71:82:93:A4:B5:C6"
  )
)

‍To get the fingerprint of a certificate, run: openssl x509 -sha256 -in <certificate.pem> -fingerprint

Extended Key Usage Verification

Since 4.23, the network() and syslog() sources and destinations support the extended-key-usage-verify(yes) option to check that the peer's certificate has the appropriate Extended Key Usage: Client Auth for clients, Server Auth for servers.

Stronger Secure Logging

Starting with 4.27, the secure logging key derivation uses an AES-CMAC key expansion over the full input, following NIST SP 800-108r1, and key files with a mismatching CMAC are rejected. Note that this changes every derived key — see the upgrade notes below.

Observability and Operations

• Latency histograms (4.24). The new syslogng_event_processing_latency_seconds and syslogng_output_event_latency_seconds metrics show how long messages take from receipt to processing and to delivery.
• More counters. syslogng_input_transport_errors_total (4.25) reports framing and TLS handshake errors; syslogng_output_batch_timedout_total and syslogng_input_window_full_total (4.23) help you spot batching and flow-control issues; and parallelize() got its own failure counter (4.23) and batch-size histograms (4.26). See the metrics reference for details, and collect AxoSyslog metrics into Prometheus to chart them.
• Discover metrics (4.24). syslog-ng --metrics-registry lists the supported metrics.
• Troubleshoot http() requests (4.23, 4.25). Failed requests are logged at debug and notice level, and error reporting for batched sending is more detailed.
• Interactive debugger (4.25). syslog-ng --interactive gained the step, continue, follow and trace commands. Combined with log tapping, it makes debugging a pipeline much easier.
• Forced shutdown (4.28). syslog-ng-ctl stop --force terminates the process without waiting for the worker threads — for the rare case when a shutdown never completes. In-memory queue contents are lost, disk-buffer contents stay on disk.
• New platforms. Packages are available for Ubuntu 26.04, Fedora 44, and RHEL/AlmaLinux 10 (4.26), and the sql() destination now ships in the AlmaLinux 9 packages (4.28).

Notes for Upgrading

A few changes in this range are intentionally incompatible, so check them before upgrading:

• disk-buffer() format v27 (4.24). The disk-buffer serialization format was upgraded, so buffer files written by 4.24 or newer cannot be read by older versions. Keep this in mind if you might need to downgrade. (For how disk buffers fit into a resilient architecture, see disk buffering for a resilient syslog architecture.)
• Renamed latency metric (4.24). syslogng_output_event_delay_sample_seconds was removed in favor of syslogng_output_event_latency_seconds. Update your dashboards and alerts.
• Line endings and empty datagrams (4.25). CR (\r) characters are removed from line endings, and empty UDP datagrams are dropped.
• repr() is now distinct from string() (4.26). repr() includes type hints, similar to Python. If you used repr() directly in an output, cast to string() instead. (In 4.23, the repr() of datetimes already switched to microsecond resolution.)
• Cross-type comparisons in FilterX (4.27). When values of different types are compared as strings, FilterX now uses the result of the string() cast. This changes how null() and datetime() objects compare against strings.
• flush-lines() in network() and tcp() destinations (4.27). With @config: 4.26 or newer, writes are coalesced by default (flush-lines(100)). Older configuration versions keep the one-write-per-message behavior.
• secure-logging keys (4.27). Because of the new key derivation, keys and logs produced by earlier versions cannot be verified by 4.27 or newer.
• arrow-flight() timeout() (4.27). Deprecated, use keep-alive() instead.
• Debian 11 (4.28). Debian 11 (bullseye) packages are no longer provided.

Why These Releases Matter

Across versions 4.23-4.28, AxoSyslog became a drop-in endpoint for Splunk and Elastic agents, gained a columnar path into Arrow-based analytics, and got substantially faster and leaner, while FilterX grew the types and functions you need to normalize, enrich, and route security data with less code.

Download AxoSyslog 4.28

You can download AxoSyslog from the following sources:

• Packages are available for Debian and Ubuntu from our APT repository.
• RPM packages are available for Fedora, Red Hat, and similar distributions from our RPM repository. See our blog post for details on installing AxoSyslog on RHEL, AlmaLinux, or Fedora!
• We also provide cloud-ready container images and Helm charts.
• AxoSyslog is a binary-compatible drop-in replacement for syslog-ng, up to version 4.7.1 (and possibly newer). To upgrade an existing syslog-ng deployment, see How to upgrade syslog-ng to AxoSyslog.

For the full list of changes and bugfixes, see the AxoSyslog release notes on GitHub.

Frequently Asked Questions

Can AxoSyslog receive data from Splunk HEC clients?

Yes. Starting with version 4.28, the splunk-hec() source accepts events from any client that sends data to the Splunk HTTP Event Collector API. Point the client's HEC URL at AxoSyslog (port 8088 by default) and set the same token in the token() option.

How do I send Elastic Agent or Beats data to AxoSyslog?

Use the elasticsearch-bulk() source, available since 4.28, and point the Elasticsearch output of your Elastic Agent or Beats at AxoSyslog. Elastic clients refuse to connect to a server that reports an older version than their own, so set the version() option to match your clients if needed.

Is AxoSyslog compatible with syslog-ng configurations?

Yes. AxoSyslog is a binary-compatible drop-in replacement for syslog-ng, up to version 4.7.1 (and possibly newer), so your existing configuration keeps working. For the steps, see How to upgrade syslog-ng to AxoSyslog, and for how the two projects have diverged, see the AxoSyslog year 2 comparison.

What should I check before upgrading to AxoSyslog 4.28?

Review the Notes for Upgrading section above. The most important changes are the new disk-buffer file format (older versions can't read it), the renamed latency metric, and the new secure-logging key derivation.

Trademark attribution

syslog-ng™ is the trademark of One Identity LLC

‍