FilterX

What FilterX is, when to use it instead of classic filters, parsers, and rewrite rules, a minimal example, and a map of the FilterX pages.

FilterX is the AxoSyslog language to filter, parse, and rewrite log messages. It replaces filters , parsers , and rewrite rules with a single language that has typed variables, and handles nested data structures like JSON.

A FilterX block is a list of statements. A message passes the block only if every statement is true for the message. For example, you can select messages from a specific host, parse the message into fields, then change or delete some of the fields.

To try FilterX in a few minutes, see Get started with FilterX.

FilterX or classic filters, parsers, and rewrite rules?

Use FilterX for new configurations, and when you process structured data like JSON, key=value pairs, or OpenTelemetry records. FilterX is faster than the classic blocks, and handles multi-level, typed objects.

You don’t have to migrate existing configurations: the classic blocks keep working. You can use FilterX and classic blocks in the same log path, for example, a parser before a FilterX block. To convert existing filters and rewrite rules, see Update to FilterX.

Define a filterx block

You can define filterx blocks inline in your log statements. (If you want to reuse filterx blocks, Reuse FilterX blocks.)

For example, the following FilterX statement selects the messages that contain the word deny and come from the host example.

Terminal window
log {
    source(s1);
    filterx {
        ${HOST} == "example";
        ${MESSAGE} =~ "deny";
    };
    destination(d1);
};

You can use filterx blocks together with other blocks in a log path, for example, use a parser before/after the filterx block if needed.

What’s in this section

Operators and functions

FilterX has the following types of operators:

  • Comparison: ==, ===, <, and others.
  • Boolean: and, or, not.
  • Regular expression match: =~, !~.
  • Arithmetic and string: +, +=, .. (slicing).
  • Null handling: ??, =??.

For the complete list, see FilterX operator reference.

FilterX has built-in functions to parse, format, and convert data. Other functions handle strings, dates, timezones, hashes, and metrics. For the complete list, see FilterX function reference.

Next steps


Get started with FilterX

Build your first FilterX block: filter, parse a key=value message, set a field, and check the output in a file.

FilterX language basics

FilterX statements, truthy and falsy values, variable scope and types, assigning and deleting values, and complex types like dicts, lists, and tuples.

Reuse FilterX blocks

Define a FilterX block once with block filterx, then call it by name from the filterx statements of multiple log paths.

FilterX use cases and examples

Common FilterX tasks: set, delete, and rename message fields, conditional rewrites, and an iptables parser written in FilterX.

Comparing values in FilterX

Compare macros, templates, and variables as strings or numbers in FilterX, including strict equality and inequality that also check the type.

Boolean operators in FilterX

Combine FilterX filter expressions with the and, or, and not operators, and invert conditions correctly using De Morgan’s laws.

Conditional statements in FilterX

Build decision trees in FilterX with if, elif, and else statements, or with switch-case expressions that also support integer ranges.

String search in FilterX

Search strings with the includes(), startswith(), and endswith() FilterX functions, and replace substrings with str_replace() or regexp_subst().

IP addresses and subnets

Represent IPv4 and IPv6 addresses and subnets with the ip() and subnet() FilterX types, and filter messages by source network.

Parsing data in FilterX

Parse CEF, CSV, date, JSON, key=value, LEEF, XML, and Windows Event Log XML data into FilterX dicts, lists, and datetime values.

Format data

Format FilterX data as CEF, CSV, JSON, key=value pairs, LEEF, RFC5424 syslog, XML, or Windows Event Log XML strings.

Handle and fix timezones and timestamps

Fix, guess, set, and inspect message timezones with the fix_timezone(), guess_timezone(), set_timezone(), and get_timezone_source() functions.

Handle SDATA in RFC5424 log records

Filter RFC5424 messages on their structured data with the get_sdata(), has_sdata(), and is_sdata_from_enterprise() FilterX functions.

Handle OpenTelemetry log records

Route, modify, and create OpenTelemetry log records in FilterX, convert syslog messages to OTEL, and reference the otel_logrecord fields.

Create metrics with FilterX

Count processed messages and maintain labeled metric counters with the update_metric() FilterX function and the metrics_labels type.

Update to FilterX

Convert existing filter expressions, rewrite rules, and parsers in your configuration to equivalent FilterX statements and functions.

FilterX operator reference

Reference of FilterX operators: arithmetic, comparison, boolean, null coalescing, assign if non-null, list membership, regexp match, slicing, ternary.

FilterX function reference

Reference of all FilterX functions, from base64_decode() to vars(), with their arguments and usage examples.

Troubleshoot FilterX

Fix common FilterX problems, like dropped messages and missing fields, and track failing statements with the failure_info functions.