Build your first FilterX block: filter, parse a key=value message, set a field, and check the output in a file.
FilterX
FilterX is the AxoSyslog language to filter, parse, and rewrite log messages. It replaces filters , parsers , and rewrite rules with a single language that has typed variables, and handles nested data structures like JSON.
A FilterX block is a list of statements. A message passes the block only if every statement is true for the message. For example, you can select messages from a specific host, parse the message into fields, then change or delete some of the fields.
To try FilterX in a few minutes, see Get started with FilterX.
FilterX or classic filters, parsers, and rewrite rules?
Use FilterX for new configurations, and when you process structured data like JSON, key=value pairs, or OpenTelemetry records. FilterX is faster than the classic blocks, and handles multi-level, typed objects.
You don’t have to migrate existing configurations: the classic blocks keep working. You can use FilterX and classic blocks in the same log path, for example, a parser before a FilterX block. To convert existing filters and rewrite rules, see Update to FilterX.
Define a filterx block
You can define filterx blocks inline in your log statements. (If you want to reuse filterx blocks, Reuse FilterX blocks.)
For example, the following FilterX statement selects the messages that contain the word deny and come from the host example.
log {
source(s1);
filterx {
${HOST} == "example";
${MESSAGE} =~ "deny";
};
destination(d1);
};You can use filterx blocks together with other blocks in a log path, for example, use a parser before/after the filterx block if needed.
What’s in this section
- FilterX language basics: statements, variables, types, and scope.
- FilterX use cases and examples: common tasks and a FilterX iptables parser.
- Comparing, boolean, conditional, and string search pages: how to write conditions.
- Parsing data in FilterX and Format data: parse and format CSV, key=value, JSON, XML, CEF, LEEF, and other formats.
- FilterX operator reference and FilterX function reference: the complete A–Z references.
Operators and functions
FilterX has the following types of operators:
- Comparison:
==,===,<, and others. - Boolean:
and,or,not. - Regular expression match:
=~,!~. - Arithmetic and string:
+,+=,..(slicing). - Null handling:
??,=??.
For the complete list, see FilterX operator reference.
FilterX has built-in functions to parse, format, and convert data. Other functions handle strings, dates, timezones, hashes, and metrics. For the complete list, see FilterX function reference.
Next steps
- Get started with FilterX
- FilterX language basics
- FilterX function reference
- FilterX operator reference
- Update to FilterX
FilterX statements, truthy and falsy values, variable scope and types, assigning and deleting values, and complex types like dicts, lists, and tuples.
Define a FilterX block once with block filterx, then call it by name from the filterx statements of multiple log paths.
Common FilterX tasks: set, delete, and rename message fields, conditional rewrites, and an iptables parser written in FilterX.
Compare macros, templates, and variables as strings or numbers in FilterX, including strict equality and inequality that also check the type.
Combine FilterX filter expressions with the and, or, and not operators, and invert conditions correctly using De Morgan’s laws.
Build decision trees in FilterX with if, elif, and else statements, or with switch-case expressions that also support integer ranges.
Search strings with the includes(), startswith(), and endswith() FilterX functions, and replace substrings with str_replace() or regexp_subst().
Represent IPv4 and IPv6 addresses and subnets with the ip() and subnet() FilterX types, and filter messages by source network.
Parse CEF, CSV, date, JSON, key=value, LEEF, XML, and Windows Event Log XML data into FilterX dicts, lists, and datetime values.
Format FilterX data as CEF, CSV, JSON, key=value pairs, LEEF, RFC5424 syslog, XML, or Windows Event Log XML strings.
Fix, guess, set, and inspect message timezones with the fix_timezone(), guess_timezone(), set_timezone(), and get_timezone_source() functions.
Filter RFC5424 messages on their structured data with the get_sdata(), has_sdata(), and is_sdata_from_enterprise() FilterX functions.
Route, modify, and create OpenTelemetry log records in FilterX, convert syslog messages to OTEL, and reference the otel_logrecord fields.
Count processed messages and maintain labeled metric counters with the update_metric() FilterX function and the metrics_labels type.
Convert existing filter expressions, rewrite rules, and parsers in your configuration to equivalent FilterX statements and functions.
Reference of FilterX operators: arithmetic, comparison, boolean, null coalescing, assign if non-null, list membership, regexp match, slicing, ternary.
Reference of all FilterX functions, from base64_decode() to vars(), with their arguments and usage examples.
Fix common FilterX problems, like dropped messages and missing fields, and track failing statements with the failure_info functions.