We are pleased to announce that the newest Logging operator release (4.4.0) is available with several notable features. The new release is focused on multi-tenancy capabilities and extends the list of syslog-ng outputs with support for some of the most popular providers.
- Multi-tenancy with namespace-based routing
- New syslog-ng outputs
- Timeout-based configuration checks
- Buffer volume metrics
- New Fluentd 1.16 image
Multi-tenancy with namespace-based routing
Logging operator now supports namespace based routing for efficient aggregator-level multi-tenancy. Using the new LoggingRoute resource, an administrator can define a routing rule that allows for sending logs from one centrally managed Fluent Bit agent to multiple different Fluentd or syslog-ng aggregators owned by individual tenants. That way each tenant receives only the logs relevant to them. Previously this wasn’t possible, tenants had to define their own Fluent Bit agents and receive all logs from the cluster to be able to run an aggregator.
In addition to that, operations teams still have the ability to receive all logs from the system, for example, to archive them on a cold storage. The following diagram demonstrates this use case given two user-level and one administrator-level tenants:
For details, see the following links:
- An overview about multi-tenancy.
- More detailed information about the new LoggingRoute resource.
Note that Logging operator also supports nodegroup level isolation for hard multi-tenancy.
New syslog-ng outputs
When using syslog-ng as the log aggregator, you can now use the following new outputs:
Other new syslog-ng features
The HTTP output now supports the log-fifo-size, response-action, and timeout fields.
You can now use the metrics-probe()
parser of syslog-ng in syslogNGFLow and SyslogNGClusterFlow. For details, see MetricsProbe.
Forwarder logs
Fluent Bit now automatically excludes aggregator logs to avoid infinitely growing message loops. With this change, you can access Fluentd and syslog-ng logs simply by running:
kubectl logs <name-of-forwarder-pod>
Timeout-based configuration checks
Timeout-based configuration checks are different from the normal method: they start a Fluentd or syslog-ng instance without the dry-run
or syntax-check
flags, so output plugins or destination drivers actually try to establish connections and will fail if there are any issues, for example, with the credentials.
Buffer volume metrics updates
The buffer metrics are now available for both the Fluentd and the SyslogNG based aggregators. The sidecar configuration has been rewritten to add a new metric and improve performance by avoiding unnecessary cardinality. The name of the metric has been changed as well, but the original metric was kept in place to avoid breaking existing clients.
See the following two metrics output how the one is different from the old:
Old:
# HELP node_buffer_size_bytes Disk space used [deprecated]
# TYPE node_buffer_size_bytes gauge
node_buffer_size_bytes{entity="/buffers"} 32253
New:
# HELP logging_buffer_files File count
# TYPE logging_buffer_files gauge
logging_buffer_files{entity="/buffers",host="all-to-file-fluentd-0"} 2
# HELP logging_buffer_size_bytes Disk space used
# TYPE logging_buffer_size_bytes gauge
logging_buffer_size_bytes{entity="/buffers",host="all-to-file-fluentd-0"} 32253
Istio support
For jobs/individual pods that run to completion, Istio sidecar injection needs to be disabled, otherwise the affected pods would live forever with the running sidecar container. Configuration checkers and Fluentd drainer pods can be configured with the label sidecar.istio.io/inject
set to false.
spec:
configCheck:
labels:
sidecar.istio.io/inject: false
fluentd:
scaling:
drain:
labels:
sidecar.istio.io/inject: false
Other improvements
- You can now configure the resources of the buffer metrics sidecar.
- You can now rerun failed configuration checks by deleting the failed configcheck pod.
- The Fluentd ElasticSearch output now supports the composable index template format. To use it, set the
use_legacy_template
option to false. - The metrics for the syslog-ng forwarder are now exported using axosyslog-metrics-exporter.
Image updates
For the list of images used in Logging operator, see Images used by Logging operator.
Support for Fluentd image versions v1.14 and v1.15 is now discontinued because they are based on ruby 2.7 that has reached EOL.
The currently supported image is v1.15-ruby3 and build configuration for v1.15-staging is available for staging experimental changes.
Fluentd v1.16 is available for testing
A new image based on Fluentd v1.16 is already available to experiment with. To use this new image, change your Logging resource like this:
spec:
fluentd:
image:
tag: v1.16
Although this image is still experimental, please give it a try and let us know how it worked!
Summary
As you can see, Logging operator 4.4 brings you several useful features. You can find the Logging operator release on our GitHub page.
To download the latest Docker image:
docker pull ghcr.io/kube-logging/logging-operator:4.4.0
To install the latest release using Helm:
helm install logging-operator oci://ghcr.io/kube-logging/helm-charts/logging-operator --version=4.4.0
Give it a try!
Live Webinar
Parsing
sucks!
What can you do
about it?
30 October
10.00 PDT • 13.00 EDT • 19.00 CET
Balázs SCHEIDLER
Founder syslog-ng™
Mark BONSACK
Co-creator SC4S
Sándor GUBA
Founder Logging Operator
Neil BOYD
Moderator
Live Webinar
Parsing
sucks!
What can you do about it?
30 October
10.00 PDT • 13.00 EDT • 19.00 CET
Follow Our Progress!
We are excited to be realizing our vision above with a full Axoflow product suite.