Collect metrics with Prometheus

Export AxoSyslog and syslog-ng metrics to Prometheus using the axosyslog-metrics-exporter and scrape them with Prometheus.

Prerequisites

• A running AxoSyslog instance
• stats(level(2)) or higher set in your configuration file
• File-level access to the AxoSyslog control socket

Deploy the metrics exporter

The axosyslog-metrics-exporter is a Go-based tool that exposes Prometheus-style metrics by connecting to the AxoSyslog control socket. It works with syslog-ng, syslog-ng Premium Edition, and all versions of AxoSyslog (syslog-ng™ is the trademark of One Identity LLC).

Run the exporter as a container:

sudo podman run -d -p 9577:9577 -v $(echo /var/*/syslog-ng/syslog-ng.ctl):/syslog-ng.ctl \
  ghcr.io/axoflow/axosyslog-metrics-exporter:latest --socket.path=/syslog-ng.ctl

Once started, the metrics endpoint is available at http://127.0.0.1:9577/metrics.

Configure Prometheus

Create a prometheus.yml file with a scrape job pointing to the metrics exporter:

scrape_configs:
  - job_name: axosyslog
    static_configs:
      - targets:
          - <prometheus-host-ip>:9577
        labels:
          app: axosyslog

Then run Prometheus:

sudo podman run \
    -p 9090:9090 \
    -v ./prometheus.yml:/etc/prometheus/prometheus.yml \
    prom/prometheus

To verify that Prometheus is scraping correctly, open the following pages in your browser:

• http://127.0.0.1:9090/config: shows the active configuration
• http://127.0.0.1:9090/targets: shows whether the AxoSyslog scrape target is up

Key metrics to monitor

For a detailed reference, see Metrics reference. The main metrics that you should monitor are the following.

Critical metrics

These metrics indicate problems that require immediate attention:

• output_unreachable: destination is unavailable
• socket_receive_dropped_packets_total: messages dropped on the source side
• output_events_total{result="dropped"}: messages dropped at the output without flow control
• socket_rejected_connections_total: number of rejected incoming connections

Core pipeline metrics

These metrics give you a basic understanding of pipeline throughput:

• input_events_total: total messages received by all sources
• output_events_total: total messages sent by all destinations
• filtered_events_total: total messages processed by filters
• parsed_events_total: total messages processed by parsers
• memory_queue_events and disk_queue_events: current buffer usage
• io_worker_latency_seconds: I/O worker latency, a sign of potential overload