kafka: Publish messages to Apache Kafka

The kafka() destination publishes log messages to the Apache Kafka message bus, where subscribers can access them.

kafka() is a configuration snippet of the AxoSyslog Configuration Library (SCL) that calls the kafka-c() destination and passes on every option unchanged. The two names behave identically, use whichever you prefer. You can find the source of this configuration snippet on GitHub.

Prerequisites

• Install the axosyslog-scl package on Debian/Ubuntu. On RHEL and compatible distributions, the SCL files are part of the axosyslog base package.

  Your configuration must also contain @include "scl.conf", which the default configuration file already does. If the SCL files are missing, AxoSyslog fails to start with an unexpected LL_IDENTIFIER error.

  @include "scl.conf"

• This feature requires a separate module. Install the axosyslog-mod-rdkafka package on Debian/Ubuntu, or the axosyslog-kafka package on RHEL and compatible distributions. If the module isn’t installed, AxoSyslog fails to start with an unexpected LL_IDENTIFIER error.

Declaration:

@include "scl.conf"

kafka(
    bootstrap-servers("1.2.3.4:9092,192.168.0.2:9092")
    topic("${HOST}")
);

Example: Sending log data to Apache Kafka

The following example defines a kafka destination, using only the required options.

@include "scl.conf"

destination d_kafka {
  kafka(
    bootstrap-servers("1.2.3.4:9092,192.168.0.2:9092")
    topic("${HOST}")
  );
};

Options

The kafka() destination accepts the options of the kafka-c() destination. For the list of options, see Options of the kafka() destination’s C implementation.