Flow control and the Kafka client

A AxoSyslog destination recognizes a message as sent when the message has been sent to the Kafka client, not when the Kafka server confirms its delivery.

If the Kafka client collects too many unsent messages, it will not accept any more messages from AxoSyslog. The AxoSyslog application detects this and stops sending messages to the Kafka client. Also, AxoSyslog’s flow control starts functioning in the direction of the sources (for example, AxoSyslog will not read from the sources in that specific logpath).

You can specify a “high water mark” limit for the Kafka client with the queue.buffering.max.messages librdkafka property, which you can set in the config() option.

For more information about how the kafka-c() destination works with AxoSyslog, click here.