Send logs, metrics, and traces to OpenTelemetry

Starting with version 4.3.0, AxoSyslog can send logs, metrics, and traces to OpenTelemetry over the OpenTelemetry Protocol (OTLP/gRPC).

The only required parameter is the url() of the destination server, which includes the port number as well.

Example: Forwarding OpenTelemetry data

log otel_forward_mode_alts {
  source {
    opentelemetry(
      port(12345)
      auth(alts())
    );
  };

  destination {
    opentelemetry(
      url("my-otel-server:12345")
      auth(alts())
    );
  };
};

Example: Sending log messages to OpenTelemetry

The following example receives syslog messages and forwards them as OpenTelemetry logs.

log non_otel_to_otel_tls {
  source {
    network(
      port(12346)
    );
  };

  destination {
    opentelemetry(
      url("my-otel-server:12346")
      auth(
        tls(
          ca-file("/path/to/ca.pem")
          key-file("/path/to/key.pem")
          cert-file("/path/to/cert.pem")
        )
      )
    );
  };
};

auth()

You can set authentication in the auth() option of the driver. By default, authentication is disabled (auth(insecure())).

The following authentication methods are available in the auth() block:

adc()

Application Default Credentials (ADC). This authentication method is only available for destinations.

alts()

Application Layer Transport Security (ALTS) is a simple to use authentication, only available within Google’s infrastructure. It accepts the target-service-account() option, where you can list service accounts to match against when authenticating the server.

source {
    opentelemetry(
      port(4317)
      auth(alts())
    );
  };
destination {
    loki(
      port(12345)
      auth(alts())
    );
  };
source {
    syslog-ng-otlp(
      port(4317)
      auth(alts())
    );
  };

insecure()

This is the default method, authentication is disabled (auth(insecure())).

tls()

tls() accepts the key-file(), cert-file(), ca-file() and peer-verify() (possible values: required-trusted, required-untrusted, optional-trusted and optional-untrusted) options.

destination {
    opentelemetry(
      url("your-otel-server:12346")
      auth(
        tls(
          ca-file("/path/to/ca.pem")
          key-file("/path/to/key.pem")
          cert-file("/path/to/cert.pem")
        )
      )
    );
  };
destination {
    loki(
      url("your-loki-server:12346")
      auth(
        tls(
          ca-file("/path/to/ca.pem")
          key-file("/path/to/key.pem")
          cert-file("/path/to/cert.pem")
        )
      )
    );
  };
destination {
    syslog-ng-otlp(
      url("your-otel-server:12346")
      auth(
        tls(
          ca-file("/path/to/ca.pem")
          key-file("/path/to/key.pem")
          cert-file("/path/to/cert.pem")
        )
      )
    );
  };

Note:

  • tls(peer-verify()) is not available for the opentelemetry() and loki() destination.
  • The gRPC-based drivers (opentelemetry() and loki()) have a different tls() block implementation from the network() or http() drivers. Most features are the same.

batch-lines()

Type:number
Default:1

Description: Specifies how many lines are flushed to a destination in one batch. The AxoSyslog application waits for this number of lines to accumulate and sends them off in a single batch. Increasing this number increases throughput as more messages are sent in a single batch, but also increases message latency.

For example, if you set batch-lines() to 100, AxoSyslog waits for 100 messages.

If the batch-timeout() option is disabled, the AxoSyslog application flushes the messages if it has sent batch-lines() number of messages, or the queue became empty. If you stop or reload AxoSyslog or in case of network sources, the connection with the client is closed, AxoSyslog automatically sends the unsent messages to the destination.

Note that if the batch-timeout() option is enabled and the queue becomes empty, AxoSyslog flushes the messages only if batch-timeout() expires, or the batch reaches the limit set in batch-lines().

For optimal performance, make sure that the AxoSyslog source that feeds messages to this destination is configured properly: the value of the log-iw-size() option of the source must be higher than the batch-lines()*workers() of the destination. Otherwise, the size of the batches cannot reach the batch-lines() limit.

batch-timeout()

Type:time in milliseconds
Default:-1 (disabled)

Description: Specifies the time AxoSyslog waits for lines to accumulate in the output buffer. The AxoSyslog application sends batches to the destinations evenly. The timer starts when the first message arrives to the buffer, so if only few messages arrive, AxoSyslog sends messages to the destination at most once every batch-timeout() milliseconds.

workers()

Type:integer
Default:1

Description: Specifies the number of worker threads (at least 1) that AxoSyslog uses to send messages to the server. Increasing the number of worker threads can drastically improve the performance of the destination.

Last modified December 4, 2023: Theme update (6377ea9)