opensearch: Send messages to OpenSearch

The opensearch() destination can directly post log messages to OpenSearch using its HTTP endpoint.

HTTPS connection, as well as password- and certificate-based authentication is supported. The content of the events is sent in JSON format.

Prerequisites

• AxoSyslog version 4.4 or later.

• Install the axosyslog-scl package on Debian/Ubuntu. On RHEL and compatible distributions, the SCL files are part of the axosyslog base package.

  Your configuration must also contain @include "scl.conf", which the default configuration file already does. If the SCL files are missing, AxoSyslog fails to start with an unexpected LL_IDENTIFIER error.

  @include "scl.conf"

  The opensearch() driver is a reusable configuration snippet (SCL). For details on using or writing such configuration snippets, see Reusing configuration blocks. You can find its source in scl/opensearch/opensearch.conf on GitHub.

• This feature requires a separate module. Install the axosyslog-mod-http package on Debian/Ubuntu, or the axosyslog-http package on RHEL and compatible distributions. If the module isn’t installed, AxoSyslog fails to start with an unexpected LL_IDENTIFIER error.

Declaration:

@include "scl.conf"
# ...

d_opensearch {
    opensearch(
        index("<opensearch-index-to-store-messages>")
        url("https://your-opensearch-endpoint:9200/_bulk")
    );
};

Example: Sending log data to OpenSearch

The following example defines an opensearch() destination, with only the required options.

@include "scl.conf"
# ...

destination opensearch {
    opensearch(
        index("<name-of-the-index>")
        url("http://my-elastic-server:9200/_bulk")
    );
};

log {
    source(s_file);
    destination(d_opensearch_http);
    flags(flow-control);
};

The following example uses mutually-authenticated HTTPS connection, templated index, and also sets some other options.

@include "scl.conf"
# ...

destination opensearch_https {
    opensearch(
        url("https://node01.example.com:9200/_bulk")
        index("test-${YEAR}${MONTH}${DAY}")
        time-zone("UTC")
        workers(4)
        batch-lines(16)
        timeout(10)
        tls(
            ca-file("ca.pem")
            cert-file("syslog_ng.crt.pem")
            key-file("syslog_ng.key.pem")
            peer-verify(yes)
        )
    );
};