kubernetes: Collect and parse the Kubernetes CRI (Container Runtime Interface) format

The kubernetes() source collects container logs managed by the Kubelet. It reads plain-text and JSON-formatted container logs (as described in the Container Runtime Interface (CRI) design proposal), for example, from the /var/log/containers or /var/log/pods files, and enriches them with various metadata retrieved from the Kubernetes API.

By default, it reads the /var/log/containers folder and extracts:

the log content, and
Kubernetes metadata, for example, namespace, pod, and container information.

The Kubernetes-related metadata is available in name-value pairs with the .k8s. prefix. The following table shows the retrieved metadata and their source:

`syslog-ng` name-value pair	source
`.k8s.namespace_name`	Container log file name.
`.k8s.pod_name`	Container log file name.
`.k8s.pod_uuid`	Container log file name or python kubernetes.client.CoreV1Api.
`.k8s.container_name`	Container log file name or python kubernetes.client.CoreV1Api.
`.k8s.container_id`	Container log file name.
`.k8s.container_image`	python kubernetes.client.CoreV1Api.
`.k8s.container_hash`	python kubernetes.client.CoreV1Api.
`.k8s.docker_id`	python kubernetes.client.CoreV1Api.
`.k8s.labels.*`	python kubernetes.client.CoreV1Api.
`.k8s.annotations.*`	python kubernetes.client.CoreV1Api.

Prerequisites

AxoSyslog version 3.37 or later.
Install the axosyslog-scl package on Debian/Ubuntu. On RHEL and compatible distributions, the SCL files are part of the axosyslog base package.

Your configuration must also contain @include "scl.conf", which the default configuration file already does. If the SCL files are missing, AxoSyslog fails to start with an unexpected LL_IDENTIFIER error.
Terminal window
```
@include "scl.conf"
```
This feature requires a separate module. Install the axosyslog-mod-python package on Debian/Ubuntu, or the axosyslog-python package on RHEL and compatible distributions. If the module isn’t installed, AxoSyslog fails to start with an unexpected LL_IDENTIFIER error.

Declaration

   kubernetes(
        base-dir("<pathname>")
    );

kubernetes() source options

The kubernetes() source has the following options:

base-dir()


Type:	path without filename
Default:	`/var/log/containers`

Description: The path to the directory that contains the log files, for example, base-dir("/var/log/pods").

cluster-name()


Type:	string
Default:	`k8s`

Description: The name of the Kubernetes cluster.

key-delimiter()


Type:	character
Default:	`.`

Description: The delimiter character to use when parsing flattened keys. Supports Only single characters.

prefix()


Synopsis:	prefix()

Description: Insert a prefix before the name part of the parsed name-value pairs to help further processing. For example:

To insert the my-parsed-data. prefix, use the prefix(my-parsed-data.) option.
To refer to a particular data that has a prefix, use the prefix in the name of the macro, for example, ${my-parsed-data.name}.
If you forward the parsed messages using the IETF-syslog protocol, you can insert all the parsed data into the SDATA part of the message using the prefix(.SDATA.my-parsed-data.) option.

Names starting with a dot (for example, .example) are reserved for use by AxoSyslog. If you use such a macro name as the name of a parsed value, it will attempt to replace the original value of the macro (note that only soft macros can be overwritten, see Hard versus soft macros for details). To avoid such problems, use a prefix when naming the parsed values, for example, prefix(my-parsed-data.)

The prefix() option is optional and its default value is ".k8s.".