Sending Kubernetes logs to OpenSearch
The following tutorial shows you how to install AxoSyslog on Kubernetes, deploy OpenSearch to the same cluster, and send Kubernetes logs to OpenSearch.
Prerequisites
You need:
-
A Kubernetes cluster. We used minikube with docker driver and Helm. We used a Ubuntu 22.04 (amd64) machine, but it should work on any system that can run minikube (2 CPUs, 2GB of free memory, 20GB of free disk space).
The OpenSearch service needs a large mmap count setting, so set it to at least 262144, for example:
Generate logs
If you don’t already have an application that generates logs deployed to the Kubernetes cluster, install kube-logging/log-generator
to generate sample logs. Complete the following steps.
-
Add the
kube-logging
repository to Helm.Expected output:
-
Update your repositories.
The output should look like:
-
Install
kube-logging/log-generator
.The output should look like:
-
Check that the
log-generator
is running:The output should look like:
Set up OpenSearch
-
Install an OpenSearch cluster with Helm:
Expected output:
-
Update your repositories.
The output should look like:
-
Install OpenSearch. This step can take a few minutes.
-
Install the OpenSearch dashboards.
The output should look like:
-
Now you should have 5 pods. Check that they exist:
The output should look like:
-
Forward the 5601 port of the OpenSearch Dashboards service (replace the name of the pod with your pod).
The output should look like:
-
Log in to the dashboard at
http://localhost:8080
with admin/admin. You will soon create an Index Pattern here, but first you have to send some logs fromsyslog-ng
.
Set up axosyslog-collector
-
Add the AxoSyslog Helm repository:
-
Create a YAML file (called
axoflow-demo.yaml
in the examples) to configure the collector. -
Check how the
syslog-ng.conf
file looks with your custom values:The output should look like:
-
Install the
axosyslog-collector
chart:The output should look like:
-
Check your pods:
The output should look like:
Check the logs in OpenSearch
-
Open OpenSearch dashboard at
http://localhost:8080/app/management/opensearch-dashboards/
. -
Create an Index Pattern called
test-axoflow-index
:http://localhost:8080/app/management/opensearch-dashboards/indexPatterns
. At Step 2, set the Time field to@timestamp
. -
Now you can see your logs on the Discover view at
http://localhost:8080/app/discover
. Opening the detailed view for a log entry shows you the fields sent to OpenSearch.