This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Manage and monitor the pipeline

1 - Provision pipeline elements

The following sections describe how to register a logging host into the Axoflow Console.

Hosts with supported collectors

If the host is running one of the following log collector agents and you can install Axolet on the host to receive detailed metrics about the host, the agent, and data flow the agent processes.

  1. Install Axolet on the host. For details, see Axolet.

  2. Configure the log collector agent of the host to integrate with Axolet. For details, see the following pages:

1.1 - AxoRouter

1.1.1 - Install AxoRouter on Kubernetes

To install AxoRouter on a Kubernetes cluster, complete the following steps. For other platforms, see AxoRouter.

Prerequisites

Kubernetes version 1.29 and newer

Minimal resource requirements

  • CPU: at least 100m
  • Memory: 256MB
  • Storage: 8Gi

Network access

The hosts must be able to access the following domains related to the Axoflow Console:

  • When using Axoflow Console SaaS:

    • <your-tenant-id>.cloud.axoflow.io: HTTPS traffic on TCP port 443, needed to download the binaries for Axoflow software (like Axolet and AxoRouter).
    • kcp.<your-tenant-id>.cloud.axoflow.io: HTTPS (mutual TLS) traffic on TCP port 443 for management traffic.
    • telemetry.<your-tenant-id>.cloud.axoflow.io: HTTPS (mutual TLS) traffic on TCP port 443, where Axolet sends the metrics of the host.
    • us-docker.pkg.dev: HTTPS traffic on TCP port 443, for pulling container images (AxoRouter only).

  • When using an on-premise Axoflow Console:

    • The following domains should point to Axoflow Console IP address to access Axoflow from your desktop and AxoRouter hosts:

      • your-host.your-domain: The main domain of your Axoflow Console deployment.
      • authenticate.your-host.your-domain: A subdomain used for authentication.
      • idp.your-host.your-domain: A subdomain for the identity provider.

    • The Axoflow Console host must have the following Open Ports:

      • Port 80 (HTTP)
      • Port 443 (HTTPS)

Install AxoRouter

  1. Open the Axoflow Console.

  2. Select Provisioning.

  3. Select the Host type > AxoRouter > Kubernetes. The one-liner installation command is displayed.

  4. Open a terminal and set your Kubernetes context to the cluster where you want to install AxoRouter.

  5. Run the one-liner, and follow the on-screen instructions.

    Current kubernetes context: minikube
Server Version: v1.28.3
Installing to new namespace: axorouter
Do you want to install AxoRouter now? [Y]

  6. Register the host.

    1. Reload the Provisioning page. There should be a registration request for the new AxoRouter deployment. Select ✓.

      Provisioning AxoRouter - registration request

    2. Select Register to register the host. You can add a description and labels (in label:value format) to the host.

      Provisioning AxoRouter - registration details

    3. Select the Topology page. The new AxoRouter instance is displayed.

Create a flow

  1. If you haven’t already done so, create a new destination.
  2. Create a flow to connect the new AxoRouter to the destination.

    1. Select Flows.

    2. Select Create New Flow.

    3. Enter a name for the flow, for example, my-test-flow.

      Create a flow

    4. In the Router Selector field, enter an expression that matches the router(s) you want to apply the flow. To select a specific router, use a name selector, for example, name = my-axorouter-hostname.

    5. Select the Destination where you want to send your data. If you don’t have any destination configured, see Destinations.

      By default, you can select only external destinations. If you want to send data to another AxoRouter, enable the Show all destinations option, and select the connector of the AxoRouter where you want to send the data.

      AxoRouter as destination

    6. (Optional) To process the data transferred in the flow, select Add New Processing Step. For details, see Processing steps. For example:

      1. Add a Reduce step to automatically remove redundant and empty fields from your data.
      2. To select which messages are processed by the flow, add a Select Messages step, and enter a filter into the Query field. For example, to select only the messages received from Fortinet FortiGate firewalls, use the meta.vendor = fortinet + meta.product = fortigate query.
      3. Save the processing steps.

      Example processing steps

    7. Select Create.

    8. The new flow appears in the Flows list.

      The new flow

Send logs to AxoRouter

By default, AxoRouter accepts data on the following ports:

  • 514 TCP and UDP for RFC3164 (BSD-syslog) formatted traffic.
  • 601 TCP for RFC5424 (IETF-syslog) formatted traffic.
  • 6514 TCP for TLS-encrypted syslog traffic.
  • 4317 TCP for OpenTelemetry log data.

To receive data on other ports or other protocols, configure the source connectors of the AxoRouter host.

Make sure to enable the ports you’re using on the firewall of your host.

1.1.1.1 - Advanced installation options

When installing AxoRouter, you can set a number of advanced options if needed for your environment. Setting the advanced options in the Axoflow Console automatically updates the one-liner command that you can copy and run.

Advanced deployment options

Alternatively, before running the one-liner you can use one of the following methods:

  • Set the related environment variable for the option. For example:

    export AXOROUTER_USER=syslogng
export AXOROUTER_GROUP=syslogng

  • Set the related URL parameter for the option. For example:

    curl -fLsH 'X-AXO-TOKEN:random-generated' 'https://<your-tenant-id>.cloud.axoflow.io/setup.sh?type=AXOROUTER&platform=K8S&parameter=value' | sh

Proxy settings

Use the http_proxy=, https_proxy=, no_proxy= parameters to configure HTTP proxy settings for the installer. To configure the Axolet service to use the proxy settings, enable the AXOLET_AVOID_PROXY parameter as well. Lowercase variable names are preferred because they work universally.

Installation options

You can pass the following parameters to the installation script as environment variables, or as URL parameters.

AxoRouter image override
Default value: empty string
Environment variable IMAGE
URL parameter image

Description: Deploy the specified AxoRouter image.

Helm chart
Default value: oci://us-docker.pkg.dev/axoflow-registry-prod/axoflow/charts/axorouter-syslog
Environment variable HELM_CHART
URL parameter helm_chart

Description: The path or URL of the AxoRouter Helm chart.

Helm chart version
Default value: Current Axoflow version
Environment variable HELM_CHART_VERSION
URL parameter helm_chart_version

Description: Deploy the specified version of the Helm chart.

Helm extra arguments
Default value: empty string
Environment variable HELM_EXTRA_ARGS
URL parameter helm_extra_args

Description: Additional arguments passed to Helm during the installation.

Helm release name
Default value: axorouter
Environment variable HELM_RELEASE_NAME
URL parameter helm_release_name

Description: Name of the Helm release.

Image repository
Default value: us-docker.pkg.dev/axoflow-registry-prod/axoflow/axorouter
Environment variable IMAGE_REPO
URL parameter image_repo

Description: Deploy AxoRouter from a custom image repository.

Image version
Default value: Current Axoflow version
Environment variable IMAGE_VERSION
URL parameter image_version

Description: Deploy the specified AxoRouter version.

Namespace
Default value: axorouter
Environment variable NAMESPACE
URL parameter namespace

Description: The namespace where AxoRouter is installed.

Axolet parameters

API server host
Default value:
Environment variable
URL parameter api_server_host

Description: Override the host part of the API endpoint for the host.

Axolet executable path
Default value:
Environment variable AXOLET_EXECUTABLE
URL parameter axolet_executable

Description: Path to the Axolet executable.

Axolet image override
Default value: empty string
Environment variable AXOLET_IMAGE
URL parameter axolet_image

Description: Deploy the specified Axolet image.

Axolet image repository
Default value: us-docker.pkg.dev/axoflow-registry-prod/axoflow/axolet
Environment variable AXOLET_IMAGE_REPO
URL parameter axolet_image_repo

Description: Deploy Axolet from a custom image repository.

Axolet image version
Default value: Current Axoflow version
Environment variable AXOLET_IMAGE_VERSION
URL parameter axolet_image_version

Description: Deploy the specified Axolet version.

Initial GUID
Default value:
Environment variable
URL parameter initial_guid

Description: Set a static GUID.

1.1.2 - Install AxoRouter on Linux

AxoRouter is a key building block of Axoflow that collects, aggregates, transforms and routes all kinds of telemetry and security data automatically. AxoRouter for Linux includes a Podman container running AxoSyslog, Axolet, and other components.

To install AxoRouter on a Linux host, complete the following steps. For other platforms, see AxoRouter.

What the install script does

The AxoRouter installer will first install a software package which deploys a systemd service.

What the install script does

When you deploy AxoRouter, you run a command that installs the required software packages, configures them and sets up the connection with Axoflow.

The installer script installs the axorouter packages, then executes the configure-axorouter command with the right parameters. (If you’ve installed the packages in advance, the installer script only executes the configure-axorouter command.)

The configure-axorouter command is designed to be run as root (sudo), but you can configure axorouter to run as a non-root user. The configure-axorouter command is executed with a configuration snippet on its standard input which contains a token required for registering into the management platform.

The script performs the following main steps:

  • Generates a unique identifier (GUID).
  • Initiates a cryptographic handshake process to Axoflow.
  • Creates the initial configuration file for AxoRouter under /etc/axorouter/.
  • Installs a statically linked executable to /usr/local/bin/axorouter.
  • Creates the systemd service unit file /etc/systemd/system/axorouter.service, then enables and starts that service.
  • The service waits for an approval on Axoflow. Once you approve the host registration request, Axoflow issues a client certificate to AxoRouter.
  • AxoRouter starts to send telemetry data to Axoflow, and keeps sending them as long as the agent is registered and the certificate is valid.

Prerequisites

Minimal resource requirements

  • CPU: at least 100m
  • Memory: 256MB
  • Storage: 8Gi

Network access

The hosts must be able to access the following domains related to the Axoflow Console:

  • When using Axoflow Console SaaS:

    • <your-tenant-id>.cloud.axoflow.io: HTTPS traffic on TCP port 443, needed to download the binaries for Axoflow software (like Axolet and AxoRouter).
    • kcp.<your-tenant-id>.cloud.axoflow.io: HTTPS (mutual TLS) traffic on TCP port 443 for management traffic.
    • telemetry.<your-tenant-id>.cloud.axoflow.io: HTTPS (mutual TLS) traffic on TCP port 443, where Axolet sends the metrics of the host.
    • us-docker.pkg.dev: HTTPS traffic on TCP port 443, for pulling container images (AxoRouter only).

  • When using an on-premise Axoflow Console:

    • The following domains should point to Axoflow Console IP address to access Axoflow from your desktop and AxoRouter hosts:

      • your-host.your-domain: The main domain of your Axoflow Console deployment.
      • authenticate.your-host.your-domain: A subdomain used for authentication.
      • idp.your-host.your-domain: A subdomain for the identity provider.

    • The Axoflow Console host must have the following Open Ports:

      • Port 80 (HTTP)
      • Port 443 (HTTPS)

Install AxoRouter

  1. Select Provisioning > Select type and platform.

    Provisioning AxoRouter on Linux

  2. Select the type (AxoRouter) and platform (Linux). The one-liner installation command is displayed.

    Provisioning AxoRouter on Linux

    If needed, set the Advanced options (for example, proxy settings) to modify the installation parameters. Usually, you don’t have to use advanced options unless the Axoflow support team instructs you to do so.

  3. Open a terminal on the host where you want to install AxoRouter.

  4. Run the one-liner, then follow the on-screen instructions.

    Example output:

    Do you want to install AxoRouter now? [Y]
y
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  4142  100  4142    0     0  19723      0 --:--:-- --:--:-- --:--:-- 19818
Verifying packages...
Preparing packages...
axorouter-0.40.0-1.aarch64
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100 31.6M  100 31.6M    0     0  2092k      0  0:00:15  0:00:15 --:--:-- 2009k
Verifying packages...
Preparing packages...
axolet-0.40.0-1.aarch64
Created symlink /etc/systemd/system/multi-user.target.wants/axolet.service → /usr/lib/systemd/system/axolet.service.
Now continue with onboarding the host on the Axoflow web UI.

  5. Register the host.

    1. Reload the Provisioning page. There should be a registration request for the new AxoRouter deployment. Select ✓.

      Provisioning AxoRouter - registration request

    2. Select Register to register the host. You can add a description and labels (in label:value format) to the host.

      Provisioning AxoRouter - registration details

    3. Select the Topology page. The new AxoRouter instance is displayed.

Create a flow

  1. If you haven’t already done so, create a new destination.
  2. Create a flow to connect the new AxoRouter to the destination.

    1. Select Flows.

    2. Select Create New Flow.

    3. Enter a name for the flow, for example, my-test-flow.

      Create a flow

    4. In the Router Selector field, enter an expression that matches the router(s) you want to apply the flow. To select a specific router, use a name selector, for example, name = my-axorouter-hostname.

    5. Select the Destination where you want to send your data. If you don’t have any destination configured, see Destinations.

      By default, you can select only external destinations. If you want to send data to another AxoRouter, enable the Show all destinations option, and select the connector of the AxoRouter where you want to send the data.

      AxoRouter as destination

    6. (Optional) To process the data transferred in the flow, select Add New Processing Step. For details, see Processing steps. For example:

      1. Add a Reduce step to automatically remove redundant and empty fields from your data.
      2. To select which messages are processed by the flow, add a Select Messages step, and enter a filter into the Query field. For example, to select only the messages received from Fortinet FortiGate firewalls, use the meta.vendor = fortinet + meta.product = fortigate query.
      3. Save the processing steps.

      Example processing steps

    7. Select Create.

    8. The new flow appears in the Flows list.

      The new flow

Send logs to AxoRouter

Configure your hosts to send data to AxoRouter.

  • For appliances that are specifically supported by Axoflow, see Sources.

  • For other appliances and generic Linux devices, see Generic tips.

  • For a quick test without an actual source, you can also do the following (requires nc to be installed on the AxoRouter host):

    1. Open the Axoflow Console, select Topology, then select the AxoRouter instance you’ve deployed.

    2. Select ⋮ > Tap log flow > Input log flow. Select Start.

    3. Open a terminal on your AxoRouter host.

    4. Run the following command to send 120 test messages (2 per second) in a loop to AxoRouter:

      for i in `seq 1 120`; do echo "<165> fortigate date=$(date -u +%Y-%m-%d) time=$(date -u +"%H:%M:%S%Z") devname=us-east-1-dc1-a-dmz-fw devid=FGT60D4614044725 logid=0100040704 type=event subtype=system level=notice vd=root logdesc=\"System performance statistics\" action=\"perf-stats\" cpu=2 mem=35 totalsession=61 disk=2 bandwidth=158/138 setuprate=2 disklograte=0 fazlograte=0 msg=\"Performance statistics: average CPU: 2, memory:  35, concurrent sessions:  61, setup-rate: 2\""; sleep 0.5; done | nc -v 127.0.0.1 514

      Alternatively, you can send logs in an endless loop:

      while true; do echo "<165> fortigate date=$(date -u +%Y-%m-%d) time=$(date -u +"%H:%M:%S%Z") devname=us-east-1-dc1-a-dmz-fw devid=FGT60D4614044725 logid=0100040704 type=event subtype=system level=notice vd=root logdesc=\"System performance statistics\" action=\"perf-stats\" cpu=2 mem=35 totalsession=61 disk=2 bandwidth=158/138 setuprate=2 disklograte=0 fazlograte=0 msg=\"Performance statistics: average CPU: 2, memory:  35, concurrent sessions:  61, setup-rate: 2\""; sleep 1; done | nc -v 127.0.0.1 514

Manage AxoRouter

This section describes how to start, stop and check the status of the AxoRouter service on Linux.

Start AxoRouter

To start AxoRouter, execute the following command. For example:

systemctl start axorouter

If the service starts successfully, no output will be displayed.

The following message indicates that AxoRouter can not start (see Check AxoRouter status):

Job for axorouter.service failed because the control process exited with error code. See `systemctl status axorouter.service` and `journalctl -xe` for details.

Stop AxoRouter

To stop AxoRouter

  1. Execute the following command.

    systemctl stop axorouter

  2. Check the status of the AxoRouter service (see Check AxoRouter status).

Restart AxoRouter

To restart AxoRouter, execute the following command.

systemctl restart axorouter

Reload the configuration without restarting AxoRouter

To reload the configuration file without restarting AxoRouter, execute the following command.

systemctl reload axorouter

Check the status of AxoRouter service

To check the status of AxoRouter service

  1. Execute the following command.

    systemctl --no-pager status axorouter

  2. Check the Active: field, which shows the status of the AxoRouter service. The following statuses are possible:

    • active (running) - axorouter service is up and running

    • inactive (dead) - axorouter service is stopped

1.1.2.1 - Advanced installation options

When installing AxoRouter, you can set a number of advanced options if needed for your environment. Setting the advanced options in the Axoflow Console automatically updates the one-liner command that you can copy and run.

Advanced deployment options

Alternatively, before running the one-liner you can use one of the following methods:

  • Set the related environment variable for the option. For example:

    export AXO_USER=syslogng
export AXO_GROUP=syslogng

  • Set the related URL parameter for the option. For example:

    curl -fLsH 'X-AXO-TOKEN:random-generated' 'https://<your-tenant-id>.cloud.axoflow.io/setup.sh?type=AXOROUTER&platform=LINUX&user=syslogng&group=syslogng' | sh

Proxy settings

Use theHTTP proxy, HTTPS proxy, No proxy parameters to configure HTTP proxy settings for the installer. To avoid using the proxy for the Axolet service, enable the Avoid proxy parameter as well. Lowercase variable names are preferred because they work universally.

Installation options

You can pass the following parameters to the installation script as environment variables, or as URL parameters.

AxoRouter capabilities
Default value: CAP_NET_BIND_SERVICE CAP_NET_BROADCAST CAP_NET_RAW CAP_SYSLOG CAP_BPF
Environment variable AXO_AXOROUTER_CAPS
URL parameter axorouter_caps

Description: Capabilities added to the AxoRouter container.

AxoRouter config mount path
Default value: /etc/axorouter/user-config
Environment variable AXO_AXOROUTER_CONFIG_MOUNT_INSIDE
URL parameter axorouter_config_mount_inside

Description: Mount path for custom user configuration.

AxoRouter image override
Default value: us-docker.pkg.dev/axoflow-registry-prod/axoflow/axorouter
Environment variable AXO_IMAGE
URL parameter image

Description: Deploy the specified AxoRouter image.

Extra container arguments
Default value: empty string
Environment variable AXO_PODMAN_ARGS
URL parameter extra_args

Description: Additional arguments passed to the AxoRouter container.

Image repository
Default value: us-docker.pkg.dev/axoflow-registry-prod/axoflow/axorouter
Environment variable AXO_IMAGE_REPO
URL parameter image_repo

Description: Deploy AxoRouter from a custom image repository.

Image version
Default value: Current Axoflow version
Environment variable AXO_IMAGE_VERSION
URL parameter image_version

Description: Deploy the specified AxoRouter version.

Package format
Default value: auto
Available values: auto, dep, rpm, tar, none
Environment variable AXO_INSTALL_PACKAGE
URL parameter install_package

Description: File format of the installer package.

Start router
Default value: true
Available values: true, false
Environment variable AXO_START_ROUTER
URL parameter start_router

Description: Start AxoRouter after installation.

Axolet parameters

API server host
Default value:
Environment variable
URL parameter api_server_host

Description: Override the host part of the API endpoint for the host.

Avoid proxy
Default value: false
Available values: true, false
Environment variable AXO_AVOID_PROXY
URL parameter avoid_proxy

Description: Do not use proxy for the Axolet process.

Axolet capabilities
Default value: CAP_SYS_PTRACE CAP_SYS_CHROOT
Environment variable AXO_CAPS
URL parameter caps

Description: Capabilities added to the Axolet service.

Configuration directory
Default value: /etc/axolet
Environment variable AXO_CONFIG_DIR
URL parameter config_dir

Description: The directory where the configuration files are stored.

HTTP proxy
Default value: empty string
Environment variable AXO_HTTP_PROXY
URL parameter http_proxy

Description: Use a proxy to access Axoflow Console from the host.

HTTPS proxy
Default value: empty string
Environment variable AXO_HTTPS_PROXY
URL parameter https_proxy

Description: Use a proxy to access Axoflow Console from the host.

No proxy
Default value: empty string
Environment variable AXO_NO_PROXY
URL parameter no_proxy

Description: Comma-separated list of hosts that shouldn’t use proxy to access Axoflow Console from the host.

Overwrite config
Default value: false
Available values: true, false
Environment variable AXO_CONFIG_OVERWRITE
URL parameter config_overwrite

Description: Overwrite the configuration when reinstalling the service.

Service group
Default value: root
Environment variable AXO_GROUP
URL parameter group

Description: The group running the Axolet service.

Service user
Default value: root
Environment variable AXO_USER
URL parameter user

Description: The user running the Axolet service.

Start service
Default value: true
Available values: true, false
Environment variable AXO_START
URL parameter start

Description: Start the Axolet service after installation.

WEC parameters

These parameters are related to the Windows Event Collector server that can be run on AxoRouter. For details, see Windows Event Collector (WEC).

WEC Image repository
Default value: us-docker.pkg.dev/axoflow-registry-prod/axoflow/axorouter-wec
Environment variable AXO_WEC_IMAGE_REPO
URL parameter wec_image_repo

Description: Deploy the Windows Event Collector server from a custom image repository.

WEC Image version
Default value: Current Axoflow version
Environment variable AXO_WEC_IMAGE_VERSION
URL parameter wec_image_version

Description: Deploy the specified Windows Event Collector server version.

1.1.2.2 - Run AxoRouter as non-root

To run AxoRouter as a non-root user, set the AXO_USER and AXO_GROUP environment variables to the user’s username and groupname on the host you want to deploy AxoRouter. For details, see Advanced installation options.

Operators must have access to the following commands:

  • /usr/bin/systemctl * axolet.service: Controls the axolet.service systemd unit. Usually * is start, stop, restart, enable, and status. Used by the operators for troubleshooting.

  • /usr/local/bin/configure-axolet: Creates initial axolet configuration and enables/starts the axolet service. Executed by the bootstrap script.

  • Command to install and upgrade the axolet package. Executed by the bootstrap script if the packages aren’t already installed.

    • On RPM-based Linux distributions: /usr/bin/rpm -Uv axo*.rpm
    • On DEB-based Linux distributions: /usr/bin/dpkg -i axo*.deb

  • /usr/bin/systemctl * axorouter.service: Controls the axorouter.service systemd unit. Usually * is start, stop, restart, enable, and status. Used by the operators for troubleshooting.

  • /usr/local/bin/configure-axorouter: Creates the initial axorouter configuration and enables/starts the axorouter service. Executed by the bootstrap script.

  • Command to install and upgrade the axorouter and the axolet package. Executed by the bootstrap script if the packages aren’t already installed.

    • On RPM-based Linux distributions: /usr/bin/rpm -Uv axo*.rpm
    • On DEB-based Linux distributions: /usr/bin/dpkg -i axo*.deb

You can permit the syslogng user to run these commands by running on of the following:

sudo tee /etc/sudoers.d/configure-axoflow <<A
syslogng ALL=(ALL) NOPASSWD: /usr/local/bin/configure-axolet
syslogng ALL=(ALL) NOPASSWD: /bin/systemctl * axolet.service
# for rpm installation:
syslogng ALL=(ALL) NOPASSWD: /usr/bin/rpm -Uv axo*.rpm
A

sudo tee /etc/sudoers.d/configure-axorouter <<A
syslogng ALL=(ALL) NOPASSWD: /usr/local/bin/configure-axorouter
syslogng ALL=(ALL) NOPASSWD: /bin/systemctl * axorouter.service
# for rpm installation:
syslogng ALL=(ALL) NOPASSWD: /usr/bin/rpm -Uv axo*.rpm
A

sudo tee /etc/sudoers.d/configure-axorouter <<A
syslogng ALL=(ALL) NOPASSWD: /usr/local/bin/configure-axorouter
syslogng ALL=(ALL) NOPASSWD: /bin/systemctl * axorouter.service
# for deb installation:
syslogng ALL=(ALL) NOPASSWD: /usr/bin/dpkg -i axo*.deb
A

sudo tee /etc/sudoers.d/configure-axorouter <<A
syslogng ALL=(ALL) NOPASSWD: /usr/local/bin/configure-axorouter
syslogng ALL=(ALL) NOPASSWD: /bin/systemctl * axorouter.service
# for deb installation:
syslogng ALL=(ALL) NOPASSWD: /usr/bin/rpm -Uv axo*.deb
A

1.2 - AxoSyslog

Onboarding allows you to collect metrics about the host, display the host on the Topology page, and to tap into the log flow.

Onboarding requires you to modify the host and the configuration of the logging agent running on the host.

  • Level 1: Install Axolet on the host. Axolet collects metrics from the host and sends them to the Axoflow Console, so you can check host-level metrics on the Metrics & Health page of the host, and displays the host on the Topology page.
  • Level 2: Instrument the configuration of the logging agent to provide detailed metrics about the traffic flow. This allows you to display data about the host on the Analytics page.
  • Level 3: Instrument the configuration of the logging agent to allow you to access the logs of the logging agent and to tap into the log flow from the Axoflow Console. The exact steps for this integration step depend on the configuration of your logging agent. Contact us so our professional services can help you with the integration.

To onboard an existing AxoSyslog instance into Axoflow, complete the following steps.

  1. Install Axolet on the host, then approve its registration on the Provisioning page of the Axoflow Console.

  2. The AxoSyslog host is now visible on the Topology page of the Axoflow Console as a source.

  3. If you've already added the AxoRouter instance or the destination where this host is sending data to the Axoflow Console, add a path to connect the host to the AxoRouter or the destination.

    1. Select Topology > + > Path.

      Add a new path

    2. Select your data source in the Source host field.

      Configure path

    3. Select the target router or aggregator this source is sending its data to in the Target host field, for example, axorouter.

    4. Select the Target connector. The connector determines how the destination receives the data (for example, using which protocol or port).

    5. Select Create. The new path appears on the Topology page.

      The new path

  4. Access the AxoSyslog host and edit the configuration of AxoSyslog. Set the statistics-related global options like this (if the options block already exists, add these lines to the bottom of the block):

    options {
    stats(
        level(2)
        freq(0) # Inhibit statistics output to stdout
        );
};

  5. (Optional) To get detailed metrics and analytics about the traffic that flows through the host, instrument your AxoSyslog configuration as follows:

    1. Download the following configuration snippet to the AxoSyslog host, for example, as /etc/syslog-ng/conf.d/axoflow-instrumentation.conf.

      # Metrics probe parser definition for a specified destination
# This will yield a reasonable number of metrics for common
# syslog-ng macros
block parser metrics-output(destination(undefined)) {

# This will probe for the number of *events*.
# key name is fixed; labels are user-specified
  metrics-probe(
    key("classified_output_events_total")
    labels(
      "app" => "${PROGRAM}"
      "host" => "${HOST}"
      "program" => "${PROGRAM}"
      "source" => "${SOURCE}"
      "dest_port" => "$DESTPORT"
      "dest_ip" => "$DESTIP"
      "src_ip" => "$SOURCEIP"
      "protocol" => "$(if ($PROTO == 6) TCP $(if ($PROTO == 17) UDP $PROTO))"
      "ip_protocol" => "$IP_PROTO"
      "transport" => "$TRANSPORT"
      "destination" => "`destination`"
    )
  );

# This will probe for the number of *bytes*.
# key name is fixed; labels are user-specified
  metrics-probe(
    key("classified_output_event_bytes_total")
    increment("${RAWMSG_SIZE}")
    labels(
        "app" => "${PROGRAM}"
        "host" => "${HOST}"
        "program" => "${PROGRAM}"
        "source" => "${SOURCE}"
        "dest_port" => "$DESTPORT"
        "dest_ip" => "$DESTIP"
        "src_ip" => "$SOURCEIP"
        "protocol" => "$(if ($PROTO == 6) TCP $(if ($PROTO == 17) UDP $PROTO))"
        "ip_protocol" => "$IP_PROTO"
        "transport" => "$TRANSPORT"
        "destination" => "`destination`"
    )
  );
};
      Download this file

    2. Include it in at the top of your configuration file:

      @version: current
@include "axoflow-instrumentation.conf"

    3. Edit every destination statement to include a parser { metrics-output(destination(<custom-ID-for-the-destination>)); }; line, for example:

      destination d_file {
    channel {
        parser { metrics-output(destination(my-file-destination)); };
        destination { file("/dev/null" persist-name("d_s3")); };
        };
    };

  6. Reload the configuration of AxoSyslog.

    systemctl reload syslog-ng

1.3 - Splunk Connect for Syslog (SC4S)

Onboarding allows you to collect metrics about the host, display the host on the Topology page, and to tap into the log flow.

Onboarding requires you to modify the host and the configuration of the logging agent running on the host.

  • Level 1: Install Axolet on the host. Axolet collects metrics from the host and sends them to the Axoflow Console, so you can check host-level metrics on the Metrics & Health page of the host, and displays the host on the Topology page.
  • Level 2: Instrument the configuration of the logging agent to provide detailed metrics about the traffic flow. This allows you to display data about the host on the Analytics page.
  • Level 3: Instrument the configuration of the logging agent to allow you to access the logs of the logging agent and to tap into the log flow from the Axoflow Console. The exact steps for this integration step depend on the configuration of your logging agent. Contact us so our professional services can help you with the integration.

To generate metrics for the Axoflow platform from an existing Splunk Connect for Syslog (SC4S) instance, you need to configure SC4S to generate these metrics. Complete the following steps.

  1. If you haven’t already done so, install Axolet on the host, then approve its registration on the Provisioning page of the Axoflow Console.

  2. Download the following code snippet as axoflow-instrumentation.conf.

    Loading…

  3. If you are running SC4S under podman or docker, copy the file into the /opt/sc4s/local/config/destinations directory. In other deployment methods this might be different, check the SC4S documentation for details.

  4. Check if the metrics are appearing, for example, run the following command on the SC4S host:

    syslog-ng-ctl stats prometheus | grep classified

1.4 - syslog-ng

Onboarding allows you to collect metrics about the host, display the host on the Topology page, and to tap into the log flow.

Onboarding requires you to modify the host and the configuration of the logging agent running on the host.

  • Level 1: Install Axolet on the host. Axolet collects metrics from the host and sends them to the Axoflow Console, so you can check host-level metrics on the Metrics & Health page of the host, and displays the host on the Topology page.
  • Level 2: Instrument the configuration of the logging agent to provide detailed metrics about the traffic flow. This allows you to display data about the host on the Analytics page.
  • Level 3: Instrument the configuration of the logging agent to allow you to access the logs of the logging agent and to tap into the log flow from the Axoflow Console. The exact steps for this integration step depend on the configuration of your logging agent. Contact us so our professional services can help you with the integration.

To onboard an existing syslog-ng instance into Axoflow, complete the following steps.

  1. Install Axolet on the host, then approve its registration on the Provisioning page of the Axoflow Console.

  2. The syslog-ng host is now visible on the Topology page of the Axoflow Console as a source.

  3. If you've already added the AxoRouter instance or the destination where this host is sending data to the Axoflow Console, add a path to connect the host to the AxoRouter or the destination.

    1. Select Topology > + > Path.

      Add a new path

    2. Select your data source in the Source host field.

      Configure path

    3. Select the target router or aggregator this source is sending its data to in the Target host field, for example, axorouter.

    4. Select the Target connector. The connector determines how the destination receives the data (for example, using which protocol or port).

    5. Select Create. The new path appears on the Topology page.

      The new path

  4. Access the syslog-ng host and edit the configuration of syslog-ng. Set the statistics-related global options like this (if the options block already exists, add these lines to the bottom of the block):

    options {
    stats-level(2);
    stats-freq(0); # Inhibit statistics output to stdout
};

  5. (Optional) To get detailed metrics and analytics about the traffic that flows through the host, instrument your syslog-ng configuration as follows:

    1. Download the following configuration snippet to the syslog-ng host, for example, as /etc/syslog-ng/conf.d/axoflow-instrumentation.conf.

      Loading…

    2. Include it in at the top of your configuration file:

      @version: current
@include "axoflow-instrumentation.conf"

    3. Edit every destination statement to include a parser { metrics-output(destination(<custom-ID-for-the-destination>)); }; line, for example:

      destination d_file {
    channel {
        parser { metrics-output(destination(my-file-destination)); };
        destination { file("/dev/null" persist-name("d_s3")); };
        };
    };

  6. Reload the configuration of syslog-ng.

    systemctl reload syslog-ng

1.5 - Axolet

Axolet is the agent software for Axoflow. Its primary purpose is to discover the log collector services that are running on the host (for example, AxoSyslog or SC4S), and report their operating statistics (counters) to Axoflow.

1.5.1 - Install Axolet

Axolet is the agent software for Axoflow. Its primary purpose is to discover the log collector services that are running on the host (for example, AxoSyslog or SC4S), and report their operating statistics (counters) to Axoflow.

It is simple to install Axolet on individual hosts, or collectively via orchestration tools such as chef or puppet.

What the install script does

When you deploy Axolet, you run a command that installs the required software packages, configures them and sets up the connection with Axoflow.

The installer script installs the axolet packages, then executes the configure-axolet command with the right parameters. (If you’ve installed the packages in advance, the installer script only executes the configure-axolet command.)

The configure-axolet command is designed to be run as root (sudo), but you can configure axolet to run as a non-root user. The configure-axolet command is executed with a configuration snippet on its standard input which contains a token required for registering into the management platform.

The script performs the following main steps:

  • Generates a unique identifier (GUID).
  • Initiates a cryptographic handshake process to Axoflow.
  • Creates the initial configuration file for Axolet under /etc/axolet/.
  • Installs a statically linked executable to /usr/local/bin/axolet.
  • Creates the systemd service unit file /etc/systemd/system/axolet.service, then enables and starts that service.
  • The service waits for an approval on Axoflow. Once you approve the host registration request, Axoflow issues a client certificate to Axolet.
  • Axolet starts to send telemetry data to Axoflow, and keeps sending them as long as the agent is registered and the certificate is valid.

Axolet communication flow

Prerequisites

Axolet should work on most Red Hat and Debian compatible Linux distributions. For production environments, we recommend using Red Hat Enterprise Linux 9.

Network access

The hosts must be able to access the following domains related to the Axoflow Console:

  • When using Axoflow Console SaaS:

    • <your-tenant-id>.cloud.axoflow.io: HTTPS traffic on TCP port 443, needed to download the binaries for Axoflow software (like Axolet and AxoRouter).
    • kcp.<your-tenant-id>.cloud.axoflow.io: HTTPS (mutual TLS) traffic on TCP port 443 for management traffic.
    • telemetry.<your-tenant-id>.cloud.axoflow.io: HTTPS (mutual TLS) traffic on TCP port 443, where Axolet sends the metrics of the host.
    • us-docker.pkg.dev: HTTPS traffic on TCP port 443, for pulling container images (AxoRouter only).

  • When using an on-premise Axoflow Console:

    • The following domains should point to Axoflow Console IP address to access Axoflow from your desktop and AxoRouter hosts:

      • your-host.your-domain: The main domain of your Axoflow Console deployment.
      • authenticate.your-host.your-domain: A subdomain used for authentication.
      • idp.your-host.your-domain: A subdomain for the identity provider.

    • The Axoflow Console host must have the following Open Ports:

      • Port 80 (HTTP)
      • Port 443 (HTTPS)

Install Axolet

To install Axolet on a host and onboard it to Axoflow, complete the following steps. If you need to reinstall Axolet for some reason, see Reinstall Axolet.

  1. Open the Axoflow Console at https://<your-tenant-id>.cloud.axoflow.io/.

  2. Select Provisioning > Select type and platform.

    Axoflow Host Deployment

  3. Select Edge > Linux.

    Axoflow agent type and platform selection

    The curl command can be run manually or inserted into a template in any common software deployment package. When run, a script is downloaded that sets up the Axolet process to run automatically at boot time via systemd. For advanced installation options, see Advanced installation options.

  4. Copy the deployment one-liner and run it on the host you are onboarding into Axoflow.

    Example output:

    Do you want to install Axolet now? [Y]
y
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100 31.6M  100 31.6M    0     0  2127k      0  0:00:15  0:00:15 --:--:-- 2075k
Verifying packages...
Preparing packages...
axolet-0.40.0-1.aarch64
Created symlink /etc/systemd/system/multi-user.target.wants/axolet.service → /usr/lib/systemd/system/axolet.service.
Now continue with onboarding the host on the Axoflow web UI.

  5. On the Axoflow Console, reload the Provisioning page. A registration request for the new host should be displayed. Accept it.

  6. Axolet starts sending metrics from the host. Check the Topology page to see the new host.

  7. Continue to onboard the host as described for your specific log collector agent.

Manage Axolet

This section describes how to start, stop and check the status of the Axolet service on Linux.

Start Axolet

To start Axolet, execute the following command. For example:

systemctl start axolet

If the service starts successfully, no output will be displayed.

The following message indicates that Axolet can not start (see Check Axolet status):

Job for axolet.service failed because the control process exited with error code. See `systemctl status axolet.service` and `journalctl -xe` for details.

Stop Axolet

To stop Axolet

  1. Execute the following command.

    systemctl stop axolet

  2. Check the status of the Axolet service (see Check Axolet status).

Restart Axolet

To restart Axolet, execute the following command.

systemctl restart axolet

Reload the configuration without restarting Axolet

To reload the configuration file without restarting Axolet, execute the following command.

systemctl reload axolet

Check the status of Axolet service

To check the status of Axolet service

  1. Execute the following command.

    systemctl --no-pager status axolet

  2. Check the Active: field, which shows the status of the Axolet service. The following statuses are possible:

    • active (running) - axolet service is up and running

    • inactive (dead) - axolet service is stopped

Upgrade Axolet

To upgrade Axolet, re-run the installation script.

Run axolet as non-root

You can run Axolet as non-root user, but operators must have access to the following commands:

  • /usr/bin/systemctl * axolet.service: Controls the axolet.service systemd unit. Usually * is start, stop, restart, enable, and status. Used by the operators for troubleshooting.

  • /usr/local/bin/configure-axolet: Creates initial axolet configuration and enables/starts the axolet service. Executed by the bootstrap script.

  • Command to install and upgrade the axolet package. Executed by the bootstrap script if the packages aren’t already installed.

    • On RPM-based Linux distributions: /usr/bin/rpm -Uv axo*.rpm
    • On DEB-based Linux distributions: /usr/bin/dpkg -i axo*.deb

You can permit the syslogng user to run these commands by running on of the following:

sudo tee /etc/sudoers.d/configure-axoflow <<A
syslogng ALL=(ALL) NOPASSWD: /usr/local/bin/configure-axolet
syslogng ALL=(ALL) NOPASSWD: /bin/systemctl * axolet.service
# for rpm installation:
syslogng ALL=(ALL) NOPASSWD: /usr/bin/rpm -Uv axo*.rpm
A

sudo tee /etc/sudoers.d/configure-axoflow <<A
syslogng ALL=(ALL) NOPASSWD: /usr/local/bin/configure-axolet
syslogng ALL=(ALL) NOPASSWD: /bin/systemctl * axolet.service
# for deb installation:
syslogng ALL=(ALL) NOPASSWD: /usr/bin/dpkg -i axo*.deb
A

1.5.2 - Advanced installation options

When installing Axolet, you can set a number of advanced options if needed for your environment. Setting the advanced options in the Axoflow Console automatically updates the one-liner command that you can copy and run.

Advanced deployment options

Alternatively, before running the one-liner you can use one of the following methods:

  • Set the related environment variable for the option. For example:

    export AXOLET_USER=syslogng
export AXOLET_GROUP=syslogng

  • Set the related URL parameter for the option. For example:

    curl -fLsH 'X-AXO-TOKEN:random-generated' 'https://<your-tenant-id>.cloud.axoflow.io/setup.sh?type=AXOEDGE&platform=LINUX&user=syslogng&group=syslogng' | sh

Proxy settings

Use theHTTP proxy, HTTPS proxy, No proxy parameters to configure HTTP proxy settings for the installer. To avoid using the proxy for the Axolet service, enable the Avoid proxy parameter as well. Lowercase variable names are preferred because they work universally.

Installation options

You can pass the following parameters to the installation script as environment variables, or as URL parameters.

API server host
Default value:
Environment variable
URL parameter api_server_host

Description: Override the host part of the API endpoint for the host.

Avoid proxy
Default value: false
Available values: true, false
Environment variable AXOLET_AVOID_PROXY
URL parameter avoid_proxy

Description: If set to true, the value of the *_proxy variables will only be used for downloading the installer, but not for the axolet service itself. If set to false, the Axolet service will use the variables from the installer.

Capabilities
Default value: CAP_SYS_PTRACE
Available values: Whitespace-separated list of capability names with CAP_ prefix.
Environment variable AXOLET_CAPS
URL parameter caps

Description: Ambient Linux capabilities the axolet service will use.

Configuration directory
Default value: /etc/axolet
Environment variable AXOLET_CONFIG_DIR
URL parameter config_dir

Description: The directory where the configuration files are stored.

HTTP proxy
Default value: empty string
Environment variable AXOLET_HTTP_PROXY
URL parameter http_proxy

Description: Use a proxy to access Axoflow Console from the host.

HTTPS proxy
Default value: empty string
Environment variable AXOLET_HTTPS_PROXY
URL parameter https_proxy

Description: Use a proxy to access Axoflow Console from the host.

No proxy
Default value: empty string
Environment variable AXOLET_NO_PROXY
URL parameter no_proxy

Description: Comma-separated list of hosts that shouldn’t use proxy to access Axoflow Console from the host.

Overwrite config
Default value: false
Available values: true, false
Environment variable AXOLET_CONFIG_OVERWRITE
URL parameter config_overwrite

Description: If set to true, the configuration process will overwrite existing configuration (/etc/axolet/config.json). This means that the agent will get a new GUID and it will require approval on the Axoflow Console.

Package format
Default value: auto
Available values: auto, dep, rpm, tar, none
Environment variable AXOLET_INSTALL_PACKAGE
URL parameter install_package

Description: File format of the installer package.

Service group
Default value: root
Environment variable AXOLET_GROUP
URL parameter group

Description: Name of the group and Axolet will be running as. It should be either root or the group syslog-ng is running as.

Service user
Default value: root
Environment variable AXOLET_USER
URL parameter user

Description: Name of the user Axolet will be running as. It should be either root or the user syslog-ng is running as. See also Run axolet as non-root.

Start service
Default value: AXOLET_START=true
Available values: true, false
Environment variable AXOLET_START
URL parameter start

Description: Start axolet agent at the end of installation. Use false for preparing golden images. In this case axolet will generate a new GUID on the first boot after cloning the image.

If you are preparing a host for cloning with Axolet already installed, set the following environment variable in your (root) shell session, before running the one-liner command. For example:

export START_AXOLET=false
curl ... # Run the command copied from the Provisioning page

This way Axolet will only start and initialize after the first reboot.

1.5.3 - Run axolet as non-root

If the log collector agent (AxoSyslog or syslog-ng) is running as a non-root user, you may want to configure the Axolet agent to run as the same user. To do that, set the AXOLET_USER and AXOLET_GROUP environment variables to the user’s username and groupname. For details, see Advanced installation options.

Operators will need to have access to the following commands:

  • /usr/bin/systemctl * axolet.service: Controls the axolet.service systemd unit. Usually * is start, stop, restart, enable, and status. Used by the operators for troubleshooting.

  • /usr/local/bin/configure-axolet: Creates initial axolet configuration and enables/starts the axolet service. Executed by the bootstrap script.

  • Command to install and upgrade the axolet package. Executed by the bootstrap script if the packages aren’t already installed.

    • On RPM-based Linux distributions: /usr/bin/rpm -Uv axo*.rpm
    • On DEB-based Linux distributions: /usr/bin/dpkg -i axo*.deb

For example, you can permit the syslogng user to run these commands by running the following commands:

sudo tee /etc/sudoers.d/configure-axoflow <<A
syslogng ALL=(ALL) NOPASSWD: /usr/local/bin/configure-axolet
syslogng ALL=(ALL) NOPASSWD: /bin/systemctl * axolet.service
# for rpm installation:
syslogng ALL=(ALL) NOPASSWD: /usr/bin/rpm -Uv axo*.rpm
A

sudo tee /etc/sudoers.d/configure-axoflow <<A
syslogng ALL=(ALL) NOPASSWD: /usr/local/bin/configure-axolet
syslogng ALL=(ALL) NOPASSWD: /bin/systemctl * axolet.service
# for deb installation:
syslogng ALL=(ALL) NOPASSWD: /usr/bin/dpkg -i axo*.deb
A

1.5.4 - Reinstall Axolet

Reinstall Axolet to a (cloned) machine

To re-install Axolet on a (cloned) machine that has an earlier installation running, complete the following steps.

  1. Log in to the host as root and execute the following commands:

    systemctl stop axolet
rm -r /etc/axolet/

  2. Follow the regular installation steps described in Axolet.

Recover Axolet after the root CA certificate was rotated

  1. Log in to the host as root and execute the following commands:

    export AXOLET_KEEP_GUID=y AXOLET_CONFIG_OVERWRITE=y PATH=$PATH:/usr/local/bin

  2. Run the one-liner from the Axoflow Console Provisioning page in the same shell.

  3. The installation may result in an error message, but Axolet should eventually recover. You can check by running:

    journalctl -b -u axolet -n 20 -f

1.6 - Data sources

1.7 - Destinations

1.8 - Windows host - agent based solution

Axoflow provides a customized OpenTelemetry Collector distribution to collect data from Microsoft Windows hosts.

Prerequisites

The Axoflow OpenTelemetry Collector supports the following Windows versions:

  • Windows Server 2025
  • Windows Server 2022
  • Windows 11
  • Windows 10

Installation

  1. Download the installation package for your platform form the Assets section of the release. We provide MSI installers and binary releases for amd64 and arm64 architectures.

  2. Run the installer. The installer installs:

    • the collector agent (by default) to C:\Program Files\Axoflow\OpenTelemetry Collector\axoflow-otel-collector.exe, and
    • a default configuration file (C:\Program Files\Axoflow\OpenTelemetry Collector\config.yaml) that must be edited before use.

Configuration

If you have already installed the agent, complete the following steps to configure it.

  1. Open the configuration file (C:\Program Files\Axoflow\OpenTelemetry Collector\config.yaml).

  2. Set the IP address and port of the AxoRouter host where you want to send data from this Windows host. Use the IP address and port of the AxoRouter OpenTelemetry connector (for example, 10.0.2.2:4317). Here’s how to find the IP address of your AxoRouter. (By default, every AxoRouter has an OpenTelemetry connector enabled.)

    exporters:
  otlp/axorouter:
    endpoint: 10.0.2.2:4317
    tls:
      insecure: true

  3. (Optional) Customize the Event log sources. The default configuration collects data from the following channels:

    • application,
    • security,
    • system.

    To include additional channels:

    1. Add a new windowseventlog receiver under the receivers section, like this:

      receivers:
  windowseventlog/<CHANNEL_NAME>:
    channel: <CHANNEL_NAME>
    raw: true

    2. Include the new receiver in a pipeline under the service.pipelines section, for example:

      service:
  pipelines:
    logs/eventlog:
      receivers: [windowseventlog/application, windowseventlog/system, windowseventlog/security, windowseventlog/<CHANNEL_NAME>]
      processors: [resource/agent, resourcedetection/system]
      exporters: [otlp/axorouter]

  4. (Optional) Configure collecting DNS logs from the host.

    1. Check the path of the DNS log file by running with the following PowerShell command:

      (Get-DnsServerDiagnostics).LogFilePath

    2. Enter the path into the receivers.filelog/windows_dns_debug_log.include section of the configuration file. Note that you have to escape the backslashes in the path, for example, C:\\Windows\\System32\\DNS\\dns.log.

      receivers:
  filelog/windows_dns_debug_log:
    include: ['<ESCAPED_DNS_LOGFILE_PATH>']
    ...

  5. (Optional) Configure collecting DHCP logs from the host.

    1. Check the path of the DHCP log files by running with the following PowerShell command:

      (Get-DhcpServerAuditLog).Path

      DHCP server log files usually start with the DhcpSrvLog (for IPv4) or the DhcpV6SrvLog (for IPv6) prefixes.

    2. Enter the path of the IPv4 log files without the filename into the receivers.filelog/windows_dhcp_server_v4_auditlog.include section of the configuration file.

      Note that you have to escape the backslashes in the path, for example, C:\\Windows\\System32\\DHCP\\.

      receivers:
  filelog/windows_dhcp_server_v4_auditlog:
    include: ['<ESCAPED_DHCP_SERVER_LOGS_PATH>\\DhcpSrvLog*']
    ...

  filelog/windows_dhcp_server_v6_auditlog:
    include: ['<ESCAPED_DHCPV6_SERVER_LOGS_PATH>\\DhcpV6SrvLog*']
    ...

    3. Enter the path of the IPv6 log files without the filename into the receivers.filelog/windows_dhcp_server_v6_auditlog.include section of the configuration file.

      Note that you have to escape the backslashes in the path, for example, C:\\Windows\\System32\\DNS\\dns.log.

  6. Save the file.

  7. Restart the service.

    Restart-Service axoflow-otel-collector

    The agent starts sending data to the configured AxoRouter.

  8. Add the Windows host where you’ve installed the OpenTelemetry Collector to Axoflow Console as a data source.

    1. Open the Axoflow Console.

    2. Select Topology > + > Source.

      Add appliance as source

    3. Select Microsoft Windows as the type of the source.

      Select data source

    4. Set the IP address and the host name (FQDN) of the host.

    5. Select Create.

  9. Create a flow between the data source and the OpenTelemetry connector of AxoRouter. You can use the Select messages processing step (with the meta.connector.type = otlp and meta.product =* windows query) to route only the Windows data received by the AxoRouter OpenTelemetry connector to your destination.

    Route Windows data that’s received by the AxoRouter OpenTelemetry connector to your destination

Metadata fields

The AxoRouter connector adds the following fields to the meta variable:

field value
meta.connector.type otlp
meta.connector.name <name of the connector>
meta.product `windows

2 - Topology

Based on the collected metrics, Axoflow visualizes the topology of your security data pipeline. The topology allows you to get a semi-real-time view of how your edge-to-edge data flows, and drill down to the details and metrics of your pipeline elements to quickly find data transport issues.

Security data pipeline topology

Select the name of a source host or a router to show the details and metrics of that pipeline elements.

If a host has active alerts, it’s indicated on the topology as well.

Alert indicator on the Topology page

Traffic volume

Select bps or eps in the top bar to show the volume of the data flow on the topology paths in bytes per second or events per second.

Filter hosts

To find or display only specific hosts, you can use the filter bar.

  • Free Text mode searches in the following fields of the host: Name, IP Address, GUID, and FQDN.

  • AQL mode allows you to search in specific labels of the hosts. It also makes more complex filtering possible, using the Equal, Contains, and Match operators. When using AQL mode, Axoflow Console autocompletes the built-in host labels and field names, but doesn’t autocomplete custom labels.

    Tip: to delete an AQL block, use SHIFT+Backspace. You can quickly navigate between the blocks using Tab and SHIFT+Tab

Topology filter

If a filter is active, by default the Axoflow Console will display the matching hosts and the elements that are downstream from the matching hosts. For example, if an aggregator like AxoRouter matches the filter, only the aggregators and destinations where the matching host is sending data are displayed, the sources that are sending data to the matching aggregator aren’t shown. To display only the matching host without the downstream pipeline elements, select Filter strategy > Strict.

Grouping hosts

You can select labels to group hosts in the Group by field, so the visualization of large topologies with lots of devices remains useful. Axoflow Console automatically adds names to the groups. The following example shows hosts grouped by their region based on their location labels.

Group hosts

You can select multiple labels to group the hosts based on different parameters.

Queues

Select queue in the top bar to show the status of the memory and disk queues of the hosts. Select the status indicators of a host to display the details of the queues.

Queue details

The following details about the queue help you diagnose which connections of the host are having throughput or connectivity issues:

  • capacity: The total capacity of the buffer in bytes.
  • usage: How much of the total capacity is currently in use.
  • driver: The type of the AxoSyslog driver used for the connection the queue belongs to.
  • id: The identifier of the connection.

Disk-based queues also have the following information:

  • path: The location of the disk-buffer file on the host.
  • reliable: Indicates wether the disk-buffer is set to be reliable or not.
  • worker: The ID of the AxoSyslog worker thread the queue belongs to.

Both memory-based and disk-based queues have additional details that depend on the destination.

3 - Hosts

Axoflow collects and shows you a wealth of information about the hosts of your security data pipeline.

3.1 - Find a host

To find a specific host, you have the following options:

  • Open the Topology page, then click the name of the host you’re interested in. If you have many host and it’s difficult to find the one you need, use filtering, or grouping.

    Topology view

  • Open the Hosts page and find the host you’re interested in.

    Hosts list

    To find or display only specific hosts, you can use the filter bar.

    • Free Text mode searches in the following fields of the host: Name, IP Address, GUID, and FQDN.

    • AQL mode allows you to search in specific labels of the hosts. It also makes more complex filtering possible, using the Equal, Contains, and Match operators. When using AQL mode, Axoflow Console autocompletes the built-in host labels and field names, but doesn’t autocomplete custom labels.

      Tip: to delete an AQL block, use SHIFT+Backspace. You can quickly navigate between the blocks using Tab and SHIFT+Tab

    You can also select one or more labels (for example, location) to group hosts in the Group by field based on different parameters.

    Group hosts on the Hosts page

3.2 - Host information

The Hosts page contains a quick overview of every data source and data processor node. The exact information depends on the type of the host: hosts managed by Axoflow provide more information than external hosts.

Host information

The following information is displayed:

  • Hostname or IP address
  • Metadata labels. These include labels added automatically during the Axoflow curation process (like product name and vendor labels), as well as any custom labels you’ve assigned to the host.

For hosts that have the Axoflow agent (Axolet) installed:

  • The version on the agent
  • The name and version of the log collector running on the host (for example, AxoSyslog or Splunk Connect for Syslog)
  • Operating system, version, and architecture
  • Resource information: CPU and memory usage, disk buffer usage. Click on a resource to open its resource history on the Metrics & health page of the host.
  • Traffic information: volume of the incoming and outgoing data traffic on the host.

For AxoRouter hosts:

For more details about the host, select the hostname, or click ⋮.

Host overview

3.3 - Custom labels and metadata

To add custom labels to a host, complete the following steps. Note that these are static labels. To add labels dynamically based on the contents of the processed data, use the processing steps in data flows.

  1. Find the host on the Hosts or the Topology page, and click on its hostname. The overview of the host is displayed.

    Host overview

  2. Select Edit.

    Edit host metadata

    You can add custom labels in <label-name>:<value> format (for example, the group or department a source device belongs to), or a generic description about the host. You can use the labels for quickly finding the host on the Hosts page, and also for filtering when configuring Flows.

    When using labels in filters, processing steps, or search bars, note that:

    • Labels added to AxoRouter hosts get the axo_host_ prefix.
    • Labels added to data sources get the host_ prefix. For example, if you add a rack label to an edge host, it’ll be added to the data received from the host as host_rack.

    On other pages, like the Host Overview page, the labels are displayed without the prefixes.

  3. Select Save.

3.4 - Services

The Services page of a host shows information about the data collector or router services running on the host.

This page is only available for managed pipeline elements.

The following information is displayed:

  • Name: Name of the service
  • Version: Version number of the service
  • Type: Type of the service (which data collector or processor it’s running)
  • Supervisor: The type of the supervisor process. For example, Splunk Connect for Syslog (sc4s) runs a syslog-ng process under the hood.

Icons and colors indicate the status of the service: running, stopped, or not registered.

Data collector services running on the host

Service configuration

To check the configuration of the service, select Configuration. This shows the list of related environment variables and configuration files. Select a file or environment variable to display its value. For example:

Configuration file of the service

To display other details of the service (for example, the location of the configuration file or the binary), select Details.

Service details

Manage service

To reload a registered service, select Reload.

4 - Log tapping

Log tapping in Axoflow samples the log flow. You can use labels to filter for specific messages (like ones with parse errors) and tap only those messages. To not get overwhelmed with events, Axoflow automatically samples the output: if many messages match the selected filter, only a subset is shown (about 1 message per second). Using log tapping, you can quickly troubleshoot both parsing/curation errors and destination ingest (API) errors, and check:

  • What was in the original message?
  • What is sent in the final payload to the destination?

To see log tapping in action, check this blog post. To tap into your log flow, or to display the logs of the log collector agent, complete the following steps.

  1. Open the Topology page, then select the host where you want to tap the logs, for example, an AxoRouter deployment.

  2. Select ⋮ > Tap log flow.

    Open Log tapping

  3. Tap into the log flow.

    • To see the input data, select Input log flow > Start.
    • To see the output data, select Output log flow > Start.
    • To see the logs of the log collector agent running on the host, select Agent logs.

    You can use labels to filter the messages and sample only the matching ones.

    Configure Log tapping

  4. When the logs you’re interested in show up, click Stop Log Tap, then click a log message to see its details.

    Details of the log message

  5. If you don’t know what the message means, select AI Analytics to ask our AI to interpret it.

    AI interpretation of the log message

Filter the messages

You can add labels to the Filter By Label field to sample only messages matching the filter. If you specify multiple labels, only messages that match all filters will be sampled. For example, the following filter selects messages from a specific source IP, sent to a specific destination IP.

Using filters in log tapping

5 - Alerts

Axoflow raises alerts for a number of events in your security data pipeline, for example:

  • a destination becomes unavailable or isn’t accepting messages,
  • a host is dropping packets or messages,
  • a host becomes unavailable,
  • the disk queues of a host are filling up,
  • abandoned (orphaned) disk buffer files are on the host,
  • the configuration of a managed host failed to synchronize or reload,
  • traffic of a flow has unexpectedly dropped.

Alerts are indicated at a number of places:

  • On the Alerting page. Select an alert to display its details.

    Alert indicator on the Alerting page

  • On the Topology page:

    Alert indicator on the Topology page

  • On the Overview page of the host:

    Alert indicator on the Host Overview page

  • On the Metrics & Health page of the host the alerts are shown as an overlay to the metrics:

    Alert indicator on the Metrics & Health page

6 - Private connections

6.1 - Google Private Service Connect

If you want your hosts in Google Cloud to access the Axoflow Console without leaving the Google network, we recommend that you use Google Cloud Private Service Connect (PSC) to secure the connection from your VPC to Axoflow.

Prerequisites

Contact Axoflow and provide the list of projects so we can set up an endpoint for your PSC. You will receive information from us that you’ll need to properly configure your connection.

You will also need to allocate a dedicated IP address for the connection in a subnet that’s accessible for the hosts.

Steps

After you have received the details of your target endpoint from Axoflow, complete the following steps to configure Google Cloud Private Service Connect from your VPC to Axoflow Console.

  1. Open Google Cloud Console and navigate to Private Service Connect > Connected endpoints.

  2. Select the project you want to connect to Axoflow.

  3. Navigate to Connect endpoint and complete the following steps.

    1. Select Target > Published service.
    2. Set Target service to the service name you’ve received from Axoflow. The service name should be similar to: projects/axoflow-shared/regions/<region-code>/serviceAttachments/<your-tenant-ID>
    3. Set Endpoint name to the name you prefer, or the one recommended by Axoflow. The recommended service name is similar to: psc-axoflow-<your-tenant-ID>
    4. Select your VPC in the Network field.
    5. Set Subnet where the endpoint should appear. Since subnets are regional resources, select a subnet in the region you received from Axoflow.
    6. Select Create IP address and allocate an address for the endpoint. Save the address, you’ll need it later to verify that the connection is working.
    7. Select Enable global access.
    8. There is no need to enable the directory API even if it’s offered by Google.
    9. Select Add endpoint.

  4. Test the connection.

    1. Log in to a machine where you want to use the PSC using SSH.

    2. Test the connection. Run the following command using the IP address you’ve allocated for the endpoint.

      curl -vk https://<IP-address-allocated-for-the-endpoint>

      If the connection is established, you’ll receive an HTTP 404 response.

  5. If the connection is established, configure DNS resolution on the hosts. Complete the following steps.

Setting up selected machines to use the PSC

  1. Add the following entry to the /etc/hosts file of the machine.

    <IP-address-allocated-for-the-endpoint> <your-tenant-id>.cloud.axoflow.io kcp.<your-tenant-id>.cloud.axoflow.io telemetry.<your-tenant-id>.cloud.axoflow.io

  2. Run the following command to test DNS resolution:

    curl -v https://<your-tenant-id>.cloud.axoflow.io

    It should load an HTML page from the IP address of the endpoint.

  3. If the host is running axolet, restart it by running:

    sudo systemctl restart axolet.service

    Check the axolet logs to verify that there’re no errors:

    sudo journalctl -fu axolet

  4. Deploy the changes of the /etc/hosts file to all your VMs.

Setting up whole VPC networks to use the PSC

  1. Open Google Cloud Console and in the Cloud DNS service navigate to the Create a DNS zone page.

  2. Create a new private zone with the zone name <your-tenant-id>.cloud.axoflow.io, and select the networks you want to use the PSC in.

  3. Add the following three A records, all of which targeted to the <IP-address-allocated-for-the-endpoint>:

    • <your-tenant-id>.cloud.axoflow.io
    • kcp.<your-tenant-id>.cloud.axoflow.io
    • telemetry.<your-tenant-id>.cloud.axoflow.io