OpenObserve

To add an OpenObserve destination to Axoflow, complete the following steps.

Prerequisites

• An OpenObserve account for Axoflow, or
• a self-hosted OpenObserve deployment.
• To configure Axoflow, you’ll need the username, password, the name of your organization, and the name of the OpenObserve stream where you want to send your data.

Steps

1. Create a new destination.

   1. Open the Axoflow Console.
   2. Select Topology.
   3. Select + > Destination.

2. Configure the destination.

   1. Select OpenObserve.

   2. Select which type of configuration you want to use:

      • Simple: Send all data into a single stream of an organization.
      • Dynamic: Send data to an organization and stream based on the content or metadata of the incoming messages (or to the default stream of the default organization).
      • Advanced: Allows you to specify a custom URL endpoint.

   3. Enter a name for the destination.

      

   4. Configure the endpoint of the destination.

      • Advanced: Enter your OpenObserve URL into the URL field, for example, https://example.com/api/my-org/my-stream/_json
      • Simple and Dynamic:
        1. Select the HTTPS or HTTP protocol to use to access your destination.
        2. Enter the Hostname and Port of the destination.

   5. Specify the OpenObserve stream to send the data to.

      • Simple: Enter name of the Organization and the Stream where you want to send the data. All data will be sent into this stream.
      • Dynamic: Enter the expression that specifies the default Default Organization and teh Default Stream. The data will be sent into this stream if no other organization and stream is set during the processing of the message (for example, by the processing steps of the Flow).

      You can use AxoSyslog macros in this field.

   6. Enter the username and password for the account you want to use.

   7. (Optional)

      By default, Axoflow rejects connections to the destination server if the certificate of the server is invalid (for example, it’s expired, signed by an unknown CA, or its CN and the name of the server is mismatched).

      Warning When validating a certificate, the entire certificate chain must be valid, including the CA certificate. If any certificate of the chain is invalid, Axoflow will reject the connection.

      If you want to accept invalid certificates (or no certificate) from the destination servers, disable the Verify server certificate option.

   8. (Optional) Set other options as needed for your environments.

      • Timeout: The number of seconds to wait for a log-forwarding request to complete, and attempt to reconnect the server if exceeded. The default (0) is unlimited. For more details, see the AxoSyslog documentation.
      • Batch Timeout: Maximal time in milliseconds to wait for a batch to be filled before sending it. The default value (-1) means that the batch should be full before sending, which may result in a long wait if the incoming traffic is low. For more details, see the AxoSyslog documentation.
      • Workers: Used for scaling the destination in case of high message load. Specifies the number of worker threads AxoRouter uses for sending messages to the destination. The default is 1. If high message throughput is not a concern, leave it on 1. For maximum performance, increase it up to the number of CPU cores available on AxoRouter. For more details, see the AxoSyslog documentation.
3. Create a flow to connect the new destination to an AxoRouter instance.

   1. Select Flows.

   2. Select Create New Flow.

   3. Enter a name for the flow, for example, my-test-flow.

      

   4. In the Router Selector field, enter an expression that matches the router(s) you want to apply the flow. To select a specific router, use a name selector, for example, name = my-axorouter-hostname.

   5. Select the Destination where you want to send your data. If you don’t have any destination configured, see Destinations.

      By default, you can select only external destinations. If you want to send data to another AxoRouter, enable the Show all destinations option, and select the connector of the AxoRouter where you want to send the data.

      

   6. (Optional) To process the data transferred in the flow, select Add New Processing Step. For details, see Processing steps. For example:

      1. Add a Reduce step to automatically remove redundant and empty fields from your data.
      2. To select which messages are processed by the flow, add a Select Messages step, and enter a filter into the Query field. For example, to select only the messages received from Fortinet FortiGate firewalls, use the meta.vendor = fortinet + meta.product = fortigate query.
      3. Save the processing steps.

      

   7. Select Create.

   8. The new flow appears in the Flows list.