Sumo Logic

Sumo Logic: Cloud-native platform for log management, metrics analysis, and security analytics.

To add a Sumo Logic destination to Axoflow, complete the following steps.

Prerequisites

  • A Sumo Logic account.

  • A HTTP Hosted Collector configured in the Sumo Logic service.

    To configure a Hosted Collector, follow the configuration instructions under the Configure a Hosted Collector section on the official Sumo Logic website.

  • The unique HTTP collector code you receive while configuring your Host Collector for HTTP requests.

  • To add custom fields of the messages to Sumo Logic:

    1. These fields must already exists in your Sumo Logic account. For details, see the Sumo Logic documentation.
    2. You must configure the HTTP Hosted Collector to process the received fields.

Steps

  1. Create a new destination.

    1. Open the Axoflow Console.
    2. Select Topology.
    3. Select + Create New Item > Destination.

  2. Configure the destination.

    1. Select Sumo Logic.

    2. Enter a name for the destination.

      Configure the Sumo Logic destination

    3. Select which type of configuration you want to use:

      • Simple: Send all data using a single source host and source category.
      • Dynamic: Set source host, name, and category based on the content or metadata of the incoming messages.
      • Advanced: Send all data to a collector with custom URL using a single source host and source category. Specifying the full URL might be needed for example if your endpoint uses a DNS alias.

    4. Configure the endpoint of the destination.

      • Simple:

        1. Enter the ID of your Sumo Logic Collector that groups and manages data sources. All data will be sent to this collector. The URL will be constructed like this: https://[endpoint]/receiver/v1/http/[collectorId].
        2. Enter the host part of the collector URL into the Endpoint field. All data will be sent to this endpoint.
        3. Enter the name of the Source Name. All data will be associated as coming from this source.
        4. Enter the host identifier (for example, server or container name) that sent the data into the Source Host field. All data will be associated as coming from this source.
        5. Enter the name of the Source Category. The data will be added to this category in Sumo Logic.

        6. Enter the metadata tags that to be assigned to your data by default into the Fields field. Use key-value pairs. Other metadata can be added during the processing of the message (based on automatic classification, or by the processing steps of the Flow). Note the following limits of Sumo Logic fields:

          • An HTTP request is limited to 30 fields.
          • A field name (key) cannot be longer than 255 characters.
          • A value cannot be longer than 200 characters.
          • You must configure your Sumo Logic instance to accept and process the received fields.

      • Dynamic:

        1. Enter the ID of your Sumo Logic Collector that groups and manages data sources. The URL will be constructed like this: https://[endpoint]/receiver/v1/http/[collectorId].
        2. Enter the host part of the collector URL into the Endpoint field. All data will be sent to this endpoint.
        3. Enter the Default Source Name. The name of the data source. The data will be associated as coming from this source, unless a different one is set during the processing of the message (based on automatic classification, or by the processing steps of the Flow).
        4. Enter the Default Source Host. The host identifier (for example, server or container name) that sent the data. All data will be associated as coming from this source, unless a different one is set during the processing of the message (based on automatic classification, or by the processing steps of the Flow).
        5. Enter the name of the Default Source Category. This is a user-defined tag to logically group or filter data in Sumo Logic. The data will added to this category, unless a different one is set during the processing of the message (based on automatic classification, or by the processing steps of the Flow).

        6. Enter the metadata tags that to be assigned to your data by default into the Fields field. Use key-value pairs. Other metadata can be added during the processing of the message (based on automatic classification, or by the processing steps of the Flow). Note the following limits of Sumo Logic fields:

          • An HTTP request is limited to 30 fields.
          • A field name (key) cannot be longer than 255 characters.
          • A value cannot be longer than 200 characters.
          • You must configure your Sumo Logic instance to accept and process the received fields.

      • Advanced:

        1. Enter the URL of the HTTPS endpoint of your Sumo Logic HTTP Source where data is sent.
        2. Enter the name of the Source Name. All data will be associated as coming from this source.
        3. Enter the host identifier (for example, server or container name) that sent the data into the Source Host field. All data will be associated as coming from this source.
        4. Enter the name of the Source Category. The data will be added to this category in Sumo Logic.

        5. Enter the metadata tags that to be assigned to your data by default into the Fields field. Use key-value pairs. Other metadata can be added during the processing of the message (based on automatic classification, or by the processing steps of the Flow). Note the following limits of Sumo Logic fields:

          • An HTTP request is limited to 30 fields.
          • A field name (key) cannot be longer than 255 characters.
          • A value cannot be longer than 200 characters.
          • You must configure your Sumo Logic instance to accept and process the received fields.

    5. (Optional) Set other options as needed for your environments.

      • Timeout: The number of seconds to wait for a log-forwarding request to complete, and attempt to reconnect the server if exceeded. If the timeout is exceeded, AxoRouter attempts to reconnect the destination. The default (0) is unlimited. For more details, see the AxoSyslog documentation.
      • Batch Bytes: Sets the maximum size of payload in a batch. If the size of the messages reaches this value, AxoRouter sends the batch to the destination even if the number of messages is less than the value of the batch lines option.
      • Batch Lines: Number of lines sent to the destination in one batch. AxoRouter waits for this number of lines to accumulate and sends them off in a single batch. Increasing this number increases throughput as more messages are sent in a single batch, but also increases message latency. For more details, see the AxoSyslog documentation.
      • Batch Timeout: Maximal time in milliseconds to wait for a batch to be filled before sending it. The default value (-1) means that the batch should be full before sending, which may result in a long wait if the incoming traffic is low. For more details, see the AxoSyslog documentation.
      • Number of Workers: Used for scaling the destination in case of high message load. Specifies the number of worker threads AxoRouter uses for sending messages to the destination. The default is 1. If high message throughput is not a concern, leave it on 1. For maximum performance, increase it up to the number of CPU cores available on AxoRouter. For more details, see the AxoSyslog documentation.

  3. Create a flow to connect the new destination to an AxoRouter instance.

    1. Select Flows.

    2. Select Create New Flow.

    3. Enter a name for the flow, for example, my-test-flow.

      Create a flow

    4. In the Router Selector field, enter an expression that matches the router(s) you want to apply the flow. To select a specific router, use a name selector, for example, name = my-axorouter-hostname.

      It also makes more complex filtering possible, using the Equals, Contains (partial match), and Match (regular expression match) operators. Note that:

      • To execute the search, click Search, or hit ESC then ENTER.
      • Axoflow Console autocompletes the built-in and custom labels and field names, as well as their most frequent values, but doesn’t autocomplete labels and variables created by data parsing and processing steps.
      • You can use the AND and OR operators to combine expressions, and also parenthesis if needed. For details on AQL, see AQL operator reference.
      • The precedence of the operators is the following: parentheses, AND, OR, comparison operators.
      • Use the usual keyboard shortcuts to undo (⌘/Ctrl + Z) or redo (⌘/Ctrl + Shift + Z) your edits.

    5. Select the Destination where you want to send your data. If you don’t have any destination configured, see Destinations.

      By default, you can select only external destinations. If you want to send data to another AxoRouter, enable the Show all destinations option, and select the connector of the AxoRouter where you want to send the data.

      AxoRouter as destination

    6. (Optional) To process the data transferred in the flow, select Add New Processing Step. For details, see Processing steps. For example:

      1. Add a Classify, a Parse, and a Reduce step, in that order, to automatically remove redundant and empty fields from your data.
      2. To select which messages are processed by the flow, add a Select Messages step, and enter a filter into the Query field. For example, to select only the messages received from Fortinet FortiGate firewalls, use the meta.vendor = fortinet AND meta.product = fortigate query.
      3. Save the processing steps.

      Example processing steps

    7. Select Create.

    8. The new flow appears in the Flows list.

      The new flow