Axoflow Zero to Hero: Stream Security Data Anywhere

Today we're going to show you how to be a hero by connecting machines and logging data to your analytics tool of choice in 12 minutes or less using the Axoflow Platform.

What you'll need

To follow the steps shown in the video, you'll need:

A virtual machine
Evaluation access to Axoflow. You can submit an evaluation request, we'll set it up for you within a business day.
Access to Splunk and a token to send data to Splunk, or another destination supported by Axoflow.

Install AxoRouter

The first step in testing the Axoflow Console is to install AxoRouter. AxoRouter is the curation solution that collects, aggregates, transforms, and routes all kinds of security data automatically – at carrier-grade scale.

Open your Axoflow Console, and select the Provisioning page.
Click Select type and platform > AxoRouter > Linux > Copy and close.
Open a terminal on your VM, then paste the install command.
Reload the Provisioning page. The new AxoRouter deployment shows up.
Click the check mark.
Add a custom label to your AxoRouter so you'll know which team it belongs to, then click Register.

After you register an AxoRouter deployment, you can see which operating system that you're running on, and some other details of the host.

Send data to AxoRouter

If you have a data source handy, configure it to send logs to AxoRouter. If you don’t, open a terminal on your VM running AxoRouter, and run the following command to generate some synthetic data.

Note that AxoRouter collects detailed, real-time metrics about the data-flows – giving you observability over the health of the security data pipeline and its components. Your security data remains in your self-managed cloud or in your on-prem instance where your sources, destinations, and AxoRouters are running, only metrics are forwarded to Axoflow Console.

for i in `seq 1 120`; do echo "<165> fortigate date=$(date -u +%Y-%m-%d) time=$(date -u +"%H:%M:%S%Z") devname=us-east-1-dc1-a-dmz-fw devid=FGT60D4614044725 logid=0100040704 type=event subtype=system level=notice vd=root logdesc=\"System performance statistics\" action=\"perf-stats\" cpu=2 mem=35 totalsession=61 disk=2 bandwidth=158/138 setuprate=2 disklograte=0 fazlograte=0 msg=\"Performance statistics: average CPU: 2, memory: 35, concurrent sessions: 61, setup-rate: 2\""; sleep 0.5; echo "<165>  id=us-west-1-dc1-a-dmz-fw sn=C0EFE3336C80 time=\"$(date -u +"%Y-%m-%d %H:%M:%S %Z")\" fw=192.168.1.239 pri=6 c=1024 gcat=6 m=537 msg=\"Connection Closed\" srcMac=00:50:56:f5:50:27 src=10.237.228.74:54406:X20 srcZone=Trusted natSrc=192.168.1.239:38377 dstMac=00:1a:f0:8b:e0:18 dst=44.190.129.212:123:X2 dstZone=Untrusted natDst=44.190.129.212:123 proto=udp/ntp sent=152 rcvd=152 spkt=2 rpkt=2 cdur=30250 rule=\"22 (LAN->WAN)\" n=490872197 fw_action=\"NA\" dpi=0"; sleep 0.5; echo "<165>$(date -u +"%b%e %H:%M:%S") us-east-1-dc1-b-edge-fw 1,$(date -u +"%Y/%m/%d %H:%M:%S"),007200001056,TRAFFIC,end,1,$(date -u +"%Y/%m/%d %H:%M:%S"),192.168.41.30,192.168.41.255,10.193.16.193,192.168.41.255,allow-all,,,netbios-ns,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,To-Panorama,$(date -u +"%Y/%m/%d %H:%M:%S"),8720,1,137,137,11637,137,0x400000,udp,allow,276,276,0,3,$(date -u +"%Y/%m/%d %H:%M:%S"),2,any,0,2800265,0x0,192.168.0.0-192.168.255.255,192.168.0.0-192.168.255.255,0,3,0"; sleep 0.5; done | nc -v 127.0.0.1 514

Wait a few seconds to give some time for the metrics to accumulate.

Check analytics data

Select Analytics (if you've navigated away, select Topology > your-AxoRouter> first)
If your AxoRouter is receiving data, some metrics should be visible. AxoRouter automatically classifies and parses the incoming data, and adds vendor and product labels to the data it recognizes.

One of the cool things you'll see is that there are a number of metrics that we collect in the AxoRouter, in the collector itself. First and foremost, you'll see that we are detecting what product is sending this particular set of data.

If you change back to the Overview tab, you can see that Axoflow raised a warning for this AxoRouter, because there are no flows configured for this particular router: right now we're collecting the data, but we're not sending it anywhere. Let's fix that by creating a destination.

Create destination

Create a Splunk destination. If you don’t want to use Splunk, use another destination supported by Axoflow.

Select Topology > + > Destination > Splunk.
Enter the required parameters. You'll need the URL of your Splunk deployment, and an access token that allows you to send data (you can get these from your Splunk administrator).
Click Create.

Create a flow

Create a flow to route the data received by AxoRouter to your Splunk destination.

Select Flows > Create New Flow.
Enter a name for the flow.
In the Router Selector field select the name of your AxoRouter.
Set the Destination field to your Splunk destination, then click Create.

Add a source

If you go back to the Topology page, you're going to see that there is a connection between your AxoRouter and your destination. To visualize the sources, register the sources that are sending data to the AxoRouter.

Select Topology > + > Source > Detected. The list of sources that are currently sending data and AxoRouter has automatically recognized are displayed.
NOTE: If you don't have separate source hosts and you're sending data with the script provided above, only a single source will be detected, since all data is coming from the same IP address. However, it will show up as three different hosts and source types, because AxoRouter classifies the incoming messages based on their content.
Select a source, then add a custom label to the source if you'd like to.

Once everything is set up, the source and metrics about the data it's sending shows up on the Topology page. So in addition to the deeper level analytics, you get a high level view of what is happening on the topology level.

Check output metrics

Let’s take a look at output events:

Click on your AxoRouter and select Analytics, then change the Input events to Output events.

Now that AxoRouter is receiving data from a source and a destination to forward it, you'll see that the output metrics are also populated. In addition to the metrics collected earlier, such as the destination port and the transport, now you see another metrics dimension called Splunk source type. Since we selected Splunk as a destination, Axoflow automatically sets the source type to the right values.

For example, using labels you can check how traffic is distributed based on splunk_sourcetype, or to which index data is sent using the splunk_index label.

So now let's go look at Splunk itself and see what this data looks like.

Check data in Splunk

Since AxoRouter takes care of parsing, classifying, and sending the data to Splunk, you don't need to do that on the Splunk side: you don’t need technology add-ons or parsers in Splunk. You can simply use the structured data and metadata that AxoRouter provides to dig into your data, create charts and graphs, and gain insights.

Summary

If you’ve successfully followed this blog or the related video, you'll already have some useful data in your analytics tool. We spent less than 12 minutes doing all this, and didn’t need to invoke or write a regular expression parser the whole time.