Azure Event Hubs

Collect data from Azure Event Hubs with a Cloud Connector, and look up the labels and field mappings for each log type.

Azure Event Hubs: Big data streaming platform to ingest and process events.

Axoflow can collect data from your Azure Event Hubs deployment with an Axoflow Cloud Connector. At a high level, the process looks like this:

Create a Connector Rule on the AxoRouter that deploys an Axoflow Cloud Connector and collects the data from your Event Hub. Axoflow Cloud Connector is a simple container that runs on an AxoRouter instance.
Configure a Flow on AxoConsole that processes and routes the collected data to your destination (for example, Splunk or another SIEM).

How Axoflow collects data from Azure Event Hubs with Axoflow Cloud Connector and AxoRouter

Prerequisites

An AxoRouter instance that:
- Was provisioned with Enable Cloudconnectors turned on.
- Can reach Azure Event Hubs on TCP port 5671. Adjust your firewall rules and the access rules of the data source as needed.
- You know the IP address the AxoRouter. To find it:
  1. Open the AxoConsole.
  2. Select the Routers or the Topology page.
  3. Select on AxoRouter instance that is going to receive the logs.
  4. Check the Networks > Address field.
An Azure account with an active subscription.
An Event Hubs connection string that includes a Shared Access Key (SAS token) and an EntityPath name.

Note To collect data from multiple EntityPaths, you must create a separate connector rule in AxoConsole for each EntityPath.

Configure Axoflow Cloud Connector on AxoRouter

To collect data from Azure Event Hubs, complete the following steps.

Select Routers > Connector Rules > Add Connector Rule. (Alternatively, you can select Add Connector > Add a connector rule on the Connectors page of an AxoRouter host.)
Select Azure Event Hubs.
Enter a name for the connector rule into the Connector rule name field.
(Optional) Add labels to the connector rule. You will be able to use these labels in Flow Processing steps, for example, in the Query field of Select Messages steps. For details about the message schema and the related connector-specific fields, see the meta.connector object.
Set the Router selector for the connector rule. The selector determines which AxoRouter instances get a connector from this connector rule.

CAUTION:

Each connector rule for Azure Event Hubs must match exactly one AxoRouter. Use the name or guid AQL selector to make sure it matches only one AxoRouter. Otherwise several Axoflow Cloud Connector instances collect the same data, and Axoflow receives every message multiple times.

You can use any labels and metadata of the AxoRouter hosts in the Router selectors, for example, the hostname of the AxoRouter, or any custom labels.
- If you leave the router selector empty, it matches every AxoRouter instance.
- To select only a specific AxoRouter instance, set the name field to the name of the instance as selector. For example, name = my-axorouter.
- If you set multiple fields in the selector, the selector will match only AxoRouter instances that match all elements of the selector. (There in an AND relationship between the fields.)
(Optional) Enter a Suffix for the connector rule. AxoConsole uses the suffix in the name of the connector instances that it creates on the AxoRouter hosts. The name of the connector instance follows the <name-of-AxoRouter>-<suffix> schema. For example, if the name of a matching AxoRouter instance is axorouter-1, and the suffix of the rule is custom, the name of the connector on that AxoRouter is axorouter-1-custom.

If the Suffix field is empty, AxoConsole uses the name of the connector rule instead. For example, if the name of the connector rule is otlp-1, the name of the connector is axorouter-1-otlp-1.
(Optional) Enter a description for the rule.
Enter your Azure Event Hubs connection string into the Connection string field.

The connection string must include a Shared Access Key (SAS token) and an EntityPath name.
(Optional) If needed, set the Consumer group of the data. By default, it’s set to $Default.
(Optional) If you fetch non-standard or custom data from Event Hubs, change Format to Raw.
Select Add. The Axoflow Cloud Connector starts forwarding logs to the AxoRouter instance.
Verify that the Axoflow Cloud Connector is running.
1. Select Routers, then select the AxoRouter you added the Axoflow Cloud Connector to.
2. Select Services and check that axoflow-cloudconnector.service is running without errors.
Create a Flow to route the data from the AxoRouter instance to a destination. You can use the Labels of this source to select its messages.
To fetch data from another EntityPath, repeat this procedure from step 1. Make sure that only one AxoRouter fetches data from each EntityPath.

Labels

Axoflow automatically adds the following labels to data collected from this source:

Analytics label	Message field	Value
`vendor`	`meta.vendor`	`microsoft`
`product`	`meta.product`	`azure-event-hubs`
`azure_event_hubs_category`	`log.body.category`	The log category of the event being logged.

You can use the labels as:

labels in Customize Graph on the Analytics page,
in the Filter labels field during log tapping.

You can use the message fields

in Flow Processing steps, for example, in the Query field of Select Messages steps,
in AQL expressions in the search bars.

Field mapping

The Axoflow Cloud Connector maps the fields of the Azure common log format to OpenTelemetry attributes. The azure.* attributes arrive as dotted keys in the message body, for example, azure.correlation.id becomes log.body["correlation.id"].

The following table shows which of these fields Axoflow uses when it processes and normalizes each log category:

Azure field	Message field	Audit	Graph activity	Provisioning	Signin
`category`	`log.body.category`	✓	✓	✓	✓
`operationName`	`log.body["operation.name"]`	✓	✓		✓
`operationVersion`	`log.body["operation.version"]`	✓	✓		✓
`correlationId`	`log.body["correlation.id"]`	✓	✓		✓
`tenantId`	`log.body["tenant.id"]`	✓	✓		✓
`durationMs`	`log.body.duration`	✓	✓		✓
`resultSignature`	`log.body["result.signature"]`	✓	✓		✓
`properties`	`log.body.properties`	✓	✓		✓
`identity`	`log.body.identity`	✓			✓
`resultType`	`log.body["result.type"]`	✓			✓
`location`	`log.body["cloud.region"]`		✓		✓
`callerIpAddress`	`log.body["network.peer.address"]`		✓		✓
`resultDescription`	`log.body["result.description"]`				✓

Note

Axoflow always sets the cloud.provider field to azure.
The properties field carries most of the data, for example, the signin logs read about 40 subfields from it, such as deviceDetail, status.errorCode, and location.geoCoordinates.

Event Hubs Audit logs labels

Analytics label	Message field	Value
`vendor`	`meta.vendor`	`microsoft`
`product`	`meta.product`	`azure-event-hubs`
`service`	`meta.service.name`	`audit`

Event Hubs Graph Activity logs labels

Analytics label	Message field	Value
`vendor`	`meta.vendor`	`microsoft`
`product`	`meta.product`	`azure-event-hubs`
`service`	`meta.service.name`	`graphactivity`

Event Hubs Provisioning logs labels

Analytics label	Message field	Value
`vendor`	`meta.vendor`	`microsoft`
`product`	`meta.product`	`azure-event-hubs`
`service`	`meta.service.name`	`provisioning`

Analytics label	Message field	Value
`vendor`	`meta.vendor`	`microsoft`
`product`	`meta.product`	`azure-event-hubs`
`service`	`meta.service.name`	`signin`

Sending data to Splunk

When you send the data collected from this source to Splunk, Axoflow uses the following sourcetype and index settings:

sourcetype	index
`mscs:azure:eventhub`	`azure-activity`

If the Axoflow classification doesn’t set the source field for the message automatically, and you haven’t set it in a flow processing step manually (by setting the meta.destination.splunk.source field), AxoRouter automatically sets the source to the name of the AxoRouter connector that received the message (for example, axorouter-syslog-tcp-514).

Sending data to Google SecOps

When sending the data collected from this source to a dynamic Google SecOps destination, Axoflow sets the following log type: AZURE_EVENTHUB.