Rehydrate data

Send selected data from AxoStore or AxoLake to a SIEM or another destination, for example, during an investigation.

Rehydrating data allows you to select a set of data stored in AxoStore and send it to any other destination. For example, that way you can send data stored in AxoLake to a SIEM for further analysis.

Start a new rehydration

To start a new rehydration job, complete the following steps.

  1. Select Rehydration > Start New Rehydration.

    Alternatively, run a search on the Search Logs page, then select Rehydrate. AxoConsole copies the query, the Store, the Router, and the time range of the search into the new rehydration job.

    Start rehydration from the Search Logs page

  2. (Optional) AxoConsole automatically sets a name for the job. Set a custom name for the job if needed. This name is also added to the metadata of the rehydrated logs in the meta.rehydration_name field in a pre-filled processing step, which can be removed or modified if desired.

  3. Specify which logs you want to rehydrate. Like on the Search Logs page, you can:

    • use Free-text or AQL Expression search
    • select logs from a specific Store or a specific Router
    • select logs from a specific time range.

    To check which logs match, select Search logs.

    Start new rehydration

  4. (Optional) Add processing steps if needed. To reuse the processing steps later, select Save as Custom Step.

  5. Select the Destination where you want to send the rehydrated data.

  6. Select Start New Rehydration to start the job.

    AxoConsole configures and starts the job. You can follow its progress on the Rehydration page.

    Rehydration in progress

For details on the possible performance impact of rehydration, see Performance considerations .

Manage rehydration jobs

All existing rehydration jobs are listed on the Rehydration page.

Rehydration jobs

For every job the following information is shown:

  • Name: The name of the job.
  • Status: In Progress, Completed, Canceled (stopped by a user), or Failed. For failed jobs, select See failure details. on the card to display the error.
  • The data selectors used for the job: Log selector, Logs from, Logs until, Store, and Router, as well as the number of processing steps (Process) and the Destination.
  • Estimated Remaining Time (only for running jobs).
  • Rehydration Events: The number of events sent to the destination, and the number of events that matched the selectors.
  • The progress bar of the job, with the Start time and the end time of the job. For running jobs, the end time is the Estimated end time. For canceled and failed jobs, it’s the Cancellation time or the Fail time.

You can perform the following actions on the jobs:

  • To cancel a running job, select Cancel on its card, then select Terminate. You can’t continue a canceled job, but you can clone it.
  • To clone a job (and optionally modify its parameters), select more_vert > Clone. For canceled jobs, you can also select Clone and Restart on the card.
  • To delete a job, select more_vert > Delete. You can’t delete a running job.

Performance considerations

Rehydration jobs use the same AxoRouter destinations that other flows might use. For example, if you’re sending data to Splunk in a flow, and you’re also rehydrating data to the same Splunk destination, then these operations use the same resources (like buffers and bandwidth). As a result, running rehydration on an AxoRouter under heavy load may cause backpressure towards the sources, and slow down data ingestion.

If the AxoRouter host has enough free resources, you can create a duplicate of the destination with a lower number of workers, and use that in the rehydration job. This can help to prevent rehydration jobs causing issues with normal workloads.