Process orphan disk-buffer files
Purpose
When AxoSyslog creates orphan disk-buffer files, you can start a separate AxoSyslog instance parallel to the AxoSyslog instance already running, and use the following procedure to process the messages in the orphan disk-buffer file.
Orphan disk-buffer files
In certain situations (for example, after modifying the disk-buffer configuration or losing the persist information), AxoSyslog creates a new disk-buffer file instead of using the already existing one. In these situations, the already existing disk-buffer file becomes a so-called orphan disk-buffer file.
Steps
Before starting a separate AxoSyslog instance to process the messages from the orphan disk-buffer file, consider the following:
- During the resolution process, a separate AxoSyslog instance will be started with its temporary files beside the AxoSyslog instance already running.
- An incorrect startup command and incorrect configurations may cause issues for the AxoSyslog instance already running.
- The disk-buffer file stores processed log messages in the format in which they would have been sent out to the destination.
- The disk-buffer file doesn’t store information about the destination.
-
Identify the orphan disk-buffer files and make a record of them. For more information, see How to get information about disk-buffer files.
It is important to know the type of the disk-buffer file. Disk-buffer file types can be normal (
.qf
) or reliable (.rqf
). In the examples during this process, the/opt/syslog-ng/var/syslog-ng-00005.rqf
orphan reliable disk-buffer file is used. -
Determine the destination of the logs. The content of the disk-buffer may help you determine the logs’ destination. For more information, see How to get information about disk-buffer files.
In the examples the destination
10.21.10.20
is used with the standardnetwork()
port514
. -
Create a directory for the temporary instance. In the examples during this process, the
/tmp/qdisk
directory is used.mkdir /tmp/qdisk
Warning Make sure that there is sufficient disk space in the directory. The minimum recommended disk space in the directory is equal to the size of the orphan disk-buffer file. -
Create the configuration file
/tmp/qdisk/qdisk.conf
for the temporary instance with the following content.@version:4.9.0 @include "scl.conf" options { keep-hostname(yes); keep-timestamp(yes); }; destination d_destination { # ADD YOUR DESTINATION HERE }; log { destination(d_destination); };
-
Add your destination statement with
disk-buffer()
to the configuration file. You can copy the destination statement from your running AxoSyslog configuration.Warning Add thedir()
option and set the disk-buffer file’s destination directory to the temporary directory (that is,/tmp/qdisk
) in your destination statement. -
Start the temporary AxoSyslog instance in the foreground.
syslog-ng -Fe -f /tmp/qdisk/qdisk.conf -R /tmp/qdisk/qdisk.persist -c /tmp/qdisk/qdisk.ctl
The AxoSyslog application will log to the console, so you will see any potential error that may occur during startup.
The following example output displays that an empty disk-buffer file has been created and the connection to the remote destination has been established.
Follow-mode file source not found, deferring open; filename='/no_such_file_or.dir' Reliable disk-buffer state saved; filename='/tmp/qdisk/syslog-ng-00000.rqf', qdisk_length='0' No server license found, running in client mode; syslog-ng starting up; version='7.0.20', cfg-fingerprint='eaa03b9efb88b87d7c1b0ce7efd042ed8ac0c013', cfg-nonce-ndx='0', cfg-signature='c0327a7f7e6418ce0399a75089377dfb662bb072' FIPS information; FIPS-mode='disabled' Syslog connection established; fd='7', server='AF_INET(10.21.10.20:514)', local='AF_INET(0.0.0.0:0)'
-
To stop AxoSyslog, press
CTRL+C
. -
Overwrite the empty disk-buffer file with the orphan disk-buffer file.
mv /opt/syslog-ng/var/syslog-ng-00005.rqf /tmp/qdisk/syslog-ng-00000.rqf
-
Start AxoSyslog using the command used in Start the temporary AxoSyslog instance in the foreground step.
syslog-ng -Fe -f /tmp/qdisk/qdisk.conf -R /tmp/qdisk/qdisk.persist -c /tmp/qdisk/qdisk.ctl
-
Open another terminal and check the progress by using one of the following methods.
-
Checking the number of stored logs in the disk-buffer (that is, the last number from the output).
/opt/syslog-ng/sbin/syslog-ng-ctl stats -c /tmp/qdisk/qdisk.ctl | grep 'dst.*queued'
-
Checking the status of the disk-buffer file.
/opt/syslog-ng/bin/dqtool info /tmp/qdisk/syslog-ng-00000.rqf
An empty disk-buffer file will look similar to this:
Reliable disk-buffer state loaded; filename='/tmp/qdisk/syslog-ng-00000.rqf', queue_length='0', size='0'
-
-
Press
CTRL+C
to stop AxoSyslog. -
Check the state of the orphan disk-buffer file. For more information, see How to get information about disk-buffer files.
-
If you have more than one orphan disk-buffer file, repeat the steps following the AxoSyslog stop (that is, the steps beginning from overwriting the empty disk-buffer file with the orphan disk-buffer file) for each orphan disk-buffer file.
-
Remove the temporary directory.
rm -rf /tmp/qdisk