This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

How to get information about disk-buffer files

Purpose

This section describes how to get information about disk-buffer files used in AxoSyslog.

1 - Information about disk-buffer files

This section describes information about disk-buffer files used in AxoSyslog.

The following list contains information about how disk-buffer files are used in AxoSyslog:

  • You can configure disk-buffer() for a remote destination in the destination() statement.

    For more information about an example of configuring disk-buffer() for a remote destination in the destination() statement, see disk-buffer().

  • By default, AxoSyslog creates disk-buffer files under /opt/syslog-ng/var directory, unless dir() option is set in disk-buffer().

  • The filenames are generated automatically by AxoSyslog with the extensions .qf for a normal disk-buffer and .rqf for a reliable disk-buffer.

  • The disk-buffer file stores processed log messages in the format in which they would have been sent out to the destination, but doesn’t store information about the destination.

2 - Getting the status information of disk-buffer files

This section describes how to get status information of the disk-buffer files used in AxoSyslog.

Command syntax

The basic command syntax for getting the status information of the disk-buffer files used in AxoSyslog looks like the following:

/opt/syslog-ng/bin/dqtool info <DISK-BUFFER_FILE>

Example commands and outputs

  • Empty, normal disk-buffer file

    /opt/syslog-ng/bin/dqtool info /opt/syslog-ng/var/syslog-ng-00000.qf/var/lib/syslog-ng/syslog-ng-00000.qf
    

    Example output:

    Disk-buffer state loaded; filename='/opt/syslog-ng/var/syslog-ng-00000.qf/var/lib/syslog-ng/syslog-ng-00000.qf', number_of_messages='0'
    
  • Non-empty, reliable disk-buffer queue file

    /opt/syslog-ng/bin/dqtool info /opt/syslog-ng/var/syslog-ng-00000.rqf
    
    Reliable disk-buffer state loaded; filename='/opt/syslog-ng/var/syslog-ng-00000.rqf', number_of_messages='10'
    

List disk-buffer state in the default directory

You can use the following one-liner command to get the state of all disk-buffer files that are stored in the default directory:

for qfile in /opt/syslog-ng/var/*.?(r)qf ; do /opt/syslog-ng/bin/dqtool info $qfile 2>&1 ; done

3 - Getting the list of disk-buffer files

Purpose

This section describes getting the list of disk-buffer files used in AxoSyslog.

The AxoSyslog application stores information (namely, the IP:PORT or DNS:PORT of the destinations, and the name of the disk-buffer file) about disk-buffer files in its persist file.

Example: command for listing the disk-buffer files in use

The following command will list the disk-buffer files in use:

   /opt/syslog-ng/bin/persist-tool dump /opt/syslog-ng/var/syslog-ng.persist/var/lib/syslog-ng/syslog-ng.persist | awk -F '["=]' '/(qfile\(|\.queue)/ { gsub(/[ \t]+/, "", $5); gsub(/^[0-9A-Fa-f]{8}/, "", $5); "echo "$5"|xxd -r -p"|& getline QUEUE; printf("%s ==> %s\n",$1,QUEUE)}'

The example output will look like the following:

   afsocket_dd_qfile(stream,10.21.10.20:601)  ==> /opt/syslog-ng/var/syslog-ng-00000.rqf

4 - Printing the content of disk-buffer files

Purpose

This section describes printing the content of the disk-buffer files used in AxoSyslog.

Command syntax

The command syntax for printing the content of the disk-buffer files used in AxoSyslog looks like the following:

   /opt/syslog-ng/bin/dqtool cat DISK-BUFFER_FILE

Short example output for printed content

Example: short output that shows the printed content of the disk-buffer files used in AxoSyslog

The following short output example shows the printed content of the disk-buffer files used in AxoSyslog:

   /opt/syslog-ng/bin/dqtool cat /opt/syslog-ng/var/syslog-ng-00000.rqf
    
    Reliable disk-buffer state loaded; filename='/opt/syslog-ng/var/syslog-ng-00000.rqf', queue_length='2952', size='-437712'
    Jul 31 12:33:48.226 10.21.10.10 <382019-07-31T12:33:36 localhost prg00000[1234]: seq: 0000000838, thread: 0000, runid: 1564569216, stamp: 2019-07-31T12:33:36 PADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADD
    ...

5 - Orphan disk-buffer files

Purpose

This section describes orphan disk-buffer files used in AxoSyslog.

Orphan disk-buffer files

In certain situations (for example, after modifying the disk-buffer configuration or losing the persist information), AxoSyslog creates a new disk-buffer file instead of using the already existing one. In these situations, the already existing disk-buffer file becomes a so-called orphan disk-buffer file.

Discovering the new disk-buffer files (orphan disk-buffer files)

To discover orphan disk-buffer files, get the list of disk-buffer files from the persist file, then compare the list with the contents of the disk-buffer files’ saving directory.

For more information about how you can get the list of disk-buffer files from the persist file, see Getting the list of disk-buffer files).

Example: difference between the list of disk-buffer files from the persist file and the content of the disk-buffer files’ saving directory

The following examples show the difference between the list of disk-buffer files from the persist file and the content of the disk-buffer files’ saving directory.

Disk-buffer file list from persist file:

   afsocket_dd_qfile(stream,10.21.10.112:514) = { "queue_file": "/opt/syslog-ng/var/syslog-ng-00001.rqf" }

Disk-buffer files’ saving directory content:

   # ls -l /opt/syslog-ng/var//var/lib/syslog-ng/*qf
    -rw------- 1 root root 2986780 Jul 31 12:30 /opt/syslog-ng/var/syslog-ng-00000.qf/var/lib/syslog-ng/syslog-ng-00000.qf
    -rw------- 1 root root 2000080 Jul 31 12:31 /opt/syslog-ng/var/syslog-ng-00000.rqf
    -rw------- 1 root root    4096 Aug  1 11:09 /opt/syslog-ng/var/syslog-ng-00001.rqf

The disk-buffer files syslog-ng-00000.qf and syslog-ng-00000.rqf don’t exist in the persist file. These two files are the orphan disk-buffer files.

For more information about orphan disk-buffer files and how to process the messages in orphan disk-buffer files using a separate AxoSyslog instance, see Process orphan disk-buffer files.

6 - Process orphan disk-buffer files

Purpose

When AxoSyslog creates orphan disk-buffer files, you can start a separate AxoSyslog instance parallel to the AxoSyslog instance already running, and use the following procedure to process the messages in the orphan disk-buffer file.

Orphan disk-buffer files

In certain situations (for example, after modifying the disk-buffer configuration or losing the persist information), AxoSyslog creates a new disk-buffer file instead of using the already existing one. In these situations, the already existing disk-buffer file becomes a so-called orphan disk-buffer file.

Steps

  1. Identify the orphan disk-buffer files and make a record of them. For more information, see How to get information about disk-buffer files.

    It is important to know the type of the disk-buffer file. Disk-buffer file types can be normal (.qf) or reliable (.rqf). In the examples during this process, the /opt/syslog-ng/var/syslog-ng-00005.rqf orphan reliable disk-buffer file is used.

  2. Determine the destination of the logs. The content of the disk-buffer may help you determine the logs’ destination. For more information, see How to get information about disk-buffer files.

    In the examples the destination 10.21.10.20 is used with the standard network() port 514.

  3. Create a directory for the temporary instance. In the examples during this process, the /tmp/qdisk directory is used.

    mkdir /tmp/qdisk
    
  4. Create the configuration file /tmp/qdisk/qdisk.conf for the temporary instance with the following content.

    @version:4.9.0
    @include "scl.conf"
    
    options {
        keep-hostname(yes);
        keep-timestamp(yes);
    };
    
    destination d_destination {
    #    ADD YOUR DESTINATION HERE
    
    };
    
    log {
        destination(d_destination);
    };
    
  5. Add your destination statement with disk-buffer() to the configuration file. You can copy the destination statement from your running AxoSyslog configuration.

  6. Start the temporary AxoSyslog instance in the foreground.

    syslog-ng -Fe -f /tmp/qdisk/qdisk.conf -R /tmp/qdisk/qdisk.persist -c /tmp/qdisk/qdisk.ctl
    

    The AxoSyslog application will log to the console, so you will see any potential error that may occur during startup.

    The following example output displays that an empty disk-buffer file has been created and the connection to the remote destination has been established.

    Follow-mode file source not found, deferring open; filename='/no_such_file_or.dir'
    Reliable disk-buffer state saved; filename='/tmp/qdisk/syslog-ng-00000.rqf', qdisk_length='0'
    No server license found, running in client mode;
    syslog-ng starting up; version='7.0.20', cfg-fingerprint='eaa03b9efb88b87d7c1b0ce7efd042ed8ac0c013', cfg-nonce-ndx='0', cfg-signature='c0327a7f7e6418ce0399a75089377dfb662bb072'
    FIPS information; FIPS-mode='disabled'
    Syslog connection established; fd='7', server='AF_INET(10.21.10.20:514)', local='AF_INET(0.0.0.0:0)'
    
  7. To stop AxoSyslog, press CTRL+C.

  8. Overwrite the empty disk-buffer file with the orphan disk-buffer file.

    mv /opt/syslog-ng/var/syslog-ng-00005.rqf /tmp/qdisk/syslog-ng-00000.rqf
    
  9. Start AxoSyslog using the command used in Start the temporary AxoSyslog instance in the foreground step.

    syslog-ng -Fe -f /tmp/qdisk/qdisk.conf -R /tmp/qdisk/qdisk.persist -c /tmp/qdisk/qdisk.ctl
    
  10. Open another terminal and check the progress by using one of the following methods.

    • Checking the number of stored logs in the disk-buffer (that is, the last number from the output).

      /opt/syslog-ng/sbin/syslog-ng-ctl stats -c /tmp/qdisk/qdisk.ctl | grep 'dst.*queued'
      
    • Checking the status of the disk-buffer file.

      /opt/syslog-ng/bin/dqtool info /tmp/qdisk/syslog-ng-00000.rqf
      

      An empty disk-buffer file will look similar to this:

      Reliable disk-buffer state loaded; filename='/tmp/qdisk/syslog-ng-00000.rqf', queue_length='0', size='0'
      
  11. Press CTRL+C to stop AxoSyslog.

  12. Check the state of the orphan disk-buffer file. For more information, see How to get information about disk-buffer files.

  13. If you have more than one orphan disk-buffer file, repeat the steps following the AxoSyslog stop (that is, the steps beginning from overwriting the empty disk-buffer file with the orphan disk-buffer file) for each orphan disk-buffer file.

  14. Remove the temporary directory.

    rm -rf /tmp/qdisk