Load balancing logs between multiple destinations

1: Load balancing with a round robin load balancing method based on the R_MSEC macro

2: Configuration generator for the load balancing method based on MSEC hashing

These sections describe a method of load balancing logs between multiple AxoSyslog destinations. The first subsection describes the round robin load balancing method based on the R_MSEC macro of AxoSyslog, while the second subsection describes a configuration generator that you can use as an alternative to using the example configuration described in the first subsection.

For more information about the R_MSEC macro and further macros of AxoSyslog, see Macros of AxoSyslog.

1 - Load balancing with a round robin load balancing method based on the R_MSEC macro

This section describes a round robin load balancing method based on the R_MSEC macro of AxoSyslog to load balance your logs between multiple AxoSyslog destinations.

Note If R_MSEC is not precise enough, you can replace it with R_USEC (which uses microseconds instead of milliseconds).

For more information about the R_MSEC macro and further macros of AxoSyslog, see Macros of AxoSyslog.

Example: round robin load balancing between multiple destinations

The following example is a round-robin load balancing method, based on AxoSyslog’s R_MSEC macro.

   destination d_lb_network { 
      channel { 
        channel { 
          filter { 
          "0" == "$(% ${R_MSEC} 2)" 
          }; 
          destination { 
            network("myhost1" 
              disk-buffer(flow-control-window-size(10000) capacity-bytes(2000000))); 
          }; 
          flags(final); 
        }; 
     
        channel { 
        filter { 
        "1" == "$(% ${R_MSEC} 2)" 
        }; 
    
        destination { 
          network("myhost2" 
            disk-buffer(flow-control-window-size(10000) capacity-bytes(2000000))); 
        }; 
        flags(final); 
        }; 
      }; 
    };

The filter {" <return value >" == "$(% ${R_MSEC} 2)"}; code snippets (in bold) serve as the basis of the method. This filter separates incoming log messages’ timestamp values based on the R_MSEC macro, using a division with remainder method, and distributes the log messages equally between two destinations based on the return value (in this case, 0 or 1).

If you need a file instead of a network destination, replace the network destination with the file in the example (and use the same analogy for any other AxoSyslog destinations).

For an alternative method to use the round robin load balancing method based on the R_MSEC macro, see Configuration generator for the load balancing method based on MSEC hashing.

2 - Configuration generator for the load balancing method based on MSEC hashing

This section describes a configuration generator SCL for the load balancing method based on MSEC hashing to load balance your logs between multiple AxoSyslog destinations.

Warning

Consider that network-load-balancer() is not a destination, only a script that generates the example configuration described in Load balancing with a round robin load balancing method based on the R_MSEC macro.

Also consider that the configuration generator script may change incompatibly in the future. As a result, Axoflow does not officially support using this script, and recommends that you only use this script at your own risk.

As an alternative to using the example configuration described in Load balancing with a round robin load balancing method based on the R_MSEC macro, a configuration generator script is also available in AxoSyslog:

destination d_lb {network-load-balancer(targets(myhost1 myhost2 myhost3))};

Where destinations share the same configuration except for the destination address, balancing is based on MSEC hashing.

Starting with version 4.24, network-load-balancer() supports the failover() option.