Enhance macOS logging with syslog-ng’s native macOS system() source

darwin-oslog(): native macOS log retrieval

The darwin-oslog() source marks a paradigm shift in syslog-ng’s approach to macOS system logs. Replacing the previous file source-based solution with a native OSLog framework, this source seamlessly integrates with the unified logging system of darwin OSes. If you install syslog-ng 4.6 on macOS 10.15 Catalina or newer, the system() source uses the darwin-oslog() source. (Note that although technically the new sources work on macOS 10.15+, the homebrew formulae only supports macOS 12 Monterey and newer.)

The darwin-oslog() source not only has some of the usual parameters similar to the file source (see a brief summary below), but it also allows you to filter the incoming logs on the source level, using native macOS predicates.

Also note that while darwin-oslog() offers powerful log retrieval capabilities, the persistent OSLog store is limited. Typically, it keeps about 7 days of logs on disk.

darwin-oslog() parameters

• filter-predicate(): String for native log message filtering. For example, the following predicate selects AirDrop logs:
  

  subsystem=="com.apple.sharing" and category=="AirDrop"

• go-reverse(): Boolean value for reverse-ordered log list (from latest to oldest).
• do-not-use-bookmark(): Boolean value to prevent log continuation from the last position.
• max-bookmark-distance(): Integer value for maximum distance in seconds for bookmark backward pointing.
• read-old-records(): Boolean value to control reading logs from the oldest available at the first start.
• fetch-delay(): Integer value controlling the time between log message reads/sends.
• fetch-retry-delay(): Integer value controlling the time before a repeated log read/send attempt.
• log-fetch-limit(): Integer value limiting the number of logs sent in one run (currently disabled due to an OSLog API bug).

darwin-oslog-stream(): real-time log streaming

The darwin-oslog-stream() source takes a different approach, and offers a live log stream feed through the macOS log tool’s “log stream” command. Unlike its darwin-oslog() counterpart, this source can provide non-persistent log events in real-time. This source has only one parameter called params(), which is a string containing macOS log tool parameters. Practically, that’s the parameter you would add to the log stream command in the macOS terminal – run the log --help stream command for a full reference.

Demo: Using the darwin-oslog() source

This short demo shows you how to install syslog-ng on macOS and how to get started.

Install syslog-ng on your Mac

You can install syslog-ng on a macOS (version 12 or newer) device with brew:

brew install syslog-ng

Configure syslog-ng

We will use a simple stdout() destination to print the messages to the console. Obviously, this is just for demonstration purposes, in a real usecase you’d configure a syslog destination to forward the logs to an aggregator or relay, or directly to one of the many destinations that syslog-ng supports.

Let’s modify the /opt/homebrew/etc/syslog-ng/syslog-ng.conf file, which is where the homebew package installs the main configuration file of syslog-ng, and replace its contents with the following:

@version: current
@include “scl.conf”

log {
    source {
        system();
    };
    destination {
        stdout();
    };
};

Run syslog-ng

Start syslog-ng in the foreground with the syslog-ng -Fe command. You’ll see that the system logs are printed to the console.

You can check that the same logs are available in the “Console” application as well. For example, you can find the logs from the “launchd” process both in syslog-ng’s output and the launchd.log file. (You can read more about the “Console” application here.)

Conclusion

As you can see, although a large part of syslog-ng development focuses on supporting cloud-related technologies and performance improvements – for example, the Google BigQuery destination which is also part of the 4.6 release, there are also more traditional features like these new macOS sources that find their way into the syslog-ng releases.