systemd-journald doesn’t just store logs; it’s a powerful, flexible system for managing their lifecycle, from what gets kept locally to where it all goes.
Let’s see it in action. Imagine you’ve got a server and you want to capture all logs, but only keep the last 100MB on disk, and forward everything to a central syslog server at 192.168.1.100 on UDP port 514.
First, let’s set up the local retention. We’ll edit /etc/systemd/journald.conf.
[Journal]
Storage=volatile
SystemMaxUse=100M
Here, Storage=volatile means logs are kept in RAM, and SystemMaxUse=100M tells journald to automatically prune old logs when the journal reaches 100 megabytes. If you want to keep logs on disk even after a reboot, you’d use Storage=persistent.
Now, let’s configure forwarding. We’ll add these lines to the same /etc/systemd/journald.conf:
[Journal]
ForwardToSyslog=yes
SyslogTransport=udp
SyslogRemote=192.168.1.100:514
ForwardToSyslog=yes tells journald to send logs to a syslog daemon. SyslogTransport=udp specifies the protocol, and SyslogRemote=192.168.1.100:514 is the destination. If your syslog server uses TCP, you’d change SyslogTransport to tcp and ensure the port is correct (often 6514 for TLS, or 514 for unencrypted TCP).
After saving the changes, we need to restart systemd-journald for them to take effect:
sudo systemctl restart systemd-journald
Now, journald will actively manage its local storage, discarding older entries once 100MB is reached, and simultaneously stream all incoming log messages to 192.168.1.100 via UDP.
The core problem journald solves is the chaotic nature of traditional syslog. Instead of disparate files managed by rsyslog or syslog-ng with their own complex parsing and rotation rules, journald provides a unified, structured binary log format. This structure allows for richer querying and more deterministic management. It’s not just about storing logs; it’s about managing them as a resource.
Internally, journald writes logs to files in /var/log/journal/. When Storage=persistent is used, these files are organized by machine-ID. Each log entry is a structured binary object, not just plain text. This means fields like MESSAGE, PRIORITY, UNIT, PID, and COMM are distinct, queryable metadata, not just parts of a string.
The journalctl command is your primary interface for interacting with these logs. For instance, to see logs from the last hour from your local journal:
journalctl --since "1 hour ago"
To see logs forwarded to the syslog server, you’d check the configuration of the receiving syslog daemon (e.g., rsyslog or syslog-ng) on 192.168.1.100.
You can also configure journald to forward logs to other destinations besides syslog, like ForwardToKmsg=yes to send them to the kernel message buffer, or ForwardToConsole=yes to log to the virtual console.
One of the most potent, yet often overlooked, features is RateLimitIntervalSec and RateLimitBurst. By default, journald will rate-limit identical messages within a short interval to prevent log flooding during system instability. This is controlled by RateLimitIntervalSec (the time window in seconds) and RateLimitBurst (the maximum number of messages allowed within that window). For example, if you see too many repeated error messages, you might adjust these:
[Journal]
RateLimitIntervalSec=30
RateLimitBurst=100
This configuration means journald will allow up to 100 identical messages in a 30-second window before it starts dropping subsequent identical messages, only reporting them again after the interval passes and a new burst of identical messages occurs. This is crucial for preventing disk filling or network saturation during high-volume error conditions.
The next step in advanced log management would be exploring structured logging with systemd-journald and integrating it with log analysis platforms like Elasticsearch or Splunk.