The systemd daemon reported "Protocol not available" when attempting to send a notification via sd_notify, indicating that the systemd supervisor process isn’t listening for these status updates from the service.
Here are the common reasons and how to fix them:
1. The Service File is Missing Type=notify
- Diagnosis: Check your service’s
.servicefile. The critical line that enablessd_notifyisType=notify. If it’s missing or set to another type (likesimple,forking,oneshot, ordbus),systemdwill not set up the necessary communication socket forsd_notify.- Command:
systemctl cat your-service-name.service - Example Output (Missing
Type=notify):[Unit] Description=My Awesome Service [Service] ExecStart=/usr/local/bin/my-awesome-service Restart=on-failure [Install] WantedBy=multi-user.target
- Command:
- Fix: Add
Type=notifyto the[Service]section of your.servicefile.- Example Fix:
[Unit] Description=My Awesome Service [Service] Type=notify ExecStart=/usr/local/bin/my-awesome-service Restart=on-failure [Install] WantedBy=multi-user.target - Why it works:
Type=notifytellssystemdto expect status updates from the service process and specifically instructs it to create a special socket ($NOTIFY_SOCKET) that the service can write to. Without this, the socket isn’t created, andsd_notifyfails. - Action: After editing, reload
systemdand restart your service:sudo systemctl daemon-reload sudo systemctl restart your-service-name.service
- Example Fix:
2. The Service is Not Actually Sending Notifications
- Diagnosis: Even if
Type=notifyis present, the service itself must be written to callsd_notify()or a similar notification function. If the service code never attempts to send a notification,systemdmight still report this error, or the notification will simply be ignored.- Check: Review the source code or documentation of
your-service-name. Look for calls tosd_notify(),sd_notifyf(), orSD_NOTIFY_USE_FD.
- Check: Review the source code or documentation of
- Fix: Implement the notification logic in your service. This typically involves including the
systemd/sd-daemon.hheader and callingsd_notify(0, "READY=1")after the service has fully initialized and is ready to serve requests.- Example C Snippet:
#include <systemd/sd-daemon.h> // ... after service initialization ... if (sd_notify(0, "READY=1") == 0) { fprintf(stderr, "sd_notify failed, but continuing\n"); } // ... rest of service logic ... - Why it works: This ensures that your application actively communicates its readiness or other status changes to
systemdvia the established notification socket.
- Example C Snippet:
3. NOTIFY_SOCKET Environment Variable Not Set
- Diagnosis: When
Type=notifyis used,systemdsets theNOTIFY_SOCKETenvironment variable within the service’s execution context. If this variable is not present, the service cannot find the socket to send its notifications.- Command:
systemctl status your-service-name.serviceand look for the "Environment:" section. - Example Output (Missing
NOTIFY_SOCKET):
(If● your-service-name.service - My Awesome Service Loaded: loaded (/etc/systemd/system/your-service-name.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2023-10-27 10:00:00 UTC; 5min ago ... Environment: ...NOTIFY_SOCKETis present, it will be listed here with a path like/run/systemd/notify).
- Command:
- Fix: This is usually automatically handled by
systemdwhenType=notifyis correctly set. If it’s missing, it points back to an issue withType=notifynot being recognized bysystemdor a problem withsystemditself. Ensuresystemdis running and healthy.- Why it works: The
NOTIFY_SOCKETvariable is howsd_notify()(and other notification functions) discover the specific socket file descriptor provided bysystemdfor communication.
- Why it works: The
4. Systemd Version Too Old
- Diagnosis: Older versions of
systemdmight have had bugs or lacked full support for theType=notifymechanism. While rare on modern systems, it’s a possibility.- Command:
systemctl --version
- Command:
- Fix: Upgrade
systemdto a more recent stable version.- Why it works: Newer versions have improved stability and feature completeness for
systemd’s service management capabilities.
- Why it works: Newer versions have improved stability and feature completeness for
5. Permissions Issues with /run/systemd/notify
- Diagnosis: The
systemdnotification socket is typically located in/run/systemd/notify. If the service process or the user it runs as lacks the necessary permissions to access or write to this socket file (thoughsd_notifytypically works via file descriptors inherited from systemd, not direct file access), it could fail. This is very uncommon assystemdusually manages permissions correctly.- Check:
ls -l /run/systemd/notify(this file might not always be visible as it’s a socket).
- Check:
- Fix: Ensure the service is not running under an extremely restrictive user or with
NoNewPrivileges=yesin conjunction with other restrictive settings that might interfere with inherited file descriptors.- Why it works: The service needs to be able to communicate through the file descriptor that
systemdassociates with the notification socket.
- Why it works: The service needs to be able to communicate through the file descriptor that
6. SELinux or AppArmor Interference
- Diagnosis: Mandatory Access Control systems like SELinux or AppArmor can sometimes block the communication between a service and
systemd, including the notification socket.- Check SELinux:
sudo ausearch -m avc -ts recentand look for denials related to your service’s executable or/run/systemd/notify. - Check AppArmor:
sudo dmesg | grep -i apparmoror check/var/log/audit/audit.logor/var/log/syslog.
- Check SELinux:
- Fix: Adjust SELinux or AppArmor policies to allow the service to communicate with
systemd’s notification socket. This is highly specific to your policy configuration.- Example SELinux (temporary permissive mode for testing):
sudo semanage permissive -a your_service_t(replaceyour_service_twith the correct SELinux type for your service). - Why it works: MAC systems enforce access controls. Allowing the specific communication path between the service and the systemd notification mechanism grants the necessary permissions.
- Example SELinux (temporary permissive mode for testing):
After applying these fixes, the next error you might encounter if your service is still not behaving as expected could be a READY=1 timeout, indicating that while systemd is receiving notifications, your service isn’t sending the READY=1 signal in a timely manner.