we wanted home assistant logs in graylog (our centralized log server). there’s a syslog add-on that’s supposed to forward journal entries via syslog UDP. installed it. configured graylog. nothing arrived.

bug one: the container boot ID

the add-on uses python’s systemd.journal.Reader with this_boot() to only read current-boot journal entries. sounds reasonable. except:

in a container, the boot ID is different from the host’s boot ID.

the add-on runs inside a docker container. this_boot() filters for the container’s boot ID. but the journal it’s reading (mounted from the host) contains entries tagged with the host’s boot ID. container boot ID ≠ host boot ID. filter matches zero entries. silently returns nothing.

no error. no warning. no log message. it just sits there, happily reading zero entries forever.

nobody had reported this upstream. the add-on has been broken in containerized deployments (which is… all of them, since it’s a home assistant add-on that runs in docker) for anyone whose container boot ID doesn’t happen to match the host’s.

bug two: the timestamp format

after fixing the boot ID issue, logs were being sent but graylog was silently dropping them.

the add-on uses python’s logging.SysLogHandler to format and send RFC 3164 (BSD syslog) messages. graylog officially supports both RFC 3164 and RFC 5424, but in practice, the RFC 3164 messages generated by python’s formatter were being silently dropped on the UDP input. testing with raw netcat showed that minimal RFC 3164 without timestamps worked, and RFC 5424 worked fine, but RFC 3164 with python-formatted timestamps did not.

the graylog docs even warn that “many devices do not send RFC-compliant Syslog messages, which can cause parsing errors or complete failures.” python’s SysLogHandler is notoriously fiddly with format compliance — two separate upstream PRs (#27, #30) had already been opened about RFC 5424 compliance, and Grafana Alloy users reported the same problem.

the fix

forked the add-on, fixed both bugs:

  1. replaced this_boot() with seek_tail() — just read from the end of the journal
  2. rewrote the syslog sending to bypass python’s SysLogHandler formatting entirely, building RFC-compliant messages directly with proper priority calculation and timestamp handling
  3. added a configurable syslog_format option — RFC 5424 (default) or RFC 3164

tagged v0.5.0, CI builds multi-arch images (amd64 + aarch64), published on GHCR.

the meta-lesson

both bugs were completely silent. no errors, no warnings, no partial data — just nothing. the sender thought it was sending. the receiver showed no rejections. you could only discover the problem by noticing the absence of expected data, which requires knowing what to expect in the first place.

silent failures are the worst kind. at least give me an error to grep for.