1

My server has a SNMP service (NET-SNMP) started with systemd. It runs well, except that every 3 days, at 3:00 PM, the service crashes.

I notice in the log that there is a reload of the agent. Then it crashes.

The service is reloaded because of a logrotate of the logs. The log file reaches the size required by logrotate after 2+ days, so on the 3rd day, the logrotate starts. There is a postrotate to reload snmpd (and snmptrapd which is not enabled).

But as seen below, it fails to reload.

Here is the log from /var/log/messages. The machine is a SLES 12.

2019-01-16T15:00:01.261639+01:00 myserver systemd[1]: Reloading LSB: Net-SNMP agent.
2019-01-16T15:00:01.270020+01:00 myserver kernel: [4192068.189683] floppy: error -5 while reading block 0
2019-01-16T15:00:01.270032+01:00 myserver kernel: [4220891.794129] snmpd[37667]: segfault at 0 ip 00007ff808105224 sp 00007ffc65011420 error 4 in libnetsnmpmibs.so.30.0.2[7ff80806a000+180000]
2019-01-16T15:00:01.271016+01:00 myserver snmpd[49093]: Reload service snmpd:..done
2019-01-16T15:00:01.282082+01:00 myserver systemd[1]: Reloaded LSB: Net-SNMP agent.
2019-01-16T15:00:01.282290+01:00 myserver snmpd[49101]: Shutting down snmpd:..done
2019-01-16T15:00:01.291082+01:00 myserver systemd[1]: Unit snmptrapd.service cannot be reloaded because it is inactive.
2019-01-16T15:00:01.481446+01:00 myserver systemd[1]: Reloading System Logging Service.

The last log line from the net-snmpd.log file, right before the crash is: Reconfiguring daemon

jayooin
  • 294
  • 1
  • 8
  • "snmpd[37667]: segfault at 0 ip 00007ff808105224 sp 00007ffc65011420 error 4 in libnetsnmpmibs.so.30.0.2[7ff80806a000+180000]" is the key here. The process crashed, and usually you need to use a debugger to learn why. – Lex Li Jan 17 '19 at 14:27
  • Well pointed out. Although now I don't know how I can solve this. But it helped to point out that my server is not up to date, as this library is part of the libsnmp30 package which isn't the last one available for SLES 12. I will upgrade to **SLES 12 SP2** in hope that this will solve the issue. – jayooin Jan 17 '19 at 16:17

1 Answers1

0

As @Lex Li pointed out, the problem was showed in that line:

2019-01-16T15:00:01.270032+01:00 myserver kernel: [4220891.794129] snmpd[37667]: segfault at 0 ip 00007ff808105224 sp 00007ffc65011420 error 4 in libnetsnmpmibs.so.30.0.2[7ff80806a000+180000]

Every time the service was reloaded, then it occured. Only on the reload. I still don't know what was exactly the problem with that library, but I upgraded my system to SLES 12 SP4, from SLES 12.0.

Now, the library is updated as well, using libnetsnmpmibs.so.30.0.3 instead of libnetsnmpmibs.so.30.0.2, and the reload works as a charm.

jayooin
  • 294
  • 1
  • 8