1

I've got an Ubuntu (9.04/Jaunty) server VPS that's not properly rotating syslogs.

Here's what I've checked so far:

  • syslogd-listfiles lists the files I think should be rotated
  • cron.daily is running (so sayeth syslog)
  • when I manually run the file checks at the beginning of /etc/cron.daily/sysklogd they all pass (test -x /usr/sbin/syslogd-listfiles, test -x /sbin/syslogd, test -f /usr/share/sysklogd/dummy)
  • when I run the cron.daily job manually as root (sudo run-parts --verbose /etc/cron.daily) the logs rotate as I would expect

Anybody have any ideas what I might try next or what I might be missing? I was thinking that perhaps the fact that sysklogd seems to be running it's process as syslog (the owner of the syslogd per ps -C syslogd -o user= | head -n 1) means that there's some sort of permissions issue, and that seems to be supported by the results of running sudo -u syslog run-parts --verbose /etc/cron.daily which ends up with a bunch of permissions erros, but I'm not sure what the best way to resolve that would be.

Contents of my sysklogd file follow, in case that's of assistance. The touch /etc/crontouchtest bit is something I inserted to verify when the file is run successfully. It updates the last used time (ls -lut /etc/crontouchtest) when I execute the run-parts as root, but not when the cron.daily runs.

#! /bin/sh

# sysklogd      Cron script to rotate system log files daily.
#
#               If you want to rotate other logfiles daily, edit
#               this script.  An easy way is to add files manually,
#               to add -a (for all log files) to syslogd-listfiles and
#               add some grep stuff, or use the -s pattern argument to
#               specify files that must not be listed.
#
#               This is a configration file.  You are invited to edit
#               it and maintain it on your own.  You'll have to do
#               that if you don't like the default policy
#               wrt. rotating logfiles (i.e. with large logfiles
#               weekly and daily rotation may interfere).  If you edit
#               this file and don't let dpkg upgrade it, you have full
#               control over it.  Please read the manpage to
#               syslogd-listfiles.
#
#               Written by Martin Schulze <joey@debian.org>.
#               $Id: cron.daily,v 1.14 2007-05-28 16:33:34 joey Exp $

test -x /usr/sbin/syslogd-listfiles || exit 0
test -x /sbin/syslogd || exit 0
test -f /usr/share/sysklogd/dummy || exit 0

touch /etc/crontouchtest
USER=$(ps -C syslogd -o user= | head -n 1)
[ -z "${USER}" ] && USER="root" || true

set -e

cd /var/log

logs=$(syslogd-listfiles)

test -n "$logs" || exit 0

for LOG in $logs
do
   if [ -s $LOG ]; then
      savelog -g adm -m 640 -u ${USER} -c 7 $LOG >/dev/null
   fi
done

# Restart syslogd
#
/etc/init.d/sysklogd reload-or-restart > /dev/null

EDIT

Outputs as requested:

ls -la /etc/cron.daily (run as root)

drwxr-xr-x   2 root root 4096 Oct 23 07:13 .
drwxr-xr-x 107 root root 4096 Oct 23 07:14 ..
-rwxr-xr-x   1 root root  314 Feb 10  2009 aptitude
-rwxr-xr-x   1 root root  111 May 11 11:49 backup-manager
-rwxr-xr-x   1 root root   89 Jan 26  2009 logrotate
-rwxr-xr-x   1 root root 1334 Oct 22 09:35 sysklogd


ps -ef | egrep '[c]ron' (run as root)

root     13369     1  0 Oct21 ?        00:00:02 /usr/sbin/cron

EDIT 2

Respective paths

from echo $PATH (after swithcing to root):

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

from /etc/crontab:

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
cori
  • 381
  • 1
  • 8
  • 16
  • currently /var/log has permission bit set as 755, which seems pretty typical across other servers I manage in other distros, but n the other servers syslogd runs as root, so I imagine that's the issue. Is there a reason that this server's syslog config is set to run syslogd as syslog rather than root? Is that typical of Jaunty? Seems like setting syslogd to run as root would resolve the issue. – cori Oct 23 '10 at 15:06
  • Please post the output of: `ls -la /etc/cron.daily`. Also: `ps -ef | egrep '[c]ron'`. – Steven Monday Oct 23 '10 at 15:07
  • OK, it's looking like anacron was simply never set up to run on system start, and /etc/crontab was set to test for anacron and only if it doesn't exist to perform the run-parts command. anacron exists on the server, so the run-parts never executes, but since anacron's not running the anacron jobs (replacing cron.daily, cron.weekly, and cron.monthly) don't execute. Fixed all of that and I'll see how it goes tonight. – cori Oct 24 '10 at 01:59
  • That's weird. On every Debian (and Debian-derived) system I've seen, anacron hasn't needed any extra configuration. It should "just work" when installed. Anyway, if this server is running all the time, why not just remove the anacron package entirely? anacron is only really meant for systems (like laptops and home PCs) that are not running 24/7. – Steven Monday Oct 24 '10 at 15:39
  • That's kind of what I thought, too. Unfortunately the server took a header unrelated to all of this and is being rebuilt, but before that happened it seemed like anacron was doing its job properly. I also thought about just removig the anacron test, too.... – cori Oct 25 '10 at 03:03
  • well after rebuilding this box with Ubuntu 10.04.1 LTS, the exact same problem occurred. Removed the anacron package and the cron job for sysklogd is working great. @Steven if you create an answer with that I'll accept it. – cori Oct 29 '10 at 12:18
  • @cori: Don't worry about it. Just glad you figured it out! – Steven Monday Oct 29 '10 at 14:17
  • Well mostly I just want an answer that someone else can easily find if they have this issue as well, and I hate answering my own questions. Seems wrong, somehow... – cori Oct 29 '10 at 18:30

2 Answers2

1

The touch /etc/crontouchtest bit is something I inserted to verify when the file is run successfully. It updates the last used time (ls -lut /etc/crontouchtest) when I execute the run-parts as root, but not when the cron.daily runs.

If I read that correctly, /etc/crontouchtest does not get updated when cron runs the cron.daily tasks. This, coupled with the fact that /etc/cron.daily/sysklogd runs correctly when launched by hand, leads me to suspect that something is causing run-parts to fail to launch /etc/cron.daily/sysklogd when run-parts is run by cron.

Since cron is running as root, and your manual test also ran as root, there is very little difference between the two environments. All I can think of is that cron perhaps runs with a different PATH as compared to the PATH that exists at the command line. Also, when a process is run by cron, there is no controlling tty. Could either of those differences explain the difference in results?

Steven Monday
  • 13,019
  • 4
  • 35
  • 45
  • I posted root's path and the path set in crontab - they look all but identical to me. crontab was setting shell to /bin/sh instead of /bin/bash, but changing that didn't change the outcome. Not sure whether or not a lack of controlling tty would make a difference.... – cori Oct 23 '10 at 22:59
0

Answering this question so the solution is more easily findable.

The default sysklogd cron job checks for anacron first, and if found doesn't run the log rotation for the system logs. As @Steven suggested I removed anacron from the system (since it's a server and intended to be on 24/7/365, anacron's features aren't really needed). Once anacron was off the system, the anacron test in the cron job fails and the system log rotation works like a champ.

Thanks @Steven

cori
  • 381
  • 1
  • 8
  • 16