Log backup clashes with log rotation

Question

On my servers I have log rotation and a backup script which, besides all, backs up the entire log directory.

For log rotation I use logroatate on Linux and newsyslog is used on BSD. For log backup I simply cp -Rf /var/log/ /backups/

From time to time the backup script fails because of a race condition occurring when log rotation renames a log file during the backup is running. So I get something like:

cp: /var/log/messages.0: No such file or directory

I believe my situation is not unique. Are there any best practices to overcome this race condition and make backup more reliable?

How about putting the backup as the last line of the cron script that does log rotation? That way, you can guarantee the backups will start as soon as, but not before, log rotation has finished. — MadHatter, Mar 09 '16 at 07:06
Not a bad idea, but unfortunately the backup script can also be invoked on-demand e.g. by an operator via the server web interface. — Andrei Korostelev, Mar 09 '16 at 07:15
Fair enough. Then I think Iain's answer is the right one: you're going to have to code for this yourselves. — MadHatter, Mar 09 '16 at 08:21

score 3 · Answer 1 · answered Mar 09 '16 at 08:12

3

It doesn't look like logrotate provides anything that you can hook into (e.g. a pid file) to solve this problem so you'll need to work around it.

I would look into using flock(1) which is available for Linux and BSD.

You can have the logrotate script create a lock and then have the backup script check for and wait on the lock.

answered Mar 09 '16 at 08:12

user9517

114,104
20
206
289

File-level locking was also the first thing popped into my mind. Just curious about other solutions. – Andrei Korostelev Mar 09 '16 at 08:23

Log backup clashes with log rotation

1 Answers1