Mongodb - proper way to rotate logs

Question

Mongo docs say that I can:

use -SIGUSR1 signal and get the old log renamed and current switched
use logrotate from OS

I want the OS's logrotate ability to zip old files and remove oldest, but see no way to tell mongod process to switch current log other than sending SIGUSR1.

So I wrote

/var/log/mongodb/*.log {
    daily
    rotate 5
    compress
    dateext
    missingok
    notifempty
    sharedscripts
    postrotate
        /usr/bin/killall -SIGUSR1 mongod
        /usr/bin/killall -SIGUSR1 mongos
    endscript
}

to /etc/logrotate.d/mongo.

And now get well-named logfiles from logrotate and empty logfiles like mongodb.log.2013-09-18T23-49-44 as traces of SIGUSR1 switching. How to get rid of the latter?

score 20 · Answer 1 · answered Jan 18 '16 at 10:15

20

Since mongodb 3.0 you can change the behavior of mongodb with the logRotate parameter, change in /etc/mongod.conf

systemLog:
  logAppend: true
  logRotate: reopen

See also Mongo Manuals.

Then you can use this logrotate configuration:

/var/log/mongodb/*.log {
    daily
    rotate 30
    size 50M
    compress
    dateext
    missingok
    notifempty
    sharedscripts
    postrotate
        /bin/kill -SIGUSR1 `cat /var/lib/mongodb/mongod.lock 2> /dev/null` 2> /dev/null || true
    endscript
}

answered Jan 18 '16 at 10:15

mmx73

301
2
2

1

A PID file created from the config file should probably be used.... See `processManagement.pidFilePath` in the config or the SystemD unit file's PIDFile setting (`/var/run/mongodb/mongod.pid` for me) – Gert van den Berg Apr 24 '19 at 08:40
BUT - keep in mind that nowaday /run is tmpfs, so it is cleared on reboot and the mongodb folder does not exist unless it is created vi tmpfs conf! – JeffRSon Oct 23 '20 at 17:14

score 15 · Accepted Answer · answered Sep 20 '13 at 10:22

15

copytruncate works pretty well for logrotation.

a config similar to this should do the job for you:

/var/log/mongodb/*.log {
  daily
  missingok
  rotate 5
  compress
  dateext
  delaycompress
  copytruncate
  notifempty
}

answered Sep 20 '13 at 10:22

steveberryman

174
1
3

1

This didn't work for me on RedHat 6.5. Logs were rotated but the original .log file continued to grow without bound. – Thomas Bratt Oct 16 '14 at 15:28
@ThomasBratt this is correct because without restarting mongo processes file handler remain open. fwics this method will not work all that well. – Mxx Oct 16 '14 at 15:31
@ThomasBratt take a look at this answer http://stackoverflow.com/a/8396266/949859 – Mxx Oct 16 '14 at 15:49
1

@Mxx Nice find - looks like copytruncate will work with the a postrotate step to signal to Mongo to truncate the logfile – Thomas Bratt Oct 16 '14 at 16:04

score 15 · Answer 3 · answered Dec 17 '14 at 03:35

The server crashed for me if you send SIGUSR1 to mongod after you have moved the logfile out of the way with logrotate.

The following config is safe for the version I've tested - 2.6.6 on ubuntu 12.04 - the previous examples crashed the server. Put this into /etc/logrotate.d/mongod:

/var/log/mongodb/mongodb.log {
    weekly
    missingok
    rotate 4
    compress
    notifempty
    create
    postrotate
        /usr/bin/pkill -USR1 mongod
        rm /var/log/mongodb/mongodb.log.????-??-??T??-??-??
    endscript
}

See: https://jira.mongodb.org/browse/SERVER-11087 for more details and a suggestion from Akshay Kumar which I used in the above (use create instead of nocreate and cp /dev/null to the logfile).

In later releases there is supposed to be a logRotate option you can use to reopen the file - not rename it - which will work around the rename problem - but it didn't work in my version (it was unsupported).

See: https://github.com/mongodb/mongo/commit/616461d294bd9f5054ca38b302b6fc5d70fde20c

I've tested this with

logrotate -v -f /etc/logrotate.d/mongod

Can confirm that if `systemLog.logRotate: reopen` in `mongod.conf`, then the pkill will work as intended and no deletion of the renamed log file is required as none is created. — Julian H. Lam, Mar 04 '17 at 21:52

gr8 · Answer 4 · 2022-07-21T11:12:38.370

My solution for MongoDB v4.4 & Ubuntu 18.04

Create logrotate config /etc/logrotate.d/mongodb

/var/log/mongodb/mongod.log {
    rotate 7
    daily
    size 100M
    missingok
    create 0600 mongodb mongodb
    delaycompress
    compress
    sharedscripts
    postrotate
            /bin/kill -SIGUSR1 $(cat /var/run/mongodb/mongod.pid)
    endscript
}

Modify /etc/mongod.conf for reopening log file at kill signal and select pid file

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log
  logRotate: reopen
processManagement:
  pidFilePath: /var/run/mongodb/mongod.pid

Modify /lib/systemd/system/mongod.service. It will create directory /var/run/mongodb at service start

[Service]
RuntimeDirectory=mongodb

Apply & check changes

sudo systemctl daemon-reload
sudo systemctl restart mongod
sudo systemctl status mongod

ls -lah /var/log/mongodb/
logrotate -f /etc/logrotate.d/mongodb
ls -lah /var/log/mongodb/

Warn!

/lib/systemd/system/mongod.service can be overwritten after updating mongo with apt upgrade

This is nice. I used `systemctl show --property MainPID --value ` to get the PID. That way you don't have to modify systemd and avoid the upgrade issue. So in the logrotate.d file you would put: `/bin/kill -SIGUSR1 $(systemctl show --property MainPID --value mongod)` — AlexPi, Aug 28 '22 at 03:00

score 0 · Answer 5 · answered Oct 16 '14 at 15:34

The following worked for me:

/var/log/mongo/mongod.log {
    missingok
    rotate 3
    size 100M
    nodateext
    postrotate
            /usr/bin/kill -USR1 $(cat /var/run/mongod.pid)
            rm -f /var/log/mongo/mongod.log.[0-9][0-9][0-9][0-9]-*
    endscript
}

Notes:

Tested on RedHat 6.5
The only way I could get a working solution was to delete the empty log files that Mongo produces
The location of the lock file depends on how MongoDB was installed
kill is a Bash builtin but logrotate runs under /bin/sh - which doesn't recognise SIGUSR1 on RedHat 6.5
I haven't tested with compress but it should be a straighforward addition

score 0 · Answer 6 · answered Jul 05 '16 at 18:13

0

Note that in version 3.0 and higher it is not required to kill your database daemon simply to rotate a log. Refer to documentation here:

https://docs.mongodb.com/manual/tutorial/rotate-log-files/

answered Jul 05 '16 at 18:13

Eric Aldinger

21
1

2

It seems Andrey already read that documentation; you're not really answering his question. – Law29 Jul 05 '16 at 19:06

Mongodb - proper way to rotate logs

6 Answers6

Linked