0

My Django app runs on an nginx/openresty server. The user running the server is root:root but typically the app is owned by webapp:webapp. This means logs get created and written on by root.

The problem is that logs aren't getting rotated because an empty one is getting created and instead of being written on, the server keeps writing to the previous one:

webapp@webapphost:~$ ls -lh logs/openresty/
total 6.7G
-rw-rw-rw- 1 root root    0 Jan 30 13:24 access.log
-rw-rw-rw- 1 root root 6.4G Mar 17 11:34 access.log.1
-rw-rw-rw- 1 root root 135M Dec 21 20:29 access.log.12.gz
-rw-rw-rw- 1 root root 146M Nov 16 12:39 access.log.15.gz
-rw-rw-rw- 1 root root 8.4K Jan 29 17:34 access.log.2.gz
-rw-rw-rw- 1 root root 3.0K Jan 19 10:24 access.log.4.gz
-rw-rw-rw- 1 root root 216K Jan 18 10:54 access.log.6.gz
-rw-rw-rw- 1 root root  58K Jan 12 10:58 access.log.8.gz
-rw-rw-rw- 1 root root 1.5M Mar 17 06:16 error.log
-rw-rw-rw- 1 root root 7.4K Jan 29 16:38 error.log.1.gz
-rw-rw-rw- 1 root root 1.2K Jan 19 10:22 error.log.2.gz
-rw-rw-rw- 1 root root 4.5K Jan 18 08:31 error.log.3.gz
-rw-rw-rw- 1 root root 1.6K Jan 12 08:55 error.log.4.gz
-rw-rw-rw- 1 root root  12K Jul 13  2020 error.log.7.gz

webapp@webapphost:~$ ls -lh /var/log/openresty/
total 352M
-rw-rw-rw- 1 root root    0 Feb 16 15:24 access.log
-rw-rw-rw- 1 root root 277M Mar 17 11:37 access.log.1
-rw-rw-rw- 1 root root  13M Oct 29 19:24 access.log.11.gz
-rw-rw-rw- 1 root root  38M Oct  6 11:19 access.log.12.gz
-rw-rw-rw- 1 root root 5.1M Jul 23  2020 access.log.13.gz
-rw-rw-rw- 1 root root 8.0K Jul 13  2020 access.log.14.gz
-rw-rw-rw- 1 root root 8.2M Jul 13  2020 access.log.15.gz
-rw-rw-rw- 1 root root 1004 Jan 29 17:34 access.log.2.gz
-rw-rw-rw- 1 root root 6.4M Jan 29 17:34 access.log.3.gz
-rw-rw-rw- 1 root root 599K Jan 19 10:24 access.log.4.gz
-rw-rw-rw- 1 root root  11K Jan 18 10:54 access.log.5.gz
-rw-rw-rw- 1 root root 3.7M Jan 18 10:19 access.log.6.gz
-rw-rw-rw- 1 root root 514K Jan 12 10:58 access.log.7.gz
-rw-rw-rw- 1 root root  50K Mar 17 06:16 error.log
-rw-rw-rw- 1 root root 2.7K Jan 29 10:24 error.log.1.gz
-rw-rw-rw- 1 root root 1.6K Jan 17 10:58 error.log.2.gz

As you can see, the *.log* files as empty, but *.log.1` keeps growing infinitely.

My logrotate configuration is the following:

webapp@webapphost:~$ cat /etc/logrotate.d/openresty
/home/webapp/logs/openresty/*.log /var/log/openresty/*.log {
        su root root
        size 100M
        missingok
        rotate 14
        compress
        nodelaycompress
        notifempty
        create 0666 root root
        sharedscripts
        prerotate
        if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
                run-parts /etc/logrotate.d/httpd-prerotate; \
        fi \
        endscript
        postrotate
                invoke-rc.d openresty rotate >/dev/null 2>&1
                service openresty reload >/dev/null 2>&1
        endscript
}

When logrotate nothing happens because "the log does not need rotation", as it's empty:

webapp@webapphost:~$ sudo logrotate -d /etc/logrotate.d/openresty
reading config file /etc/logrotate.d/openresty

Handling 1 logs

rotating pattern: /home/webapp/logs/openresty/*.log /var/log/openresty/*.log  104857600 bytes (14 rotations)
empty log files are not rotated, old logs are removed
considering log /home/webapp/logs/openresty/access.log
  log does not need rotating
considering log /home/webapp/logs/openresty/error.log
  log does not need rotating
considering log /var/log/openresty/access.log
  log does not need rotating
considering log /var/log/openresty/error.log
  log does not need rotating
not running prerotate script, since no logs will be rotated
not running postrotate script, since no logs were rotated

Forcing the rotation with -f doesn't work neither.

Howerver if I do the following, forcing the rotation does work:

invoke-rc.d openresty rotate
sudo systemctl reload openresty
sudo logrotate -f /etc/logrotate.d/openresty
dabadaba
  • 101
  • 3
  • Issuing the `reload` command to `openresty` makes it close the log file and switch to the new one. Are you sure that is executed properly? – Tero Kilkanen Mar 17 '21 at 15:41
  • @TeroKilkanen how can I tell? When I run `sudo service openresty reload` from console it seems to go through just fine. – dabadaba Mar 17 '21 at 17:09
  • In your case it seems that your openresty wasn't restarted properly when log was first rotated. Therefore logrotate didn't take any action on subsequent runs, because log file was empty. If openresty now writes to new log, does the log rotate work when its time comes up? – Tero Kilkanen Mar 17 '21 at 18:04

0 Answers0