9

I'm using rotatelogs to create my daily apache logs in format host.<day>.<month>.<year>.access.log. Now I want to gzip and move log to different directory after it's been finished. How to do that?

Update: There was a little mistake. logrotate -> rotatelogs

chicks
  • 3,639
  • 10
  • 26
  • 36
Poma
  • 1,289
  • 6
  • 22
  • 35
  • Thanks for correcting the question, I've removed my inaccurate answer. – jscott Dec 23 '11 at 15:23
  • You can write a simple script that gzip the old files and move/copy them to a different location. This script can be invoked in a daily cron job. – Khaled Dec 23 '11 at 15:37
  • I'm not familiar with shell scripts. Specifically, I don't know how to generate filename string for yesterday's date. – Poma Dec 23 '11 at 15:39

5 Answers5

8

You could use rotatelogs option -p to use a program to compress the log after the rotation. (See for reference: https://httpd.apache.org/docs/2.4/programs/rotatelogs.html)

-p program

If given, rotatelogs will execute the specified program every time a new log file is opened. The filename of the newly opened file is passed as the first argument to the program. If executing after a rotation, the old log file is passed as the second argument. rotatelogs does not wait for the specified program to terminate before continuing to operate, and will not log any error code returned on termination. The spawned program uses the same stdin, stdout, and stderr as rotatelogs itself, and also inherits the environment.

Example:

CustomLog "|bin/rotatelogs -p '/path/to/compress.sh' -l /var/log/logfile.%Y.%m.%d 86400"

compress.sh:

#!/bin/bash
file_to_compress="${2}"
compress_exit_code=0

if [[ "${file_to_compress}" ]]; then
    echo "Compressing ${file_to_compress} ..."
    tar --gzip --create --file "${file_to_compress}.tar.gz" "${file_to_compress}"

    compress_exit_code=${?}

    if [[ ${compress_exit_code} == 0 ]]; then
        echo "File ${file_to_compress} was compressed."
    else
        echo "Error compressing file ${file_to_compress} (tar exit code: ${compress_exit_code})."
    fi
fi

exit ${compress_exit_code}
Community
  • 1
Enrique Rivera
  • 81
  • 1
  • 2
  • IMHO this is the best answer (+1). 1) If you don't want to keep the uncompressed files, include `--remove-files` into `tar`. 2) [Depending on your case](http://www.gnu.org/software/tar/manual/html_section/tar_80.html#verify), it might be also interesting to include `--verify`. – Mark Messa Aug 13 '19 at 15:56
6

I've came up with the following script

#!/bin/sh
for file in $(ls /var/log/apache2/*.$(date +"%y.%m.%d" --date="1 day ago").access.log); do
    gzip $file
    mv $file.gz /var/log/apache2/archive
done;

And following cron entry

15 0    0 0 0   root    /mypath/myscript.sh
Poma
  • 1,289
  • 6
  • 22
  • 35
2

Logrotate will happily do the compressing for you. Just add:

compress

To the logrotate config for apache. There is also a neat option that delays the compressing by one day:

delaycompress

As for moving them, logrotate can't help you but a cron job like this can:

@daily mv /var/log/apache/*.gz /var/log/archive/
Ladadadada
  • 25,847
  • 7
  • 57
  • 90
  • 2
    `logrotate` offers the `olddir` options which can move the logs -- see `man logrotate`. The OP edited the question to indicate he's using [`rotatelogs`](http://httpd.apache.org/docs/2.0/programs/rotatelogs.html) *not* `logrotate`. – jscott Dec 23 '11 at 17:46
  • Ah, that means I'm barking up the wrong tree. And I didn't even know about the `olddir` option. The shell script above seems like the best solution. – Ladadadada Dec 23 '11 at 18:01
  • 2
    logrotate is not rotatelogs! – MUY Belgium Apr 26 '13 at 09:45
1

I use find and crontab to accomplish this

# crontab -e

5 0 * * * /bin/find /path/to/logs/* -type f \( -mtime 1 ! -name "." ! -name ".gz" \) -print0 | xargs -0 gzip >/dev/null 2>&1;

-mtime n File’s data was last modified n*24 hours ago.

! -name "not ".*" hidden files

! -name "not ".gz" files which should already be compressed

-print0 & -0 to ensure spaces or special characters are piped correctly to xargs -> gzip (for filename safety)

find -exec could be used but has flaws where | xargs is more stable

user3258557
  • 29
  • 4
1
CustomLog "|/opt/IHS/bin/rotatelogs -l /some/path/access_log.%Y.%m.%d 86400" common | gzip -9 /some/path/access_log.`date '+%Y.%m.%d'`
Jenny D
  • 27,358
  • 21
  • 74
  • 110
arek
  • 11
  • 1
  • Hello arek and welcome on SF. Please take a moment to read [this](https://serverfault.com/help/how-to-answer) article. – Marco Oct 04 '17 at 19:39