Log rotation of stdout?

Question

I have a Linux program which can write information to stdout and stderr.

I have a shell script which redirects that output to a file in /var/log. (Via >> and 2>&1.)

Is there a way to make that log file rotate? (max size, then switch to a different file, keep only a limited number of files)

I've seen a few answers which talk about the logrotate program, which sounds good, but they also seem to be focused on programs which are generating log files internally and handle HUP signals. Is there a way to make this work with a basic output redirection script?

Answer 1

As an alternative, you could pipe the output through tools designed with the primary purpose of maintaining size-capped, automatically rotated, log file sets, such as:

• Dan Bernstein's multilog from daemontools
• Bruce Guenter's multilog from daemontools-encore
• Laurent Bercot's s6-log from s6
• Gerrit Pape's svlogd from runit
• Wayne Marshall's tinylog from perp
• My cyclog from nosh

Tools to then process multilog-format log file sets include, amongst others:

• Russ Allbery's multilog-watch
• logrange
• Paul Kremer's multilog-stamptail
• My follow-log-directories from nosh
• My export-to-rsyslog from nosh

Further reading

• Jonathan de Boyne Pollard (2015). "Logging". The daemontools family. Frequently Given Answers.
• Jonathan de Boyne Pollard (2016). Don't use logrotate or newsyslog in this century.. Frequently Given Answers.
• https://unix.stackexchange.com/a/326166/5132

Answer 2

the rotatelogs tool shipped with apache (in the bin dir) (see docs) takes input from stdin and rotates the log after some specific amount of time

Answer 3

If you can have it go to one of the standard log streams (syslog, daemon, cron, user, security, mail, etc.) you can use the logger command and pipe to it instead.

echo "Hello." | logger -p daemon.info

Otherwise, you may be better off piping your logged content to a custom program or script to handle it, or look at setting up the logrotate configuration.

EDIT: JdeBP's answer seems to have what you may be looking for.

Answer 4

I had similar problem and had initially discard logrotate but it turned out logrotate can actually do this well, the key directive is "copytruncate". For some reason that term didn't come up on any of the googling I did, so I am adding this answer to clarify exactly how to use it for this case.

The trick is this only works if the redirect is done with ">>" (append) instead of ">" (create).

Config File (truncate.cfg):

/tmp/temp.log {
    size 10M
    copytruncate
    rotate 4
    maxage 100
}

Test Program (never gives up file). You can watch it filling disk and though deleting logfile will appear to work it will not actually free up any space on the disk:

cat /dev/urandom >> /tmp/temp.log

Running log rotate:

logrotate truncate.cfg

Answer 5

So is there a way to make logrotate work with an ongoing process's redirected stdout?

Yes! Check out the "copytruncate" directive offered by logrotate. Specifying that instructs logrotate to handle this very situation: a simple program that keeps its log file open indefinitely.

One caveat may or may not be a problem in your situation:

Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost.

Anecdotally, I've seen some "real world" log sources that do encourage users to apply this directive. There's some discussion of this option here.

Answer 6

Use split, it's part of coreutils. It can take stdin and split it into chunks (based on chunk size, or number of lines, etc.).

Example:

app | split --bytes 1G - /var/logs/put-prefix-here

Note dash (-) instructs "split" to use stdin instead of file.

Answer 7

I like multilog for my use case, but my use case is so trivial/simple that it is not laid out very simply in the docs/examples I found. Here is a simple multilog rotate example:

mkdir /tmp/myapp
./myapp | multilog t s10000 n5 '!tai64nlocal' /tmp/myapp 2>&1

Some notes:

• this dumps logs into that /tmp/myapp/ directory
• the s10000 represents 10,000 bytes*
• the n5 represents 5 files.* The 'current' log counts as one of the files, so this includes 4 older logs + 'current'
• this is based on, adapted from the examples provided by François Beausoleil at: http://blog.teksol.info/pages/daemontools/best-practices
• I don't understand many of the options - I refer you to the various documentation to extend this...
• The docs warn that: "Note that running processor may block any program feeding input to multilog." where 'processor' is the '!tai64nlocal' portion of the command

*For many applications, these are poor choices for long term use. They do allow you to observe the behavior of filling and rotating the logs more quickly than large logs do.

Finally, don't forget to nohup if required! With nohup, you do not need the 2>&1 (s=10e6 and n=30 here):

mkdir -p /tmp/myapp
nohup ./myapp | multilog t s10000000 n30 '!tai64nlocal' /tmp/myapp &

That command should get you started.

Answer 8

I just wanted to add to Sam Hendley's comment above:

The trick is this only works if the redirect is done with >> (append) instead of > (create).

I ran into the same problem where the original file just keeps growing if you use > (create) but if you use >> (append) Logrotate copytruncate works beautifully and as expected. The original file goes back down to zero bytes and the program continues writing.

Redirect STDOUT and STDERR to a rotating logfile:

1. some-program.sh >> /tmp/output.txt 2>&1 &

2. Create a logrotate config file under /etc/logrotate.d called whatever, output_roll in my case.

   Sample config for my case:

   /tmp/output.txt {
       notifempty
       missingok
       size 1G
       copytruncate
       start 0
       rotate 15
       compress
   }
   

3. Setup your cron job inside the /etc/crontab file

   *  *  *  *  * root /usr/sbin/logrotate /etc/logrotate.d/output_roll
   

   This will check the file every minute. You can adjust to suit your needs.

4. Start it up:

   $> service crond restart
   

5. That's it

Note: I also had a problem with SELinux being set to SELINUX=enforcing so I set it to SELINUX=disabled.

Answer 9

I wrote a logrotee this weekend. I probably wouldn't if I've read @JdeBP's great answer and multilog.

I focused on it being lightweight and being able to bzip2 its output chunks like:

verbosecommand | logrotee \
  --compress "bzip2 {}" --compress-suffix .bz2 \
  /var/log/verbosecommand.log

There's a lot of to be done and tested yet, though.