7

Usually log messages are written to stderr. I'm wondering if it is a good idea/practice to split log messages so that errors and warnings go to stderr while debug/informational/notice messages go to stdout instead? Or is this irrelevant given that many dedicated logging processes only read from stdin anyway, which requires source log messages in stderr and stdout be combined and redirect to logger's stdin.

[Update]

The two answers below both mentioned syslog, I think I need to clarify the setup in detail.

The daemon processes I asked are running in foreground by themselves. Daemonization is managed by supervising processes such as runit or supervisord. In both cases, the stderr and stdout of the daemon processes will be captured by the supervising processes, and it's the supervising processes' job to decide how and where to store the logs (might be syslog, or somewhere else in the network over UDP). The daemon processes don't have to worry about what and where to write logs, as they just write to stdout/stderr.

In the case of runit, its logging facility svlogd will read from its stdin for the redirected log messages, which are combined stderr/stdout of the managed daemon process. As for supervisord, it can record stderr and stdout to separate log files.

So in this specific settings, is it a good practice to split logs between stderr and stdout, or just write to one of them?

Andrew B
  • 31,858
  • 12
  • 90
  • 128
Rio
  • 325
  • 2
  • 10
  • 1
    Part of the confusion is that you are referring to the child processes as daemon processes. [The documentation for supervisord agrees that they are not daemons.](http://supervisord.org/subprocess.html#nondaemonizing-of-subprocesses) – Andrew B Mar 10 '13 at 18:31
  • 1
    It seems like you wrote the child programs yourself, given that you are asking whether to write to stdout or stdout/stderr. If so, what are you gaining by using `runit` or `supervisord`? Is there a reason you don't want to write a regular daemon and use [monit](http://mmonit.com/monit)? – Alastair Irvine Mar 11 '13 at 07:15
  • If you are set on your original plan, I'd suggest that errors and warnings go to stderr while debug/informational/notice messages go to stdout instead. This way you could also run your program from `cron`, with stdout redirected to a log file. If there are any real errors, `cron` will e-mail root. – Alastair Irvine Mar 11 '13 at 07:21
  • 1
    I see that you're still active on ServerFault. To prevent this from showing up on the list of unanswered questions, please either accept an answer or clarify your question further. Thanks! – Andrew B Jun 16 '13 at 20:01

4 Answers4

5

First, something important to clarify: STDOUT and STDERR are rarely relevant within the context of a daemon process once it has successfully started, unless you're invoking one with a debugging switch.

The whole point of a daemon is that it needs to disassociate itself from having a controlling terminal, so that it can persist after logout. Once there is no terminal, all messages need to be sent to either a syslog daemon or a log file managed directly by the process.

If you are not actually referring to daemons, and really just mean any shell script or similar that you're writing yourself, then the logic should generally be this:

  • STDOUT: Anything you want being trapped by pipes or basic output redirection.
  • STDERR: "Out of band" messages. Things you want hitting someone's terminal anyway, even if they're doing some kind of redirection. (hence why they're associated with errors) Let the user decide if they also want to redirect these messages as well, i.e. redirecting STDERR to STDIN as you mentioned.
Andrew B
  • 31,858
  • 12
  • 90
  • 128
  • Thanks for answering! I added more background information to the question according to your answer. Could you please take a look again? Thanks a lot! – Rio Mar 10 '13 at 15:45
  • 1
    I think that my answer remains valid after the edit. If one is designing any kind of foreground process, to conform to the principle of [least surprise](http://en.wikipedia.org/wiki/Principle_of_least_astonishment), it should be designed as if it might be run inside of a terminal. Putting a supervising daemon process in front of it should not change how software is designed. – Andrew B Mar 10 '13 at 18:35
4

I would not suggest co-mingling stdout and stderr inside your program, but how you handle it outside is up to you. stdout was intended for processed data meant for a pipeline, while stderr is specifically for non-data messages. This makes certain behaviors possible, such as batch processing, without altering the existing interactive behavior. Your case is different - there is a good chance stdout means little to nothing.

Because runit has a slightly different take on logging via svlog, and the service you are writing most likely will not daemonize (which in this case means "detach from a tty") then it will be up to you if you want to capture everything to a single log, or not, via the /etc/sv/<servicename>/run script. For the most part, most run scripts use

exec 2>&1

to fuse the two streams together, as most services don't pass along data via stdout. If you do want to use svlog you will need to create a script at /etc/sv/<servicename>/log/run with the appropriate command to start it. It probably looks similar (but not exactly like) this:

#!/bin/sh
exec 2>&1
exec svlog -tt main

where main is a symlink to a logging directory.

Avery Payne
  • 14,326
  • 1
  • 48
  • 87
3

If it's a daemon, I recommend using the syslog interface to log your messages direct to the logs. Run "man 3 syslog" for info. If necessary, your program can also have a debug option (with a parameter that specifies the priority) that tells the daemon program not to run in the background, and to log to stderr too.

I'd write a logging function that takes a priority and string as an argument. It should call syslog() to log the message properly, and if the priority is greater than or equal to the global debug priority value, output message to stderr as well as syslog(). Call syslog(..., "%s", msg) to avoid percent characters in the message from being screwed up. (Or make your function take a variable number of arguments and pass the arglist to vsyslog().)

Make sure you call openlog(..., LOG_PID, LOG_DAEMON) in your program's initialisation. You could add | LOG_PERROR after LOG_DAEMON if you are in debug mode, which would avoid having to write the logging function mentioned above. But then you'd lose the ability to filter based on priority.

You might (if using Ubuntu) need to configure /etc/rsyslog.conf to ensure that daemon messages are being logged.

PS -- Use the logger command if your daemon is a shell script

Alastair Irvine
  • 1,172
  • 10
  • 22
  • Thanks for answering! I added more background information to the question according to your answer. Could you please take a look again? Thanks a lot! – Rio Mar 10 '13 at 15:46
3

I don't think you should use both stdout and stderr for logging. If you have log messages with different severities, they should be tagged with a prefix to make it easy to filter/sort them using svlogd.

I'd say using both stdout and stderr for logging would violate the principle of least surprise.

András Korn
  • 641
  • 5
  • 13