50

I am running the following command every 5 minutes in my crontab to keep Phusion Passenger alive.

*/5 * * * * wget mysite.com > /dev/null 2>&1

When I run this it performs a wget on the site url routes STDOUT/STDERR to /dev/null. When I run this from a command line it works fine and doesn't produce an index.html file in my home directory.

When it runs from cron, it creates a new index.html file every five minutes, leaving me with a ton of index files which I don't want.

Is my syntax incorrect for running the cron job? From a command line it works without a problem but from cron it generates a index.html file in my home directory.

I'm sure I'm making a simple mistake, would appreciate it if anyone could help out.

Martin Schröder
  • 315
  • 1
  • 5
  • 24
nulltek
  • 1,171
  • 3
  • 13
  • 22
  • 1
    Another question is why this isn't creating a file when you run it from the command line by hand. As far as I can tell from the documentation, the only difference between running `wget` from a terminal and otherwise is whether a progress bar is displayed. – Barmar Aug 12 '14 at 21:42

5 Answers5

78

You could do it like this:

*/5 * * * * wget -O /dev/null -o /dev/null example.com

Here -O sends the downloaded file to /dev/null and -o logs to /dev/null instead of stderr. That way redirection is not needed at all.

kasperd
  • 29,894
  • 16
  • 72
  • 122
20

Do you need to actually download the contents or just receive the 200 OK? If you only have to have the server process the request, why not simply use the --spider argument?

Nacht
  • 252
  • 1
  • 10
  • That's a good thought. I really only need the 200 OK response. – nulltek Aug 11 '14 at 19:32
  • I was hoping somebody unbiased would point it out, but... which solution did you end up using? My answer is really the correct way to do this :) – Nacht Aug 18 '14 at 05:20
10

I would use the following:

/5 * * * * wget -O - mysite.com > /dev/null 2>&1

The -O - option makes sure that the fetched content is send to stdout.

Peter Lamby
  • 251
  • 1
  • 3
7

You say you only need the "200 OK" response in a comment.

That allows for solution with some additional advantages over those of
wget -O /dev/null -o /dev/null example.com. The idea is not to discard the output in some way, but not create any output at all.

That you only need the response means the data that is downloaded into the local file index.html does not need to be downloaded in the first place.
In the HTTP protocol, the command 'GET' is used to download a document. To access a document in a way that does everything except actually downloading the document, there is a special command 'HEAD'.
When using 'GET' for this task, the document is downloaded and discarded locally. Using 'HEAD' does just what you need, it does not transfer the document in the first place. It will allways return the same result code as 'GET' would, by definition.

The syntax to use the method HEAD with wget is a little odd: we need to use the option --spider. In this context, it just does what we want - access the URL with 'HEAD' instead of 'GET'.
We can use the option -q (quiet) to make wget not output details about what it does.

Combining that, wget will neither output anything to stderr, nor save a document.

wget -q --spider 'http://example.com/'

The exit code tells us whether the request was successful or not:

$ wget -q --spider 'http://example.com/'
$ echo $?
0
$ wget -q --spider 'http://example.com/nonexisting'
$ echo $?                                          
8

For a command in crontab, the fact that there is no output in both cases means you can use getting no output as an indication of errors again.

Your example command would be changed to this:

*/5 * * * * wget -q --spider mysite.com

This has the same advantages as wget -O /dev/null -o /dev/null example.com. The additional advantage is that the log output, and the document output, are not generated, instead of generated and discarded locally. Or course the big difference is avoiding to download and then discard the document, index.html.

Volker Siegel
  • 175
  • 9
3

to keep Phusion Passenger alive.

May your question should be about this, the webpage says:

A fast and robust web server and application server for

This shouldn't require any keepalive scripts.

Otherwise kasperd's solution is perfect.

user237113
  • 31
  • 1
  • Thanks for the feedback, although it isn't very constructive. Application servers do fail - although it's usually not the container's fault. – Felix Frank Aug 11 '14 at 17:29
  • 1
    I agree it shouldn't require any cronjobs to keep it alive. But it was a quick fix while I research Nginx/Passenger tuning. Was really just looking for the best way to output to /dev/null. I've had passenger fail or hang for 2 minutes at a time when there's no load, so requesting the url keeps passenger fired up for now. – nulltek Aug 11 '14 at 19:41
  • 1
    It would be good to understand, what it is that is being kept alive by the `wget` commands. In many situations the need for keep alive messages is a symptom of an underlying design flaw which should be fixed. But even if all of those are fixed, there will still be a few cases left where a keep alive message is the right solution. Even if keep alive messages are not needed, the cron job could still be a useful part of a monitoring setup. – kasperd Aug 12 '14 at 11:06
  • This would be better as a comment than an answer. – moopet Aug 12 '14 at 11:34