Running a cron job manually and immediately

Question

(I have already read How can I test a new cron script ?.)

I have a specific problem (cron job doesn't appear to run, or run properly), but the issue is general: I'd like to debug scripts that are cronned. I am aware that I can set up a * * * * * crontab line, but that is not a fully satisfactory solution. I would like to be able to run a cron job from the command line as if cron were running it (same user, same environment variables, etc.). Is there a way to do this? Having to wait 60 seconds to test script changes is not practical.

Answer 1

Here's what I did, and it seems to work in this situation. At least, it shows me an error, whereas running from the command line as the user doesn't show the error.

---

Step 1: I put this line temporarily in the user's crontab:

* * * * *   /usr/bin/env > /home/username/tmp/cron-env

then took it out once the file was written.

Step 2: Made myself a little run-as-cron bash script containing:

#!/bin/bash
/usr/bin/env -i $(cat /home/username/tmp/cron-env) "$@"

---

So then, as the user in question, I was able to

run-as-cron /the/problematic/script --with arguments --and parameters

This solution could obviously be expanded to make use of sudo or such for more flexibility.

Hope this helps others.

Answer 2

I present a solution based on Pistos answer, but without the flaws.

• Add the following line to the crontab, e.g. using crontab -e

  * * * * *  /usr/bin/env > /home/username/cron-env
  

• Create a shell script which executes a command in the same environment as cron jobs run:

  #!/bin/sh
  
  . "$1"
  exec /usr/bin/env -i "$SHELL" -c ". $1; $2"
  

Use:

run-as-cron <cron-environment> <command>

e.g.

run-as-cron /home/username/cron-env 'echo $PATH'

Note that the second argument needs to be quoted if it requires an argument. The first line of the script loads a POSIX shell as interpreter. The second line sources the cron environment file. This is required to load the correct shell, which is stored in the environment variable SHELL. Then it loads an empty environment (to prevent leaking of environment variables into the new shell), launches the same shell which is used for cronjobs and loads the cron environment variables. Finally the command is executed.

Answer 3

As crontab don't do the job, you'll to manipulate it's content :

crontab -l | grep -v '^#' | cut -f 6- -d ' ' | while read CMD; do eval $CMD; done

What it does :

• lists crontab jobs
• remove comment lines
• remove the crontab configuration
• then launch them one by one

Answer 4

By default with most default cron daemons that I have seen, there is simply no way of telling cron to run right here right now. If you're using anacron, it may be possible I think to run a separate instance in the foreground.

If your scripts aren't running properly then you are not taking into account that

• the script is running as a particular user
• cron has a restricted environment (the most obvious manifestation of this is a different path).

From crontab(5):

Several environment variables are set up automatically by the cron(8) daemon. SHELL is set to /bin/sh, and LOGNAME and HOME are set from the /etc/passwd line of the crontab’s owner. PATH is set to "/usr/bin:/bin". HOME, SHELL, and PATH may be overridden by settings in the crontab; LOGNAME is the user that the job is running from, and may not be changed.

In general PATH is the biggest problem, so you need to:

• Explicitly set the PATH within the script, while testing, to /usr/bin:/bin. You can do this in bash with export PATH="/usr/bin:/bin"
• Explicitly set the proper PATH you want at the top of the crontab. e.g. PATH="/usr/bin:/bin:/usr/local/bin:/usr/sbin:/sbin"

If you need to run the script as another user without a shell (e.g. www-data), use sudo:

sudo -u www-data /path/to/crontab-script.sh

The first thing to test before all of that, of course, is that your script actually does what it is supposed to do from the command line. If you can't run it from the command line, it will obviously not work from with cron.

Answer 5

Marco's script didn't work for me for some reason. I didn't have time to debug, so I wrote a Python script which does the same thing. It's longer, but: first, it works for me, and second, I find it easier to understand. Change "/tmp/cron-env" to where you saved your environment. Here it is:

#!/usr/bin/env python
from __future__ import division, print_function

import sys
import os

def main():
    if len(sys.argv) != 2 or sys.argv[1] in ('-h', '--help'):
        print("Usage: {} CMD\n"
              "Run a command as cron would. Note that CMD must be quoted to be only one argument."
              .format(sys.argv[0]))
        sys.exit(1)
    _me, cmd = sys.argv
    env = dict(line.strip().split('=', 1) for line in open('/tmp/cron-env'))
    sh = env['SHELL']
    os.execvpe(sh, [sh, '-c', cmd], env)

if __name__ == '__main__':
    main()

Answer 6

Marco's solution didn't work for me but Noam's python script worked. Here is slight modification to Marco's script that made it work for me:

#!/bin/sh
. "$1"
exec /usr/bin/env -i "$SHELL" -c "set -a;. $1; $2"

The added set -a export variables defined in script $1 and made it available to command $2

p.s. Noam's python worked because it 'exported' environment to the child process.

Answer 7

I noodled on Marco's answer. Code is shown below, but I will maintain this script here.

Given this crontab:

# m h  dom mon dow   command

X=Y
1 2 3 4 5 6 echo "Hello, world"
1 2 3 4 5 6 echo "Goodby, cruel world"
1 2 3 4 5 6 echo "Please spare me the drama"

Sample usage session:

$ cronTest
This is the crontab for  without comment lines or blank lines:
     1  X=Y
     2  echo "Hello, world"
     3  echo "Goodby, cruel world"
     4  echo "Please spare me the drama"
Which line would you like to run as  now?
55
55 is not valid, please enter an integer from 1 to 4
2

Evaluating 1: X=Y

Evaluating 2: echo "Hello, world"
Hello, world

This is cronTest2, which needs to be properly invoked to set up the environment variables the same way as cron does:

#!/bin/bash

# Prompt user for a user crontab entry to execute

function deleteTempFile {
  rm -f $TEMP_FILE
}

function debug {
  if [ "$DEBUG" ]; then >&2 printf "$1\n"; fi
}

function isValidLineNumber {
  # $1 - number of lines
  # $2 - requested line number
  if [[ -n "${2//[0-9]+/}" ]] && (( $2 <= $1 )); then echo true; else echo false; fi
}

function isVariableAssignment {
  [[ "$( echo "$1" | grep "=" )" ]]
}

function makeTempCrontab {
  local -r ASTERISK=\\*
  local -r NUMBER='[[:digit:]]{1,2}'
  local -r NUMBERS="$NUMBER(,$NUMBER)+"
  local -r CRON="^(($ASTERISK|$NUMBER|$NUMBERS)[[:space:]]+)"
  local -r CRON5_REGEX="$CRON{5}"
  local -r CRON6_REGEX="$CRON{6}"

  rm -f "$TEMP_FILE"

  local -r ALL_LINES="$( crontab -l )"

  # Ignore empty lines and lines starting with # (comment lines)
  local -r LINES="$( 
    echo "$ALL_LINES" | \
    grep -v '^[[:space:]]*#' | \
    grep -v '^[[:space:]]*$'
  )"

  if [[ -z "$LINES" ]]; then
    echo "Your crontab is empty, nothing to do"
    exit 1
  fi

  IFS=$'\n' 
  for LINE in $LINES; do
    LINE="$( echo "$LINE" | sed 's/\s\+$//e' )" # remove trailing space
    if [ "$( echo "$LINE" | grep "^$" )" ]; then  
      debug ""  # ignore empty line
    elif [ "$( echo "$LINE" | egrep "$CRON6_REGEX" )" ]; then
      debug "6 field date/time specifier: $LINE"
      # strip out when to run debug, leaving just the command to execute
      echo "$LINE" | cut -f 7- -d ' ' >> "$TEMP_FILE"
    elif [ "$( echo "$LINE" | egrep "$CRON5_REGEX" )" ]; then
      debug "5 field date/time specifier: $LINE"
      # strip out when to run debug, leaving just the command to execute
      echo "$LINE" | cut -f 6- -d ' ' >> "$TEMP_FILE"
    elif [ "$( echo "$LINE" | grep '^@' )" ]; then
      debug "@declaration: $LINE"
      # strip out @declaration, leaving just the command to execute
      echo "$LINE" | cut -f 2- -d ' ' >> "$TEMP_FILE"
    elif [ "$( echo "$LINE" | grep '=' )" ]; then
      debug "Variable assignment: $LINE"
      echo "$LINE"  >> "$TEMP_FILE"
    else
      debug "Ignored: $LINE"
    fi
  done
  unset IFS
}

function runUpToLine {
  # Scans up to given line number in $TEMP_FILE
  # Evaluates variable assignment
  # Executes specified line
  # Ignores remainder of file
  # Function definitions are not supported
  #
  # $1 - line number to run

  readarray CONTENTS < "$TEMP_FILE"
  for (( i=0; i<=$1; i++ )); do
    # >&2 echo "\$i=$i, \$1=$1, isVariableAssignment: $( isVariableAssignment $CONTENTS[$i] ), CONTENTS[$i]=${CONTENTS[$i]}"
    if isVariableAssignment ${CONTENTS[$i]} || (( $i == $1 )); then
      printf "\nEvaluating $(( i+1 )): ${CONTENTS[$i]}"
      eval "${CONTENTS[$i]}"
    fi
  done
}

function selectLine {
  >&2 echo "This is the crontab for $USER without comment lines or blank lines:"
  cat -n "$TEMP_FILE" >&2
  >&2 echo "Which line would you like to run as $USER now?"

  local -r NUM_LINES=$( cat "$TEMP_FILE" | wc -l )
  read LINE_NUMBER
  # >&2 echo "NUM_LINES=$NUM_LINES, LINE_NUMBER=$LINE_NUMBER;  valid: $( isValidLineNumber $NUM_LINES $LINE_NUMBER )"
  while [[ $( isValidLineNumber $NUM_LINES $LINE_NUMBER ) == false ]]; do
    >&2 echo "$LINE_NUMBER is not valid, please enter an integer from 1 to $NUM_LINES"
    read LINE_NUMBER
    # >&2 echo "NUM_LINES=$NUM_LINES, LINE_NUMBER=$LINE_NUMBER;  valid: $( isValidLineNumber $NUM_LINES $LINE_NUMBER )"
  done
  (( LINE_NUMBER-- ))
  echo ${LINE_NUMBER}
}

function doIt {
  export USER=$1
  local -r TEMP_FILE="$( mktemp crontabTest.XXX )"
  trap deleteTempFile EXIT

  makeTempCrontab
  local -r LINE_NUMBER="$( selectLine )"
  runUpToLine $LINE_NUMBER
}

doIt "$1" 

cronTest runs cronTest2 with the proper environment variables set:

#!/bin/bash

# Execute a user crontab entry with the proper environment

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

env -i bash --noprofile --norc -c "$DIR/cronTest2 $USER"

Answer 8

In most crontabs like e.g. vixie-cron you can place variables in the crontab itself like this and then use /usr/bin/env to check if it worked. This way you can make your script work in crontab once you found out whats wrong with the run-as-cron script.

SHELL=/bin/bash
LANG=en
FASEL=BLA

* * * * *   /usr/bin/env > /home/username/cron-env

Answer 9

If it's a shell script, this should get you most of the way:

sudo su  # (assuming it's run as root, if not switch to the user you want it to run as)
cd  # Switch to home folder
sh <full-path/my-shell-script>

It'll definitely highlight some problems, if not everything.

Answer 10

Running a task as cron would is tricky. It requires a modified environment, a non-interactive shell, no attached input terminal, and possibly also a specific shell (e.g. bin/sh instead of /bin/bash).

I have made a script that handle all these issues. Run it with your command/script to run as first argument, and you're good to go. It is also hosted (and possibly updated in Github).

#!/bin/bash
# Run as if it was called from cron, that is to say:
#  * with a modified environment
#  * with a specific shell, which may or may not be bash
#  * without an attached input terminal
#  * in a non-interactive shell

function usage(){
    echo "$0 - Run a script or a command as it would be in a cron job, then display its output"
    echo "Usage:"
    echo "   $0 [command | script]"
}

if [ "$1" == "-h" -o "$1" == "--help" ]; then
    usage
    exit 0
fi

if [ $(whoami) != "root" ]; then
    echo "Only root is supported at the moment"
    exit 1
fi

# This file should contain the cron environment.
cron_env="/root/cron-env"
if [ ! -f "$cron_env" ]; then
    echo "Unable to find $cron_env"
    echo "To generate it, run \"/usr/bin/env > /root/cron-env\" as a cron job"
    exit 0
fi

# It will be a nightmare to expand "$@" inside a shell -c argument.
# Let's rather generate a string where we manually expand-and-quote the arguments
env_string="/usr/bin/env -i "
for envi in $(cat "$cron_env"); do
   env_string="${env_string} $envi "
done

cmd_string=""
for arg in "$@"; do
    cmd_string="${cmd_string} \"${arg}\" "
done

# Which shell should we use?
the_shell=$(grep -E "^SHELL=" /root/cron-env | sed 's/SHELL=//')
echo "Running with $the_shell the following command: $cmd_string"


# Let's route the output in a file
# and do not provide any input (so that the command is executed without an attached terminal)
so=$(mktemp "/tmp/fakecron.out.XXXX")
se=$(mktemp "/tmp/fakecron.err.XXXX")
"$the_shell" -c "$env_string $cmd_string" >"$so" 2>"$se" < /dev/null

echo -e "Done. Here is \033[1mstdout\033[0m:"
cat "$so"
echo -e "Done. Here is \033[1mstderr\033[0m:"
cat "$se"
rm "$so" "$se"

Answer 11

you can program the job to start the next minute :)

Answer 12

Well, the user is the same as the one you put in the crontab entry (or whose crontab you put it into, alternately), so that's a no-brainer. crontab(5) should give you the list of environment variables set, there's only a few.

Answer 13

I've never found a way to run cron jobs manually but this write-up suggests setting the same environment as the cronjob would have and running the script manually.

Answer 14

Inspired by @DjangoJanny's answer, here is what I use to launch the job at the 5th line of my crontab:

 eval "$(crontab -l | sed -n '5p' | tr -s ' ' | cut -d' ' -f 6-)"

Explanation:

• launches the command given by:
• displays the crontab
• gets the 5th line
• replaces multiple spaces by a single space
• takes everything from 6th column to the end