0

I am setting up single minute crons to the crontab via a bash script for a load testing purpose. There is no issue with the script executing and the crons are added up and i can monitor crons are being executed via the /var/log/cron.

But the issue is when I informed the script the other day to 106 crons it added and they were executed nicely. But today I reset the crontab from scratch and I only able to set up 85 crons.Then it starts to say

crond[31243]: (root) INFO (Job execution of per-minute job scheduled for 08:32 delayed into subsequent minute 08:33. Skipping job run.)

and eventually all the crons will started to get skipped and nothing will get executed. But initial conclusion I had was 106 is the maximum value that can be st up on this server. But today it got reduced to 85. Server configurations weren't changed;same environmnet as it when it was 106

Is this because of the setting up crons too frequently or something else. I'm new to cron and its workings. Please help

CK LZEM7
  • 3
  • 3

1 Answers1

0

It sounds like you're setting up a cron schedule with crontab with many entries with identical schedules in the expectation that cron will execute those in parallel for you. 

* * * * * /path/to/task1
* * * * * /path/to/task2
* * * * * /path/to/task3
...

That is not exactly what happens:

In most cron implementations I know the tasks will be started in sequential order, not in parallel, in the order the tasks are listed in the crontab file. Cron will start them one after another...
Cron won't need to wait for task1 to complete before starting task2, but will only start task3 after task1 and task2 are started etc. Once started the tasks will run in parallel (for as long as it takes the tasks to complete.)

The tasks that need to be started and the load on the system at the time will limit how fast cron can start new tasks. When cron can't start all tasks in the allotted time interval and the next iteration of your schedule (a new minute) starts, it will forfeit the tasks that couldn't be started. That will result in the error:

Job execution of per-minute job scheduled for 08:32 delayed into subsequent minute 08:33. Skipping job run.

It sounds like you want to start many identical tasks in parallel: 

* * * * * /path/to/task
* * * * * /path/to/task
* * * * * /path/to/task
...

simply use the parallel helper program to that that for you; 

* * * * * parallel -j 50 /path/to/task

Please note that when your parallel tasks don't finish within a single minute, the next minute 50 additional runs of your task will start regardless, which, with the additional load of those tasks that are still running, will also be running longer that 1 minute. After some time your system will probably run out of resources and become completely unresponsive.

HBruijn
  • 72,524
  • 21
  • 127
  • 192