Bash Script to generate <name> and pass this as command line argument to programm

Question

I have an python script which gets 1 command line argument

python arg

Now I want a bash-script which:

• calls python arg & with the argument arg=000
• make this x-times

so therefore:

python 000 &
python 001 &
...
python 00x &

and if one of these processes finishes the script should fork a new process

python 00x+1 &

until a number N.

---

Footnotes:

• I am pretty aware that this sounds like an ugly solution what I am trying to do - but I am forced to do it that way by a various number of reansons, which are out of my domain
• I know i leave all(!) of the work to community, but all my googling and binging went to zero lines of script and therefore I have to leave it as it is.

EDIT: I need a solution that does not use parallel.

Answer 1

It sounds like you want to run multiple instances of the python script in parallel, but only run so many at once and when one finishes, start another. There is a handy utility for that called "parallel", which can be combined with the command "seq" to generate the list of numbers.

 seq -f '%03.0f' 0 N | parallel -j x "python {}"

In the above, replace N with the upper limit. Replace x with the maximum number of scripts to run simultaneously.

seq generates the list of numbers.

$ seq 5
1
2
3
4
5

$ seq 0 5
0
1
2
3
4
5

$ seq -f '%03.0f' 0 5
000
001
002
003
004
005

The last part looks like the numbers sequence that you are wanting to use... 5 being an arbitrary endpoint that you can change.

The parallel -j x "command {}" runs x copies of command simultaneously each time substituting the {} with one of the values from the list generated by seq. So if you have a sequence of 000-100 and you supply -j 3 it will run:

 python 000
 python 001
 python 002

all at the same time. When one of these exits, say "python 001" gets finished first, it will then start "python 003". Suppose "python 000" finishes next, it will substitute "python 004" next and so on until it finishses all 101 jobs.

Obviously you don't want to make -j too high or it will overload your system.

seq is a very standard linux command. parallel is offered as an optional package installable via the package manager for your system. Oddly RedHat based systems don't include it, but you can get source and binaries from the official site:

https://www.gnu.org/software/parallel/

Answer 2

Building on the previous answer, but instead of using GNU parallel, using the standard command "xargs" with option -P option:

seq -f '%03.0f' 0 999 | xargs -n 1 -P 4 python

Try this to see how the processes run in parallel:

seq 0 10 | shuf | xargs -n 1 -P 3 -I % sh -c "echo start %; sleep %; echo done %"