Program to manage process workflow across machines?

Question

I have an in-house Perl script that is in great need of refactoring. In the interest of not reinventing the wheel, I'm looking to see if an open-source equivalent to it exists, and I'm having trouble finding one.

The script runs as a daemon on all hosts, and allows me to do things like:

1. Run ProcessA on Host1.
2. On Host2, once ProcessA has finished successfully on Host1, run ProcessB.
3. On Host1, once ProcessB has finished successfully on Host2, run ProcessC.

It's not rocket science by any means, but I can't seem to find anything out there that replicates that functionality. RunDeck comes close, but I don't see where that can manage process workflows across different hosts.

Does anyone here have anything that I should look into?

Answer 1

This can be done with RunDeck, using a Job for each separate host step, and a higher-level Job to coordinate those jobs. Although rundeck doesn't have exactly that level of coordination within a workflow, it does allow you to chain together multiple Jobs to get the same effect

E.g:

1. Create a Job for each the actions "run processX on hostY" (JobA-C). These jobs would use the Node Dispatch feature with the correct filter to apply to the hostY only.

2. create a top-level Job with a workflow that uses Job References to execute jobs A-C in order, and not using Node Dispatching.

Running the top-level job would step through and run the other jobs in order, waiting until each one finishes before going to the next step.

You could configure the top-level job to also "keepgoing": that is, even if one step fails, continue to execute all subsequent steps.

You could also configure your lower-level jobs to target multiple nodes in parallel, and the top-level job would still do only one step at a time.

Answer 2

Skybot Scheduler may be able to do what you want - not 100% certain it can handle the cross-host dependency bits, but it's pretty robust and it's the first thing that jumps to mind.