1

(Not sure if this is the right group)

I have a situation where I have a number of apps that get created/built. These apps change over time, so they need to be tested. I have a number of distributed test servers, so I can essentially test the apps in parallel on the different test boxes.

The situation I'm facing is one where we can have ~200-300 separate apps that need to be iteratively run/tested, where each run might take ~1-2 hours to run.

I'm looking for some kind of app that would allow me to create a batch/job kind of process, where each batch/job contains the app(s) that are to be run, and I can then schedule the batch to run as required. The process would then have the client machine request the batch/job to run when the machine is ready to run the next batch/job.

I've searched the usual Sourceforge/Freshmeat, as well as looked at some of the globus/cluster management kinds of apps. Haven't quite seen anything close to what I'm looking for.

Thoughts/comments would be helpful.

Thanks

tom smith
  • 441
  • 2
  • 6
  • 10

2 Answers2

2

Buildbot is a system to automate the compile/test cycle. Buildbot supports clusters of machines.

Phil Hollenback
  • 14,647
  • 4
  • 34
  • 51
alvosu
  • 8,357
  • 24
  • 22
1

There are many answers to this question. Keep in mind however that all will require some scripting and customization on your part.

alvosu's suggestion of using Buildbot is excellent and you should definitely check that out first. Some other ideas I can think of include:

Distributed Hudson is worth investigating. Hudson is the standard continuous integration framework that everyone uses today, and you can distribute it's work among a farm of machines.

You mention you've already looked at cluster management software, that's another popular choice although more setup will be required. For example, an interesting approach would be to set up a ROCKS Cluster and use it's default remote execution engine SGE to launch jobs on build hosts.

I'm personally interested currently in the idea of using GNU Parallel as a simple queuing system.

Phil Hollenback
  • 14,647
  • 4
  • 34
  • 51