Situation:
At work, we have an in-house tool for data crunching. When a job is triggered, it starts multiple copies of itself in separate processes and communicates with them in order to crunch in parallel. It is currently set up to use 4 parallel processes on a 4-core machine, and the OS naturally allocates each to a core. The program currently runs under Windows 7 64 bit.
Question:
Would it be possible to utilize the processing power of a networked machine to run more parallel processes? I'm wondering whether my case would be easier than some, as the algorithm is already parallelized across multiple processes, which can simply be farmed out to different cores by the system scheduler.
Limitations:
The process virtualization must be transparent to the process; I.e. It can still see all the hardware/drives/NICs in the host system.
The source for the crunching program is lost to history, so I'm limited to using the existing parallelization scheme.
Potential solutions found so far:
- Setting up a VM which can be run across multiple machines that behaves and appears to the virtual OS as a single high-powered machine. (Not sure of the terminology to describe this so struggling to research it).
- A solution such as Incredibuild to farm out processes to other machines (tricky as it seems to require separate executables that can be allocated to different ,cabinets, and also less transparent).
Ideally I suppose I am looking for an out-of-the-box virtualization system, which can virtual use a Windows OS across a cluster. I hope this isn't an insultingly tall order!
Note: I am a control systems engineer and not a systems administrator, so there may be some obvious points that I'm missing. This is also my first question on a SE site!
Thank you.
