1

I'm searching for open source software to create my own cloud computing environment, like Ubuntu UEC or Eucalyptus.

What I need is that each virtual machine can use resources from multiple nodes (CPU, RAM). And apparently, neither UEC nor Eucalyptus offer this feature.

An example:

I have 5 hardware nodes with each 2x Quadcore. I want that one VM can use up to 20 cores (in this case).

My question

Which open source software can meet this requirement?

#Edit#

At first, I think this is possible with current technology, with computers connected using gigabit network.

Companies like the VPS.net offer services that make a single virtual machine to be formed from the resources of multiple nodes. (At least it seems to be so)

Is this just an abstraction made ​​by VPS.net? Is this approach recommended?

Paulo Coghi
  • 588
  • 1
  • 11
  • 22
  • A single HT bus on a single Opteron (They usually have two or more) runs around 200Gbps, you're 1Gbps network connection ain't even in the ballpark of what 5 dual CPU nodes would take. – Chris S Jun 08 '11 at 13:01

2 Answers2

2

You might need to look at changing the architecture of your hardware to suit the problem. I'm not sure what type of problems you are actually trying to solve with this. You are likely to end up with one very slow VM if you set things up like this.

You might be better served with a four-socket server with 6 or 8-cores in each socket. This would allow you to use 20 cores spread over 3-4 sockets, which would be better for performance.

Using multiple cores across different sockets on a single machine can be tricky to tune for performance. Trying to spread the work across multiple hardware nodes over the network will slow things down further as all data has to pass through the network layer.

The traditional solution to this sort of computing requirement is to use something like MPI or MOSIX. However, it is a solution for a specific type of work-load - where the data is nicely partitioned or can be spread out easily and where synchronisation does not regularly happen.

sybreon
  • 7,357
  • 1
  • 19
  • 19
  • I think you are confusing what they are selling with hardware nodes. I think that they mean VM instances when they say nodes. So, if you need more computing power, you can spawn more VM instances (not have a single VM spread across multiple hardware machines). – sybreon Jun 08 '11 at 11:28
  • 1
    Also, they seem to max out at 12GHz, 7.6GB RAM, 200GB HDD. This would seem to be the size of a single server - Hex-core 2GHz, 8GB RAM, 200GB HDD. – sybreon Jun 08 '11 at 11:30
1

I don't know of any open source software that does this, but there is a commercial product made by a company called scalemp http://www.scalemp.com/ that does exactly this.

Note that they recommend Infiniband rather than ethernet, and even so, the NUMA factor is rather large, so it's far from a general purpose solution.

janneb
  • 3,761
  • 18
  • 22