28

I am trying to find some documentation or best practice guides for virtualization with respect to provisioning vCPUs per physical core (of a CPU). If it matters, I am looking at vmWare for the virtualization implementation. For example, an Intel Xeon CPU may have 4, 8, etc. cores. I am interested in learning more about provisioning beyond just one vCPU per one physical core. The vendor I am talking to definitely thinks that a single core can be provisioned into multiple vCPUs.

What I commonly see in my research thus far is, "Well, it depends on your application." And in that case, my application is editing code, compiling/linking, testing, and configuration management. Of course not all of the VMs need to be configured with multiple vCPUs per core, but in the general case.

CC.
  • 1,186
  • 1
  • 10
  • 22
Dr. Watson
  • 383
  • 1
  • 3
  • 7

4 Answers4

24

A single physical CPU can be utilized as many vCPUs. You rarely run out of CPU resources in virtualization solutions. RAM and storage are always the limiting factors...

Remember, in VMware, CPU utilization is represented in MHz used, not cores... Unless you're pegging all of your virtual CPUs at 100% ALL OF THE TIME, I don't think your vendor is correct.

Let's look at the following cluster of systems...

  • 9 ESXi hosts.
  • 160 virtual machines
  • 104 physical CPU cores across the cluster.
  • The average virtual machine profile is: 4 vCPU and 4GB to 18GB RAM.
  • CPU can safely be oversubscribed... but remember, it can also be limited, reserved and prioritized at the VM level.

enter image description here enter image description here

from another active cluster - 3 hosts 42 virtual machines enter image description here

ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • 4
    After seeing 3936 vMotion migrations I don't feel too bad about our 1,200 – Mark Henderson May 03 '13 at 07:54
  • 2
    I was looking at the stats on our VM cluster yesterday which bear out the figures given by @ewwhite. We have 3 hosts, totaling 24 processors and 55 GHz. We have 59 VMs with a total of 79 vCPUs allocated. According to vSphere's stats, over the last 6 months, we have averaged a bit over 14 GHz used (min 9 GHz, max 25 GHz), and during that time there was 0 CPU Core Count Contention. – Paul Gear May 07 '13 at 22:16
7

To expand on ewwhite's write-up, unless you have applications which can explicitly take advantage of multiple vCPUs, or multiple cores per vCPU there is absolutely zero benefit in allocating multiple vCPUs/cores to a VM. In fact, more often than not you will actually end up with lower performance as opposed to running on a single vCPU that has one core assigned to it, in part because of the scheduling overhead required to run multiple vCPUs.

FWIW, in a VDI setting the often cited number is 5 vCPUs per physical core. Of course that's taking office work desktops into account. If your VMs are really busy with compiling code all the time you may not be able to fit 5 vCPUs per physical core.

The reason why so many people say that "it depends" is because it really does. Look at your CPU Ready values and then decide whether you can put more CPU load on a particular system. CPU Ready is a measurement of the vCPU being ready to execute a command but it has to wait for physical CPU time to become available.

In your case, if you are compiling large programs, it's entirely possible that your VMs will actually need a lot of CPU time. As ewwhite noted, normally virtualization tends to be disk I/O and RAM constrained rather than CPU constrained though.

Reality Extractor
  • 1,480
  • 2
  • 14
  • 23
  • 13
    `absolutely zero benefit in allocating multiple vCPUs/cores to a VM` - not entirely correct. We have a single-threaded application that used to hang on a weekly basis. When a single vCPU was at 100% it was impossible to get onto that system and we had to do a hypervisor level reset of the VM. We added a 2nd vCPU and when the app hung, we were able to easily get in and kill the offending thread. This is a bit of an edge case true, but you can never deal in absolutes. – Mark Henderson May 03 '13 at 07:52
  • 4
    What Mark wrote is the reason why we use two cores as the lower limit for every VM - not matter wether they need it, or not. – Nils May 03 '13 at 08:05
  • 2
    Ready value is the best way to tell if you are over provisioning your host, your ready value should be as low as possible. It is the %of time a VM is "ready" to use a CPU cycle, but has to wait as the CPU is busy with another task. To explain why giving your vm many cores can have a performance impact I will use a simple example: if I have 4 physical cores, and 4vms, if you have 3 VMs with 2cores and one vm with 4 cores, your smaller VMs will actiually get more cycles, as they can "fit" better in multiples. Go as conservative as possible with you vcpus! – Rqomey May 03 '13 at 21:52
  • @MarkHenderson I can see how that could be beneficial, and something like that could potentially happen to the guy who asked the question since he works with compilers. – Reality Extractor May 04 '13 at 03:26
  • @Nils I believe that giving every VM 2 cores whether there is a business need for it or not is a really bad idea. It could easily negatively affect you in so many ways, slot size, not enough cores available to restart/failover, lost performance due to what Rqomey wrote, just countless other things. You can pretty much always get into your VM via vCenter and DCUI. – Reality Extractor May 04 '13 at 03:28
  • @RealityExtractor so the VM-admin is the same person as the VCenter-admin? In that case - ok. Else the work is going from the VM-admin to the VCenter-admin if there are overload-situations. – Nils May 06 '13 at 09:01
3

The underlying problem is basically the same as with process-scheduling on a physical system. As long as the system load is below the number of cores (or even logical processors, in case of HyperThreading) all is well and the processors can handle the load.

So as long as the concurrent load on all used vCPUs does not exceed the load that can be handled by your physical cores all is well.

For your demands only compiling is a CPU-intesive work, which is only needed from time to time. For the compiler-VMs we allocat as many CPUs as available. So if there is a need to compile, it will be done as fast as possible (if your compiler supports paralell compiling).

This might not be true for a compiler-VM that is under constant load (e.g. if you provide a internet-service to make compiles and that is constantly being used).

Nils
  • 7,657
  • 3
  • 31
  • 71
2

One rule of thumb i've seen (possibly in VMware's documentation) is not to allocate more cores to a VM than physically exist on the host, because that would cause multiple vCores to be emulated on a single core, adding unnecessary overhead.

Paul Gear
  • 3,938
  • 15
  • 36
  • 1
    What about the reverse ? What if I have a 6-core Xeon and I only allocate 4 cores in the VM ? What would be the performance then ? Will the VM system be able to harvest power from all 6 physical cores or will it be limited to 4 ? – Overmind Sep 20 '16 at 11:49
  • If you only give it 4 cores, it will only have access to 4 cores at a time. But my understanding (not confirmed) is that those 4 virtual cores might be allocated on any 4 of the 6 physical cores, unless you've configured pinning. – Paul Gear Sep 21 '16 at 23:11