3

I’ve heard that if you have a guest with N virtual cores under ESXi, the hypervisor waits for N total logical processors on the host to become available simultaneously before delegating work from the guest to the hardware. Therefore, the advice is that you should very carefully consider increasing the guest by X cores unless you truly need the processing cycles because you’ll incur an increase in that wait proportionate to X, and you want to ensure your gain from adding vcpus outweighs the cost of this increased delay.

In the extreme example, suppose ESXi hosts hA and hB have identical hardware and configurations, and each has a single guest (gA and gB, respectively) and the guests are identical other than the fact that gA has 1 virtual cpu and gB has 2. If you put the same (non-parallelizable) workload on both hosts, gA “should” compete the task “faster.”

  1. Is this proc wait delay real? Confirmable by VMWare documentation?
  2. If it is real (and not obvious from provided documentation), are there equations to measure projected impact of guest vcpu increases ahead of time? Tooling to measure what kind of real world impact the delay is already having?

—- If it’s at all relevant, the real world problem sparking this question is we have an MS SQL Server 2014 with 4 cores that run on average 60% capacity during the day with regular spikes to 100% and we’re having an internal debate as to whether it’s smart to increase the guest to 6 or 8 cores to alleviate some performance problems we’re experiencing. The host has (I don’t know the model, only the specs) dual socket Intel hexcore at 2.6GHz w/ hyper-threading - so 24 logical cores per host.

Chris Tonkinson
  • 465
  • 2
  • 6
  • 18

2 Answers2

2

There's a lot of poor VMware advice and defunct best-practices out there.

Even VMware's website promotes outdated solutions due to poor SEO.

But the right way to assess this is to use a tool like vSphere Realize Operations Manager (vROPs) to get a sizing recommendation based on your real activity.

Otherwise, just increase and test the impact. Go to 5 vCPU and measure. Then 6 vCPU... etc.

Also, read one of the Technical Deep Dive books for better understanding of resource allocation: https://www.amazon.com/gp/product/1540873064

Example:

enter image description here

ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • I would accept both answers if I could. While you were the first to answer, and I don't think your advice is off base in the least, @John-Mahowald came through with specific documentation to co-stop/start scheduling and esxtop which I think will more precisely answer the exact question as-asked for future travelers. – Chris Tonkinson Oct 10 '17 at 10:24
  • @ChrisTonkinson Ok. – ewwhite Oct 10 '17 at 12:07
1

Hyper-threading is lying about the number of cores to get a few percent boost. You do not have 24 cores, you have 12. Although, I would feel a little better about fully using those 12 cores for guests with hyper-threading.

When a guest crosses the size of a node, you will have NUMA effects upon accessing remote CPUs or RAM. For those hex-core sockets, definitely by vCPU 8. These would also apply to operating systems on the physical server without a hypervisor. Probably manageable, given ESXi and MS SQL have scaled much bigger. Just know that there are diminishing returns.

Strict co-scheduling, where all vCPUs of a VM are stopped if there is a scheduling skew, has not been used since ESX 2. Relaxed co-scheduling is more of a per vCPU decision. You can measure if CPUs are making more progress than others by %CSTP in esxtop.

Whatever the scheduler, for the maximum throughput and minimum latency, do not oversubscribe vCPU. Those dual hexcores get guests totaling 12 vCPU. There is no waiting for idle CPU when you effectively have some dedicated to the guest.

Community
  • 1
John Mahowald
  • 30,009
  • 1
  • 17
  • 32
  • I would accept both answers for different reasons if I could. While @ewwhite answers "correctly" by forgoing the theoretical discussion and promoting a sane measure->change->measure cycle, you directly provide documentation around CPU scheduling under ESXi v5. – Chris Tonkinson Oct 10 '17 at 10:22
  • @ChrisTonkinson You can oversubscribe CPU. Thats one of the major reasons we virtualize. [Please read the book](https://rads.stackoverflow.com/amzn/click/1540873064). – ewwhite Oct 10 '17 at 12:07
  • Oversubscribing CPU is certainly possible, and great for those single user dev test guests. It may not be optimum if the goal is to squeeze the maximum throughput out of a CPU socket and the associated per core software licensing. – John Mahowald Oct 10 '17 at 13:13