2

I'm in the process of attempting to benchmark a multithreaded application on my new HP Proliant Server which has an 2x Opteron 6272 and 64gb of ram.

When I run the application on a desktop machine (a range of i7s and a Xeon X5675 processors) the application will cause all cores to hit a near 100% utilization.

When I run the application on my server, no matter how many threads I run, the total cpu utilization of the application hovers around 20-25%. That is if I'm running with 32 threads, all 32 cores will hang at around 20%, if I run 16 threads they'll hang around 40%, and so on.

  1. At first I suspected this had to do with the operating system, so I installed Windows 7 on the server so that the desktops and the server had the same OS.
  2. Then I suspected it was the hardware, I changed the power management in the bios to High Performance. Even though this did increase the benchmark time, the same 20% utilization problem persists.
  3. I can get all 32 cores at 100% using the y-cruncher benchmark. My custom benchmark is written in .NET, could this possibly have anything to do with it?

I'm perplexed by this problem. Anyone have an idea of what could cause this?

vpiTriumph
  • 73
  • 6
  • What does the application do? Are you sure it's CPU limited? – David Schwartz Apr 21 '12 at 01:35
  • @DavidSchwartz, the application runs a set of advanced satellite access calculations. I'm stuck using 3rd party libraries, so I haven't entirely ruled out something being screwy with that code, I'm chasing that possibility down too. As I mentioned in the post though, the fact I'm able to get CPU utilization up to 100% on a machine with 2x Xeon X5675s (24 cores total) makes me feel like the 3rd party libraries are doing their multithreading correctly. – vpiTriumph Apr 21 '12 at 06:24
  • 1
    @vpiTriumph what David meant was: did you check that some other resource (I/O seems likely) is not bottlenecking your threads? – the-wabbit Apr 21 '12 at 07:45
  • I have not @syneticon-dj is there a tool you would recommend for doing that? – vpiTriumph Apr 21 '12 at 17:37
  • 2
    Perfmon. Take a look at the [most interesting relevant counter values in the Windows docs](http://technet.microsoft.com/en-us/library/cc768048.aspx). As a rule of thumb, if the average queue length exceeds the number of disks in your storage subsystem, you are seeing a storage bottleneck. If you see any significant amount of idle time, you probably are not. – the-wabbit Apr 22 '12 at 14:12
  • @syneticon-dj I am hovering around 100 for PhysicalDisk % Idle Time and near 0 (0.000-0.001) Avg. Disk Queue length. Anything else I should check out? – vpiTriumph Apr 23 '12 at 15:20

1 Answers1

4

If your app is processing a large amount of data, try following the data's path - if the input data is fed from the network, check for possible latencies, bandwidth limitations or transmission errors. You already checked disk I/O which otherwise would be a likely candidate for a bottleneck.

Last but not least, since it is a highly multithreaded .NET application, you should make sure that server garbage collection is used, otherwise you might see weird load characteristics as described in this post from stack overflow.

Community
  • 1
the-wabbit
  • 40,319
  • 13
  • 105
  • 169