1

We have an application that sends and receives a lot of network traffic - but interrupts are causing us to get lower than max performance. At 20 Mbps input and 20 Mbps output (on a 1 Gbps line), the network interrupts use 100% of core 0 and the operating system freezes up (checked through Windows Performance Analyzer). Using the RSS feature on the network card (BroadCom) didn't ameliorate the situation.

I understand you can edit the affinity of specific interrupts through the registry - my aim being to use a second core for network interrupts like I would do on linux - but don't know how to go about finding the right key to edit. Any other advice is very welcome.

akonskarm
  • 13
  • 1
  • 4

1 Answers1

1

I'll provide a partial answer with clues as to how and where I believe you may find a concrete answer specific to your system.

The function is heavily tied to the NDIS version of the operating system.

An early reference is the ProcessorAffinityMask parameter of the HKLM\SYSTEM\CurrentControlSet\Services\NDIS\Parameters key. It sets which processors are to be made available for servicing network adapter driver interrupts in Windows 2000.

The Receive Side Scaling subsystem evolved in later versions to map network processing queues to the available processors, described here for Windows 2008.

Here is the documentation for RSS in Windows 2012 R2 (NDIS 6.40 I believe) and a clue is given in the early phrase:

To process received data efficiently, a miniport driver's receive interrupt service function schedules a deferred procedure call (DPC). Without RSS, a typical DPC indicates all received data within the DPC call. Therefore, all of the receive processing that is associated with the interrupt runs on the CPU where the receive interrupt occurs.

At the end of this link is a link to reference material for all 6.x NDIS versions. I believe you can use these references to work out an answer for your particular system, as well as identify precise terms to google ambiguities with.

Just based on the info the question provides (which isn't too detailed), what do I see in the docs?

The first processor in the processor set to be made available to RSS is set using the HKEY_LOCAL_MACHINE\\SYSTEM\CurrentControlSet\Services\NDIS\Parameters\RssBaseCpu parameter in Windows 2008 as described here. Hopefully this is valid also for Windows 2012 R2.

Similarly, the max number of processors to be available to RSS is set using the HKEY_LOCAL_MACHINE\\SYSTEM\CurrentControlSet\Services\Ndis\Parameters\MaxNumRssCpus as described here.

Given this, on Win2008 and hopefully also in Win2012, setting RssBaseCpu to 1 and MaxNumRssCpus to 1 should lock RSS processing to the second logical CPU and no others, assuming RSS has successfully been activated. RSS status can be checked from within Windows through an administrative command prompt like so:

netsh int tcp show global

as documented here.

Given this context, it is quite possible that your problem is simply due to a poorly written NIC device driver too. I've had quite a bit of trouble with NIC offloading mechanisms (the general category of which RSS belongs to), specifically with Broadcom integrated NICs on Dell servers and the associated drivers. I would encourage you to also pursue that possibility if the system doesn't act as one would expect based on the quadruple-checked configuration.

ErikE
  • 4,676
  • 1
  • 19
  • 25
  • But is RSS a technology that depends on the protocol used? Is it only for tcp? – akonskarm Aug 05 '15 at 14:55
  • No, it is multiprotocol these days: https://technet.microsoft.com/en-us/library/hh997036.aspx – ErikE Aug 05 '15 at 17:11
  • 1
    RSS has not been working for me, but I came upon this: https://msdn.microsoft.com/en-us/library/windows/hardware/ff570726(v=vs.85).aspx All my traffic is using the same source and the same destination IP always. So my guess is that the hash function assigns all traffic to the same single RSS queue which is assigned to the same single cpu core. – akonskarm Aug 19 '15 at 10:41
  • 1
    Thank you for your help! mixing the traffic with more (src IP, dest IP) packets made RSS work :) – akonskarm Aug 21 '15 at 12:56