Motherboard says PCIe 3.0, but chipset only supports PCIe 2.0. Who's right?

30

4

As part of integration testing for our product, we received a client machine with a Supermicro X10DRi motherboard in it, with 2 Xeon E5-2620 v4 CPUs and a Quadro P5000. That motherboard is using the Intel C612 chipset.

While doing basic research for the integration, I noticed that the motherboard specs say it has 3X PCIe 3.0 x16 and 3X PCIe 3.0 x8 slots. However, the C612 chipset apparently only supports PCIe 2.0, and only up to x4 connections.

While the machine does work, I would like to know if we are taking full advantage of the Quadro. I currently believe that we are limited by the maximum speed of the chipset, as it controls the speed of communication between the CPUs and the GPU.

  1. Am I correct in that belief?
  2. Is there something I am missing/unaware of/incorrect about relating to the communication between the CPU and GPU, whereby the chipset is not a limiting factor?
  3. Is there a way to test the speed of the connection to the GPU?

William Handrigan

Posted 2019-02-07T21:50:07.273

Reputation: 363

Answers

42

Your CPU has 40 PCIe 3.0 lanes: Intel® Xeon® Processor E5-2620 v4

enter image description here

The communications with your GPUs will be direct from your CPU. This is good, after all the memory controller is also integrated on the CPU meaning that the fastest route for data in memory is from RAM -> CPU -> GPU rather than RAM - CPU -> Chipset -> GPU

The motherboard connects the PCIe connections from the CPU to GPU, but that does not mean that they need to pass through the chipset. They go direct from the CPU to the CPU served PCIe slots.

Somewhere in your motherboard documentation it should tell you if any of the PCIe slots are served by chipset or CPU. Chances are the chipset PCIe lanes are used to serve an m.2 slot or SATA controller. Possibly even USB3 or Gigabit Ethernet. How it is wired up and what to is down to the motherboard designer.

Found your effective chipset/CPU/GPU layout on the Intel page for that chipset:

enter image description here

Mokubai

Posted 2019-02-07T21:50:07.273

Reputation: 64 434

1I understand the CPU can support more lanes, but does not the chipset actually define those lanes? In other words, do the lanes go from the CPUs, through the chipset, and then on to the GPU? – William Handrigan – 2019-02-07T21:56:05.217

3I've added details. The motherboard holds the PCIe connections from CPU to GPU, but they do not need to pass through the chipset. They are direct from the CPU to the CPU served PCIe slots. Somewhere in your motherboard documentation it should tell you if any of the PCIe slots are served by chipset or CPU. Chances are the chipset PCIe lanes are used to serve an m.2 slot or SATA controller. – Mokubai – 2019-02-07T21:58:37.683

4Perfect. Thanks for your concise, timely response! – William Handrigan – 2019-02-07T22:01:24.277

8@WilliamHandrigan I found the Intel technical page that shows the effective layout. – Mokubai – 2019-02-07T22:19:50.253

1The X10DRi manual includes a block diagram on page 1-10 which shows how the lanes are split. Slot 2 gets 16 lanes to CPU 1, slots 1 and 3 get 8 lanes each to CPU 1, slots 4 and 6 get 16 lanes each to CPU 2, slot 5 gets 8 lanes to CPU 2. The PCIe 2.0 lanes from the chipset are used for the BMC. The X540/i350 LAN chipsets and the C612 are connected to CPU 1. – Stephen Kitt – 2019-02-08T19:31:31.557

Asked: 2019-02-07T21:50:07.273

Viewed: 5 400 times

Active: 2019-02-07T22:19:12.317