PCIe 3.0 downgraded to 2.0 speed

Question

I have a dual Xeon board with Intel C602 chipset and two E5-2670 CPUs. The board comes with five PCIe 3.0 x16 slots and a PCIe 3.0 x4 slot; all of which are directly connected to the CPUs.

I added a PCIe 3.0 card to a x16 slot connected to the first CPU ("BSP") as well as another card to the x4 slot (also connected to BSP). In both cases I find the that they are downgraded to PCIe 2.0 speed.

# lspci -vvs 82:00.0 | egrep '(82:00|GT)'
82:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 (prog-if 02 [NVM Express])
            LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
            LnkSta: Speed 5GT/s (downgraded), Width x4 (ok)
            LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
# lspci -vvs 02:00.1 | egrep '(02:00|GT)'
02:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10G X550T (rev 01)
            LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <16us
            LnkSta: Speed 5GT/s (downgraded), Width x4 (ok)

How may I assess why the link speed was downgraded and possibly rectify the problem?

Huh. I also have the AsRock EP2C602. I found the same thing, with everything `downgraded`. In the BIOS, I changed all slots from `2.0` to `3.0`. Since then, only shortly after booting, I get a few btrfs read errors on multiple drives, which indicates some type of hardware problem besides the drives. The reads are corrected, and everything is fine until the next reboot. Are you noticing any issues that sound like this? Changing them back to `2.0` has not fixed the problem. I have an identical system running `3.0` just fine. — user1902689, Jan 07 '19 at 02:10
BTW, make sure NOT to use the 4 white SATA ports closest to the CPUs. Those are ran by the Marvel controller, which has a design defect. Under heavy load, the drives can drop out. It's intermittent, and caused months of headaches for me. ASRock is aware of it, and even with their latest BIOS and Marvel firmware upgrade, the Marvel ports just can't handle heavy load consistently. — user1902689, Jan 07 '19 at 02:13
Thank you for sharing your experience! Our OS is situated in a BTRFS partition on a SATA SSD that we ensured to connect directly to the CPU, and additionally we use ext4 on the NVME connected via PCI-E adapter card, also mounted at boot time. Fortunately, we didn't observe any read errors so far during boots or after. — ypnos, Jan 07 '19 at 09:53

score 2 · Answer 1 · answered Sep 25 '18 at 15:51

2

The C602 only supports PCIe 2.0, it was launched over six years ago.

https://ark.intel.com/products/63984/Intel-C602-Chipset

answered Sep 25 '18 at 15:51

Chopper3

100,240
9
106
238

Yes, but the chipset is not used for the PCIe slots. See also https://serverfault.com/questions/764137/c602-chipset-and-pcie-gen3 Btw. PCIe 3.0 is from 2010. – ypnos Sep 25 '18 at 16:02
Yes but we'd need to know more details of your actual systemboard, we don't know what it is – Chopper3 Sep 25 '18 at 16:23
It's the ASRock EP2C602. – ypnos Sep 25 '18 at 16:45

score 2 · Accepted Answer · answered Sep 25 '18 at 19:09

2

Have you tried reviewing the BIOS, I have found a few times now you can set the PCI-E slots to either work on revision 2.0 or 3.0. If its set to 2.0 in the BIOS the cards will only work at 2.0 even though the card and slot theoretically both support 3.0.

Mike

answered Sep 25 '18 at 19:09

Mike Smith

90
7

Yes, good suggestion, I will have a look at the BIOS at the next scheduled downtime. I saw the option to do so in the manual. – ypnos Sep 26 '18 at 09:24
It appears(?) that the BIOS set these two slots to 2.0 after we added the cards. However after changing it to 3.0 it stays and I get the 8GT/s. – ypnos Oct 12 '18 at 17:17

PCIe 3.0 downgraded to 2.0 speed

2 Answers2