3

We are testing the speed of two Intel DC P3600 NVMe drives on a CentOS 7.1 system. They are configured in a software RAID1 using mdadm.

After upgrading the kernel from the (very old) 3.10 to the latest 4.3 (or 4.2.5) we see can observe a very dramatic read speed degradation - from 4GB/s to max 500MB/s. We can replicate the test by selecting the kernel version at boot time.

The tests are conducted using fio with using the following command:

fio --name=read --ioengine=libaio --iodepth=16 --rw=read \
    --bs=128k --direct=1 --size=32G --numjobs=8 --runtime=300 \
    --time_based=1 --group_reporting

What change in the Kernel could cause this? Everywhere I read, the upgrade is even recommended as the NVMe driver reached version 1.0 after kernel 3.19. Any suggestions would be more than welcome.

markz
  • 173
  • 1
  • 4
  • Is Centos 7.1 with a 4.2.5/4.3 kernel supported by Centos? Serverfault is for professional sysadmins keen to run supportable platforms, perhaps superuser.com might be more appropriate for more 'experimental' scenarios – Chopper3 Nov 04 '15 at 18:10

1 Answers1

2

Red Hat kernels are quite different from vanilla ones. Even tunables (eg: sysctl) have significant different default values.

I strongly suggest you to stay with RH kernels unless absolutely necessary. Please at least consider using ELRepo kernels, if you really need a more update kernels.

Anyway, this question should be issued in the linux kernel mailing list, I suppose.

shodanshok
  • 44,038
  • 6
  • 98
  • 162
  • We did use the ELRepo kernel, for production we intend to stay on the official ones. NVMe developer documents suggests upgrading the kernel to a version greater than 3.19 as it should have better driver support. Thanks for the answer – markz Nov 04 '15 at 18:16