We recently got two QCT 2U4Node high-density servers. Each node is equipped with a XL710 dual 40G NIC with two QSPF+ ports and two QCT mezz cards that sport Intel 82599 and dual 10G SPF+ ports for a total 4 10G ports per node. We use genuine Intel DACs for connecdting XL710s' QSFP+ ports to the 40G QSFP+ ports of QCT LY8 swithes. For 10G ports, we use a set of inexpensive third party DACs connecting them to the 10G ports of two Arista 7280SE-68 switches.
Each of the eight nodes runs CentOS 7.1 x86_64.
Upon testing, we observed some packet drops (via netstat -i
) over some 10G ports. As a result, I decided to build RPMs for the latest stable i40e
and ixgbe
drivers published by Intel on sourceforge. The steps are listed in the README
of each driver's source, so I got both done and installed them on each node using yum -y --nogpgcheck localinstall
.
Then, I rebooted all nodes. After they all came back up, I did modinfo i40e
and modinfo ixgbe
and saw both were the latest. Thus, I thought the update was done.
But today, I was in need of reviewing ethtool -i
output for all nodes, and I got a surprise! Please see below. In summary:
What I did:
- Built the RPMs per the REDME in both drivers' source distributions
- On each node, `yum -y install --nogpgcheck localinstall ...
- Rebooted all the nodes. Observe
modinfo
output to ensure the new version is loaded into the OS
But why ethtool -i
still shows the "stock" version"? Did I miss any step in updating the two Intel drivers?
Thanks for any hints!
40G i40e
$ ansible topfses:bottomfses -a "/sbin/modinfo i40e" -u root|grep version
version: 1.3.38
rhelversion: 7.1
srcversion: AC44711F498386802A6D524
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 1.3.38
rhelversion: 7.1
srcversion: AC44711F498386802A6D524
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 1.3.38
rhelversion: 7.1
srcversion: AC44711F498386802A6D524
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 1.3.38
rhelversion: 7.1
srcversion: AC44711F498386802A6D524
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 1.3.38
rhelversion: 7.1
srcversion: AC44711F498386802A6D524
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 1.3.38
rhelversion: 7.1
srcversion: AC44711F498386802A6D524
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 1.3.38
rhelversion: 7.1
srcversion: AC44711F498386802A6D524
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 1.3.38
rhelversion: 7.1
srcversion: AC44711F498386802A6D524
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
$ ansible topfses:bottomfses -a "/sbin/ethtool -i ens10f0" -u root|grep version
version: 1.0.11-k
firmware-version: f4.22 a1.1 n04.24 e800013fc
version: 1.0.11-k
firmware-version: f4.22 a1.1 n04.24 e800013fc
version: 1.0.11-k
firmware-version: f4.22 a1.1 n04.24 e800013fc
version: 1.0.11-k
firmware-version: f4.22 a1.1 n04.24 e800013fc
version: 1.0.11-k
firmware-version: f4.22 a1.1 n04.24 e800013fc
version: 1.0.11-k
firmware-version: f4.22 a1.1 n04.24 e800013fc
version: 1.0.11-k
firmware-version: f4.22 a1.1 n04.24 e800013fc
version: 1.0.11-k
firmware-version: f4.22 a1.1 n04.24 e800013fc
10G ixgbe
$ ansible topfses:bottomfses -a "/sbin/modinfo ixgbe" -u root|grep version
version: 4.1.2
rhelversion: 7.1
srcversion: 11CE41D9B741B44F4507285
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 4.1.2
rhelversion: 7.1
srcversion: 11CE41D9B741B44F4507285
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 4.1.2
rhelversion: 7.1
srcversion: 11CE41D9B741B44F4507285
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 4.1.2
rhelversion: 7.1
srcversion: 11CE41D9B741B44F4507285
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 4.1.2
rhelversion: 7.1
srcversion: 11CE41D9B741B44F4507285
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 4.1.2
rhelversion: 7.1
srcversion: 11CE41D9B741B44F4507285
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 4.1.2
rhelversion: 7.1
srcversion: 11CE41D9B741B44F4507285
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
version: 4.1.2
rhelversion: 7.1
srcversion: 11CE41D9B741B44F4507285
vermagic: 3.10.0-229.11.1.el7.x86_64 SMP mod_unload modversions
$ ansible topfses:bottomfses -a "/sbin/ethtool -i ens20f0" -u root|grep version
version: 4.0.1-k-rh7.1
firmware-version: 0x800004e0
version: 4.0.1-k-rh7.1
firmware-version: 0x800004e0
version: 4.0.1-k-rh7.1
firmware-version: 0x800004e0
version: 4.0.1-k-rh7.1
firmware-version: 0x800004e0
version: 4.0.1-k-rh7.1
firmware-version: 0x800004e0
version: 4.0.1-k-rh7.1
firmware-version: 0x800004e0
version: 4.0.1-k-rh7.1
firmware-version: 0x800004e0
version: 4.0.1-k-rh7.1
firmware-version: 0x800004e0