Linux e1000e (Intel networking driver) problems galore, where do I start?

Question

I'm currently having a major problem with e1000e (not working at all) in Ubuntu Maverick (1.0.2-k4), after resume I'm getting a lot of stuff in dmesg:

[ 9085.820197] e1000e 0000:02:00.0: PCI INT A disabled
[ 9089.907756] e1000e: Intel(R) PRO/1000 Network Driver - 1.0.2-k4
[ 9089.907762] e1000e: Copyright (c) 1999 - 2009 Intel Corporation.
[ 9089.907797] e1000e 0000:02:00.0: Disabling ASPM  L1
[ 9089.907827] e1000e 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[ 9089.907857] e1000e 0000:02:00.0: setting latency timer to 64
[ 9089.908529] e1000e 0000:02:00.0: irq 44 for MSI/MSI-X
[ 9089.908922] e1000e 0000:02:00.0: Disabling ASPM L0s 
[ 9089.908954] e1000e 0000:02:00.0: (unregistered net_device): PHY reset is blocked due to SOL/IDER session.
[ 9090.024625] e1000e 0000:02:00.0: eth0: (PCI Express:2.5GB/s:Width x1) 00:0a:e4:3e:ce:74
[ 9090.024630] e1000e 0000:02:00.0: eth0: Intel(R) PRO/1000 Network Connection
[ 9090.024712] e1000e 0000:02:00.0: eth0: MAC: 2, PHY: 2, PBA No: 005302-003
[ 9090.109492] e1000e 0000:02:00.0: irq 44 for MSI/MSI-X
[ 9090.164219] e1000e 0000:02:00.0: irq 44 for MSI/MSI-X

and, a bunch of

[ 2128.005447] e1000e 0000:02:00.0: eth0: Detected Hardware Unit Hang:
[ 2128.005452]   TDH                  <89>
[ 2128.005454]   TDT                  <27>
[ 2128.005456]   next_to_use          <27>
[ 2128.005458]   next_to_clean        <88>
[ 2128.005460] buffer_info[next_to_clean]:
[ 2128.005463]   time_stamp           <6e608>
[ 2128.005465]   next_to_watch        <8a>
[ 2128.005467]   jiffies              <6f929>
[ 2128.005469]   next_to_watch.status <0>
[ 2128.005471] MAC Status             <80080703>
[ 2128.005473] PHY Status             <796d>
[ 2128.005475] PHY 1000BASE-T Status  <4000>
[ 2128.005477] PHY Extended Status    <3000>
[ 2128.005480] PCI Status             <10>

I decided to compile the latest stable e1000e to 1.2.17, now I'm getting:

[ 9895.678050] e1000e: Intel(R) PRO/1000 Network Driver - 1.2.17-NAPI
[ 9895.678055] e1000e: Copyright(c) 1999 - 2010 Intel Corporation.
[ 9895.678098] e1000e 0000:02:00.0: Disabling ASPM  L1
[ 9895.678129] e1000e 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[ 9895.678162] e1000e 0000:02:00.0: setting latency timer to 64
[ 9895.679136] e1000e 0000:02:00.0: irq 44 for MSI/MSI-X
[ 9895.679160] e1000e 0000:02:00.0: Disabling ASPM L0s 
[ 9895.679192] e1000e 0000:02:00.0: (unregistered net_device): PHY reset is blocked due to SOL/IDER session.
[ 9895.791758] e1000e 0000:02:00.0: eth0: (PCI Express:2.5GB/s:Width x1) 00:0a:e4:3e:ce:74
[ 9895.791766] e1000e 0000:02:00.0: eth0: Intel(R) PRO/1000 Network Connection
[ 9895.791850] e1000e 0000:02:00.0: eth0: MAC: 3, PHY: 2, PBA No: 005302-003
[ 9895.892464] e1000e 0000:02:00.0: irq 44 for MSI/MSI-X
[ 9895.948175] e1000e 0000:02:00.0: irq 44 for MSI/MSI-X
[ 9895.949111] ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 9895.954694] e1000e: eth0 NIC Link is Up 10 Mbps Full Duplex, Flow Control: RX/TX
[ 9895.954703] e1000e 0000:02:00.0: eth0: 10/100 speed: disabling TSO
[ 9895.955157] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 9906.832056] eth0: no IPv6 routers present

With 1.2.20 I get:

[ 9711.525465] e1000e: Intel(R) PRO/1000 Network Driver - 1.2.20-NAPI
[ 9711.525472] e1000e: Copyright(c) 1999 - 2010 Intel Corporation.
[ 9711.525521] e1000e 0000:02:00.0: Disabling ASPM  L1
[ 9711.525554] e1000e 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[ 9711.525586] e1000e 0000:02:00.0: setting latency timer to 64
[ 9711.526460] e1000e 0000:02:00.0: irq 45 for MSI/MSI-X
[ 9711.526487] e1000e 0000:02:00.0: Disabling ASPM L0s 
[ 9711.526523] e1000e 0000:02:00.0: (unregistered net_device): PHY reset is blocked due to SOL/IDER session.
[ 9711.639763] e1000e 0000:02:00.0: eth0: (PCI Express:2.5GB/s:Width x1) 00:0a:e4:3e:ce:74
[ 9711.639771] e1000e 0000:02:00.0: eth0: Intel(R) PRO/1000 Network Connection
[ 9711.639854] e1000e 0000:02:00.0: eth0: MAC: 3, PHY: 2, PBA No: 005302-003
[ 9712.060770] e1000e 0000:02:00.0: irq 45 for MSI/MSI-X
[ 9712.116195] e1000e 0000:02:00.0: irq 45 for MSI/MSI-X
[ 9712.117098] ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 9712.122684] e1000e: eth0 NIC Link is Up 100 Mbps Full Duplex, Flow Control: RX/TX
[ 9712.122693] e1000e 0000:02:00.0: eth0: 10/100 speed: disabling TSO
[ 9712.123142] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 9722.920014] eth0: no IPv6 routers present

But, I'm still getting these

[ 9982.992851] PCI Status             <10>
[ 9984.993602] e1000e 0000:02:00.0: eth0: Detected Hardware Unit Hang:
[ 9984.993606]   TDH                  <5d>
[ 9984.993608]   TDT                  <6b>
[ 9984.993611]   next_to_use          <6b>
[ 9984.993613]   next_to_clean        <5b>
[ 9984.993615] buffer_info[next_to_clean]:
[ 9984.993617]   time_stamp           <24da80>
[ 9984.993619]   next_to_watch        <5d>
[ 9984.993621]   jiffies              <24f200>
[ 9984.993624]   next_to_watch.status <0>
[ 9984.993626] MAC Status             <80080703>
[ 9984.993628] PHY Status             <796d>
[ 9984.993630] PHY 1000BASE-T Status  <4000>
[ 9984.993632] PHY Extended Status    <3000>
[ 9984.993635] PCI Status             <10>
[ 9986.001047] e1000e 0000:02:00.0: eth0: Reset adapter
[ 9986.176202] e1000e: eth0 NIC Link is Up 10 Mbps Full Duplex, Flow Control: RX/TX
[ 9986.176211] e1000e 0000:02:00.0: eth0: 10/100 speed: disabling TSO

I'm not sure where to start troubleshooting this. Any ideas?

Here is the result of ethtool -d eth0

MAC Registers
-------------
0x00000: CTRL (Device control register)  0x18100248
      Endian mode (buffers):             little
      Link reset:                        reset
      Set link up:                       1
      Invert Loss-Of-Signal:             no
      Receive flow control:              enabled
      Transmit flow control:             enabled
      VLAN mode:                         disabled
      Auto speed detect:                 disabled
      Speed select:                      1000Mb/s
      Force speed:                       no
      Force duplex:                      no
0x00008: STATUS (Device status register) 0x80080703
      Duplex:                            full
      Link up:                           link config
      TBI mode:                          disabled
      Link speed:                        10Mb/s
      Bus type:                          PCI Express
      Port number:                       0
0x00100: RCTL (Receive control register) 0x04048002
      Receiver:                          enabled
      Store bad packets:                 disabled
      Unicast promiscuous:               disabled
      Multicast promiscuous:             disabled
      Long packet:                       disabled
      Descriptor minimum threshold size: 1/2
      Broadcast accept mode:             accept
      VLAN filter:                       enabled
      Canonical form indicator:          disabled
      Discard pause frames:              filtered
      Pass MAC control frames:           don't pass
      Receive buffer size:               2048
0x02808: RDLEN (Receive desc length)     0x00001000
0x02810: RDH   (Receive desc head)       0x00000001
0x02818: RDT   (Receive desc tail)       0x000000F0
0x02820: RDTR  (Receive delay timer)     0x00000000
0x00400: TCTL (Transmit ctrl register)   0x3103F0FA
      Transmitter:                       enabled
      Pad short packets:                 enabled
      Software XOFF Transmission:        disabled
      Re-transmit on late collision:     enabled
0x03808: TDLEN (Transmit desc length)    0x00001000
0x03810: TDH   (Transmit desc head)      0x00000000
0x03818: TDT   (Transmit desc tail)      0x00000000
0x03820: TIDV  (Transmit delay timer)    0x00000008
PHY type:                                IGP2

and ethtool -c eth0

Coalesce parameters for eth0:
Adaptive RX: off  TX: off
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0

rx-usecs: 3
rx-frames: 0
rx-usecs-irq: 0
rx-frames-irq: 0

tx-usecs: 0
tx-frames: 0
tx-usecs-irq: 0
tx-frames-irq: 0

rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0

rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0

Here is also the lspci -vvv for this controller

02:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet Controller
    Subsystem: Lenovo ThinkPad X60s
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin A routed to IRQ 45
    Region 0: Memory at ee000000 (32-bit, non-prefetchable) [size=128K]
    Region 2: I/O ports at 2000 [size=32]
    Capabilities: [c8] Power Management version 2
        Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
    Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Address: 00000000fee0300c  Data: 415a
    Capabilities: [e0] Express (v1) Endpoint, MSI 00
        DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
            ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
        DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
            RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
            MaxPayload 128 bytes, MaxReadReq 512 bytes
        DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
        LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <128ns, L1 <64us
            ClockPM+ Surprise- LLActRep- BwNot-
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
            ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
    Capabilities: [100 v1] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
        AERCap: First Error Pointer: 14, GenCap- CGenEn- ChkCap- ChkEn-
    Capabilities: [140 v1] Device Serial Number 00-0a-e4-ff-ff-3e-ce-74
    Kernel driver in use: e1000e
    Kernel modules: e1000e

I filed a bug on this upstream, still no idea how to get more useful information.

Here is a the result of the running that script

EEPROM FIX UPDATE

$ sudo bash fixeep-82573-dspd.sh eth0
eth0: is a "82573L Gigabit Ethernet Controller"
This fixup is applicable to your hardware
Your eeprom is up to date, no changes were made

Do I still need to do anything? Also here is my EEPROM dump

$ sudo ethtool -e eth0
Offset      Values
------      ------
0x0000      00 0a e4 3e ce 74 30 0b b2 ff 51 00 ff ff ff ff 
0x0010      53 00 03 02 6b 02 7e 20 aa 17 9a 10 86 80 df 80 
0x0020      00 00 00 20 54 7e 00 00 14 00 da 00 04 00 00 27 
0x0030      c9 6c 50 31 3e 07 0b 04 8b 29 00 00 00 f0 02 0f 
0x0040      08 10 00 00 04 0f ff 7f 01 4d ff ff ff ff ff ff 
0x0050      14 00 1d 00 14 00 1d 00 af aa 1e 00 00 00 1d 00 
0x0060      00 01 00 40 1f 12 07 40 ff ff ff ff ff ff ff ff 
0x0070      ff ff ff ff ff ff ff ff ff ff ff ff ff ff 4a e0

I'd also like to note that I used eth0 every day for years and until recently never had an issue.

I posted a QnA Here that may help https://serverfault.com/questions/616485/e1000e-reset-adapter-unexpectedly-detected-hardware-unit-hang — Kyle Coots, Nov 05 '18 at 01:38

score 23 · Accepted Answer · edited Jan 07 '11 at 08:58

23

Please try booting the kernel with the pcie_aspm=off kernel parameter.

edited Jan 07 '11 at 08:58

Jeff Atwood

12,994
20
74
92

answered Jan 06 '11 at 23:50

Jesse Brandeburg

246
2
3

Amazing! That was it, I'm connected again and everything is working!! – Evan Carroll Jan 24 '11 at 18:45
I found a little bit of info [here](http://lists4.opensuse.org/opensuse-bugs/2010-08/msg00720.html). – Evan Carroll Jan 24 '11 at 18:48
And, on the [Redhat forum](https://bugzilla.redhat.com/show_bug.cgi?id=647077). – Evan Carroll Jan 24 '11 at 18:55
5

btw, I added `kopt=pcie_aspm=off` to `/boot/grub/menu.lst` -- I know this is basic (how to add kernel params), but for those that don't know that's all there is too it if you're using grub. You can also just add `pcie_aspm=off` to your kernel line, but then it only works until you get the next security release. – Evan Carroll Jan 26 '11 at 17:36
Some new systems don't have menu.lst so you should edit /etc/default/grub and add this parameter to GRUB_CMDLINE_LINUX_DEFAULT="pcie_aspm=off" – Josir Mar 26 '21 at 21:35

score 15 · Answer 2 · edited Dec 07 '10 at 17:03

From Intel's driver readme: downloadmirror.intel.com/9180/eng/README.txt

82573(V/L/E) TX Unit Hang Messages

Several adapters with the 82573 chipset display "TX unit hang" messages during normal operation with the e1000 driver. The issue appears both with TSO enabled and disabled, and is caused by a power management function that is enabled in the EEPROM. Early releases of the chipsets to vendors had the EEPROM bit that enabled the feature. After the issue was discovered newer adapters were released with the feature disabled in the EEPROM.

If you encounter the problem in an adapter, and the chipset is an 82573-based one, you can verify that your adapter needs the fix by using ethtool:

 # ethtool -e eth0
 Offset          Values
 ------          ------
 0x0000          00 12 34 56 fe dc 30 0d 46 f7 f4 00 ff ff ff ff
 0x0010          ff ff ff ff 6b 02 8c 10 d9 15 8c 10 86 80 de 83
                                                           ^^

The value at offset 0x001e (de) has bit 0 unset. This enables the problematic power saving feature. In this case, the EEPROM needs to read "df" at offset 0x001e.

A one-time EEPROM fix is available as a shell script. This script will verify that the adapter is applicable to the fix and if the fix is needed or not. If the fix is required, it applies the change to the EEPROM and updates the checksum. The user must reboot the system after applying the fix if changes were made to the EEPROM.

Example output of the script:

 # bash fixeep-82573-dspd.sh eth0
 eth0: is a "82573E Gigabit Ethernet Controller"
 This fixup is applicable to your hardware
 executing command: ethtool -E eth0 magic 0x109a8086 offset 0x1e value 0xdf
 Change made. You *MUST* reboot your machine before changes take effect!

The script can be downloaded at http://e1000.sourceforge.net/files/fixeep-82573-dspd.sh

score 4 · Answer 3 · answered Aug 08 '18 at 04:24

4

As a gravedigger, I'd like to dig up this old ass thread to share my solution for Ubuntu. Download the latest Intel network driver (3.4.0.2 at time of writing). Decompress it, append a thing to a line in the Makefile, get the patch, apply the patch, install, readd the module, and finally, reboot. Good luck.

wget https://downloadcenter.intel.com/download/15817/Intel-Network-Adapter-Driver-for-PCIe-Intel-Gigabit-Ethernet-Network-Connections-Under-Linux-?product=60019
tar -zxvf e1000e-3.4.0.2.tar.gz
vi e1000e-3.4.0.2/src/Makefile
    ## Locate line 152: EXTRA_CFLAGS += $(CFLAGS_EXTRA)
    ## Append -fno-pie
## Download the patch from https://sourceforge.net/p/e1000/bugs/_discuss/thread/9048ab8e 
## wget/curl won't work here. THANKS SOURCE FORGE.
patch -p0 < e1000e-3.4.0.2-timer-updates.patch
cd e1000e-3.4.0.2/src/
sudo make install
sudo rmmod e1000e && sudo modprobe e1000e
sudo reboot

This wad completed on Ubuntu 18.04, kernel 4.15.

answered Aug 08 '18 at 04:24

Justin

167
1
3
12

1

I'll give it a shot and see if it's fixed. you can find the version you're using with `modinfo -F version e1000e`. – Evan Carroll Aug 08 '18 at 04:28
I would be curious what errors yo uwere getting with 18.04. Having not run this, it seems as if the problem has resolved itself. I'm not getting any more hangs though it may have been a router /band/freq issue – Evan Carroll Aug 08 '18 at 04:30
1

Uhh, the interface would just hang for seconds or minutes. Eventually, it would usually crash the system. – Justin Aug 08 '18 at 05:30
1

Thanks for this; I was seeing this with an intel nuc on ubuntu 19.04 each time the interface was initialized. With the manually installed driver, I don't see this any more. I just upgraded to 19.04 because I had mysterious full system hangs (infloop). I wonder if the two things are related? – Wez Furlong Jul 21 '19 at 07:35

score 2 · Answer 4 · edited Oct 06 '18 at 07:31

I had the same issue with a e1000e on Ubuntu 18.04.1, 4.15.0-36 kernel and nothing helped. I've updated the driver directly from intel to 3.4.2.1-NAPI, without any improvements. All things listed in this and the other thread did not help. Guessing from reading through this Ubuntu bug report I have figured as a workaround to significantly reduce the ringbuffer size:

sudo ethtool -G eth0 rx 256 tx 256

This solves the problem completely for me and appears to not impact performance at all (Desktop use, only did basic benchmarks).

Linux e1000e (Intel networking driver) problems galore, where do I start?

4 Answers4

82573(V/L/E) TX Unit Hang Messages

Linked