Recently we bought 3 Emulex OC10 cards to try in our proxmox test setup. They are the IBM type 49Y4201. The goal is to setup a normal 10G ethernet connection(so no iSCSI or FCoE) in Linux. The cards seem to work in the system, you can get into the firmware bios and setup and flash the leds on both ports etc.
In the OS be2net driver was loaded and complained about old firmware and loss of irqs so after a lot of hassle I flashed the newer 4.1 firmware. After a verify and a reboot, it is again able to go into the bios.
Setup both NICs with proxmox or with ifconfig and then ifup gives a kernel message as follows:
[80221.424544] be2net 0000:21:00.0 enp33s0f0: Link is Down
[80292.286008] be2net 0000:21:00.1 enp33s0f1: Link is Down
and on the other host
[79712.635565] be2net 0000:21:00.0 enp33s0f0: Link is Down
[79716.421629] be2net 0000:21:00.1 enp33s0f1: Link is Down
Cables we have are Cisco branded SFP-H10GB-CU3M and they are listed as supported on IBM's site, but no link LEDs are burning on the card as the cables are plugged in.
The output of ethtool on both hosts:
Settings for enp33s0f0:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: Unknown!
Duplex: Unknown! (255)
Port: FIBRE
PHYAD: 1
Transceiver: internal
Auto-negotiation: off
Supports Wake-on: d
Wake-on: d
Current message level: 0x00002000 (8192)
hw
Link detected: no
be2net driver info:
[ 1.077098] be2net 0000:21:00.0: be2net version is 12.0.0.0
[ 1.077106] be2net 0000:21:00.0: enabling device (0000 -> 0002)
[ 1.077422] be2net 0000:21:00.0: PCIe error reporting enabled
[ 3.252360] be2net 0000:21:00.0: FW config: function_mode=0x20003, function_caps=0x7
[ 3.372359] be2net 0000:21:00.0: Max: txqs 1, rxqs 5, rss 4, eqs 16, vfs 0
[ 3.372361] be2net 0000:21:00.0: Max: uc-macs 30, mc-macs 64, vlans 15
[ 3.372446] be2net 0000:21:00.0: enabled 1 MSI-x vector(s) for NIC
[ 3.480586] be2net 0000:21:00.0: created 1 TX queue(s)
[ 3.504350] be2net 0000:21:00.0: created 1 RX queue(s)
[ 3.603279] be2net 0000:21:00.0: FW version is 4.1.334.36
[ 3.609848] be2net 0000:21:00.0: HW Flow control - TX:1 RX:1
[ 3.617493] be2net 0000:21:00.0: Adapter does not support HW error recovery
[ 3.617678] be2net 0000:21:00.0: Emulex OneConnect: PF vNIC-2 port 0
[ 3.617705] be2net 0000:21:00.1: be2net version is 12.0.0.0
[ 3.617712] be2net 0000:21:00.1: enabling device (0000 -> 0002)
[ 3.617924] be2net 0000:21:00.1: PCIe error reporting enabled
[ 3.868352] be2net 0000:21:00.1: FW config: function_mode=0x20003, function_caps=0x7
[ 3.988362] be2net 0000:21:00.1: Max: txqs 1, rxqs 5, rss 4, eqs 16, vfs 0
[ 3.988364] be2net 0000:21:00.1: Max: uc-macs 30, mc-macs 64, vlans 15
[ 3.988449] be2net 0000:21:00.1: enabled 1 MSI-x vector(s) for NIC
[ 4.096359] be2net 0000:21:00.1: created 1 TX queue(s)
[ 4.120348] be2net 0000:21:00.1: created 1 RX queue(s)
[ 4.242124] be2net 0000:21:00.1: FW version is 4.1.334.36
[ 4.248700] be2net 0000:21:00.1: HW Flow control - TX:1 RX:1
[ 4.256697] be2net 0000:21:00.1: Adapter does not support HW error recovery
[ 4.256884] be2net 0000:21:00.1: Emulex OneConnect: PF vNIC-2 port 1
[ 4.257756] be2net 0000:21:00.1 enp33s0f1: renamed from eth1
[ 4.272475] be2net 0000:21:00.0 enp33s0f0: renamed from eth0
[80221.424544] be2net 0000:21:00.0 enp33s0f0: Link is Down
[80292.286008] be2net 0000:21:00.1 enp33s0f1: Link is Down
Does anyone know how to troubleshoot this? Am I missing something?
