We have a 3 Unix (1 Ubuntu 2 Redhat) servers connected via their 10 Gbit network cards to a 10 Gbit switch (our IT person made sure we got the correct cables). We experience some odd behaviour when sending files between the servers (either via scp 10Gbit-ip or a folder mounted via fstab using 10 Gbit ip). The speed is usually far from 10 Gbit but slightly over 1 Gbit. And on one of the server the progess bar is very quick but then gets stuck on 100 % for a long long time. Also true for scp on that server.
To make sure we got Full-Duplex and 10000 Mbit/s I used two different functions and they give conflicting information. Does someone have any ideas of what is going on? mii-tools and ethtool don't give the same information:
em1 = 1 Gbit network (via 10 Gbit port) em2 = 10 Gbit network (via 10 Gbit port)
ifconfig:
[admin@XXXX ~]$ ifconfig
em1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet XXXX netmask 255.255.255.0 broadcast XXXXX
inet6 fe80::eef4:bbff:fed5:c230 prefixlen 64 scopeid 0x20<link>
ether ec:f4:bb:d5:c2:30 txqueuelen 1000 (Ethernet)
RX packets 3938043336 bytes 17103106879433 (15.5 TiB)
RX errors 0 dropped 28 overruns 0 frame 0
TX packets 8178327495 bytes 8360136815032 (7.6 TiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
em2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet XXXXX netmask 255.255.255.0 broadcast XXXX
inet6 fe80::eef4:bbff:fed5:c232 prefixlen 64 scopeid 0x20<link>
ether ec:f4:bb:d5:c2:32 txqueuelen 1000 (Ethernet)
RX packets 3314001 bytes 28089131102 (26.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 7632971 bytes 7936298958 (7.3 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions
mii-tool:
[admin@XXX ~]$ sudo mii-tool em1
[sudo] password for admin:
SIOCGMIIREG on em1 failed: Invalid argument
SIOCGMIIREG on em1 failed: Invalid argument
SIOCGMIIREG on em1 failed: Invalid argument
em1: 1000 Mbit, half duplex, link ok
[admin@XXX ~]$ sudo mii-tool em2
SIOCGMIIREG on em2 failed: Invalid argument
SIOCGMIIREG on em2 failed: Invalid argument
SIOCGMIIREG on em2 failed: Invalid argument
em2: 1000 Mbit, half duplex, link ok
ethool:
[admin@XXXX ~]$ sudo ethtool em1
Settings for em1:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: umbg
Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
[admin@XXXX ~]$ sudo ethtool em2
Settings for em2:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 10000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: umbg
Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
As you can see everything looks good with ethtool (full dubplex and full speed) but not with mii-tool (1000 Mb/s and half duplex). What could be the reason and potential test / fix?