Ubuntu 12.04 Server - eth0 1Gbps NIC eth1 10Gbps NIC - all traffic using eth0?

Question

UPDATE: (per request below)

serveradmin@FILESERVER:~$ cat /etc/hosts
127.0.0.1   localhost
127.0.1.1   FILESERVER

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Ubuntu Server 12.04.1 x64

Primary role is an NFS fileserver, for Mac OSX Clients.

Hardware:

Eth0: 00:19.0 Ethernet controller: Intel Corporation 82579V Gigabit Network Connection (rev 04)

Eth1: 07:00.0 Ethernet controller: MYRICOM Inc. Myri-10G Dual-Protocol NIC

Config:

ifconfig
eth0      Link encap:Ethernet  HWaddr <MACADDRESS>  
      inet addr:192.168.0.150  Bcast:192.168.0.255  Mask:255.255.255.0
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:460042020 errors:0 dropped:148 overruns:0 frame:0
      TX packets:231906707 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:581431978417 (581.4 GB)  TX bytes:259057368617 (259.0 GB)
      Interrupt:20 Memory:f7d00000-f7d20000 

eth1      Link encap:Ethernet  HWaddr <MACADDRESS>  
      inet addr:192.168.0.100  Bcast:192.168.0.255  Mask:255.255.255.0
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:6832208 errors:0 dropped:2 overruns:0 frame:0
      TX packets:376 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:513826442 (513.8 MB)  TX bytes:33688 (33.6 KB)
      Interrupt:59 

lo        Link encap:Local Loopback  
      inet addr:127.0.0.1  Mask:255.0.0.0
      UP LOOPBACK RUNNING  MTU:16436  Metric:1
      RX packets:507 errors:0 dropped:0 overruns:0 frame:0
      TX packets:507 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0 
      RX bytes:45057 (45.0 KB)  TX bytes:45057 (45.0 KB)

nano /etc/network/interfaces

#The loopback network interface
auto lo
iface lo inet loopback

#The primary network interface
auto eth0
iface eth0 inet static
    address 192.168.0.150
    netmask 255.255.255.0
    network 192.168.0.0
    broadcast       192.168.0.255
    gateway 192.168.0.1
    dns-nameservers 192.168.0.1 8.8.8.8

#second network interface
auto eth1
iface eth1 inet static
    address 192.168.0.100
    netmask 255.255.255.0
    network 192.168.0.0
    broadcast       192.168.0.255
    gateway 192.168.0.1
    dns-nameservers 192.168.0.1 8.8.8.8

Currently I am using on the OSX clients: nfs://192.168.0.100/Volumes/Storage to mount the NFS share.

My problem is why would all the data (and I have checked using various monitoring tools bmon, iftop, glances, etc) be going over the slower connection??

Also, after configuring /etc/network/interfaces with the above setup I always get an error message at bootup something about waiting for network configuration. Are these connected?

score 2 · Answer 1 · answered Jun 28 '13 at 23:15

2

You can restrict interfaces to only respond to ARP broadcasts for addresses configured on the interface itself (rather than any on the system) via the following:

sysctl net.ipv4.conf.eth[num].arp_filter=1

eg, in this case:

sysctl net.ipv4.conf.eth0.arp_filter=1
sysctl net.ipv4.conf.eth1.arp_filter=1

Make the change permanent by adding the lines (minus the sysctl command) to /etc/sysctl.conf

answered Jun 28 '13 at 23:15

James Yale

5,042
1
16
20

This will solve the question asked, though I suspect setting up link aggregation would be a much better solution to the problem. – Chris S Oct 02 '13 at 03:00

score -2 · Answer 2 · answered Nov 21 '12 at 05:32

-2

You are directly accessing 192.168.0.100, thus, all traffic will be routed through that IP.

If this is the faster interface and it's routing through x.x.x.150, please post /etc/hosts; it may contain some valuable info.

answered Nov 21 '12 at 05:32

Ajax

121
6

I've been told elsewhere I have borked it up. Please note, these are not bridged interfaces. Just two connections to the same subnet. I've been told this is not ideal, as if eth0 is brought up before eth1 regardless of IP address, eth0 will respond to ARP requests "first". I thought I could just have two connections and choose over which network interface (by assigning IP) I could route traffic. – James Nov 21 '12 at 05:47
Bridging the NICs might be wise; when serving multiple files, the bridge should be able to use both connections at once... But, for the sake of answering your question, could you try running 'tracepath nfs://192.168.0.100'? There was nothing interesting in /etc/hosts :) – Ajax Nov 24 '12 at 07:33
Using wireshark will also help you identify what data is getting routed where. Use it on both the server and the client to get a clear picture of the data routes. – Ajax Nov 24 '12 at 07:34

Ubuntu 12.04 Server - eth0 1Gbps NIC eth1 10Gbps NIC - all traffic using eth0?

2 Answers2