16

I have a server running Ubuntu Server with four IP addresses aliased on a single NIC. 

eth0       192.168.1.100
eth0:0     192.168.1.101
eth0:1     192.168.1.102
eth0:2     192.168.1.103

(Using 192.168.x.x for sake of example, assume these are NAT-ed to a range of public IP addresses)

One of our clients publishes their inventory via FTP, so we log in nightly to download a large file from their server. Their firewall expects our (passive) FTP connection to be made from 192.168.1.100.

Given that my server logically has four IP addresses on a single adapter, how does the operating system determine which IP address is used as source for outbound TCP/IP connections?

Let's say I ssh into my server on 192.168.1.101 and run FTP interactively. Will the outbound TCP/IP connection use 192.168.1.101 because the OS knows that's the interface over which my shell is connected?

What if the FTP task is run non-interactively via a cron job where there is no shell?

As you can probably tell, this has me quite confused, so I hope my questions have at least made sense.

Edit

To clarify why I'm asking -- I haven't made any changes to the routing table and it actually lists 'eth0' as the IFace for the 0.0.0.0 routes. However, all indications are that it is actually using eth0:0 as the source. 

Destination    Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0
0.0.0.0        192.168.1.1     0.0.0.0         UG    100    0        0 eth0

I can fiddle with the routing table or have our client change their firewall rules to get the behavior I need, but I'm trying to gain a little insight into how this works to know if there's a bug in the OS or just my naive understanding of how all the pieces fit together.

Thanks

Joe Holloway
  • 1,829
  • 3
  • 19
  • 17

6 Answers6

12

By default, on Linux, if an interface has multiple addresses that are on different subnets, traffic destined for the respective subnets will have the proper source IP. That is, if eth0 has two addresses 192.168.1.1/24 and 10.1.1.1/8, then traffic to anything on the 10.0.0.0 subnet will have source 10.1.1.1, and traffic to anything on the 192.168.1.0 subnet will have source 192.168.1.1. You can also assign source addresses explicitly in this case by using the "src 1.2.3.4" option to "ip route".

In your case, though, all your addresses are on the same subnet, so the "primary" one (as revealed by "ip addr list dev eth0") is used as the source IP for traffic exiting on that interface. I think it's possible to control the source IPs in this case just using "ip route", but I've found it easier to use iptables to rewrite the source addresses for traffic of interest.

If you want to force a specific source address to be used for specific destinations, you can do it with a SNAT rule:

iptables -t nat -I POSTROUTING -o eth0 -d dest-IP-or-net/mask -s primary-IP-of-eth0 -j SNAT --to-source desired-source-IP

So if your "primary" eth0 IP is 192.168.100.1, but you want traffic to 1.2.3.4 to have a source of 192.168.100.2, then do this:

iptables -t nat -I POSTROUTING -o eth0 -d 1.2.3.4/0 -s 192.168.100.1 -j SNAT --to-source 192.168.100.2

Note that the "-s 192.168.100.1" is important: it prevents forwarded traffic's source addresses being rewritten by this rule.

If you are going to implement complex network configurations on Linux, you should read the Linux Advanced Routing and Traffic Control documentation, http://lartc.org

user14017
  • 136
  • 3
5

It uses whatever the default gateway is in the routing table, unless there is a specific route telling it to use another: route -n

EDIT: I read your question too quick it seems...

Since you are using passive mode and the client will always be initiating the connection, I think src ip field in the IP header will always appear as whatever IP the client connected to. If it were active mode the server was initiating the connection, I think it would always be the 'Primary' IP. If your addresses are in the same subnet, Linux will make the first address you added 'Primary' and the others secondary.

I am not entirely sure though, I would run tcpdump -n and see what it sees as the src IP.

EDIT2: Okay, I wrote the above from the standpoint that you were running the server, so since you are the client and initiating the connection, I think it will always appear to come from the Primary IP address, but again, try it and see with tcpdump.

Kyle Brandt
  • 82,107
  • 71
  • 302
  • 444
  • The routing table is just the default and there's only one subnet. Another poster indicated that I misused the term multihoming. However, I'd still expect it to be using the eth0 alias. I used wget to download http://whatsmyip.net/ and it shows me that I'm somehow using eth0:0 instead. – Joe Holloway May 26 '09 at 19:29
  • That doesn't really make sense to me, whatsmyip.net should show your public ip... – Kyle Brandt May 26 '09 at 19:46
  • To be more clear, it shows the public IP address that is NAT-ed to the private IP associated with eth0:0, whereas I would expect it to show the public IP address that is NAT-ed to the private IP associated with eth0 – Joe Holloway May 26 '09 at 20:50
  • 1
    This answer here is upvoted too much and contains to many edits and confuses many things and does not help. The answers of tbman and jknapka are excellent and helped me very much. – Christian Aug 15 '10 at 10:19
5

I see in your example that all the ips are too close not to be in the same network

are you sure that you are actually multihoming and not simply having 4 IP aliases?

if the latter is the case then you can set the source ip on a route with something similar to this

/sbin/ip route show 192.168.222.0/24 dev eth0 proto kernel scope link src 192.168.222.178 169.254.0.0/16 dev eth0 scope link default via 192.168.222.1 dev eth0

sudo /sbin/ip route replace default via 192.168.222.1 src 192.168.222.178

/sbin/ip route show
192.168.222.0/24 dev eth0 proto kernel scope link src 192.168.222.178 169.254.0.0/16 dev eth0 scope link default via 192.168.222.1 dev eth0 src 192.168.222.178

see man interfaces on how to make it persistent between reboots

Aleksandar Ivanisevic
  • 3,327
  • 19
  • 24
4

Unless your FTP job has a way of specifying the interface to use for connections, I believe it defaults to the first physical interface on the relevant subnet (eth0 in this case). If you had a server with two NICs on different subnets, it'd figure out which interface to use based on the routing table.

As there is only a single physical interface on the system (eth0) and four virtuals/aliases (eth0:0 through eth0:2) on the same subnet, outbound traffic will use the eth0 IP address as the source unless the application is smart enough to declare an outbound interface.

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
  • 3
    This was my assumption, but all of my tests indicate that it's using eth0:0 as the source. – Joe Holloway May 27 '09 at 02:37
  • Then perhaps the default route is configured to use the eth0:0 interface. I have an install that uses an ethernet bridge, and it was configured to use the virtual bridge interface for the default route. – sysadmin1138 May 27 '09 at 05:00
  • 1
    I set up a trivial program on another machine to print the IP it received a connection from. Connecting to it with `nc -s ` or whatever always shows the source address is actually the ip of eth0:0, even though netcat did a `bind(2)` before `connect(2)`. So it seems like aliases don't work for giving a machine the ability to make connections from multiple source addresses. – Peter Cordes Apr 01 '15 at 16:21
4

You could see which device and src ip address will be used by ip route get command like below:

$ /sbin/ip route get 1.1.1.1
1.1.1.1 via 2.2.2.2 dev eth0  src 2.2.2.2 
    cache  mtu 1500 advmss 1460 hoplimit 64

I haven't tried this in aliased environment, but hope this helps.

tomoe
  • 430
  • 2
  • 5
1

When establishing an outbound connection, your server will look in its routing table to determine which of your four interfaces to use; your TCP connections will have a source IP of your exit interface.

netstat -rn

Will give you the output of your routing table; look for any specific entries matching the client IP you are trying to connect to. If none exist, then you will be using a default route (0.0.0.0, mask 0.0.0.0). If you have multiple default routes, the one with the lowest cost will be the one used.

Murali Suriar
  • 10,166
  • 8
  • 40
  • 62