0

I have a multi-homed system (Ubuntu server, 11.04) where eth0 is lan and eth1 is wan. There is internet connectivity through both, but via different routes.

When the system boots, I cannot connect to e.g. apache(TCP 443) on the WAN interface dispite the fact it listens on *.443.

If I bring down eth0 I can, however. After bringing eth0 back up I can still access apache.

There are no IPTables rules active (Policy: accept).

Information summary:

System & Kernel

Ununtu Server 11.04
$ uname -r
Linux myserver 2.6.38-11-server #50-Ubuntu SMP Mon Sep 12 21:34:27 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

Network interfaces:

eth0 is lan, dhcp assigned.
eth1 is wan, static ip.

IPTables

No rules set.
Policy is ACCEPT

eth1 outbound connectivity works.

I have tested outbound connectivity on eth1 using nmap, and this works:

$ nmap -sT -P0 -p80 -eeth1 google.com)
port is open.

Test from WAN there shows no access.

 # nmap -sS -P0 -p443 myhost 

 states the port is filtered

Test from LAN succeeds using Chrome.

Apache is listening correctly:

$ sudo lsof -i :443
COMMAND PID     USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
apache2 822     root    4u  IPv4   7965      0t0  TCP *:https (LISTEN)
apache2 857 www-data    4u  IPv4   7965      0t0  TCP *:https (LISTEN)
apache2 858 www-data    4u  IPv4   7965      0t0  TCP *:https (LISTEN)

The following will enable inbound connections:

# ifconfig eth0 down

# ifconfig eth0 up

Contents of /etc/network/interfaces

# The loopback network interface
auto lo
iface lo inet loopback


# The primary network interface
auto eth0
iface eth0 inet dhcp

auto eth1
iface eth1 inet static
    address xx.yy.zz.ww
    netmask 255.255.255.128
    gateway xx.yy.zz.1
    nameserver 8.8.8.8

Question

I'd like to simply ask "What is wrong?", but I' not expecting there to be enough info in this question for that to work.

Instead, I'll ask: Where do I start looking for the cause of the problem ? What information do I need to fetch ?

Edit: netstat -rvn output

Just after reboot

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface 
xx.yy.zz.0      0.0.0.0         255.255.255.128 U         0 0          0 eth1
10.10.53.0      0.0.0.0         255.255.255.0   U         0 0          0 eth0
0.0.0.0         10.10.53.1      0.0.0.0         UG        0 0          0 eth0
0.0.0.0         xx.yy.zz.1      0.0.0.0         UG        0 0          0 eth1

eth0 down

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
xx.yy.zz.0      0.0.0.0         255.255.255.128 U         0 0          0 eth1
0.0.0.0         xx.yy.zz.1      0.0.0.0         UG        0 0          0 eth1

eth0 back up

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
xx.yy.zz.0      0.0.0.0         255.255.255.128 U         0 0          0 eth1
10.10.53.0      0.0.0.0         255.255.255.0   U         0 0          0 eth0
0.0.0.0         xx.yy.zz.1      0.0.0.0         UG        0 0          0 eth1
Thomas
  • 1
  • 2
  • can you provide the output of 'netstat -rvn' when eth0 is up and when eth0 is down? – paulos Oct 14 '11 at 11:26
  • Post updated with netstat output – Thomas Oct 14 '11 at 12:16
  • traceroute from external world to Apache, pls! And parts, related to host, from httpd.conf. BTW, you can even *not* hide IP, because it's not a trouble for correctly-configured host and will allow some debug for outsiders. For *my hosts* I published - distro -version -kernel version -applied pathes and didn't get any penetrations – Lazy Badger Oct 14 '11 at 16:19
  • @LazyBadger: Traceroute to apache ? You mean traceroute to the WAN IP? That'd give nothing, as there clearly is a functioning route to the WAN IP. Regarding the config files: They are completely irrelevant. As you can see, the service listens on all interfaces. The error is not at the HTTP level, it is at the TCP level. (And you didnt get any penetrations _that you know of_) – Thomas Oct 17 '11 at 09:37
  • Yes, I mean **this exactly!!!** And for normal real admin I haven't "translate" obvoious thing. If you are so genius, I don't want spent my free time on your troubles. AMF! `man interfaces`, LOL! – Lazy Badger Oct 17 '11 at 11:04

2 Answers2

1

Based on that I'd suggest that your default route out of eth0 is taking precedence. Routing only cares about the best way to send a given packet, it doesn't care about general data flow. A client will attempt to connect to your server on xx.yy.zz.1 via eth0, but the response will be routed out of eth1 on 10.10.53.1. Even if the reply packet successfully reaches it's destination it will have the wrong IP address as a source and the client won't know what to do with it. You'll be able to verify if that's the case just by running packet captures on your interfaces as you make requests to the server.

Check out this guide on how to route packets back out of the interface they arrived on

paulos
  • 1,694
  • 9
  • 12
  • I'll look at that and return with an update – Thomas Oct 17 '11 at 09:37
  • Running tcpdump before and after eth0 is shutdown shows that just after a reboot the kernel is dropping the packets (Meaning, tcpdump -i eth1 -vv shows no packets and showing that they were dropped by kernel). After ifconfig eth0 down tcpdump shows the expected trafic. – Thomas Oct 17 '11 at 09:55
  • Are you saying that when eth0 is up no traffic arrives at eth1 at all?! That doesn't sound right. With eth0 up get two terminals open, tcpdump on eth1 & eth0 then attempt a connection to your server... – paulos Oct 17 '11 at 11:02
-1

Add a line specifying the wan address in httpd.conf (sub in your wan IP for 1.2.3.4).

Listen 1.2.3.4:80

Not sure if there is a way to bind the process to a specific port. If this was RHEL I'd be able to help a bit more, not sure if there are route scripts in debian (well there are, just not sure how they work).

Matthew
  • 2,666
  • 8
  • 32
  • 50
  • `TCP *:https (LISTEN)` shows that it is listening on all interfaces, including the WAN interface. – Mark Wagner Oct 14 '11 at 20:20
  • As mentioned in the first post, Apache was just an example. The same goes for all services. As @embobo states, you can see that it is listening on all interfaces. (And I do need it to listen on both eth0 and eth1) – Thomas Oct 17 '11 at 09:38