3

On my office network I have two internet connections and one CentOS server running a website (HTTPS on port 443). The website should be publicly accessible through the public IP of the first internet connection (ISP-1). The other internet connection, ISP-2, id the default gateway on the network. Both internet connections have routers (the household-kind) with NAT, SPI firewalls etc. The router on ISP-2 is a Netgear WNDR3700 (aka N600) with original firmware.

The problem is that the website is unreachable. Looks like incoming traffic on ISP-1 will reach the server but the returning traffic is routed through ISP-2, effectively making the site unreachable. As far as I can tell I can't do port based routing on the WNDR3700.

What are my options to make this work? I've been looking at implementing an iptables / routing based solution on the server itself but haven't been able to make that work.

Update: Note that the server has one network interface connecting it to both routers.

Marnix van Valen
  • 193
  • 1
  • 4
  • 13

5 Answers5

5

I had the same issue, but I solved only with iproute2 (source routing). Marking with iptables wasn't necessary:

echo "101 webtraffic" >> /etc/iproute2/rt_tables
ip route add default table webtraffic via $ISP1_GW_LAN_IP
ip rule add from $ISP1_IP table webtraffic

The point is that not only web may use the ISP1 connection. You may choose. It's good because you may connect to the server from ssh from both connections if someone fails. As a CentOS user I created the following files so my changes weren't lost after reboot:

echo "default table webtraffic via $ISP1_GW_LAN_IP" >> /etc/sysconfig/network-scripts/route-eth1
echo "from $ISP1_IP table webtraffic" >> /etc/sysconfig/network-scripts/rule-eth1
João Borsoi
  • 51
  • 1
  • 1
3

If I am understanding your intentions correctly, you want your webserver to normally use ISP-2 as its default gateway for outgoing traffic, with the exception of its responses to external web requests, which must transit via ISP-1 instead. Here is a sketch of a solution using policy routing:

echo "101 webtraffic" >> /etc/iproute2/rt_tables

ip route add default table webtraffic via $ISP1_GW_LAN_IP
ip rule add fwmark 1 table webtraffic
iptables -t mangle -A OUTPUT -d \! $LAN_NET_PREFIX \
                             -p tcp -m tcp --sport 443 \
                             -j MARK --set-mark 1

where:

  • LAN_NET_PREFIX is your LAN's network prefix (e.g. 192.168.100.0/24), and
  • ISP1_GW_LAN_IP is the LAN IP address of your gateway to ISP-1 (e.g. 192.168.100.100).

The first ip command sets the default route on the webtraffic table to your ISP-1 gateway, and the second ensures that packets marked 1 are routed using the webtraffic table. Finally, the iptables rule marks the appropriate outgoing packets, ensuring that their next hop will be towards ISP-1.

Here is an alternate solution that uses an experimental iptables module, the ROUTE target:

iptables -t mangle -A POSTROUTING -d \! $LAN_NET_PREFIX \
                                  -p tcp -m tcp --sport 443 \
                                  -j ROUTE --gw $ISP1_GW_LAN_IP

This rule would override the routing decision for outgoing web response packets, sending them to your ISP-1 gateway, instead of to the default ISP-2. All other traffic, including web responses to clients on your LAN, would not be affected. As has been pointed out in the comments, the ROUTE target is very likely not to be implemented on any system that has not explicitly patched it into the kernel, since it is experimental.

Steven Monday
  • 13,019
  • 4
  • 35
  • 45
  • Yes, that is exactly what I intended. Unfortunately the iptables version on the server doesn't support the ROUTE target... Any thoughts? – Marnix van Valen Jan 21 '11 at 21:39
  • @Marnix van Valen: Try using MARK and policy routing instead. I have updated my answer with an example. – Steven Monday Jan 22 '11 at 06:23
  • ROUTE was never part of the Linux kernel, nor has been endorsed as an out-of-tree module (-- policy routing is the right way). As such, it would only apply to a very customized subset of installations. – user61188 Jan 24 '11 at 01:56
  • @user61188: Thanks for pointing this out. I have updated my answer accordingly. – Steven Monday Jan 24 '11 at 02:39
  • @Steven Thanks, that worked! I needed to replace -t tcp with -p tcp in the iptables command though. – Marnix van Valen Jan 25 '11 at 11:03
  • @Marnix van Valen: Whoops, sorry about the mixup. I have updated my answer. I'm glad this worked for you! – Steven Monday Jan 25 '11 at 16:36
  • I am using CentOS 6.5. @Steven Monday 's answer is almost right, but in my case, `rp_filter` had to be disabled to make it work. To do that, add the following line to `/etc/sysctl.conf`: net.ipv4.conf.eth1.rp_filter = 0 where `eth1` is the device of ISP-1's network. And then run: # sysctl -p to reflect the configuration. – Akihiro HARAI Jan 22 '14 at 01:48
0

This is called asynchronous routing. You have to point the default gateway of the webserver to the IP address of the router on ISP-1. If the clients that are accessing the web server came from the same IP you can route this without changing the default gateway, or, you can implement NAT on the ISP1 gateway to act like a reverse proxy and then route it at the web server. Regards.

voodooo
  • 254
  • 2
  • 6
  • I need ISP2 to be the default gateway on the server, except for web traffic. Changing the default gateway is not an option. ISP1 runs DD-WRT I'm not sure it'll do NAT on the inside, but I get the idea. – Marnix van Valen Jan 21 '11 at 21:46
0

Firewall people do inside nat to get over this problem. The inbound connection is natted to the firewall inside interface address and therefore does not have a routing problem at the server.

If the both inbound connections end up in same interface in same IP you have very limited (none?) options. See if you can get another IP for the same computer/interface and have one inbound connections in one IP and anoter inbound in another IP. After this you can do source routing easily.

If you can't get different sources connect to different IP addresses you can use iptables mac to redirect packets depending on MAC they were coming from. 

   [!] --mac-source address
          Match  source  MAC  address.    It   must   be   of   the   form
          XX:XX:XX:XX:XX:XX.   Note that this only makes sense for packets
          coming from an Ethernet device and entering the PREROUTING, FOR‐
          WARD or INPUT chains.

Hope this helps.

Antti Rytsölä
  • 651
  • 4
  • 9
  • Inside nat is not an option. The router won't be able to handle that. Interesting idea to route based on the mac. Hadn't thought of that. I'll definitely try to assign multiple IPs to the network interface, that would make things a lot simpler. – Marnix van Valen Jan 21 '11 at 22:03
0

Assuming your Webserver is on the same subnet as your workstations, why not just setup split DNS so that on the inside, yourwebsite.com resolves to it's internal IP address? Sure beats complicated asynchronous routing.

Also, investing a router than can support multiple WAN connections would make life a lot easier for you (and allow you do load balancing/failover between both Internet connections).

gravyface
  • 13,947
  • 16
  • 65
  • 100
  • The server and workstations are on the same subnet, but I don't really see how a split DNS setup would help here. The problem is with people trying to access the server from the outside. I agree that a router with muliple WAN connections would help, but the way things are there is no budget for it. Besides, if a couple of lines of firewall config on the webserver will do the trick, why spend hundreds of euro's on hardware? – Marnix van Valen Jan 25 '11 at 10:32