5

We are migrating from one set of addresses to another set, both /24, and trying to minimize any down time during the migration. Ideally we'd run both for a period of time as we shut down the old circuits. There are a total of 4 internet connections, with each pair running BGP.

Each of these is then routed to a Cisco ASA, which is connected to a switch which has multiple servers connected on that subnet.

Netowrk-Diagram

In the above diagram, the left hand portion is what exists today, and I'm looking to add the right side.

I've connected the ASA and have both of them on the 10.20.20.0/24 subnet, with the first ASA interface as 10.20.20.1 and the second ASA interface as 10.20.20.254.

The issue here is that all of the servers have 10.20.20.1 as their default route, and I'd really like to route traffic back the way it came in. That is, internet -> ASA #2 -> server -> back to ASA #2. As it is today, of course, it sends the response back to ASA #1 and it doesn't find a translation for it.

Am I going about this the wrong way?

Edit: I should mention that Outside #1 and Outside #2 have different public /24 networks. We're migrating from an ISP provided block to our own block.

fullstop
  • 51
  • 5
  • 1
    Is there a reason you need the traffic to go back out the way it came in? – Zypher Feb 11 '16 at 23:19
  • Yes. It will be sent to the other ASA which knows nothing about the state of that packet. As such, the packet is dropped by the ASA. – fullstop Feb 11 '16 at 23:48
  • 1
    Yea, is there a hard requirement for two seperate ASAs? I'd probably run all ISPs into one ASA (or since you have two now, cluster them) – Zypher Feb 11 '16 at 23:50
  • There is not, but I am out of interfaces on them. We are not using vlans on them yet, though, so that may be an option. – fullstop Feb 11 '16 at 23:55
  • Everything sounds like you have forwarded ports, am I right ? Otherwise you just don't need the traffic to be sent through the original channel. – drookie Feb 12 '16 at 02:48
  • Yes. Ports forwarded to many hosts. – fullstop Feb 12 '16 at 05:31
  • Clustering is not an option, as that is not supported on the hardware that we have. I got it working with iptables fwmark and policy based routing, but this will need to be configured on every server which needs to be on both networks concurrently. It's certainly not ideal, but it would work. – fullstop Feb 12 '16 at 17:04

1 Answers1

0

This is what I ended up doing:

#!/bin/sh
echo 200 asa1 >> /etc/iproute2/rt_tables
echo 201 asa2 >> /etc/iproute2/rt_tables
ip route add table asa1 default via 10.20.20.1 dev eth0 metric 100
ip route add table asa2 default via 10.20.20.254 dev eth0 metric 100
ip rule add prio 100 from all fwmark 1 lookup asa1
ip rule add prio 110 from all fwmark 2 lookup asa2
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark
iptables -t mangle -A INPUT -m mac --mac-source $(MAC_ASA1) -j MARK set-mark 1
iptables -t mangle -A INPUT -m mac --mac-source $(MAC_ASA2) -j MARK --set-mark 2
iptables -t mangle -A INPUT -j CONNMARK --save-mark

Replace MAC_ASA1 / MAC_ASA2 with the hardware address of the connected interface on the ASA. This can be gathered from the ARP table.

You'll also have to be mindful of the ethernet device name, especially if you are using systemd with the newer style interface names.

Moshe Katz
  • 3,053
  • 3
  • 26
  • 41
fullstop
  • 51
  • 5