Multi-homed OpenBSD system: Policy based routing vs. mpath default routes

Question

TL;DR Will OpenBSD policy based routing help with a multi-homed server/gateway situation? If so, how do I configure it?

Long Form

I'm managing an OpenBSD with two ISP links and VPN tunnels to remote routing nodes.

Initially we used multiple default routes with varying metrics -- the preferred route through a static IP address a NAT router which, in turn has dynamically allocated addresses (it's basically a cable modem).

In practice this was not ideal but it works well enough. New connections established from the gateway (hereinafter referred to simply as 'gw') would select the higher speed, lower latency route if it was up; and go out through the cable modem if the link was down. Inbound connection could only come through the better route since the other IP addresses were behind NAT (not routable from the outside.

Now we need to route traffic through an additional proxy/VPN router nodes out "in the cloud" to mitigate risks DDoS on our static IP addresses.

Those are connecting to the gateway via tunnels.

first. Then we found that our admin access would sporadically drop.

To complicate matters further this gateway has additional active interfaces to specific VLANs. They're unrelated to this problem but can't be disturbed.

Possible solution

It's my impression that we should be using policy based routing, rdomains. I guess that means I create routing tables for each of my three involved interfaces and any connection on any of those (including the tun0 tunnel interface) should be routed through the table for that domain (and thus each can have its own default route).

Am I on the right track?

Here's a diagram and a sanitized list if interface settings:

 ________
| tunnel |                       _______
 ~~~+~~~~                       | GW    |======++
    |                            ~+~+~+~       ||                   
    |      _________              | | |        ||                                        
    +-----| prefISP |-------------+ | |      __||____       .........               
           ~~~~~~~w~                | +-----| Switch |-----( Cluster )                           
                                    |        ~~~~~~~~       ^^^^^^^^^           
           _________           .....|......    ||                              
          | fallISP |---------( LAN / WiFi )===++
           ~~~~~~~~~           ^^^^^^^^^^^^

    Diagram: I want to avoid asymmetric routing when accessing GW through the tunnel, through                                           the preferred ISP, and when accessing GW or the cluster (through the GW or from the LAN).


 Sanitized interface info:

    em3:  inet 123.45.67.118 netmask 0xfffffff8 broadcast 123.45.67.119        description: prefISP
    em0:  inet 10.1.1.100    netmask 0xffffff00 broadcast 10.1.1.255           description: fallISP
    tun0: inet 192.168.2.2 --> 192.168.2.1 netmask 0xffffff00                  description: tunnel
    em1:  VLAN_TRUNK
          vlan1000: inet 172.29.1.1 netmask 0xffffff00 broadcast

As noted: em3 is our link to the preferred (faster) ISP; tun0 goes through it; em0 is on the same segment as the office LAN/Wifi and serves as our fallback ISP; and GW has additional links to the cluster and the switch.

score 0 · Accepted Answer · answered Mar 21 '17 at 17:22

Welcome to the dream of load balancing.

This is posible, but your best route and pain free mode is to use BGP routing protocol and manage the Downstream and Upstream traffic using policies.

For this to succeed, you have to negotiate with both ISPs that they include you as an internal iBGP node so you can push your routes paths to the internet.

The correct way would be for you to request your own Autonomous System Number. and manage all your IPs that you own. this is a little bit complicated to accomplish due to the requirements.

http://teamarin.net/2014/01/31/how-to-request-an-asn-from-arin/

If you are qualifying under the multihomed policy you will need to provide the exterior gateway protocol to be used, the IP addresses currently in use on your network, the AS number and name of each of your upstream providers and/or peers along with contractual verification of service with at least two of them.

If you are qualifying under the unique routing policy, you must demonstrate the AS’s routing policy will differ from the routing policies of its border peers.

No matter which policy you qualify under, if this is not your first time requesting an ASN, you will also need to show us how the network you are requesting an ASN for is autonomous from all existing ASes in your network as well.

Here is a nice paper on Multihoming using BGP: http://aspath.net/BGP-MHing-HOWTO-whitepaper.pdf

if you are not willing, unable to create BGP sessions with your ISPs, then the other solution is to puchase a hardware based load balancer. (technically speaking, most hardware run some modified BSD to achieve the products features. so if you have the knowledge you could set it up on a server running BSD. but you will never get the trhoguput of a hardware appliance with dedicated hardware for network processing, but if your load is not big (more than 50 Mbps i would say) you can do it)

Multi-homed OpenBSD system: Policy based routing vs. mpath default routes

1 Answers1