Load balancing based on client certificate with Apache mod_proxy_ajp and Tomcat

Question

Currently, I am using mod_proxy_ajp to load balance requests to multiple Tomcat servers. Callers are authenticated using client certificates and an ACL. This has been working fine, callers are routed to either server at an even rate. Now I want to pre-fetch and cache data per user, so each Tomcat server will serve a known set of users. So I'm thinking it would be fine to split caller's alphabetically; user's with a names (CN or DN) starting with A-M go to server 1, users with names starting with [N-Z] go to server 2. But I don't know how to make this work.

Currently I have:

<Proxy balancer://lb-service>
   BalancerMember ajp://svr1:8009 loadfactor=1 
   BalancerMember ajp://svr2:8009 loadfactor=1 
</Proxy>

<Location /lookup>
   ProxyPass balancer://lb-service/lookup
</Location>

I'm considering using mod_rewite to change the URL to include the user's CN but I can't seem to make that work.

I'm pretty much stuck using apache mod_proxy_ajp and Tomcat for load balancing. I can change the services running the Tomcat to accept different requests.

Anyone have a good way to do this?

Answer 1

Interesting idea for caching! Let's see if we can get it working..

I've got a couple of options: one that would be more efficient but I really have no idea whether it will work, and one that's less efficient but I'm reasonably sure will work just fine.

The issue with the more efficient option (and, more generally, using something like the client cert to determine load balancer assignment) is that as far as I know there's no way to change mod_proxy_balancer's mind on node assignment after the request has come in - there's probably some creative hackery that could make it happen, but I'm not coming up with any better ideas off the top of my head. The crux of this is that it's doing a redirect to the same location that it's at, in order to have the cookie in place for the first request that gets proxied, to get the right route - and I'm not sure if that will work. Worth a shot:

RewriteEngine On

RewriteCond %{ENV:BALANCER_ROUTE_CHANGED} 1
RewriteCond %{SSL:SSL_CLIENT_S_DN} ^[a-m] [NC]
RewriteRule ^(.*)$ /.$1 [R,CO=ROUTEID:route.1:.yourdomain.com]

RewriteCond %{ENV:BALANCER_ROUTE_CHANGED} 1
RewriteCond %{SSL:SSL_CLIENT_S_DN} ^[n-z] [NC]
RewriteRule ^(.*)$ /.$1 [R,CO=ROUTEID:route.2:.yourdomain.com]

<Proxy balancer://lb-service>
    BalancerMember ajp://svr1:8009 loadfactor=1 route=1
    BalancerMember ajp://svr2:8009 loadfactor=1 route=2
    ProxySet stickysession=ROUTEID
</Proxy>

<Location /lookup>
   ProxyPass balancer://lb-service/lookup
</Location>

If that doesn't work, then here's plan B; just match the client certificate's DN every time. Adding extra overhead to each request, but it shouldn't be too bad of a performance impact overall.

<Proxy */lookup>
    RewriteEngine On

    RewriteCond %{SSL:SSL_CLIENT_S_DN} ^[a-m] [NC]
    RewriteRule ^/lookup((?:/.*|))$ ajp://svr1:8009/lookup$1 [P,L]

    RewriteCond %{SSL:SSL_CLIENT_S_DN} ^[n-z] [NC]
    RewriteRule ^/lookup((?:/.*|))$ ajp://svr2:8009/lookup$1 [P,L]

    # This will take care of any other orphan requests (say, those with no client
    # cert) - we'll just lump those on server 1; if there's a lot then we can do
    # something else to balance randomly (like just leave the current config there)
    RewriteRule ^/lookup((?:/.*|))$ ajp://svr1:8009/lookup$1 [P,L]
</Proxy>

I've got this matching for a trailing slash after /lookup to avoid a nasty security vulnerability where an attacker could get to your internal network - if that won't work for your application then we can work around it.