1

I am setting up a web application which uses SSL for all requests. It needs to have room for scaling and also be highly available. It seems that the "recommended" way to handle this is to setup a pair of Layer 4 load balancers for HA, which pass through to a farm of servers for SSL decryption, which then go to a farm of Layer 7 load balancers, and then finally to the web servers. (See: http://1wt.eu/articles/2006_lb/index_09.html)

I've used HAProxy in the past, so I gave that a try with a setup like this. Basically I had HAProxy in TCP mode => STunnel => HAProxy in HTTP mode => Apache. When I did this, the client IP that was available to apache was that of the 2nd HAProxy server. I could improve that slightly by having STunnel add an X-Forwarded-For header, but that only gave me the IP address of the 1st HAProxy server. I was not able to find a way to let Apache know the original client's IP address.

Based on the documentation I have read, I don't think that HAProxy is capable of passing through the original client's IP address in tcp mode. Is that correct? If so, are there any opensource, software load balancers that can do this? If not, are there any commercial/hardware load balancers that can?

I guess I should also note that this load balancer doesn't, strictly, need to be a layer 4 load balancer. I just needs to be able to pass through the SSL traffic to STunnel (or whatever I end up using for SSL decryption).

As a side note, I tried setting it up without the layer 4 load balancer (i.e. STunnel is the first part of the chain). This solved the client IP problem. However, as predicted in the above article, STunnel was the bottleneck. It started losing requests somewhere between 500 and 600 requests per second. And, of course, this can't scale (while maintaining HA) by just adding more boxes. Based on the requirements I have been given, this app should be able to handle peak traffic of 1000-5000 requests per second.

phylae
  • 319
  • 4
  • 10
  • I am wondering if HAProxy's "usesrc clientip" option might be the best solution. However, it looks like that depends on TPROXY. I'm running RHEL 5.5, which has kernel 2.6.18. TPROXY isn't included until 2.6.28. I'd rather not have to recompile the kernel, but I guess we'll see. – phylae Apr 10 '11 at 06:29
  • This seems like a very related question: http://serverfault.com/questions/190570/haproxy-ssl-farm-options – phylae Apr 11 '11 at 17:51

3 Answers3

2

I highly recommend looking into LVS. http://www.linuxvirtualserver.org/

It has the ability to do normal NAT routing, but also IP tunneling and direct routing which send directly back to the client.

rfelsburg
  • 767
  • 3
  • 7
  • 1
    This is looking like the best option. Unfortunately (for me) the more I read about LVS the less it seems I know about networking! – phylae Apr 11 '11 at 17:41
  • Trust me, I understand that, just think of it as an opportunity to learn more, and the added incentive of knowing you have to :-) – rfelsburg Apr 11 '11 at 17:54
0

Have you looked at: http://httpd.apache.org/docs/2.3/mod/mod_remoteip.html that will rewrite the IP based on the X-Forwarded-for for header.

n8whnp
  • 1,316
  • 7
  • 9
  • Yes. As I mentioned in the original post, this is exactly how I have STunnel setup. The problem with this approach is that request is being decrypted on the load balancer. That creates a bottleneck because you can't just add more load balancers (while maintaining HA) like you can add more app servers. That is the reason for a load balancer _before_ decrypting SSL. – phylae Apr 10 '11 at 06:18
0

The BigIP from F5 is a commercial solution that works very well. X-Forwarded-For is implemented as an http profile option. It isn't cheap though.

Apache TrafficServer can be set up to insert an X-Forwarded-For header and it doesn't cost anything. I've never used it though but it does have a configuration variable: proxy.config.http.append_xforwards_header

mahnsc
  • 1,776
  • 13
  • 11
  • The X-Forwarded-For header is very useful. But this doesn't get at the root of the problem. Both the BigIP and TrafficServer have SSL termination at the load balancer level. Based on my current benchmarks, SSL termination is my bottleneck. If I can get an F5 BigIP (and I might be able to), then it may be fast enough. However, that doesn't answer the theoretical problem (which I still think is interesting). My goal is to allow an SSL farm and still have the original client IP passed along to the app servers. This guarantees (theoretically) that SSL will never be the bottleneck. – phylae Apr 11 '11 at 17:44
  • Sorry! I was commenting on the first part of your question. – mahnsc Apr 11 '11 at 18:09