0

I think I am doing something wrong with HAProxy conf because my throughput drops to 25% in a real-world test done with HAProxy and one single AWS instance. Following is my relevant (extremely simple) configuration:

global
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     20000
    user        haproxy
    group       haproxy
    daemon
    stats socket /var/lib/haproxy/stats
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m

timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 30000
frontend localnodes
    bind *:80
    mode http
    default_backend nodes
backend nodes
    mode http
    balance roundrobin
    hash-type consistent
    option httpchk /health
    server w1 xx.xx.xx.xx:80 check id 1

I had enabled logging. A typical entry in log looks like this:

Dec  2 10:56:59 localhost haproxy[25988]: xx.xx.xx.xx:39789 [02/Dec/2016:10:56:58.729] main app/app1 0/0/1000/1/1002 200 449 - - ---- 13/13/13/7/0 0/0 "GET / HTTP/1.1"
Dec  2 10:56:59 localhost haproxy[25988]: xx.xx.xx.xx:39803 [02/Dec/2016:10:56:58.730] main app/app1 0/0/999/1/1000 200 377 - - ---- 12/12/12/7/0 0/0 "GET / HTTP/1.1"
Dec  2 10:56:59 localhost haproxy[25988]: xx.xx.xx.xx:39804 [02/Dec/2016:10:56:58.730] main app/app1 0/0/999/1/1000 200 277 - - ---- 11/11/11/7/0 0/0 "GET / HTTP/1.1"

My throughput is 25% of what a direct traffic to my instance would be. This is terrible performance. Am I doing something really wrong?

Maxsteel
  • 101
  • 3
  • You're right, this is unexpected. What version of HAProxy? Why `option http-server-close`? (Not wrong, just asking why.) Also, how did you benchmark before and after? Note that `38/0/0/1/41` suggests the client is causing the slowness, not sending a complete request until 38 ms after connect. 0 + 0 ms internal delays, 1 ms to connect to server, and 41 ms total => 41 - 1 - 0 - 0 - 38 = 2 ms response from backend. – Michael - sqlbot Dec 02 '16 at 12:38
  • @Michael-sqlbot i) Version 1.5 and 1.7, same result. ii) I have 1000s of connection. I read that this is a good way to let server close them. iii) I ran same test trace using httpperf and other tools like ab. Same result. Very high throughput in single machine, gets down to 1/4th with HAProxy. iv) See the new trace. Proxy consumes lot of time to connect to server. Thanks! – Maxsteel Dec 02 '16 at 20:06
  • Ah, not quite. Third number is `Tc`: *"total time to establish the TCP connection to the server. It's the time elapsed between the moment the proxy sent the connection request, and the moment it was acknowledged by the server, or between the TCP SYN packet and the matching SYN/ACK packet in return."* http://cbonte.github.io/haproxy-dconv/1.6/configuration.html#8.4 HAProxy is waiting for the server to respond, at this point. Check the behavior without `option http-server-close`. – Michael - sqlbot Dec 02 '16 at 20:19
  • @Michael-sqlbot Thanks, I will try this and get back to you. Just curious, would `option http-server-close` really affect this? – Maxsteel Dec 02 '16 at 21:16

0 Answers0