2

I have number of peer-to-peer HTTP proxies and I need to distribute my requests among them evenly, so I added multiple cache_peer directives and set round-robin as peer selection algorithm. This is my config:

# Basic configuration
visible_hostname squid_proxy_1
unique_hostname squid_proxy_1

http_port 3130
pid_filename /vagrant/squid_proxy_1.pid


cache deny all
http_access allow all

# Proxies
cache_peer MY_PROXY_HOST parent PROXY_PORT 0 round-robin no-query name=FIRST login=MY_PROXY_LOGIN-1:PASSWORD1
cache_peer MY_PROXY_HOST parent PROXY_PORT 0 round-robin no-query name=SECOND login=MY_PROXY_LOGIN-2:PASSWORD2
cache_peer MY_PROXY_HOST parent PROXY_PORT 0 round-robin no-query name=THIRD login=MY_PROXY_LOGIN-3:PASSWORD3

never_direct allow all

But I ran into following problem: cache_peer switch is made not after every request. Here's how my access.log looks like:

sudo tail -f /var/log/squid3/access.log
1460977613.775  23783 127.0.0.1 TCP_MISS/200 11371 GET http://www.google.com/ - ROUNDROBIN_PARENT/104.131.79.140 text/html
1460977621.171   4843 127.0.0.1 TCP_MISS/200 11413 GET http://www.google.com/ - ROUNDROBIN_PARENT/104.131.79.140 text/html
1460977628.367   5583 127.0.0.1 TCP_MISS/200 11412 GET http://www.google.com/ - ROUNDROBIN_PARENT/104.131.79.140 text/html
1460977634.270   4726 127.0.0.1 TCP_MISS/200 11354 GET http://www.google.com/ - ROUNDROBIN_PARENT/104.131.79.140 text/html
1460977642.429   6531 127.0.0.1 TCP_MISS/200 12841 GET http://www.google.com/ - ROUNDROBIN_PARENT/104.131.79.140 text/html
1460977645.563   1747 127.0.0.1 TCP_MISS/200 11360 GET http://www.google.com/ - ROUNDROBIN_PARENT/104.131.79.140 text/html
1460977653.731   7242 127.0.0.1 TCP_MISS/200 11408 GET http://www.google.com/ - ROUNDROBIN_PARENT/107.170.96.123 text/html
1460977658.559   3763 127.0.0.1 TCP_MISS/200 11374 GET http://www.google.com/ - ROUNDROBIN_PARENT/45.55.163.40 text/html
1460977660.516   1387 127.0.0.1 TCP_MISS/200 11398 GET http://www.google.com/ - ROUNDROBIN_PARENT/45.55.163.40 text/html
1460977662.863   1758 127.0.0.1 TCP_MISS/200 11414 GET http://www.google.com/ - ROUNDROBIN_PARENT/45.55.163.40 text/html
1460977664.667   1305 127.0.0.1 TCP_MISS/200 11419 GET http://www.google.com/ - ROUNDROBIN_PARENT/45.55.163.40 text/html
1460977676.157  10901 127.0.0.1 TCP_MISS/200 11384 GET http://www.google.com/ - ROUNDROBIN_PARENT/45.55.163.40 text/html
1460977678.565   1348 127.0.0.1 TCP_MISS/200 11321 GET http://www.google.com/ - ROUNDROBIN_PARENT/45.55.163.40 text/html
1460977681.742   2516 127.0.0.1 TCP_MISS/200 11383 GET http://www.google.com/ - ROUNDROBIN_PARENT/45.55.163.40 text/html

It happens ONLY IF I USE THESE PARTICULAR P2P PROXIES. I tried same config with free proxies I found in the interned and everything worked fine.

I guess the problem is that my proxies may be considered dead by Squid (because they really can be very slow sometimes) and therefore we're not routing traffic to them, but I'm not sure.

Is there any way to get more information about peer selection algorithm? Can I tweak cache_peer options somehow to solve this problem?

Ales Dokshanin
  • 51
  • 5
  • Squid appears to be doing round-robin at a higher level than per request. HTTP has a keep-alive capability, which allows a single connection to hand multiple requests. – BillThor Apr 18 '16 at 12:46
  • @BillThor I just tried to use random free HTTP proxies from internet and again got the expected result: they rotate in round-robin way. – Ales Dokshanin Apr 18 '16 at 13:00
  • I would expect public proxies to drop the connection after one request. This would be to conserve connection resources. Local repository may conserve. different resources by keeping connections open. Round-robin may not work as you expect if you also have ICP enabled, – BillThor Apr 18 '16 at 13:16
  • @BillThor regarding to ICP - as you may see, I've got `no-query` option in my config which disables ICP for this peer. Regarding to open connections: maybe there is way to force close all the connections after every request or something like that? It'd show us if the problem is in persistent connections. Anyway thanks much for help! – Ales Dokshanin Apr 18 '16 at 15:45
  • @BillThor I turned off persistent connections and it worked! Thanks much! Would you like to submit this as answer or should I do it myself? – Ales Dokshanin Apr 18 '16 at 16:37

1 Answers1

0

This problem is solved by forbidding persistent connections. In my case I added following line to config and everything is working fine now.

server_persistent_connections off

Huge thanks to @BillThor, who helped me in the comment section.

Ales Dokshanin
  • 51
  • 5