2

Until recently our setup consisted of 4 web-servers sharing sessions to a single server running memcache. We are hosted on Amazon cloud and had a crash at peak load on 2 consecutive days. The problem was that the memcache service crashed (the load to our site has been increasing steadily).

So, we took the following measures:

1) Added 2 more servers for storing sessions

2) Set following variables in php ini file on all web-servers

session.save_handler = memcache

session.save_path = tcp://ip1.port, tcp://ip2.port, tcp://ip3:port

memcache.hash_strategy = consistent

memcache.allow_failover = 1

Things are running in order now. To ensure availability, we tried testing by randomly killing one of our session servers, and the site keeps running (some users got logged out, which is business-wise acceptable at the moment).

But there is one major problem. I expected the load on memcache servers to be more or less equally distributed. But it is not!

If I look at the "Max Network Out (Bytes)" in CloudWatch, then I can see that the load is roughly in the ratio of 10:5:1. In other words, In and Out network bandwidth wise, first server is 10 times as loaded as the third one. And the second server is 5 times as loaded as the third one.

Any ideas?

Kundra
  • 21
  • 2
  • 1
    Four front-end webservers managed to crash a memcached server that is only storing sessions? I have my doubts. – womble Jul 15 '11 at 06:25
  • @womble: Well, let me be more precise. The server was storing session data, and also cache (we cache a bunch of stuff). Now we split session on 3 servers (as described above), and also cache on 3 additional servers. The behavior is same on cache servers as well -> load is split in the ratio of 10:5:1 (approx). – Kundra Jul 15 '11 at 07:52

1 Answers1

0

Changing order of memcache IPs for session.save_path parameters might change the session distribution ratio over memcache servers. For example on the 1st web server :

session.save_path = tcp://ip1.port, tcp://ip2.port, tcp://ip3:port

on the 2nd web server :

session.save_path = tcp://ip2.port, tcp://ip3.port, tcp://ip1:port

on the 3rd web server :

session.save_path = tcp://ip3.port, tcp://ip1.port, tcp://ip2:port

on the 4th web server :

session.save_path = tcp://ip1.port, tcp://ip3.port, tcp://ip2:port
uguraslan
  • 55
  • 2