1

I'm running an apache2 2.2.9 webserver with modpython and mpm_worker_module.

The current config for the mpm is

ServerLimit 32
StartServers         10
MaxClients          800
MinSpareThreads      25
MaxSpareThreads      75
ThreadsPerChild      25
MaxRequestsPerChild   0

The server has 1G of ram and a 100Mbit connection.

Checking netstat -na | grep ESTABLISHED | wc -l gives me a number between 50 - 60. The load is about 1.0 Every pageload is also cached by memcached.

I can't see why the server is so slow in responding to new connections, sometimes droping them completely?

Also tried disabling iptables to make sure it's not because of a full state table or something like that.

The only thing in dmesg is a lot of spam about "TCP: Treason uncloaked!"

Even connections to localhost with apache2ctl status fails, so it shouldn't be modpython related. When the status do work, it shows around 110-128 requests being processed. About half are status C (closing connection), lots of _ (Waiting for connection), the rest being R and W

baloo
  • 135
  • 1
  • 5

1 Answers1

0

What does the error_log report? It's the best place to start debugging these kind of problems.

top is also a good way to debug this. Run top and check if the Apache processes are using too much CPU. Then type M to sort the processes by memory usage and check if they're consuming too much RAM.

Check also if you are swapping:

free

The memcached runs on the same server? If so, how much memory does it have allocated?

Also, are you having trouble with i/o wait? Check the %wa value when running top.

Regarding the network, are you having errors or CRCs? Check it with netstat:

netstat -i

Take a look the RX-ERR, RX-DRP, TX-ERR and TX-DRP columns. Ideally, these values should be 0.

Hope this helps.

Marco Ramos
  • 3,100
  • 22
  • 25
  • Thanks for the list, here is the output: The CPU is mostly about 75% idle. Each apache (about 7 in the list) consume about 3-8% cpu. Each apache has 39M (RES) and 310M (VIRT). %wa maxes out on 7%, most of the time it's 0%. Zero errors on netstat. Memcached is on the same server and has 165M. The only error in the apache log is from modpython that complains about "IOError: Write failed, client closed connection." – baloo May 20 '10 at 14:43
  • And I do have 284M used on swap – baloo May 20 '10 at 14:48
  • Even connections to localhost (apache2ctl status) fails to establish at times (after waiting several minutes) – baloo May 20 '10 at 15:16
  • Also make sure you look at your application. You said you're using the threaded MPM; I don't suppose there are any situations where you may be blocking on hot locks? Check the output of `free`, what does it list for available memory? What about after +/- buffers? – McJeff May 20 '10 at 15:21
  • I'm not sure about the "hot locks"? This is a python django application. Mem: 1034680 1021432 13248 0 9420 455428. -/+ buffers/cache: 556584 478096 – baloo May 20 '10 at 19:21
  • So, yeah, the RAM is a bit on the low side, but I wouldn't think that would cause a situation like you're seeing. My guess is that the slowness is caused by the application. A slow DB query, a DB connection delay, a threading.Lock mutex that multiple threads are waiting on... something like that. – McJeff May 21 '10 at 20:40