Apache performance is really heavy

Question

My company runs a heavy traffic site, about 150k daily.

Each time after apache restarted, site load is very fast. The problem always comes after a couple of hours.

Server config :

Intel Core i3-2100 (3.1 GHz)
8 GB RAM

Server apache running processes :

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                    
35986 apache    20   0 1046m  15m 6912 D 24.9  0.1   0:00.86 httpd                                                                                                      
35984 apache    20   0 1046m  15m 6916 D 21.9  0.1   0:00.78 httpd                                                                                                      
35987 apache    20   0 1046m  15m 6916 D 21.6  0.1   0:00.76 httpd                                                                                                      
35774 apache    20   0 1054m  35m  19m D  7.6  0.2   0:04.76 httpd                                                                                                      
35947 apache    20   0 1061m  43m  29m D  7.6  0.3   0:02.05 httpd                                                                                                      
35949 apache    20   0 1061m  44m  29m D  7.6  0.3   0:02.42 httpd                                                                                                      
35951 apache    20   0 1046m  15m 6912 D  7.6  0.1   0:01.69 httpd                                                                                                      
35952 apache    20   0 1046m  15m 6912 D  7.6  0.1   0:01.70 httpd                                                                                                      
35953 apache    20   0 1045m  15m 6900 D  7.6  0.1   0:01.48 httpd                                                                                                      
35983 apache    20   0 1046m  15m 6912 D  7.6  0.1   0:01.00 httpd                                                                                                      
35418 apache    20   0 1067m  84m  64m D  7.3  0.5   1:14.12 httpd                                                                                                      
35761 apache    20   0 1054m  36m  19m D  7.3  0.2   0:10.78 httpd                                                                                                      
35770 apache    20   0 1066m  59m  39m D  7.3  0.4   0:04.12 httpd                                                                                                      
35771 apache    20   0 1056m  36m  19m D  7.3  0.2   0:05.51 httpd

This is the ab result right after apache restart :

ab -n 300 -c 30 http://www.vibiznews.com/

Server Software:        Apache
Server Hostname:        www.vibiznews.com
Server Port:            80

Document Path:          /
Document Length:        182372 bytes

Concurrency Level:      30
Time taken for tests:   40.708 seconds
Complete requests:      30
Failed requests:        15
   (Connect: 0, Receive: 0, Length: 15, Exceptions: 0)
Write errors:           0
Total transferred:      5475675 bytes
HTML transferred:       5471175 bytes
Requests per second:    0.74 [#/sec] (mean)
Time per request:       40708.364 [ms] (mean)
Time per request:       1356.945 [ms] (mean, across all concurrent requests)
Transfer rate:          131.36 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        5    6   1.4      5       8
Processing: 17815 28229 9292.6  33340   40703
Waiting:    17731 28121 9339.8  33295   40666
Total:      17820 28235 9292.4  33345   40708

Percentage of the requests served within a certain time (ms)
  50%  33345
  66%  35458
  75%  37776
  80%  38302
  90%  39178
  95%  39302
  98%  40708
  99%  40708
 100%  40708 (longest request)

Here is my current Apache configuration :

Apache API Version  20120211

StartServers 5
MaxClients 220
ServerLimit 220
MinSpareServers 8
MaxSpareServers 16
MaxRequestsPerChild 4000

After a couple of hours, ab resulted in apr_pollset_poll: The timeout specified has expired (70007

I calculated Max Client this way : 8196 (Physical Memory) / 35 (Apache average memory usage) = 234.17, rounded to 230 to give space for other processes besides apache.

My question is the apache configuration is optimal for the given condition?

I usually sees other's configuration for StartServer and MinSpareServer to be around 10, and MaxRequestsPerChild to 0.

Is that configuration safe, especially MaxRequestsPerChild to be 0? If so what is the benefit?

Any thought will be greatly appreciated.

Thanks!

What is the max concurrent visitors on your site at it's peak? — Drew Khoury, Sep 10 '13 at 08:12
What's the state of the `Keepalive` directive in your config? — Tom O'Connor, Sep 10 '13 at 08:20
Have you done a top, is it the CPU that's getting max'd out? — Drew Khoury, Sep 10 '13 at 08:40

score 1 · Answer 1 · answered Sep 10 '13 at 09:26

This is not an answer - but it's too long for a comment, and the question can't be answered in it's current form.

about 150k daily

150k what? The only metrics that really matter are concurrent hits/average duration.

You've provided no details of what the traffic is - it is obviously NOT serving static content. You've provided no details.

1.3 seconds per request is very slow.

ServerLimit 220

Seems very low.

35 (Apache average memory usage)

How did you get that?

This is not the right way to calculate serverlimit/maxclients. Try measuring the free memory (less buffers/cache) at different load levels and plot this against number of servers.

Working out what the right questions to ask about the performance here would take at least a couple of hours for someone who knows what they're doing - finding the answers will take longer.

You should be able to run a lot more servers then this - but your performance problem is not in the httpd config.

Sorry, I'm really new at this. 150k means about 150.000 requests daily. And 35 (Apache average memory usage) is from top. Can you point me on how to do concurrent hits/average duration calculation? I have been trying to google with no luck — Samuel Adam, Sep 10 '13 at 10:45
You can get a snapshot of the concurrent connections from netstat or server-status. Memory management on modern operating systems is incredibly complex - don't try to work out how much space a process is using and don't believe the reports - see how much space it leaves behind. As for duration - add %D to your apache logs. — symcbean, Sep 10 '13 at 13:47

Apache performance is really heavy

1 Answers1