8

After scratching my head trying to figure out why my site was responding so slowly even though the server resources are fine, I finally checked the Apache status and found:

78 requests/sec - 0.7 MB/second - 8.5 kB/request
256 requests currently being processed, 0 idle workers

It appears that my apache is literally maxed out with connections. Anyone trying to visit my site gets put on a "waiting list" until Apache is free again.

It seems I have two options.

A) Raise the max connections limit above 256. Although according to this article it's not so easy:

By default, the MaxClients parameter has a compiled in hard limit of 256. This can be changed by recompiling Apache however. Some distributions, or hosting companies raise this limit to a very high value, such as 512 or even 1024 in order to cope with large loads.

B) Locate scripts that are taking up too much time. This seems a lot more tricky to me, since most apache processes just appear and then disappear again. Also, my sites PHP scripts are optimized pretty well...and once again, server resources are fine:

Server load 2.69 (8 CPUs)   
Memory Used 25.33% (2,039,108 of 8,048,804) 
Swap Used   1.32% (54,156 of 4,095,992)

Which option (if either of them) should I chose and how should I do it?

EDIT

More information: Server Version: Apache/2.2.23 (Unix) mod_ssl/2.2.23 OpenSSL/1.0.0-fips DAV/2 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635

HTTP Conf: http://pastebin.com/yBeLt6mP

Parital Request sample: http://pastebin.com/vzUVDMPR

Toggle Text-Wrap if the paste-bins show up weird.

kmoney12
  • 205
  • 1
  • 3
  • 6
  • The article you cited is ~6 years old. Software changes. See Shane's Answer. – Chris S Feb 23 '13 at 21:12
  • 1
    @ChrisS Even further out of date - the compiled-in limit was a 1.x thing (2.0 came out in 2002), and the article even explicitly links to the 2.0 docs. – Shane Madden Feb 23 '13 at 21:46

2 Answers2

10

That article is inaccurate; MaxClients can be raised above 256 when using the prefork MPM (which is what I assume you're using currently based on your description of the problem). From the documentation:

For non-threaded servers (i.e., prefork), MaxClients translates into the maximum number of child processes that will be launched to serve requests. The default value is 256; to increase it, you must also raise ServerLimit.

ServerLimit is the one that has the hard-compiled limit, but it's way past where you should ever reach without your server running into some other bottleneck. Documentation:

There is a hard limit of ServerLimit 20000 compiled into the server (for the prefork MPM 200000). This is intended to avoid nasty effects caused by typos.

So, if you want to raise your client limit to something like 512, then:

MaxClients 512
ServerLimit 512

You should also take a look at which MPM you're using, as MPMs other than prefork are better for scale. See here for more information.

Community
  • 1
Shane Madden
  • 112,982
  • 12
  • 174
  • 248
  • I see core.c and worker.c from running `httpd -l`. I guess that means I am running worker MPM? – kmoney12 Feb 23 '13 at 21:03
  • @hellohellosharp Indeed - in that case, you'll want to adjust potentially your `MaxClients`, `ServerLimit`, `ThreadsPerChild`, and `ThreadLimit`. Can you provide your current worker config from `httpd.conf`? – Shane Madden Feb 23 '13 at 21:09
  • Yes...is there any danger in posting this publicly? Hopefully not, here it is: http://pastebin.com/yBeLt6mP – kmoney12 Feb 23 '13 at 21:11
  • Nope, no problems in posting most httpd.conf files. Just make sure you don't have something odd like comments with passwords/usernames/etc. These sometimes creep up in proxy configuration sections and the like too. Many people like to edit out IP and Domain names too, just so they don't attract attention – Chris S Feb 23 '13 at 21:14
  • Okay, I put it in the comment above :) – kmoney12 Feb 23 '13 at 21:16
  • @hellohellosharp You're running into a limit on both the `ThreadsPerChild` and `ServerLimit` (16 * 16 = 256) and `MaxClients 256` - To double your client limit, set `ServerLimit 32` and `MaxClients 512`. Based on the comments of the file, you'll want to edit the cpanel template file (probably `/var/cpanel/templates/apache2/main.default`) instead of editing `httpd.conf` directly, and then regenerate the config using the cpanel `build_apache_conf` command. – Shane Madden Feb 23 '13 at 21:25
  • Yeah it is edited from a WHM. I was wondering about the Server limit in there. The Server limit was updated to 32 but for some reason I got `The following settings are invalid and were rejected: maxclients: 512`. Not sure if that is a cpanel restriction or what....any ideas? – kmoney12 Feb 23 '13 at 21:34
  • @hellohellosharp That's interesting. Maybe just change it in `httpd.conf` directly, then? You're using Apache 2.x, not 1.3, right? (1.3 had the compiled-in behavior) – Shane Madden Feb 23 '13 at 21:43
  • Added some info to the post... – kmoney12 Feb 23 '13 at 21:48
  • I edited the http.conf directly and restarted apache. Is there a way for me to tell if the MaxClients is really 512? – kmoney12 Feb 23 '13 at 21:54
  • @hellohellosharp I'm not aware of any way to check it during runtime, I don't think it's included in the output from mod_status. If it's set in the file and didn't throw any warnings during start (check Apache's error log), then it should be active. – Shane Madden Feb 23 '13 at 21:57
  • 1
    the status module can show you these details. – ETL Feb 24 '13 at 02:50
-1

I would suggest that you use a reverse proxy, something like nginx or lighttpd can handle many more connections than apache. Depending on how your sites use htaccess you can also use nginx/lighttpd with fcgi and forego apache completely.

Shoshomiga
  • 84
  • 7
  • 4
    Really?? He needs to add two lines of configuration to `httpd.conf` and your solution is to rip out most of the software he has running and replace it?! Every piece of software has it's problems, but uneducated misuse causes worse problems. – Chris S Feb 23 '13 at 21:11
  • Ultimately nginx can handle more active connections, its up to the OP to decide what to use, I was just putting forward my suggestion. – Shoshomiga Feb 24 '13 at 12:06