Apache, Out of memory

Question

I get very often the following messages:

I guess this happens because I have not enough RAM. Is this correct ?

Is there something I can do ?

thanks

update: I've MPM prefork installed

<IfModule mpm_prefork_module>
StartServers          5
MinSpareServers       5
MaxSpareServers      10
MaxClients          150
MaxRequestsPerChild   0

Answer 1

This is your problem:

MaxRequestsPerChild   0

A zero value means that the individual child processes will never exit, slowly consuming more and more RAM (it's code dependent how long it will take). Set that value to something within reason (try 5000 to start), then after a good bit and Apache is spun up to handle load you will see a point in your 'ps' of where the Apache child is topping out in resident (RSS) memory.

Using this value as your maximum child size, you then have to set MaxClients to be the memory you want to use. So if you have 256mb, you'd want to leave some for the OS so let's say 248MB for Apache (this isn't exact, depends on what else is on your system). This means you could have at most 1.5MB for each Apache child which is kind of silly - most Apache processes tend to run in the 15MB range on average (some higher).

Let's say you have an average 20MB Apache child process with 5000 requests - you could decrease that to 15MB (example) if you dropped the requests to 4000; it's something you have to play with. Whatever combination you come up with, do the math and set your MaxClients to top out (248/15 = 16, e.g.).

You have to do a little playing around with the values to narrow in on where you want each one to get the memory settings just right. With your current setting of 0 you're letting Apache run amok.

Answer 2

I guess this happens because I have not enough RAM. Is this correct ?

System kill Apache processes because it needs more memory

Is there something I can do ?

Add memory, swap or tune mpm settings(/etc/apache2/apache2.conf, search MPM specific)

Answer 3

The biggest thing you can do is trim all daemons to only use the memory that they need in order to get the job done. Reduce your database memory cache, reduce the number of httpd children, disable unused httpd modules, etc. Obviously this will affect performance, so you'll have to monitor the system for a while after and tweak as necessary.

Resist the temptation to patch the problem by adding swap space. This is actually a non-solution, since system performance will be driven into the ground.

Answer 4

I'm assuming you're using the preform MPM (which you can check by running apache2 -V and looking for forked: yes in the output).

First, try editing /etc/apache2/apache2.conf and lowering the MaxClients value. This value determines how many concurrent clients apache can serve. Obviously you'll need to restart/reload apache for the change to take effect.

Alternatively, if you don't have to use the prefork MPM (the most likely reason you might have to would be that you're running PHP), you can switch to the worker MPM. This uses threads rather than processes for concurrency and is therefore a lot more memory efficient.

To do this, run:

aptitude install apache2-mpm-worker

Answer 5

you probably have a buggy application running inside apache, for example a badly written php extension or custom apache module or just a runaway script. your maxrequestsperchild is set to zero so apache's fork()ed children will be kept alive "forever" to serve requests (where forever is relatively defined to the universe of your apache main process). while doing that your buggy application requests memory without ever freeing it so the children will keep sucking memory. OS marks these processes as good candidates for the next outofmem bloodshed. if you want to hide the bug and pretend nothing happened go and set maxrequestsperchild to something like 500 (this value really depends on how many requests per second your service handles, e.g. if you serve 500reqps then 5000 would be a good starting point so you will not cause fork thundering herds). don't use mpm worker, that will totally kill entire thread pools when OS decides to forcibly free some memory, plus most of php extensions arent exactly threadsafe (supposing you're using php). best thing to do is to start debugging. if not with gdb then i believe you can find some apache modules written for debugging and profiling running applications.