Apache with mod_php high memory utilization

Question

We have Magento application deployed on Apache with mod_php and mysql. I have observed that sometime apache server starts consuming high memory which causes memory swapping and results in high load on servers. whenever there is high load on apache server, the apache processes which are causing the high load were in sleep mode at mysql end and CLOSE_WAIT state at client side. Any help is appreciated to resolve this issue.

Answer 1

@raj you'll probably find that the processes are in an S state as they are waiting on another process (in a D state) bound by I/O ie. For the machine to stop swapping.

So the key is to address two key issues, standard memory allocation and memory commitment.

When running Magento, it is fairly easy to massively over commit on RAM simply by setting PHP Max memory limits too high. Under normal circumstances, it won't have any ill effects, but the moment the app tries to use that memory, it will almost immediately go into swap (if you've run out of RAM that is).

Then, you'll likely find that Apache isn't the main concern for memory usage, but rather MySQL is more likely the culprit.

You haven't mentioned your server specification or any specific details for your store, so it would be impossible to accurately advise. But simply revising your Apache and MySQL configuration would be an easy place to start. Drop the number of Apache threads, MySQL connections, allocated memory commitments. Then start graphing with a tool like Munin to actually be able to correlate traffic, threads, CPU usage and memory usage. Without doing so, you are working blind.

But I would advise to stop poking around netstat as you are a long way off needing to diagnose TCP states; you need to start from the beginning.

Answer 2

Go read some good books on the subject of HTTP tuning and performance management (really - you're unlikely to get the answer here).

which causes memory swapping

Then your server is badly configured - a webserver should never start swapping. Apply the usual apache tuning ritual and reduce php max memory.

whenever there is high load on apache server, the apache processes which are causing the high load were in sleep mode at mysql end

Eh? A sleeping process by definition does not contribute to load. What mhysql end?

Answer 3

Use this script: http://saiweb.co.uk/linux/apache/bash%20script/linux-the-sysadmin-script-part-3/

to determine how much memory your apache processes are using.

Then limit MaxClients in apache config, to amount of process you can run without swapping. In this case when the problem occurs you will not have high LA, and you can find the real source of problems. Here's my version of script (link above seems to be broken now)

#!/bin/bash
       if [ -z "$1" ]; then
                echo "Usage: sysadmin appmem app_name i.e. (sysadmin appmem apache)";
        else
                RRES=(`ps aux | grep "$1" | grep -v 'grep' | grep -v "$0" | awk '{print $6}'`);
                VRES=(`ps aux | grep "$1" | grep -v 'grep' | grep -v "$0" | awk '{print $5}'`);
                COUNT=0;
                VMEM=0;
                RMEM=0;
                for RSS in ${RRES[@]}
                do
                        RMEM=$(($RSS+$RMEM));
                done;
                for VIRT in ${VRES[@]}
                do
                        VMEM=$(($VIRT+$VMEM));
                        COUNT=$(($COUNT+1));
                done;
                VMEM=$(($VMEM/$COUNT));
                VMEM=$(($VMEM/1024));
                RMEM=$(($RMEM/1024));
                echo -e "$YELLOW ----- MEMORY USAGE REPORT FOR '$1' ----- $CLEAR";
                echo "PID Count: $COUNT";
                echo "Shared Mem usage: $VMEM MB";
                echo "Total Resident Set Size: $RMEM MB";
                echo "Mem/PID: $(($RMEM/$COUNT)) MB";
        fi