-1

Today I saw that a website of mine isn't working so I ssh'd to the server and executed ps -eF. I see about 200 PHP processes that are running all for 4 hours.

Apache is built with mpm event and mod fcgid.

I killed all the PHP processes and now it's running fine, why does this happen? is this expected behavior?

I don't really understand how processes how Apache keeps track of the number of PHP processes and their process IDs, so it would be nice if someone can also give some reference when I can read about this.

Also, I used the "ab" command (Apache Benchmark) to see if this happens all the time, so I ran it about 4-5 times in a row with 30 concurrent requests and again there are like 150 PHP processes running, when I keep running "ab" now it doesn't spawn more processes and the website is still working. To be clear the 4-5 times I executed "ab" were not a the same time, as soon as one "ab" process finished I executed another one, but each time I executed "ab" I did 30 concurrent requests.

Please shed some light on this! Thank you :)

fiftyeight
  • 157
  • 1
  • 7

1 Answers1

2

Yes this is expected behaviour. It would appear you had 5 copies of ab running at the same time. This resulted in your 150 PHP processes to handle the 30 requests for each of your 5 ab processes.

Apache works best when all its threads can run in memory. There will be a load at which the threads will need to be swapped to disk. Response time will degrade rapidly when this occurs. Killing processes as you did will improve performance. Server performance may become poor in this case. If the maximum threads are configured too high it is relatively easy to perform a DOS (Denial Of Service) using this behavior.

Another reason you can end up with problems is if two processes deadlock on two resources. If the other processes need either of these resources, they may fail to respond. Apache will limit the number of processes created in this case. Killing either of the deadlocked processes should clear the problem. Depending on the normal locking order it may require a specific process to be killed to prevent another deadlock. This will have little or no impact on server performance.

Programming errors may also cause threads to hang. This can result in the theads sticking as you saw. Finding what happened is difficult after the fact, but the contents of your apache server's logs may help. Check the error log for problems. The access log may have entries for the requests that hung around the time you killed the processes.

Apache should kill off some of the processes if they are not used for a period of time. This is controlled by the MaxSpareThreads parameter. Check your control file which should have some comments about the parameters for Threads, Servers, and Clients. The default values are usually good.

If there are problems with code, setting MaxRequestsPerChild to somewhere in the range 5000 to 100 often helps. The smaller the number the more often new threads need to be created and the harder the apache server needs to work.

EDIT: Apache will open up a number of threads at startup. This depends on the interaction of several parameters. See the MPM configuration documentation for details on the parameters. This is independent of any loading of the server, and normally be the minimum number of threads a running server will keep running.

BillThor
  • 27,354
  • 3
  • 35
  • 69
  • This is a good answer however I executed "ab" (apache benchmark) 5 times but not concurrently. Each time I did 30 concurrent requests, but the 5 times I executed "ab" were not at the same time, just in case it influences the answer. I'll also edit the question – fiftyeight Nov 27 '11 at 20:26
  • @fiftyeight: As noted in my edit. It is possible that 150 is the minimum number of threads your server keeps running. – BillThor Nov 27 '11 at 20:52
  • The processes are PHP processes, not threads, is there some value you know about that might cause a minimum of PHP processes open or make PHP processes last for too long? – fiftyeight Nov 27 '11 at 22:45
  • Each thread may need a PHP process to run the code being served up. – BillThor Nov 28 '11 at 03:05