3

We have a webserver with apache 2.2.14, PHP 5.3.2.

PHP is executed using mod_fcgid (see bottom). Everything works fine but sometimes, and we still have to figure out what triggers this, when php processes are "rotated" they remain active and orphaned: apache spawns new php processes and the old ones remains on the system. Killing them doesn't always kicks them away. More likely an "apache2ctl graceful" frees the system from this stale processes. We found this in error log: [Tue Jun 18 20:49:54 2013] [warn] mod_fcgid: process 2009 graceful kill fail, sending SIGKILL from what I found searching around this is quite normal, but that's all I found in apache logs during one of this processes leak.

This event is luckily seldomly happening, usually apache and php operate normally and with no problems during fcgid childrens renewal. How can we understand what's going wrong in these situations?

mod_fcgid config in site:

<IfModule mod_fcgid.c>
SuexecUserGroup domain domain
<Directory /var/www/fomain.it/htdocs/>
AddHandler fcgid-script .php
FCGIWrapper /var/www/fcgi/domain.it/fcgi-starter-php .php
Options +ExecCGI -Indexes
AllowOverride FileInfo Options
Order allow,deny
Allow from all
</Directory>
<Directory /var/www/fcgi/domain.it/>
AllowOverride None
Options +ExecCGI MultiViews -Indexes
Order allow,deny
Allow from all
</Directory>
</IfModule>

/var/www/fcgi/domain.it/fcgi-starter-php:

#!/bin/sh

PHPRC=/var/www/fcgi/domain.it/php/

export PHPRC
PHP_FCGI_CHILDREN=8
export PHP_FCGI_CHILDREN
PHP_FCGI_MAX_REQUESTS=5000
export PHP_FCGI_MAX_REQUESTS

exec /usr/lib/cgi-bin/php $1

We currently found a workaround to avoid the server to fill up with orphaned processes. You can get rid of the stale processes with:

apache2ctl graceful

and kill those processes with:

pkill -f -x /usr/lib/cgi-bin/php -P 1

scripting and scheduling those two commands (with proper checks) will avoid the server to host plenty of useless procs, but the issue is still present.

Maxxer
  • 282
  • 4
  • 18

3 Answers3

0

I don`t have a real answer, but maybe more information can help to solve the problem. I like to tell, I have the same problem on a windows server with PHP 5.3.5.

Some cgi-processes remain as a kind of zombie task, after the done their real execution. They even ignore settings like max_execution time.

Currently I have a scheduled script what kills these old processes to. problem with this solution is, even "normal" cgi processes where killed, maybe it would be usefull to detect the runtime of the processes and only kill them if they have expired their max_execution time.

Radon8472
  • 185
  • 8
0

I had a similar problem that fcgid ran out of available process slots after some period of activity.

The log message was roughly:

[fcgid:warn] mod_fcgid: can't apply process slot for /var/www/cgi-bin/xxx/php-cgi, referer: ...

I tracked the issue down to this:

[fcgid:emerg] (35)Resource deadlock avoided: [client ....] mod_fcgid: can't get pipe mutex, referer: ...

which gets caused by improper locking. In my case, Apache used fcntl() locking (by default on debian), so I changed that to flock() in apache2.conf:

Mutex flock:${APACHE_LOCK_DIR} default

Reference that led me to the solution: https://bz.apache.org/bugzilla/show_bug.cgi?id=53999

Documentation about various locking options (fcgid has a warning advice against usign it with anything that would involve threads): https://httpd.apache.org/docs/2.4/mod/core.html#mutex

exa
  • 571
  • 4
  • 14
  • thanks for your feedback. I'll accept the answer even if I cannot test it anymore, having happily switched to nginx+php-fpm long time ago. Thanks again for reporting back! – Maxxer Mar 11 '16 at 10:39
0

Changing to "sem" as Thomas suggested in https://bz.apache.org/bugzilla/show_bug.cgi?id=53999 worked for me.

So I would recommend anyone to edit his apache2.conf, comment the Mutex line, and put:

# Mutex file:${APACHE_LOCK_DIR} default   # Original debian config
Mutex sem  # Solves orphaned PHP processes.
deavidsedice
  • 101