daemon running for several hours, and dies

Question

After restart server, this is running:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19653 lighttpd 16 0 67584 39m 16m R 21.3 15.3 0:17.60 php-cgi
8006 mysql 15 0 137m 22m 4800 S 5.3 8.8 1:07.91 mysqld
13364 lighttpd 16 0 63560 39m 20m S 2.3 15.3 0:34.57 php-cgi
1 root 15 0 2068 648 556 S 0.0 0.2 0:00.55 init
1741 root 15 0 2196 1012 816 R 0.0 0.4 0:00.01 top
7449 root 15 -4 2152 552 344 S 0.0 0.2 0:00.00 udevd
7839 root 15 0 1724 608 512 S 0.0 0.2 0:00.00 syslogd
7888 root 18 0 7024 1056 660 S 0.0 0.4 0:00.00 sshd
7899 root 18 0 2724 872 700 S 0.0 0.3 0:00.00 xinetd
7934 root 18 0 3616 1288 1116 S 0.0 0.5 0:00.00 mysqld_safe
11499 root 18 0 9180 1676 660 S 0.0 0.6 0:00.03 sendmail
13330 smmsp 20 0 8132 1472 608 S 0.0 0.6 0:00.00 sendmail
13344 lighttpd 15 0 6792 2732 756 S 0.0 1.0 0:01.30 lighttpd
13347 lighttpd 21 0 47144 8984 6168 S 0.0 3.4 0:00.09 php-cgi
13356 lighttpd 15 0 47144 8984 6168 S 0.0 3.4 0:00.09 php-cgi
13358 lighttpd 15 0 68128 43m 20m S 0.0 17.0 0:08.42 php-cgi
13360 lighttpd 21 0 47144 8988 6168 S 0.0 3.4 0:00.08 php-cgi
13365 lighttpd 15 0 47144 8992 6168 S 0.0 3.4 0:00.09 php-cgi
13446 root 18 0 4408 1108 560 S 0.0 0.4 0:00.00 crond
13669 xfs 18 0 3456 1180 728 S 0.0 0.5 0:00.00 xfs
13681 root 18 0 5580 708 428 S 0.0 0.3 0:00.00 saslauthd
13682 root 18 0 5580 444 164 S 0.0 0.2 0:00.00 saslauthd
13767 root 15 0 7496 5432 1604 S 0.0 2.1 0:00.03 miniserv.pl
21715 lighttpd 16 0 78908 49m 15m S 0.0 19.2 0:16.40 php-cgi
32674 root 18 0 10016 2828 2284 S 0.0 1.1 0:00.03 sshd
32747 root 15 0 3752 1536 1236 S 0.0 0.6 0:00.02 bash

but after several hours, it will become like this:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 15 0 2068 656 556 S 0.0 0.3 0:03.89 init
3723 root 15 0 4408 1112 564 S 0.0 0.4 0:00.16 crond
9220 root 18 0 9860 2828 2288 S 0.0 1.1 0:00.08 sshd
9688 root 15 0 3752 1540 1236 S 0.0 0.6 0:00.11 bash
10210 root 15 0 2196 1004 816 R 0.0 0.4 0:00.00 top
21529 root 15 0 1724 608 512 S 0.0 0.2 0:03.19 syslogd
21602 root 18 0 7024 1052 660 S 0.0 0.4 0:13.52 sshd
21623 root 18 0 2724 868 700 S 0.0 0.3 0:00.00 xinetd
21717 mysql 15 0 138m 24m 4988 S 0.0 9.4 79:19.17 mysqld

This will happen over and over again.. The webmin also dies

Im using:

-CentOS 5.3

-lighttpd
"max-procs" => 1
"PHP_FCGI_CHILDREN" => "6"
"PHP_FCGI_MAX_REQUESTS" => "500"


-PHP
use eaccelerator

-monit
monit error in respawn: Id "mo" respawning too fast: disabled for 5 minutes
monit code in inittab: mo:234:respawn:/usr/bin/monit -Ic /etc/monit.d/monitrc

I also dont know how to fix monit error in inittab

Before this, nothing like this happen when the server running with below config:
-default mod_fastcgi config
-without monit

So, I start use monit because lighttpd dies when Im sleeping and also my website a bit slow, plus high RAM usage on php-cgi (My RAM only 256MB, host more than 20 websites in it)

Answer 1

You are probably running afoul of the out-of-memory killer.

Increase the amount of RAM in the server, or if not possible, increase swap.

I'm a little surprised you can get CentOS 5.3 to do anything useful with so little memory.

Answer 2

Reduce your FCGI_CHILDREN to zero. This is recommended for newer lighttpd versions. Set more max-procs instead. Try tweaking the number of max-procs. If your site is not hammered all the time, just having 2 will be more than enough.