I have a high load server using php/apache/APC working fine for months. In the last 2 days it has crashed midday. if I restart apache it works so the problem is probably in the apache/php/apc stack. It runs at up to 200 hits/sec running highly optimized PHP script with some algorithms. I use newrelic for monitoring and as far as I can see the problem is not related to databases or disk access.
Here is what I see in newrelic
I don't see anything special in syslog or error log of apache.
Does the high spike in running instances count tells something?
What can I check to get to the bottom of it?