Centos 6 server is locked, cannot access via putty and cannot ping

Question

I have rented a Centos6 server and running an Apache Tomcat 7 server with an Apache httpd server proxying it.

Sometimes my server gets locked and I cannot access it via putty even ping.

My question is: how can I be sure that this is about connection problems or application servers (I currently have a hibernate problem that causes the app servers not to respond because of unneeded heavy queries)?

Thanks in advance.

(PS: Fixing the problem by restarting the server)

EDIT: Following are the applications and props on my server:

*Apache httpd server --> proxy, *Apache Tomcat server --> app server, *Mysql DBMS

And got hd images appr. 69000 (99GB) serving from one folder.

EDIT2: The system comes back normal when I restart machine and run the servers again.

Answer 1

The easiest approach is probably to set up a cron job that runs every minute and does something trivial that you can check later whether it happened or not at a specific time. For example, create a file /etc/cron.d/datelog with the contents:

* * * * * root date >> /var/log/date.log

Let this run for a while. The next time the VPS stops responding, wait a few minutes before rebooting it, then check /var/log/date.log once the VPS comes back up.

If that file has entries for the time when the VPS was unresponsive, you can be fairly sure it was running normally and for whatever reason was just not accessible over the network, so you can focus on investigating why the VPS became inaccessible over the network. If the file doesn't have entries for when the VPS was unresponsive, that does not conclusively prove that the VPS stopped or crashed, but it's a very strong indication that something of that sort happened and that you should investigate further based primarily on such a hypothesis.

This approach has the advantage of not requiring the installation or configuration of any additional software; a cron daemon is included in the standard installation on almost any distribution, and I doubt CentOS is any different in this regard, because it is useful for so many other things.

Answer 2

It could be a number of things.

First off you need to have a look in the system log files and see if there is anything of note. I'd start with /var/log/messages and look around the time you lost contact.

You have stated its a VPS - I presume you have no access to the physical underlying host or hypervisor?

It could be you are hitting a limit or a trigger of some sort and the hosting provider is suspending or killing your VM instance. You should check in the control panel - it should report the states of any VM's you may also find it offers a 'KVM' or console access to the VM's where you could try and access the instance when its in its 'broken' state.

I would try monitoring the instance from within using something like Icinga, Nagios or Cacti. They all have plugins to monitor a number of applications and systems. Install and setup to monitor your httpd, network connectivity, CPU, Memory etc. That way you will at least be able to see what state things where in before the machine became unreachable.