I have a cloud VPS at Digital Ocean , Recently it went down on its own, i was using pingdom alert which notified me about it, so i booted up the VPS again to find out what caused this . How can find what caused the unexpected system halt ?
System Info : OS : Cents Os 6.4 x64
i did
[root@user1 myserver]# cat /var/log/messages
Sep 8 03:12:02 user1 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="970" x-info="http://www.rsyslog.com"] rsyslogd was HUPed
Sep 9 23:33:52 user1 init: tty (/dev/tty1) main process (1295) killed by TERM signal
Sep 9 23:33:52 user1 init: tty (/dev/tty2) main process (1297) killed by TERM signal
Sep 9 23:33:52 user1 init: tty (/dev/tty3) main process (1301) killed by TERM signal
Sep 9 23:33:52 user1 init: tty (/dev/tty4) main process (1303) killed by TERM signal
Sep 9 23:33:52 user1 init: tty (/dev/tty5) main process (1305) killed by TERM signal
Sep 9 23:33:52 user1 init: tty (/dev/tty6) main process (1307) killed by TERM signal
Sep 9 23:34:00 user1 acpid: exiting
Sep 9 23:34:00 user1 auditd[954]: The audit daemon is exiting.
Sep 9 23:34:00 user1 kernel: type=1305 audit(1378769640.655:2459): audit_pid=0 old=954 auid=4294967295 ses=4294967295 res=1
Sep 9 23:34:00 user1 kernel: type=1305 audit(1378769640.757:2460): audit_enabled=0 old=1 auid=4294967295 ses=4294967295 res=1
Sep 9 23:34:00 user1 kernel: Kernel logging (proc) stopped.
Sep 9 23:34:00 user1 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="970" x-info="http://www.rsyslog.com"] exiting on signal 15.
Sep 10 01:15:01 user1 kernel: imklog 5.8.10, log source = /proc/kmsg started.
Sep 10 01:15:01 user1 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="960" x-info="http://www.rsyslog.com"] start
Sep 10 01:15:01 user1 kernel: Initializing cgroup subsys cpuset
Sep 10 01:15:01 user1 kernel: Initializing cgroup subsys cpu
Sep 10 01:15:01 user1 kernel: Linux version 2.6.32-358.6.2.el6.x86_64 (mockbuild@c6b8.bsys.dev.centos.org) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Thu May 16 20:59:36 UTC 2013
Sep 10 01:15:01 user1 kernel: Command line: root=LABEL=DOROOT ro
Sep 10 01:15:01 user1 kernel: KERNEL supported cpus:
Sep 10 01:15:01 user1 kernel: Intel GenuineIntel
Sep 10 01:15:01 user1 kernel: AMD AuthenticAMD
Sep 10 01:15:01 user1 kernel: Centaur CentaurHauls
Sep 10 01:15:01 user1 kernel: BIOS-provided physical RAM map:
Sep 10 01:15:01 user1 kernel: BIOS-e820: 0000000000000000 - 000000000009dc00 (usable)
Sep 10 01:15:01 user1 kernel: BIOS-e820: 000000000009dc00 - 00000000000a0000 (reserved)
Sep 10 01:15:01 user1 kernel: BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
Sep 10 01:15:01 user1 kernel: BIOS-e820: 0000000000100000 - 000000003fffd000 (usable)
Sep 10 01:15:01 user1 kernel: BIOS-e820: 000000003fffd000 - 0000000040000000 (reserved)
Sep 10 01:15:01 user1 kernel: BIOS-e820: 00000000feffc000 - 00000000ff000000 (reserved)
Sep 10 01:15:01 user1 kernel: BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
Sep 10 01:15:01 user1 kernel: DMI 2.4 present.
Sep 10 01:15:01 user1 kernel: SMBIOS version 2.4 @ 0xFDAD0
- more 304 lines
Memory is enough i guess
[root@]# free -m
total used free shared buffers cached
Mem: 996 213 783 0 9 90
-/+ buffers/cache: 113 883
Swap: 2047 0 2047
HDD space is fine too
[root@]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda 30G 27G 2.0G 94% /
none 499M 0 499M 0% /dev/shm
UPDATE: I contacted vps provider and asked them the cause, and i got reply
There has been a response to your ticket:
It looks like the "power off" was due to a kernel panic from your server if you disable
/dev/shm
from fstab it should help you out
received more reply
There has been a response to your ticket:
To be more clear, there are a number of possible reasons that your machine could power off, including disk corruption. The /dev/shm item in /etc/fstab is a dynamically sized RAM-based filesystem mounted to /tmp on our CentOS droplets. If this partition grows larger than its maximum size (500MB) it will cause a system failure on your droplet. This could be caused by a large build job, for example. You can either increase the size of the shm in fstab (no larger than your maximum RAM), or unmount it.
I also recommend running a fsck on your root filesystem (/dev/vda) by powering off your droplet and loading our custom recovery kernel, DO-recovery-fsck-static, from the droplet control panel. You can then boot into it and run fsck -y /dev/vda. Recovered files will be located in /lost+found.
