1

How do you determine why MySQL stops running?

I recently upgraded an Ubuntu server to 16.04, and now MySQL will suddenly stop after a few hours. It's acting as the backend for a Wordpress install. My MySQL version is now:

mysql  Ver 14.14 Distrib 5.5.54, for debian-linux-gnu (x86_64) using readline 6.3

Checking who -b shows the server hasn't rebooted in a few days, so it's not an issue with MySQL not being started properly.

The log /var/log/mysql/error.log is empty, as well as all the archived ones.

The log /var/log/syslog.log shows nothing unusual or referencing mysql.

I have the unattended-upgrades package installed to automatically install security updates, and the log /var/log/unattended-upgrades/unattended-upgrades.log shows:

2017-03-22 07:00:32,459 INFO Packages that will be upgraded: libc-bin libc-dev-bin libc6 libc6-dev locales multiarch-support
2017-03-22 07:00:32,459 INFO Writing dpkg log to '/var/log/unattended-upgrades/unattended-upgrades-dpkg.log'
2017-03-22 07:00:38,595 INFO All upgrades installed
2017-03-22 07:00:39,846 ERROR No '/usr/bin/mail' or '/usr/sbin/sendmail',can not send mail. You probably want to install the 'mailx' package.
2017-03-22 07:00:39,846 WARNING Found /var/run/reboot-required, rebooting

indicating it did install some updates, and has scheduled a reboot, but it hasn't rebooted yet.

How do I find out definitively what caused MySQL to stop or crash?

Cerin
  • 3,497
  • 17
  • 57
  • 72
  • The log should never be completely empty, even when mysqld is running normally. Check the config if a different log file is configured, increase log level if needed. – Gerald Schneider Mar 22 '17 at 13:49

3 Answers3

1

It can be stopped due to OOM (out of memory killer). In short: when memory, swap included, is almost completely full, the Linux Kernel can kill some demanding processes to alleviate memory pressure.

Does dmesg | grep -i oom report something?

shodanshok
  • 44,038
  • 6
  • 98
  • 162
  • I considered that too. However, I have an external server remotely monitoring cpu and memory usage, and it was showing no more than 30% memory in use prior to MySQL stopping. – Cerin Mar 22 '17 at 18:21
0

First, I would enable logging since your mysql error log is empty: https://dev.mysql.com/doc/refman/8.0/en/error-log.html

Add this:

/etc/my.cnf  # most common location

log-error=/var/log/mysqld.log

Also, I would not allow outside connections to Port 3306. If you need remote access to MySQL, use a SSH Tunnel instead.

Marc Pope
  • 36
  • 6
0

Try to look for something that could be killing mysqld. Check this blogpost on what tools to use. Auditctl worked for one case I was involved in. https://www.percona.com/blog/2015/03/06/stopped-mysql-tracing-back-signals-sent-mysql/

jerichorivera
  • 489
  • 1
  • 4
  • 12