0

My VPS server has been acting up lately, for example last weekend the PHP module Imagemagick simply stopped working and I had to do a PERC uninstall / reinstall.

Today my server has been totally unresponsive for up to 20 minutes. This is mission critical software and the downtime is totally unacceptable. Here are the logs prior to shutdown. I would greatly appreciate any advice, we have contacted our host, but ultimately this falls on my shoulders to understand and fix. I'm ultimately just hoping to get to the next step of my understanding of this problem!

[THU AUG 13 13:11:24 2015] [notice] SIGUSR1 received.  Doing graceful restart
[THU AUG 13 13:11:26 2015] [warn] RSA server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
[THU AUG 13 13:11:26 2015] [warn] Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[THU AUG 13 13:11:26 2015] [notice] mod_bw : Memory Allocated 0 bytes (each conf takes 48 bytes)
[THU AUG 13 13:11:26 2015] [notice] mod_bw : Version 0.92 - Initialized [0 Confs]
[THU AUG 13 13:11:26 2015] [notice] Apache/2.2.29 (Unix) mod_ssl/2.2.29 OpenSSL/1.0.1e-fips DAV/2 mod_bwlimited/1.4 configured -- resuming normal operations
[THU AUG 13 13:42:51 2015] [error] [client 88.252.32.143] ModSecurity: Access denied with code 406 (phase 1). Operator EQ matched 0 at REQUEST_HEADERS. [file "/opt/mod_security/hg_rules.conf"] [line "91"] [id "900161"] [msg "XMLRPC Request with no UA/Ref"] [hostname "www.REDACTED.com"] [uri "/xmlrpc.php"] [unique_id "VczXGzJ0SbEAAD9dIycAAADI"]
[THU AUG 13 14:33:49 2015] [error] [client 72.27.221.129] ModSecurity: Access denied with code 406 (phase 1). Operator EQ matched 0 at REQUEST_HEADERS. [file "/opt/mod_security/hg_rules.conf"] [line "91"] [id "900161"] [msg "XMLRPC Request with no UA/Ref"] [hostname "www.REDACTED.com"] [uri "/xmlrpc.php"] [unique_id "VczjDTJ0SbEAAD9eJIEAAACR"]
[THU AUG 13 14:40:44 2015] [error] [client 52.13.23.41] ModSecurity: Access denied with code 403 (phase 1). Pattern match "CONNECT" at REQUEST_METHOD. [file "/opt/mod_security/10_asl_rules.conf"] [line "59"] [id "340361"] [rev "2"] [msg "CONNECT method denied"] [data "CONNECT"] [severity "CRITICAL"] [hostname "www.wikipedia.org"] [uri "/"] [unique_id "VczkrDJ0SbEAAD9cIuEAAABX"]
[THU AUG 13 15:09:28 2015] [warn] RSA server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
[THU AUG 13 15:09:28 2015] [warn] Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[THU AUG 13 15:09:28 2015] [notice] suEXEC mechanism enabled (wrapper: /usr/local/apache/bin/suexec)
[THU AUG 13 15:09:28 2015] [notice] ModSecurity for Apache/2.8.0 (http://www.modsecurity.org/) configured.
[THU AUG 13 15:09:28 2015] [notice] ModSecurity: APR compiled version="1.5.1"; loaded version="1.5.1"
[THU AUG 13 15:09:28 2015] [notice] ModSecurity: PCRE compiled version="8.36 "; loaded version="8.36 2014-09-26"
[THU AUG 13 15:09:28 2015] [notice] ModSecurity: LUA compiled version="Lua 5.1"
[THU AUG 13 15:09:28 2015] [notice] ModSecurity: LIBXML compiled version="2.9.2"
[THU AUG 13 15:09:28 2015] [notice] Status engine is currently disabled, enable it by set SecStatusEngine to On.
[THU AUG 13 15:09:29 2015] [warn] RSA server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
[THU AUG 13 15:09:29 2015] [warn] Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[THU AUG 13 15:09:29 2015] [notice] mod_bw : Memory Allocated 0 bytes (each conf takes 48 bytes)
[THU AUG 13 15:09:29 2015] [notice] mod_bw : Version 0.92 - Initialized [0 Confs]
[THU AUG 13 15:09:29 2015] [warn] pid file /usr/local/apache/logs/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
[THU AUG 13 15:09:29 2015] [notice] Apache/2.2.29 (Unix) mod_ssl/2.2.29 OpenSSL/1.0.1e-fips DAV/2 mod_bwlimited/1.4 configured -- resuming normal operations Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)
[THU AUG 13 15:11:12 2015] [notice] SIGUSR1 received.  Doing graceful restart
[THU AUG 13 15:11:13 2015] [warn] RSA server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
[THU AUG 13 15:11:13 2015] [warn] Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[THU AUG 13 15:11:13 2015] [notice] mod_bw : Memory Allocated 0 bytes (each conf takes 48 bytes)
[THU AUG 13 15:11:13 2015] [notice] mod_bw : Version 0.92 - Initialized [0 Confs]
[THU AUG 13 15:11:13 2015] [notice] Apache/2.2.29 (Unix) mod_ssl/2.2.29 OpenSSL/1.0.1e-fips DAV/2 mod_bwlimited/1.4 configured -- resuming normal operations
Peter Bartlett
  • 1

1 Answers1

0

Mod_security is a seriously nice application level firewall. Based on the messages you are receiving my bet is that your site isn't actually going down / crashing but that mod_security is blocking access. I've seen it misconfigured so that it generates only a blank white page more than once.

Based on the lines out of your log containing ModSecurity: Access denied with code 406 we have the ID of the potentially problematic filters 900161. You should be able to disable this using SecRuleRemoveById in your host / virtual host, assuming an Apache server.

You might also want to look at monitor mode for mod_security.

Tim Brigham
  • 15,465
  • 7
  • 72
  • 113