1
I've a web page that sometimes give a blank page with a 200 status and I want to automatically restart the service when this happen. To do it I want to use Monit and a nagios plugin that allow to search text in the page instead only check the status, but my problem is that monit takes a long time to detect when the page is down and it restart the service a lot of times even when is running fine again.
My Monit config file:
check program nginx with path "/etc/monit/files/check_nginx.sh"
every 1 cycle
start program = "/etc/init.d/nginx start"
stop program = "/etc/init.d/nginx stop"
if status != 0 then restart
As Monit don't allow to pass arguments, I'm using this simple script in bash:
#!/bin/bash
/etc/monit/files/check_http -H 127.0.0.1 -w 5,0 -c 10,0 -p 88
exit $?
When nagios is running then is detected without problem by Monit, but when I stop the daemon it took about 2 cycles to detect that is down (this is acceptable because are 2 minutes). After detect the problem, Monit runs a restart command without problem:
CEST Sep 26 18:31:47] error : 'nginx' connect to address localhost and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:31:47] info : 'nginx' trying to restart
[CEST Sep 26 18:31:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:31:47] info : 'nginx' start: /etc/init.d/nginx
That is fine, but the problem is that don't detect when is up again and it keeps restarting the service every cycle until I disable the Monit service:
[CEST Sep 26 18:31:47] error : 'nginx' connect to address localhost and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:31:47] info : 'nginx' trying to restart
[CEST Sep 26 18:31:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:31:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:32:47] error : 'nginx' connect to address localhost and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:32:47] info : 'nginx' trying to restart
[CEST Sep 26 18:32:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:32:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:33:47] error : 'nginx' connect to address localhost and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:33:47] info : 'nginx' trying to restart
[CEST Sep 26 18:33:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:33:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:34:47] error : 'nginx' connect to address localhost and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:34:47] info : 'nginx' trying to restart
[CEST Sep 26 18:34:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:34:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:35:47] error : 'nginx' connect to address localhost and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:35:47] info : 'nginx' trying to restart
[CEST Sep 26 18:35:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:35:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:36:47] error : 'nginx' connect to address localhost and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:36:47] info : 'nginx' trying to restart
[CEST Sep 26 18:36:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:36:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:37:47] error : 'nginx' connect to address localhost and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:37:47] info : 'nginx' trying to restart
[CEST Sep 26 18:37:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:37:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:38:47] error : 'nginx' connect to address 127.0.0.1 and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:38:47] info : 'nginx' trying to restart
[CEST Sep 26 18:38:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:38:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:39:47] error : 'nginx' connect to address 127.0.0.1 and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:39:47] info : 'nginx' trying to restart
[CEST Sep 26 18:39:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:39:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:40:47] info : 'nginx' status succeeded
[CEST Sep 26 18:44:47] error : 'nginx' connect to address 127.0.0.1 and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:44:47] info : 'nginx' trying to restart
[CEST Sep 26 18:44:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:44:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:45:47] error : 'nginx' connect to address 127.0.0.1 and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:45:47] info : 'nginx' trying to restart
[CEST Sep 26 18:45:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:45:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:46:47] error : 'nginx' connect to address 127.0.0.1 and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:46:47] info : 'nginx' trying to restart
[CEST Sep 26 18:46:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:46:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:47:47] error : 'nginx' connect to address 127.0.0.1 and port 88: Connection refused
HTTP CRITICAL - Unable to open TCP socket
[CEST Sep 26 18:47:47] info : 'nginx' trying to restart
[CEST Sep 26 18:47:47] info : 'nginx' stop: /etc/init.d/nginx
[CEST Sep 26 18:47:47] info : 'nginx' start: /etc/init.d/nginx
[CEST Sep 26 18:48:26] info : Shutting down monit HTTP server
[CEST Sep 26 18:48:26] info : monit HTTP server stopped
[CEST Sep 26 18:48:26] info : monit daemon with pid [4187] killed
I've tested the bash script to see if detect when the service is up again and is working fine, so looks like Monit doesn't update the status or doesn't try to run the command again.
Someone knows what can be happening?, because I don't know what to do...
Thanks!!
Daniel Carrasco Marín
Posted 2016-09-29T14:03:37.900
Reputation: 31