I am using Monit 5.5 with the CHECK PROGRAM
directive to execute an external script that does some work to verify if my app is functioning properly. I want to do this check every few cycles so as to avoid being a burden on the app. My config is this:
CHECK program mydaemon with path "/usr/local/sbin/my_check.sh"
ALERT monit@mycompany.pagerduty.com ON { exec }
START PROGRAM "/etc/init.d/mydaemon start"
STOP PROGRAM "/etc/init.d/mydaemon stop"
if status = 1 for 2 cycles then restart
# Trick monit into doing a restart + hitting our local alert
if status = 1 for 4 cycles then exec "/bin/true"
if status = 1 for 6 cycles then unmonitor
every 3 cycles
Things work almost as expected - every 3 cycles monit performs the check or takes an action, though as you might guess by the comment in the config, what I am seeing in my logs is that on cycle 3, 4, and 5 monit will also perform the restart action:
May 24 14:03:24 monit[19488]: 'mydaemon' status failed (1) for /usr/local/sbin/my_check.sh -- Error: testing!
May 24 14:03:54 monit[19488]: 'mydaemon' status failed (1) for /usr/local/sbin/my_check.sh -- Error: testing!
May 24 14:03:54 monit[19488]: 'mydaemon' trying to restart
May 24 14:03:54 monit[19488]: 'mydaemon' stop: /etc/init.d/mydaemon
May 24 14:03:54 monit[19488]: 'mydaemon' start: /etc/init.d/mydaemon
May 24 14:04:24 monit[19488]: 'mydaemon' status failed (1) for /usr/local/sbin/my_check.sh -- Error: testing!
May 24 14:04:24 monit[19488]: 'mydaemon' trying to restart
May 24 14:04:24 monit[19488]: 'mydaemon' stop: /etc/init.d/mydaemon
May 24 14:04:24 monit[19488]: 'mydaemon' start: /etc/init.d/mydaemon
May 24 14:04:54 monit[19488]: 'mydaemon' status failed (1) for /usr/local/sbin/my_check.sh -- Error: testing!
May 24 14:04:54 monit[19488]: 'mydaemon' status failed (1) for /usr/local/sbin/my_check.sh -- Error: testing!
May 24 14:04:54 monit[19488]: 'mydaemon' exec: /bin/true
May 24 14:04:54 monit[19488]: 'mydaemon' status failed (1) for /usr/local/sbin/my_check.sh -- Error: testing!
May 24 14:04:54 monit[19488]: 'mydaemon' trying to restart
May 24 14:04:54 monit[19488]: 'mydaemon' stop: /etc/init.d/mydaemon
May 24 14:04:54 monit[19488]: 'mydaemon' start: /etc/init.d/mydaemon
May 24 14:05:25 monit[19488]: 'mydaemon' status failed (1) for /usr/local/sbin/my_check.sh -- Error: testing!
May 24 14:05:25 monit[19488]: 'mydaemon' status failed (1) for /usr/local/sbin/my_check.sh -- Error: testing!
May 24 14:05:25 monit[19488]: 'mydaemon' exec: /bin/true
May 24 14:05:25 monit[19488]: 'mydaemon' status failed (1) for /usr/local/sbin/my_check.sh -- Error: testing!
May 24 14:05:25 monit[19488]: 'mydaemon' trying to restart
May 24 14:05:25 monit[19488]: 'mydaemon' stop: /etc/init.d/mydaemon
May 24 14:05:25 monit[19488]: 'mydaemon' start: /etc/init.d/mydaemon
Why is monit performing my restart
action on unspecified cycles?
PS - My monit cycle length is 10s, hence the actions in the log snippit are 30s apart.