2

I have a monit script that does something like this:

check process sidekiq_1 with pidfile /tmp/pids/sidekiq_1.pid
 start program = "/bin/bash -l -c 'bundle exec sidekiq start" as uid jim and gid jim with timeout 250 seconds
 stop program = "/bin/bash -l -c 'bundle exec sidekiq stop" as uid jim and gid jim with timeout 120 seconds
 if cpu usage > 25% for 18 cycles then restart
 if mem > 1500.0 MB for 18 cycles then restart

This is great, however I need to have the check made conditional based on the existence of a trigger file like so:

Only execute the check (start the process), if the file /tmp/do_not_start_sidekiq.txt is NOT present.

In this way i could do a touch /tmp/do_not_start_sidekiq.txt if I wanted to shut down the processes and not have monit starting them again, until I do a rm /tmp/do_not_start_sidekiq.txt

How would I do change this monit script to get that behavior?

ewwhite
  • 194,921
  • 91
  • 434
  • 799
Niels Kristian
  • 358
  • 2
  • 13

1 Answers1

3

The right way to handle this with monit is to "unmonitor" the process...

An example:

monit unmonitor sidekiq_1

Will not attempt to restart or report issue with the process.

You can restore monitoring of the check with:

monit monitor sidekiq_1

These can also be grouped or kicked off by cron. A good real-life deployment may have applications monitored during business hours and unmonitored during downtime windows, controlled by cron...

################################################################################
# Shutdown Cucumber
################################################################################
01  15 * * 1-5 monit unmonitor `/bin/hostname`
50  23 * * 0-5 monit -g servers stop all
51  23 * * 0-5 monit -g base  stop all
52  23 * * 0-5 monit stop all

Edit:

If you need an unprivileged user to be able to control this behavior, you can leverage /etc/sudoers entries for the monit unmonitor/monitor commands.

Something like: 

jim ALL=NOPASSWD: /usr/bin/monit unmonitor sidekiq_1

Would allow that specific command to be run by your unprivileged user, jim.

ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • I get it, and i thought of it, however I'm not so sure that it's the case here. Now monit is being run by a privileged user with root access. This is to have monit monitoring all kinds of low level stuff like memcached and so on. These services should the less privileged user `jim` not have access to. The user `jim` is a user that several people in my team have authentication to, that can only execute higher level stuff like, restart the application, stop/start the workers (sidekiq) etc. So determining if monit itself should monitor something or not is not my team members' responsibility. – Niels Kristian Jul 23 '14 at 12:56
  • @NielsKristian See my edit above. – ewwhite Jul 23 '14 at 13:05
  • Hmm okay thanks, at least that's a solution, however I don't really like having coupling to project specific behavior, through monit, in my `/etc/sudoers` file. – Niels Kristian Jul 23 '14 at 13:09
  • 2
    @NielsKristian Your other option is to run multiple Monit instances. The "Cucumber" example I gave above is from a firm that runs Monit as a non-root user so their developers can control the application, but they also have another system-level Monit that handles server monitoring. They *may* have customized things to make this work. – ewwhite Jul 23 '14 at 13:11