Monit can, according to the documentation generate a number of alerts:
Event: | Failure state: | Success state:
---------------------------------------------------------------------
action | "Action done" | "Action done"
checksum | "Checksum failed" | "Checksum succeeded"
bytein | "Download bytes exceeded" | "Download bytes ok"
byteout | "Upload bytes exceeded" | "Upload bytes ok"
connection | "Connection failed" | "Connection succeeded"
content | "Content failed", | "Content succeeded"
data | "Data access error" | "Data access succeeded"
exec | "Execution failed" | "Execution succeeded"
fsflags | "Filesystem flags failed" | "Filesystem flags succeeded"
gid | "GID failed" | "GID succeeded"
icmp | "Ping failed" | "Ping succeeded"
instance | "Monit instance changed" | "Monit instance changed not"
invalid | "Invalid type" | "Type succeeded"
link | "Link down" | "Link up"
nonexist | "Does not exist" | "Exists"
packetin | "Download packets exceeded" | "Download packets ok"
packetout | "Upload packets exceeded" | "Upload packets ok"
permission | "Permission failed" | "Permission succeeded"
pid | "PID failed" | "PID succeeded"
ppid | "PPID failed" | "PPID succeeded"
resource | "Resource limit matched" | "Resource limit succeeded"
saturation | "Saturation exceeded" | "Saturation ok"
size | "Size failed" | "Size succeeded"
speed | "Speed failed" | "Speed ok"
status | "Status failed" | "Status succeeded"
timeout | "Timeout" | "Timeout recovery"
timestamp | "Timestamp failed" | "Timestamp succeeded"
uid | "UID failed" | "UID succeeded"
uptime | "Uptime failed" | "Uptime succeeded"
We were able to fix this on our side by setting (addresses changed to protect the innocent):
SET ALERT important-messages@projectlocker.com ON { invalid, nonexist, timeout, resource, size, timestamp}
SET ALERT less-important-messages@projectlocker.com ON {action, permission, pid, ppid, instance, status}
This successfully routes the messages to the adresses we care about. You can set them globallly or locally, but our alerts are just global.
The subheadings under SERVICE TESTS at:
http://mmonit.com/monit/documentation/monit.html
correspond fairly neatly to the types above.
For each scheduled process or feature of your server, you should be able to come up with what matters to you in plain English, and match that desire to one of the tests mentioned in SERVICE TESTS. For example, if I'm running Apache, I know that I care about:
- Is the PID in the PID file still running? (nonexist)
- Did the PID change without my knowledge? (pid)
- Is the service responding in a timely fashion to a restart? (timeout)
For a custom daemon that polls, I may care about whether the log file is getting updated with status messages regularly (timestamp).