Output additional information if nagios/icinga check fails

Question

We use check_postgres to monitor our PostgreSQL servers.

Sometimes there are peaks in the number of locks and the check alerts.

Up to now the check only tells us the number of locks. This does not help in debugging where the locks come from.

I would like to see additional output. For example this output:

select * from pg_stat_activity order by datname

The output of the above statement is long and AFAIK check scripts must only output one line.

How to output additional information if nagios/icinga check fails?

Answer 1

You can use an Event Handler to do this.

The event handler would get called on any state change, and you write some logic to run that additional select when it's in CRITICAL state (e.g., via ssh with keys).

This also allows you to do whatever you want with the output, in case it is long. You could write it to a file, send it as email, whatever.