HAproxy: Run script on health check change

Question

I have set up a haproxy configuration with a backend with two servers looking like that:

...
default
        option log-health-checks
...
mailers mta
        mailer smtp1 127.0.0.1:25
...
backend s_api
        balance roundrobin
        option httpchk GET /sites?site=q&limit=1

        http-check expect rstatus (2|3)[0-9][0-9]

        server s1_a 1.2.3.4:3600 check inter 5s fall 4 rise 1
        server s2_b 1.2.3.5:3600 check backup

        timeout queue 60s
        timeout server 60s
        timeout connect 60s

        email-alert mailers mta
        email-alert level notice
        email-alert from haproxy@example.com
        email-alert to ops@example.com

I now get noticed via email when a server gets down and up again. What I would like to archive is to start a local bash script when a server is going up and another one when the server is going down.

How could that be done?

score 5 · Accepted Answer · answered Nov 22 '17 at 13:07

I would probably do this with an external check script and use this to control the rise and fall values. Then when you meet the rise or fall value you can run your own script. Set the rise and fall in your haproxy config to be 1 and change the check to external

    external-check command ping.sh
    server s1_a 1.2.3.4:3600 check inter 5s fall 1 rise 1
    server s2_b 1.2.3.5:3600 check backup

    timeout queue 60s
    timeout server 60s
    timeout connect 60s

Then as a rough example with an external ping check

#!/bin/bash

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

# Set the maximum time in seconds to allow ping to run, 1-10 is a sensible range.
TMEOUT=1
# Interval between pings in seconds, 0.2-1 is a sensible range.
INT=1
# Number of pings to send.
NUM=1
RIP=$(echo ${3}))
VIP=$(echo ${1}) 
#Optionally provides source IP for Ping
SRC=

if [ "$SRC" != '' ]; then
        SRC_IP="-I $SRC"
fi
if ping ${SRC_IP} -n -w${TMEOUT} -i${INT} -c${NUM} ${RIP} >& /dev/null; then
            COUNTER="$(cat count)"
            if [COUNTER != 0]; then
                //run up script here
            fi
            echo 0 >> count         
            exit ${?}
else
            COUNTER="$(cat count)"
            if [COUNTER >= 3]; then
                //run down script here
            fi

            COUNTER=$[COUNTER + 1]
            echo COUNTER >> count
            exit ${?}
fi

You may also be able to do this with lua but I've not looked at that. I've done a couple of other options for email alerts like polling the stats socket or hacking haproxy which could be adapted but the external check will probably work best here. However if you wish to try them out you can find them here https://www.loadbalancer.org/blog/3-ways-to-send-haproxy-health-check-email-alerts/ .

Thanks for your suggestion. It's not quite the thing I was looking for but my current state of research brought nothing else up. Especially no native haproxy way of doing this. So I will accept your answer. — mr.simonski, Dec 07 '17 at 21:58

HAproxy: Run script on health check change

1 Answers1