As this is an old answer I'm not sure if you are still looking for an answer. But I stumbled upon it looking for the best way on how to do this.
The way Hetzner assigns a failover IP to a dedicated server is not to allow it to be configurred on the server but to route the traffic to the original server IP. Therefore it is possible to not change anything on your server and manually switch the IP in their admin interface. However; this is not a suitable solution for most as I would not like to get out of bed to manually fail over. This should be done automatically and then notify the admin that the failover has been done. Maybe even with a small report which issues the system has seen and why it did fail over.
Keepalived can do this for you, the only thing required is to configure keepalived to run the script while failing over. But if there is no IP to failover, how can we then failover?
Simple; create an internal network between the servers and assign your own non-routed internal IP to keepalived. As this internal network uses the same interface as the external network it does not really matter. A benefit of this approach is that you can keep all internal traffic 100% internal by using this internal VIP.
Once Keepalived fails over you order it to run the script from Hetzner to also switch the external ip using:
notify
An example:
vrrp_script chk_haproxy {
script "killall -0 haproxy" # cheaper than pidof
interval 2 # check every 2 seconds
weight 2 # add 2 points of prio if OK
}
vrrp_instance VI_1 {
state MASTER
interface enp0s31f6.4000
virtual_router_id 51
priority 101
virtual_ipaddress {
192.168.100.3/24 # this is the shared IP I was using
}
track_script {
chk_haproxy
}
notify /usr/sbin/hetzner_failoverIP.sh database set $THIS_SERVER_IP
}
Ofcourse the Hetzner script can be adjusted to be much smarter, selecting the server IP by itself for example.
The downside that should be noted is that the external IP will take between 40 and 60 seconds. For me, a minimum of 40 seconds and a maximum of 1 minute is too long.
Another option is to use the Hetzner cloud instances to enable HA without the use of the failover IP and the script above. In the cloud there is another solution: Cloud floating IP.
This option will set you back for about €8,50 euro's per month for:
- two cloud instances (1 basic cpu, 2GB memory and 20TB traffic each)
- two cloud floating IP's
Then use keepalived to manage the cloud floating IP (virtual_ipaddress section) and HAProxy to send all traffic to the dedicated servers. HAProxy will then do the healthchecks and you don't need to worry about:
- Switching IP's using the Hetzner API
- 40 to 60 seconds of additional downtime
It's is worth to mention that Hetzner cloud servers do not support an internal network. But it is not required if you use them in this way and it will not cost you extra as internal traffic is free of charge. For security, secure the load balancers (Keepalived+HAProxy cloud instances) with SELinux/AppArmor and Firewalld. Use encrypted traffic between the two clusters (cloud <-> Dedicated) to prefent packet sniffing. I would also encrypt the traffic between your dedicated servers even if you are using a private VLAN, the traffic is still being send out via the same NIC. Something to keep in mind.
Sources used:
- https://wiki.hetzner.de/index.php/Failover/en
- https://wiki.hetzner.de/index.php/Failover_Skript/en
- https://wiki.hetzner.de/index.php/Vswitch/en
- https://wiki.hetzner.de/index.php/Cloud_floating_IP_persistent/en
- https://www.hetzner.com/cloud
- https://twitter.com/hetzner_online/status/955781300513857536