40

I have a web sever that connects to an internal database through a VPN. There are 2 IPs (primary and secondary) to the database from the web server. How can I setup my /etc/hosts file so that if the primary IP is not available then the secondary IP will be used?

Would this work for my hosts file?

141.131.286.1   abc.efg.datastore.com   #primary

141.131.286.237 abc.efg.datastore.com   #secondary
dmourati
  • 24,720
  • 2
  • 40
  • 69
Mike T
  • 413
  • 1
  • 4
  • 5
  • http://www.linuxmisc.com/4-linux/391d5fa7270c26bc.htm – dmourati Jan 30 '13 at 18:00
  • As an alternative and corret solution, install a local dns server. For example the easy-peasy to configure *dnsmasq* http://www.thekelleys.org.uk/dnsmasq/doc.html – Massimo May 26 '20 at 19:26

8 Answers8

31

The hosts file does not provide such mechanism. If you list two IPs for the same name, only the first one will be used. So, there is no such thing as primary and secondary IPs.

Also, the hosts file does not handle URLs. It just handles names like the ones provided in the question. A URL contains complete path and protocol such as http://host/path/to/resource.

fengqi
  • 103
  • 3
Khaled
  • 35,688
  • 8
  • 69
  • 98
  • Edited title and naming in OPs question to eliminate the confusion about URLs/hosts. – dmourati Jan 30 '13 at 17:56
  • 1
    This is not true, at least not blanketly. If the hosts file is used and has multiple entries, every resolver I have seen _returns_ all of them to the program, just as it does when a DNS server returns multiple A or AAAA records. What the _program_ does varies -- some use only the first entry, some do more complicated things that might or might not match what the user would like. – dave_thompson_085 Feb 06 '20 at 09:19
16

You can't provide resilience or round robin load balancing via the /etc/hosts file - it is not designed for that purpose.

Instead, your options are ... (in no particular order)

  1. Configure your network properly, so that routes change when a link is dropped
  2. Use DNS round-robin load balancing (not A Good Idea TM) using a managed service (eg. loaddns.com or dnsmadeeasy.com etc.)
  3. Use a local L3 load balancer for the outbound traffic (HAProxy?) with the back-ends defined as necessary
  4. Build the resilience into your web application itself
Ben Lessani
  • 5,174
  • 16
  • 37
  • DNS round-robin load balancing isn't usually resilient. One is selected and others are not tried. – Antti Rytsölä Sep 19 '12 at 17:09
  • Another option could be to use netcat or another software to forward the connection to an IP. Then change the forward if one IP is lost. – Antti Rytsölä Sep 19 '12 at 17:11
  • 1
    @anttiR DNS RR alone has no resilience, but used via a managed DNS service provider it does. I've edited my answers and given some examples to be clearer. – Ben Lessani Sep 19 '12 at 17:19
  • I doubt it would work great with database. They have the tendency to fetch one IP and stick to it. An internet website on the other hand would work great. – Antti Rytsölä Sep 19 '12 at 17:30
  • That would depend on the resolver of the host machine. If the DNS resolver is set to be a non-caching service - or polls the DNS registrars DB directly, then it would work. But like I said, ***its not a good idea***, its just *an idea*. – Ben Lessani Sep 19 '12 at 17:38
  • I think I blew up your network point because there are none in his real question. /etc/hosts maps IPs to hostnames, not URLs or "network paths" – dmourati Jan 30 '13 at 18:05
  • What I see tells me that he's got one database server with two addresses. No round robin needed. It looks like the single server is setup for failover (why put two IP addresses on the database otherwise?). So, two DNS entries weighted differently provides failover, no? BTW, I suspect that his DNS admin already has done this. Though one of those addresses is a .1 which worries me. – Jeter-work Dec 18 '18 at 16:12
  • .1 is typically used as a gateway. Is a load balancer installed as a gateway? Could his DNS admin have set up the database server behind a load balancer? If so, the .1 is probably the load balancer, and the .237 is the actual address of one of the database servers. Either way, best way forward is to use the fqdn of the database server and leave the IPs to the DNS guys. – Jeter-work Dec 18 '18 at 16:22
4

/etc/hosts doesn't support round robin but you can write a simple bash script to sed replace an entry tagged with a #RoundRobin comment (or any other tag you wish to use, just reflect it in the grep line in the script).

#!/bin/bash
fqdnips=( $(nslookup sub.domain.com|grep Address:|awk -F\  '{ print $2 }'|grep -v '#') )

new=`printf "${fqdnips[@]}"`
old=`grep "#RoundRobin" /etc/hosts|awk -F\  '{ print $1 }'`
sed -i "s/$old/$new/g" /etc/hosts

The above script grabs the output of nslookup for sub.domain.com and stores it in an array. It then prints the top most value to $new and grabs the existing value for tag #RoundRobin assigned in /etc/hosts ... lastly, it performs a sed replace

/etc/hosts file entry would look like this

127.0.0.1        localhost
::1              localhost
11.12.13.14      sub.domain.com      #RoundRobin

Lastly, place this script in the root's crontab to run every hour or so and you'll now have an /etc/host round-robin.

This is particularly useful if you have a coded page that is pulling some data from an API and the DNS lookup for the API server is causing a lot of hang time in the page's script execution... resulting in high cpu consumption for what would otherwise appear to be a simple page. To avoid the costly DNS lookup (particularly if your site is doing hundreds of them per minute do to heavy traffic), you should use /etc/hosts to resolve the FQDN of the remote API server. This will dramatically reduce the CPU usage for pulling the API data and generating the page.

Satalink
  • 178
  • 7
2

Note that at least on macOS, in contradiction to what the other answers say, the system resolver will return all entries associated with a host name in /etc/hosts instead of stopping at the first.

zneak
  • 318
  • 2
  • 14
  • 1
    I don't think the standards address a hosts file at all, but all the systems I have looked at -- Linux (with glibc), FreeBSD, Solaris _and_ Windows -- do this. Whether and how a program calling the resolver _uses_ multiple addresses is up to that program and varies wildly. – dave_thompson_085 Feb 06 '20 at 09:24
  • @dave_thompson_085, per other answers, that at least seems to be unusual, unless every platform changed how they handle this in the last 8 years and we’re just the first people to notice. – zneak Feb 06 '20 at 17:23
  • 1
    I doubt it's changed; I know I used this 'feature' (for testing, at a prior job) by around 2012 at the latest. And `gethostbyname` was designed to return multiple addresses from its creation in the 1980s. I think people were misled by the fact that many _programs_ use only the first result from the resolver -- _either_ from hosts file _or_ from DNS. (But DNS servers can easily vary the order on each request, and some do, so that the selected first entry changes, whereas doing that with hosts file is very clumsy.) – dave_thompson_085 Feb 10 '20 at 05:28
  • I verified that glibc on a 2021 debian system only supports one ip per host in /etc/hosts, and looking at changelogs and documentation, it is unlikely that this ever was different. What is the source for the claim that this ever was supported? The fact that gethostbyname can return multiple IPs has no relöevance on whether /etc/hosts supports this. – Remember Monica May 07 '21 at 16:04
  • The systemd-resolverd stub resolver on my Ubuntu 20 system appears to support returning more than one IP per host. Running `dig ` shows both IPs. – Gareth Dec 12 '21 at 21:09
2

Easy to setup, please follow the instruction:

  1. install dnsmasq
  2. edit /etc/resolv.conf and set "nameserver 127.0.0.1" as a first DNS
  3. add normal DNS as alternative (google one for example) "nameserver 8.8.8.8" as a second line
  4. make sure two required records are in your /etc/hosts file
  5. now check with a command host abc.efg.datastore.com

    that should respond two records with RR-DNS so if one node from the list is down - your application will be connected to another one
  • 1
    IP stack automatically checks hosts file before consulting DNS. I believe it is a part of the IP definition, but I know that Windows and Linux both follow this pattern. He does not need dnsmasq, and if he changes his /etc/resolv.conf in this way, he will lose connection to his real DNS server. – Jeter-work Dec 18 '18 at 16:06
  • not true, he can put multiple lines to resolv.conf so next line can be "nameserver 8.8.8.8", for example, + nscd for caching existing records, this approach exactly solve problem that was described – Ihor Kolodyuk Dec 21 '18 at 13:55
  • That's not what you told him to do. You told him to set his nameserver to loopback. You did NOT say add loopback as a nameserver. But if the host is not running named, or even if it is and it's not listening on loopback, then dns queries to that address will be unanswered. Simply adding the addresses to the hosts file will allow them to be resolved, but if they have the same hostname, only the first will ever be used and we're back at the beginning. The answer is in setting up a virtual host in DNS with weighted aliases, or in fronting the database with a load balancer. – Jeter-work Dec 21 '18 at 13:59
  • That is exactly I told to him, dnsmasq is acting as a nameserver proxy. This approach works. – Ihor Kolodyuk Dec 21 '18 at 14:23
  • 1
    I have tried Ihor's solution and the round robin works for records with the same name in the /etc/host file. In Ubuntu 18.04.3 "bionic" I had to disable systemctl-resolved and enable dnsmasq, and added my previous DNS servers to /etc/resolv.conf, and also the line "dns=dnsmasq" to /etc/NetworkManager/NetworkManager.conf, and restarted Networkmanager service. Round robin works great, so it's the best answer for the second question- Would this work for my hosts file?. +1 point and thanks for sharing. – Clon Jan 30 '20 at 22:34
  • 1
    @Xalorous: Ihor is right- round robin works well with dnsmasq. It does not with, for example, systemd-resolvd, which is the default service for the latest Ubuntu versions. – Clon Jan 30 '20 at 22:44
0

Easy way to make this happen would be to just use a public DNS service, like AWS Route53. You can enter multiple IP addresses per A record with priority

abc.efg.datastore.com

10 141.131.286.1  
20 141.131.286.237 

As long as no certificates are involved this is working and afaik not even against any norms or best practices.

NSLookup or other domain queries will return both addresses. Your app needs to be able to handle this. And yes, the domain needs to be a publicly registered one, not just a local hostname.

bortran
  • 109
  • 1
  • Browsers will be able to handle this, if 10 isn't available it will fallback on 20. It surely depends on the app and there should also be the point where you check for availability - if you want to get around setting up something more complicated and expensive with load balancing. – bortran Dec 18 '18 at 17:31
  • `A` records don't have a priority field. Some authoritative DNS services have record weighting, but that's done by probabilistically including or excluding the records. – Matt Nordhoff Jun 19 '19 at 18:02
-2

Yes multiple IPs in /etc/hosts works. Example, and test;

> echo "192.168.0.141 test
192.168.0.142 test
192.168.0.143 test" >> /etc/hosts

> dig test +short
192.168.0.141
192.168.0.142
192.168.0.143
user1133275
  • 195
  • 1
  • 11
  • I tried on one server, I confirm you: it doesn't work. Yours is a specific case, not valid as a generic answer, if you can't say how you solved it. – Massimo May 27 '20 at 21:37
  • @Massimo please provide a specific counter example if you are having trouble making this work. – user1133275 May 28 '20 at 17:08
  • dig does not use the hosts file at all - either those entries come from the dns server or this answetr was made up – Remember Monica Nov 06 '21 at 05:40
  • @RememberMonica The man page said "Unless it is told to query a specific name server, dig will try each of the servers listed in /etc/" . I just re-tested this and it works. Can you show a counter test/example? – user1133275 Nov 06 '21 at 20:46
  • It says "/etc/resolv.conf" not "/etc". Which entry in /etc/resolv.conf do you think corresponds to your hosts file? – Remember Monica Nov 08 '21 at 14:54
  • @RememberMonica I don't know the internals but I assume it's some systemd thing that is going on in Ubuntu 20.04. Just try it yourself already it only takes 2 seconds. – user1133275 Nov 09 '21 at 15:48
-2

Yes, it would work.

However, the search mechanism simply goes does the list until it finds a match.

So while the answer to the question as written is YES, it is going to be a challenge. But nothing insurmountable.

Try this: Each of those IP addresses really do need to have differing names.

SDsolar
  • 157
  • 1
  • 1
  • 11
  • 2
    No, it won't work. Second occurrence of a value in /etc/hosts would never be used. And if he renames the second IP address with a different host name, now he has to do something in his app to do load balancing. That's going to be expensive and a kludge, when DNS or DNS + Load Balancer is the right answer. – Jeter-work Dec 18 '18 at 16:24