1

I want to make a ubuntu server image to run in a public cloud that discovers when other servers from the same image come online. I came up with this list of solutions, but I'm not sure what is best?

Possible solutions:

  • Layer-2 multicasts with Link Layer Discovery Protocol (LLDP) much like CDP
  • Assign a link-local address to self then send a broadcast
  • Avahi (a zeroconf implementation on linux, much like Bonjour by Apple)
  • Use a DHCP server track servers

More possible solutions: (servers must have an IP address already)

  • SNMP broadcast/multicast
  • SSDP multicast messages (Simple Service Discovery Protocol) (used in UPnP)
  • I'm not sure but maybe m(ulticast)DNS or DNS-SD (Service Discovery)

Are there any more ways to discover nodes? What is commonly done in large cloud clusters?

dan
  • 131
  • 3
  • 1
    I'd be wary of using layer 2 mechanisms because, inevitably, you'll need to have servers in different subnets and cross-subnet discovery. – Evan Anderson Oct 08 '14 at 20:41
  • Cross-subnet discovery is even more challenging. Do you know any cloud providers that have multicast or broadcast support? – dan Oct 08 '14 at 20:52
  • The "best" answer will depend on _why_ you are trying to do this, which unfortunately you neglected to mention. – Michael Hampton Oct 09 '14 at 21:13

2 Answers2

6

The public cloud providers generally don't support broadcast/multicast methods, so any solution that relies on those will not be 'best'. At least for values of 'best' that include significant components of platform independence.

In unicast-land, you have a bunch of options and a lot depends on just what it is you want to do with those images and what service is actually doing the discovery.

If you're entirely within a single cloud infrastructure provider, the common way is to ask the infrastructure about what's there. You'll get a list of servers, with IP address, that you can then use to build your dynamic cluster.

If you can't do that for some reason, maybe you're on multiple providers or you need a list of known-alive hosts, you're going to have to go to some kind of application-specific method of building a dynamic cluster. The methods for this are various, but I've seen:

  • Provide a few hints for nodes to check when they come online, and the application maintains a list of checkins and does heartbeating to be sure that only live nodes are in the list. Think of it like bit-torrent seeders.
  • Service registrations with an actual Service Registration service, of which you've listed a few.
  • Service registrations with self-built registration service, such as something cobbled out of Redis.

What's best for you depends on what you're doing. But go for the API lookups first, and only get fancy if those don't work for you.

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
3

As I understand it, the proper way of doing this is by using the API that your cloud provider offers to you. Trying to do this from the hosts is a little bit "stone-axe" territory. Use the nice space-age infrastructure that you're paying for.

mfinni
  • 35,711
  • 3
  • 50
  • 86