1

Is it possible to create different core os clusters all linked up to 1 etcd cluster ? If yes, how ?

I have a scenario where I need to create multiple clusters but I want to keep one etcd cluster. Currently I end up having all instances in the same cluster even etcd it self. How do I make sure there are separate clusters ?

I'm following the approach in which etcd2 service in each instance has a dropin environment file (in cloud config) which includes details about initial cluster

Hazim
  • 113
  • 7

1 Answers1

1

The cluster in CoreOS is managed by fleet, and if you want to see how, look under the 'hidden' etcd key /_coreos.com/fleet

$ etcdctl ls /_coreos.com/fleet
/_coreos.com/fleet/machines
/_coreos.com/fleet/engine
/_coreos.com/fleet/lease
/_coreos.com/fleet/unit
/_coreos.com/fleet/job
/_coreos.com/fleet/state
/_coreos.com/fleet/states

If you could somehow change the key prefix used by fleet, then you should be able to have separate clusters using the same etcd.

The cloud-config section for fleet does support this via the etcd_key_prefix setting. By default this is /_coreos.com/fleet/ so set this to a similar sort of value with a leading and trailing slash and you should be good to go.

I've not tried this, but I'd be interested to hear whether this works for you. If the cloud-config setting works, then each machine should have a configuration in something like /run/systemd/system/fleet.service.d/20-cloudinit.conf which includes a FLEET_ETCD_KEY_PREFIX environment variable. Then, you should be able to use etcdctl to view the key and verify fleetctl operations are working.

Paul Dixon
  • 1,436
  • 3
  • 21
  • 35
  • 1
    Since you asked whether it worked or not, it works perfectly! Thanks a lot for the explanation. – Hazim Aug 01 '16 at 08:03
  • I'm having an issue with this configuration, when I submit a unit to my cluster using fleet and private IP of an instance in the cluster as endpoint, the unit ends up getting submitted to the etcd cluster instead of the cluster whose IP I provided, any clue why? – Hazim Aug 01 '16 at 12:02
  • Just figured it out, had to add "--etcd-key-prefix=/my/prefix/" argument while using `fleetctl` – Hazim Aug 01 '16 at 12:05