I am writing chef scripts to setup an etcd cluster. The list of nodes (including their IPs) will be hardcoded in the chef script (as an attribute) and it will pass them to etcd using static bootstrapping with --initial-cluster and --initial-cluster-state new parameters. So the chef script calls etcd with these parameters, they will setup the cluster and in future runs (e.g. reboots) ignore these --initial-cluster parameters.
For first time setup, this works well. Now assume I want to later add a new node to the etcd cluster. I first add another node to the hardcoded list in the chef attribute and bootstrap the new node. When the script then calls etcd, the --initial-cluster parameter will also contain the additional node, but etcd requires it to be called with --initial-cluster-state existing instead of --initial-cluster-state new. So I need a separate chef script for adding new nodes than for starting the first few nodes. This doesn't seem a good solution to me.
A first idea for a solution is to have case distinction in the chef script. We keep a separate hardcoded list of the IPs of the initial nodes. If the bootstrapped node belongs to the initial set, it will use --initial-cluster-state new and if it doesn't, it will use --initial-cluster-state existing. However, this doesn't work when one of the initial nodes dies. When deleting and re-bootstrapping it, it will get the same IP as before (which qualifies it as an initial node) and will try to connect to the cluster using --initial-cluster-state new.
If there was some kind of automatic recognition in etcd that would choose the --initial-cluster-state parameter automatically, that would be great. But as far as I know, etcd doesn't offer this.
What is the recommended way to setup an etcd cluster with chef?
