Indicating multiple clusters with ceph-ansible

Question

I am working through a Ceph course right now (ceph learning path from packtpub). The course is ok, but has a lot of errors, and isn't always the most accurate. The course expects that you use ceph-ansible to do a lot of the work. You start by creating a 3 node cluster.

So, the starting value of /etc/ansible/hosts is:

[mons]
ceph-node1
ceph-node2
ceph-node3
[osds]
ceph-node1
ceph-node2
ceph-node3
[clients]
client-node1

Now, the current section has you create a second 3-node cluster to serve as a remote backup.

To do this, the course has you modify the /etc/ansible/hosts file to the following:

#Primary site (ceph-node1):
[mons]
ceph-node1
ceph-node2
ceph-node3
[osds]
ceph-node1
ceph-node2
ceph-node3
[clients]
client-node1

#Secondary site (ceph-node5):
[mons]
ceph-node5
ceph-node6
ceph-node7
[osds]
ceph-node5
ceph-node6
ceph-node7

The installation of ceph-ansible is done via ceph-node5 with a different all.yml file, indicating a different cluster name. What I am trying to understand is how this would indicate to ceph-ansible that there are two separate clusters. If the /etc/ansible/hosts file is in .ini format, which ansible hostfiles are, any lines that begin with a # should be parsed as comments.

So, does ceph-node5 see the other installations on nodes 1,2,3 and ignore them as a separate cluster? After all, ceph-node5 has it's own all.yml with instruction that the hosts should be part of the backup cluster. Will it try to add 1,2,3 as nodes?

The text I am working with is really sparse on details, and trying to dig into the ceph-ansible github page hasn't been especially helpful.

I'm just trying to understand the mechanics of how this works. Any help would be appreciated, thank you!

Answer 1

Duplicate groups names in one inventory does not make sense. Further, you can store inventory files in different places.

Consider this YAML format inventory/siteA.yml:

---
all:
  vars:
    datacenter: a
  children:
    cephnodes:
      hosts:
        ceph-node1:
        ceph-node2:
        ceph-node3:
    osds:
      children:
        cephnodes
    mons:
      children:
        cephnodes

    clients:
      children:
        client-node1:

Because each node has an osd and a mon, I did the DRY thing and made that reference one list via child groups.

inventory/siteB.yml would naturally be the same thing with 3 different nodes.

If you wish to target multiple inventories for one play, you can specify a directory of them as inventory: ansible-playbook -i inventory/. osds will then target 6 nodes. However, if you need to target the A and B site separately in one play, you may want to restructure the groups a bit to distinguish those.