Cross-reference Ansible variables within the same mapping

Question

How can I define an Ansible variable whose value is another variable in the same mapping structure?

To allow sensible namespacing of variables, I am defining mapping structures like this, where some values depend on other variables in the same structure:

acme:
  directory:
    hostname: "acme-staging-v02.api.letsencrypt.org"
letsencrypt:
  config_dir: "/etc/letsencrypt"
  keys_dir: "{{ letsencrypt.config_dir }}/keys"
  csrs_dir: "{{ letsencrypt.config_dir }}/csr"
  certs_dir: "{{ letsencrypt.config_dir }}/certs"
  accounts_dir: "{{ letsencrypt.config_dir }}/accounts"
  csr_file: "{{ letsencrypt.csrs_dir }}/{{ site_domain }}.csr"
  account_key_file: "{{ letsencrypt.csrs_dir }}/{{ acme.directory.hostname }}"
  email_address: "certificate-reminders@{{ site_domain }}"

This fails because Ansible can't resolve the values which reference others within the same data structure:

recursive loop detected in template string: {{ letsencrypt.config_dir }}/keys

So I thought the lookup vars would allow deferring that resolution:

acme:
  directory:
    hostname: "acme-staging-v02.api.letsencrypt.org"
letsencrypt:
  config_dir: "/etc/letsencrypt"
  keys_dir: "{{ lookup('vars', 'letsencrypt.config_dir') }}/keys"
  csrs_dir: "{{ lookup('vars', 'letsencrypt.config_dir') }}/csr"
  certs_dir: "{{ lookup('vars', 'letsencrypt.config_dir') }}/certs"
  accounts_dir: "{{ lookup('vars', 'letsencrypt.config_dir') }}/accounts"
  csr_file: "{{ lookup('vars', 'letsencrypt.csrs_dir') }}/{{ site_domain }}.csr"
  account_key_file: >-
    {{ lookup('vars', 'letsencrypt.csrs_dir') }}/{{ acme.directory.hostname }}
  email_address: "certificate-reminders@{{ site_domain }}"

This fails, because Ansible is attempting to resolve that lookup immediately:

No variable found with this name: letsencrypt.config_dir

Of course I could split them out so they're separate variables. That defeats my purpose, though, of keeping the strongly related variables all grouped in the same namespace.

So what will allow me to define the data structure so that some values can depend on other variables in the same structure?

It's a long standing issue in Ansible, see [issue #8603](https://github.com/ansible/ansible/issues/8603) for the extended discussion. — Michael Hampton, Aug 22 '21 at 23:38

score 0 · Accepted Answer · answered Aug 23 '21 at 05:43

(Thanks to @michael-hampton for leading to this answer.)

As described in Ansible issue#8603, the configuration parser is reading variable values and immediately attempting to render templates it encounters while defining the variables. This causes the parsing to fail when a template references a variable not yet completely defined.

A comment by ‘rquelibari’ gives a good analysis:

This feature is actually well-defined in terms of meaning exactly one thing. The syntax and semantics (in terms of logic) are both well defined for this kind of "back reference". […]

and explains in detail how this happens.

A subsequent comment by ‘cmpunches’ directly states the solution needed:

This issue is a direct result of ansibles use of a yaml parser's interpolation feature instead of loading the yaml object and doing a second pass for interpolation. This is not just a jinja bug this is an implementation bug in ansible. Please inspect. Loading as raw string and then processing the initialised object members in a second pass should fix this.

So, until the YAML parser in Ansible is corrected to read variable values as plain text without attempting to immediately render templates (and so postpone rendering until all variables are defined), this cross-reference in values can't yet be done in Ansible variables.

This is a misleading explanation of what is happening, because all variable evaluation happens lazily (on use), not during parsing. Variables are not able to refer to themselves because a variable has to be fully rendered before any part of it can be accessed. Changing this would involve difficult-to-reason-about changes to variable handling, and be of questionable value. A better solution is to namespace vars with prefixes (`letsencrypt_config_dir` etc.) instead of trying to pass everything as a single var. — flowerysong, Aug 23 '21 at 22:15

Cross-reference Ansible variables within the same mapping

1 Answers1