Hiera lookup based on pattern

Question

I am trying to wrap my head around how I can transition from something I am doing in my large site.pp file into a structure I can use in hiera. It isn't clear to me from reading the puppet docs, exactly how the hiera data is evaluated and when it fits into the picture. I have recently upgraded from puppet 2.7.x to 3.3.x. This includes the hiera as part of standard package, so I wanted to finally look at using this since it should make my setup easier to read/understand.

I am using to support systems several external organizations. This includes configuring systems unique to each organization. At the top of my site.pp I have a structure like below. Which I use to set facts for each organization based on a regular expression that matches against the clientcert fact which are configured and issued in a way that they will reliably identify each organization.

# match organization
case $::clientcert {

  /.*example1.org/ :
      { $snmp_ro_community='...'
        $snmp_location='Example Org 1' 
        ... }
  /.*example2.org/ :
      { $snmp_ro_community='...'
        $snmp_location='Example Org 2' 
        ... }
  /.*example3.org/ :
      { $snmp_ro_community='...'
        $snmp_location='Example Org 3' 
        ... }
  /.*example4.org/ :
      { $snmp_ro_community='...'
        $snmp_location='Example Org 4' 
        ... }
}

I browsing through examples I don't see any way to do any kind of pattern matching in my hiera.yaml file. I suspect I must be missing something obvious though.

I do not want to rely on a custom fact for this. I prefer to stick with the client certificate, since I am certain this will correctly identify the organization and system and has been confirmed using strong cryptography. I don't want to give the one organizations values to another organization.

Since this is data from something the node already has (the FQDN and client cert CN), as opposed to something you're trying to set to the node, I'd say this is probably something that wouldn't really belong in Hiera - this bit of logic should coexist just fine in `site.pp` with a Hiera setup for your classes and class parameters, though? — Shane Madden, Oct 22 '13 at 06:13

Walid · Answer 1 · 2014-11-21T08:04:44.757

I was looking for the same thing for a similar reason. and have found that hiera backedn in github helpful jjulien/hiera-regex . As an example from the project itself readme file: you have to configure a new backend that deals with the grouping as in:

/etc/puppet/hiera.yaml:

:backends:
  - regex
  - yaml
:yaml:
  :datadir: /var/lib/hiera
:regex:
  :datadir: /var/lib/hiera
:hierarchy:
  - "fqdn/%{::fqdn}"
  - common

/var/lib/hiera/fqdn/fqdn.regex:

---
 - '^mailin-trusted.example.org$':
     postfix::smtp_relay: 'mailout-dmz.example.org'
 - '^mailout.*':
     postfix::smtp_relay: 'smtp.mailgun.org'
 - '^mailin.*':
     postfix::smtp_relay: 'localhost'

however i see several puppet-users posts and puppetlabs video are warning against using hiera for classification, especially now with the release of PE 3.7 with the node manager.

score 1 · Accepted Answer · answered Nov 11 '13 at 10:16

You can setup hierarchy of YAML files in order to this. Let's start with hiera.yaml:

---
:hierarchy:
    - "host/%{fqdn}"
    - "domain/%{domain}"
    - "env/%{::environment}"
    - "ops/%{operatingsystem}"
    - "os/%{osfamily}"
    - common
:backends:
    - yaml
:yaml:
    :datadir: /etc/puppet/data

For the folder structure you can use any fact, that you can see in output of facter -y. e.g. you can have hiera config files for each CPU architecture. Then you would add line

- "arch/%{::architecture}"

and hiera would look let's say into arch/amd64.yaml

To debug hiera you can dump your current facts:

   $ facter -y > myhost.yaml

And look for some variable:

   $ hiera -y myhost.yml snmp_location --debug

Hiera will go through all rules and try to find the variable:

DEBUG: Mon Nov 11 11:00:23 +0100 2013: Hiera YAML backend starting
DEBUG: Mon Nov 11 11:00:23 +0100 2013: Looking up snmp_location in YAML backend
DEBUG: Mon Nov 11 11:00:23 +0100 2013: Looking for data source host/myhost.example.com
...
DEBUG: Mon Nov 11 11:00:23 +0100 2013: Looking for data source ops/Ubuntu
DEBUG: Mon Nov 11 11:00:23 +0100 2013: Cannot find datafile /etc/puppet/data/ops/Ubuntu.yaml, skipping

For matching $::clientcert it might be good idea to extract the top domain to a separate fact and then just have yaml files for cert/example1.org.yaml which would contain something like this:

---
snmp_location: 'Example Org 1'

It's good to know, that even if you have module that doesn't contain any hiera function call at all you can easily setup parameter values:

class snmp (
  $location = 'foo',
) { 
# ...
}

some hiera config:

---
snmp::location: 'Example Org 1'

I guess I am not being clear enough in my question. My problem is that there is no fact provided the client that I can use, and I don't to create a custom fact ant trust the what the client provides for that fact. I need to do something based on a regular expression match against the client certificate which I know has been verify using strong cryptography. I want to know if I can use the contents of a variable I set in my site.pp within my hiera config. — Zoredache, Nov 12 '13 at 02:42
Hiera automatically receives all of Puppet’s current variables. This includes facts and built-in variables, as well as local variables from the current scope. There are many types of variables: http://docs.puppetlabs.com/puppet/latest/reference/lang_variables.html — Tombart, Nov 19 '13 at 23:34

Hiera lookup based on pattern

2 Answers2