Satellite 6's puppet is unable to find the hiera data sources

Question

Using Satellite 6 which comes with Foreman 1.6.0.53.

By default, Puppetlabs' documentation states that hiera configuration should be in $config/hiera.yaml.

# puppet config print confdir hiera_config
confdir = /etc/puppet
hiera_config = /etc/puppet/hiera.yaml

Looking at our hiera config:

# cat /etc/puppet/hiera.yaml
---
:backends: yaml
:yaml:
  :datadir: /var/lib/hiera
:hierarchy:
  - users
  - groups
  - global

The data file exists:

# cat /var/lib/hiera/users.yaml
---
users:
  bfernandez:
    uid: 300
    fullname: Belmin Fernandez

And, to test it, I use hiera's CLI and a puppet apply:

# hiera --conf=/etc/puppet/hiera.yaml --debug -h users
DEBUG: 2015-05-06 14:11:37 -0400: Hiera YAML backend starting
DEBUG: 2015-05-06 14:11:37 -0400: Looking up users in YAML backend
DEBUG: 2015-05-06 14:11:37 -0400: Looking for data source users
DEBUG: 2015-05-06 14:11:37 -0400: Found users in users
DEBUG: 2015-05-06 14:11:37 -0400: Looking for data source groups
DEBUG: 2015-05-06 14:11:37 -0400: Looking for data source global
{"bfernandez"=>{"uid"=>300, "fullname"=>"Belmin Fernandez"}}

# puppet apply -e '$foo = hiera_hash(users) notify { $foo: }'
Notice: Compiled catalog for foosat.example.com in environment production in 0.08 seconds
Notice: {"bfernandez"=>{"uid"=>300, "fullname"=>"Belmin Fernandez"}}
Notice: /Stage[main]/Main/Notify[{"bfernandez"=>{"uid"=>300, "fullname"=>"Belmin Fernandez"}}]/message: defined 'message' as '{"bfernandez"=>{"uid"=>300, "fullname"=>"Belmin Fernandez"}}'
Notice: Finished catalog run in 0.30 seconds

So far, all looks good. However, when I reference hiera_hash('users') in a module and apply it to a node, I get this error:

May  6 13:49:04 foo1 puppet-agent[8688]: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find data item users in any Hiera data file and no default supplied at /etc/puppet/modules/accounts/manifests/init.pp:10 on node foo1.example.com

Any ideas of what I should look at? Feel like I'm missing something on the Foreman side perhaps.

Update 1:

Per @lsd, tried using /etc/hiera.yaml for the config instead by creating a symlink. Tested it out with hiera CLI to confirm the config:

# hiera --conf=/etc/hiera.yaml --debug -h users
DEBUG: 2015-05-06 14:31:13 -0400: Hiera YAML backend starting
DEBUG: 2015-05-06 14:31:13 -0400: Looking up users in YAML backend
DEBUG: 2015-05-06 14:31:13 -0400: Looking for data source defaults
DEBUG: 2015-05-06 14:31:13 -0400: Cannot find datafile /var/lib/hiera/defaults.yaml, skipping
DEBUG: 2015-05-06 14:31:13 -0400: Looking for data source users
DEBUG: 2015-05-06 14:31:13 -0400: Found users in users
DEBUG: 2015-05-06 14:31:13 -0400: Looking for data source groups
DEBUG: 2015-05-06 14:31:13 -0400: Looking for data source global
{"bfernandez"=>{"uid"=>300, "fullname"=>"Belmin Fernandez"}}

Still got the error on the agent however so that did not address the issue.

It is entirely possible that it is trying to user /etc/hiera.yaml instead of /etc/puppet/hiera.yaml. Everything I've seen has recommended linking /etc/hiera.yaml to /etc/puppet/hiera.yaml. I'm not saying that is the problem, but you could try that as a first step. — lsd, May 06 '15 at 18:21
@lsd Thanks for the suggestion, still getting the error though. — Belmin Fernandez, May 06 '15 at 18:35

score 3 · Accepted Answer · answered May 07 '15 at 13:12

After thinking about it on my commute, decided to check out SELinux and there it was:

[root@foosat hiera]# grep yaml /var/log/audit/audit.log | head -n1
type=AVC msg=audit(1430926955.728:75727): avc:  denied  { getattr } for  pid=17099 comm="ruby" path="/var/lib/hiera/users.yaml" dev="dm-2" ino=25185161 scontext=system_u:system_r:passenger_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file

Changed the file contexts on the hiera file to puppet_etc_t (if someone knows of something more appropriate, please comment):

[root@foosat hiera]# semanage fcontext -a -s system_u -t puppet_etc_t "/var/lib/hiera(/.*)?"
[root@foosat hiera]# restorecon -R -v .
restorecon reset /var/lib/hiera/users.yaml context unconfined_u:object_r:var_lib_t:s0->unconfined_u:object_r:puppet_etc_t:s0

Working now. Hope this helps someone else.

Ah, that makes sense. I have my hieradata in /etc/puppet/hieradata, which has a default context that hiera can read, but of course /var/lib doesn't. — lsd, May 07 '15 at 14:27
Sorry to post on an old question, but how are you syncing your hieradata? Is it set up as a repo under Products? Or are you just manually copying it to Satellite/capsules? — shearn89, May 14 '16 at 13:58

Satellite 6's puppet is unable to find the hiera data sources

Update 1:

1 Answers1