Using Puppet to incrementally add lines to a file, from multiple classes

Question

I am trying to use Puppet to automatically configure Splunk monitoring. This involves adding a list of file paths to a Splunk configuration file (inputs.conf).

Each role (webserver, db, etc.) in our application has its own Puppet module, which includes some base modules common to all roles.

The monitored files will vary between roles - for webservers we want /var/log/nginx/error.log to be monitored. For db servers, /var/log/postgresql/postgresql.log. There are some files (/var/log/syslog) that should be monitored for every role.

I populate the inputs.conf file using an EBR template, e.g.:

# $files is actually a parameter to the logging class
$files = ['/var/log/syslog', '/var/log/nginx/error.log']
file {"/opt/splunkforwarder/etc/apps/search/local/inputs.conf":
    ensure  => "present",
    content => template("logging/splunk_inputs.conf.erb"),
    require => Package["splunkforwarder"],
}

# splunk_inputs.conf.erb:
<% files.each do |file| %>
    [monitor://<%= file %>]
    disabled = false  
<% end -%>

Ideally, I want to include a "logging" module in each of these classes and incrementally build up a list of files that should be monitored. This list should be combined, and the $files variable used in the template.

Something like this:

class base {
    class {'logging':
        files => ['fileA', 'fileB']
    }
}

class webserver {
    require base

    class {'logging':
        files => ['file1', 'file2']
    }
}

… would result in the following files being monitored: fileA, fileB, file1, file2.

Splunk will need to be restarted once the final line has been added. I have an exec that does this, but am not sure of the 'best' time to run it. I would preferably like to avoid restarting Splunk after every additional line.

I am having difficulty designing a clean module layout that supports this. I've tried a few different layouts, but unfortunately without success so far. Virtual resources might be a good solution, but I don't think it is possible to have a "virtual variable" that is realized just before it is used in the File's template. Augeas also does not seem like the right solution, although perhaps I am missing something here.

Does anyone have any suggestions as to how I could achieve this?

score 4 · Accepted Answer · answered Dec 06 '12 at 13:34

4

This should do the trick for you

https://github.com/ripienaar/puppet-concat

answered Dec 06 '12 at 13:34

bart613

548
1
6
15

score 3 · Answer 2 · answered Dec 06 '12 at 13:28

3

you can try to look into using the concat module to build your config file from fragments declared in different classes

answered Dec 06 '12 at 13:28

Daniel

1,703
1
12
16

score 2 · Answer 3 · answered Dec 06 '12 at 18:38

2

Depending on what exactly you want, the file_line resource in puppet-stdlib may be of use to you:

answered Dec 06 '12 at 18:38

Not Now

3,532
17
18

score 2 · Answer 4 · answered Dec 06 '12 at 18:59

2

You can do this in two ways. using puppet concat, or using exported resources. Look at the tutorials of automatically adding nagios checks using exported resources. Something similar should work for splunk.

answered Dec 06 '12 at 18:59

Sirex

5,447
2
32
54

Using Puppet to incrementally add lines to a file, from multiple classes

4 Answers4