10

Notice: there is a lot of theoretical questions.

Recently I'm reading about Puppet (and similar systems), which - as I believe - can make my work easier, a lot. But I try - and unfortunately can't - to understand what all I can "puppetize". I can imagine "clouds" or HA clusters, where is the same config on more servers. But what about workstations? I have one pc (centos with kvm), one notebook (fedora) and personal server, can (or should) it be puppetized? What are (dis)advantages? Or in our company we have hundreds of servers (mainly with centos), but each of them is a little bit different. Can't decide if it's better to have a lot of configs on one place.. (Dis)advantages? I will be happy for all your opinions or links with this topic.

stderr
  • 871
  • 6
  • 15
  • I would advise not trying to "Puppetize" any of your Windows systems. Oh, and [reading our FAQ](http://serverfault.com/faq) as to what kind of questions you should ask here. – HopelessN00b Oct 24 '12 at 20:53
  • 7
    The amount of things you puppet, and how much puppet stuff they have should be directly proportional to the amount you care about the task being done for you. Start small, just puppet the ntp configuration, or rsyslog. Then build from there if and as you need to, when you need to. – Sirex Oct 24 '12 at 21:01
  • 1
    Since you already have lots of servers in my place, my suggestion is that you start simple with the bits that are common on every single system, then start going into more specific detailed things as you have time. – Zoredache Oct 24 '12 at 21:14
  • 1
    Windows support has dramatically improved in recent versions of Puppet. I manage hundreds of Windows nodes with Puppet. Puppet on POSIX nodes is much more straightforward and powerful, but using Puppet for at least some things on Windows can be incredibly useful. – czervik Oct 25 '12 at 01:07

4 Answers4

16

The degree to which you can puppetize an entire environment is dependent upon several variables:

  • The willingness of the automation staff to write automation for every. little. thing.
  • The cultural conditioning that allows "I'll just change this one thing, it's a one-off anyway" to turn into "I'll just change this one thing in this puppet manifest, and apply it now; it's just a one-off."
  • The degree of heterogenaity in an environment.

It is definitely possible to puppetize every ********** thing that can be puppetized, but getting there requires the right culture and buy-in from everyone that can touch a puppet-able device. Some devices are fundamentally hard to manage that way, things like workstations, and puppet is better as a staging tool than a configuration management engine.

Puppet is awesome when you're managing a fleet of VMs all doing largely the same thing. Total win, and not a lot of effort to get there.

On the other end of the spectrum you have what I had at my last job, which was 200+ servers providing 130 services and only a small group of them doing so with more than one machine. There are absolutely companies (and universities) that have puppeted that kind of thing, but it's a lot of effort and takes a lot of buy-in. It requires that the first step of your new-machine deploy process not be "Install OS", but "create manifests".

Ultimately its an effort vs efficiency cultural issue you'll have to solve among all of your IT staff.

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
13

PUPPET ALL THE THINGS

Anything that's reasonably similar across all systems (or a subset of them), or that you can base a template off a fact you can get out of facter is fair game.

Things that are really unique you probably shouldn't bother, and should just serve the configs out of a filebucket.

What falls into either category is a decision we can't make without knowing your environment intimately, so that's for you to figure out.

Michael Hampton
  • 237,123
  • 42
  • 477
  • 940
6

I think others have covered the why so I'll take a shot at the how. I think by understanding how someone might use Puppet to do what you want, it'll make the decision clearer.

Do the basic case first

Your Puppet module for Apache shouldn't do much by default. Install Apache, config it to a minimum standard, and start the service. Make this work on all distros you need to support.

Add flexibility second

We need to add vhosts. You'll end up with a system that can drop file or remove them from a set of conf.d or vhosts.d/ directories according to what you need. Same thing with enabling or configuring modules.

Use role or hostgroup classes to tie your building blocks together

I think the best way to use Puppet is to make sure it's additive. Using the examples above we should have a module that does

  1. Install Apache
  2. Set basic configs
  3. Add vhosts to apache
  4. Configure any extra settings
  5. Start Apache

Rather than overload our default Apache module to do exactly what we need for a particular host or group we should handle this is a role or hostgroup class.

class role::web_cust1 {
  include apache
  apache::vhost {'www.domain.com': }
  apache::vhost {'www.domain2.com': priority => '99', }
  include php
  include php-fpm
  include mysql
}

Again additive.

Put special cases in Hiera

I'm a big fan of letting Puppet's Hiera, think of it as a database for Puppet, store the special bits. If a certain host or hostgroup needs a special setting, first put a sane default into the module so that normal users don't need to know about it. Then insert data for those special hosts or hostgroups so Hiera can consume it pass it to Puppet as needed.

My use case is Listen port. Some servers have a Varnish or haproxy in front of them. By default the Puppet module has Apache use port 80, but if Hiera finds data it will override that default.

kashani
  • 3,922
  • 18
  • 18
  • I've been using the role module hierarchy and it works well. It makes it easier to build an environment where you might have many servers playing many roles (ie. some of the role::web servers might be role::storage also). – Andy Shinn Nov 15 '12 at 16:38
5

I'm currently in the transition between Puppetize reasonably similar systems to Puppetize everything and am convinced that long-term, Puppetize everything is a better approach.

If you version control your Puppet manifests (we all do this, right) you gain all the benefits of version control for your infrastructure. Your team becomes operations engineers. This is as important for special, one-off systems as homogeneous cattle farms. You get a log of who changed something, when they changed it, what the exact change was, and the ability to roll back the change.

Personally, I also find that forcing myself to make every change through Puppet makes me think more carefully about the change. As I'm writing manifests, I'm more attentive to each change than I typically am hacking away at the command line.

Your Puppet modules will also become better. Do you have more than one Nginx module? Maybe that means your Nginx module isn't that great, and you need to make it flexible enough to handle all your special needs. At least abstract the similarities into a core Nginx module that you extend for "custom" modules.

Further, how confident are you that you can restore all your special needs servers to their current state (with respect to configuration) when the disaster happens? If every change needed to get a factory Ubuntu server into your internal wiki is Puppetized, you can easily rebuild the current state of your wiki, yesterday's Tomcat memory tweak by Bob included.

Finally, this can be really hard. Managing lots of very different servers can lead to some hacktastic Puppet code if you don't take the time to do things right. If you aren't using Puppet Enterprise, consider hiera and/or an ENC such as Foreman to help separate your data from your manifests. Ever day Puppetize something else. Have a coworker drive while you explain how this works in Puppet. Each change will get easier.

czervik
  • 241
  • 1
  • 3