71

I'm learning my way through configuration management in general and using puppet to implement it in particular, and I'm wondering what aspects of a system, if any, should not be managed with puppet?

As an example we usually take for granted that hostnames are already set up before lending the system to puppet's management. Basic IP connectivity, at least on the network used to reach the puppetmaster, has to be working. Using puppet to automatically create dns zone files is tempting, but DNS reverse pointers ought to be already in place before starting up the thing or certificates are going to be funny.

So should I leave out IP configuration from puppet? Or should I set it up prior to starting puppet for the first time but manage ip addresses with puppet nonetheless? What about systems with multiple IPs (eg. for WAN, LAN and SAN)?

What about IPMI? You can configure most, if not all, of it with ipmitool, saving you from getting console access (physical, serial-over-lan, remote KVM, whatever) so it could be automated with puppet. But re-checking its state at every puppet agent run doesn't sound cool to me, and basic lights out access to the system is something I'd like to have before doing anything else.

Another whole story is about installing updates. I'm not going in this specific point, there are already many questions on SF and many different philosophies between different sysadmins. Myself, I decided to not let puppet update things (eg. only ensure => installed) and do updates manually as we are already used to, leaving the automation of this task to a later day when we are more confident with puppet (eg. by adding MCollective to the mix).

Those were just a couple of examples I got right now on my mind. Is there any aspect of the system that should be left out of reach from puppet? Or, said another way, where is the line between what should be set up at provisioning time and "statically" configured in the system, and what is handled through centralized configuration management?

HopelessN00b
  • 53,385
  • 32
  • 133
  • 208
Luke404
  • 5,708
  • 3
  • 44
  • 58
  • 1
    Nice question. I'm curious myself if there's anything other than machine-specific configs you shouldn't use puppet for. Well, that and Windows machines. – HopelessN00b Aug 06 '12 at 17:07
  • 6
    You should not manage things in puppet when it is better/easier to manage them some other way. :p – Zoredache Aug 06 '12 at 17:24
  • 1
    With the prevalence of companies using Puppet these days, I can see this question garnering much attention over the next few years – Daniel Li Aug 29 '12 at 15:12

6 Answers6

26

General rule: If you're using configuration management, manage every aspect of the configuration that you can. The more you centralize the easier it will be to scale your environment out.

Specific examples (cribbed from the question, all "This is why you want to manage it" narratives):

IP Network configuration

OK, sure, you configured an address/gateway/NS on the machine before you dropped it in the rack. I mean if you didn't how would you run puppet to do the rest of the config?

But say now you add another nameserver to your environment and you need to update all your machines -- Don't you want your configuration management system to do that for you?

Or say your company gets acquired, and your new parent company demands that you change from your 192.168.0.0/24 addressing to 10.11.12.0/24 to fit into their numbering system.

Or you suddenly get a massive government contract -- Only catch is you have to turn up IPv6 RIGHT FREAKIN' NOW or the deal is blown....

Looks like network configuration is something we'd like to manage...

IPMI Configuration

Just like with IP addresses, I'm sure you set this up before you put the machine in the rack -- It's just good common sense to enable IPMI, remote console, etc. on any machine that has the capability, and those configurations don't change much...

... Until that hypothetical acquisition I mentioned in IP Configuration above -- The reason you were forced to vacate those 192.168-net addresses is because that's IPMI-land according to your new corporate overlords, and you need to go update all your IPMI cards NOW because they're gonna be trampling on someone's reserved IP space.

OK, it's a bit of a stretch here, but like you said - all of it can be managed with ipmitool, so why not have Puppet run the tool and confirm the configuration while it's doing all of its other stuff? I mean it's not going to hurt anything, so we may as well include IPMI too...

Updates

Software updates are more of a gray area -- In my organization we evaluated puppet for this and found it "sorely lacking", so we use radmind for this purpose. There's no reason Puppet can't call radmind though -- In fact if/when we migrate to Puppet for configuration management that's exactly what's going to happen!

The important thing here is to have all your updates installed in a standard way (either standard across the organization, or standard within platforms) -- There's no reason Puppet shouldn't be launching your update process, as long as you've thoroughly tested everything to ensure that Puppet won't mess up anything.
There's also no reason why Puppet can't call out to a tool that's better suited for this task if you've determined that Puppet can't do a good job on its own...

Community
  • 1
voretaq7
  • 79,345
  • 17
  • 128
  • 213
  • Re: Updates. One thing that can get you in trouble with puppet running your updates is when the critical service is being patched, e.g.: mysql, apache - you don't want those restarting on the whim. Puppet provides ways to lock in the version of those packages, that's how I've avoided it while enjoying general updates for other nuts and bolts. – thinice Apr 24 '15 at 19:06
  • @thinice That's a good point, but my usual counterpoint to it is to smack people on the back of the head and shout REDUNDANCY really really loud :-) (It's a worse situation with radmind because that just blasts the filesystem. Our policy is to have the load-balancer unload half the servers so we can patch/test those, then we move everyone to the patched machines so we can do the other half. Works well, but you need the redundancy built into your environment.) – voretaq7 Apr 24 '15 at 21:04
11

Don't reinvent the wheel.

Yes, you can have 50 puppet virtual user resources and realize them as needed in your modules... but if you can, use LDAP.

I speak from bitter experience. Although ldap isn't an option here, yet.

Anouther example is pushing out hosts files, rather than just using DNS.

Sirex
  • 5,447
  • 2
  • 32
  • 54
  • 3
    I recognize all those words, but I'm still not sure what you're trying to say. – Chris S Aug 07 '12 at 00:18
  • 2
    I'm trying to say; puppet is a central place for "information". So are DNS and LDAP. Don't try to do their jobs with puppet, it's rubbish at it.... I say this having seen giant /etc/hosts files being pushed out with puppet every time a new host joins the network. – Sirex Aug 07 '12 at 00:24
  • 3
    Do people really use Puppet instead of LDAP to manage user accounts?? – Joel E Salas Aug 08 '12 at 00:30
  • Yes. We do... politics. Believe me, I'm working on it. – Sirex Aug 08 '12 at 01:00
  • 2
    Every tool has its place, but using puppet for user account management or LDAP for file storage is *abuse*. – Hubert Kario Dec 13 '12 at 23:35
  • 1
    It's certainly *ab*use... – jldugger Jan 01 '13 at 19:28
  • 1
    How then do you effectively manage ssh keys and authorized_keys across a large number of servers? – Josh Smeaton Mar 21 '13 at 02:49
  • 1
    We now use freeipa (just called "ipa" in centos & redhat). It's awesome. Doing this in puppet was just awful, although we still do it for service accounts (i.e: non-humans) – Sirex Mar 21 '13 at 04:07
  • Can you expand on the problems you had with virtual user resources in the answer? – 7yl4r Aug 07 '18 at 14:27
  • yeah, it's not the right tool for the job. Say, the week after when you get a requirement that all users must have complex passwords, or change those every 90 days, or they must have a minimum length. Or you abuse the hosts file instead of dns and now you want an alias, etc etc. Using config management where it doesn't fit will bite you in the arse eventually. What you can (should) do is use it to setup the dns server, and use it to add records for hosts into it. – Sirex Aug 07 '18 at 20:11
8
  • Puppet is not an orchestration system. In particular:
    • Puppet is not well suited to VM orchestration, as VMs have a lifecycle of their own which should be respected.
    • Puppet is not well suited to application release management/complex upgrades. Standalone puppet runs could be exploited for this, but then at least it's not Puppet in control, instead it's your wrapper scripts or a human robot, which is fine.
  • Puppet is not a good user management system (It has to manage every user entry, even deleted users, to be effective. So find another solution)
  • Puppet is not a good configuration database (look at using an external database of some sort, and an ENC, Hiera, or some similar glue)

Of course you can do all of these things with Puppet.. but it’s just not the best solution for them. Sometimes you should put down the hammer, and go look for a wrench.

Puppet is however exceedingly good at maintaining the base configuration for a machine, and installing the tools that let you do VM and release orchestration, user management etc.

Robert M Thomson
  • 81
  • 1
  • More of a quibble: Puppet can be configured to add and remove users with or without managing *every* user. That said... it is sorta useless to *not* manage every user, and terrible at managing every user. I use puppet to add service accounts, but not user accounts. – Art Hill Oct 10 '18 at 22:45
2

In my case, I have a bootstrap script which loads the minimal system config (Ubuntu): Ruby, Rubygems, build-essential, git, etc. My Puppet manifests are kept under version control, and I simply clone the repository. From there, my bootstrap script makes an assumption that hostname --short is valid, and attempts to puppet apply /root/infrastructure/puppet/hosts/$( hostname --short ).pp.

To answer your question:

  • My scripts assume basic network connectivity (DNS, IP), and don't manage or change it;
  • My scripts assume the machine's identity is correct, and don't change it;
  • My scripts assume Ruby / Rubygems / Git is present, but do manage it afterwards.
François Beausoleil
  • 751
  • 2
  • 9
  • 20
2

I mostly agree with voretaq7, but with a couple of caveats.

  • I rarely ever configure the IP addressing in puppet, unless the system uses DHCP (i assume most large "cloud" providers do this). I've had situations where i broke network configurations with puppet, but couldn't correct them with puppet because the node didn't have any way to contact the puppetmaster.

  • I'm quite firm in my belief that update management belongs in the hands of the system tools, and i don't see using puppet as a glorified cron to be an improvement.

Paul Gear
  • 3,938
  • 15
  • 36
0

Think you don't need use puppet for network configuration. It is once configured thing usually. Also you can get some crap if you will have a mistake(s) with IP or MAC or something similar that will brings by puppet.

Evgenii Iablokov
  • 630
  • 2
  • 8
  • 15
  • 2
    Never had to change default gateway on 100+ server by hand? Lucky you ;) –  Aug 06 '12 at 21:36
  • @EricDANNIELOU I suppose that could be taken as a +1 for letting puppet manage IP configuration of network interfaces ;) – Luke404 Aug 06 '12 at 23:01
  • @EricDANNIELOU Try to do this with bash, "for" cycle and appropriate user permissions (sudo to root or root directly) and sed/perl/etc. :) – Evgenii Iablokov Aug 07 '12 at 04:37
  • 1
    I don't think bash "for" cycle and dirty sed/awk/vi scripts are safer than scm for network configuration. And once you have puppet set up for everything else, it's not very convenient to have ssh "for" loop only for network configuration. –  Aug 07 '12 at 10:38