29

This is probably a simple question for those of you already running configuration management tools. Are configuration management tools such as Puppet or Chef the right approach for keeping installed packages up to date?

Suppose I run a number of servers, mostly based on Debian and Ubuntu. Do configuration management tools make it easier to update packages installed from the repositories when security updates or bug fixes come along?

I currently run "unattended upgrades" to let the systems automatically install security updates, but I still have to connect to the servers and run aptitude update && aptitude safe-upgrade every so often. Naturally this gets boring, tedious and error-prone the more servers there are.

Are tools such as Puppet or Chef the right approach to keeping installed packages up to date? Do any of you use these tools to avoid manually running aptitude or an equivalent on 15 servers? I am quite certain the answer to these questions is "Yes, of course!"

But where can I find more information about this particular use case? I have not yet had the time to study Puppet or Chef in-depth, and the example cookbooks or classes only show more or less trivial examples of installing one particular package, such as ssh. Do you have any resources to recommend, other than the official documentation (I am, of course, going to study the docs once I know which, if any, of the tools are right for me).

Cristian Ciupitu
  • 6,226
  • 2
  • 41
  • 55
daff
  • 4,729
  • 2
  • 26
  • 27
  • nice question, i've read some tutorial [ which i cannot find ] mentioning keeping debian up to date with puppet but never tried it myself. it'll be interesting to see answers of those using it in production – pQd Dec 14 '09 at 13:00

10 Answers10

22

You can do it with puppet, you either do:

ensure => latest,

or

ensure=> "1.0.2",

to specify the latest/required version. i.e.

package { apache2: ensure => "2.0.12-2" }
package { apache2: ensure => latest }

This does at least mean you can specify the same version across all systems, as well as preventing servers from (potentially dangerously) automatically upgrading themselves. I've used this method in production on a number of sites, and it works very well.

Running unattended upgrades scares me a bit, especially if they're upgrading mission-critical packages, kernels, mysql libraries, apache, etc. Especially if the install script might want to restart the service!

Tom O'Connor
  • 27,440
  • 10
  • 72
  • 148
  • Thanks for the reply! So it seems that keeping packages that were installed via Puppet up to date is at least possible. Good to know. But what about servers running different versions of packages? As in Debian Lenny vs Ubuntu 8.04 and 9.10? Do I have to take care of versions manually? I have some more research to do, it seems. I am not sure what I was expecting, maybe something along the lines of [Canonical's Landscape](http://www.canonical.com/projects/landscape) which offers a web interface and other more or less fancy stuff for updating packages on multiple servers. – daff Dec 14 '09 at 19:34
  • For servers running different versions, this is where it gets complicated. You need to have different blocks within your puppet manifest, where it uses Facter to retrieve the lsb_release or debian_version keyword from /etc and then makes decisions based on that about which package to install. I've not seen Landscape used in anger on production servers, I gather it's quite expensive. – Tom O'Connor Dec 15 '09 at 07:37
  • 2
    `ensure => latest` will always make sure everything's up-to-date with whatever your set of repository provides. – womble Dec 16 '09 at 07:41
18

I think this is probably the wrong question. Certainly using configuration management tools like Puppet and Chef to maintain your infrastructure is a huge leap forward from trying to do it all manually. The issue of keeping your package versions up to date and in sync is not one that any of these tools solves directly. To automate this properly you need to bring the package repositories themselves under your control.

The way I do this is to maintain a dedicated Yum repo (for Redhat/Fedora/CentOS; an APT repository for Debian/Ubuntu) which contains the packages I care about for a particular site. These will generally be the dependencies of the application itself (Ruby, PHP, Apache, Nginx, libraries and so on) and security-critical packages.

Once you have this set up (usually you can just mirror the required packages from the upstream repo to start with) you can use Puppet's "ensure => latest" syntax to make sure that all your machines will be up to date with the repo.

It would be wise to use a 'staging' repo to enable you to test updated versions of packages before rolling them blithely out to production. This is easily done with Puppet without any duplication of code by using repository templates.

Automating your package versioning strongly encourages you to bring all of your production systems into sync, as maintaining multiple repos and packages for different OS distros, versions and machine architectures is very time consuming and likely to lead to all sorts of obscure problems and incompatibilities.

All of this advice applies equally to Ruby gems, Python eggs and other package systems which you may use.

I've written a little Puppet tutorial which should help you get up and running with Puppet quickly. You could deploy a custom repo definition to your machines using Puppet as the first step in bringing package versions under control.

John Arundel
  • 539
  • 3
  • 4
9

Puppet (I'm pretty sure chef does also) ties in with your apt-get/yum software repositories. Since they do the heavy lifting of figuring out which packages are available, that means ensure => latest just works for Ubuntu/CentOS/Debian the like. As long as you set up the appropriate files correctly (/etc/apt/sources.list, etc).

Cristian Ciupitu
  • 6,226
  • 2
  • 41
  • 55
Perlchild
  • 114
  • 2
  • 1
    Answers that involve Puppet or similar managing each package mean that you must track every package in Puppet, even the ones that are part of the basic Linux distribution installation. Using tools such as `unattended-upgrades` or `yum-cron` to [automate the updates](https://serverfault.com/a/861468/79266) is much less work - just use Puppet/Chef/Ansible to configure those tools. – RichVel Jul 11 '17 at 09:48
5

This question is old, but I thought i'd answer in an up-to-date way as a currently existing answer was unavailable back then.

If you are using puppet or chef, look into mcollective. It is a very nice tool by the puppetlabs guys that allows you to send commands to groups of servers. http://docs.puppetlabs.com/mcollective/

It also has an apt plugin, which can be used to do an apt update on any number of servers: http://projects.puppetlabs.com/projects/mcollective-plugins/wiki/AgentApt

Walter Heck
  • 92
  • 2
  • 7
5

Whilst Puppet/Chef are possible contenders for this functionality, to make them keep everything on the system up-to-date requires either custom types or listing every package (including underlying system libraries like libc6) as resources with ensure => latest. For the specific case of automated package updates, you might want to look into the cron-apt package, which does what you want as well.

womble
  • 95,029
  • 29
  • 173
  • 228
  • or just push out an exec job of "yum update" witha high splay time. Works for me anyhow. – Sirex Nov 09 '11 at 16:04
4

I realize this is a bit late for your original question, but here it is in the spirit of "better late than never".

I use Cfengine 3 to do this on several servers. I specify an explicit list of packages for automatic update, thus avoiding updating all packages without being a little careful about it. It works great, and cfengine 3 is very lightweight.

Here's a promise snippet from my cfengine configuration:

    packages:
            "apache2"
                    package_method => "apt",
                    package_policy => "update";

Hope this helps.

Jonathan Clarke
  • 1,657
  • 2
  • 11
  • 25
2

I agree with Jonathan. The Cfengine 3 approach is nice because you can control all aspects of package management without having to recode at a low level.

SAnnukka
  • 69
  • 3
2

We use puppet + apt-dater.

ptman
  • 27,124
  • 2
  • 26
  • 45
1

You can also use package management tools such as Canonicals Landscape which is designed to manage and monitor Ubuntu / Debian systems. It manages multiple systems, allows you to update them simultaneously and provides some basic monitoring capabilities.

0

Security updates

Generally I think it's simplest to use Ansible or similar to set up the robust unattended-upgrades package for Ubuntu/Debian (or yum-cron for RHEL/CentOS). You can use Puppet, Chef or other tools, but I will discuss Ansible here.

  • unattended-upgrades can be used to make non-security updates at the same time if you prefer, which is much easier than running a command via Ansible every day.

  • unattended-upgrades takes care of auto updates every day, and is normally constrained to security updates only (to increase stability). If the server needs a reboot after the update, this tool can auto-reboot at a certain time.

If your reboots are more complex, unattended upgrades can email you, and it also creates /var/run/reboot-required, so that Ansible (or similar) can manage the reboots at a suitable time (e.g. rolling reboots of a cluster of web or DB servers to avoid downtime, waiting for each server to become available on a certain TCP port before continuing).

You can use Ansible roles such as jnv.unattended-upgrades for Ubuntu/Debian systems, or the simple but effective geerlingguy.security, which also covers RHEL/CentOS (and hardens SSH config).

If rapid security updates are less important, you could put them through a test process on less important servers first, and run the unattended-upgrade command once tests show there are no problems - however it's quite rare for server-oriented security fixes to cause problems, in my experience.

General updates

Updates other than security should go through a normal continuous integration and testing process, to ensure things don't break.

I have seen aptitude safe-upgrade cause serious problems on servers in the past, so it's best not to run this automatically, whereas security updates are generally quite safe.

Community
  • 1
RichVel
  • 3,524
  • 1
  • 17
  • 23