Best practice for testing chef recipes?

Question

My team has created a large number of chef cookbooks. I was wondering what methods and frameworks I should look into so we can start creating tests to assure that our nodes are configured properly?

Answer 1

You might (also) look into Vagrant for this.
A video is showing how to use the combo of Vagrant and Chef cookbooks. And there's a dedicated page to Vagrant and Chef on the Opscode site (update 2015-01-23: page has gone...).

Answer 2

There's the cucumber-chef ruby gem and a recently released book Nelson-Smith et al - "Test-driven Infrastructure with Chef" that makes use of that gem - http://www.cucumber-chef.org/

Just remembered this from my github watchlist: https://github.com/gregretkowski/vmth/

The VMTH (Virtual Machine Test Harness) provides a mechanism to unit-test your infrastructure automation - puppet policies, chef recipes, release deployment scripts, etc. It uses features of modern VM monitors (like qemu) to snapshot system state, and then reset that state after each test, so that a series of tests can be performed on a VM instance in rapid succession

Answer 3

There's a number of options.

• cucumber-chef rubygem by Stephen Nelson-Smith, with a related / accompanying book.
• minitest cookbook by AJ Christensen.
• vagrant to provide test virtual machines thanks to Mitchell Hashimoto

Opscode has on the Chef development roadmap "no-op" support which will help with testing. We do argue that "no-op" doesn't necessarily do what you want, but understand the desire for the feature.

Answer 4

Some of the answers here seem a bit old. For CI and local testing I would look into leveraging a test harness tool like Test Kitchen which also supports Vagrant if you've been using that already.

It will allow you to run your coobkooks against many cloud and virtualizations solutions out there today: EC2, Digital Ocean, Vagrant, Docker, etc..

It also has plugins to allow you to run one of the many infrastructure test tools that exist out there today:

• serverspec - Most popular testing framework.
• goss - YAML, simple, self-contained binary, extremely fast.
• inspec - think of it as serverspec improved (written by the chef guys).
• testinfra - Python based server testing framework.

Answer 5

In my experience, the only way to really do this is with a staging environment and careful development practices. Mocking out the actual work to be done is incredibly messy and fragile, and CI test suites running on commit or push are a lot of effort to write and maintain (although Chef is probably a lot better at this than Puppet, which is my main experience), and take a ridiculous amount of time to run.

EDIT

Just after I wrote this, I suddenly realised there might be a way to do reasonably efficient "unit" testing of Chef/Puppet recipes with a union filesystem and a significant amount of scaffolding. However, I can't find anything out there that anyone's done already in this area. I might have just found myself a project.