3

Coming from a development background i'm used to automated (unit) testing before stuff goes live.Now i'd like to use the same approach to (new) linux (and some windows) servers and networkdevices.

I want to be able to define tests / conditions which these systems should pass before they can go live, or run the tests against live hosts to check if they still comply with our standards.

Tests I'd like to run go beyond networkscanning. For instance:

  • I want to check if SSH is enabled, but no root login is allowed and keybased logins are enforced.
  • On a printer I want to check if certain SNMP communities are set.
  • On a linux host i'd want to check the ntp settings
  • I'd want to be able to define custom check in some specific cases

Do you know if such an automated system exist, which one would meet my requirements best? Or should I built on an existing unittest framework?

Erwin Vrolijk
  • 225
  • 3
  • 5
  • Related questions: http://serverfault.com/questions/299895/best-practice-for-testing-chef-recipes http://serverfault.com/questions/36407/test-driven-development-for-infrastructure-deployments – oliver Mar 27 '16 at 16:25

2 Answers2

3

This is what you use monitoring for. There are plenty of monitoring systems, with different positives and negatives, and this isn't the place for an exhaustive discussion of the different options. Your monitoring should, in principle, represent all of the assertions you make about your system, both in terms of outputs as well as response times. I encourage the use of "monitoring-first systems administation" in my team, and the parallels to development should be obvious.

Now, just as some things are hard to unit test, there are also some things that are hard to monitor. Your SSH example is one of them -- while you can certainly try to login, and if that fails, then say you're done, but plenty of things could confound your test -- "try to login as root with a password fails" could be screwed up by someone changing the root password to test123 and turning on password auth -- you don't know what the password's been set to, so of course your test login will fail.

For those things, you want a configuration management system, like Chef or Puppet. These systems allow you to effectively make assertions about the state of a system (like "The PasswordAuthentication option in /etc/ssh/sshd_config should be set to false"), and the config management system will ensure that is the case each time it runs. Good systems can also provide you with exception reporting ("hey, I thought you might like to know that PasswordAuthentication was set to true; it's OK, I fixed it, but you might want to go break someone's fingers") so you can know when something odd does happen.

womble
  • 95,029
  • 29
  • 173
  • 228
  • Thanks for your reply. I'll look into monitoring and config. management. Maybe I'll supplement it with some custom unittest cases where things are very application dependent or other methods will fail. – Erwin Vrolijk Jul 17 '12 at 08:09
1

You might be able to use Serverspec. It is mainly useful for tests which run on the target host - eg. checking for file existence/content, installed packages, used server ports.

oliver
  • 395
  • 1
  • 5
  • 18