6

I do fieldwork with a number of computers running ubuntu performing critical tasks doing fieldwork. The computers are similarly configured with slight variations.

Since we've had some configuration issues in the past, my boss is pressing for us to take an image of the installation on each computer, and restore each computer to that image before they are to go into the field.

My preferred solution would be to write a common script that checks to ensure that the configuration of the system is correct and that the system is operational. If the computer has been verified, isn't restoring it to that configuration redundant? And are there any inherent problems with doing so?

My reluctance stems from the fact that our software and configuration is subject to change in the field, but these changes must be made across all the computers. That means that when a change is made, all the restoration images have to be updated as well. The differences in the configuration of each of the computers live in /etc. In the event that restoration is required, I would prefer to keep a single image containing everything that is common to all machines, and have a snapshot of each computer's /etc directory to be used for restoring the state of that particular machine.

What's the better approach?

field_guy
  • 61
  • 1

2 Answers2

13

It sounds like what you really need is a configuration management system like puppet. With a proper setup, it should verify that everything is the way it is supposed to be setup on every puppet run.

Periodically re-installing systems just to get them to a known state should not really be necessary, and sure seems like it will create a lot of work, that you shouldn't ever need to do. When you have a lot of Linux systems it is much better to have a really basic automated install, and then have a system to pull in all the proper configurations for a given node after defining a unique system name/id.

Zoredache
  • 128,755
  • 40
  • 271
  • 413
  • 2
    +1 - you are describing exactly the problem configuration management tools were built to solve. Use them. – voretaq7 Apr 13 '12 at 22:58
  • Of course, it might still be a good idea to throw up a VM, run puppet on it, and make it works every once in a while (to test that your puppet configs are correct). – Brendan Long Apr 14 '12 at 00:18
  • @voretaq7, thank you, i wasn't even aware such tools existed. I've read puppet and it looks like an excellent tool, especially because it scales. Something I may do right away is simply have an automated install for the common configuration using something like remastersys, and keeping /etc of each system under version control. – field_guy Apr 14 '12 at 09:58
1

If you could virtualize these servers, then you can periodically return to a base snapshot which was taken after everything is verified. This has the advantage of being guaranteed known state, because configuration management works but does not guarantee returning to a known state of the entire server.

johnshen64
  • 5,747
  • 23
  • 17
  • 3
    All of which would still be moderately onerous without a configuration management solution, and more or less unneeded WITH configuration management. – Magellan Apr 13 '12 at 23:10
  • Yes, and additionally, returning to a base snapshot will may also wipe data that is still needed, such as logfiles, temporary files or result data. You could store this on a different partitions, but it still means extra work (and risk). – sleske Apr 13 '12 at 23:55