23

We haven't been doing any documentation at my workplace. I'm completely new to it and ask for some guidance getting started.

I have a few questions:

  • What are the essential documents that a sysadmin should write and maintain? And why are these so important?

  • How do you keep your documentation in sync with the system? How do you minimize duplication of information?

  • Recommended guides, best practices, anti-patterns?

chickeninabiscuit
  • 1,094
  • 6
  • 17
  • 33
  • Related: [How do you document a network?](http://serverfault.com/questions/12378/how-do-you-document-a-network) – Zoredache Sep 02 '13 at 08:15

6 Answers6

15

since 2003 I'm documenting everything in our inhouse wiki.

Servers

  • hardware specs
  • waranty information
  • network information
  • and of course installed software and configuration

Workflows

e.g. how-to add or delete a user and give him/her access to all relevant services

Important links

  • link to all your web interfaces
  • link to the monitoring URLs (nagios, munin, apc-monitoring...)
  • link to the wiki (for the printed version!)

Emergency instructions

what to do if intranet server/internet/web server/etc are down

Important:

Choose a wiki engine with easy export to PDF!
Its not useful if you are in holiday, the server running your wiki is down and no one knows what to do because your documentation is offline

Have a look at twiki, docuwiki or mediawiki.

BTW:

there is a OpenOffice.org plugin to write directly to mediawiki - very convenient.

EDIT:

Its also nice to write down some infos to /home/adminuser/maintenance. This is done quick and can be very helpfull, if several admins work on a server. eg:

2009-06-27 -thorsten-
          running aptitude update && aptitude full-upgrade
          everything seems ok
2009-06-25 -andreas-
          cups-pdf wasn't reachable. restarted cups
2009-06-23 -thorsten-
          deleted old log under /var/log/squid
etc.
Tommiie
  • 5,547
  • 2
  • 11
  • 45
ThorstenS
  • 3,084
  • 18
  • 21
13

While you realize that while everyone wants (and needs) documentation, you also need to recognize that no-one has time to read and study the stuff.

So, don't write documentation that needs to be studied - instead, structure your documentation in a way that allows someone to quickly find the information they need, when they need it - which may well be while the system is down and the CTO is breathing down his/her neck.

With this in mind, some suggestions ...

  • Avoid large blocks of text
  • Bullet lists are your friend
  • A clear diagram is golden
  • Repetition is a good idea (1)
  • Make it easy to update and extend

(1) Don't create one source of truth and force people to hunt it down. The more important the idea, the more you should repeat it.

Bevan
  • 621
  • 1
  • 3
  • 8
  • 2
    With more than one source of documentation however, you have more than one place that needs updating should it become outdated and need to be changed. A good way round this (if you have a wiki or something similar) is to try to make one true source of truth and link to it from as many places as needed. – Mark Jul 27 '09 at 03:35
  • To some extent, I agree - links and cross references can be very useful indeed. There is a trade-off though - in database design, its common to de-normalise tables to aid in reporting. I think the same approach is relevant here - to make *consumption* of the documentation easier, repeating key facts can be valuable. – Bevan Jul 27 '09 at 09:52
  • its ok to distribute principles widely, but for things like IP addresses, passwords, configuration management a centralize single authoritative source of data, with adequate backups is key to sane administration. – Tom Dec 21 '11 at 05:25
  • I'd agree - as long as it is both *well known* and *easily accessed* - a single authoritative *secret* source is an all too common antipattern. – Bevan Dec 22 '11 at 03:17
  • I vehemently disagree with the repetition, because one will get updated, but others won't. Or they'll be inconsistently updated. Instead the more important docs should be more readily *linked*. – gWaldo Sep 09 '12 at 20:58
  • "DRY" is a great principle for machines, a lousy one for people. What I've done (with reasonable success) is to *repeat the principle* and *reference the details*. So I'd have the exhaustive detail about "which Active Directory groups are used to control security" listed in one document, but the principle that "system access is controlled by membership in active directory groups" would be repeated wherever relevant, in documents detailing installation, configuration, troubleshooting, management, training and so on. – Bevan Sep 09 '12 at 22:38
5

Essential docs:

  • Server documentation - specs/disk layouts/installed software/anything of note
  • Common procedures - anything that is done that isn't 'trivial' should have a procedure documented, especially if it's something that hasn't been done before.

Keeping documentation in sync can be pretty much a 'fix it as you see mistakes' affair. Along with this needs to come the realization that documentation can and will be out of date, and that it should not be followed blindly without taking this into account. Documentation is there to assist an admin in tasks, not to be a step by step set of rules that replace critical thought.

Minimizing duplication - using something like wikis where you can link documentation together can help with this, instead of repeating information, you just link to it. The problem is that the person writing the documentation needs to know that the information they are about to duplicate already exists. This is usually a question of good organization.

Mark
  • 2,846
  • 19
  • 13
4

I've found that creating a template is a big help. In my case it's a Word template but use whatever suites you. Create a skeleton file, complete with table of contents field and sections as desired. Once you've used it a couple of times, and made any fine tuning adjustments, you'll create new documents much faster. The consistency of format will be a great help, both for document creation and later use. The documentation needs to be stored in a logical place and in a logical directory structure.

I'm personally opposed to repetition for the simple fact that it makes upkeep unnecessarily difficult and time consuming. Rather than duplicate documents, or parts of documents, create references to other documents where appropriate. If something changes you should never have to change the relevant documentation more then once or in more than one place, otherwise you will have a collection of conflicting documents, which helps nobody.

While creating your documentation just keep in mind what it's for. Somebody at a later time will need to use it. Will it be usable to do the job without prior knowledge?

John Gardeniers
  • 27,262
  • 12
  • 53
  • 108
3

Not a direct answer to your question, but a pointer in the right direction:

I found The Practice of System and Network Administration, by Limoncelli and Hogan (aka the Sysadmin Bible) to be quite valuable because it is about "best practice" issues, such as documentation. If you don't already know about it, make sure you investigate it whenever you get the chance.

Tiberiu Ana
  • 256
  • 2
  • 5
  • The 2nd edition of that book has a chapter on documentation. A related book, "Time Management for System Administrators" has a chapter on documentation that is more focused on what you need to do, rather than what your organization needs to do. – TomOnTime Mar 20 '14 at 20:43
0

For me, the most important consideration is making it easy to use. If it's difficult to orchestrate then people will avoid it. I choose Trac's wiki as the medium for our documentation for these reasons:

  • Centrally located.

    More than one active copy of any one document leads to confusion. If you're able to refer everybody to the same place, both contributors and audience, then you can simplify the process.

  • Simple editing and formatting.

    So much time is wasted on pretty Word templates and conforming with the last author's styling. If you don't bog people down with this, then it's easier to edit on-the-go and contributors are more inclined to do so. Separate out items as much as you wish with TracLinks.

  • Audit history.

    It's important to know who made what change, when and why. If you can tie it into change request tickets and configuration commit logs then even better. SVN commit hooks are great for this.

Dan Carley
  • 25,189
  • 5
  • 52
  • 70
  • I'm also using trac for the documentation of one project. What`s _really_ missing is a kind of breadcrump in the wiki. I hope this is comming soon. – ThorstenS Jul 27 '09 at 16:28