How can I version control server configuration files that are changed at application runtime, using git or another VCS?

Question

I'm running a Minecraft server (craftbukkit) and I have several other admins who want access to modifying server configuration files. It is important that I track all their changes, so naturally, git seems like a good choice.

Please note that this question could pertain to many situations, it is not specific to Minecraft.

Several tutorials exist on using git to manage websites. The most common solutions seems to be using the post-receive hook to run a checkout operation to the web directory. However, in my situation this poses a few issues.

First, some files admins would need to edit are changed by the server at runtime. I'm assuming this would confuse git if I made the server directory itself into the repository. Using the post-receive checkout solution, this would be less problematic, if I am correct (I am still learning git).

I also would need changes made by the server to be pushed into the repo, so admins can fetch those changes down to their local repos. I've seen solutions using inotifywait but these all seem only to account for single file changes. My server has 50-80 config files that I would need to track and autocommit when the server runtime changes them. What would be the best way to handle this?

Note that using git is not a requirement. I simply like what I've used of it so far. If there is a better tool for the job, I am open to using it, so long as it is user-friendly. Note that my server admins are not coders, nor are they Linux power users, so user-friendliness helps.

I originally posted this on StackOverflow, but was told it was better suited for here.

Answer 1

This isn't a complete answer, but hopefully some useful thoughts:

inotifywait will work happily with multiple files and can recursively set watchpoints on directories. For example, I can run:

inotifywait -m -r -e close_write /etc

And get the following log after editing /etc/passwd and /etc/postfix/main.cf:

/etc/ CLOSE_WRITE,CLOSE .passwd.swpx
/etc/ CLOSE_WRITE,CLOSE .passwd.swp
/etc/ CLOSE_WRITE,CLOSE 4913
/etc/ CLOSE_WRITE,CLOSE passwd
/etc/ CLOSE_WRITE,CLOSE .passwd.swp
/etc/postfix/ CLOSE_WRITE,CLOSE .main.cf.swx
/etc/postfix/ CLOSE_WRITE,CLOSE .main.cf.swp
/etc/postfix/ CLOSE_WRITE,CLOSE 4913
/etc/postfix/ CLOSE_WRITE,CLOSE main.cf
/etc/postfix/ CLOSE_WRITE,CLOSE .main.cf.swp

You could very easily work this into a script that, upon each close_write event, would commit the file to the local repository and push the changes to a remote server.

Also note that incron is a nifty tool for automating this sort of inotify-based workflow (but wouldn't substantially change the nature of the solution).

You're going to run into difficulties if your admins will be editing the same files that get updated by the server at runtime. This suggests that you're going to have to set up some sort of automatic conflict resolution on the server, which will invariably result in losing some information (well, apparent loss, at least, you can obviously preserve conflicting changes in two distinct branches of the repository but the server only gets to see one branch).

I don't think any parts of this problem or solution are particular to git. You're going to run into these issues with any sort of distributed, asynchronously synchronized shared maintenance of files.

Answer 2

What I've done in similar situations is make the live configuration a repository (I tend to use Bazaar), and written a cronjob to automatically commit any changes. Having the live configuration a VCS repository shouldn't be a problem -- the only difference is the .git directory, which should just be ignored in most cases. The interval of the cronjob would be whatever granularity you are happy with. Every hour has been good enough for my situations, but it could be every 5 minutes, or even 1 minute if necessary.

Answer 3

Several tutorials exist on using git to manage websites. The most common solutions seems to be using the post-receive hook to run a checkout operation to the web directory.

For the configuration files, take a look at etckeeper:

Name       : etckeeper
Arch       : noarch
Version    : 0.63
Release    : 1.el5
Size       : 36 k
Repo       : epel
Summary    : Store /etc in a SCM system (git, mercurial, bzr or darcs)
URL        : http://kitenet.net/~joey/code/etckeeper/
License    : GPLv2+
Description: The etckeeper program is a tool to let /etc be stored in a git,
           : mercurial, bzr or darcs repository. It hooks into yum to automatically
           : commit changes made to /etc during package upgrades. It tracks file
           : metadata that version control systems do not normally support, but that
           : is important for /etc, such as the permissions of /etc/shadow. It's
           : quite modular and configurable, while also being simple to use if you
           : understand the basics of working with version control.
           : 
           : The default backend is git, if want to use a another backend please
           : install the appropriate tool (mercurial, darcs or bzr).
           : To use bzr as backend, please also install the etckeeper-bzr package.
           : 
           : To start using the package please read /usr/share/doc/etckeeper-0.63/README

First, some files admins would need to edit are changed by the server at runtime.

No problem.

I also would need changes made by the server to be pushed into the repo, so admins can fetch those changes down to their local repos.

By default, etckeeper runs daily as a cron job:

/etc/cron.daily/etckeeper

#!/bin/sh
set -e
if [ -x /usr/bin/etckeeper ] && [ -e /etc/etckeeper/etckeeper.conf ]; then
    . /etc/etckeeper/etckeeper.conf
    if [ "$AVOID_DAILY_AUTOCOMMITS" != "1" ]; then
        # avoid autocommit if an install run is in progress
        lockfile=/var/cache/etckeeper/packagelist.pre-install
        if [ -e "$pe" ] && [ -n "$(find "$lockfile" -mtime +1)" ]; then
            rm -f "$lockfile" # stale
        fi
        if [ ! -e "$lockfile" ]; then
            AVOID_SPECIAL_FILE_WARNING=1
            export AVOID_SPECIAL_FILE_WARNING
            if etckeeper unclean; then
                etckeeper commit "daily autocommit" >/dev/null
            fi
        fi
    fi
fi

If you also want to push into the remote repo, edit the above file as belows:

        if etckeeper unclean; then
            etckeeper commit "daily autocommit" >/dev/null
            cd /etc && git push origin master
        fi

If daily or even minutely interval is not enough, Watcher can commit your configuration files immediately after changing:

/etc/watcher.ini

[DEFAULT]
logfile=/tmp/watcher.log
pidfile=/tmp/watcher.pid
[job1]
watch=/etc
events=create,delete,write_close
recursive=true
autoadd=true
command=cd /etc && etckeeper commit "daily autocommit" >/dev/null && git push origin master

Give it a try.