We are trying to eliminate a single point of failure for our files without a SAN. We currently have a single server for file serving (locally), web server, and database (in VM). Looking at using DRBD with GFS2 to create a cluster file system and then run as much as possible from that file system, including the Host OS, VM's, and shared files.
We would like to have two servers setup in a master/slave setup running from a shared source; but the servers would have slightly different hardware configurations (drive port assignments, actual brands and models of main board, NICs, etc) so the hardware configuration files would be kept local (fstab and mdadm for example).
Ideally, one server would act as primary. Package management would happen by the primary server only on the clustered file system because the slave server should not need to if it is using the same data (data from same block device).
The master server would also be the single point of administration for both servers simply because they share the data and configuration files, and the second server simply sits as a redundant set of hardaware to run the software that is configured on server 1 and does so via STONITH, Hearbeat, Pacemaker, etc.
But as I read, it seems as though most people administer the servers independently of each other, and try to automate using cron jobs and rsync.
It also seems that most of the files are fairly static and user modify; that is the number of files/folders that are actively modified by the OS (logs, swapfile, etc) independent of user input and/or dependent on hardware are very small.
We would like to keep the files served by the native OS, (not in a VM).
So now the questions.
Can the the software administration be separated from the hardware administration? It seems the only thing that needs administered independently is the hardware on each machine; the software should be managed at once via the clustered file system. (Could be wrong, hence the question)
Can the Host OS be ran from a shared set of files by either of the two machines, one at a time? That is, can both machines be ready to administer the Host OS, but only the master does until failover at which point the slave machine picks up the administration work? Or is this assumption inaccurate?
Can we set a machine into a slave/idle state until the master goes down? (Suspend cron jobs, suspend services, etc) I am assuming some of this will be a matter of of the fencing software?
(This is not a question of how to run both machines from the same set of data at the same time; they would be effectrively running from the same set of data at different times. The files the servers each need to constantly modify would be on the local file system)
We simply want to setup a redundant server and minimize the redundant administration.
