49

I'm backing up a Linux server and storing it on another server.

I began with a simple

rsync -aPh --del server.example.com:/ /mnt/backup

Then someone pointed out that I shouldn't back up /proc, since you don't want to restore the /proc of one server on another.

Is there anything else that I should/shouldn't include?

For instance, what about /sys?

SDsolar
  • 157
  • 1
  • 1
  • 11
Amandasaurus
  • 30,211
  • 62
  • 184
  • 246

14 Answers14

48

Both /proc and /sys are virtual filesystems which reflect the state of the system, and allow you to change several runtime parameters (and sometimes do more dangerous things, like directly writing to the memory or to a device). You should never backup or restore them.

In most modern distributions, /dev is dynamically created at boot (it is a memory filesystem filled by udev and friends). There is no point in backing it up, and attempting to restore it is futile. However, if your distribution is configured to use a static /dev, this does not apply (check /proc/mounts, if /dev is a tmpfs it is a memory filesystem).

There are other filesystems you should not back up; usbfs (usually at /proc/bus/usb, if mounted at all), debugfs (supposed to be at /sys/kernel/debug if mounted at all, but some people put it somewhere else; you probably do not have this one), devpts (mounted at /dev/pts), other tmpfs instances (often found at /dev/shm, /var/run, /var/lock, and other places; backing them up and restoring them should be harmless but pointless, as their contents are lost on shutdown), and any remote filesystems or magic automounter directories (attempting to backup or restore them could end up in disaster, as you could end up backing up/restoring to a different machine). You should also be careful with /media and /mnt, as external devices (like a CD you forgot in the drive) could be found there, but you might also have used them on purpose to mount something which should be backed up.

Note that, other than mostly harmless tmpfs instances, network filesystems/automounters, and removable media, the filesystems you should not back up are all descendents of /dev, /proc, or /sys. If you have no network filesystems (or automounters), and no removable media, excluding /sys and /proc and rebooting after a restore (to wipe the tmpfs instances) should be enough.

CesarB
  • 2,368
  • 15
  • 9
25

This really depends on how you are going to restore your system. If you will rebuild then you only need the configuration/data files for your services (eg: /etc, /opt, /var, /home)

If you are after a full system restore, then it you could omit /proc, /boot & /dev. Then you can install the minimum OS from your boot media and then restore your system via your backup.

Of course, the best backup is one that has been tested and verified.

So omit what you don't think you need, try to restore in a VM and verify you can get back your system using this data.

Wayne
  • 3,084
  • 1
  • 21
  • 16
  • 6
    Don't omit `/boot` entirely -- you might need to compare old boot config with new boot config. Just be sure not to *restore* `/boot` except manually. – quack quixote Oct 15 '09 at 13:24
  • 5
    And exclude /sys as well... And to restore to bare metal, you'd better exclude /etc/udev/rules.d/ too. – wazoox Oct 15 '09 at 13:31
  • 2
    also lost+found which for the file system, /mnt and /media to not copy any mounted device – blade May 19 '15 at 14:26
19

See The Tao Of Backup, chapter 1.

Teddy
  • 5,134
  • 1
  • 22
  • 27
8

Some of the special files in /proc and /sys confuse rsync. You don't want to back up mounted network filesystems either, usually. Sparse files can also cause problems.

Add -x to limit it to one file system. That avoids all the network file systems and /proc etc. However you then need to run one rsync for each file system you have mounted.

Add -S to handle sparse files sensibly.

pjc50
  • 1,720
  • 10
  • 12
4

/boot, /dev and /proc are quite useless to backup -- though, if you know what you're doing you can backup /boot.

I would also not backup /lib, /media, /mnt, /sbin, /bin, /srv, /sys, or /tmp.

/usr is optional, depending on whether you have anything in /usr worth backup up. If I were you I"d worry most about backing up user's $HOMEs, /var, and /etc (for configuration files).

Again though, this really all depends on the type of backup you want to do. Is this a web server? Is this a personal computer? Is this a shell server with tons of directories under /home?

Michael Pobega
  • 934
  • 5
  • 12
  • I'd like to restore by use cloning the backup onto a new machine – Amandasaurus Oct 15 '09 at 12:18
  • What do you mean by "cloning"? You could always just backup the raw partitions using dd and sfdisk sfdisk -d > partition_table.part dd if=/dev/sda1 of=dev.sda1.img (do this for each partition) then, on your new system: sfdisk /dev/sda < partition_table.part dd if=dev.sda1.img of=/dev/sda1 (for each partition, again) – Michael Pobega Oct 15 '09 at 14:03
  • Since the comment system doesn't like code tags, I posted another answer. – Michael Pobega Oct 15 '09 at 14:08
  • @MichaelPobega If you backup raw partitions as you're saying then you'd have to copy the entire disk size. Why copy 512GB when you've used only 80BG of your disk? With `rsync` you not only copy just what you have used, but also enable future syncing, so you can safely run a cron job for it. – Max Aug 03 '19 at 16:04
  • 1
    Coming back to this years later, my decade of experience has taught me that you are correct @Max. – Michael Pobega Aug 26 '19 at 16:37
4

You could achieve a total backup using sfdisk and dd.


To backup the partition scheme of each hard drive, you'd use sfdisk like this:

sfdisk -d /dev/sda  > parttable_sda.part

To backup each partition you could use dd, like so:

dd if=/dev/sda1 of=devsda1.img

Where /dev/sda1 is unmounted, such as with a live-CD boot.

(keep in mind you're going to need to have a a lot of free space to write this file; so you may want to write it to an external media) Do that for each partition, one at a time, and back everything up.


Then, to restore on another computer you can do:

sfdisk /dev/sda < parttable_sda.part
dd if=devsda1.img of=/dev/sda1    # do this for each partition
SDsolar
  • 157
  • 1
  • 1
  • 11
Michael Pobega
  • 934
  • 5
  • 12
  • 5
    WARNING: only do this if the partition is unmounted or mounted read-only. Dumping the raw contents of a partition while it is being written to can end up with a badly inconsistent filesystem on the backup (because blocks near the beginning of the filesystem are copied "earlier" than blocks near the end, and the filesystem algorighms do not expect this; you can avoid this problem if you can somehow do an atomic snapshot of the filesystem). fsck will not help you, since its algorithms also depend on the order the filesystem writes to the disk. – CesarB Oct 16 '09 at 20:42
  • dd is the way to go. Boot on a LiveCD, of course. And it is important to `dd if=/dev/urandom of=/dev/sdb bs=512 count=12` to wipe MBR and partition table of the target drive. – SDsolar Jun 16 '18 at 05:12
4

As pointed by this great community:

/dev /proc /sys /tmp /run /media /lost+found /boot ( /boot is optional see other comments)

For reference my final rsync command (under Arch with external media mounted in '/run/media/fred/INTENSO/' and backing up to a folder named 'fred') is:

$ sudo rsync -Pazhmxv --exclude /run/media --exclude /dev --exclude /lost+found --exclude /tmp --exclude /proc --exclude /boot --exclude /sys / /run/media/fred/INTENSO/fred/.

(excluded files could also be specified with curly brackets (--exclude={/dev,/proc}) under Bash or with a text file (--exclude-from='excude.txt')).

-P: show progress -a: archive mode -z: compress during transfer -h: output numbers in a human-readable format -m: prune empty directories -x: limit to one file-system -v: verbose

3

Instead of excluding, I usually only backup what I want. Including: /home /etc /var (except /var/log)

Nixphoe
  • 4,524
  • 7
  • 32
  • 51
DennyHalim.com
  • 491
  • 3
  • 10
1

Basically, pseudo-filesystems (/proc, /sys, /dev/shm...) need not be backed up.

sendmoreinfo
  • 1,742
  • 12
  • 33
1

I'm on a Ubuntu 18.04 machine and I have these excluded:

/dev/
/proc/
/sys/
/tmp/
/run/
/mnt/
/media/
/lost+found/
/cdrom/
/swapfile

Also, specificially for my setup, I exclude these:

/home            <-- Backed up separately
/backup          <-- Mount point for backup disks
/data            <-- Mount point for data disks, which are backed up off-site
/scratch         <-- Mount point for volatile fast SSD scratch disk
0

I generally make a habit of backing up everything on a system, even the stuff that I know for certain is useless to back up. It's simpler to set up and you can be 100% sure that you're going to get everything you do need included in the backup.

Maximus Minimus
  • 8,937
  • 1
  • 22
  • 36
0

I'm using an Ubuntu linux box as a testing server for website development, and for hosting a documentation wiki. Each night a crontab dumps the MySQL database into /var/www, and then all of /var/www is zipped up and replicated to the backup server. It's not ideal, but it's enough. I had to rebuild the server at one point, and all I really missed were the Apache and Samba configuration files.

Nic
  • 13,025
  • 16
  • 59
  • 102
0

I would assume you don't have Linux on a virtual machine. If at all possible, I would urge considering moving to virtualisation. Backups on vm level are a whole new level of consistency and ease of use. There exist free virtualisation tools, so you don't necessarily have to invest into VmWare or other expensive monster tool.

Gnudiff
  • 533
  • 5
  • 20
0

Question: Which directories should I exclude when backing up a server?

Here is a script I use often, from a Ubuntu 16.04 LTS laptop to a Ubuntu 16.04 LTS server. It clearly shows which directories should be skipped while making a full backup:

echo "EMPTYING TRASH"
rm -rf ~/.local/share/Trash/* >/dev/null 2>&1
echo "DELETING OLD LOGS"
sudo rm -f /var/tmp/* >/dev/null 2>&1
sudo rm -f /var/log/*.gz >/dev/null 2>&1
sudo rm -f /var/log/kern* >/dev/null 2>&1
sudo rm -f /var/log/messages* >/dev/null 2>&1
echo "DELETING CHROMIUM CACHE"
rm -rf /home/pi/.cache/chromium/Default/Cache/* >/dev/null 2>&1
echo "====================================================================="
echo "      BEGINNING RSYNC from PAV root to PRIME5:/mnt/full/pav"
echo "====================================================================="
time sudo rsync -aAXv \
          / \
          --bwlimit=500 \
          --delete \
          --delete-excluded \
          --ignore-errors \
          --exclude="/dev/*" \
          --exclude="/proc/*" \
          --exclude="/sys/*" \
          --exclude="/tmp/*" \
          --exclude="/run/*" \
          --exclude="/mnt/*" \
          --exclude="/media/*" \
          --exclude="/lost+found" \
          abc@prime5:/mnt/full/pav
echo "====================================================================="
df -h

Note the exclusion of /mnt - which is where each Ubuntu system has a full-time backup drive mounted for cron-based rsync self-backups 4 times a day. These drives are mounted by entries in fstab and are always present. To include them in a backup to another system would be duplicative.

Similarly, /media is where USB drives get mounted. They are backed up separately.

SDsolar
  • 157
  • 1
  • 1
  • 11