Backing up Zimbra 8.0.x, data.mdb issues

Question

When trying to create a backup of zimbra open source edition 8.0.1+ I ran into issues with data.mdb 'exploding'.

While my /opt/zimbra directory is only about 8GB my backup is around 80GB.

both rsync and copy fail to work properly.

Answer 1

The data.mdb file is actually a thin provisioned 86GB (by default) file.
Rsync and sometimes cp commands will 'explode' the file to it's full size when copying to a backup directory.
This obviously doesn't help if you want to create an offsite backup or if you are limited in resources.

This behavior seems new in 8.0.1

The proper way to prevent this is to split your backup into 3 parts. Below my example backup script (to be executed as root).

Frist Stop zimbra and kill whatever is left.

#!/bin/bash
# Zimbra Backup Script
echo 'Starting Zimbra backup script'
date
echo 'Stopping Zimbra'
su zimbra -c '/opt/zimbra/bin/zmcontrol stop'
echo 'Kill Zimbra'
ps auxww | grep zimbra

Next backup the /opt/zimbra directory EXCLUDING the /opt/zimbra/data/ldap directory.

echo 'Saving base'
rsync -aHK --delete --exclude '*data/ldap*' /opt/zimbra/ /backup/zimbra-base

Next convert the ldap files into ldif files with the build in zmslapcat command.
You can choose to first start the server again as this command can be ran while zimbra is running. but just to be sure we get a backup of a frozen environment we'll leave it stopped for now.

echo 'Saving ldap config'
su zimbra -c '/opt/zimbra/libexec/zmslapcat -c /backup/zimbra-ldap'
echo 'Saving ldap main'
su zimbra -c '/opt/zimbra/libexec/zmslapcat /backup/zimbra-ldap'
echo 'Restarting zimbra'
su zimbra -c '/opt/zimbra/bin/zmcontrol start'
echo 'Finished'

This script was tested on zimbra 8.0.1 under ubuntu server 14.04.
It should work on most distributions and versions above 8.0.1

Answer 2

Depending on which tool you use for backups, some do support sparse files, though that support isn't as quick as the real data would suggest.

rsync -S  # it works, but it isn't "efficient"
cp --sparse=always
tar -S    # the GNU version.

Unfortunately, my favorite backup too, rdiff-backup, doesn't have sparse file support currently. I have seen a patch to make it support sparse files, but the devs haven't pull'ed. I haven't done anything with it either.

Duplicity doesn't appear to have sparse file support either. I didn't look very deep.

There are probably others which support it. I know that moving a qcow2 image around with a sparse file inside worked as expected.

As for backups, with something as complex as Zimbra, I hope everyone is using LVM2 snapshots or ZFS snapshots to avoid downtime, then run the backup against the snapshot storage.