10

I want a robust method of taking backup of KVM VPS of qcow2 file, while running. After searching for a long time I found that we can use rsync for live backup. I used following command for rsync:

rsync -avh /vms/base.qcow2 /backup/backup.qcow2

It worked properly. But I am afraid it may corrupt backup qcow2 image if backup is taken while VPS is being changed due to write and delete operations (Like installing new software or update).

Therefore taking backup from snapshot of live VPS seems more interesting option.

I used following commands for taking backup of running VPS:

1.Create Snapshot:

qemu-img create -f qcow2 -b base.qcow2 snapshot.qcow2

2.Convert snapshot in raw image format (.img) :

qemu-img convert -O raw snapshot.qcow2 /backup/backup.img

To restore VPS :

3.Convert raw image in qcow2 image:

qemu-img convert -O qcow2 /backup/backup.img base.qcow2

Then we can start VPS by virsh create base.xml command.

Even though this method also work perfectly I am still not sure if this is proper method to take backup of running VPS. I found nice documentation on qemu-img commands here and here. But they didn't mention anything about converting snapshot of live VPS in raw image.

Which method is more suitable for live VPS backup? Is converting snapshot in raw image safe?

If I could get nice detailed documentation on how snapshot and base files behave in certain conditions that will be very helpful.

Maybe we can also convert base qcow2 image into raw backup file instead of converting snapshot. So command flow will be :
1. Create snapshot
2. Convert base qcow2 into backup raw image
3. Commit snapshot changes into base image
4. Delete snapshot

I tried this and it worked fine on small VPS of 5GB. But is there any possibility of error or VPS corruption when VPS size is huge?

Thanks in advance...

Community
  • 1
Bosco D'mello
  • 101
  • 1
  • 1
  • 4
  • 2
    That's one of the reasons I like having my virtual machines reside on logical volumes. They have inherent snapshot support. – Halfgaar Aug 18 '15 at 09:02

2 Answers2

13

The only thing you are missing is to quiesce the guest filesystem before taking the snapshot, to ensure that it is consistent. This can be done with virsh domfsfreeze if you are using libvirtd.

For example, the order of operations is:

# Freeze guest filesystems
virsh domfsfreeze $VM_NAME

# Create snapshot
qemu-img create -f qcow2 -b $VM_NAME.qcow2 snapshot.qcow2

# Thaw guest filesystems
virsh domfsthaw $VM_NAME

# Take backup from snapshot
qemu-img convert -O raw snapshot.qcow2 /backup/backup.img

Note well that this requires the qemu-guest-agent be installed and running in the VM. Depending on the distribution and installation choices, this may or may not be the case.

The qemu-guest-agent is also extensible; you can add scripts to it to support specific software. For instance, it includes a sample script to put MySQL in a consistent state.

Michael Hampton
  • 237,123
  • 42
  • 477
  • 940
  • The only thing this answer is missing is snapshot removal after backup :) – dyasny Aug 18 '15 at 15:16
  • I guess using `virsh domfsfreeze` will prevent any write operation on VPS and that means there will be downtime for running VPS. – Bosco D'mello Aug 19 '15 at 05:53
  • 2
    @BoscoD'mello Unless you have very bad hard drives, it should be less than a fraction of a second. It's not worth worrying about. – Michael Hampton Aug 19 '15 at 05:55
  • I tried using `virsh domfsfreeze` but it shows 'unknown command' error. I didn't find any reference of domfsfreeze in virsh man page. Please let me know about how can I use that command to freeze VPS file system. – Bosco D'mello Aug 19 '15 at 10:24
  • @BoscoD'mello Just how old is your system? – Michael Hampton Aug 19 '15 at 17:12
  • @dyasny (or @Michael-Hampton) , are there any gotchas with removing a snapshot or is it as straightforward as `rm snapshot.qcow2`? EDIT, looks like virsh has a separate command for deleting a snapshot: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/sect-Managing_guest_virtual_machines_with_virsh-Managing_snapshots.html#sect-Managing_snapshots-snapshot_delete_domain `virsh snapshot-delete $VM_NAME` – a coder May 23 '17 at 14:34
  • In order to delete a snapshot in the middle of a COW chain, you need to run the virsh command, so it's contents can be merged in the underlying image – dyasny May 23 '17 at 14:45
  • 1
    Alas, the snapshot did not work for me. When attempting to `virsh domfsfreeze $VM_NAME`, I get permission denied on my guest filesystem. My guest filesystems are encrypted, not sure that matters. – a coder May 23 '17 at 15:10
  • If the virsh domfsfreeze is going to interrupt a running KVM instance, wouldn't it be better to schedule a shutdown of the instance for backup? I'm asking cause I'm working on this issue myself. – Edward_178118 Dec 29 '19 at 13:36
  • 2
    @Edward_178118 It depends on whether your workload can stand to be suspended for a fraction of a second. – Michael Hampton Dec 29 '19 at 19:49
  • @MichaelHampton Is there any particular operation a KVM instance might be performing at the time a virsh domfsfreeze is executed that it might not be gracefully frozen and thawed to resume operating normally? – Edward_178118 Dec 30 '19 at 06:29
  • 2
    @Edward_178118 Extremely low latency network operations, perhaps? I've never run into anything that was a problem. – Michael Hampton Dec 30 '19 at 07:00
2

There is another approach which involves creating temporary snapshots during the backup and committing them back afterwards.

# Create a temporary backup snapshot which records changes during backup
virsh snapshot-create-as vm backup-tmp --disk-only --atomic --quiesce --no-metadata --diskspec sda,file=/tmp/sda_temp.qcow2
# Now the main image is frozen and the changes are going into /tmp/sda_temp.qcow2

# Do your backup here
...

# Now we commit back data from temp snapshot into the main image
virsh blockcommit vm sda --active --pivot
# Remove the temporary snapshot file
rm /tmp/sda_temp.qcow2

For more information see: https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit

Thuffir
  • 21
  • 2
  • 1
    If you need your backed up disk again later, which comes from a running vm, how would you restore? Don't you need the ram contents too for restoring? – Adrian Zaugg Dec 26 '19 at 00:30