How to "really" reduce the size of KVM VMs images?

Question

I've spent many hours searching the web about how to shrink KVM virtual disk images, especially for Windows guests, with no luck.

All I've found is to zeroize the VM free space, defrag the virtual disk (from Windows), and then run qemu-img convert -c ... (-c flag to compress).

I've a Windows 7 VM, with a 100 GB virtual drive size. Initially, this VDD spent 40 GB on the host storage. Once it's zeroized, the VDD eats for real 100 GB on the host. And qemu-img -c ... creates a 91 GB, which is not at all what I expected.

On the modernie web site, we can download W7 VMs which are less than 10 GB, how is this possible? Is there a way to "really" compress the VM images?

---

Thanks to @dyasny, I made a small test with virt-sparsity. I cleaned up the W7 VM disk, disable hibernation, so the VDD only consumes 20 GB. Degraded the disk again, and ran again sdelete -z. Running virt-sparsity with the --compress flag gives a 80 GB virtual drive. Far from what I would have hoped.

---

EDIT-2016-02-16: "Refreshing" this question because the method to shrink a VM discussed here is very efficient but has a major drawback: it deletes all VM snapshots. If someone knows how to shrink a VM while preserving snapshots, feel free to share!

Answer 1

To shrink a Windows Guest OS, you have to shrink the partition inside the guest, shutdown the VM, create a new smaller disk of the desired size, copy the data from the old disk to the new smaller disk, swap the disk names and reboot the VM.

It’s straightforward, yet if done improperly could lead to loss of data – and hair.

Here are the steps for KVM with a Windows Server 2012 guest of 100 GB that we want to shrink to 35 GB, using the QCOW2 format.

IMPORTANT: This method involves no modification of the virtual machine definition. Instead, it requires only disk image manipulations.

Assumptions for the guest:

• Guest is a Windows Server 2012
• Disk image of 100 GB in QCOW2 format
• Two partitions:
  • 350 MB of boot
  • 99.6 GB of C: drive with 20 GB of used space
• We want to shrink C: from 99.6 GB to 34 GB

Assumptions for the Host:

• Ubuntu 16 LTS server
• KVM (libvirt)
• 250 GB drive
• Virtual images located in /var/lib/libvirt/images

STEP 1: Preparation of the Windows Guest, shrinkage of the main C: partition

In this step, we'll just reduce our windows partitions directly from Windows. The resulting disk image at the end of this step will be the sum of the boot partition, the C: drive (reduced) and a leftover unused space that we will delete (by not copying it over to a new disk).

1. Login to the Windows Guest
2. Open the “Computer Management” utility, by using the start menu search function to locate it.
3. On the left side, click on “Storage->Disk Management” Storage Disk Management screenshot
4. On the new screen, right click on the C: partition, click on “Shrink Volume…” which should take a little bit of time before a dialog appears. Be patient.

5. Once the “Shrink C:” dialog window appears, enter the amount of space in “Amount of space to shrink” that makes the “Total size after shrink in MB” value approach the desired 35 GB. Then click “Shrink”.

   NOTE: You may get an error message if the new space is too small, in this case you should reduce the “Amount of space to shrink” by 1GB incrementally until the error disappears and the shrinking takes place. In practice, we like to keep 10 GB of free space.

   Let’s assume you were able to shrink the C: partition to 34 GB.

6. Once done, shut down the VM by opening a command prompt and typing: shutdown /s /t 0

7. Your windows guest is ready.

STEP 2: Shrinkage of the disk on the VM host

The procedure is not really a shrinkage, but instead we're going to create a new disk (of the final size) in which we will copy the two partitions from the original disk, and skip carrying over the unused space.

The goal is to create a disk whose total size = boot partition + C: partition. We'll also end up with some tiny leftover space (unless your math was perfect) not to worry about because we'll deal with in the last step.

1. Login to the linux host
2. switch to superuser: sudo su
3. go to where the virtual images are stored: cd /var/lib/libvirt/images
4. list the files: ls -l
5. Find your guest image (plenty of tutorials on that elsewhere). Let’s assume our windows guest image is called “windows.qcow2”

6. we make a backup:

   mkdir backup
   cp windows.qcow2 backup/windows.qcow2.bak
   

   (go have coffee because this will take a while for a large disk)

7. install guestfs packages you might be missing:

   apt-get install libguestfs-tools
   

8. Alright, let’s double check our windows disk by exploring the windows image with virt-filesystems:

   virt-filesystems --long --parts --blkdevs -h -a windows.qcow2
   

   which outputs this:

   Name       Type       MBR  Size  Parent 
   /dev/sda1  partition  07   350M  /dev/sda
   /dev/sda2  partition  07   34G   /dev/sda
   /dev/sda   device     -    100G  -
   

   Notice that we have /dev/sda1 which is our windows boot partition of 350 MB, /dev/sda2 which is our C: partition of now 34 GB and that the total disk image /dev/sda/ is of 100 G leaving us with a bunch of space to trim.

   So here is the important step: do your math: 34 G + 350M fits in 35 G, therefore we are going to create an image of 35 GB. We’ll inevitably end up with some leftover space – unless your math is perfect – but don’t worry about it, we’ll deal with it below.

9. let’s create the new virtual QCOW2 disk that we are calling newdisk.qcow2 of total size 35 GB:

   qemu-img create -f qcow2 -o preallocation=metadata newdisk.qcow2 35G
   

   which outputs:

   Formatting 'newdisk.qcow2', fmt=qcow2 size=37580963840 encryption=off cluster_size=65536 preallocation=metadata lazy_refcounts=off refcount_bits=16`
   

10. Let’s resize the disk by copying the old disk into the newly allocated one. This is the bit that is absolutely awesome. Most other guides show some awfully complicated stuff. This is simply done by this command after which you should go get more coffee – it’s likely going to take a a while:

    virt-resize windows.qcow2 newdisk.qcow2`
    

    which outputs this:

    [   0.0] Examining windows.qcow2
    100% ?¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦? --:--
    **********
    Summary of changes:
    /dev/sda1: This partition will be left alone.
    /dev/sda2: This partition will be left alone.
    There is a surplus of 439.8M.  An extra partition will be created for the surplus.
    **********
    [   8.8] Setting up initial partition table on newdisk.qcow2
    [   9.9] Copying /dev/sda1
    100% ?¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦? 00:00
    [  15.1] Copying /dev/sda2
    100% ?¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦? 00:00
    Resize operation completed with no errors.  Before deleting the old disk, carefully check that the resized disk boots and works correctly.
    

    Notice the tool found the surplus of space... recall the comments about Math... So you can cancel that and recreate the disk or just move on as we do here and expand the sda2 partition as is done on STEP 3.

11. Once done. Inspect the resulting image:

    virt-filesystems --long --parts --blkdevs -h -a newdisk.qcow2
    

    which outputs this:

    Name       Type       MBR  Size  Parent
    /dev/sda1  partition  07   350M  /dev/sda
    /dev/sda2  partition  07   34G   /dev/sda
    /dev/sda3  partition  83  439.8M   /dev/sda
    /dev/sda   device     -    35G  -
    

    Notice how the type of the /dev/sda3 is of type linux for the leftover space. Leftover space is OK, unless you did your math exactly right. We’ll deal with this extra partition from the windows guest further below. Right now, just ignore it.

12. Swap the disk images:

    mv windows.qcow2 backup/
    mv newdisk.qcow2 windows.qcow2
    

13. Start your VM.

STEP 3: Finalization of the disk operation on the Window Guest

In this step, we're confirming windows boots fine, and we're going to expand our C partition into the extra bit of space.

1. Login to the windows guest

2. Open the “Computer Management” utility, by using the start menu search function to locate it.

3. On the left side, click on “Storage->Disk Management”

4. You should see 3 partitions: boot, C: and a small 439 MB partition (to the far right). Screenshot of Computer Management showing the 3 partitions

5. Delete the linux partition by right click->delete volume. (click yes to any prompts)

6. Right click on the C: partition and click on “Extend”, then Next and OK on the dialogs. It should only offer to extend by the amount of the last partition. Once done you have resized C: and be left with only two partitions.

7. That’s it. Your windows guest is now using only 35 GB or so. Remember the actual disk image may be larger (it could be closer to 38 GB) due to all the overhead, etc.

Check that everything works fine and delete your image backups or move them offline to storage.

Answer 2

I finally managed to really shrink the VM space. At the beginning, the W7 VM ate 107 GB on the host storage. The virtual HDD size is 100 GB and currently, the VM only eats 18 GB of its virtual storage.

Here is what I did:

1. Clean up the virtual drive (remove temps files, etc)
2. Defrag with the open source UltraDefrag software with "full optimisation"
3. Run sdelete -c c:
4. Run sdelete -z c:
5. Run qemu-img convert -c -f qcow2 w7-64.qcow2 -O qcow2 w7-64-compressed.qcow2

This way, the qcow2 file was shrunk from 107 GB to...7 GB!

Answer 3

When you run qemu-img -c, you compress the image, which, while being able to reduce some space, can really hurt performance. If you want to deduplicate the zeroes on the disk, you need to run qemu-img convert, basically as if you're trying to convert the image from one format to another (even if the src and dst formats are the same).

This process will write a new converted image, sans the zeroes, effectively deduplicating the zeroed space on the drive.

Another option would be to simply use virt-sparsify of course.

Answer 4

The method that worked for me for a Windows VM on KVM is as below.

On the Windows guest:

1. Defragment if necessary.
2. sdelete -z c: (sdelete -z is for VMs. It does not increase the actual QCOW2 image size on disk and zeroes out empty space. sdelete -c on the other hand is not required and it will increase QCOW2 image size on disk)

On the Linux host:

1. qemu-img convert -O qcow2 -c inputVM.img outputVM.img (note that this does not preserve snapshots and will take the current state)

Answer 5

In Xenial and Bionic, the utility virt-sparsify from package libguestfs-tools should work. Note that:

• You DO NOT have to run a tool like sdelete inside the guest beforehand (but it won't hurt)
• You can use the --in-place flag to free up space without copying the file (useful if the disk image is already larger than the remaining free space on your drive!)
• The tool supports qcow2 and raw format images

Answer 6

You just need to "hole punch" or "sparsify" the empty space. To do that, you need the space to contain only 0's and holes. A filesystem's "empty space" is just unallocated, but may contain old junk data, not 0's. So the first step is to zero it. There are tools to do that, but here's an easy minimal way to do it...

1. Boot the vm.

2. Make a file with only 0's on each filesystem, and remove it. Here I assume /tmp is writable by you and is on the rootfs. (in windows you could do the same command in cygwin, or use another tool.)

   dd if=/dev/zero of=/tmp/zeros bs=1M
   rm /tmp/zeros
   

   Now the empty space is just 0's. But also, this disk is now the full size on disk... the opposite of what you want in the end. So this won't work if you were out of space before.

3. Stop the VM.

4. Punch holes. There are a few ways to do this... here is a fast way, using a python script. First stop the vm, then run the script on the disk file(s). If it's a qcow2 file or another format, it should work the same, but there might be something I am forgetting, or simply an easier way.

And be aware that a hole is not allocated, so the file is not all in one place; the filesystem may become fragmented, hurting performance. This should not be in any way noticable on typical Linux/UNIX filesystems unless you were very low on space while writing files, but just be aware of the possibility. It is recommended to keep at least 10% free space to avoid fragmentation.

Also, there are tools that do other things too... like zeroing only the non-zero empty space (so they don't grow before you punch holes), zeroing swap too, doing it while online (probably requires hypervisor support), etc.. I tried these ways and found they all were terribly unreliable, sometimes barely shrinking 5% as much as manually zeroing it does, so I won't even bother listing the tools; others can list their favorites.

Answer 7

You may want to disable System Restore or delete any existing volume shadow copies in the VM disk. That alone can take up lots of space, and will appear hidden to the filesystem. Then run sdelete and zero out the free space, after a vm reboot.