Is there a disk image file format that only store sectors which are filled (non empty)?

Question

I have a 500GB hard drive that contains several NTFS partitions. I would like to create a single disk image of all partitions. My ultimate goal is to mount that drive later and scan it for possible file recovery.

I would like to know if there is a disk image file format that allow to "compress" the disk image (it would be smaller than the actual size of the drive) by only storing the sectors which are not empty. I only know about the ISO file format but AFAIK this is a raw copy of the disk. I could compress that image later (eg : using gz compression) but it would make the image unusable (non mountable) unless the actual image is fully uncompressed first.

I ask this because the used space of the drive is only about 80GB. If I open the disk using an tool (eg : HxD) I see that about almost all sectors beyond a certains point (about 65%) contains null bytes (those sectors are empty).

EDIT: I am on Windows 10.

Answer 1

Some imaging programs do have their own formats which are made sparse at format level, not at filesystem level. Common image formats used by VM software (VHD, VHDX, Qcow2) almost always support holding partial data, as part of the "dynamically growing disk image" feature.

The same formats can usually be used for imaging physical drives and restoring them back, without having to involve a VM at all. (For example Microsoft distributes Disk2Vhd for imaging a physical disk into a VHD(X) image, which can then be restored onto another physical disk with various tools.)

However, you specifically asked for a mountable image. What is mountable heavily depends on the OS: for example, Windows can natively mount VHD(X) images but not raw dd images, while Linux is the opposite and its "loop device" system only supports raw images.

(Not sure about macOS capabilities, but it also has its own .dmg format.)

So with current OS versions (and avoiding third-party tools) a good choice would be:

• For Windows 7 or later – create a VHDX image using Microsoft Disk2Vhd.
• For Linux in general – create a filesystem-sparse "raw" image using dd or cp or ddrescue, as mentioned in Kamil's post. Note that if you losetup and mount it, you can use fstrim (yes, the same fstrim as used with SSDs) to turn a non-sparse image into a sparse one.
• For Linux when imaging an NTFS partition, you can also use ntfsclone which intelligently skips all areas which NTFS would consider "free", even if they happen to contain leftover non-zero data, allowing the image to be more sparse than what dd could produce. (This is what CloneZilla uses to create NTFS images as well.)

Answer 2

Saving a raw image as a sparse file will automatically take advantage of sectors that are filled with zeros. Some possibilities in Linux:

• dd conv=sparse (mind this)
• ddrescue -S
• cp --sparse=always

I don't know Windows tools. I know NTFS supports sparse files, so the general concept should work in Windows as well.

---

The method gets more effective if one overwrites the empty space with zeros first. By "empty space" I mean sectors not allocated to any file or metadata according to the filesystem(s) you want to store. But since your goal is to scan for possible file recovery, you shouldn't overwrite anything. This paragraph is for users interested in cloning/storing only non-deleted data.

Answer 3

All respectable backup products can create a disk image with only used sectors, but the format of the backup file not disclosed.

You may find a list of such products with reviews in the article Best Free Drive Cloning Software.

The product Clonezilla is open-source, so conceivably its format can be extrapolated from the sources (but what's the point).