1

note: all = all that i've found

All the "cloning with dd" information talks about how dd copies all the "empty space", but literally all writings seem to be responding to a situation where filesystems fill up the entire disk.

What about the situation where i have two partitions that together fill up only 10% of 160G disk. What is the image size going to be for the good ol "dd if=/dev/sda of=disk.img" run: 16G or 160G?

Unless i'm blind, Using DD for disk cloning does not have this information.

lkraav
  • 756
  • 1
  • 8
  • 21

2 Answers2

2

sda is the disk as a whole - 160 GB

sda1 is the first partition on that disk - 8 GB (say)

sda2 is the second partition on that disk - 8 GB (say)

If you only want to clone the 16 GB of filesystem data, use dd on sda1 and sda2.

To restore the data to a different disk you'd have to first partition that disk (ideally identically to the original disk). I expect you'd probably have to reinstall the bootloader too.

See, for example, http://www.backuphowto.info/linux-backup-hard-disk-clone-dd

RedGrittyBrick
  • 3,792
  • 1
  • 16
  • 21
  • yeah, if the goal is to have minimum whole-disk image size with a single dd command, you are saying there is no difference to dd whether empty space resides inside filesystem or in non-partitioned space. right? – lkraav Nov 06 '10 at 16:55
  • `dd` can't tell the difference between "empty space" and space filled with data. You can minimize the image size by compressing it with `gzip` but obviously this takes time. I suppose you could minimise the size of a gzipped dd of sda by creating a new partition to occupy the remaining disk space and filling that partition with zeroes or some regular pattern. However I doubt there are many cases where this would be worthwhile. – RedGrittyBrick Nov 06 '10 at 17:04
1

First I want to answer to your question:

what about the situation where i have two partitions that together fill up only 10% of 160G disk. what is the image size going to be for the good ol "dd if=/dev/sda of=disk.img" run: 16G or 160G?

If you run dd on /dev/sda, i.e. the whole disk sda of 160G, then disk.img will be 160G, since dd will just copy all bits on sda one by one. It makes no difference if the space after /dev/sda1 and /dev/sda2 is partitioned or not! Unfortunately, if the hard disk ever was full of data before, even if you try to compress the disk.img, it will not really get smaller because of the fact that the "empty space" on the disk surface in reality is not empty but contains the leftover bits of the previously used file systems and data. I do not know what the case for brand new hard disks that have never been used before is, though!

Additionally to this, I want to answer to your comment that you gave to RedGrittyBrick's answer. If your goal is to make a minimum size whole-disk image, then a possible solution is as follows

  1. Create a third partition with the empty space, in your example: /dev/sda3. There is no need (but also no harm) to format it with a file system since the partition will be overwritten with zeros in the next step, eliminating any file system formatting again

  2. Now fill the entire partition with zeros: dd if=/dev/zero of=/dev/sda3. This will take quite a long time and dd has no progress bar, so just be patient. At some point dd will report that the disk is full and some time after that will exit with a final report about the bits copied

  3. If you do not want this partition to exist, you can delete it again now. The zeros will remain on the disk surface!

  4. Now make the single disk image as you where planning in first place: dd if=/dev/sda of=disk.img. By the way: There is no need for any further dd-parameter whatsoever, such as conv=notrunc, etc. as reported in many other forums. notrunc has absolutely nothing to do with what we want to do here (clone a disk to a file) and is used for something totally different (in short: Insert data in a file that is already existing without cutting off [=truncating] the remains of the existing file should the newly added data be shorter than the existing file). Do not listen to those who write that notrunc is especially needed when cloning disks to a file, that is 100% bollocks

  5. The disk.img will be 160GB again, no surprise here. But the difference now to before is that if you compress the disk.img, this time it will almost become the same size as the sum of the compressed images of /dev/sda1 and /dev/sda2 would be, since the zeros on the remaining unpartitioned disk surface will compress down to practically nothing!

  6. There you go, now you have a single-file full disk image of the whole disk but which is almost only the size of the really used data area of /dev/sda1 and /dev/sda2

  7. In the case that you would not have created /dev/sda1 and /dev/sda2 yet, you could also just write zeros with dd to the entire disk /dev/sda first, or use a disk wiping tool that writes zeros to an entire disk, etc. and then just create /dev/sda1 and /dev/sda2. The result should be the same

HTH! Best regards An anonymous internet native on planet earth

Anonimiatis
  • 11
  • 2
  • Or you can instead truncate the data at the right place (`head -c …` for example). Then even the uncompressed image doesn't need to be 160GB; and writing back incomplete raw image is totally fine; if you really need, you can always fill with zeroes the remaining space when restoring. The tricky part is to figure out the correct size… – Display Name Jun 18 '18 at 07:48