Ability to use dd to perform a partial image backup?

2

This question is related to one from several years ago, where I was looking for a way to do a partial image backup. That piece of it was never answered, so this is a dedicated question for it.

The Problem

  • I planned to shrink a Windows partition and wanted to first make an image backup, as insurance. If recovery was required, the backup would need to be restored to the shrunken partition.

  • Since the purpose was insurance for the system as-is, the backup had to be the "virgin" contents, prior to any kind of modification that would change anything or introduce any new risks.

  • The Windows partition shrink tool tells you ahead of time how much it will shrink the partition. I wanted an image of only the portion of the partition that would remain after shrinkage.

  • I had a file-based backup, so I was not concerned about any files that might be outside the shrink area and potentially not captured by the backup.

Solution Concept

Conventional wisdom says that these requirements can't be met using dd. However, knowing just enough to be dangerous, there is an approach that seems like it ought to be a solution (I'm assuming a dd-based solution would be performed from within Linux).

The idea:

  • dd has parameters available to specify starting and ending locations.
  • Any partition management tool will reveal where that partition sits on the drive.
  • The Windows partition shrink tool displays how much it will shrink the Windows partition.

That would seem to be the raw material for calculating the dd parameters. Putting the pieces together is something I haven't done before. So the question is:

  • how to determine the dd parameters reflecting the shrunken portion of the partition
  • the dd command to copy that relevant portion to a backup drive
  • the dd command to restore that image to the same place on the original drive

Caveat

Defining and copying a desired section of a partition off bare metal looks like it ought to be straightforward. However, one point that isn't clear is whether the NTFS MFT is tied to the original partition size, so that restoring it to the shrunken partition would leave it corrupted, at least beyond the ability of something like chkdsk to repair.

Question Objective

The ultimate objective is to create a partial image backup of a Windows partition as described. This problem never attracted solutions in the original question, so this question is intended to offer the seed of a potential approach. However, my concept may not work. Thus, while a dd solution is the focus of this question, I wouldn't preclude an alternative solution using built-in or commonly available free tools. So a successful answer can be:

  • A working dd-based solution
  • An authoritative and definitive explanation of why a dd-based solution cannot possibly work (or at least cannot be trusted to work reliably)
  • An alternative solution that accomplishes the objectives.

Addendum

Kamil Maciorowski's answer identifies another question, Clone only space in use from hard disk, that is fairly similar, but I don't believe it is a duplicate. Some key differences:

  • That question seeks to clone only used space. This one seeks to clone a specific section of a partition.
  • That question doesn't have any particular constraints on getting to the result. This one requires that the image be of the pristine current state, with no alteration of, or risk to, the content, which precludes any type of "preparation".
  • That question is about an entire drive and a Linux filesystem. This one is about a section of a partition and NTFS.

fixer1234

Posted 2017-03-20T03:40:58.403

Reputation: 24 254

Answers

1

I may say it's XY problem and the real question is: How to clone only space in use from hard disk (or partition)? I don't want to point to another answer without explanation why it should be better than your proposed idea; so here it is:

In general it is not guaranteed that the free space aggregates at the very end of the filesystem, even after defragmentation. I don't know NTFS nor FAT family good enough to be sure, but I suspect there may be some data structures that occupy far sectors of large filesystem, regardless of how empty it is.

Therefore I am skeptical about the idea of saving a partial image only. Maybe there are filesystems that would survive this, maybe NTFS is one of them -- I don't know.

Many times I have done something similar to what you want to achieve, mostly with NTFS. My way works regardless of where in the filesystem the free chunks are. Basically:

  • write zeros to a file until there is no space left,
  • sync,
  • remove the file of zeros,
  • umount,
  • take image to a sparse or compressed file.

It can take a lot of time. Make sure the filesystem is healthy beforehand. Also don't do this if you suspect the disk to be faulty.

You can find more details in my answer to already mentioned question. The important difference is you will be reading your single partition with dd, not the entire drive.

Unfortunately such an image cannot be restored to the shrunken partition. The image will be useful if shrinking process fails and you lose your original files. I think you need a tool that can backup files (and metadata inside a filesystem) to be able to restore to a smaller partition. Well, dd is not such a tool.

Kamil Maciorowski

Posted 2017-03-20T03:40:58.403

Reputation: 38 429

Your answer there is interesting and I'm still digesting it. A couple of considerations in comparing the two cases. In my task, I was going to use the Windows partition shrink tool after full defrag. Windows sticks some non-movable stuff near the middle of the partition. You can often deal with that manually, but I wasn't going to mess with it. Simply running the shrink tool shrinks the partition to the unmovable stuff, which frees up about half the space, and that tool normally does a clean job. I haven't run into a situation with a near-empty drive where (cont'd) – fixer1234 – 2017-03-20T08:28:46.603

there was file content beyond the immovable stuff that the shrink tool made disappear. I don't recall whether the tool relocates outlier movable stuff to below the immovable stuff or just limits shrinkage to the "outermost" content. What would remain after shrinkage was the target for backup, and the shrink tool tells you ahead of time where it will shrink to. If there was file content in the nether regions of the partition that the shrink tool would relocate, my backup beforehand could miss it. But that wasn't a concern because it I had a file backup, (cont'd) – fixer1234 – 2017-03-20T08:29:40.250

and nothing in that area of the partition would be critical. The other consideration is that I wanted to backup exactly what was there before doing anything other than defrag. Shrinking, itself is pretty safe, and that's what I wanted insurance for. So any kind of preparatory changes defeats what I was trying to accomplish. re: your last paragraph, is the issue that the NTFS MFT is structured for the entire partition, so pasting the image back into a smaller partition leaves the MFT corrupted (placeholders or pointers to non-existent space)? – fixer1234 – 2017-03-20T08:29:49.137

@fixer1234 My basic concern is there may be some information (metadata maybe?) in the area you want to omit, that will be silently relocated at the exact moment of shrinking, not before, so your image will lack it; or there may be some information (metadata) that is adjusted accordingly at the exact moment of shrinking to correspond to the smaller filesystem size, so your partial image will be inconsistent. But as I said: I don't know enough about NTFS to foresee whether or not it will survive. – Kamil Maciorowski – 2017-03-20T08:58:22.170

I'm still thinking about your other answer. It's a good answer for that question and I upvoted it there. I don't think that solution will work for these requirements, though. You might be right about no good solution here. Let's see what ideas, if any, come in. In the meantime, thanks for the input. I appreciate your thoughts on this. – fixer1234 – 2017-03-20T11:03:22.990

@fixer1234 I'm curious of other ideas too. I read about NTFS today and now I think your unorthodox approach might work (I'm not sure though). – Kamil Maciorowski – 2017-03-20T11:43:29.553