Convert NTFS partition to ext4 - How to copy the data?

1

1

I wanted to convert my D:\ partition with all my data from NTFS to ext4 and the internet told me that I should do the following:

  1. Shrink all partitions to a minimum size
  2. Back up my data (done by cloning the partition in GParted)
  3. Format the partition to ext4 (Not done yet, but not that hard)
  4. Copy the backup to the drive
  5. Delete backup and grow all partitions

And I don't know how to do part 4. Can I really copy a whole drive like I would do for folders? Are there tools specialized in this? How can I make sure I don't miss any folders hidden by the system or symbolic links etc.? I have an Ubuntu live USB-Stick to work from if that is important.

piegames

Posted 2017-11-26T16:33:09.720

Reputation: 137

2You don’t know how to copy files? – Ramhound – 2017-11-26T17:29:34.623

I do know how to copy files. The question is about if I can copy whole drives like I do for files or if there are special things to take care of to not break the system. – piegames – 2017-11-26T20:39:00.750

There are tools that read each sector of a drive and outputs an image of said drive. – Ramhound – 2017-11-26T20:40:38.267

Answers

4

1. FreeFileSync GUI tool (Windows, Mac, Linux)

To ensure you don't miss any data, I highly highly recommend using a good file-copy tool like FreeFileSync. It is graphical, no cost, free and open source, and cross-platform (Windows, Mac, Linux). Just be sure not to screw it up and copy backwards, deleting everything! That's not hard to get right, however, since it's graphical and very user-friendly.

Just set up your folders with source on the left and destination on the right, then click the gear at the top right and choose the "Mirror" option to mirror from left to right, then press the "Compare" button at the top left. Once it determines what it needs to copy, it will pop up an info screen showing what it's going to do (how many files it will create, overwrite, or delete, etc.). Ensure you see nothing next to the trash can (indicating what is to be deleted), or that you see expected files to be deleted, as marked with the trash can and a minus sign, and then press the "Synchronize" button to begin. Voila! Easy to use. Here's a screenshot: enter image description here

2. rsync Command-line tool (Linux, Windows with Cygwin) (AKA: "How to use rsync"):

If you are doing the copy from Linux, you may also use FreeFileSync of course, but if your'e a command-line guru you might like rsync better. It's a command-line-only tool, and harder to use, but still very powerful, albeit not quite as powerful as FreeFileSync since it doesn't handle two-way syncing very well. Here's a few of my favorite rsync commands:

  1. COPY from source to destination, leaving anything in the destination that isn't conflicting still intact:

    sudo rsync -rah --stats --info=progress2 '/path/to/source' '/path/to/destination'
    

    Optionally, add --verbose, or just -v (or -rahv in my case) to also get a full list of exactly what will be copied.

    Optionally, add --dry-run to show what would happen but NOT ACTUALLY DO IT YET.

    Optionally, you may also log the entire list of files to be copied to a stdout.log file like this, and any errors to a stderr.log file like this. Here I remove the progress indicator by removing --info=progress2, just so the log files don't get polluted with progress info everywhere:

    (sudo rsync -rah --verbose --dry-run --stats '/path/to/source' '/path/to/destination' | tee -a '/path/to/stdout.log') 3>&1 1>&2 2>&3 | tee -a '/path/to/stderr.log'
    

    Remove --dry-run from the above when ready to actually make the copy. Be sure to review the stats paragraph printout (which printed because you had --stats) at the end of the stdout.log file before running the script for real, to make sure it all looks good.

    Here's an example of a --stats output obtained from doing the MIRROR option below. Notice it is deleting some files. Pay careful attention to that, as deleting files can't be undone (at least not without special software recovery tools). Just keep in mind it is permanently deleting them, NOT sending them to some sort of recycle bin or something, so watch out!

    Number of files: 102,033 (reg: 89,638, dir: 9,867, link: 2,502, special: 26)
    Number of created files: 3 (reg: 3)
    Number of deleted files: 9,861 (reg: 9,360, dir: 474, link: 27)
    Number of regular files transferred: 25
    Total file size: 8.12G bytes
    Total transferred file size: 35.30M bytes
    Literal data: 35.30M bytes
    Matched data: 0 bytes
    File list size: 917.43K
    File list generation time: 0.007 seconds
    File list transfer time: 0.000 seconds
    Total bytes sent: 38.33M
    Total bytes received: 1.02M

    sent 38.33M bytes received 1.02M bytes 7.15M bytes/sec
    total size is 8.12G speedup is 206.37

  2. To MIRROR from source to destination, which means also deleting anything in destination that is not in source (notice the "Number of deleted files" line in the stats output shown just above), add --delete --delete-excluded (even though we aren't excluding anything yet, once you learn about the --exclude=PATTERN syntax you'll also realize you want the --delete-excluded flag).

    First, the safe dry-run version:

    (sudo rsync -rah --verbose --dry-run --stats --delete --delete-excluded '/path/to/source' '/path/to/destination' | tee -a '/path/to/stdout.log') 3>&1 1>&2 2>&3 | tee -a '/path/to/stderr.log'
    

    And now the danger danger danger--could delete everything if you do source and destination backwards version (so check the stats output at the end of stdout.log before continuing):

    (sudo rsync -rah --verbose --stats --delete --delete-excluded '/path/to/source' '/path/to/destination' | tee -a '/path/to/stdout.log') 3>&1 1>&2 2>&3 | tee -a '/path/to/stderr.log'
    

    And now the version that doesn't save to any log files in case you want to just see the live progress and the stats paragraph at the end, but NOT have a log of individual files copied or deleted in stdout.log, or errors printed in stderr.log:

    sudo rsync -rah --info=progress2 --stats --delete --delete-excluded '/path/to/source' '/path/to/destination'
    
  3. And this is just the beginning of rsync. Read the manual: man rsync, or online: https://linux.die.net/man/1/rsync, and keep in mind you can mix and match many of the options I've shown above. Other really useful things to look into include setting up a source.txt file, and an exclude.txt file, to include and exclude, respectively, lists of files and paths. Do this with the option --files-from='/path/to/source.txt' and --exclude-from='/path/to/exclude.txt', respectively. This is extremely useful. Lastly, this goes really well with the --relative option. In the end, a full Bash script might look something like this:

     (sudo rsync $OPTIONS $SRC $DEST | tee -a $LOG_STDOUT) 3>&1 1>&2 2>&3 | tee -a $LOG_STDERR
    

    Where $SRC is your source path (in this case, a relative path for all files in the source.txt file, such as simply / to make everything relative to your root directory), $DEST is your destination path, $LOG_STDOUT is your path to where you'd like to log your stdout info, $LOG_STDERR is your path to where you'd like to log your stderr info, and $OPTIONS might look something like this:

    -rah --verbose --stats --relative --info=progress2 --delete --delete-excluded --dry-run --files-from='/path/to/source.txt' --exclude-from='/path/to/exclude.txt'
    

    And, of course, as stated above, you might want to remove --verbose, --relative, --info=progress2, --delete and --delete-excluded, and --dry-run, as desired, depending on what you're doing.

Related:

  1. [MY ANS] https://unix.stackexchange.com/questions/65077/is-it-possible-to-see-cp-speed-and-percent-copied/567828#567828

Gabriel Staples

Posted 2017-11-26T16:33:09.720

Reputation: 625

1I just tried FreeFileSync based on this answer and it works really well for my use-cases (mirroring and synching directories with over 1 million files each. – Reb.Cabin – 2019-12-11T20:48:18.190