31

I would like to write a simple backup script that saves some data to a FAT drive. Should I reformat the drive and use a better file system or is it possible to use rsync with FAT? If so, what problems might I run into? Would performance be a lot worse?

EDIT: This is on linux, didn't even know there was a rsync for windows. The sources are various file systems (it's a mess), and the destination is currently formatted with FAT32.

Thank you for your answers, I'll probably go for a reformat, since I'm not completely sure about the file sizes we'll have.

Kim
  • 729
  • 1
  • 5
  • 12

5 Answers5

44

I use rsync to backup my photos I store and process on laptop running Linux (Ubuntu 10.4). I backup them to a very basic NAS with 1TB hard disk formatted as FAT32. The NAS case and firmware is very basic, so it doesn't allow to reformat the drive.

The command I use is:

$ rsync --progress --modify-window=1 --update --recursive --times \
  /home/mloskot/Pictures /mnt/nas/Pictures

To allow correct time comparison --modify-window=1 option is used, because FAT32 records file timestamps with 2-seconds resolution which is different to filesystem(s) used on Linux. The --update to avoid unnecessary copying of existing files - it behaves like incremental backup.

In order to do size-based comparison, you can specify --size-only option.

mloskot
  • 543
  • 1
  • 4
  • 6
  • Thanks for sharing, I will be checking out your combination of flags :-) The rsync manual is a bit bulky... – DutchUncle Jul 05 '12 at 14:36
  • 3
    the --modify-window=1 makes a huge difference. It solved the - i'm now done in milliseconds instead of minutes. – Wolfgang Fahl Nov 13 '14 at 10:33
  • 4
    For some reason I need to use `--modify-window=2`. When using `1` it still copies all files. Copying from NTFS to FAT32. – Luc Dec 07 '15 at 18:40
  • Occasionally, even with `--modify-window`, rsync would insist on copying over all my files. I am using the `--size-only` option now and that's working ok. That was from Mac OS Extended (Case-sensitive, Journaled) to FAT32. – diachedelic Apr 03 '18 at 04:02
  • Probably a good idea to add '--ignore-errors --max-size=4GB' to avoid crashing on files that would not fit into FAT32. – Maksym Jul 08 '18 at 14:56
12

I would recommend reformatting to a linux fs if you possibly can. As mentioned, FAT has relatively low file size limits and might not handle permissions and ownership quite right. More importantly, FAT doesn't track modification times on files as precisely as, say ext3 (FAT is only precise to within a 2 second window). This leads to particularly nasty behavior with rsync as it will sometimes decide that the original files is newer or older than the backup file by enough that it needs to re-copy the data or at least re-check the hashes. All in all, it makes for very poor performance on backups. If you must stick with FAT, look into rsync's --size-only and --modify-window flags as workarounds.

thraxil
  • 271
  • 1
  • 3
2

Is this rsync on linux to/from a FAT (which version of FAT?) disk or are you using the Windows version cwRsync?

Either way FAT16 and 32 work with both versions (haven't tried FAT12 myself). If you're on a linux system I would expect performance from a FAT filesystem to be a little worse than using a linux-format filesystem as in general they are slightly slower (happy for someone to put me right on this one if I'm wrong).

Chopper3
  • 100,240
  • 9
  • 106
  • 238
1

Yes reformat! You should always use a better file system than FAT unless you have to for compatibility reasons!

I'd suggest you format the drive using the same file system that you are rsyncing from otherwise you will potentially lose permissions and attributes that are set on your files and directories.

FAT (presumably FAT32) could be problematic to due maximum file sizes (4GB), maximum volume sizes (2Tb - one day anyway!), fragmentation etc.

However if none of this matters and you just want a simple, fast & maximum compatible file system then FAT will be OK. Speed wise, due to the lack of journalling, permissions etc, FAT could possibly out perform the alternatives - NTFS/Ext3/HFS...

Jon Rhoades
  • 4,989
  • 3
  • 30
  • 47
1

Is there a reason you're using the FAT drive? Rsync should work to save data, but you'll lose meta information about files...ownership, permissions, etc...and you risk having issues if you ever rsync large files since FAT has a filesize limit of 2 gig.

Performance I can't comment on, I've never run benchmarks on it compared to others. I imagine it also has a lot to do with the drive's own specifications as well as filesystem in regards to that.

If you could, I'd reformat to the native format you're planning to use with the computer (OS X? Linux?), if for nothing else than to keep the metadata and lose the forehead slap in a few months when I end up syncing something that errors out due to filesystem limitations. often sharing with other systems can be done through sharing exports or if you use NTFS you can keep some metadata but still get most cross-system compatibility since the Mac, NT, and Linux all support NTFS R/W ability now.

Bart Silverstrim
  • 31,092
  • 9
  • 65
  • 87