4

I put work files in Git repositories and sync between computers/laptops using an external SSD. So for example if I want to sync repo1 from my PC to my laptop, these are the steps that are done:

  1. plug and mount external SSD on PC
  2. on PC: cd /media/externalssd/repo1 && git pull ~/repo1
  3. unmount and unplug external SSD on PC
  4. plug and mount external SSD on laptop
  5. on laptop: cd ~/repo1 && git pull /media/externalssd/repo1

Additional information: I'm using ZFS on Linux using zfs-fuse.

Sometimes, in the middle of the pull, the USB cable connection is not good so the filesystem is "disconnected" or sometimes zfs-fuse daemon dies. After I restart zfs-fuse and remount the filesystems, sometimes I will end up with a few files being truncated on the laptop and the repository on the laptop is no longer recognized as a git repo ("fatal: Not a git repository (or any parent up to mount point /blah)"). And sometimes, the repository on the SSD also gets corrupted.

This scares me.

Which element(s) probably truncate the files? The SSD? Linux? zfs-fuse? Git? How should I avoid this corruption and have a reliable sync process?

janos
  • 798
  • 1
  • 5
  • 22
Gerry Lufwansa
  • 41
  • 2
  • Just out of curiosity, why aren't you using `git push` to sync your repo TO the external disk and `git pull` to sync it FROM the disk? – Gerald Schneider Dec 20 '17 at 14:18
  • 1
    How should you avoid this corruption? Get a USB drive that doesn't disconnect in the middle of an IO operation. This is not a problem you need to fix in hardware. Also, as @GeraldSchneider says, there are alternate ways of using git where you won't have that issue. – Mark Henderson Dec 20 '17 at 16:54
  • @GeraldSchneider What's the differene if I may ask? I thought `git pull` and `git push` should be pretty much equivalent if done on the opposite side? – Gerry Lufwansa Dec 22 '17 at 12:52
  • @MarkHenderson I've actually just bought a new SSD, but sometimes disconnect can still happen e.g. cable accidentally got shaken when laptop is on my lap. – Gerry Lufwansa Dec 22 '17 at 12:54

1 Answers1

1

It sounds to me like you're having issues with the reliability of the connection of the SSD, which intrinsically will cause issues with any software looking to read or write from that drive. In your case, this software is git.

If you cannot trust your connections when moving from device to device, then you should probably re-evaluate your workflow.

From the above information, I believe your PC and laptops are in the same vicinity, or could at least be connected to the same LAN. Why can't you connect the SSD to the PC permanently, then use git to push/pull files over the network? That solution should be relatively as quick as swapping the USB drive, for all intents and purposes of file versioning.

Aaron St Clair
  • 11
  • 1
  • No, half of the time I work on a laptop away from my PC (in my office or at home I would prefer doing work on the PC all the time). I push to SSD regularly as a backup/fallback mechanism. I avoid network because sometimes I have no connectivity (e.g. travelling between cities on a train), or the connection is expensive (mobile data quota is small), and/or the data is private (I know and use tunnelling to a private VPS mounting encrypted filesystem, but this is not my regular workflow). – Gerry Lufwansa Dec 27 '17 at 09:26
  • What I don't understand is why unreliable (USB) connection to the SSD can cause corrupted/garbled data. Aren't there layers of protocols that are supposed to prevent this from happening? – Gerry Lufwansa Dec 27 '17 at 09:27
  • Ugh, another lockdown. – Gerry Lufwansa Dec 27 '17 at 09:28