3

Here is the scenario I am trying to make work. I have three machines, J, T and R that need to eventually get the same data (slight temporary differences are fine). J and T are desktops that are turned on and off each day. R is a remote fileserver. The users on J and T can make changes to shared data in one directory (shared via group permissions) and on their own home directories. It is fairly common that only one of the two J and T machines is running at any one time. The Internet connection to R is fairly reliable, but R is far away and the latency makes it undesirable to use as the main file server. I tried with NFS and it was pretty painful. The connection is not reliable enough to depend on it for access at all times.

It is not an option, at this point, to put in a local server and just serve up a shared drive via NFS or CIFS. That would make like much, much easier, but it is not an option.

I tried cross mounted NFS exports, and managed to get a couple simple shell scripts to mount the drives etc. but if J mounted the drive from T and used that as the base file share and then T was turned off, the data changes would be lost. So I need some way to constantly sync data.

I looked at rsync and eventually found csync (not csync2 which is an entirely different project). This allows me to run periodic syncs between the machines. However, it is in cron and not done on demand. I also found lsyncd which looks like it would possibly be good for copying changes when both workstations were up, but not good at figuring out what changed when the local system was down and the other workstation was making changes.

I looked at Ubuntu One, Dropbox etc. and the free file limits are a problem. The budget is very, very limited and recurring costs are to be avoided. iFolder might be an option, but it looks like it needs a server somewhere.

What I would prefer would be something like this:

  • mount both remote and local copies into a working directory. Make changes there and have the filesystem push them out to the other locations.

  • On login, start a resync with both other machines to catch up on any changes made while the machine was off.

It is possible for one of the two workstations to be off for days at a time, so a lot of changes can accumulate. There are only two usual users, so the rate of changes isn't that high, but after a week, they do build up.

Is there a FUSE module with something like RAID1 over file systems? Because of the long duration of possible disconnects, things like Gluster FS, AFS and NBD do not seem that appropriate.

If there was a way to do NFS client failover (other workstation went down, use the local copy instead) that would work too. I've done some research on that and other than some mentions that "autofs should support that, but doesn't", I haven't found much.

I would prefer an NFS/CIFS based solution because then I could get file locks and we would not have potential problems with users trying to modify the same file at the same time. But I'm not sure how to solve the client-side failover.

How would you solve this? Again, having a local file server is not currently an option.

Kyle
  • 31
  • 3
  • 1
    I also like this question for this scenario: complex LAN has a central file server that just went down and I need to log in to fix it. But either admins have their home directories scattered to the wind or they are on the central server. Solving this improves the ability to recover during nasty outages, as long as the central login system is responding (e.g. ActiveDirectory). – zerolagtime Oct 29 '10 at 17:15

2 Answers2

1

Coda should meet your requirements. It is a network file system that strongly supports local caching and disconnected client operation.

ephemient
  • 1,420
  • 1
  • 11
  • 8
  • It did not look like Coda would cache all the files. And, in this case, where is the server? It was not clear to me that this would work. Is there a way to make Coda pre-cache all the files on the server? – Kyle Oct 28 '10 at 01:47
  • No, it won't cache everything by default. You can however advise the cache manager by "Hoarding". Look for hoarding in the coda documentation. – Kjetil Joergensen Oct 29 '10 at 17:43
0

If you have want to fall back to a sync tool be sure to look at unison. It is a bi-directional sync based on the rsync libraries and it can handle things being modified. It can save files where there was a conflict (updated on both sides) to a backup folder.

The sync will still need to be initiated by the either the user, cron, or some kind of startup script.

Zoredache
  • 128,755
  • 40
  • 271
  • 413
  • I have looked at unison, but, as with csync, there are two issues: – Kyle Oct 28 '10 at 01:45
  • 1) not demand triggered when a change happens, and – Kyle Oct 28 '10 at 01:45
  • 2) still no file locks :-( – Kyle Oct 28 '10 at 01:46
  • 1
    I know this question is ancient but regarding triggering: The Linux kernel has a change detection capability - inotify - that is available to the end user. So while a bit hackish, you can always use inotify together with unison to trigger syncronisation. – Andre de Miranda Aug 12 '19 at 21:42