4

I have a fairly simple (not really) requirement but I've looked at a few solutions and can't find a good solution. I've got a Red Hat EL 6 server environment in my co-location and office, and some linux and windows xp and win7 workstations my office.

The co-location and office do not have a high-bandwidth link. Let's say it's 5Mbps.

I have a collection of files which are processed by batch jobs at my co-location, but are viewed and edited by workstation users, i.e. the files need to be accessible at both locations in a read-write fashion.

Additionally, some individual files are huge - too big to wait for the file to transfer from the "other" location if needed on-demand. So a copy of each file should be immediately "synced" to the other location if created/changed.

I've looked at AFS, which is almost perfect, except that the distributed "replica" is always read-only, and re-synchronizing the read-only replicas happens after an administrative command, AFAICT.

I'd prefer, also, for each client (e.g. workstation) to not have to have a large cache, but rather have a server at each site which provides local (subnet) access to the shared FS (maybe re-shares the content using local NFS or samba...)

Is there any such fs available for linux rhel 6?

dmansfield
  • 161
  • 1
  • 4

2 Answers2

2

Based on the information above I think you have a similar situation to us. We have a main data center with a few remote offices that need offsite mirroring.

We are using rSync. It works very well.

RSync is a free software computer program for Unix and Linux like systems which synchronizes files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. An important feature of rsync not found in most similar programs/protocols is that the mirroring takes place with only one transmission in each direction.

Here is a quick overview and guide http://www.cyberciti.biz/tips/linux-use-rsync-transfer-mirror-files-directories.html

You may also want to looking into what ubuntu is doing with their orchestra server. not a direct comparison for RHEL however it is just a compilation of other services rolled into a pre-installed package.

Thomas

DaffyDuc
  • 512
  • 2
  • 7
  • I should have mentioned I have considered rsync, in fact there is a project called lsyncd that I have considered which uses rsync but it directly monitors the filesystem for changes. However that is pretty brute-force. If a filesystem will do this for me, I'd much prefer it. – dmansfield Mar 01 '12 at 21:20
  • Rsync is fairly light weight. Usually our update only takes a few seconds to complete. Hope you find what your looking for. – DaffyDuc Mar 03 '12 at 23:08
  • the main issues with a straight rsync are 1. tons of files (more than 75000) - running rsync every few seconds against 75000 files means statting the files each time - quite resource intensive when almost none of them have changed. 2. bidirectional - it needs to sync both directions. – dmansfield Mar 05 '12 at 22:57
  • The main issue with RSYNC is that it does not detect file moves or renames. So that if you rename a folder with a lot of data, RSYNC will consider that the folder was removed and a new one was added. This would cause a lot of data to be pointlessly transferred over a slow link. –  Feb 15 '13 at 21:58
2

You can try MoozeFS or http://www.xtreemfs.org/
Both can be okay for your needs.

Forget about GlusterFS (bugs, bugs, bugs) and Coda (need bandwidth).
You can also have a look at tahoe-lafs (I never tested it).

Gregory MOUSSAT
  • 1,737
  • 2
  • 25
  • 48