7

I'm in the process of moving files from an Ubuntu server to a Snow Leopard server. The Ubuntu server has an NFS share of around 6TB which I'd like to clone to the Snow Leopard server.

I mounted the nfs share on the Snow Leopard server and then ran

rsync -av /Volumes/FromUbuntu /Volumes/LocalCopy

After it copies around 100GB it complains that files have vanished. I'm assuming that for some reason the NFS links are going stale? What could be causing this? The Ubuntu server is not crashing and there are not connectivity issues that I am aware of. I wouldn't mind running to the rsync command over and over until the copy was finished, but it takes about 6 hours just for rsync to make it's file list.

Any suggestions? Would it be faster to just rsync over SSH? Thanks!

(PS: I've tried just using 'cp -arv' which doesn't seem to fail but according to the network traffic monitor on the Mac it seems to take twice as long to copy files as rsync after rsync has built its file list?)

UPDATE: I'm trying rsync over ssh from the mac server to the ubuntu server and it seems to been going much faster (it took less than two hours to create a file list and start transferring, where as when rsyncing from the nfs mount took around six hours just to build the initial file list.) It definitely looks like there is some problem with mounting the NFS share on the mac. Has anyone had problems with this? What about the other way around, mounting nfs shares on the mac on an ubuntu client? I was planning on hosting an NFS share for Ubuntu clients on the Mac, but now I'm getting nervous. Thanks for your input!

Evan
  • 227
  • 1
  • 3
  • 8
  • Is there anything changing the contents of this share after rsync has built it's file list? – DerfK Dec 17 '10 at 20:11
  • @DerfK Yes, some of the files will probably change so I would expect the error on a few files, just not every file after so many had been transfered? Or would I need to unmount the NFS share from other clients for it to work properly? – Evan Dec 17 '10 at 20:16

3 Answers3

7

This is a problem with special characters in the file names, as discussed here: https://superuser.com/questions/91967/rsync-character-set-problems

Community
  • 1
Dan Bolser
  • 175
  • 1
  • 5
6

As far as I know, the "file-has-vanished" condition is not a fatal error condition for rsync, and the process should continue just fine. Are you saying that rsync is halting upon reporting the "vanished file" messages?

If you are backing up a live filesystem, it is always possible to get the "vanished" message from rsync. Between the time that rsync adds a file to its transfer list and the time that rsync actually gets around to reading it, a file can "vanish" for any number of legitimate reasons (e.g. a temporary lock file was removed, or a file was renamed). Note that some applications work around limitations in NFS file locking by creating and removing hidden lock files, and it's possible that rsync may be seeing some of those.

I would suggest to keep your source filesystem as quiescent as possible while it is being backed up. Ideally, no one other than the backup process should be accessing it, but I realize that that may not be feasible. If your source filesystem is on an LVM logical volume, then you could consider creating a read-only snapshot to use as your backup source.

Steven Monday
  • 13,019
  • 4
  • 35
  • 45
  • Thanks for your answer! No it's not a fatal error, but what's happening is that it's reporting "file-has-vanished" for every file after a certain amount of time passed/GB have been transfered. That's why I was thinking something about the NFS mount was at fault? – Evan Dec 17 '10 at 20:31
  • @Evan: So `rsync` is reporting that files you definitely know to exist are vanishing? That sounds bad. Well, you could try eliminating NFS from the picture by running the `rsync` daemon on your Ubuntu server, and configuring a read-only module in `/etc/rsyncd.conf` to act as the source for your `rsync` transfer. – Steven Monday Dec 17 '10 at 21:01
  • Yeah, it's definitely saying files that are there have vanished and it seems to happen after a specific amount of time goes by. I've run rysnc a few times and each time it copies roughly the same amount in the same amount of time before deciding every other file has vanished. Not sure what would be causing that though because the Ubuntu NFS server has worked without problems for a few years and it doesn't seem to happen if I use a cp command instead of rsync. I'll try rsync without using NFS and update you in a few days if it works! Thanks for your help. – Evan Dec 17 '10 at 21:21
2

What I do in my daily backup script, this script is a rsync wrapper :

#!/bin/bash

rsync "$@" 2> >(grep -v vanished)
ret=$?
((ret==24)) && exit 0 || exit $ret
Gilles Quenot
  • 1,150
  • 9
  • 15