RSync to WebDAV mounted volume very slow

I have a mounted WebDAV store, which I am playing around with to try and put together my own pseudo-DropBox set up. However, it seems very slow to upload larger files (my bandwidth graph goes ok for a little bit, then just stops dead for long periods of time).

Has anyone tried this? I'm wondering if the way it gets mounted makes rsync think it's a local drive or something and so may be chattier than it would be if it knew it was a mounted network drive ?

I'm syncing using the following command:

rsync -rv /source/directory/ /mounted/webdav/directory/

This is running using MacOS 10.6, and WebDAV is mounted using the native mounting mechanism.

AlexC

Posted 2010-09-25T22:13:09.503

Reputation: 211

Answers

Check the man page for

--size-only because most webdav implementation do not accept setting modification time
--no-whole-file to tell rsync its handling a remote filesystem
--inplace having rsync replacing files directly, instead of uploading an then replacing

Btw, I am using another encfs layer to encrypt files on the webdav, and I am quite happy with the performance.

ribx

Posted 2010-09-25T22:13:09.503

Reputation: 131

How --inplace help? In manual: This option is useful for transferring large files with block-based changes or appended data, and also on systems that are disk bound, not network bound. It can also help keep a copy-on-write filesystem snapshot from diverging the entire contents of a file that only has minor changes. – msa7 – 2015-11-11T14:22:14.860

My command is sudo mount -t davfs https://webdav.yandex.com/ ~/mnt/yd; rsync -avrc --size-only --no-whole-file /home/msa/xranitel/webdav /home/msa/mnt/webdav – msa7 – 2015-11-11T14:24:58.033

I have found that Mac OSX webdav mounting is unreliable (10.11.6 El Capitan). If I use rsync, tar, cp, or any other tool, some of the files come across corrupted. I can repeat and repeat, but different files get corrupted each time. I then mount the same exact webdav folder, in a windows VM inside the mac (same network, same hardware, same everything except now it's a windows VM instead of Mac OSX) and then everything runs fine.

Edward Ned Harvey

Posted 2010-09-25T22:13:09.503

Reputation: 1 070

I have tried this on a server for which I have both rsync and WebDAV access to just to see the difference and the rsync over WebDAV method was much slower. Must just be a protocol thing. I don't think that rsync is locally chattier than over a network.

fideli

Posted 2010-09-25T22:13:09.503

Reputation: 13 618

I'm pretty sure this is because rsync calculates the checksum for comparing and after transmission. Since the file is stored remotely it means for this it's transferred several times.

You could run rsync as daemon or through ssh to do the calculation on the server. Or just disable the checksum.

bdecaf

Posted 2010-09-25T22:13:09.503

Reputation: 458