rsync always uploads all files from macOS => Linux

Question

I'm testing rsync with WebDAV server in Docker container:

• local directory: /Users/user/files/
• "remote" mounted server: /Volumes/webdav/

Here's the initial state:

# remote
➜  cat /Volumes/webdav/remotefile
change
➜  ls -la  /Volumes/webdav/remotefile
-rwx------  1 user  staff  7 Dec  2 01:39 /Volumes/webdav/remotefile

# local
➜  cat /Users/user/files/remotefile
change
➜  ls -la  /Users/user/files/remotefile
-rwx------  1 user  staff  7 Dec  2 01:39 /Users/user/files/remotefile

Now let's change local file and upload it to "remote" server:

➜  files echo 'add#0' > ./remotefile
➜  files cat remotefile
add#0
➜  files \rsync -varP /Users/user/files/* /Volumes/webdav/ --delete
building file list ...
1 file to consider
remotefile
           6 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/1)

sent 132 bytes  received 42 bytes  348.00 bytes/sec
total size is 6  speedup is 0.03

File uploads successfully, as it was changed. But if I run rsync again local => remote, it re-uploads it:

➜  files \rsync -varP /Users/user/files/* /Volumes/webdav/ --delete
building file list ...
1 file to consider
remotefile
           6 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/1)

sent 132 bytes  received 42 bytes  348.00 bytes/sec
total size is 6  speedup is 0.03

Now I test opposite direction:

➜  files \rsync -varP /Volumes/webdav/* /Users/user/files/ --delete
building file list ...
1 file to consider
remotefile
           6 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/1)

sent 132 bytes  received 42 bytes  348.00 bytes/sec
total size is 6  speedup is 0.03

File downloads succesfully. And one more time...

➜  files \rsync -varP /Volumes/webdav/* /Users/user/files/ --delete
building file list ...
1 file to consider

sent 80 bytes  received 20 bytes  200.00 bytes/sec
total size is 6  speedup is 0.06

...shows 0 transfers, as file was not changed. Thats the expected behavour.

Why it doesn't work with local => remote upload and always re-uploads file?

Thanks.

Answer 1

Have you confirmed that the clocks are in sync on both local and remote?

From the rsync man page:

 Rsync finds files that need to be transferred using a lqquick checkrq algorithm 
 (by default) that looks for files that have changed in size 
 or in last-modified time.

If the clocks are out of sync, and you are not able to edit one, try telling rsync to be a little "fuzzier" with how it checks for time. Again from the man page for rsync

 --modify-window
    When comparing two timestamps, rsync treats the timestamps 
    as being equal if they differ by no more than the modify-window value. 
    This is normally 0 (for an exact match), but you may find it useful 
    to set this to a larger value in some situations. 
    In particular, when transferring to or from an MS Windows FAT filesystem 
    (which represents times with a 2-second resolution), 
    --modify-window=1 is useful (allowing times to differ by up to 1 second).

Try running rsync with the --size-only option and seeing if the behavior of rsync changes.

Another alternative is to use the --checksum option to rsync. This requires more disk I/O and cycles to generate checksums, so it can be slower. It would be interesting to know if it shifts the problem.

---