Why does rsync update NTFS directories when the source has not changed?

I'm taking classes with lots of phonetic and grammatical analysis. They've given me an account on a Windows 10 network. Since I am much more comfortable with Mac and Linux (and since the Windows availability hours are limited), I do my work on my Mac laptop.

I auto-mounted my NTFS directories, but there's something flaky with their LAN, so that quite often some apps have read-only access or no access at all.

Consequently, I work in a local copy and frequently rsync changes. But rsync* always updates every NTFS directory—the directory only, never any files (except those I actually changed). Most of those directories, I have never navigated into, yet rsync reports updates all the way down the subdir tree. It's not major—finishes in seconds—but it's still pretty weird.

What do Windows and NTFS do to directories that are never accessed that makes rsync think they have changed?

WGroleau

Posted 2016-08-10T04:17:37.157

Reputation: 438

Well, that's not it either. It's crazy. I noticed it was updating unvisited directories again, and I immediately repeated the rsync command. No directories! Then I logged into the windows machine while repeating rsync on the Mac. The moment the Windows desktop appear, rsync immediately updated all those directories. But, it continued to do so as long as Windows was logged in. WELL, I thought, maybe it is wasting CPU time doing unnecessary refreshes. So I logged out. rsync continues to update those directories! So I'm back to the beginning, i.e., DUNNO. – WGroleau – 2016-08-11T20:36:37.660

Adding the modify-window in “rsync -av --bwlimit=200 --delete --modify-window=5 (HFS+) (NTFS)” made it stop updating unvisited directories. But a day later it did it again. An immediate repeat did not update them. Logging into Windows made it happen again. BUT, since then, it happens EVERY time, whether logged into Windows or not. Even if the rsync commands are only a second apart. Again, even with the unnecessary updates, it only takes a second, so it's not a big deal. But it does make reading the file list less convenient. – WGroleau – 2016-08-11T20:50:11.083

Answers

Allow comparison of mod-times with reduced accuracy

It appears in this type of setup where you mount to an NTFS partition you want to ensure you use appropriate parameters such as --modify-window=5 to increase the default value to allow comparison of mod-times with reduced accuracy.

--modify-window 

    When comparing two timestamps, rsync treats the timestamps as
    being equal if they differ by no more than the modify-window value. This 
    is normally 0 (for an exact match), but you may find it useful to set 
    this to a larger value in some situations. In particular, when
    transferring to or from an MS Windows FAT filesystem (which represents               
    times with a 2-second resolution), --modify-window=1 is useful (allowing 
    times to differ by up to 1 second).

Further Resources

Pimp Juice IT

Posted 2016-08-10T04:17:37.157

Reputation: 29 425

First to answer the programs read-only problem. Since the network share is controlled by windows, windows only allows certain users privileges to do things. It could be its a public share, IE anyone can view it to download files ect, but not change it unless specifically added to be able to do so. Since you are using Mac, which is similar to Linux. Not every linux application you are using, is using the same user name that you are currently logged into on your Mac or Linux. Therefore, since the program is not using the logged in username of yours, its not given privilege to edit the files according to windows since the username does not match.

Secondly, Rsync is recursive. Meaning, it will go through all the directories. Unless you limit it to a specific directory, but even if you do limit it to 1 directory, it will keep going through all directories that are in the directory it started from. This is what its designed to do. Hence its called recursive synchronization. Now the reason for the updating the directories and not the files could be how rysnc is obtaining the timestamp from the directories. It could be that because its windows 10 directories don't have a modified or accessed timestamp, only a timestamp of when the directory was created whereas the files do have all these timestamps which is what rsync uses to determine if it needs to update directories or files.

Frostalf

Posted 2016-08-10T04:17:37.157

Reputation: 429

It was the timestamps. But your first part doesn't make sense. The directories are auto-mounted. That means Samba is getting user name and password from my operating system, NOT from each individual app. Second, I said "quite often," not "always." Mac OS doesn't change my username or password at random times, and Windows shouldn't. The access problem occurs with both the public share and the directory that is supposedly mine alone. Also, it's sometimes read-only, sometimes no access at all. Since the symptoms change so often, I stand by my "something flaky with their LAN." – WGroleau – 2016-08-10T05:09:36.557

Mac OS operates similar to that of Linux. On linux, not all the software that is installed on it, runs under the same user name you are logged in under. Given the example Apache Web Server if you installed it, uses by default the user www-data. But you can still use it under your currently logged in username. Now, with windows shares, if the program wasn't running under your currently logged in username which is what the windows share expects to be using the share, it could be given different set of permissions. Also it depends how samba is configured as well. – Frostalf – 2016-08-10T05:14:09.917

(1) None of my applications have the sticky-bit set. They all run as me. (2) If it were a permissions error, it would say so every time instead of sometimes no problem, sometimes permissions, sometimes other more cryptic messages. (3) If it were a permissions error, rsync wouldn't work either. – WGroleau – 2016-08-10T05:19:44.653

the third one is not entirely true, for sending files the other direction yes, for recieving no as you can rsync anything in the direction to your computer given it can read the directories. Anyways besides the point. Then I would look at your samba configuration to ensure it is correct. Windows has a few different variants of its file sharing protocol so it is possible samba is using an older version of the windows protocol. Not entirely sure. Anyways worth a look to ensure your settings are not the problem. The other problem could be due to the applications, either a bug or something else. – Frostalf – 2016-08-10T05:42:29.317

I have two directories in the same disk on the same server mounted with the same syntax. The two don't always behave the same as each other, or the same as they did five minutes earlier when /etc/auto_smb has not changed. An error in the syntax of that file will not make it work intermittently. The network is flaky. – WGroleau – 2016-08-10T06:21:07.133