4

I have two NTFS formatted drives on different machines and I do a robocopy mirror between them like this:

robocopy "D:\My Documents" "X:\Backup\My Documents" /MIR /V /NP /XD .* /XO /XA:SHTO /FAT /Z /ETA /DST /R:3 /W:30

But it will falsely mark files as newer, but I haven’t altered it, and it is a lots of files so I don’t think windows have altered them either.

I am running the robocoy in a steady loop and it takes about 15 minutes, so I think something is a bit wrong, anyone have this issue?

Example output:

  • same 17548 draft1.jpg
  • same 98108 draft2.jpg
  • same 104 Spiral3DCopy1.mtl
  • Newer 104 Sphere3DCopy1.mtl
  • Newer 345985 swamp1.jpg
  • Newer 75209 test.jpg

I have done a compare and no differences are found:

C:\>fc D:\Documents\"My Documents"\3d\Sphere3DCopy1.mtl X:\Backup\"My Documents"\3d\Sphere3DCopy1.mtl
Comparing files D:\DOCUMENTS\MY DOCUMENTS\3D\Sphere3DCopy1.mtl and X:\BACKUP\MY DOCUMENTS\3D\SPHERE3DCOPY1.MTL
FC: no differences encountered
HopelessN00b
  • 53,385
  • 32
  • 133
  • 208
Cinaird
  • 141
  • 1
  • 1
  • 4
  • Is the time set right on both machines? – Sven Jan 29 '12 at 16:46
  • Yes, it is, not all files is marked "newer" about 20% maybe – Cinaird Jan 29 '12 at 16:52
  • Is there anything common about the filetypes that are getting marked newer? – Trevor Sullivan Jan 30 '12 at 17:54
  • @TrevorSullivan: No, it is randomly distributed among my most common filetypes in the folder. Here is a sample of the most common filetypes: mtl, obj, jpg, png, pdf, pdf, odt, html, txt, psd, rar – Cinaird Jan 30 '12 at 20:41
  • Are these machines in different time zones? I'm wondering why you're using /DST and whether it could somehow be to blame. – Larry Silverman Feb 01 '12 at 15:16
  • Might help if you included some output from the command so we can check your assumptions. – Larry Silverman Feb 01 '12 at 15:19
  • @LarrySilverman Good observation but no, they are in the same building, and the only reason I have /FAT /DST /XO is to overcome the “Newer” bug, but obviously it does not work. I have tried with, without and all the combination of them. – Cinaird Feb 02 '12 at 16:27
  • Is robocopy the only thing that thinks the old files have become "freshened"? I wonder if you watched D:\My Documents with something like SysInternals procmon if you could detect some other process touching those files. – Larry Silverman Feb 02 '12 at 19:59
  • @LarrySilverman I’m pretty new to this, so bear with me if I don’t do things right. I run the backup script twice after each other and look in the Process Monitor it doesn’t show any *.mtl files ass accessed, but shouldn’t it show at least twice? And the Modified date Is still 2008-04-07. I think I have missed something. – Cinaird Feb 02 '12 at 20:38
  • Are you absolutely sure there are no processes that might be modifying these files on either side? It could be something touching the files on the sending side, making them seem newer (antivirus?); or something touching files on the receiving side, modifying to make them seem older. – Larry Silverman Feb 02 '12 at 23:11
  • I thought perhaps there was an option to only copy files if the size had changed, ignoring the timestamps, but I'm not seeing a switch for that in the robocopy /? help output. Perhaps I'm thinking of rsync. – Larry Silverman Feb 02 '12 at 23:14
  • This might be useless to you if you're trying to script this, but have you seen/tried SyncToy from Microsoft? It's a GUI file sync tool, rather than command line. – Larry Silverman Feb 02 '12 at 23:15
  • @LarrySilverman, Actually I’m not sure how to be absolutely sure, but I think the files are untouched what I can see in Procmon, maybe it’s on the receiving end. The option to only copy files if the size had changed would be perfect, but sadly I haven’t found it either. I think my best option is to test with some other synchronization software. Thanks for all the help. – Cinaird Feb 04 '12 at 07:41
  • @LarrySilverman ok, now i have done a fc and a compare, and both say no differences – Cinaird Feb 04 '12 at 14:08
  • Take a look at http://stackoverflow.com/questions/2871931/robocopy-compare-modified-dates-of-files -- Is it possible your /XO is competing with /MIR? Perhaps try getting rid of /MIR and explicitly setting /XO. – Larry Silverman Feb 05 '12 at 16:02
  • 1
    And one more idea. Try running it with /L. This will do a "what if" scenario, without actually making changes. Let robocopy tell you which files it thinks are newer. Then pick one or two and go check the timestamps on both machines and see if they are actually different. Honestly, you've got a very odd situation, and it makes me think you've got some rogue program changing timestamps on you. – Larry Silverman Feb 05 '12 at 16:04
  • @LarrySilverman Ok, no I did a test I should have done much earlier. I tried a different destination. Same computer as the source, but different hard drive. It worked like a charm, so now I know the problem is my destination. The destination is a Boxee box, so I don’t have so much control. Thaks for all the help. I tested what you suggested, but there was no difference on the files timestamp, and sadly just /XO did not do the trick. I will try some new settings on the box and hopefully it will work. – Cinaird Feb 06 '12 at 20:59
  • You said both source and destination are NTFS-formatted partitions. You have an NTFS-formatted partition mounted to a Boxee Box? – Larry Silverman Feb 06 '12 at 21:45
  • @LarrySilverman I formatted the disc as NTFS and then mounted it, so if Boxee Box dose'nt formated it the, it should be NTFS, else i don't know – Cinaird Feb 07 '12 at 08:09

1 Answers1

6

We found that using the /FFT Robocopy switch will reduce the timestamp to about 2 seconds difference when comparing file changes. It eliminated the 2 hours copying everything when the changed files would normaly take under 3 minutes.

RoboCopy /? says to use this switch to "assume FAT File Times (2-second granularity)."

Thanks to the folks at connetrix.

http://www.conetrix.com/Blog/post/Robocopy-Incorrectly-Detects-Files-as-Newer-When-Copying-Between-Different-File-Systems.aspx

Try this:

robocopy "D:\My Documents" "X:\Backup\My Documents" /FFT /MIR /V /NP /XD .* /XO /XA:SHTO /FAT /Z /ETA /DST /R:3 /W:30

It worked for us.

finch
  • 125
  • 1
  • 1
  • 7
JohnD
  • 61
  • 1
  • 3