1

I am trying to copy a large amount of data (several hundred gigs, with folders and files of various sizes) from local storage on a Windows 2008 server to a CIFS share on another server. The CIFS share space can only be accessed via CIFS, and the system hosting the CIFS share cannot be "logged into" (via ssh or RDP or similar) in any way. All permissions must be maintaned, all folder trees and file names must be maintained, and the copy has to be done in a reasonable amount of time. The servers cannot be offline for a long period of time, and whatever method/tool is used has to be free.

Microsoft's "RichCopy" utility doesn't seem to be able to override permissions on folders the account it is being run under doesn't have access to. However, it provides threaded copying which seems to be fairly vital to copying the data quickly due to the large number of very small files.

I read through several other posts with similar issues, but most of them have to do with Linux environments or do not have all of the same requirements. Lots of specifics here but I'm hoping someone can help me find a good solution!

kingfish
  • 410
  • 3
  • 11

3 Answers3

2

Windows 2008's version of Robocopy handles permissions correctly, invoking the Backup privilege (via the /B or /ZB switches) when needed to override ACL's restrictions that would deny the account running the copy direct access to files\directories. It also supports multithreaded copies (via the /MT switch, default is 8 max is 128). The /Z (or /ZB) switch provides for partial copy\resume.

The /B options require a user that can invoke the backup privilege - a user that is either an administrator or member of the Backup Operators group can invoke this. The account you use should meet those requirements on both source and destination.

One quirk with Robocopy that I've noticed is that the /B or /ZB switches only work reliably when you are running Robocopy from the source system to the destination, in pull mode where you are running it on the destination system I've seen access denied errors.

Helvick
  • 19,579
  • 4
  • 37
  • 55
  • It's telling me that the MT switch is not a valid option, and I don't see the MT switch in any of the Microsoft documentation. Thoughts? – kingfish Oct 16 '09 at 18:18
  • Ah, my bad. The version of Robocopy that supports /MT is the one included with 2008 R2 or Windows 7. – Helvick Oct 16 '09 at 19:45
  • I admit I haven't Googled for this yet but is there a way to get the 2008 R2 version of Robocopy into 2008 Vanilla? – kingfish Oct 16 '09 at 22:20
  • You can download an eval copy from Microsoft so you should be able to pull it's version of Robocopy out of that without too much trouble. I've run the "wrong" version of robocopy on systems for years with no major issues that I can recall. – Helvick Oct 16 '09 at 22:34
  • Eval copy is 64-bit only, and of course the machines I'm working with are 32-bit. I tried copying the Robocopy.exe from the Eval version to my 32-bit version just to test, but as suspected it did not work. Any other ideas? (Thank you for all of your input btw.) – kingfish Oct 19 '09 at 15:11
  • Aargh - seems like you're being stumped at every turn. You could try a copy from a 32bit Windows 7 install - should be the same code. – Helvick Oct 20 '09 at 18:44
0

I have used TeraCopy many times very successfully. I believe that it properly maintains all permissions. Are the computers within the same domain? If not, there might be an issue with users existing on one computer but not the other, I don't know how any solution would resolve that though.

TeraCopy also uses threaded copying so it is much faster than the built in Windows copy methods. It will resume where it left off if you need to pause or stop for any reason during the copy.

As for copying files that you do not have permission to, I am not sure that any utility will be able to do that. On my Windows 2003 domain there is a Builtin Security Group called Backup Operators, described as "Members can bypass file security to back up files". If you were a member of this group, you might be able to do what you want.

minamhere
  • 859
  • 7
  • 18
0

I tried downloading the Windows 2008 R2 Evaluation copy as suggested by Helvick above. I made a VM to run the OS on, as I cannot upgrade the data source server to R2 at this time.

I ran into a fair amount of speed bumps along the way, and ultimately discovered that even with the /B and /MIR switches set, Robocopy can only properly transfer the ACLs if being run on the source (and perhaps the destination) server. If run on an intermediary device, setting the ACLs on the destination end fails, even though the data is copied. Robocopy then applies the ACL of the deepest folder it has access to to all the data below that level of the tree, which is unacceptable. In addition to this, the overhead of running the copy through an intermediary server was much too high and drastically reduced copy speeds.

However, when run on the source server, the 2008 Vanilla version of Robocopy is able to transfer and implement the ACLs without a hitch. Luckily we have also discovered that the speeds of non-threaded Robocopy should suffice for this data migration. However, if I could update the source server to 2008 R2, I have no doubt that the data copy would absolutely fly using multi-threading.

Hopefully, the boys at MS will soon add a "Backup" mode to RichCopy and it will become a super useful little utility.

kingfish
  • 410
  • 3
  • 11