Continuously copy all file changes from one folder to another on change (Windows)

I'm working on a massive Java web application which is kept under centralized version control, but the source files are used to build and run the actual server, which then copies all the files I work on into a random temporary directory while it's running.

My problem is that I need to either rebuild this huge application to see any changes to my files, or keep track of all my changes and copy over the files I've changed back to their source location before committing them back to source control.

My question is: is there a command line script I can run that will monitor any changes in the source folder and automatically copy changed files across to temporary folder X?

The idea is to remove the human point of failure in having to manually mitigate the two-locations problem.

Barney

Posted 2013-02-25T15:10:50.853

Reputation: 220

Do you want a folder that it a perfect copy or a folder that consists only of changed files? – David – 2013-02-25T15:15:48.790

Perfect copy. When I build the application, it makes one big copy. I'd like my manual changes to the source to be automatically migrated to the copy folder. – Barney – 2013-02-25T15:17:59.703

Robocopy can do it but I am not sure if that's a good idea. – kush – 2013-02-25T15:18:36.317

Thanks @kush, wasn't aware of robocopy. Not sure it's a good idea… Because robocopy is flaky? Implementation is dangerous? Inherently bad idea?

– Barney – 2013-02-25T15:20:48.813

You should not have to come up with your own implementation to something that sounds like (please correct me if I am wrong) a pretty common scenario. You are taking files on the production server and updating your temp folder (and not the other way), right? – kush – 2013-02-25T15:27:16.430

Keeping two copies of the exact same file is a bit like using the goto statement in programming, sometimes it's necessary, but most of the time, there is a better way. – David – 2013-02-25T15:28:55.747

Actually, both locations are on my local filesystem. Ironically I know of plenty of tools which will persist any changes on my system to a remote location via SFTP, but nothing that'll do it from one 'source' location on disk to another arbitrary 'cache' location, also on my local disk. Turns out robocopy can't copy open files.

– Barney – 2013-02-25T15:29:41.470

@Jikag I agree, I shouldn't be fighting this huge semi-automated application to sanitize my workflow. The problem is I'm one of the few people working on files that need a rapid trial-and-error workflow, while most of the work on the application needs conditional compilation based on local system etc. – Barney – 2013-02-25T15:53:38.980

Would a Continuous Integration solution such as Hudson or Jenkins help you perhaps?

– Karan – 2013-02-25T22:51:55.467

Answers

I suggest FreeFileSync, which appears to have the ability to create a script that will automatically sync the two folders every few seconds. I haven't played with it personally, but it looks promising.

enter image description here

David

Posted 2013-02-25T15:10:50.853

Reputation: 6 593

Great soft! Good online documentation. Thx ! – M'sieur Toph' – 2018-07-04T06:58:41.653

Thanks for this — it makes use of Volume Shadow Copy, which seems to be the crucial caveat to the robocopy solution, so it looks promising. I'll give it a twirl and qualify in a minute.

– Barney – 2013-02-25T15:56:07.107

1This works great. There's a bunch of granular options to specify behaviour when Item exists on left side only, Right side is newer, etc — the batch tool can execute every 0 seconds for near instantaneous response without any noticeable performance hit (for 300 files over 150MB), it reports and optionally alerts/pauses/logs on change… GUI interface is a bit weird and it takes tweaking, but it's powerful and flexible. Again, many thanks. – Barney – 2013-02-26T13:53:31.980

Have you considered using RoboCopy? http://ss64.com/nt/robocopy.html

RoboCopy can be set to copy the data after a set period of time has elapsed and/or a number of changes to the datasets have occured.

/MON:n : MONitor source; run again when more than n changes seen.
/MOT:m : MOnitor source; run again in m minutes Time, if changed.

BobJim

Posted 2013-02-25T15:10:50.853

Reputation: 980

You could also add the resulting batch file to the Windows Task Scheduler to run at specific times. – BobJim – 2015-05-12T14:21:31.423

You also might want to have a look at the (ageing) DeltaCopy client and server. Basically, it’s a Linux-like rsync implementation for Windows, scriptable (using the Windows scheduler, if wanted) and its main advantage is that only the changed parts of a modified file (yes, file!) are transmitted. This can save a lot of traffic on big projects. A disadvantage is, of course, that it has to build and transmit a file list before.

Also, there are some (small) caveats:

As it is older software, you might have to replace cygwin1.dll on Windows machines. (If you sync files between Linux and Windows, only. It uses cygwin as the underlying framework which didn’t handle 16-bit Unicode/UTF-8 filename conversion correctly at that time.)
It’s not exactly that user-friendly, you should have a look at Linux’ rsync manpage to exploit its full potential.
On Windows, it requires a »client« and a »server« machine.
If the server is a Linux machine, you’ll have to setup rsyncd correctly.
No support for Windows’ Volume Shadow Copy (might not be needed, though).

Well, it also has advantages (that’s why I still use it daily):

Fully compatible with *NIX rsync and rsyncd. A proven, well-known and highly efficient syncing system. Still.
Keeps overhead small: Only the changed data (even inside files!) gets transported over the network. So it’s also very nice if you are teleworking or connect to your server via slow connections.
Failsafe, apparently. I’ve rsync’d many many gigabytes of data over the years, and not one flaw happened when »patching up« the target files. Even if I found the concept a little worrying, at first.
I still use DeltyCopy to backup files from Windows machines to both a central Linux server and a Windows 2003 server on a daily basis. Works extremely well and safe, if installed correctly.

Well, up to you. Have fun! (And let us know about FreeFileSync!)

Moonbase

Posted 2013-02-25T15:10:50.853

Reputation: 61