Sync Only New Files

I have a large collection of photos. I only want to sync the latest (in my case right now 1 year) to a laptop.

I have tried syncovery and Goodsync without seeing any modifiers that would only select new/recent files. I have emails pending with syncovery because someone thought it was possible but I didn't see an option.

Another reader suggested Robocopy command. Reading over the modifiers I couldn't find the exact way to filter only new files AND removing all old files from the destination directory. (If I copy it over I would want to have it removed after 1 year. While still copying new files over).

Thank you for any help

davefromcamp

Posted 2013-12-15T11:48:43.540

Reputation: 112

Answers

I have tried the following and it works, but please make backups first before trying this out. I did.

Because you want this to always delete the old files on the destination, it is a little tricky to achieve the results desired, but definitely doable.

Download a software called Syncback, the best in my opinion.

You're gonna have to create 2 profiles of mirrors.

Profile #1 - Main Profile:

1 - Create a new profile

2 - Choose Mirror

3 - Choose Source and destination

4 - Click on Expert on the left side of the screen

5 - Navigate to Compare Options / Date & Time

6 - Set to: Ignore files that have NOT been modified within the last 365 days.

Or, change accordingly to what you need.

enter image description here

That's it for this profile

Profile #2 - Cleanse Profile

This is a dummy profile created only to delete old files from destination.

1 - Create a new profile

2 - Choose mirror

3 - As a source, choose a folder that will always be empty. You can create a folder anywhere in your drive, just make sure it'll never be touched and it will always be empty.

4 - The destination folder will be the same as the main profile

5 - Click on Expert on the left side of the screen

6 - Navigate to Compare Options / Date & Time

7 - Set to: Ignore files that have been modified within the last 365 days.

Or, change accordingly to what you need.

Please note, 1st profile have not been, 2nd profile have been.

You can use the software to run both profiles as a group, run both manually, or schedule them both. Make sure profile #1 runs first and that there is enough time for everything to be copied over.

That's all.

It seems complicated but it is very simple. That is now that I figured out how to do this.

Enjoy.

enter image description here

riseagainst

Posted 2013-12-15T11:48:43.540

Reputation: 295

This will not delete files in the destination older than 365 days. Or is there an additional option for that? If I copy it over I would want to have it removed after 1 year. – Rik – 2013-12-20T09:59:57.500

I'm not following, do you want to copy everything first, then only copy the new ones, or do you want to only copy the new ones? Anyway, on the above picture, click on Decision - Files and it'll give you the option of what to do if a file exists on the left or not on the right and vice-versa, if a file is older but the same. There's also the versions option. You only have to set it up the way you want it, but it can most definitely get the job done for what you need. – riseagainst – 2013-12-20T12:10:30.920

Another thought is, you can create a folder just to try-out your rules. Make a local backup copy of your photo folder and try the rules on that. When you copy a file, the "date created" changes, but the "modified date" does not. This way you can try with no fear. – riseagainst – 2013-12-20T12:11:16.830

As i understand it (summed up in my answer) is he wants a backup of the photo-directory on his laptop with only files newer than a year. So the copy function needs to copy new files (newer than a year) but also delete any files older than a year on the laptop. The files older than a year should stay on the original. – Rik – 2013-12-20T12:19:57.853

If that's the case, which was my initial understanding as well, the original answer would work. That is only syncing over the files newer than 365 days. – riseagainst – 2013-12-20T12:30:45.250

And that would still leave any photos older than 365 days on the laptop, wouldn't it ;-? – Rik – 2013-12-20T12:40:58.270

You're confusing me haha. If he wants all the photos on the laptop, but only synchronize the new ones, he can copy and paste the whole folder manually the first time, then do what my original answer stated. If he does not want the files older that a year on the laptop at all, just do what the original answer says. – riseagainst – 2013-12-20T13:13:54.453

You're correct for that moment in time. But after a week, there could be photos on the laptop which are 372 days old. And a week after that 379 days old. He wants a mechanism to copy new ones from the original AND also delete those photos (which already are on the laptop) *after they become older than a year* from the laptop. – Rik – 2013-12-20T13:53:18.350

@Rik I finally got what you're saying. Ok, in order for that to work, what he has to do is change step 2 from synchronize to mirror. Mirror will make sure that left and right are exactly the same, but with that exception of the date. Therefore, files older than 365 days will be deleted on the right. Thanks for clarifying that, will update the answer. Cheers. – riseagainst – 2013-12-20T14:14:48.370

Nope. Still doesn't work. I just tested this. If on the source and destination there are files older than 365 days they should be deleted from destination. Now they are just ignored. I almost thought this setting would do the trick but that doesn't work if the files are still on the source (which they are). As far as i can see there is no setting to delete files from the destination if they are still on the source. BTW Mirror is indeed a better way to go over Sync because the changes on the laptop should not go to the original.

– Rik – 2013-12-20T14:21:56.043

You sir are correct, I also just tried it and it only ignores the files, does not delete them. I will try to find an answer and updated it tonight. – riseagainst – 2013-12-20T14:56:50.303

@Rik updated the answer and confirmed that it works now. – riseagainst – 2013-12-20T19:11:33.083

Yeah, that looks right. Haven't tested it, but you have, so this should work. Basically it does the same as my solution with robocopy so now it's the choice of the OP to use a nice GUI or a batch-file. (I was sure it had to be done in two steps here too, but nice work figuring it out) – Rik – 2013-12-20T19:42:19.313

@Rik I never heard of robocopy but will take a look at it and your answer too. Thanks. – riseagainst – 2013-12-23T12:31:08.460

Thanks! Sorry for the late response I had a new baby. I will work on this soon. Looks like others want this too? I only want the files from the last year. Once older I want deleted from the small SSD. This 'year' may be adjusted later as well. – davefromcamp – 2014-01-03T11:37:39.743

Np, enjoy your baby! You can change the 365 days to years, months, weeks, minutes, hours, seconds... Whatever you need. – riseagainst – 2014-01-03T15:38:16.233

Let me sum up what i think you want (for clarity):

You want to copy, from a photo-directory, only the newer files, not older than 365 days to a laptop.
On that laptop you want to delete the files (from that directory) older than 365 days.

With robocopy you can do this in a couple of lines. (I don't think there is a way to do it in a 'one-liner'.)

@echo off

:: set your directories
set photo_collection=\\server\data\photos
set laptop_photos=\\laptop\data\photos

:: this is a temporary delete directory where robocopy move older files.
set robo_delete=%temp%\~robodel

:: this will move photos older than 365 to a "delete"-directory
:: /CREATE files are NOT actually copied but created as 0-files, this is faster
robocopy "%laptop_photos%" "%robo_delete%" /e /MOVE /CREATE /MINAGE:365

:: this temporary directory is deleted
rmdir "%robo_delete%" /s /q

:: this will copy only newer photos, never older than 365 days, to the laptop
robocopy "%photo_collection%" "%laptop_photos%" /MIR /MAXAGE:365

Notes:

The :: lines are just comments and can be removed.
I used \\\server\data and \\laptop\data to show you can use UNC-paths. You can, of course, just use C:\source_of_photos and D:\laptop_directory or something.
The laptop directory is always a backup of files (photos) newer than 365 days. If things go wrong you always have the originals in \\server\data\photos. Don't delete any photos from the original directory because then they will be gone forever.
The older files are temporarily moved to %temp%\~robodel. With the /CREATE they are not actually moved, but a 0-file is created for it. This is much faster.

Rik

Posted 2013-12-15T11:48:43.540

Reputation: 11 800

Thank you for your help! I will try this and get back to you – davefromcamp – 2014-01-03T11:39:39.063

You may try rsync with the --ignore-existing flag. We provide a Windows version called cwRsync. A free edition is available.

itefix

Posted 2013-12-15T11:48:43.540

Reputation: 21

No. Rsync is a one-way sync tool. Deleting files older than one year is a kind of maintenance task. However, we wrote a Nagios plugin called check_winfile which can help - check_winfile --target directory --filter "age lt -1 year" --delete will do the job. You can create a batch file running rsync first and then check_winfile for maintenance.

– itefix – 2013-12-20T12:05:00.647