3

I have a large Windows file server with about 2TB of data, about half of that is over 2 years old (based on modification date). What would be the best way of archiving off that old data, using scripts or whatever, but without spending big money on a full archiving (HSM) system?

The purpose is to reduce the backup window, because all that old data is backed up every week when really it never changes and can be backed up much less frequently, thus reducing tape requirements.

BTW the archive would be on another disk, with read-only permissions.

Has anyone implemented something similar? How would users access the archive easily?

PowerApp101
  • 2,604
  • 1
  • 20
  • 28

3 Answers3

5

I do all of my backups with Rsync. Even if the server is Windows (which mine are) you can backup with a Linux computer or even use a Windows version of Rsync.

Rsync will check which files have been modified since the last time they were backed up and only sync the difference. It will even do binary diffs and only transfer the changes of the file instead of transferring the entire file.

The bottom line is Rsync is really useful and extremely efficient. The only downside is that you have to make your own scripts to use it effectively. I wrote an article on how to make daily, weekly, monthly backups here: http://www.marksanborn.net/howto/use-rsync-for-daily-weekly-and-full-monthly-backups/

sanbornm
  • 105
  • 1
  • 6
  • 1
    Instead of having a daily/weekly/monthly directory and writing over them you should really look at --link-dest, so you can make properly incremental backups without overwriting them. – theotherreceive Aug 25 '09 at 05:13
  • Or dirvish if you have a linux box. It'll handle all the linking and expiring of snapshots. – Ryaner Aug 25 '09 at 14:52
2

If you use PowerShell, you could use a script like the following. This would go through each directory on the root of E:, match only files that are older or equal to 8/24/2007 (two years old) and move them to the path stored in $archiveTarget.

$sourceDir = "E:\"
$archiveTarget = "\\server\share\arhive\folder\"

Get-ChildItem $sourceDir -recurse | Where-Object {!$_.PsIsContainer -and  $_.LastWriteTime -le "8/24/2007"} | ForEach { Move-Item -path $_.FullName -destination $archiveTarget }

You could change the variable to target specific drives or folders, one at a time, until you're done. Use PowerShell's -whatif argument to see what would happen without actually moving anything. Could also pipe the results to a text file.

Kai
  • 470
  • 2
  • 7
  • FYI, this is just an example. The script includes no logic for handling duplicates, although, it wouldn't be difficult with Test-Path and Rename-Item. – Kai Aug 25 '09 at 02:58
  • 1
    What would also be good with that is when it move the file it then writes a text file with the same name and in the text file is says this file moved to archive server xxxx on date – SpaceManSpiff Aug 25 '09 at 12:13
  • SpaceMan, probably overkill for me. But easy to add by inserting more code into the ForEach loop. e.g., Add-Content "$_.txt" "Moved to $archiveTarget on $(Get-Date)" – Kai Aug 25 '09 at 13:35
0

Try http://www.mltek.co.uk/archivemanager.aspx.

I was searching the web for the same kind of solution and I found them. Looks like they do a really cheap 'lite' version that can move the files somewhere else (using a UNC path), then even leave seamless stubs behind so it looks like the files are still there.

Could be worth a shot.

Mark
  • 1