0

Our Backup Script runs every day. At the end of the month, every file should be moved to the "Staging" location, in order to be copied "to tape"...

So, I wrote a PS-Script, moving all the files. Until now, this happened "synchronous" which was perfectly working fine..., but two days of backups where lost, cause moving the files took about 2 day, blocking daily calls of the same script for daily backups...

Now i want to improve this, by moving files (through robocopy) in an asynchronous way, so the next-days backup can start whenever required...

I modified the existing script to the following - however, it throws an exception after starting "all jobs" (If it's last day of month!):

enter image description here

And here's the script:

#definitions
$backupJobName = "MyCompany Komplett"
$backupCopyJobName = "CopyToTape"
$backupFileLocation = "V:\MyCompany Komplett"
$backupFileToTapeLocation = "T:\"

##############################################################
# Modify content bellow only if aware of what you are doing! #
##############################################################
. C:\shared.ps1

Start-Transcript -Path "C:\daily_backup.txt" -Append

# imports
Add-PSSnapin VeeamPSSnapIn

#script
$dateTime = Get-Date;
$firstDayOfMonth = getFirstDayOfMonth
$lastDayOfMonth = getLastDayOfMonth

if (isSameDay $dateTime $lastDayOfMonth){
  $dateTime.ToString() + ": Last day of Month:"
  $dateTime.ToString() + ": 1.) Run final incremental Backup of this month."
  #$job = Get-VBRJob | where {$_.Name –eq $backupJobName} | Start-VBRJob

  #output for logs
  #$job

  #Copy Files to staging folder.
  $dateTime = Get-Date;
  $dateTime.ToString() + ": 2.) Copy Files to Tape-Staging-Location"
  $elements = Get-ChildItem -Path $backupFileLocation -filter "*.vrb"| ? { $_.lastwritetime.month -eq $dateTime.month -and $_.lastwritetime.year -eq $dateTime.year}
  $elements_vbk = Get-ChildItem -Path $backupFileLocation -filter "*.vbk" | ? { $_.lastwritetime.day -eq $dateTime.day -and $_.lastwritetime.month -eq $dateTime.month -and $_.lastwritetime.year -eq $dateTime.year}
  $elements_vbm = Get-ChildItem -Path $backupFileLocation -filter "*.vbm" 

  "List of relevant vrb files"
  $elements | Select Name | ForEach-Object { write-host $_.Name } #output for logs

  "List of relevant vbk files"
  $elements_vbk | Select Name | ForEach-Object { write-host $_.Name } #output for logs

  # copy elements to Staging location (Async!)
  # copy: vrb from current month
  # copy: vbk from today.
  "Copy Job started for:"
  $elements_vbm | Select Name | ForEach-Object { Start-Job -ScriptBlock { robocopy $backupFileLocation $backupFileToTapeLocation $_.name } }
  $elements_vbk | Select Name | ForEach-Object { Start-Job -ScriptBlock { robocopy $backupFileLocation $backupFileToTapeLocation $_.name } }
  $elements | Select Name | ForEach-Object { Start-Job -ScriptBlock { robocopy $backupFileLocation $backupFileToTapeLocation $_.name } } 

  ###
  ###
  ###
  ###  Here I get the exception. Running robocopy without "Start-Job" works, but takes tooo long...
  ###
  ###
  ###
} elseif (isSameDay $dateTime $firstDayOfMonth){

  #do full Backup.
  $dateTime.ToString() + ": First day of month: Running Full Backup this time."
  $job = Get-VBRJob | where {$_.Name –eq $backupJobName} | Start-VBRJob -FullBackup
  #Output for logs
  $job
}else{
  #do Reverse Incremental Backup.
  $dateTime.ToString() + ": Regular Day of Month: Running Reverse Incremental Backup."

  $job = Get-VBRJob | where {$_.Name –eq $backupJobName} | Start-VBRJob

  #Output for logs
  $job
}

Stop-Transcript
dognose
  • 164
  • 10
  • If I understood you correctly, every day you're starting a new job that takes two days to complete? That means the jobs will overlap. Assuming these jobs are I/O bound (you're using your maximum bandwidth with even one job), then each overlapping job will slow things down even more. You'll get many jobs overlapping by the end, which is what your screenshot looks like. You probably need to find another approach. Consider simply starting the next copy job as soon as the first finishes, or only copying changed files. – Matthew Wetmore May 06 '18 at 05:10
  • No, that job just is triggered the first of every month. But since it runs 2 days, it will cause the execution of the reverse incremental backups to be blocked 2 days, which it shouldn't if possible. – dognose May 09 '18 at 06:28

0 Answers0