Linux script , Incremental backup in the same server

Question

Could you please someone guide for the below incremental backup script, i have already written the script. I am facing an issue while looping the script.

Requirement i need : Consider a source folder contains A, B, C when i ran the script A, B, C will move to the destination folder. Again in source folder new files/directories added i.e. D,E when again if i ran the script now it should check for already present files. Here in our case A,B,C is already present in the destination. So only D,E have to move to the destination.

From the below code am able to backup , but how to make in loop Please guide

#!/bin/bash
####################################
#
# Backup IS server script script.
#
####################################

# What to backup. 
Integrationserver="/home/ec2-user/source"

# Where to backup to.
dest="/home/ec2-user/destination"


# Create archive filename.
#date=$(date +%F)
IS=source
hostname=$(hostname -s)
#archive_file="$hostname-$IS-$date.tar.gz"
archive_file="$hostname-$IS.tar.gz"

# Print start status message.
echo "Backing up $Integrationserver to $dest/$archive_file"
date
echo

# Backup the files using tar.
tar --exclude=/home/ec2-user/source/logs* --exclude=/home/ec2-user/source/TC*  -zcf $dest/$archive_file $Integrationserver

# Print end status message.
echo
echo "Backup finished"
date

The below code is working fine with all the conditions and requirements , But not sure how to exclude the multiple directories.. please guide

#!/bin/bash

source=/home/ec2-user/source
dest=/home/ec2-user/destination

for file in $(find $source -printf "%P\n") ; do
  if [ -a $dest/$file ] ; then
    if [ $source/$file -nt $dest/$file ]; then
      echo "Newer file detected, copying .."
      cp -r $source/$file $dest/$file
    else
      echo "File $file exists, skipping"
    fi 
  else
    echo "$file is being copied over to $dest"
    cp -r $source/$file $dest/$file
  fi 
done

You want the script to keep executing in an infinite loop, is it? — Vinicius, Jul 05 '22 at 13:24
@Vinicius, yes i want to like that , but in that i need to exclude few directories. Could you please help — Varun K M, Jul 11 '22 at 09:36
@Vinicius, Thank you for sharing the script. I have used the first script , its working fine. After running the script , the script is not ending from the result terminal . Manually i am stopping the script .. please assist , Automatically how can the script stop after copying. — Varun K M, Jul 19 '22 at 10:09
I don't understand exactly what you need, you first asked for a infinite loop. If you don't want it to loop for ever just run the `rsync`, you don't need the `while true`. What am I missing? — Vinicius, Jul 21 '22 at 09:32
@Vinicius , I mean to say the script is going to infinite loop. Once if the files copied from source to destination, then the loop should breakout. — Varun K M, Jul 25 '22 at 06:52
You're welcome. Please remember to accept the answer so it can help others with the same issue. — Vinicius, Aug 04 '22 at 09:17

Vinicius · Accepted Answer · 2022-08-04T09:18:16.910

You can achieve this with a much simpler script using rsync:

#!/bin/bash

source=/home/ec2-user/source
dest=/home/ec2-user/destination

changed=0

while [[ $changed -eq 0 ]]; do
    # The next command just count changes, does not copy anything
    changed=$(rsync -rin $source/ $dest | grep "^>f" | wc -l)
    sleep 1
done

echo "Copying $changed files"

rsync -qrt $source/ $dest

I used the sleep 1 to avoid a resource intensive loop, but there is a better way using inotify-tools:

inotifywait -e modify,create -r $source && \
  rsync -qrt $source/ $dest

The inotifywait command will stay blocked until some file is either modified or created (the -e modify,create option) and it is very efficient.

Linux script , Incremental backup in the same server

1 Answers1