Divide out local files in different servers with limited space with rsync

1

1

I have 80 GB of files in a local directory that I want to back up, dividing them out onto different servers with a limited space quota, using rsync, like so:

LOCAL 80 Gb ====> server1: first 30 Gb
             ||
             ||=> server2: following 15 Gb
             ||=> server3: last 35 Gb

Every server is mounted on the local machine using WebDAV connection and davfs2, so I can avoid all SSH, keys, and similar ramblings. How can I split my files onto several destinations?

fradeve

Posted 2012-03-23T09:14:54.257

Reputation: 393

What's the question? – Chris2048 – 2012-04-09T13:27:08.250

Sorry for the lack of clearness: is there a way to "divide out local files in different servers" using rsync? – fradeve – 2012-04-10T18:01:10.227

How close to the quota do you want to get? Do you wish to specify the figure yourself, or is the system quota enforced (i.e. you won't be able to store files once over it), without a 'grace' period, that is. – Chris2048 – 2012-04-11T19:05:44.150

To make the rsync script simple, the quota cold be specified manually – fradeve – 2012-04-13T14:57:35.067

Answers

1

If these files are just for backup you could

  • create a (dated?) tar file of all the files you want to backup
  • then use the 'split' command to break it up into appropriately sized chunks
  • distribute one chunk to each server

As far as I now rsync doesn't have the functionality to distibute a file across servers. Another possibility is this:

  • create one directory representing each server
  • manually move/copy the files you want in each server to each of these directories, but if you copy, use hard-links so no more space is used.
  • rsync each directory to its respective server. create a script to do this if this is a regular task.

Chris2048

Posted 2012-03-23T09:14:54.257

Reputation: 543

Asked: 2012-03-23T09:14:54.257

Viewed: 587 times

Active: 2016-02-28T18:40:42.190