2

Hi all into my server linux with Ubuntu10 in etc/crontab I have this:

0 0 * * * user rdiff-backup -v9 user@111.111.111.111::/var/www/ /mnt/server1/www
0 1 * * * user rdiff-backup -v9 user@222.222.222.222::/var/www/ /mnt/server2/www
0 2 * * * user rdiff-backup -v9 user@333.333.333.333::/var/www/ /mnt/server3/www

The server has an hard disk of 2Tb for example and with my backup I have occupied 1Tb about.
I done the backup every night, now if for an year or more this backup work the space of my hard disk I think that decrease because rdiff-backup make history of changed file.
Is possible to limit into my crontab to limit the history and mantain controlled the space of the hard disk? Something like: "after 30 version cancel the oldest".
is possible to do it into my file crontab?

Alessandro Minoccheri
  • 185
  • 1
  • 5
  • 16

3 Answers3

5

Since your cron jobs are run every day and you want to retain 30 versions, this is the same as deleting all incremental backups older than 30 days.

This is possible with rdiff-backup using the --remove-older-than option specifiying 30D or 1M. You can not combine --remove-older-than with backing up or restoring data in a single command. You will need to add lines to your crontab.

Another caveat is that --remove-older-than refuses to delete multiple increments at the same time, unless you specify the --force option. You could try running it once manually with --force and then let the cron jobs delete one increment each day. This could cause trouble if the cron jobs are not run for whatever reason (such as the server being down). I would include the --force option in the cron jobs.

Your new crontab could look something like this:

0 0 * * * user rdiff-backup -v9 user@111.111.111.111::/var/www/ /mnt/server1/www
0 1 * * * user rdiff-backup -v9 user@222.222.222.222::/var/www/ /mnt/server2/www
0 2 * * * user rdiff-backup -v9 user@333.333.333.333::/var/www/ /mnt/server3/www
0 3 * * * user rdiff-backup -v9 --remove-older-than 1M --force /mnt/server1/www
20 3 * * * user rdiff-backup -v9 --remove-older-than 1M --force /mnt/server2/www
40 3 * * * user rdiff-backup -v9 --remove-older-than 1M --force /mnt/server3/www

Whether or not you delete the old increments before creating the new backup or at what time you run which job is something you should decide.

To be clear: only the information needed to restore to an incrementally backed-up state earlier than 30 days ago will be removed, not old data that is still present right now or in a more recent incremental backup.

For more information about the --remove-older-than option you can read the man page:

--remove-older-than time_spec
          Remove the incremental backup  information  in  the  destination
          directory  that  has  been  around  longer  than the given time.
          time_spec can be either an absolute time, like "2002-01-04",  or
          a  time  interval.   The time interval is an integer followed by
          the character s, m, h, D, W, M, or Y, indicating  seconds,  min-
          utes,  hours,  days,  weeks, months, or years respectively, or a
          number of these concatenated.  For example, 32m  means  32  min-
          utes,  and 3W2D10h7s means 3 weeks, 2 days, 10 hours, and 7 sec-
          onds.  In this context, a month means 30 days,  a  year  is  365
          days, and a day is always 86400 seconds.

          rdiff-backup  cannot remove-older-than and back up or restore in
          a single session.  In order  to  both  backup  a  directory  and
          remove old files in it, you must run rdiff-backup twice.

          By  default,  rdiff-backup will only delete information from one
          session at a time.  To remove two or more sessions at  the  same
          time,  supply  the --force option (rdiff-backup will tell you if
          --force is required).

          Note that snapshots of deleted files are covered by this  opera-
          tion.  Thus if you deleted a file two weeks ago, backed up imme-
          diately afterwards, and then  ran  rdiff-backup  with  --remove-
          older-than  10D  today,  no  trace  of  that  file would remain.
          Finally, file selection options such as --include and  --exclude
          don't affect --remove-older-than.
Kenny Rasschaert
  • 8,925
  • 3
  • 41
  • 58
  • ok but it cancel all files oldest than 1 month? If I have a file that isn't changed since 30 days I don't want that it will be cancel... I think that the system cancel only the copy not the original oldest than 30 days – Alessandro Minoccheri Apr 03 '13 at 08:23
  • A complete duplicate of the `/var/www/` folder on your server1 is kept in your `/mnt/server1/www`, as well as a `rdiff-backup-data` directory which contains metadata to help you restore to certain increments (to a specific date if you back up every day). The only thing that now will change is you can only go back 30 days in time when restoring a backup. The entire directory as it exists on the server is also present locally, always, even if the files on your server haven't been changed in years. – Kenny Rasschaert Apr 03 '13 at 08:27
  • what i have to understand: if a file doesn't change in 30 or more days, will the function delete it or it will remain available? Because I have file that doesn't change for months because is an archive. But if I want to retrieve It from my backup their has to be in my backup. – Alessandro Minoccheri Apr 03 '13 at 09:17
  • If it is still on the server, it will still be in your backups. – Kenny Rasschaert Apr 03 '13 at 09:19
  • Thanks this is the answer that I wanted! Thabnks a lot +1 for the perfect explain – Alessandro Minoccheri Apr 03 '13 at 09:21
  • I quickly ran this test to prove that the archive remains even after deleting all incremental backups. This should put your mind at ease. http://pastebin.com/vgDBEPcx – Kenny Rasschaert Apr 03 '13 at 09:38
1

You could try

rdiff-backup --remove-older-than 1M /mnt/server1/www
user9517
  • 114,104
  • 20
  • 206
  • 289
Txeyen
  • 5
  • 1
  • Can i put it into my file crontab? And it delete all files oldest than 1 month? – Alessandro Minoccheri Mar 26 '13 at 08:44
  • You could also do "find /mnt/server1/www -mtime +30 -exec rm {} \;" etc as a separate crontab entry. – J Adams Apr 03 '13 at 16:13
  • 1
    @JAdams no, this is extremely dangerous! Using find and rm will delete all files that exists and are older than 30 days, including files that were not changed in between (i.e. the result is that only files that were changed during the last 30 days are kept!). The OP wants to keep a complete, valid backup of his data, and only discard past versions of existing, newer files. – MoonCactus Aug 13 '16 at 08:08
1

You need to add another line to your crontab to tell rdiff-backup to remove the files.

From http://www.nongnu.org/rdiff-backup/examples.html:

This section assumes that rdiff-backup has been used in the past to back up to host.net::/remote-dir, but all commands would work locally too, if the hostname were ommitted.

This commands deletes all information concerning file versions which have not been current for 2 weeks:

rdiff-backup --remove-older-than 2W host.net::/remote-dir Note that an existing file which hasn't changed for a year will still be preserved. But a file which was deleted 15 days ago cannot be restored after this command is run. As when restoring, there are a variety of ways to specify the time. The 20B below tells rdiff-backup to only preserve information from the last 20 rdiff-backup sessions. (nnB syntax is only available in versions after 0.13.1.)

rdiff-backup --remove-older-than 20B host.net::/remote-dir

Thai Wood
  • 353
  • 2
  • 8