1

So the place where I'm working at has a nifty little Red Hat Enterprise server and a tape backup system that they want me to get working backing up the large amounts of video content and resources produced by the company.
The desired situation would be to keep a local copy of all the data on said Red Hat server and then copy all changed files to tapes which will then be sent offsite. I know that I can use either unison or rsync for the initial local data backup but what I need to know is if there is a way to get a list of only the changed files which will then be copied (in full) to the tape.
I've found several possible solutions through auditing software but they all seem to require a kernel recompile (which is not feasible right now). I also was promised that unison could do this "out of the box" but I haven't figured out how.
Is there a solution to my presented question or am I going about this at a fundamentally wrong way? Any help/advice would be appreciated!
Cheers! -Russell C

Russell C
  • 115
  • 1
  • 1
  • 8

2 Answers2

1

I would use a backup program like Bacula to do this. It keeps a database of your backups so it don't need to compare file sizes and time stamps against already backed up files like rsync.

If you want to use rsync, you could use find to find files changed after your last backup, write this to a file and read the file names to copy from this file with rsync's --files-from option. This opens timing issues, though.

Sven
  • 97,248
  • 13
  • 177
  • 225
  • Bacula looks really cool but it may be a bit much for a single junior engineer like me to set up without support from anyone else here. :( I'll look into it some more anyways. – Russell C Feb 09 '11 at 19:07
  • Granted, Bacula is not the easiest software, but it's quite powerful. But beside Bacula, there are other possibilites like Amanda, basic tools like tar (which stands for tape archiver), commercial packages like Arkeia or BRU and countless other options. – Sven Feb 09 '11 at 19:23
  • But one comment which I hope you don't take personal: If you think Bacula is over your head and you are just a lone beginner with no support from colleagues, things are seriously wrong at your company: Backup is **extremely** important, it's among the most important tasks for a system adminstrator, very difficult to get right and test and really nothing you would give a junior admin to "do it somehow". I would really recommend that you ask more senior colleagues to do this together with them, so you get it right and don't learn you missed a small, but important bit after disaster struck. – Sven Feb 09 '11 at 19:32
  • Yup, I'm actually at a small startup that's fairly new and still hiring people (we're on the lookout for an actual IT guy now). Technically I'm more supposed to be a scripting guy but I said I'd give a crack at this anyway. They have a series of drive mirroring on the server but want some offsite backups, too. Hopefully we can get this working before we actually have anything important to back up. – Russell C Feb 09 '11 at 19:42
  • I also know that I'll be using tar quite a bit because of the tap backups. I may start another thread later about some specifics of tar if I can't find answers already here or on stack overflow. – Russell C Feb 09 '11 at 19:49
1

Whats wrong with the normal rsync output?

/tmp # ls -l ra
total 8
-rw-r--r-- 1 phemmer unix  5 2011/02/09-16:42:49 adsf
-rw-r--r-- 1 phemmer unix 29 2011/02/09-16:49:41 date

/tmp # date > ra/date

/tmp # rsync -Haxv  ra rb
building file list ... done
ra/date

sent 180 bytes  received 42 bytes  444.00 bytes/sec
total size is 34  speedup is 0.15

Just pipe rsync out somewhere rsync ... 1>/path/to/log and then parse through the log for the list of files changed

P.S. You do not want to use unison for this. Unison is a 2-way rsync, for backups you want 1-way.

phemmer
  • 5,789
  • 2
  • 26
  • 35
  • I was just wondering if there was a simpler way to redirect the files instead of having to parse an additional file. I guess I must have also missed the fact that I could capture the simplified output to a file with a simple redirection. – Russell C Feb 09 '11 at 18:56
  • But if you store your backup data on a tape, there is no easy/fast way for rsync to access the files on the tape to make the comparison, this only works if you have a normal tree of files which rsync can access. – Sven Feb 09 '11 at 19:17
  • @russel you could use something like unionfs and get exactly what you want in 1 step, but that'd be even more complex than parsing a log file – phemmer Feb 10 '11 at 00:12
  • @svenw, he stated that he was going to be keeping a copy of the files on the local machine, so it would work fine. the only way to do it if all the files were on tape would be to generate checksums of the files before they got sent to tape, and then re-checksum every file to see if its change. which isnt what he's doing. – phemmer Feb 10 '11 at 00:13
  • The way I understood the question is that the local copy is the primary data/backup source, so there is no easy way for rsync to do a comparison. In this case, tar is a better tool anyway, with either the -g or the -N options. – Sven Feb 10 '11 at 00:22