Backup tape compression

Question

What things should I check to confirm that compression is actually happening on our tape backup system? Although the tapes are marked as 200G/520G (native/compressed) capacity, they seem to fill up before the 200G mark (some less than 100G).

I'm using

Sony AIT-4 tape autochanger
Sony SDX4-200C (AIT-4) tapes
Ubuntu Lucid
Bacula

I've tried checking hardware compression with: tapeinfo -f /dev/nst0, which gives

Product Type: Tape Drive
Vendor ID: 'SONY    '
Product ID: 'SDX-900V        '
Revision: '0102'
Attached Changer API: No
SerialNumber: '0001000036'
MinBlock: 2
MaxBlock: 8388608
SCSI ID: 1
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: Not Loaded
Density Code: 0x33
BlockSize: 0
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0x3
DeCompType: 0x3
BOP: yes
Block Position: 0
Partition 0 Remaining Kbytes: 201778000
Partition 0 Size in Kbytes: 201779000
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 0

... so I presume it's on.

Notes:

The Bacula documentation says hardware compression needs to be enable with "system tools such as mt"
The data "should" compress well. The largest proportion is text files with lots of redundancy (genomic data). bzip often reduces to 10-30%
I have compression (i.e. software compression) turned off in Bacula

How are you determining that your tapes are "filling up before the 200G mark"? — voretaq7, Sep 06 '12 at 21:13
In the Bacula GUI, under media. The tapes are marked eg 79G, but "Full". It's not a job limit, because the tape only has one job written to it — pufferfish, Sep 07 '12 at 08:54
If your tapes are filling up before even getting close to the uncompressed capacity you may be appending to the tapes and only seeing what has been added, rather than the full amount on them. — John Gardeniers, Sep 07 '12 at 10:20
Do you have `UseVolumeOnce = yes` or a low `Maximum Volume Jobs = ###` setting? in your Bacula configuration? Bacula's tape usage counter should be for the whole volume, but if you aren't seeing even the uncompressed capacity of your tapes it sounds like Bacula is misconfigured somehow. (You should however perform the tests Mircea and I outlined below to confirm that the problem is not the drive itself) — voretaq7, Sep 07 '12 at 15:10

score 2 · Answer 1 · answered Sep 06 '12 at 12:59

2

DataCompEnabled: yes means that the compression is enabled.

answered Sep 06 '12 at 12:59

Mircea Vutcovici

16,706
4
52
80

Yes, that's why I'm puzzled – pufferfish Sep 06 '12 at 13:01
May be your data is not compressible. Try to write text files or HTMLs on the tape. – Mircea Vutcovici Sep 06 '12 at 13:03
2

Try disabling compression and see what happens, whether you get the exact 200GB you expect (and I presume that this isn't 1024 gigs vs 1000 gigs that is throwing you off?). – Dan Pritts Sep 06 '12 at 14:21
Most of the large data files are text files with lots of redundancy. Compress well with for example gzip. – pufferfish Sep 07 '12 at 08:56

score 0 · Answer 2 · answered Sep 06 '12 at 21:18

As Mircea pointed out the DataCompEnabled: yes in your tapeinfo output means that hardware compression is enabled for your tape drive.

Compression is not a magic thing that takes the input and reduces it in size by 50% which is what most tape vendors claim in their advertising -- If your Bacula installation is already doing gzip (or some other) compression your tape drive's hardware compression may not be optimally effective (in fact if Bacula is doing effective compression your tape drive's hardware compression may make the data take up more space).

If you want to test your tape drive's compression to be sure it's working properly the best way to go about it is usually to use tar to write a tape that consists of highly compressible data (text files are good for this) and see how much tape you use to store that data.

score 0 · Answer 3 · answered Nov 28 '12 at 13:49

0

How old are your tapes? Our LTO-1 tapes often fill before the 100GB because they're OLD (new tapes fill at 140-150GB).

answered Nov 28 '12 at 13:49

AndrewQ

390
3
13

Backup tape compression

3 Answers3