7

My question is... what multiplexing factor in NetBackup is recommended / do you use for Oracle RMAN backups over 1Gb/s management network to LTO3?

JB

Background:

With enterprise backup tools like NetBackup, there is the concept of Multiplexing which is the combining of data from a number of backup clients simultaneously in order to feed modern high speed tape drives as quickly as possible.

The number of simultaneous client data streams interleaved is determined by the Multiplexing factor. The higher the multiplexing factor, the more data fed to the tape drive, but the slower any restores.

As overall restore speed is mostly determined by the messing about (log incident, determine if tapes are available, recall from offsite, load, inventory, etc. ) than by the actual tape restore speed, I feel confident using a high factor for filesystem backups.

Oracle backups with large datasets, which are more often restored all together, pose a different challenge to filesystem backups.

Jonathan Bourke
  • 91
  • 2
  • 7

3 Answers3

1

The first thing to check is how much network (TCP) throughput your server can handle. Use netcat, etc. If it is less than around 30 MB/s, multiplexing from network is of no use to you, and my further advice can be ignored. Work on tuning your network throughput instead. Now, to the point.

The LTO3 drive, just like any other linear tape drive, works well only when it gets a stream of data with a certain constant throughput.

The tape is passing under the head at a high speed, and you don't want to stop it. At each stop the drive has to perform lengthy procedure: decelerate to full stop, accelerate back, pass the end-of-data point, decelerate again, accelerate forth to reach the end-of-data point. When data is not feeded by NetBackup fast enough, the buffer underruns frequently and so the drive has to stop/rewind/start frequently. The performance is hurt dramatically. This is called "start-stop" operation or "shoe shining".

Drive adjusts the speed of the tape somewhat, but not very much, it can drop to about 50% of maximum speed.

The whole point of Netbackup multiplexing is to provide a better streaming throughput and avoid start-stop operation. Check the throughput of your RMAN backup, if it is 30 MB/s or less you have a classic start-stop operation.

Now, let me make one thing clear. If you do not have start-stop I would not recommend multiplexing RMAN backups at all. RMAN is complicated enough without multiplexing. I don't want to mess with RMAN, I want my restore to be as fast, easy and seamless as possible.

However, if you find your backup throughput unacceptably low, I would suggest implementing around three multiplexing streams for starters. Increase the number each night until you will not gain any more throughput. And make sure each stream is coming from the different disk spindle(s). Not from different partitions/tablespaces/filesystems/databases/servers/LUNs/other-virtualization-layers. These matter little, if any. Physical disk spindles. If you feed many streams from the same spindles you will just cause thrashing and overall performance will drop even more.

Note: NetBackup theoretically can also de-multiplex a restore. If I remember correctly, it pauses a little before a restore, to give a chance for more restore attempts to launch. In this case they will run jointly, just like multiplexed backups. But please verify this with a manual, I am only 90% sure on this one.

kubanczyk
  • 13,502
  • 5
  • 40
  • 55
  • Thanks for the great explaination. I am aware of the shoe shining issue, and each single RMAN stream does about 20mb/s+ taking into account the SAN, the network, the hardware, etc. And, like you suggested, it looks like 3-4 times MPX might be the sweet point. – Jonathan Bourke Jul 30 '09 at 05:19
1

That depends entirely on if your oracle server can move data fast enough to keep the LTO3 drives streaming. I do not multiplex Oracle data because the large files get served up fast enough to keep the drives running at an acceptible speed.

Before we replaced the Oracle servers however, and they only backed up at about half their current speed I did in fact multiplex them.

It is important to note that restores are slightly slower when multiplexed with NetBackup, but not much slower. I know for certian you can de-multiplex on a restore. We do it all the time both to do restore tests, and on rare occasion to actually replace lost data.

I strongly recommend testing both ways and seeing if you can keep your LTO3 drives moving fast enough without multiplexing.

Laura Thomas
  • 2,825
  • 1
  • 26
  • 24
  • Hi Laura, What do you consider an acceptable speed for your LTO3 drives? I want to get them running at 80mb/s+ and my Oracle RMAN backups seem to be able to manage approx 20 mb/s per stream, which would imply an MPX of 4 or so. In the past, while I have hear the "restores take longer" comment, I have yet to meet a DBA who would restore to production, even a compromised production db. They much rather restore to another system, and work with the data, hence they are prepared to wait a little longer. – Jonathan Bourke Jul 30 '09 at 05:12
0

I found the simplest way to solve this was to write the initial backup to disk then duplicate the backup images to tape.

Multiplexed backups are more likely to span tapes, are harder to import or use outside netbackup, are slower to restore, and are an all-around ugly hack created to prevent shoe-shining tapes.

Netbackup has really nice direct-to-disk functionality and the CLI tools make it pretty easy to script up image duplication mechanisms.

chris
  • 11,784
  • 6
  • 41
  • 51
  • All in good time :-) backing up 7TB oracle databases, requires a significant amount of staging space for this type of backup. – Jonathan Bourke Jul 30 '09 at 05:20
  • Anyone who can afford LTO4 can afford one of these http://www.iqss.com/fc_to_sata_raid.html with 8 big disks... – chris Jul 30 '09 at 11:31
  • 1
    You would be surprised... often it is not the money, it is the projects, the budgets, the signoffs, the approved vendors list, the tendering process, etc. And backup isn't always the "sexiest" of project areas to be pushing... – Jonathan Bourke Jul 31 '09 at 00:21
  • Oh yeah, people don't care about their backups... only the restores. I'm completely happy to be away from doing backups. – chris Jul 31 '09 at 01:22