What are the good ways to migrate a RAID array to bigger disks?

Question

I'm looking to replace my hard-drives with bigger-ones and I would like to know what are the best practices in doing this. I have a HP ProLiant with its 256MB version of RAID controller:

03:00.0 RAID bus controller: Hewlett-Packard Company Smart Array G6 controllers (rev 01)

I also have a pair of 10.000 rpm 146GB SAS drives in RAID 1. The OS is Ubuntu 10.04, 64-bit. I'm not using LVM.

zduric@pbksrv002:~$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p1     130G  119G  9.7G  93% /
none                  3.0G  192K  3.0G   1% /dev
none                  3.0G     0  3.0G   0% /dev/shm
none                  3.0G   64K  3.0G   1% /var/run
none                  3.0G     0  3.0G   0% /var/lock
none                  3.0G     0  3.0G   0% /lib/init/rw
zduric@pbksrv002:~$ mount
/dev/cciss/c0d0p1 on / type ext4 (rw,errors=remount-ro)

We experienced a huge growth in content stored there and now I need to replace this array with another pair, 300GB disks. Ubuntu 10.04 is up there with a tomcat and a document management application (which is filling up most of the drive). I'm looking for the best way to do it.

I was thinking that I could do one of the following:

Option I: replacing a drive at a time. Basically, I'd do something like this:

1. Replace drive 1 with the new drive.
2. Rebuild the array to put the data on the new drive.
3. Replace drive 2 with the new drive.
4. Rebuild the array.

I don't like the two-times rebuilding part about it. I also don't know, if that would work at all and even worse, how much downtime is that. (around 115GB data, OS included.)

Option II: Create a new array. The idea here is that I would add the new disks and create a second array. I could then migrate my tomcat application to the new array - this would mean a simple cp operation.

I like this option much better, because it would also leave me the first array to use solely for the OS and I could do OS upgrades more easily in the future.

Is the option II doable at all? Where could I find some good resources on doing this whole procedure?

Are there other, better, smarter options I should try?

Thanks in advance.

Answer 1

You have options...

You specific case is covered on page 81 of the HP Smart Array Controllers User Guide.

Follow that with the information on page 20 of the HP Smart Array Reference Guide for an overview of what to do once the new capacity is available...

• Do you have a battery-backed or flash-backed cache unit on your RAID controller?
• Can you post the output of df -h?
• Are you using LVM?

Expansion of a logical drive using an HP Smart Array RAID controller can be accomplished by replacing each disk with a larger drive, one at a time, allowing time to rebuild in between. That will result in a larger logical drive (in HP terms) that will provide unused space that can be allocated to the existing drive or carved into a new logical drive.

Doing the drive replacement will not require much downtime, other than a reboot to recognize the new array size. An online expansion (using the HP Array Configuration Utility) or an offline expansion (using the bootable SmartStart CD) is needed to expand the logical drive. And whatever needs to be done at the filesystem level to resize your partitions. I can do this in one or two reboots, depending on how the server OS is configured.

It may also make sense to install the HP management agents and drivers for your OS. Also updating the firmware of the server components may also be useful. There's a bootable firmware DVD available to accomplish that.

Answer 2

If you just replace the drives one at a time you might have problems as the new array build will be the same size as the previous; the filesystem won't know about the expansion.

Then you would have to try resizing the partition and filesystem on the fly. Assuming you RAID controller wouldn't have a problem with this. And the filesystem. Some systems are better suited for these acrobatics than others (using LVM?)

What I personally would do is look at building another server and running it in parallel with the original, then when everything is in the clear, cut it over to the new production system.

If you couldn't do that, I'd schedule a time to shut down the server, create a brand-spankin' new array entirely from scratch, then restore from backup. This would also have the benefit of having the old drives available if there was a problem and you had to abort the rebuild.

Those two approaches would save the "taking chances" part, test your backups, and reduce the chances of having to cut over to "plan B" (or C or D) at three in the morning, when you're already cranky, tired, and more apt to make stupid mistakes.

Otherwise you're going to have to look at your filesystem and volume management software (LVM and filesystem resize utilities) and hope your RAID controller doesn't do something goofy to cause issues which might make you end up having to restore from backup anyway.

That's my take on it...perhaps someone with a model controller and server more like yours can give personal experiences. My approach tends to be more generic and I try to cut out the "just in case" edge problems.

Answer 3

Option 1 is not really an option, since copying an existing RAID array's contents to a new disk does not increase the array size.

Unless the HP RAID controller offers online array resizing it's not even physically possible - and even then this only enlarges the array, not the partitions on them.

Go for option 2: add the drives to the controller, make a new RAID-1 array, create a new partition using the full size of the array, and move your content over.

There will be some downtime while you are copying the data, but you can minimize this by using rsync after the first full copy action, and while the applications are stopped.