mdadm: safest way to grow RAID 5 (4x2TB) with 2x4TB

Question

My situation:
I am currently running a RAID 5 with 4x2 TB disks and would like to grow the array with 2 x 4 TB disks. The goal would be that there is a RAID 5 with 4 x 4 TB (the two 4 TB disks and two RAID 0 with each two 2 TB disks). Unfortunately I don't have enough space to make a full backup, so I can't build a new RAID. The important data is backed up on an offsite server, which leaves about 4TB of unprotected, less important data which I still try to avoid loosing.

My question:
What is the safest way to achieve that?

• make a new RAID 1 with the two 4 TB disks copy the data and then add the other disks and change the RAID 1 to RAID 5 and grow it?
• replace two of the 2 TB disks with the new 4 TB disks, merge the freed disks to a RAID 0, add it to replace a third 2 TB disk with the RAID 0, remove the last 2 TB disk and make another RAID 0 and replace the missing 4th disk with it and then grow the Array?

Which of the two options is safest? Is there a better Way of doing what I want?

Answer 1

I am not a big fan of layering RAID the way you are considering (RAID5 on top of RAID0).

I would prefer to create two separate RAID5s.

You have proposed to turn your 4x 2TB hard drives into (virtually) 2x 4TB HDs, and add 2 more 4TB HDs. This will give you 12TB of usable space.

As an alternative, consider chopping the two new 4TBs in half and expanding the original RAID5 to 6x 2TB. This leaves 2x 2TB for the bottom halves of the new drives. Those can be combined into a RAID1. The end result is 12TB of usable space.

Another arrangement that gives you the same amount of usable space is to build 2 RAID5s each from 4x 2TB partitions. Combine the top half of the new drives with two of the old drives. Combine the bottom half of the new drives with the other two old drives. You end up with 12TB of usable space.

I have invented a theory (which was probably first invented about 1 week after the original scientist invented RAID) that the maximum amount of usable space you can get out of a RAID5 is equal to the sum of all the available drives without the largest of those drives. You can RAID them in a way that gives you less, but you can't RAID them in a way that gives you more.

For your particular situation, the safest procedure is the middle one on the diagram where you expand the existing RAID5 with two new partitions, and then add a mirror with the remainder of the new drives. If you are already using LVM (and you should totally be using LVM) you can expand the PV on the old raid and add the new mirror's PV to the volume group.

Answer 2

Your only "safe" option is to create a degraded RAID5 with your 4tb disks (7.9TB), use something like ZFS or LVM + EXT4 on that so you can grow the filesystem later. Copy on your data, turn your 4 x 2TB disks into 2 x 4TB RAID0 arrays, add them to the degraded array pushing it up to 12TB & then grow the file system

If all that works without any data loss, go out & buy yourself some lottery tickets.

(Disclaimer: I would not attempt this, I would go out and purchase 1 more 4TB disk)