3

I am making a huge RAID10 device (8 TB) for storage on CentOS. I have created the 4 partitions on my 4 physical hard disks and have created the raid device /dev/md2 out of those 4 partitions.

Now it's time to put a filesystem on it. I am wondering whether I should do just:

mkfs.ext4 /dev/md2

which totally works (I tried it), or I should first create a single partition on that device (with fdisk, gdisk or parted?), that takes up all the space, and then do the mkfs stuff on that partition.

Googling the Web, I found another person asking the very same question here: http://ubuntuforums.org/showthread.php?t=2174067 but, although a responder said:

Partitions go on the array members.
Filesystem goes on the array.

they didn't give him a definitive, non-ambiguous answer or an explanation when one approach should be preferred over the other.

Pavel Tankov
  • 367
  • 3
  • 15

2 Answers2

5

The only method I have ever used was to partition the physical disks, then combine the partitions into a RAID, and put file system (or swap) directly on the md device.

As far as I know partitioning the md device is a feature, which was added later, but never became widely used.

I would recommend against attempting to create a RAID directly on a physical device without partitioning that device first. Lot of software expects hard disks to be partitioned, and the partition table contains a type field for each partition, which will tell software reading the partition table that this partition is part of a RAID.

Whether to use a partition table on top of the md device may be a matter of taste. It might be entirely due to historical reasons, that it is usually not done. The tiny amount of additional disk space consumed by the extra layer of partition table is certainly not a reason to avoid it, and just as at the lower layer, the partition table on top of the md device would indicate what to expect to find inside.

However the flexibility gained by being able to partition the md device, I consider to be negligible, because you can create multiple partitions on the physical device instead and create multiple md devices.

Multiple md devices which are not partitioned give you more flexibility than one md device which is partitioned. You have additional freedom when time comes to replace one of the underlying media, and you can configure the different md devices with different RAID levels.

For example if you have six disks, you could make /boot be RAID-1 with a copy on every disk. / could be a RAID-1 with copies on three disks and a RAID-1 with copies on the other three disks could be used for swap. Remaining space could be a large RAID-6 across all six disks.

kasperd
  • 29,894
  • 16
  • 72
  • 122
  • It seems best-practice is to set the Partition-Code-Type of the partition holding the md device to "FD00 Linux RAID", right?. And if you don't make partition(s) on the md device, you will not have any partition with the "right" Partition-Type-Code like "8300 Linux filesystem" and "8200 Linux swap" and that doesn't seem right to me, but neither is partitioning the MD driver....what's the solution? I must be missing something? – MrCalvin Jun 21 '18 at 01:14
  • @MrCalvin The underlying partitions definitely should have the correct type indicating that it's a Linux RAID. I don't recall the exact value to use but FD00 cannot be correct as it's an 8-bit value on MBR and a 128-bit value on GPT. In normal operation the way a Linux system knows what's on each RAID is from entries in `/etc/fstab`. Thus it isn't strictly needed to have a partition table to distinguish between file system and swap. Moreover the 8-bit values used by MBR aren't sufficient to distinguish between different file system types anyway. – kasperd Jun 21 '18 at 15:10
  • It's 2018.....lets keep it at GPT (unless MBR mentioned specific) :-P Yes, the actual and true UUID of the RAID partition type code is "A19D880F-05FC-4D3B-A006-743F0F84911E", but to keep it simple I used the unofficial short-codes defined by gdisk and perhaps also fdisk. But I'm still not sure what the "perfect" setup would be. Are you saying the partition code type UUID really doesn't matter for Linux as it will use the fstab? So it's just usable for other OSs that might access the disk at some time. – MrCalvin Jun 21 '18 at 19:10
0

As others have said, normal practice with md-RAID metadevices is to put the file system directly on it. The metadevice should already only be made up of partitions (putting RAID devices on top of whole-disc devices is a practice that I find involves shooting yourself in the foot more often than not), so you're not violating the advice you received.

But an option that's not already been mentioned is that if you want to partition the RAID device, you can assign it to LVM then use LVM to divide up the device into partitions. In addition, you get the handy advantages of LVM when it comes to resizing partitions, allocating extra space, and so on.

Here's an example from one of my boxes:

[root@risby ~]# pvs
  PV         VG         Fmt  Attr PSize PFree
  /dev/md127 VolGroup01 lvm2 a--  1.82t    0 
[root@risby ~]# vgs
  VG         #PV #LV #SN Attr   VSize VFree
  VolGroup01   1   4   0 wz--n- 1.82t    0 
[root@risby ~]# lvs
  LV       VG         Attr       LSize  Pool Origin Data%  Move Log Cpy%Sync Convert
  LogVol00 VolGroup01 -wi-ao----  5.00g                                             
  LogVol01 VolGroup01 -wi-ao---- 50.00g                                             
  LogVol02 VolGroup01 -wi-ao----  1.75t                                             
  LogVol03 VolGroup01 -wi-a----- 11.02g

As you can see, there's a single RAID metadevice, on which four separate partitions exist. They correspond to swap, /, a big data partition, and a little extra space in case it's needed anywhere.

MadHatter
  • 78,442
  • 20
  • 178
  • 229