Areca RAID volume and LVM alignment

Question

I have the following hardware:

1x - Areca ARC-1680IX-24 PCIe x8 SAS RAID Card
24x - Seagate Barracuda LP 2 TB 5900RPM SATA 3 GB/s 32 MB Cache 3.5-Inch Internal Hard Drive ST32000542AS

I plan on setting up the following volumes on the RAID controller.

Volume Info for RAIDSET #1, 4x 2tb drives:
    V#1    RAID10,10gb    - 4k stripe size
    V#2    RAID10,3tb    - 4k stripe size
    V#3    RAID0,2tb    - 128k stripe size

Volume Info for RAIDSET #2, 6x 2tb drives:
    V#4    RAID0,300gb    - 4k stripe size
    V#5    RAID5,7.8tb    - 64k stripe size

Volume Info for RAIDSET #3, 6x 2tb drives:
Volume Info for RAIDSET #4, 6x 2tb drives:
    V#6    RAID50,20tb or RAID60,16tb    - 128k stripe size

I have concerns with alignment and stripe size when I setup these volumes in the RAID controller. I know with a regular HDD it is important to get the partition properly aligned for good performance, I am assuming I need to worry about this also when setting up the RAID volumes? Or will the RAID controller make sure the alignment is optimal?

My next concern is how this alignment affects the LVM volumes. Does the RAID controller notify the LVM driver how to properly align within the RAID volume?

I have a choice of many stripe sizes, from 4k all the way up to 128k. Will the selection of the stripe size affect my alignment and/or allocation sizes of the RAID volumes, e.g. if I choose 128k strip sizes on a RAID set with 4 disks that will be setup in RAID 0, should I make my RAID volume size a multiple of 512k? Also, what is an optimal stripe size for a linux root and swap partition or just general use?

Lastly, the RAID controller exposes a 3 channel SCSI controller, is there any performance benefit of splitting up my load across the 3 channels or is that just so it can support more disks?

Answer 1

There are up to 3 levels of alignment you need to keep in mind - 1). volume manager, 2). volume partitioning, 3). file system. If you are not using LVM then 1 is irrelevant. If you are not partitioning you volumes with fdisk, then 2 is irrelevant as well. The most important alignment for performance is 3. With proper alignmet you may see up to 15% boost in performance.

For cases 1 and 2 a good general rule would be aligning to a megabyte boundaries.

1). LVM usually does a good job by a) placing it's metadata at the end of the volume and b) giving you an option of specifying size of metadata (for example "pvcreate -M2 --metadatasize 2048K --metadatacopies 2 ")

2). If you need to partition any of these volumes with fdisk then again, try to stick to MB boundaries. Modern Linux fdisk versions have this option as well as recent version of gparted.

3). Aligning of file system is most important of all. I have experience with aligning xfs and ext3 (ext4 should be similar to ext3) and you will need to do some math here and then specify right parameters when creating the file system. Look at the documentation for specific parameters, namely something called "stripe width". Be careful though with interpretation - depending on fs type it is either expressed in 512B blocks or in bytes, so you will need to do calculations accordingly. This interpretations is also depends on number of drives in the RAID array and RAID level. You may also find some useful info in this thread.

Also, you can specify parameters when mounting a file system that may improve performance even further. Here are parameters I use with my 18TB xfs file system "noatime,attr2,nobarrier,logbufs=8,logbsize=256k". But be careful, these are not universal rules and if used incorrectly may compromise reliability of your system (especially "nobarrier").

Another thing to keep in mind that if you planning for future expansion of any of these RAID arrays you should take it into account when you create file systems since it will imminently affect your perfect alignment ;-)

I hope this points you in a right direction. Have fun :-)