Linux MD software raid stripe cache size

Question

From what I can piece together the /sys/block/md0/md/stripe_cache_size is for raid5 and 6 set at a very low default value to avoid starving machines for memory.

By increasing this value you can get a significant speed boost from your raid array. The default value is 256 and as far as I have understood this "translates" to 256 * 4096 * disks. So for a 8 disk raid 6 this would mean that 8 megabytes of memory is used.

I think this would translate to about one megabyte per disk. Now, today most disks have at least 32mb of cache and sometimes even more. To be able to use all the cache of the disks I would speculate that a value of 8192 or a little less would be good and give no higher worst case scenario for corruption than the default value. Also the raid array now consumes 268MB of memory, but that is a tradeoff I am willing to make unless corruption or data loss risk is significantly increased by alternating this value.

1. What exactly is stripe_cache_size?
2. How does it work?
3. Does increasing the value increase the risk of corruption / data loss?
4. Is the memory usage the only reason it is set low by default?

Answer 1

1. As I understand this is cache for completed stripe of data, ready to be written to disk including parity data.
2. Before stripe is written to disk it must be formed somewhere.
3. That depends. I usually increase stripe_cache_size, BUT decrease dirty bites cache, so all data passed directly to MD driver. This way I had no data corruption in many years

here how I increase /etc/udev/rules.d/83-md-stripe-cache.rules KERNEL=="md*", ACTION=="add", TEST=="md/stripe_cache_size", ATTR{md/stripe_cache_size}="32768"

this is decreased value vm.dirty_background_bytes = 2097152 4. It is set low by default for safety purpose, not for memory saving.