mdadm RAID Fast Setup with Empty Drives?

I am recreating some RAID5 disks as RAID6 with mdadm. There is no data on the drives that I care about.

Setting up RAID takes a while to setup the shape - I accept that when there there is data that needs to be striped and parity calculated, however with these drives, they are empty - or at least I want them to be considered empty.

So is there a way to skip the parity calculation and tell mdadm to just set up the superblocks and be done, or otherwise, what exactly is it spending all this time on when there is no data to move around?

md3 : active raid6 sdf3[5] sde3[4] sdd3[3] sdc7[2] sdb3[1] sda3[0]
      1953114112 blocks super 1.2 level 6, 512k chunk, algorithm 2 [6/6] [UUUUUU]
      [>....................]  resync =  1.3% (6790144/488278528) finish=409.3min speed=19604K/sec

Note that I am not talking about --assume-clean where you are rebuilding an array from a pre-existing set of disks that contain a RAID array that you know is correct. I am talking about an array that should be considered empty, not considered correctly striped.

So lets say for the sake of this question that the devices have been pre-populated with zeros.

Paul

Posted 2012-06-18T22:24:57.293

Reputation: 52 173

Answers

You can use --assume-clean but unless you are using raid5 ( not raid6 ) and the disks actually are full of zeros, the first time it runs a parity check, it will come up with errors that will need corrected, so you should not do this. You don't need to wait for the resync to finish before you can start using the array; it will chug along in the background until it is done.

psusi

Posted 2012-06-18T22:24:57.293

Reputation: 7 195

Thanks psusi - this doesn't address the question. – Paul – 2012-06-18T23:13:29.097

2@Paul, umm.. yes, it does. There is no such thing as "empty" there is only in sync, or not in sync. – psusi – 2012-06-19T02:17:07.940

It doesn't address the question I am asking. The xor of a block of zeros is zero. So if I could tell mdadm that the space can be considered empty, it could create parity very quickly. The parity would only need to be calculated properly when a specific block is written to, at which point it doesn't matter that it was "wrong" previously. I could even actually zero the disk. – Paul – 2012-06-19T03:09:35.410

2@Paul, again, there is no such thing as "empty". Generally yes, it won't matter that the parity on stripes that have never been written is wrong, which is why you can get away with --assume-clean, even when the disks are not full of zeroes, but the wrong parity will be detected and corrected when mdadm does a parity check. – psusi – 2012-06-19T03:18:17.313

Is this an issue of semantics I am missing? When I say "empty" I mean without data, and more specifically, with zeros. So parity of zeros is zero. If I can assert to mdadm that something contains zeros and it actually does contain zeros, there is no wrong parity to correct during a parity check. – Paul – 2012-06-19T03:27:40.170

2@Paul, the meaning of "empty" is not well defined. Most people use it to mean they have not put any files or a filesystem on the device, and do not know or care what it currently contains. If you know the disks are all full of zeros, then --assume-clean is what you want. It will take your word for it that the disks are all zeroed and not recompute the parity, and as you mention, a parity of zero will be correct for data disks of all zeros, at least for raid5 ( not for raid6 ). – psusi – 2012-06-19T03:35:36.077

If you update your answer with this clarification I'll accept it. Btw, I reckon even with raid6 it wouldn't take much to do the same - while the parity wouldn't be zeros, it would be the same for each block. So mdadm could have a feature where you say "I promise there is nothing but zeros" and mdadm would do reed-solomon codes based on that promise. I imagine there isn't much demand for this though. – Paul – 2012-06-19T04:28:15.983

There are a good reason to let the RAID write all checksums, and that is to early check for bad sectors on the disks. I do recommend that you look into blogs written by the engineers and computer scientist who wrote ZFS. There are a lot of usefull info about large disk systems and unique problems which they tried to solve. And it was a lot fun reading too. :-) – Anders – 2012-06-19T11:34:01.747

@Paul, edited... – psusi – 2012-06-19T13:13:10.340

You can't do this with a software or hardware RAID. All checksums need to be written on the disks, which takes time. You can do it later, but then the parts of the disk that isn't written to, will have to do that before you can use them.

This is basicly because the RAID system and file systems doesn't know a thing about each other. ZFS has a sollution for this, but there the RAID parts are deep integrated with the file system. So the RAID subsystem actually knows what parts of the disks are used to store data on and which can be used later and then write the check sums to them.

You can add throughput speed to the software RAID or you start using the RAID before all checksums are written, and let the software RAID handle this for you later. Witch is what @psusi wrote.

Anders

Posted 2012-06-18T22:24:57.293

Reputation: 145

I understand how RAID works, I am trying to understand what takes place with checksums on an empty disk. Checksums need to be calculated every time something is written to disk, so calculating them before any data is present doesn't make much sense - "doing it later" is sensible when there is nothing there. There isn't a file system to consider in this scenario. – Paul – 2012-06-19T01:10:41.543

6There's always data present. It might not be important or meaningful, but every sector always has a value. (Zero is a value too!) The RAID implementation doesn't know which sectors contain important data, so it has to treat them all as important and calculate their parity. – Wyzard – 2012-06-19T04:42:26.010