As far as I know, zfs uses checksums to protect against data loss caused by bitrot.
But what happens if a bitrot affects the data of the checksum. Then, does zfs think the data is corrupt or think the checksum is corrupt?
Thx 4 any answer,
tbol
Asked
Active
Viewed 991 times
0
-
Do you have a specific problem? – ewwhite Jul 29 '16 at 12:51
-
not a specific problem, just some thoughts about data ingetrity – tbol Jul 29 '16 at 12:53
-
Welcome and thank you for posting. ServerFault requires a clear and useful question which is [well written](http://meta.serverfault.com/a/3609/37681) , [on-topic](http://serverfault.com/help/on-topic) and contains sufficient details about your actual business problem to provide you with a good solution. - Please don't ask overly broad, conceptual and/or hypothetical (background) questions . – HBruijn Jul 29 '16 at 20:59
2 Answers
2
ZFS provides fault isolation between data and checksum by storing the checksum of each block in its parent block pointer -- not in the block itself. Every block in the tree contains the checksums for all its children, so the entire pool is self-validating.
Edit: because you asked about the parent:
Observation 1: ZFS detects all [on disk] corruptions due to the use of checksums. In our fault injection experiments on all metadata and data, we found that bad data was never returned to the user because ZFS was able to detect all corruptions due to the use of checksums in block pointers. The parental checksums are used in ZFS to verify the integrity of all the on-disk blocks accessed. The only exception are uberblocks, which do not have parent block pointers. Corruptions to the uberblock are detected by the use of checksums inside the uberblock itself.
End-to-end Data Integrity for File Systems: A ZFS Case Study
You a can test this yourself. Insert a random block in the middle of a ZFS device and see if it maintains integrity.
Note that in the next section of that paper, they show that in memory corruptions went undetected.
John Mahowald
- 30,009
- 1
- 17
- 32
-
and what happens if the block-pointer of the data in the parent is corrupted by bitrot? – tbol Jul 29 '16 at 12:50
-
Parents have a checksum in parent block pointers, up to the uberblocks which have them embedded. See the paper I linked. – John Mahowald Jul 30 '16 at 12:18
-
1
I've found the right explanation:
A ZFS storage pool is really just a tree of blocks. ZFS provides fault isolation between data and checksum by storing the checksum of each block in its parent block pointer -- not in the block itself. Every block in the tree contains the checksums for all its children, so the entire pool is self-validating. [The uberblock (the root of the tree) is a special case because it has no parent; more on how we handle that in another post.]
When the data and checksum disagree, ZFS knows that the checksum can be trusted because the checksum itself is part of some other block that's one level higher in the tree, and that block has already been validated.
There's just a single point of failure: if the root node in the tree is sorrupted, but there should be a solution for this
Read @ https://blogs.oracle.com/bonwick/entry/zfs_end_to_end_data
-
This answer is correct and the solution for the uberblock (the parent of all other blocks on the top) is simply to have multiple copies of it because they are pretty small anyway (I think it was about 5). – user121391 Jul 29 '16 at 13:01
-
@user121391 And backups. No solution is perfect, so we hedge our bets and make sure that even if we lose the bet, we still have enough in the bank to make it home. – user Aug 17 '16 at 14:25