1
We have a SLES12 SP1 System running as VM in a MS HyperV Environment.
After a reboot the system won't boot anymore because the vmlinux isn't there anymore. Grub says "you must load the linux kernel first" or so (I weren't able to copy the exact error message). After I digged deeper and tried to load an older kernel I realized that there are still transactions hanging in the btrfs system, at least there corrupting checksums.
So I downloaded the latest OpenSUSE Rescue CD and tried to mount the devices. I can mount the home partition, but the partition with the /boot etc. won't mount:
linux@localhost:~> sudo mount -t btrfs -o ro,usebackuproot /dev/sda2 /mnt
mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda2, missing codepage or helper program, or other error.
So I followed these steps: https://lists.opensuse.org/opensuse/2017-02/msg00930.html
Mounting doesn't work at all, I tried different options like disable cache etc., so scrubbing didn't work either.
btrfs check says:
linux@localhost:~> sudo btrfs check /dev/sda2
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found E4E3BDB6 wanted 00000000
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
bytenr mismatch, want=75659116544, have=5432601639048425051
ERROR: cannot open file system
Restoring doesn't work either:
linux@localhost:~> sudo btrfs restore /dev/sda2 /run/media/linux/c03ad26a-a896-468d-bf9d-9953f87b2661/Administrator/Backup_sda2/
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found E4E3BDB6 wanted 00000000
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
bytenr mismatch, want=75659116544, have=5432601639048425051
Could not open root, trying backup super
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found E4E3BDB6 wanted 00000000
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
bytenr mismatch, want=75659116544, have=5432601639048425051
Could not open root, trying backup super
ERROR: superblock bytenr 274877906944 is larger than device size 42952818688
Could not open root, trying backup super
So I backuped the image using dd.
super-recover:
linux@localhost:~> sudo btrfs rescue super-recover /dev/sda2
All supers are valid, no need to recover
btrfs rescue zero-log:
linux@localhost:~> sudo btrfs rescue zero-log /dev/sda2
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found E4E3BDB6 wanted 00000000
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
bytenr mismatch, want=75659116544, have=5432601639048425051
ERROR: could not open ctree
The chunk-recover didn't work either but I tried to mount afterwards anyway:
linux@localhost:~> sudo btrfs rescue chunk-recover /dev/sda2
Scanning: DONE in dev0
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found E4E3BDB6 wanted 00000000
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
bytenr mismatch, want=75659116544, have=5432601639048425051
open with broken chunk error
Chunk tree recovery failed
linux@localhost:~> sudo mount -t btrfs -o ro,usebackuproot /dev/sda2 /mnt
mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda2, missing codepage or helper program, or other error.
check repair:
linux@localhost:~> sudo btrfs check --repair /dev/sda2
enabling repair mode
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found E4E3BDB6 wanted 00000000
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
bytenr mismatch, want=75659116544, have=5432601639048425051
ERROR: cannot open file system
I get almoust the same error with init-extent-tree:
linux@localhost:~> sudo btrfs check --repair --init-extent-tree /dev/sda2
enabling repair mode
Checking filesystem on /dev/sda2
UUID: b70dc1a4-4c19-446c-98c9-eeee88c4fd47
Creating a new extent tree
checksum verify failed on 75638358016 found 9B64BB37 wanted 6156F702
checksum verify failed on 75638358016 found 9B64BB37 wanted 6156F702
checksum verify failed on 75638358016 found E4E3BDB6 wanted 00000000
checksum verify failed on 75638358016 found 9B64BB37 wanted 6156F702
bytenr mismatch, want=75638358016, have=267958633426704061
Error reading tree block
error pinning down used bytes
ERROR: attempt to start transaction over already running one
extent buffer leak: start 89771360256 len 16384
also init-csum-tree:
linux@localhost:~> sudo btrfs check --repair --init-csum-tree /dev/sda2
enabling repair mode
Creating a new CRC tree
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
checksum verify failed on 75659116544 found E4E3BDB6 wanted 00000000
checksum verify failed on 75659116544 found 61415E80 wanted 96E73A64
bytenr mismatch, want=75659116544, have=5432601639048425051
ERROR: cannot open file system
I didn't find any more information regarding this error so I'm trying to get some help on stack-exchange, knowing that there are already some Q&A on this topic. We only have a backup of the host server, so we really want to get this running. There must a better way to recover the btrfs partition :(
Thanks in advance!
Lukas Florea
Posted 2018-05-30T12:54:59.780
Reputation: 11