Linux XFS partition could not be mounted (can't read superblock) due to suddenly power shutdown, xfs_repair: read failed: Input/output error

Question

After suddenly shutdown, my home directory which is a XFS partition could not be mounted and I can login to system by my account. But I can login with root account (because root and home directories are separate). My OS is OpenSuse thumbleweed.

I tried following steps to recover my home partition:

1- Trying to mount the partition manualy:

# mount /dev/sda4 /home

and I see this error: can't read superblock.

2- Trying to repair by xfs_repair:

# xfs_repair /dev/sda4

ERROR: The filesystem has valuable metadata changes in a log which needs to
be replayed.  Mount the filesystem to replay the log, and unmount it before
re-running xfs_repair.  If you are unable to mount the filesystem, then use
the -L option to destroy the log and attempt a repair.
Note that destroying the log may cause corruption -- please attempt a mount
of the filesystem before doing this.

After that I run the command with -L option:

# xfs_repair -L /dev/sda4
Phase 1 - find and verify superblock...
Phase 2 - using internal log
        - zero log...
        - scan filesystem freespace and inode maps...
agi unlinked bucket 3 is 247427 in ag 0 (inode=247427)
agi unlinked bucket 51 is 115 in ag 0 (inode=115)
agi unlinked bucket 27 is 105361179 in ag 2 (inode=1179103003)
agi unlinked bucket 53 is 147510965 in ag 2 (inode=1221252789)
sb_icount 9534016, counted 9534272
sb_ifree 1508, counted 2201
sb_fdblocks 12934900, counted 13126723
        - found root inode chunk
Phase 3 - for each AG...
        - scan and clear agi unlinked lists...
        - process known inodes and perform inode discovery...
        - agno = 0
imap claims a free inode 407153 is in use, correcting imap and clearing inode
cleared inode 407153
data fork in ino 14927433 claims free block 1865944
correcting imap
        - agno = 1
data fork in regular inode 540301004 claims used block 75087592
correcting nextents for inode 540301004
bad data fork in inode 540301004
cleared inode 540301004
correcting imap
        - agno = 2
        - agno = 3
xfs_repair: read failed: Input/output error
cannot read inode 1900567232, disk block 1512628928, cnt 32
Aborted (core dumped)

I tried this command more than 3 times (without -L option) and each time I see the same error.

4- Using xfs_metadump to copy meta of partition and use it to recover data (base on the this article: https://www.suse.com/support/kb/doc/?id=000018858):

# xfs_metadump -aowg /dev/sda4 /tmp/dump_xfs_meta
Copied 2011648 of 9534272 inodes (1 of 4 AGs)              Unknown directory buffer type!
Unknown directory buffer type!
Copied 9243200 of 9534272 inodes (3 of 4 AGs)              xfs_metadump: read failed: Input/output error

xfs_metadump: cannot read inode block 3/36244312
Copied 9246656 of 9534272 inodes (3 of 4 AGs)              xfs_metadump: read failed: Input/output error

xfs_metadump: cannot read inode block 3/36416788
Copied 9251776 of 9534272 inodes (3 of 4 AGs)              xfs_metadump: error - read only 16384 of 32768 bytes

xfs_metadump: cannot read inode block 3/36437896
Copied 9279808 of 9534272 inodes (3 of 4 AGs)              xfs_metadump: error - read only 8192 of 32768 bytes

xfs_metadump: cannot read inode block 3/36723656
Copied 9292736 of 9534272 inodes (3 of 4 AGs)              xfs_metadump: error - read only 16384 of 32768 bytes

xfs_metadump: cannot read inode block 3/36818108
Copied 9332032 of 9534272 inodes (3 of 4 AGs)              xfs_metadump: read failed: Input/output error

xfs_metadump: cannot read inode block 3/37269284
Copying log   

However, I continued commands in the article as the following:

# xfs_mdrestore /tmp/dump_xfs_meta /tmp/workable_xfs_dump
# xfs_repair /tmp/workable_xfs_dump
# mount /tmp/workable_xfs_dump /mnt/test

In this state, I mount was successful and I did see the almost files and folders but none of the files could be opened!

Now, here is my questions:

1. Is there any way to repair the partition or restore the files? Or my files are went permanently :(
2. Is there any way to fix the problem in the step 2 (of my trying steps) such that the repair process could be continued even with losing some little data? For example by force the xfs_repair to ignore some blocks!

Thanks all already to your help.

Answer 1

xfs_metadump does not copy data by design. Per the manual page

The most common usage scenario for this tool is when xfs_repair(8) fails to repair a filesystem and a metadump image can be sent for analysis.

"Sent for analysis" meaning paying someone who understands XFS in detail to advise on recovery. Possibly your operating system support, but likely a company specializing in data recovery. Easier to send them a metadata only dump, much smaller and possibly with sensitive file names removed.

Time for a cost benefit analysis. How much will you spend for your data, in time and money? People have their preferred do it yourself tools, for example How to recover XFS file system with "superblock read failed". Starting with dd_rescue style imaging as much as possible to healthy storage and attempt to mount the copy. However, there is the risk that actions taken will damage the corrupt file system further. Zeroing the log like you did, for example. Price out professional software and recovery services, versus doing it yourself.

Answer 2

You may have an underlying physical disk error based on the output from xfs_repair -L. From your question I assume backups haven't been a priority for you to date. That's really the first thing you should change from now on, whether or not you manage to recover any data.

It's probably a good idea to clone the disk or partition before proceeding with any kind of rescue operation. Worst-case you can try dd to a device that's as large or larger than the volume you're trying to save. I'm sure other tools may be useful too. Back in the day Clonezilla was a favorite of mine, but I haven't used it in years, so I don't know how well it manages today's systems.