Linux (exfat-fuse) is unable to read from GPT 4TB external exFAT disk

1

I have a 4tb external WD RED. It was formatted as an exFAT disk on a Windows 10 machine. I have no problem reading the files from both the original Windows 10 machine, and a 2017 macbook pro (High Sierra).

On either linux machine I use (Ubuntu 16.10 x64 + Slackware 14.2), the system never detects the disk as readable or mountable. Confusingly, if I erase and format the disk on my linux machine as exFAT, then neither the windows 10 machine, nor the macbook can read it, although both linux devices can. This leads me to suspect the exfat-fuse implementation is incomplete... although admittedly I don't know enough to be sure.

[As a side note, I also erased + formatted as an exFAT from the macbook pro, and the Windows 10 machine could still read/write. No linux machine could.]

Here's what I have investigated so far:

sudo lsblk

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
...
sdj      8:144  0   3.7T  0 disk 
├─sdj1   8:145  0   6.4G  0 part 
└─sdj2   8:146  0   7.3G  0 part

Note the garbage filesystem sizes lsblk detects. The volume is formatted with only 1 exFAT partition, spanning all 4TB.

parted -l /dev/sdj

Error: /dev/sdj: unrecognised disk label
Model: WDC WD40 EZRZ-00GXCB0 (scsi)                                       
Disk /dev/sdj: 4001GB
Sector size (logical/physical): 512B/4096B
Partition Table: unknown
Disk Flags:

parted is at least honest enough to say it has no idea what we are dealing with. It did detect the model right, it is a WD red 4tb.

fdisk -l /dev/sdj

Disk /dev/sdj: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x3258a7ab

Device     Boot Start       End   Sectors   Size Id Type
/dev/sdj1           1     76805     76805  37.5M ee GPT
/dev/sdj2       77056 976754431 976677376 465.7G  7 HPFS/NTFS/exFAT

The fdisk output is interesting because it is almost right. It maps one partition as the GPT metadata. The other, it correctly finds the id (0x07 == microsoft data ), but the size is wrong. The partition is 4tb, not 465.7GB.

I verified the partition really spans 4tb by writing 4tb of test content from windows, and successfully read all the content back again from the macbook pro.

Attempting to mount any of the detected devices does not work:

sudo mount.exfat-fuse -d /dev/sdj1 /mnt/foobar

FUSE exfat 1.2.4
ERROR: exFAT file system is not found.

sudo mount.exfat-fuse -d /dev/sdj2 /mnt/foobar

FUSE exfat 1.2.4
ERROR: exFAT file system is not found.

Even in debug mode, exfat-fuse prints no useful output. Looking at the source, it fails very early, just after reading the superblock.

I took a peek at the first 1MB of data on the device with a hexdump, and it looks sane, with no obvious garbage. (Pastebin: https://pastebin.com/BVLWW3kf )

Any additional debugging steps I should try? Any ideas on what is going on here?

Edit:

Using gdisk, it appears the drive has a hybrid MBR+GPT. However, gdisk detects the GPT as invalid:

sudo gdisk /dev/sdj
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
  MBR: hybrid
  BSD: not present
  APM: not present
  GPT: not present

***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format
in memory. THIS OPERATION IS POTENTIALLY DESTRUCTIVE! Exit by
typing 'q' if you don't want to convert your MBR partitions
to GPT format!
***************************************************************

Afforess

Posted 2017-12-10T18:58:13.843

Reputation: 85

2I suspect, the disk contains a GPT partition table AND an MBR partition table (not the absurd sizes of sdj1 and sdj2). On Windows and Mac, the GPT will be used, on Linux MBR. Have you tried gptsync or friends to rule this out? – Eugen Rieck – 2017-12-10T19:04:02.477

gptsync fails to read the GPT table: https://pastebin.com/r2GswgBL

You are right though, it did detect a MBR table.

– Afforess – 2017-12-10T19:08:00.770

That could explain it! Windows & Mac will use a GPT partition table, where only the secondary (backup) Version exists without complaint, Linux will not. You could try wiping start and end (partition tables MBR, GPT primary, GPT backup), then partitioning under Linux with gdisk - this should work, I used it to exchange Video files between Apple (editing) and Linux (broadcasting) – Eugen Rieck – 2017-12-10T19:13:29.447

I'll try that - but I am curious why linux thinks the GPT is invalid. It's clearly valid enough that windows and macOS can read it. – Afforess – 2017-12-10T19:16:54.290

A GPT partition table exists twice - Windows and MacOS will use it, when one copy is OK. Linux is more cautious – Eugen Rieck – 2017-12-10T19:32:27.863

I'm pretty sure the linux kernel will use the backup GPT just fine. parted and gdisk will certainly let you know the primary is corrupt and offer to repair it using the backup. – psusi – 2017-12-10T19:41:10.097

The tools will - but the kernel won't auto-detect the partitions – Eugen Rieck – 2017-12-10T21:48:55.253

I'm actually not sure there is anything wrong with the GPT, it seems that the core issue is the 512e vs 4k, and that linux thinks the drive is 512e. – Afforess – 2017-12-10T23:18:54.550

Answers

3

The problem appears to be that the disk was partitioned as if it were a 4k native sector size disk. The GPT starts at 4096 bytes. Since it is a 512e drive, it is supposed to start at 512 bytes into the disk. I would guess that for whatever reason, Windows is incorrectly detecting the drive as 4kn.

psusi

Posted 2017-12-10T18:58:13.843

Reputation: 7 195

It is a 4k native sector size disk. WD adopted the 4ke spec first, (and the drive is brand new). It is linux that is not detecting it right. – Afforess – 2017-12-10T23:05:36.917

Actually, I am wondering if the USB enclosure is at fault for this. The gdisk output makes sense if you multiply it by 8 (4096 / 512 -> 8), which suggests the linux tools are reading it as 512e when it is 4k. – Afforess – 2017-12-10T23:17:55.123

(I upvoted this answer, but I am not marking it right, because strictly speaking, it's not. It's close though.) – Afforess – 2017-12-11T06:27:35.377

@Afforess, when you access it through Windows are you using the same enclosure? If not, then that would explain where the 512e comes from: it's the enclosure. – psusi – 2017-12-12T22:30:48.307

I was using the same enclosure. – Afforess – 2017-12-13T16:52:57.500

@Afforess, I wonder if Windows sees it as 4k or 512. I wonder if it has some workaround where if it sees a GPT that claims it is 4k then it goes with that instead of what the drive says. – psusi – 2017-12-15T00:45:25.310

Any easy way to tell which way windows is seeing the disk? – Afforess – 2017-12-15T00:51:12.937

Asked: 2017-12-10T18:58:13.843

Viewed: 1 094 times

Active: 2017-12-10T19:48:09.117