10
6
My Seagate Momentus XT hybrid hard drive is corrupting files on Linux. I would appreciate help from anyone, but I'd particularly like to know if other Momentus XT users are able to reproduce this problem; I have provided step-by-step instructions for reproducing this issue on the Seagate Community Forums.
So far, four users have reproduced this problem on the following laptops and OS/distributions:
- Five laptops: Lenovo Thinkpad T60, T61, T510, MSI MS-1656-ID1, and MacBook Pro (15" late 2009).
- Four OS/distributions: Ubuntu 11.04, Fedora 15, openSUSE, and Mac OS X.
The instructions for reproducing the problem are simple. Here is a brief verbal description:
- Create a large test file, save it to another storage device (not the Momentus XT), and compute the SHA-1 checksum.
- Write the test file to the Momentus XT.
- Read the test file from the Momentus XT, calculate the SHA-1, and compare this checksum with the checksum of the original. We should have a match. We have probably reproduced the problem if they don't match. (Only 'probably', because it is possible for other issues to cause a mismatch. See the Seagate thread about identifying this specific problem by comparing the files with
cmp -l.) - Repeat from step (2).
The Seagate thread has more details. Here are some notes from my testing (I have been able to reproduce this problem on three consecutive Momentus XT drives; I RMA'd twice and am now on the third one):
- What seems to be happening is that the Momentus XT sometimes neglects to write data to the drive, so that when I read from the drive, I get what was originally on the sector, and not the correct data. This occurs in blocks of different sizes; typical sizes are 1 MiB and 512 KiB.
- Problem occurs on ext2, ext4, Btrfs, NTFS, and FAT32. Strangely, I was not able to reproduce this problem on ext3.
- Writing with the
oflag=directoutput flag inddavoids this problem. Rapidly commiting data to disk withwhile true; do sync; sleep 0.01; donealso prevents the problem. - I have only been able to reproduce this problem through a SATA and an eSATA interface. A USB connection seems to prevent the problem. (Not sure if this is due to transfer speed.)
- Problems occur more often with large files (>2 GB). I was not able to produce problems with files smaller than about 85 MB.
- I was not able to reproduce the problem on Windows XP with NTFS.
- Gazoi at the Seagate forums was unable to reproduce the problem on FreeBSD 8.2 with UFS2.
- The Momentus XT passes both the extended SMART test and
badblocks -wwith no issues. - My laptop (MS-1656-ID1) has successfully passed through 24 hours each of Memtest86+, Memtest86, memtester, and MPrime.
- I have tested two other storage devices (a Seagate Momentus 7200.4 and an Intel 320 series SSD) with the same procedure, and they both pass with no issues.
If you have a Momentus XT, please try reproducing this problem and let me know what happens.
What else can I do to diagnose the problem?
Vincent Yu
Posted 2011-07-19T17:05:52.953
Reputation: 473
1Are you rebooting between the write and the read? Are you flushing the cache to make sure that it is actually read back from the disk instead of cache? If not, that may be why you can't reproduce it with smaller files since they are more likely to still be in the cache – psusi – 2011-07-19T19:29:48.717
I am bypassing the page cache by reading with the
iflag=directinput flag with dd. When I am not using dd, I flush the cache withsudo sh -c "sync && echo 1 > /proc/sys/vm/drop_caches"– None – 2011-07-19T20:22:38.287I just realized that you might be talking about the disk buffer that is physically on the Momentus XT, instead of the Linux page cache. You have a point in this case - I am mostly not taking any precautions against reading the disk buffer, and I am also immediately reading the file after writing it. Perhaps I should commit writes to multiple files, and then read them on a FIFO basis. With that said, the Momentus XT disk cache is only 32 MB, and I have also done some tests in the past where I have wrote >10 GB of small files (~64 MB), and read them afterwards without any finding corruption. – None – 2011-07-19T20:40:33.413
Maybe there's a bug in the drive's firmware (integer overflow?) that causes it to loose track of unwritten blocks under high speed write. Or there are some bad blocks on the flash that the firmware failed to detect. NTFS verifies writes (and ntfs-3g is very slow), so the problem is unlikely to appear. If you can't find a fix, you may want to use ZFS and enable checksum. – billc.cn – 2011-07-21T21:08:56.623
@billc.cn, I have reproduced the problem with NTFS. Btrfs and ZFS with data checksums will conspicuously fail to read corrupted files, but the corrupted data will be unrecoverable. More redundancy (e.g., FS-level mirroring or RAID mirroring) will be needed to recover these files. In any case, I am of course not using the Momentus XT until I figure out what's happening. – Vincent Yu – 2011-07-21T21:24:08.737
I've got a Momentus XT, but its running Windows 7. I'll test it on Win7. I could boot into Linux using a USB device, but I don't know a whole lot about Linux commandline, so that'd probably end in disaster, heh – William Lawn Stewart – 2011-07-22T00:27:30.133
@William: Great! I would still recommend booting Linux from a live USB, simply because all my instructions on the Seagate thread are for Linux. There are only three user-specified parameters required; everything else is just copy and paste. However, a Windows 7 test could be interesting.
– Vincent Yu – 2011-07-22T00:52:46.173My Momentus XT (SD22, NTFS) failed the test with a 4GB file running from an Ubuntu 11.04 LiveCD on an HP dv7-6014tx. I'll try it on Windows 7 once I've figured out how to make sure its not caching anything. – William Lawn Stewart – 2011-07-22T02:37:20.397
I just compared the two files on Windows 7, and they are exactly the same. These are the same two files that Ubuntu came up saying they were different. – William Lawn Stewart – 2011-07-22T02:54:23.963
@William: Thank you very much for the information. Strange. How did you compare the files on Windows? – Vincent Yu – 2011-07-22T03:09:19.383
@Ironclaw I wrote a C# program that compared each file byte by byte. Also double-checked by running WinDiff. According to both the files are identical. Your data isn't at risk, it seems. Maybe Linux makes some assumption about buffering which stands for other drives, but not this one? – William Lawn Stewart – 2011-07-22T03:18:23.287
@William: Argh. I've just discovered there seems an occasional problem with my instructions when used with NTFS partitions - dd refuses to read files, and I made the stupid decision of redirecting standard error to /dev/null, meaning that messages stating this are suppressed. Sorry about this. It's because of the
iflag=directinput flag I'm using withdd- I need to figure out how to fix this. I'll comment here again when I've got it fixed. – Vincent Yu – 2011-07-22T03:31:23.880@William: I have updated the instructions on the Seagate post with different commands for NTFS. I have tested the new commands with an NTFS partition and I was able to reproduce file corruption. Would you mind going through the test on Linux again? The previous problem was due to direct I/O not being supported on the open-source NTFS-3G driver. – Vincent Yu – 2011-07-22T04:18:54.687
@Ironclaw: Sure =) The Seagate forums won't let me register, so I'll have to post the results here again. – William Lawn Stewart – 2011-07-22T04:44:59.390
Turns out that flushing buffers while using a Live CD install is a great way to make it lock up, I'll go find a spare drive to install to instead, heh – William Lawn Stewart – 2011-07-22T06:17:48.233
@William: Whoops... Looks like I just keep making things worse. I'll add this to the Seagate post. Thanks for taking the time to do this. – Vincent Yu – 2011-07-22T06:23:48.183
@William: An installation on spare drive would work. Alternatively, if you have space, you can resize your NTFS partition and create an ext4 partition on the Momentus XT. Then you can write the test files to that instead, without having to flush the page cache. – Vincent Yu – 2011-07-22T06:33:14.513
Installed Ubuntu on my USB drive, but it had some error message about a radeon-switcheroo and refused to boot, so I tried the Live CD again. Didn't lock up this time (possibly because I didn't try listening to music at the same time? =P). The results are weird - It copied the 4GB file five times, and they were all fine. No errors at all. I guess I could make an ext4 partition for testing and see if that changes anything? Or perhaps 5 times wasn't enough. – William Lawn Stewart – 2011-07-22T08:24:12.723
@William: Well, it's good that you're not having any problems. I would suggest looping through the test more than 5 times - the second Momentus XT that I had would normally fail only after about 15 copies. Looping the test 20-30 times is probably sufficient. Are you saving the test file on an external hard drive? – Vincent Yu – 2011-07-22T08:36:15.780
I've been using my laptop's secondary internal hard drive for the input directory. – William Lawn Stewart – 2011-07-22T08:41:34.143
I'll be able to run an extended test tomorrow, today my laptop is doing some number crunching. It might be useful if someone with a Mac could run the tests, perhaps this explains why Mac folks have freezing issues with the XT. – William Lawn Stewart – 2011-07-23T22:58:07.447
@William: Okay, thanks! I agree about testing on a Mac, especially since Mac OS X is Unix-based. If you are able to reproduce the problem, I think I'll submit something to Slashdot and Digg, so that this can be tested on more hardware and OSes. Otherwise, if you can't reproduce the problem, I'll wait for more information from other users - I don't want to pollute news sites with a problem that affects only a small fraction of users. – Vincent Yu – 2011-07-23T23:19:03.427