How to force a remap of sectors reported in S.M.A.R.T C5 (Current Pending Sector Count)?

38

27

The S.M.A.R.T C5 value of my Samsung HM640JJ Hard Drive (in an HP Pavilion dv6 laptop) is "yellow status = caution"

C5 was 10 yesterday, and it's 21 today.

C4 (Reallocation Event Count) = 0 and 05 (Reallocated Sectors Count) = 0

How can I force the firmware to reallocate them?

  • I removed the partitions, recreated them again and formatted the entire drive.
  • I ran chkdsk /r /f
  • I ran the BIOS disk check utility and other diagnose/repair tools

edteke

Posted 2012-01-30T20:59:25.527

Reputation: 523

I use boot disk spinrite6 on level 2 to do this....http://www.grc.com/intro.htm

– Moab – 2012-01-30T23:03:57.353

Answers

120

Short answer: Write something new to the sector (even zeros - which a long format does).

Long Answer

Hard drives today try to hide bad sectors from the host computer. The host computer simply asks the drive to return the contents of a particular sector number. Normally the drive reads the sector, returns it to the host machine, and everything is fine.

The hard drive knows if the value it read is valid or not, because the drive uses Error-correcting code (ECC) to validate that the contents it read are correct. If the drive detects that the contents of the sector are invalid, it will retry the read. The hope is that if it simply reads it again, it might get the correct sector contents. It will keep retrying until it gets a good value, or it's reached it's time limit (formally known as the Command completion time limit, or CCTL).

During these retries, the drive will appear dead; as it is no longer responding to commands.

Spare Sectors

Most modern drives contain a number of "spare" sectors (e.g. 1,024 spare sectors). If the drive recognizes a sector as bad, it will stop using it. Any requests to read or write to that damaged sector will transparently be redirected to a spare sector. This marking off of a bad sector, and reallocating its data to a spare sector, is called a Reallocation Event. And the total number of sectors that have been reallocated (and so how many of your spare sectors have been used up) is the Reallocated Sector Count.

In this example from one of my own hard drives, 64 sectors were found to be bad. That means that 64 of the drive's spare sectors have been called into use:

ID                             Current  Worst  Threshold  Raw  
=============================  =======  =====  =========  ===
(05) Reallocated Sector Count  192      192    140        64

On this same hard drive, there have been 4 Reallocation Events. This means that there were four occasions where the drive marked sectors as bad, and used spare sectors instead.

ID                             Current  Worst  Threshold  Raw  
=============================  =======  =====  =========  ===
(05) Reallocated Sector Count  192      192    140         64
(C4) Reallocated Event Count   196      196    0            4

What if it can't ever read the data?

These actions of re-reading sectors, consuming spares, all behind the computer's back is a good thing. It means the host operating system doesn't have to deal with the issue of failing sectors. The drive itself can handle those details itself.

Bonus Chatter: In the olden days, your hard drive shipped with a sticker fastened to it. This sticker contained the Factory Defect List; the list of all known bad spots on the drive.

enter image description here

If you performed a low-level format of the drive, you had to use a tool to type in all the Cylinder-Head-Sector locations of the bad spots.
SCSI drives have a command, IOCTL_DISK_REASSIGN_BLOCKS, to tell them to reallocate a bad spot on the drive after the operating system detects it. In IDE drives this all happens automatically, without the need for operating system intervention.

Ideally the drive would recognize the sector is failing, move the data to a spare sector, and never use the original sector again. But what happens if the drive hasn't been able to read the sector successfully?

This is what Pending Sectors are. The drive has detected that a sector is failing, and needs to be remapped to a spare. But it cannot do that until it can successfully read the data. When the drive knows that a sector is bad, and needs to be remapped, but it cannot do it yet because it's waiting to get a good read from the sector: that's called the Pending Sector Count:

ID                             Current  Worst  Threshold  Raw  
=============================  =======  =====  =========  ====
(05) Reallocated Sector Count  192      192    140          64
(C4) Reallocated Event Count   196      196    0             4
(C5) Current Pending Sector    100      100    0             2

My hard drive has 2 sectors that the drive recognizes as bad, but cannot be reallocated yet. If you were to read one of these ‘Pending sectors’, the drive would likely retry (and retry, and retry), and eventually return a read error to the host operating system:

enter image description here

Give up on the pending sector and it will get reallocated

There are two ways that the drive can finally reallocate the sector, and consume another spare sector:

  • it finally gets a good read
  • you don't care what's in the sector anymore

If the drive finally read the sector, then it knows that it can reallocate the sector.

The other way the drive can reallocate the sector is if you let it know that the contents of that sector are irrelevant; that you don't care what's in it anymore. How do you do that?

By writing something new to the sector.

Whenever you read from, or write to, a sector on a hard drive, you have to read/write the entire 512-byte sector1. You are not able to write only part of a sector. When the OS writes data to a sector, it has to specify the entire 512-bytes. If you tell the hard drive that you want these new contents to replace this bad sector, the drive then knows you don't even care what's currently in the bad sector. It can then Reallocate a bad sector to one of the spares, and the sector is no longer Pending.

This is why when people ask about having some Current Pending Sectors, the common advice is to use a tool (such as Western Digital's Data LifeGuard) to write all zero's to the drive.

enter image description here

By writing zeros to every sector on the drive, you are telling the drive that it can finally reallocate all those pesky pending sectors. After the wipe, all your Pending Sectors will become Reallocated Sectors:

ID                             Current  Worst  Threshold  Raw  
=============================  =======  =====  =========  ====
(05) Reallocated Sector Count  192      192    140          66
(C4) Reallocated Event Count   196      196    0             5
(C5) Current Pending Sector    100      100    0             0

Note: It's not strictly necessary to use a "low level" tool like Western Digital's Data LifeGuard. If you instruct Windows perform a full format (i.e. non-Quick format) of a volume, it will writes zeros to every sector in the volume.

The OS filing system supports marking sectors as bad

Armed with this knowledge, we'll explore a commonly confusing scenario.

Before the advent of Integrated Drive Electronics (IDE), the host operating system was responsible for detecting bad sectors, retrying the reads, moving data to another sector, and marking old sectors as bad.

If you were to run a chkdsk /r c: using the host operating system, it would recognize that the "pending" sectors are bad, and mark them as bad itself, and never try to use them again:

> C:\Windows\system32>chkdsk /r c:
The type of the file system is NTFS.
Volume label is OS.

        12 KB in bad sectors.

So assuming a 512-byte sector hard drive, 12 KB of ‘Pending Sectors’ or in this example 12KB marked by the OS as ‘bad sectors’, that would correspond to decimal 24 or hexadecimal 0x18 as would be shown by a S.M.A.R.T. disk utility such as Crystal Disk Information:

ID   Attribute Name            Current  Worst  Threshold  Raw  
=============================  =======  =====  =========  ====
(C5) Current Pending Sector    100      100    0            18

Note: Western Digital’s Data LifeGuard v1.31 (latest as of 8/31/2017) utility does not appear to show the current S.M.A.R.T. ‘Raw’ counter values correctly.

Now if you perform a full format (which writes zeros to every sector in the volume):

enter image description here

That means all those sectors that were Pending are going to get reallocated. It is now safe for the filing system to use those sectors again. In order to instruct the filing system that those sectors are no longer "bad", you perform an option where it reevaluates bad sectors:

>chkdsk c: /B

where the command documentation says

/B              NTFS only: Re-evaluates bad clusters on the volume
                (implies /R)

Or

According to: https://technet.microsoft.com/en-us/library/cc730714(v=ws.11).aspx

/B              NTFS only: Clears the list of bad clusters on the volume and 
                  rescans all allocated and free clusters for errors. /b includes
                  the functionality of /r. Use this parameter after imaging a
                  volume to a new hard disk drive.

This was a whole lotta writing, and a whole lotta screenshots, for something that will never be read.

Ian Boyd

Posted 2012-01-30T20:59:25.527

Reputation: 18 244

2One thing I don't quite understand: is it possible to (a) show which files contain the current C5 sectors, and (b) 'nuke' those sectors, i.e. move them to C6, and stop trying to read them? I don't want my HDD to get stuck in an endless failing read cycle; would rather black out those sectors and delete the files in question (manually restoring them from some other source if they're important). – M.M – 2014-12-29T21:54:48.747

@MattMcNabb That's what Western Digital's Data Lifeguard Diagnostic tool can do. It can zero out the entire driv.

– Ian Boyd – 2015-01-01T17:30:20.597

1I don't want to zero the entire drive though, just work around the bad bit. Will have a look at that program , thanks – M.M – 2015-01-01T22:15:22.247

So "pending sectors" is actually worse than "reallocated sectors"? Is it possible to identify which files the bad sectors are in? – endolith – 2015-12-14T16:29:10.817

This answer sounds reasonable. It still makes me wonder, why Acronis tells us in there official documentation that "Degradation of this parameter may indicate imminent drive failure. Urgent data backup and hardware replacement is recommended."

– Matthias Weiler – 2016-01-13T08:25:02.030

@M.M. Yes, it's possible. I've done it on EXT4 filesystem but it should be possible on NTFS too. First you need a list of bad blocks/sectors, then map them to file ID/inode equivalent in NTFS and then map those file IDs to file paths. I'm not sure if there is software that could do it for you though (you can always write one yourself, it isn't all that hard). – matt – 2016-07-28T07:05:46.893

@matt Bad sectors are outside the filesystem; they're on the physical medium. Is it possible to un-mark bad sectors of the filesystem. See my question here.

– Ian Boyd – 2016-07-28T15:00:30.060

I did read all of it and it did answer some questions I had. My problem is, however, that HDD keeps incrementing pending sectors count and sometimes decrements it without clear reason (zero reallocations). It does not report uncorrectable errors either! I did not yet try to zero the drive because I run my OS from it but I will do it later. – Euri Pinhollow – 2017-06-28T18:53:42.777

@Euri Pinhollow, got the same behavior recently - blind written the sector to provoke reallocation, but both Pending and Reallocated counts are now zero. My guess it that these sectors were "soft-bads", with some low level markers or tech data damaged on HDD surface. Writing to them could have restored that data. Is your drive made by Hitachi/HGST by the way? – ogurets – 2018-01-30T00:12:39.827

@ogurets it surely was not Hitachi/GHST but I cannot remember what exactly it was anymore. It probably was Fujitsu. Those could not be soft-bads because I could not read those sectors with dd-rescue (no access timeout, just read that **** already!) and there were zero UNCs even after read failure. This drove me mad at the time. Backup is love, backup is life. – Euri Pinhollow – 2018-01-30T12:21:36.433

@IanBoyd You had me right until the end of the answer, I'm confused by what exactly chdksk does. The documentation you quoted says it "clears the list of bad clusters on the volume". That sounds to me like it's clearing the G-LIST, the same G-LIST that you wrote zeroes to force the drive to allocate sectors to. Doesn't this basically mean that chdksk /b is undoing the work of mapping bad sectors? – Hashim – 2018-04-11T23:52:37.200

@IanBoyd Also, for clarification; a sector is marked as Pending Reallocated when a read fails. Is it then marked as truly bad (i.e. Reallocated proper) when a subsequent write to it fails, or when a write to it completes successfully? Thank you. – Hashim – 2018-04-12T00:28:26.260

@Hashim - A spare LBA is used for that write - it notices you are writing to a bad LBA, then it allocates a hidden spare, then writes to the spare. Since the drive hides spares from you (with some sort of translation or lookup table maintained internally by firmware), it looks like you're writing to the same LBA - but internally writing to the spare is happening. – LawrenceC – 2018-04-12T14:53:25.057

"By writing something new to the sector." - This is the answer I was looking everywhere for – Patrick – 2018-11-16T02:07:45.483

I have a 200MB file that has a problem at 87%. I got 3 pending sectors. I tried hashing the file to get good reads and claim reallocated sectors, but they are not getting reallocated! Hashing results: ok, fail, ok, ok, fail, fail. Every time the OS generated several Disk Errors in the event viewer, and on ok hashings, it hung for half a minute at 87%. All hashes are equal. C5 is still 3 and C4 still 0. What's your take on this? – Mark Jeronimus – 2014-05-11T20:25:51.753

@MarkJeronimus Try using a file wiper, which will write all zeros (or all random data) to the file. That should get the drive to realize you don't care about those sectors anymore. – Ian Boyd – 2014-05-11T23:31:41.390

I did a full format, but still no change. – Mark Jeronimus – 2014-05-14T08:53:35.163

@MarkJeronimus The only think i can think is that there are pending sectors in an area outside of a partition (i.e. outside of "Drive C"). Using something like Western Digital's hard drive tool to write zero's to the entire drive - including partition tables and boot sectors. – Ian Boyd – 2014-05-14T17:26:11.137

5

You don't need to do anything - yet. The reallocation count means "disk found defective sector and successfully reallocated it". The firmware already fixed the issue.

You just need to be aware that the drive is starting to fail. At some point in the future, the drive won't have spare sectors left for the reallocation and you will lose data. If some important block is lost, you might lose all the data on the drive.

So keep an eye on this value or buy a new drive right away so you can forget about it.

Aaron Digulla

Posted 2012-01-30T20:59:25.527

Reputation: 6 035

3

I had good experience with the method describe here https://www.smartmontools.org/wiki/BadBlockHowto. My disk was making noise and stop working so I use smart report and the formula to calculate the bad block and use dd to clear it. I was lucky because smart is still reporting a current sector count but the disk is working since 2 years or so. I did bought a replacement but I've never used it. It seems to me that smart isn't that accurate either.

Gigamegs

Posted 2012-01-30T20:59:25.527

Reputation: 1 784

oflag=direct option for dd when writing zeros to bad locations to force reallocation (as suggested in that link) was the part I was looking for. Thanks! – Anton – 2018-11-24T17:55:49.897

2

Maybe you do not know this tool: HD Sentinel

Hard Disk Sentinel (HDSentinel) is a multi-OS SSD and HDD monitoring and analysis software. Its goal is to find, test, diagnose and repair hard disk drive problems, report and display SSD and HDD health, performance degradations and failures. Hard Disk Sentinel gives complete textual description, tips and displays/reports the most comprehensive information about the hard disks and solid state disks inside the computer and in external enclosures (USB hard disks / e-SATA hard disks). Many different alerts and report options are available to ensure maximum safety of your valuable data.

This seems to be a great tool to do such actions. But take care, some run options can completely destroy data.

Estevan TH.

Posted 2012-01-30T20:59:25.527

Reputation: 21

1

You need to use a tool that writes to every sector of the drive. Otherwise, there's no way to do it. If the drive allocated a spare sector and remapped, what data should it put in the spare sector? It has no idea, since its attempt to read from the existing sector failed. Only a write operation can clear the error.

David Schwartz

Posted 2012-01-30T20:59:25.527

Reputation: 58 310

Formatting the entire hard drive is a write operation to every sector? I already formatted the drive (windows-based format command, no 'quick format'). Thanks – edteke – 2012-01-31T00:29:23.120

2

A quick format is not a write operation to every sector, a regular format usually is. You can always use DBaN. Did the regular format at least reduce the count?

– David Schwartz – 2012-01-31T00:32:36.963

I did a regular format (not a quick format), I expected the count down or a remap after the format, but number was the same. Thanks – edteke – 2012-01-31T00:40:30.173

That's very strange, actually. Was the partition you formatted the size of the entire drive? – David Schwartz – 2012-01-31T00:41:54.797

@edgh: A regular format doesn't help when you have such an error. I suggest you try my method. – Gigamegs – 2012-01-31T01:07:44.273

@David: A regular format writes to every sector. A write to a sector with a pending remap will force the remap. – David Schwartz – 2012-01-31T01:10:40.080

@DavidSchwartz I deleted the 3 partitions I had, create a single one (full size) then did the slow format. I thought that would force the remap, but it didn't. That's where my question comes from. – edteke – 2012-01-31T01:18:00.853

@edgh: It certainly should have. A write to a sector with a pending remap must force an immediate remap -- otherwise, what would the drive do with the data? – David Schwartz – 2012-01-31T01:22:07.377

@DavidSchwartz: I don't have an explain but it's my experience. Read the link it's seems dd and write zeros to it is better than format. – Gigamegs – 2012-01-31T01:32:34.210

1

Zero write, otherwise known as Low Level Format. You can only do this with a manufacturer disk utility usually. If there is a hard drive failure causing this error it may come back almost right away.

Chris Abbott

Posted 2012-01-30T20:59:25.527

Reputation: 11

1

Try Spinrite 6 or HDD regenerator bootable CDs. They do write surface tests so it should clear out the pending bad sectors and bump up the reallocated count.

danwat1234

Posted 2012-01-30T20:59:25.527

Reputation: 37

0

To add another solution, I recently used Victoria to repair 2 disks that failed with pending sectors, yet the disk wouldn't remap them automatically on it's own.

Victoria is a low level graphical interface to your disk, and includes (if you really want to!) actual byte by byte editing of the disk, force sector remapping on failures, supports USB drives and those cheap China-bay USB-2-sata adaptors, change factory set smart attributes and features, and more.

The downside - it's developed by a Russian developer, and I leave you to open that Pandora's box if you have issues with it...

With Victoria, I grabbed the first sector that failed to remap, ran a "Read" scan of each sector after that with a force remap. I then ran a "Verify" of the entire disk, re-ran the smart check, and it is now fine...

...But, I have spare disks, so I'm in the process of swapping them out now. This was a little scary, as I'm also between backup solutions, of all times!

KolonUK

Posted 2012-01-30T20:59:25.527

Reputation: 163