Are hard disk's bad sectors filesystem dependent?

Question

I was wondering where defective sectors are stored on the hard disk: - in a filesystem independent area, so that if the hard disk is formatted again with same/different filesystem they will show as bad?

I ask this because my digital sat PVR's hard disk was showing a buggy behavior, so I have attached the HD to my PC, however as it uses a proprietary filesystem (XTV), I have formatted it with NTFS and issued CHKDSK /F /R to verify sectors: as a result 24 sectors were marked as bad.

Unmounted the disk and plugged into the PVR, the disk was re-initialized with XTV filesystem.

At this point I wonder if the sectors marked as bad are still flagged or not

Answer 1

Disk bad sectors are FileSystem independent.

There are two bad sector list which are stored in the disk in the area called SystemArea which is on the platter also and comes before user (e.g. O.S.) accessable sectors and they are managed by disk itself. Their names are P-List and G-List. The P-List is Primary List that is permenantly written to disk after it is manufactured and tested by defective sectors. The G-List is Grown List that is managed and maintained on the fly. After some time sectors can go bad magnetically, which is absolutely normal, or can go bad resulted via ECC errors. With these and any other on the fly causes, the G-List keeps track of bad sectors you can see. There are of course some disk tools which you can manipulate the G-List and the G-List only. You can't change the contents of P-List.

A file system can keep track the bad sectors, also. You can think of this as in access speed considerations. And that tracking mechanism is FS dependent, sadly. We cannot say "Oh, my fs is keeping the tracking the bad sectors correctly" until we see the source code of that fs itself. We don't know whether it updates G-List from the disk or not when issuing a disk format or a disk check, as it should, to sync with real bad sectors.

Back to your original question, a file system must be one of the most sophisticated parts of an O.S. and your XTV file system should, should do that sycn with the physical bad sector trackings.

Answer 2

The best thing you can do with a disk which already has 24 bad sectors is to throw it away and replace it with a new one (hope that your PVR is not locked to a particular disk and will accept such replacement). If the disk started going bad, usually the process does not stop, and soon you will see either thousands of bad sectors or a complete disk failure. Or the disk just may become very slow, which will result in broken recording or playback when using it in a PVR.

As for your question, the CHKDSK /F /R command marks bad sectors only on the filesystem level — the information about those sectors is stored only in NTFS structures, and is lost when the disk is reformatted. So now you have a disk with 24 unreadable sectors, but your PVR does not know anything about it, and will happily try to use those sectors.

However, hard disks are able to perform bad sectors remapping themselves — there is a “Grown Defect List” stored on the disk itself, which is used to redirect attempts to access sectors which have gone bad to a spare area. This remapping is not visible to the computer (or any other device which is using the disk), except when using special low-level utilities.

The remapping is usually triggered by an attempt to write into a sector which has been previously observed as unreadable by the disk. So you may try to ignore the problem and hope for the best, because your PVR should not try to read those sectors that it did not write yet after reformatting, and the disk knows about those 24 bad sectors (because it tried to read them during the CHKDSK test) and should remap them when some data is written there. Of course, this will not help if some other sectors will go bad later (which is very likely to happen with a disk which started to break).

One more thing you can do is go to the disk manufacturer's web site and download a disk testing utility (e.g., SeaTools for a Seagate disk), then attach the disk to your computer and perform a full media test. This utility may be able to force bad sectors remapping by the disk, so that the disk will not appear to have bad sectors when accessed in a normal way (at least while no other bad sectors appear due to ongoing deterioration of the disk). If the disk is in a particularly bad shape, the utility may show that it requires replacement (but just having several bad sectors which could be remapped is usually not considered to be a severe problem by these utilities).

Note that if you choose to run the disk manufacturer's testing utility, you must attach the disk directly to a SATA controller (or IDE, if your disk and computer are that old), and not through an external adapter with an USB or FireWire interface, because the disk utility must be able to send arbitrary ATA commands to the disk, which is not possible when using many USB-SATA or FireWire-SATA controller chips (and even if it is possible, most likely the utility does not support it). In some cases you may also need to change BIOS settings to make the disk utility work (e.g., disable AHCI or RAID mode if they are not supported by the utility); in this case you must be especially careful to restore those settings, or your OS may no longer boot (or, even worse, a RAID array may be destroyed if RAID mode was used).

Answer 3

In a nutshell, if you observe poor blocks in any of the file systems, it is likely that the service area to redirect the bad sector is full. I suggest you give this HDD enemy)).

See disk error at eventlog:

powershell Get-Eventlog system -newest 10 -EntryType Error -Source Disk

See all disk event at eventlog:

powershell Get-Eventlog system -Source Disk

Test S.M.A.R.T. status, smartChk.vbs:

Dim strComputer 
Dim strLogin 
Dim strPassword 
On Error Resume Next 

strComputer = "."

If strLogin = "" or strComputer = "." then 
   Set    objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\WMI") 
else 
   Set objSWbemLocator = CreateObject("WbemScripting.SWbemLocator") 
   Set objWMIService = objSWbemLocator.ConnectServer(strComputer, _ 
  "root\CIMV2", _ 
  strLogin, _ 
  strPassword, _ 
  "MS_409") 
end If 

Set    SmartDataItems = objWMIService.ExecQuery("SELECT * FROM MSStorageDriver_FailurePredictData", "WQL", 48) 

    InstanceName = Array() 
    SmartData = Array() 
    SmartLength = Array() 
    RowCount = 0 
    For Each objItem In SmartDataItems 
      Redim Preserve InstanceName(RowCount) 
      InstanceName(RowCount) = objItem.InstanceName 
      Redim Preserve SmartData(RowCount) 
      SmartData(RowCount) = objItem.VendorSpecific 
      Redim Preserve SmartLength(RowCount) 
      SmartLength(RowCount) = objItem.Length 
      RowCount = RowCount + 1 
Next 
Set    SmartDataItems = Nothing 

ThresholdData = Array() 
Redim Preserve ThresholdData(RowCount) 
PredictFailure = Array() 
Redim Preserve PredictFailure(RowCount) 
Reason = Array() 
Redim Preserve Reason(RowCount) 

RowCount = 0 
    Set    ThresholdItems = objWMIService.ExecQuery("SELECT * FROM MSStorageDriver_FailurePredictThresholds", "WQL", 48) 
For Each objItem In ThresholdItems 
      ThresholdData(RowCount) = objItem.VendorSpecific 
      RowCount = RowCount + 1 
Next 
Set    ThresholdItems = Nothing 

RowCount = 0 
    Set    PredictStatusItems = objWMIService.ExecQuery("SELECT * FROM MSStorageDriver_FailurePredictStatus", "WQL", 48) 
    For Each objItem In PredictStatusItems 
          PredictFailure(RowCount) = objItem.PredictFailure 
          Reason(RowCount) = objItem.Reason 
          RowCount = RowCount + 1 
    Next 
Set    PredictStatusItems = Nothing 

Set oDict = CreateObject("Scripting.Dictionary") 
CreateDict(oDict) 

Set objFSO = CreateObject("Scripting.FileSystemObject") 

If strComputer = "." then 
   Set objTextFile = objFSO.OpenTextFile("HDD_Smart.csv", 2, True) 
else 
   Set objTextFile = objFSO.OpenTextFile("HDD_Smart_" & strComputer & ".csv", 2, True) 
end If 

For CurrentDisk = 0 to RowCount - 1 
     objTextFile.WriteLine("Drive: " & Replace(Mid(InstanceName(CurrentDisk), 9, InStr(InstanceName(CurrentDisk), "__") - 9), "_", " ")) 
     objTextFile.WriteLine("PredictFailure: " & PredictFailure(CurrentDisk)) 
     objTextFile.WriteLine("Reason: " & Reason(CurrentDisk)) 
     objTextFile.WriteLine("ID;Attribute;Type;Flag;Threshold;Value;Worst;Raw;Status;") 
     aSmartData = SmartData(CurrentDisk) 
     aThresholdData = ThresholdData(CurrentDisk) 

     If IsArray(aSmartData) AND IsArray(aThresholdData) Then 
         LastID = 0 
         For x = 2 To SmartLength(CurrentDisk) + 2 Step 12 
              If LastID > aSmartData(x)  then 
                 x = 514 
              else 
                 LastID = aSmartData(x)                      

              If aSmartData(x) <> 0 Then 
                  objTextFile.Write(aSmartData(x) & ";") 
                  If oDict.Item(aSmartData(x)) = "" Then 
                        objTextFile.Write("VendorSpecific(" & aSmartData(x) & ");") 
                  else 
                        objTextFile.Write(oDict.Item(aSmartData(x)) & ";") 
                  end If 

                  If aSmartData(x + 1) MOD 2 Then 
                        objTextFile.Write("Pre-Failure;") 
                  Else 
                        objTextFile.Write("Advisory;") 
                  End If 

'                      objTextFile.Write(aSmartData(x + 1) & ";") 
                  aFlag = aSmartData(x + 1) 
                  txtFlag = "" 
                  If (aFlag And &H1) <> 0 Then txtFlag = "LC," End If 
                  If (aFlag And &H2) <> 0 Then txtFlag = txtFlag & "OC," End If 
                  If (aFlag And &H4) <> 0 Then txtFlag = txtFlag & "PR," End If 
                  If (aFlag And &H8) <> 0 Then txtFlag = txtFlag &  "ER," End If 
                  If (aFlag And &H10) <> 0 Then txtFlag = txtFlag &  "EC," End If 
                  If (aFlag And &H20) <> 0 Then txtFlag = txtFlag &  "SP," End If 
                  If txtFlag <> "" then 
                     txtFlag = Left(txtFlag, Len(txtFlag)-1 ) 
                  end If 
                  objTextFile.Write txtFlag & ";" 

                  objTextFile.Write(aThresholdData(x + 1) & ";") 'Threshold 
                  objTextFile.Write(aSmartData(x + 3) & ";")      'Value 
                  objTextFile.Write(aSmartData(x + 4) & ";")      'Worst 
                  objTextFile.Write((aSmartData(x + 8) * 65536 + aSmartData(x + 7) * 4096 + aSmartData(x + 6) * 256 + aSmartData(x + 5)) & ";")      'Raw 
                  If aSmartData(x + 3) >= aThresholdData(x + 1) Then 
                       objTextFile.WriteLine("OK;") 
                  else 
                       objTextFile.WriteLine("NOT OK;") 
                  end If 
              end If 
              end If 
         Next 
         else 
             objTextFile.WriteLine("NO DRIVE WITH SMART FOUND;") 
         end If 
         objTextFile.WriteLine 
    Next 

    objTextFile.WriteLine 
    objTextFile.WriteLine("LC - life critical;") 
    objTextFile.WriteLine("OC - online collection;") 
    objTextFile.WriteLine("PR - performance related;") 
    objTextFile.WriteLine("ER - error rate;") 
    objTextFile.WriteLine("EC - event count;") 
    objTextFile.WriteLine("SP - self preserving;") 
    objTextFile.Close 

    Wscript.echo "HDD_SMART Is Done!" 

Function CreateDict(oDict) 
    oDict.Add 1, "Raw Read Error Rate" 
    oDict.Add 2, "Throughput Performance" 
    oDict.Add 3, "Spin-Up Time" 
    oDict.Add 4, "Start/Stop Count" 
    oDict.Add 5, "Reallocated Sectors Count" 
    oDict.Add 6, "Read Channel Margin" 
    oDict.Add 7, "Seek Error Rate Rate" 
    oDict.Add 8, "Seek Time Performance" 
    oDict.Add 9, "Power-On Hours (POH)" 
    oDict.Add 10, "Spin Retry Count" 
    oDict.Add 11, "Recalibration Retries Count" 
    oDict.Add 12, "Device Power Cycle Count" 
    oDict.Add 13, "Soft Read Error Rate" 
    oDict.Add 184,"End-to-End error"
    oDict.Add 187,"Reported UNC Errors"
    oDict.Add 190,"Airflow Temperature (WDC)"
'    oDict.Add 190, "HDA Temperature" 
    oDict.Add 191, "G-Sense Error Rate Frequency" 
    oDict.Add 192, "Power-Off Park Count" 
    oDict.Add 193, "Load/Unload Cycle Count" 
    oDict.Add 194, "HDA Temperature" 
    oDict.Add 195, "Hardware ECC Corrected Count" 
    oDict.Add 196, "Reallocated Event Count" 
    oDict.Add 197, "Current Pending Sector Count" 
    oDict.Add 198, "Off-Line Scan Uncorrectable Sector Count" 
    oDict.Add 199, "UltraDMA CRC Error Count" 
    oDict.Add 200, "Write Error Rate" 
    oDict.Add 201, "Soft Read Error Rate" 
    oDict.Add 202, "Address Mark Errors Frequency" 
    oDict.Add 203, "ECC errors (Maxtor: ECC Errors)" 
    oDict.Add 204, "Soft ECC Correction" 
    oDict.Add 205, "Thermal Asperity Rate (TAR)" 
    oDict.Add 206, "Flying Height" 
    oDict.Add 207, "Spin High Current" 
    oDict.Add 208, "Spin Buzz" 
    oDict.Add 209, "Offline Seek Perfomance" 
    oDict.Add 210, "Vibration During Write" 
    oDict.Add 211, "Vibration During Read" 
    oDict.Add 212, "Shock During Write" 
    oDict.Add 220, "Disk Shift" 
    oDict.Add 221, "G-Sense Error Rate" 
    oDict.Add 222, "Loaded Hours" 
    oDict.Add 223, "Load/Unload Retry Count" 
    oDict.Add 224, "Load Friction" 
    oDict.Add 225, "/Unload Cycle Count" 
    oDict.Add 226, "Load 'In'-time" 
    oDict.Add 227, "Torque Amplification Count" 
    oDict.Add 228, "Power-Off Retract Cycle" 
    oDict.Add 230, "GMR Head Amplitude" 
    oDict.Add 231, "Temperature"
    oDict.Add 240, "Head Flying Hours" 
    oDict.Add 250, "Read Error Retry Rate" 
End Function