$ guides / smartctl-disk-monitoring / smartctl-health-passed-but-failing ▌

Operations Guides

SMART says PASSED but the drive is failing: why the health check lies

The PASSED verdict from smartctl -H tells you one narrow thing: no pre-fail SMART attribute has crossed its vendor-defined threshold. It does not mean the drive is healthy, that data is safe, or that the drive will survive the week.

A drive can report PASSED while hundreds of sectors are pending reallocation, dozens have already been remapped, I/O latency is spiking to seconds, and the spare pool is burning down. Google’s 2007 study of over 100,000 drives found that 36% of failed drives had zero prior SMART warnings. Backblaze’s fleet analysis showed that 23.3% of failed drives had no non-zero values across the five attributes they consider most predictive (IDs 5, 187, 188, 197, 198). The health check is a last-resort binary, not a health indicator.

What the PASSED verdict actually checks

The SMART Overall Health Self-Assessment is a binary pass/fail judgment from the drive firmware. On ATA/SATA drives, smartctl -H issues the SMART RETURN STATUS command. The drive compares its internal attribute values against vendor-defined thresholds and reports whether any pre-fail attribute has crossed its threshold.

SMART attributes carry a TYPE column in smartctl -A output: Pre-fail or Old_age. Only Pre-fail attributes participate in the health verdict. Old_age attributes (like Power-On Hours) never trigger FAILED regardless of their value. Even among Pre-fail attributes, the verdict only fires when the normalized VALUE drops to or below the THRESH value. Vendors set these thresholds so that FAILED triggers only at an advanced state of degradation. A drive can accumulate significant media damage and still have every normalized value sitting comfortably above its threshold.

This matters because most monitoring setups, naive scripts, and even some commercial tools treat PASSED as proof of health. They check the exit code or the verdict string and move on. By the time SMART says FAILED, the drive has been dying for weeks or months, and you are at the point of maximum damage and minimum recovery time.

How it works

The threshold comparison

For ATA drives, each SMART attribute has three key fields in the smartctl -A output:

VALUE: The normalized current value, on a vendor-defined scale (typically 1-253, where higher is better).
WORST: The lowest normalized value the attribute has ever reached.
THRESH: The vendor-defined threshold below which the attribute is considered failing.

The health check logic is straightforward: for each Pre-fail attribute, if VALUE is at or below THRESH, the drive reports FAILED. Vendors set THRESH low. A Reallocated_Sector_Ct (ID 5) attribute might have a threshold so low that the raw reallocated count can reach into the hundreds before the normalized value drops far enough to trip the verdict.

The exit code bitmask

smartctl encodes diagnostic information in its exit status as a bitmask. The relevant bits for health assessment:

Bit 3: SMART RETURN STATUS check returns “DISK FAILING.”
Bit 4: A prefail attribute is at or below its threshold.
Bit 5: SMART status check returned “DISK OK” but some usage or pre-fail attributes were at or below threshold in the past.
Bit 6: The device error log contains errors.
Bit 7: The device self-test log contains errors.

A clean PASSED with exit code 0 means bits 3 and 4 are not set: the drive is not reporting “DISK FAILING” and no prefail attribute is currently at or below threshold. This says nothing about attributes trending downward, pending sectors, error logs, or any signal that has not yet crossed the vendor’s line.

A reported case illustrates the gap: smartctl -H returned exit code 0 on a failing drive, while smartctl -a returned exit code 192 (bits 6 and 7 set: error log and self-test log both contain errors). A monitoring script checking only -H saw nothing.

Note: With smartctl -H, smartmontools 7.3 and later no longer set bit 2 solely because ATA attributes are available. Bit 2 still reports a failed SMART/ATA command or a checksum error. Do not treat bit 2 as a catch-all “something is wrong” signal.

NVMe: a different mechanism

NVMe drives do not use ATA SMART attribute thresholds. The smartctl -H health assessment for NVMe reads the Critical Warning byte from the SMART/Health Information Log (Log Page 02h). Each bit represents a specific condition:

Bit 0: Available spare below threshold
Bit 1: Temperature is above or below its threshold
Bit 2: NVM subsystem reliability degraded
Bit 3: Media placed in read-only mode
Bit 4: Volatile memory backup device failed
Bit 5: Persistent memory region is read-only or unreliable

If no Critical Warning bit is set, the drive reports PASSED. This catches some conditions the ATA check misses, but it still does not catch gradual wear. An NVMe drive can report 188% Percentage Used with the Critical Warning byte at 0x00 and still return PASSED. The wear is extreme, the drive is far beyond its rated endurance, but no threshold has been crossed because Available Spare has not yet dropped below the vendor threshold and no media errors have occurred.

flowchart TD
    A["smartctl -H /dev/sdX"] --> B{"ATA or NVMe?"}
    B -->|"ATA/SATA"| C["SMART RETURN STATUS\ncommand"]
    B -->|"NVMe"| D["Read Critical Warning\nbyte from Log Page 02h"]
    C --> E{"Any Pre-fail attribute\nVALUE at or below THRESH?"}
    E -->|"No"| F["PASSED"]
    E -->|"Yes"| G["FAILED"]
    D --> H{"Any Critical\nWarning bit set?"}
    H -->|"No"| F
    H -->|"Yes"| G
    F --> I["Drive can still have:\nhundreds of pending sectors\ndozens of reallocated sectors\n188% Percentage Used\nrising error log entries"]
    G --> J["Drive firmware declares\nfailure. Evacuate immediately."]

Where it shows up in production

The monitoring script that only checks exit code 0

The most common pattern: a cron job or monitoring check runs smartctl -H /dev/sdX, greps for “PASSED,” and alerts only on “FAILED.” This setup is blind to everything except the vendor’s final verdict. Drives with rapidly growing reallocated sectors, climbing pending counts, and degrading performance sail through with a clean bill of health. When the drive finally fails, the monitoring system had nothing to say for weeks.

USB enclosures and the attribute check fallback

USB-to-SATA bridges often do not pass the ATA SMART RETURN STATUS command through to the drive. When this happens, smartctl -H falls back to an attribute-based check and prints a warning: “This result is based on an Attribute check.” The result may still say PASSED, but it is a best-effort guess, not the drive’s own firmware verdict. Drives behind USB bridges are effectively unmonitored for health unless you use the correct -d sat or bridge-specific flag and verify the data path works.

NVMe drives past rated endurance

NVMe Percentage Used is explicitly allowed to exceed 100% per the NVMe specification. Real-world reports show drives at 123%, 188%, and higher, all reporting PASSED because the Critical Warning byte remains 0x00. The Available Spare threshold is vendor-specific. As long as the spare pool has not dropped below that threshold, the drive considers itself fine. It is operating well beyond warranty, NAND retention is degrading, and the next bad block could exhaust spares, but the health check says PASSED.

Hardware RAID controllers masking everything

Behind hardware RAID controllers (MegaRAID, Smart Array, Adaptec), standard SMART queries fail or return virtual device data. If your monitoring runs smartctl -H /dev/sda where /dev/sda is a RAID virtual disk, you are not checking physical drive health at all. You must use controller-specific passthrough:

smartctl -a -d megaraid,0 /dev/sda

Without the correct -d type, every physical drive could be failing and the health check would still say PASSED on the virtual device.

Counter resets and tampered drives

SMART counters are cumulative and should never decrease. A drive that shows Reallocated_Sector_Ct dropping to zero, or Power-On Hours decreasing, has either been swapped, had a firmware update that reset counters, or been tampered with. In 2025, used Seagate enterprise drives with 15,000-50,000 hours were sold as new with SMART values reset to near zero. The fraud was initially caught because Seagate drives keep a second log (FARM) that scammers did not modify. Later variants compromised FARM values too. If you are monitoring secondhand drives, a PASSED verdict with suspiciously clean counters is a red flag, not reassurance.

Common misuses

Treating PASSED as proof of health. PASSED means “no pre-fail attribute has crossed the vendor threshold yet.” A drive can be actively degrading for weeks before the verdict changes.

Alerting only on FAILED. By the time the drive firmware says FAILED, you have lost recovery time. Individual attributes provide warnings days to weeks earlier.

Assuming exit code 0 means no problems. The exit code bitmask includes bits for error log entries (bit 6) and self-test log errors (bit 7). A drive can return PASSED with a non-zero exit code from those bits while the health verdict string still says PASSED. Scripts that check only for the string “PASSED” miss this.

Using the same thresholds for all drive types. A 60C temperature reading is critical for an HDD but within spec for an enterprise NVMe. Seagate raw values for IDs 1 and 7 can be in the billions and be normal. Alert thresholds must be drive-type-aware and vendor-aware.

Ignoring rate of change. A drive with 5 reallocated sectors accumulated over 5 years is stable. A drive with 5 reallocated sectors gained this week is actively dying. Most monitoring checks absolute values only, missing the acceleration signal that distinguishes historical damage from active failure.

Signals to watch in production

Signal	Why it matters	Warning sign
Reallocated_Sector_Ct (ID 5)	Drive is consuming its finite spare pool. Growth rate is the key metric.	Any increase from baseline, especially if accelerating.
Current_Pending_Sector (ID 197)	Sectors the drive cannot reliably read. Each one can cause multi-second I/O latency spikes.	Any non-zero value. Sustained non-zero across polls indicates active deterioration.
Offline_Uncorrectable (ID 198)	Permanent data loss at the media level. Data in these sectors is gone.	Any increase from baseline. Verify the attribute name, not just the ID: some WD models use ID 198 for a different purpose.
UDMA_CRC_Error_Count (ID 199)	Physical layer problem between drive and host. Usually a cable or backplane issue, not the drive.	Any increase. Commonly misdiagnosed as drive failure, leading to unnecessary replacement.
NVMe Available Spare	Remaining capacity to handle bad blocks. When it hits 0%, the next bad block means permanent data loss.	Below vendor threshold. Any declining trend.
NVMe Percentage Used	Endurance consumption estimate. Can exceed 100% per spec.	Above 90%: plan replacement. Above 100%: operating beyond rated life.
NVMe Critical Warning bits	Drive firmware flagging specific critical conditions via a bitmask.	Any non-zero value. Each bit maps to a distinct condition (spare, temperature, reliability, read-only, PLP).
NVMe Media and Data Integrity Errors	Uncorrectable data errors from the NAND media. The NVMe equivalent of Offline_Uncorrectable.	Any increase from baseline.
ATA Error Log entries	History of I/O errors with type, LBA, and timing relative to power cycle.	Any non-zero count, especially UNC (uncorrectable) errors. The summary log holds only 5 entries; old errors are overwritten.
Kernel I/O errors (dmesg)	Host-level perspective on drive health. Catches failure modes SMART cannot self-report.	“I/O error”, “medium error”, “timeout”, or “reset” messages for the specific device.
Self-test results	Active surface scanning that finds latent bad sectors before production I/O hits them.	Any result other than “Completed without error”. Requires scheduled tests; drives do not run them automatically.

How Netdata helps

Netdata collects the individual signals that predict failure, rather than relying on the PASSED/FAILED verdict as the primary health indicator.

Per-second collection of SMART attributes means rate-of-change tracking is continuous. You see when Reallocated_Sector_Ct or Current_Pending_Sector starts climbing in real time, alongside whatever workload triggered the degradation.
Correlation between SMART attributes and I/O latency surfaces the zombie drive pattern: pending sectors causing await spikes in disk metrics while the health check still reports PASSED.
NVMe health log collection including Available Spare, Percentage Used, Critical Warning bits, and Media and Data Integrity Errors provides the full endurance and reliability picture. A drive at 150% Percentage Used with declining Available Spare gets attention before the Critical Warning byte flips.
Anomaly detection on SMART attribute trends flags unusual changes that static thresholds miss, such as slow reallocated sector accumulation that has not yet crossed any configured alert line.
Error log and self-test monitoring catches UNC errors and self-test failures that the health check ignores.
Kernel I/O error correlation provides the host-side perspective that complements drive-side SMART data, catching firmware bugs and controller issues that SMART cannot self-report.