NVMe volatile memory backup device failed: CriticalWarning bit 4 and lost power-loss protection

You run smartctl -H /dev/nvme0n1 and get FAILED. The detail line reads - volatile memory backup device has failed. The Critical Warning byte shows 0x10. The drive is still serving reads and writes at full speed, latency is normal, Available Spare is fine, and Percentage Used is well within spec.

This is NVMe Critical Warning bit 4. It does not mean the NAND is failing or the controller is dying. It means the drive’s power-loss protection (PLP) hardware, typically supercapacitors on enterprise NVMe SSDs, has failed. Under stable power, the drive operates normally. But if power drops unexpectedly, data sitting in the drive’s volatile write buffer will be lost because the capacitor bank can no longer flush it to NAND.

The drive continues to serve I/O until the next power event, at which point the PLP safety net that justified deploying enterprise drives is already gone. This article covers what bit 4 means, how to verify it is not a false positive, how to correlate it with unsafe shutdown count, and what remediation options exist.

What this means

The NVMe SMART/Health Information Log (Log Page 02h) contains a 1-byte field called Critical Warning. Each bit maps to a specific condition the drive firmware considers critical:

When bit 4 is set, smartctl reports the overall health as FAILED and appends the specific condition text. Any non-zero Critical Warning byte triggers this FAILED declaration, regardless of which bit is set. This is expected smartctl behavior: it prints FAILED whenever any Critical Warning bit is present, whether or not that bit indicates an immediate media failure.

The NVMe specification defines this field as valid only if the controller has a volatile memory backup solution. On enterprise NVMe drives with PLP, this means the supercapacitor bank that provides power to flush the write buffer during an unexpected power loss has failed self-test. On drives without PLP hardware, bit 4 should not be set, but firmware bugs or spec misinterpretation on some consumer drives can produce false positives.

The operational impact is specific: the drive cannot guarantee data integrity across a power-loss event. The failure is latent, waiting for a power event to manifest as data loss.

Common causes

Quick checks

All commands below are read-only and safe to run on production drives.

# Check the Critical Warning byte value
smartctl -A /dev/nvme0n1 | grep "Critical Warning"

# Get the full health assessment text
smartctl -H /dev/nvme0n1

# Get the full SMART/Health output for context
smartctl -a /dev/nvme0n1

# Check Unsafe Shutdowns count
smartctl -A /dev/nvme0n1 | grep "Unsafe Shutdowns"

# Identify the drive model and firmware
smartctl -i /dev/nvme0n1

# Check for kernel-level events around the time the bit appeared
dmesg | grep -iE "nvme|power|shutdown|reset" | tail -30

# Confirm Available Spare is not also degraded (broader failure)
smartctl -A /dev/nvme0n1 | grep "Available Spare"

# Check Media and Data Integrity Errors to rule out NAND failure
smartctl -A /dev/nvme0n1 | grep "Media and Data Integrity Errors"

How to diagnose it

Step 1: Confirm the bit and decode the byte

Read the Critical Warning value. If it shows 0x10, only bit 4 is set. If the value is different (for example 0x14), multiple bits are active and you have a compound problem. Decode the full byte before acting on bit 4 in isolation.

Step 2: Verify the drive actually has PLP hardware

This is the most important diagnostic step. The NVMe spec says bit 4 is only valid if the controller has a volatile memory backup solution. If you are running a consumer NVMe SSD with no PLP capacitors, bit 4 may be a firmware false positive.

Check the drive model against the vendor datasheet. Enterprise NVMe drives commonly ship with PLP capacitors. If the datasheet does not mention power-loss protection or capacitors, the drive likely has no PLP hardware and bit 4 is meaningless.

Step 3: Correlate with Unsafe Shutdowns

# Check current Unsafe Shutdowns count
smartctl -A /dev/nvme0n1 | grep "Unsafe Shutdowns"

The Unsafe Shutdowns counter increments when the drive loses power without receiving a shutdown notification (CC.SHN). If PLP has failed, every future unsafe shutdown is a potential data loss event. A rising count combined with bit 4 set means the drive has already experienced power events without its safety net active.

Step 4: Rule out firmware bugs

Check whether the bit appeared after a firmware update:

# Current firmware version
smartctl -i /dev/nvme0n1 | grep -i firmware

Compare against known-good firmware versions. Check vendor advisories and community forums for reports of bit 4 false positives on this specific model and firmware combination.

Step 5: Assess the full SMART picture

Confirm that the rest of the drive’s health indicators are normal. If Available Spare is also below threshold (bit 0 set) or Media and Data Integrity Errors are non-zero and increasing, the drive may have broader age-related or hardware failures beyond the PLP capacitor.

flowchart TD
    A["Critical Warning = 0x10"] --> B{"Drive has PLP hardware?"}
    B -->|Yes| C["PLP capacitor failure
Safety net is gone"]
    B -->|No| D["Likely firmware false positive
Suppress or update firmware"]
    C --> E{"Unsafe Shutdowns
count rising?"}
    E -->|Yes| F["High risk: data loss
has likely occurred"]
    E -->|No| G["Latent risk: data loss
on next power event"]
    F --> H["Replace drive
or add UPS redundancy"]
    G --> H
    D --> I["Use smartd.conf to suppress
if supported by your version"]

Metrics and signals to monitor

Fixes

Replace the drive (enterprise with PLP requirement)

If the drive was deployed specifically because PLP is a data integrity requirement (database write cache, journal device, write-heavy workload without reliable UPS redundancy), the only correct fix is replacement. The drive cannot be repaired in the field. The capacitor bank is not a serviceable component.

Tradeoff: replacement requires a maintenance window, data migration, and potential RAID rebuild. But continuing to operate a PLP drive without PLP defeats the purpose of deploying enterprise drives.

Suppress the alert for consumer drives without PLP

If the drive does not actually have PLP hardware and bit 4 is a firmware false positive, you can configure smartd to ignore the bit.

# smartd.conf: proposed syntax to ignore bit 4 (0x10) for this device
# NOTE: -o on and -S on are ATA-only directives, not applicable to NVMe
/dev/nvme0n1 -d nvme -a

If bitmask masking is not supported in your version, the alternative is -d ignore in smartd.conf, which disables all SMART monitoring for the device. This is strongly discouraged because it creates a complete monitoring blind spot for every other failure mode. A better approach may be to exclude the specific drive from smartd and monitor it with a custom script that checks the Critical Warning byte and masks bit 4 before alerting.

Tradeoff: suppressing the bit is appropriate only for drives that genuinely lack PLP hardware. Suppressing it on an enterprise drive with actual PLP hides a real hardware failure.

Accept the risk temporarily (not recommended)

If the server has reliable UPS backup with generator failover, the probability of an unsafe shutdown may be low enough that some teams accept the risk temporarily while awaiting a replacement. This is a business decision, not a technical fix. Document the acceptance explicitly, set a hard deadline for replacement, and track the Unsafe Shutdowns count closely during the interim.

Prevention

• Track PLP health across the fleet: Monitor Critical Warning bit 4 on all enterprise NVMe drives. A PLP failure is silent until a power event exposes it.
• Monitor Unsafe Shutdowns trend: A rising count on any drive indicates power infrastructure issues. On a drive with failed PLP, each increment is a potential data loss event.
• Maintain UPS redundancy: PLP is defense in depth, not the primary protection. Reliable power infrastructure is the first line of defense against data loss from unexpected shutdowns.
• Track firmware versions: Some bit 4 false positives are triggered by specific firmware versions on consumer drives. Maintain a firmware inventory and watch vendor advisories.
• Baseline at deployment: Capture the Critical Warning byte value and Unsafe Shutdowns count when a drive is first deployed. Alert on change from baseline, not on absolute values, to avoid false alarms from historical counts.

How Netdata helps

• Per-second metric collection: Netdata collects NVMe SMART attributes including the Critical Warning byte at per-second granularity, so you see the moment bit 4 transitions from 0 to 1 without waiting for the next polling cycle.
• Correlation with Unsafe Shutdowns: When bit 4 is set, the Unsafe Shutdowns counter becomes the critical correlated signal. Netdata’s dashboard lets you overlay both metrics on the same timeline to assess whether power events have already occurred without PLP protection active.
• Anomaly detection: Netdata’s ML-based anomaly detection can flag the transition of Critical Warning from 0x00 to 0x10 as anomalous even without a configured static threshold.
• Fleet-level visibility: Across multiple servers, Netdata aggregates SMART data so you can see whether bit 4 is isolated to one drive or affecting multiple drives of the same model, which may indicate a batch defect or firmware issue.

Related guides

• NVMe Critical Warning bits: decoding the health-log bitmask
• SMART overall-health self-assessment: FAILED is the drive’s own death notice
• NVMe Available Spare below threshold: the spare block pool is running out
• NVMe Percentage Used at or above 100%: rated endurance consumed
• SMART says PASSED but the drive is failing: why the health check lies
• How S.M.A.R.T. actually works: a mental model for operators
• smartctl disk monitoring checklist: the SMART signals every server needs
• SMART monitoring maturity model: from survival to expert