NVMe percentage used at 100%: reading the endurance-consumed estimate

Your monitoring just told you an NVMe drive is at 100% “percentage used”, or 123%, or 167%. The number looks like a fuel gauge hitting empty, and the instinct is to treat it as an emergency. It is not one. percentage_used is a vendor estimate of rated endurance consumed, the NVMe specification explicitly allows it to exceed 100, and drives routinely operate past 100% for months or years. It is also monotonic: it never goes back down.

That does not make it noise. It makes it a procurement and planning signal rather than a paging signal, and its rate of change is one of the best leading indicators you have for workload mismatch and premature wear-out.

What percentage_used actually is

The field lives in the NVMe SMART / Health Information log page (Log ID 0x02) as percent_used. The spec definition: a vendor-specific estimate of the percentage of NVM subsystem life used, based on actual usage and the manufacturer’s prediction of NVM life. Three properties follow from that definition and matter operationally:

• It is an estimate, not a measurement. The vendor’s firmware decides how to map observed wear onto a percentage, and different vendors calculate it differently. Two drives from different vendors at the same host write volume can report very different numbers.
• It is monotonic. It behaves like a fuse counter: it only increases. A drive at 87% will never report 86%. There is no recovery, so any alert on the absolute value will eventually fire on every drive that lives long enough.
• It is updated infrequently. The spec allows updates as infrequently as once per power-on hour. Polling it every second gives you a stale number almost all of the time; trend it over days and weeks, not minutes.

The value anchors to the drive’s rated endurance, expressed as TBW (total bytes written) or DWPD (drive writes per day). At 100%, the drive has consumed its rated endurance. Rated endurance is where the vendor’s warranty and reliability projections stop applying, not where the NAND stops working. Enterprise TLC drives are typically rated at 1-10 DWPD and often have real margin past that. Consumer QLC drives may be rated at 100-300 TBW total, which a write-heavy production workload can burn through in under a year.

A useful cross-check is data_units_written from the same SMART log. Multiply by 512,000 bytes (1000 x 512, not 1024) to get host-visible bytes written, and compare against the drive’s rated TBW. Two caveats: data units count host writes, not NAND writes, so write amplification inside the FTL is invisible, and the vendor’s estimate may use assumptions you cannot see. If percentage_used says 100% but your computed host writes are well under rated TBW, the vendor is counting something you are not, such as internal write amplification or a conservative endurance model. Treat the drive’s number as authoritative for warranty purposes and yours as a sanity check.

Why values above 100% are not a failure

The NVMe specification explicitly permits values greater than 100, and caps the field at 255 (percentages above 254 are reported as 255). A value of 100 means the estimated endurance has been consumed; it does not indicate an NVM subsystem failure. This is spec-valid behavior, not a firmware quirk.

Operators routinely see enterprise drives at 110%, 150%, 200% and beyond with zero media errors and full available spare. Some drives operate at 200-300% with no errors; others start failing at 95%. The number is an estimate, not a cliff. What 100% actually changes:

• The drive is outside its rated reliability envelope. Vendor failure-rate projections no longer apply.
• Warranty and support claims tied to endurance are typically void past this point.
• The probability of media errors and spare-block exhaustion rises, but on a timeline of weeks to months, not minutes.

The actual failure signals (media errors, reliability degradation, available spare exhaustion, read-only mode) have their own SMART fields and their own critical warning bits. percentage_used is the early warning that lets you buy the replacement before those fire.

Reading the field

# Read the endurance estimate and its companion fields
nvme smart-log /dev/nvme0 | grep -E "percentage_used|available_spare|media_errors|data_units_written|critical_warning"

# JSON output for scripting
nvme smart-log /dev/nvme0 -o json | jq '{percent_used, avail_spare, spare_thresh, media_errors, data_units_written}'

Both commands are read-only SMART log queries, safe to run on a production device. Note that available_spare and percentage_used track different things: spare capacity is the pool of replacement NAND blocks, while percentage used is the endurance estimate. You can have 100% used with 100% spare, or 50% used with declining spare (which points at a NAND quality problem rather than normal wear). Check both.

Alerting thresholds that reflect reality

The number alone is never page-worthy. The severity ladder that works in production:

Two rules behind this design:

1. Never page on the number alone. A drive at 105% with zero media errors and stable available spare is a scheduling problem, not a 3 a.m. problem. Paging on the absolute value trains the team to ignore the alert, and since the counter is monotonic, the alert will never clear.
2. Page when wear crosses into active failure. Past rated endurance, any new media_errors increment or I/O error in the kernel log means the drive is failing in the region where the vendor no longer guarantees anything. That combination (high percentage used plus active errors) is the page condition. Reliability degradation shows up independently as critical warning bit 2, and read-only mode as bit 3; those page on their own merits regardless of the percentage. See NVMe critical_warning is nonzero for the per-bit severity breakdown.

flowchart TD
  A[percentage_used check] --> B{Value?}
  B -->|< 80%| C[Track trend only]
  B -->|80-90%| D[PLAN: start procurement]
  B -->|90-100%| E[TICKET: schedule replacement]
  B -->|> 100%| F{New media or I/O errors?}
  F -->|No| G[TICKET: replace soon, monitor weekly]
  F -->|Yes| H[PAGE: replace immediately, verify backups]
  A -.-> I{Rate > 1%/week?}
  I -->|Yes| J[Investigate workload: swap, logging, write amplification]

Rate of change matters more than the number

A drive at 45% used after four years is boring. A drive at 45% after four months will hit 100% before its first birthday, and the absolute value gives you no hint of that. Compute the slope: a sustained increase of more than about 1% per week suggests write amplification problems or a workload that does not match the drive class. Common causes:

• Swap on NVMe. Paging under memory pressure generates small random writes that wear NAND fast.
• Excessive logging or journaling. Verbose application logs, filesystem journal modes that double-write, and chatty WAL configurations all amplify writes.
• No TRIM/discard. Without it, the FTL treats the device as full, garbage collection runs at maximum pressure, and write amplification climbs.
• Drive class mismatch. A consumer QLC drive rated at a few hundred TBW sitting under a database workload is the wrong hardware. No configuration change fixes that; plan the swap to enterprise TLC.

Runway estimation: days to rated end of life = (100 - current percentage) / daily percentage increase. Apply a safety factor (2x is reasonable) in the final 10%, because spare-block consumption accelerates non-linearly near end of life and the estimate itself is coarse.

What to check when it crosses 100%

When a drive goes past 100%, run one correlation pass before deciding urgency:

• Media errors. nvme smart-log field media_errors. Any rate of increase above zero past 100% is the page condition. A static low count on an old drive is often tolerable.
• Available spare. Compare available_spare against spare_thresh. Declining spare past 100% used means the wear is now consuming the safety margin. See NVMe available spare below threshold.
• Critical warning bitmask. Bit 0 (spare below threshold), bit 2 (reliability degraded), bit 3 (read-only). Any of these set changes the response from “schedule” to “act now”.
• Data units written vs. rated TBW. Confirms whether the vendor estimate roughly matches host-visible writes, or whether internal amplification is doing the damage.
• Kernel log. dmesg | grep -i nvme for I/O errors or timeouts that SMART counters have not yet reflected. See blk_update_request: I/O error, dev nvme0n1.

If all of those are clean, the drive goes on the replacement schedule with weekly trend checks, not the incident bridge.

Signals to correlate

How Netdata helps

• Netdata collects percentage_used as the nvme.device_estimated_endurance_perc chart, so the monotonic climb is visible as a trend rather than a number someone checks twice a year.
• The rate of change is derivable directly from the chart history, which is what makes the “> 1% per week” workload-mismatch check possible without manual SMART polling.
• Endurance sits next to nvme.device_available_spare_perc and nvme.device_media_errors_rate on the same device dashboard, so the “100% plus errors” page condition is a single-screen correlation, not three separate CLI sessions.
• The nvme.device_critical_warnings_state chart breaks the critical warning bitmask into per-bit dimensions, so spare-below-threshold, reliability-degraded, and read-only states are distinguishable at a glance when endurance runs out.
• Historical retention lets you back-calculate when the wear rate changed, which usually lines up with the deploy that introduced the chatty logger or the swap configuration.

Related guides

• NVMe available spare below threshold: critical warning bit 0 and end-of-life wear
• NVMe critical_warning is nonzero: decoding the SMART critical warning bitmask
• NVMe drive in read-only mode: critical warning bit 3 and rejected writes
• blk_update_request: I/O error, dev nvme0n1: reading NVMe I/O errors in the kernel log
• NVMe monitoring checklist: the signals every production SSD needs
• How NVMe actually works in production: a mental model for operators