NVMe sanitize, format, and secure erase: expected events versus red flags

Sanitize, Format NVM, and secure erase are the three ways an NVMe controller destroys data on its own media. They are admin commands executed inside controller firmware, and once started they are largely outside the host’s control. In a healthy fleet they appear during commissioning (occasionally) and during decommission or repurpose. Any other occurrence is a security or integrity event.

The operational problem is that these operations look alarming even when they are working correctly. Progress counters sit at 0% or 50% for hours. The controller briefly shows a resetting state. Estimated completion times report meaningless values. Operators page on all of this, or worse, they try to “fix” a running sanitize and discover it cannot be stopped.

What these commands actually do

All three are issued on the admin queue and executed by controller firmware. The host’s role after submission is polling a log page.

Sanitize (NVMe 1.3 and later) puts the controller into a dedicated sanitize state machine. It supports several actions, selected with -a / --sanact in nvme-cli: block erase (0x02), overwrite (0x03), cryptographic erase (0x04), and exit failure mode (0x01). Sanitize operates on the whole controller, so it must be issued against the character device (/dev/nvme0), not a namespace block device (/dev/nvme0n1). Once started, a sanitize is uninterruptible: it survives power cycles and reboots, resuming automatically until it completes. There is no abort.

Format NVM reformats one or all namespaces, optionally changing the LBA size and metadata format. Its destructiveness is controlled by the Secure Erase Settings field (--ses in nvme-cli): 0 means no secure erase (metadata and mapping only), 1 is User Data Erase, and 2 is Cryptographic Erase. With --ses 0 the format is fast and not a secure erase at all; with --ses 1 or 2 it is a destructive erase. Format accepts either the character device or a namespace block device. nvme-cli enforces a 10-second confirmation delay on destructive formats unless --force is given.

Secure erase is the umbrella term operators use for the erase paths above (SES 1/2 format, or sanitize block/crypto erase). It is not a separate NVMe command.

The important property for monitoring: sanitize progress and status are reported through the Sanitize Status log page, polled with nvme sanitize-log /dev/nvme0. Format has no progress log; it either returns or the controller becomes busy until it finishes.

The sanitize lifecycle

stateDiagram-v2
    [*] --> Idle: never sanitized / complete
    Idle --> InProgress: sanitize start (block/overwrite/crypto)
    InProgress --> InProgress: power cycle resumes automatically
    InProgress --> Complete: SSTAT = success
    InProgress --> Failed: SSTAT = failed
    Failed --> InProgress: exit failure mode + retry
    Complete --> Idle: subsequent sanitize cycles

Two fields in the sanitize log tell you where the controller is:

SPROG (sanitize progress): a 16-bit fraction of completion. 65535 (0xFFFF) means 100%. Anything below is SPROG / 65535 * 100 percent. On drives with NVMe 2.1 Post Sanitize Media Verification, SPROG covers the processing states separately from post-verification deallocation, so the interpretation differs by drive generation.
SSTAT (sanitize status): the low three bits carry the state as independent flags: bit 0 = most recent sanitize completed successfully, bit 1 = in progress, bit 2 = failed. A reported value of 0x101 is a successful completion (bit 0 set, plus higher status bits).

Expected events: what a healthy operation looks like

These behaviors generate false pages. All of them are normal during a legitimate sanitize or destructive format.

Progress stuck at 0% or 50% for hours. SPROG reporting is coarse. Drives update it in large steps, and some sit near zero for most of the operation before jumping. Block erase on a 512GB consumer drive can take 2-3 hours with SPROG near 1% for most of that time. Coarse reporting is not a hang.
Estimated completion times of 4294967295. This is 0xFFFFFFFF, the “no time period reported” value. The drive is declining to estimate, not reporting a 136-year runtime.
Brief resetting controller state. /sys/class/nvme/nvmeX/state may show resetting transiently during firmware-activated sanitize. This resolves in seconds. The controller state alert should only fire if a non-live state persists for more than 30 seconds; check nvme sanitize-log before escalating.
The operation surviving a reboot. Sanitize is uninterruptible by design. If the host power-cycles mid-operation, the sanitize resumes after the controller reinitializes. This is the spec working as intended, not corruption.
Inability to abort. nvme reset, admin-passthru abort, and even sanitize -a 0x01 (exit failure mode) will not stop a sanitize in progress; the controller returns “Sanitize In Progress”. The only way out is completion. Do not build runbooks that promise cancellation.
Namespace I/O errors during the operation. The controller is busy erasing. Host I/O to the affected namespaces will stall or fail. On a drive you are intentionally erasing, the filesystem errors in dmesg are downstream noise.

The unifying check for all of the above: poll nvme sanitize-log /dev/nvme0. If SSTAT shows in progress and SPROG changes between samples spaced hours apart, the operation is alive. If you expected the sanitize, nothing here is page-worthy.

Red flags: when this is a security or integrity event

These commands are destructive, they require admin queue access (effectively root), and on bare metal nothing intercepts them. An unexpected one is an incident.

Any sanitize or format on a production drive outside a maintenance window. This is either malicious destruction, accidental targeting (wrong device node in a decommission script), or automation with a bad device mapping. Severity: INFO if expected, TICKET if unexpected or failed. An actively running unauthorized erase on a live volume is a page, not a ticket, because data loss is in progress.
A sanitize you cannot attribute. These commands leave almost no host-side trace by default. Check dmesg | grep -i "format\|sanitize" | grep nvme, shell history, configuration management runs, and decommission automation. If nothing explains it, you have a privileged-access problem, not a storage problem.
SSTAT showing a failed sanitize on a drive holding data. A failed sanitize (bit 2 set) means the drive may be partially erased and is in sanitize failure mode. The data state is indeterminate. Do not return the drive to service; the supported recovery is exit failure mode followed by a successful sanitize, then retire the drive.
Format with --ses 1 or --ses 2 where you expected a metadata-only format. --ses 0 reformats mappings without erasing user data; SES 1 and 2 erase. Automation that silently defaults to a secure erase turns a repartitioning task into data destruction. Audit scripts and playbooks for explicit --ses flags.
Format or sanitize commands arriving through a USB-to-NVMe adapter. nvme-cli expects native PCIe. Translation layers can mangle admin commands, and failures here can leave the drive in a bad state. Do not run destructive commands over USB bridges.
A second sanitize starting while one is believed complete. Re-check SSTAT. If the in-progress bit is set and no one issued it, investigate as above.

Verifying state during and after

# Check sanitize status and progress (safe, read-only)
nvme sanitize-log /dev/nvme0

# Controller state - expect live, tolerate brief resetting
cat /sys/class/nvme/nvme0/state

# Look for host-side evidence of who issued what
dmesg | grep -i "format\|sanitize" | grep nvme
journalctl -k --no-pager | grep -i nvme | grep -i "format\|sanitize\|reset"

# Post-operation: confirm completion and drive health baseline
nvme smart-log /dev/nvme0 | grep -i "critical_warning\|media_errors\|unsafe_shutdown"

When issuing these commands deliberately, pin the procedure: unmount all namespaces on the target, confirm the device node twice (character device for sanitize, /dev/nvme0, never /dev/nvme0n1), and record the start time and expected duration so the on-call can distinguish a long operation from a stuck one. A format against a mounted filesystem will fail or warn; treat that as a safety feature, not an inconvenience.

Signals to watch in production

Signal	Why it matters	Warning sign
`nvme sanitize-log` SSTAT/SPROG	Only source of sanitize state and progress	In-progress sanitize with no change record; SSTAT failed bit
`/sys/class/nvme/nvmeX/state`	Distinguishes transient sanitize resets from stuck resets	Non-`live` persisting > 30s with no sanitize in log
dmesg format/sanitize lines	Host-side attribution for destructive commands	Any entry outside a maintenance window
`nvme.device_critical_warnings_state`	Post-operation health; bit 3 read-only means the drive is done	Any nonzero value after an erase completes
`nvme.device_media_errors_rate`	Media errors after a failed sanitize indicate real damage	Rate > 0 on a drive that just failed a sanitize
Firmware slot log (`nvme fw-log`)	Sanitize interacts with firmware state; unexpected firmware change is its own incident	Slot change concurrent with an unexplained sanitize

How Netdata helps

Netdata collects controller state and SMART health continuously, so a drive entering resetting during a sanitize is visible in context: you can see it recover in seconds instead of paging on a single sample.
The nvme.device_critical_warnings_state chart breaks the critical warning bitmask into per-bit dimensions, so post-sanitize health verification is a glance at one chart rather than manual bit math.
Media errors and error log entry rates (nvme.device_media_errors_rate, nvme.device_error_log_entries_rate) establish the baseline you compare against after a failed or completed erase.
Unsafe shutdown counts matter here because sanitize survives power cycles: correlating nvme.device_unsafe_shutdowns_count with a sanitize timeline explains why an operation took longer than the drive’s estimate.
Retaining per-second history across the maintenance window gives you the before/after record for the change ticket and for any later integrity review.