$ guides / smartctl-disk-monitoring / smartctl-firmware-version-change ▌

Operations Guides

Unexpected drive firmware or identity change: counter resets and identity drift

A cumulative SMART counter decreased between polls. Power-On Hours went backward. The drive’s model, serial number, or firmware version changed since the last snapshot. SMART counters are designed to be monotonically increasing. Identity fields are set at the factory. When either changes without a documented reason, determine whether this is routine maintenance, a firmware update, or something requiring forensic review.

The severity depends on context. A counter reset correlated with a ticketed drive replacement is informational. A firmware version change with no change management record is a TICKET. A serial number change on a drive that was never touched is a security event.

What this means

SMART cumulative counters (Reallocated Sector Count, Power-On Hours, UDMA CRC Error Count, error log entries) should never decrease. Drive identity fields (model, serial number, firmware version) should remain stable unless the drive is physically replaced or its firmware is updated.

When either type of value changes unexpectedly, one of the following has occurred:

Physical swap. A new drive in the same slot has a different serial number, lower Power-On Hours, and zeroed counters. Most common legitimate cause.
Firmware update. The firmware version string changes. Some vendors reset certain counters during an update, though behavior is vendor-specific and not standardized.
Sanitize or reset cleared error history. Some drives support sanitize operations that clear error logs and may reset counters. Identity stays the same, but the history is gone.
Firmware manipulation. Counterfeit drives can have SMART data reflashed to show a different model or reset counters for resale. smartctl cannot detect this directly.
Firmware bug. Known firmware bugs have caused Power-On Hours to report values offset by hundreds of thousands of hours, or counters to behave erratically.

The key diagnostic principle: if the change is not explained by a documented operation, investigate before trusting any subsequent SMART data from that drive.

flowchart TD
    A["Counter decreased or identity changed"] --> B{"Change management record?"}
    B -->|Yes| C["Classify as INFO"]
    B -->|No| D{"What changed?"}
    D -->|Serial or model| E["Physical swap without documentation"]
    D -->|Firmware version only| F["Undocumented firmware update"]
    D -->|Counter reset, identity same| G{"GLTSD bit set?"}
    G -->|Yes| H["Counters not saved to NV storage"]
    G -->|No| I["Sanitize or firmware manipulation"]
    E --> J["TICKET: forensic review"]
    F --> J
    I --> J
    H --> K["TICKET: fix GLTSD or accept"]
    C --> L["Update baseline"]

Common causes

Cause	What it looks like	First thing to check
Physical drive swap (legitimate)	Serial changed, Power-On Hours much lower, all counters near zero	Change management ticket for this slot or device
Physical drive swap (undocumented)	Same as above but no maintenance record	Physical inspection, dmesg for hot-plug events
Firmware update	Firmware version string changed, some counters may reset	Whether a firmware update was scheduled or run
Manufacturer sanitize/reset	Error log empty, some counters zeroed, identity unchanged	Whether a SANITIZE or secure erase command was issued
GLTSD bit set	Counters reset to zero after power cycle, identity unchanged	`smartctl -a` output for GLTSD warning
Counterfeit or refurbished drive	Counters suspiciously clean, physical labels mismatch smartctl output	Compare physical drive labels to `smartctl -i`
Firmware bug	Impossible values, such as Power-On Hours offset by ~894,794 hours on affected Intel SSDs	Known firmware bugs for this drive model
Self-test log lifetime wrap	LifeTime(hours) field in self-test log wraps	Compare against Power-On Hours attribute ID 9

Quick checks

# Current drive identity
smartctl -i /dev/sdX | grep -iE "Model|Serial|Firmware"

# Power-On Hours (check for impossible values)
smartctl -A /dev/sdX | grep -i "Power_On_Hours"

# Diff current SMART snapshot against a stored baseline
smartctl -A /dev/sdX > /tmp/smart_current.txt
diff /tmp/smart_previous.txt /tmp/smart_current.txt

# GLTSD warning (counters not saved to non-volatile storage)
smartctl -a /dev/sdX 2>&1 | grep -i "GLTSD"

# Self-test log for lifetime hour wrapping
smartctl -l selftest /dev/sdX

# Kernel log for hot-plug or device removal events
# Use journalctl -k on systemd hosts if dmesg buffer has rotated
dmesg | grep -iE "sdX|hotplug|remove|rescan" | tail -30

# NVMe firmware version and update slot info
smartctl -x /dev/nvme0n1 | grep -i "firmware"

How to diagnose it

Capture the current identity and attribute snapshot. Run smartctl -i /dev/sdX and smartctl -A /dev/sdX and save both. These are your baseline going forward.
Compare against your last known-good snapshot. Without a previous snapshot, you cannot determine what changed. If you are not storing periodic SMART snapshots per drive, start now.
Check change management records. Was a drive replacement or firmware update scheduled for this slot? If yes, verify that the new identity and firmware version match the intended hardware and release.
Check kernel logs for hot-plug events. dmesg (or journalctl -k) shows SCSI device removal and addition events with timestamps. Correlate these against maintenance windows.
Verify the GLTSD bit. If only counters reset (not identity), check whether GLTSD (Global Logging Target Save Disable) is set. When set, the drive does not save error counters, power-up hours, and other data to non-volatile storage, so they reset to zero on every power cycle. smartctl -a will warn if GLTSD is set. Some manufacturers ship drives with this bit enabled by default.
Check for known firmware bugs. Some firmware versions produce impossible SMART values. For example, certain Intel SSD firmware versions (330 firmware 300i, 520 firmware 400i) report Power-On Hours with an offset of approximately 894,794 hours. Cross-reference the drive model and firmware version against the smartmontools drive database and vendor advisories.
Compare physical labels to smartctl output. If you suspect a counterfeit or refurbished drive, compare the model and serial number printed on the physical label against smartctl -i. Counterfeit drives can have firmware reflashed to report a different model or reset counters entirely. smartctl cannot detect this, but physical verification can.
Document findings and classify severity. If the change correlates with documented maintenance, classify as INFO and update your baseline. If unexpected, classify as TICKET and escalate.

Metrics and signals to monitor

Signal	Why it matters	Warning sign
Drive serial number	Set at manufacture, should never change on the same physical drive	Any change without a documented swap
Firmware version	Tracks firmware updates and catches known-buggy versions	Change without a change management record
Power-On Hours (ID 9)	Cumulative since manufacture, should only increase	Any decrease or impossible value
Reallocated Sector Count (ID 5)	Cumulative lifetime count, should never decrease	Reset to zero on a drive that previously had nonzero values
UDMA CRC Error Count (ID 199)	Cumulative interface error count, carries with the drive	Reset to zero
Error log entry count	Historical error record; clearing erases forensic history	Count drops or log is suddenly empty
GLTSD bit status	Indicates whether counters persist across power cycles	Bit is set on a drive where historical tracking matters
Self-test log LifeTime(hours)	Can wrap on some drives	Values that appear to reset but are actually a field limitation

Fixes

Documented drive swap

If the identity change is explained by a legitimate drive replacement:

Update the monitoring baseline for this slot with the new drive’s identity and starting counter values.
Verify the replacement drive’s SMART health before putting it into production.
Record any factory-shipped nonzero counters (some SSDs ship with a small number of reallocated sectors from manufacturing QA) as the new baseline.

Undocumented drive swap

If the serial number changed but there is no maintenance record:

Physically inspect the slot to confirm which drive is present.
Check dmesg timestamps for when the swap occurred.
Determine who performed the swap and why it was not documented.
In high-security environments, treat as a potential security incident.

Firmware update (documented)

If the firmware version changed as part of a planned update:

Verify the new firmware version is the intended release.
Re-baseline all SMART counters, since some vendors reset counters during firmware updates.
Check whether the update addressed a known bug. Cross-reference against vendor advisories and the smartmontools drive database.
Monitor the drive closely for the first 48 hours, as firmware changes can introduce new behavior.

Firmware update (undocumented)

If the firmware version changed with no change management record:

This is a TICKET. Determine who updated the firmware, what tool was used, and whether the update came from the manufacturer or a third party.
Some manufacturers have released fixed firmware without changing the version number, making it difficult to distinguish versions by string alone.
In supply-chain security contexts, an undocumented firmware change could indicate compromise.

GLTSD bit causing counter resets

If the GLTSD bit is set and counters reset on every power cycle:

Determine whether this is the manufacturer’s default for this drive model.
If historical tracking matters, clear the GLTSD bit if the drive supports it. This allows counters to persist in non-volatile storage across power cycles.
If the bit cannot be cleared, treat every power-on as a fresh baseline. Historical tracking across reboots is not possible.

Counterfeit or tampered drive

If physical inspection confirms the drive does not match its reported identity:

The drive may be counterfeit or refurbished with tampered SMART data.
Remove the drive from production. Do not trust SMART data from a drive whose identity has been manipulated.
In procurement contexts, trace the supply chain for this drive batch.

Firmware bug producing impossible values

If Power-On Hours or other counters show impossible values:

Cross-reference the drive model and firmware version against known bugs in the smartmontools drive database.
Use the -F (firmwarebug) option with the appropriate type if smartmontools has a workaround. For example, -F swapid corrects byte-swapped ATA identify strings returned by buggy drivers. Multiple -F options can be combined.
If the bug makes the counter unreliable, track drive health using other signals (self-test results, error log entries, reallocated sector trends).

Prevention

Snapshot SMART data at deployment. Capture smartctl -a output for every drive at installation. This establishes the baseline for all future change detection.
Track drive identity in your inventory system. Record model, serial number, and firmware version per slot. Alert on any change not tied to a change management ticket.
Maintain a firmware version registry. Track approved firmware per drive model. When a vendor advisory is published, you need to know immediately which drives are affected.
Store periodic SMART snapshots. Without historical snapshots you cannot determine when a change occurred or what the previous values were.
Integrate change management with monitoring. When a swap or firmware update is scheduled, mark it in the monitoring system so identity and counter changes are expected.
Use smartd for continuous tracking. The smartd daemon logs SMART attribute changes over time, providing an audit trail that catches silent counter resets.

How Netdata helps

Netdata’s smartctl integration collects drive identity (model, serial, firmware version) and SMART attributes on each collection cycle, making counter resets and identity drift immediately visible.

Identity tracking across cycles surfaces firmware version changes and serial number swaps the moment they occur, not during the next manual audit.
Rate-of-change anomaly detection flags when a monotonically increasing counter suddenly decreases, which is the core signal of a reset or tampering event.
Historical metric retention provides the before-and-after snapshots needed to determine what changed and when, without requiring manual baseline storage.
Correlation with host-level signals (dmesg events, I/O errors, device presence changes) helps classify whether an identity change correlates with a physical hot-plug event or occurred silently.
ML-based anomaly detection can flag the pattern of multiple counters resetting simultaneously, which is characteristic of a sanitize operation or drive swap rather than a single-attribute glitch.