MySQL InnoDB: Database page corruption — detection and recovery

Your error log shows InnoDB: Database page corruption on disk or a failed file read. MySQL may abort the connection, mark the tablespace read-only, or crash. Page corruption is a storage-layer failure, not a query bug: a 16KB InnoDB page failed checksum validation.

The critical distinction is between secondary index corruption, which is rebuildable without data loss, and clustered index corruption, which affects the table data itself. Detect the failure, classify the damage, extract what you can, and rebuild safely.

What this means

InnoDB stores data and indexes in fixed-size pages, typically 16KB. Each page carries a checksum. When InnoDB reads a page into the buffer pool, it validates the checksum. If the value does not match, InnoDB logs InnoDB: Database page corruption and may raise an assertion failure. The server refuses to use the page because it cannot distinguish a failed storage read, a torn write, or silent bit rot.

Once detected, the affected tablespace is unsafe to write to until you intervene. Do not restart the server and hope the error goes away. Your next steps depend on whether the damage is in an index structure or in the row data itself.

Common causes

Quick checks

Run these read-only checks to scope the damage before making changes.

# Confirm the exact error string and timestamp
grep "InnoDB: Database page corruption" /var/log/mysql/error.log

-- Review InnoDB engine status for assertion details
SHOW ENGINE INNODB STATUS\G

-- Check a specific table online (safe but expensive on large tables)
CHECK TABLE schema_name.table_name;

# Check for underlying hardware I/O errors
dmesg | grep -i "error"

# Check SSD health for media wear indicators
smartctl -a /dev/sdX

How to diagnose it

1. Read the error log carefully. Look for InnoDB: Database page corruption, the tablespace ID, the page offset, and any adjacent assertion failure. This tells you which table is affected and whether the server aborted a thread or the entire instance.

2. Map the tablespace to a table. Query the data dictionary using the tablespace ID from the error log:

   SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES WHERE SPACE = <tablespace_id>;
   

3. Run CHECK TABLE. Execute it against the affected table. If it returns error or warning, note whether the message references an index record or the primary key and data rows. CHECK TABLE does not catch every kind of corruption, so a result of OK does not fully clear the table if the error log persists.

4. Validate offline with innochecksum. Shut down MySQL and run innochecksum against the .ibd file or the system tablespace. This recomputes per-page checksums and reports mismatches without requiring a running server. Do not run it against files the server has open.

   # Run against a file-per-table .ibd after shutdown
   innochecksum /var/lib/mysql/db_name/table_name.ibd
   

5. Check hardware logs. Correlate the timestamp of the first corruption error with storage controller, SSD SMART, and memory ECC logs. If hardware errors coincide with the first InnoDB error, the root cause is outside MySQL.

6. Classify the damage. If CHECK TABLE or innochecksum points to a secondary index, you can rebuild. If the clustered index is affected, you are looking at data loss and must rely on logical dumps or backups.

Metrics and signals to monitor

Fixes

If the server is running and the table is accessible

If CHECK TABLE reports secondary index corruption but the clustered index is intact:

-- Rebuild all indexes for the table (online in MySQL 8.0)
ALTER TABLE t ENGINE=InnoDB;

This rebuilds the secondary indexes from the data rows. It preserves all data but consumes I/O and temporarily requires extra disk space. Run it during a low-traffic window.

If the server crashes or refuses to start

Add innodb_force_recovery to your MySQL configuration, then restart the instance. Start with the lowest value that allows startup.

[mysqld]
innodb_force_recovery = 1

innodb_force_recovery is for extraction only. Values above 0 put InnoDB in a degraded read-only mode. Do not run production traffic against it and do not write to the database while it is active. Higher values increasingly bypass safety checks to allow startup; avoid them unless there is no other way to start the server.

Once the server starts:

# Dump the affected table logically; this may fail if the dump hits the corrupted region
mysqldump --single-transaction db_name t > t_recovery.sql

Shut down, remove innodb_force_recovery from the configuration, and restore the table from your most recent clean backup. If the instance will not start normally because crash recovery still hits the corrupted page, restore to a fresh data directory or keep force recovery only long enough to extract data. Never write to the database while innodb_force_recovery is active.

If the clustered index is corrupted

When the primary key data pages are damaged, ALTER TABLE ... ENGINE=InnoDB will not help because the row data itself is unreadable.

1. Start with innodb_force_recovery at the minimum level required.
2. Attempt a logical dump of the table.
3. If the dump fails due to the corrupted region, restore directly from backup.
4. Accept that any data written since the last clean backup may be lost.

Prevention

• Keep the doublewrite buffer enabled. It is ON by default and protects against torn-page writes by staging pages before writing them to data files.
• Schedule periodic CHECK TABLE or offline innochecksum validation. Catching corruption early prevents it from spreading through replication or backups.
• Monitor storage hardware health. Track SSD wear levels and memory ECC error rates at the OS level.
• Validate backups before you need them. A backup taken after corruption began is itself corrupted. Periodically restore backups to a staging instance and run CHECK TABLE.

How Netdata helps

Netdata correlates MySQL error log strings such as InnoDB: Database page corruption with system-level disk latency and memory health metrics. Sudden Uptime resets and increased crash recovery duration highlight corruption-induced restarts. Spikes in Innodb_data_reads and disk latency help confirm storage-layer faults rather than query bugs.

Related guides

• How MySQL actually works in production: a mental model for operators: /guides/mysql/how-mysql-works-in-production/
• MySQL Aborted_connects and Aborted_clients climbing: diagnosis: /guides/mysql/mysql-aborted-connections/
• MySQL adaptive hash index latch contention: high CPU, low throughput: /guides/mysql/mysql-adaptive-hash-index-latch-contention/
• MySQL binary logs filling the disk: expiry, lagging replicas, and purge: /guides/mysql/mysql-binary-log-disk-full/
• MySQL InnoDB buffer pool hit ratio collapse: the cliff edge: /guides/mysql/mysql-buffer-pool-hit-ratio-collapse/
• MySQL slow after restart: buffer pool warm-up and the cold cache: /guides/mysql/mysql-buffer-pool-not-warming-up/
• MySQL innodb_buffer_pool_size tuning: 60-80% of RAM and when that breaks: /guides/mysql/mysql-buffer-pool-sizing/
• MySQL Innodb_buffer_pool_wait_free > 0: buffer pool memory pressure: /guides/mysql/mysql-buffer-pool-wait-free/
• MySQL InnoDB checkpoint age: the redo log capacity signal nobody watches: /guides/mysql/mysql-checkpoint-age-monitoring/
• MySQL connection exhaustion: detection, diagnosis, and prevention: /guides/mysql/mysql-connection-exhaustion/
• MySQL stuck in InnoDB crash recovery: why startup hangs after an unclean shutdown: /guides/mysql/mysql-crash-recovery-slow-startup/
• MySQL innodb_deadlock_detect=OFF: when deadlock detection becomes the bottleneck: /guides/mysql/mysql-deadlock-detect-off-high-concurrency/