$ guides / postfix / postfix-no-space-left-on-device ▌

Operations Guides

Postfix No space left on device: disk and inode exhaustion halting the queue

Postfix logs show fatal: ... No space left on device or warning: not enough free space in mail queue. You run df -h and the queue filesystem has free space. Mail acceptance stalls, delivery stops, and the queue grows.

The most common cause is inode exhaustion. Postfix creates one file per queued message across its queue subdirectories: maildrop, incoming, active, deferred, bounce, defer, hold, and corrupt. A mail storm, a backscatter loop, or months of accumulated deferred messages can exhaust millions of inodes while disk space barely moves. df -h looks fine. df -i shows 100%.

The failure mode is cliff-edge. One moment file creation works; the next, every create() and write() syscall fails. Queue files cannot be written, the PID file at /var/spool/postfix/pid/master.pid cannot be updated, and lock files cannot be acquired. Different operations fail at different points depending on which file creation hits the wall first, so acceptance and delivery stall unpredictably.

What this means

No space left on device is the string representation of ENOSPC. The kernel returns it when either disk blocks or inodes are exhausted. Postfix passes the error through.

Postfix also has an internal free-space check in the SMTP server. With queue_minfree at its default of 0, smtpd rejects MAIL FROM with 452 4.3.1 Insufficient system storage and logs not enough free space in mail queue: N bytes < 1.5*message size limit when free space drops below 1.5 times message_size_limit (default: 10240000 bytes). A non-zero queue_minfree raises that floor to the configured value.

This check only gates new inbound mail; it does not stop writes for messages already accepted. The actual ENOSPC can still arrive later from the kernel when cleanup or qmgr attempts a write() the filesystem refuses.

flowchart TD
    A["Postfix: No space left on device"] --> B["Run df -i on queue path"]
    B --> C{"Inodes at 100%?"}
    C -->|Yes| D["Inode exhaustion:
bulk-delete queue files"]
    C -->|No| E["Run df -h on queue path"]
    E --> F{"Disk at 95%+?"}
    F -->|Yes| G["Disk exhaustion:
free space or enlarge"]
    F -->|No| H["Check ext4 reserved blocks
or XFS inode limit"]

Common causes

Cause	What it looks like	First check
Inode exhaustion	`df -h` shows free space, `df -i` shows near 100%	`df -i /var/spool/postfix`
Disk space exhaustion	`df -h` shows 95%+ used on queue partition	`df -h /var/spool/postfix`
ext4 reserved blocks	`df` reports available space but Postfix cannot write	`tune2fs -l <device> \| grep -i reserved`
queue_minfree or message_size_limit mismatch	452 “Insufficient system storage” rejections plus warning despite ample space and inodes	`postconf -h queue_minfree message_size_limit`
XFS inode allocation limit	`df -i` shows free inodes but writes still fail with ENOSPC	`xfs_db -r -c 'sb 0' -c 'p imax_pct' <device>`

Quick checks

# Check inodes first - this is the #1 trap
df -i /var/spool/postfix

# Check disk space
df -h /var/spool/postfix

# Confirm queue directory location
postconf -h queue_directory

# Check Postfix free-space parameters
postconf -h queue_minfree message_size_limit

# Find ENOSPC errors in mail logs
grep -i 'No space left on device' /var/log/mail.log | tail -20

# Find Postfix advisory warnings about free space
grep -i 'not enough free space' /var/log/mail.log | tail -20

# Count files per queue subdirectory
for d in maildrop incoming active deferred bounce defer hold corrupt; do
  count=$(find /var/spool/postfix/$d -type f 2>/dev/null | wc -l)
  echo "$d: $count"
done

# Check ext4 reserved blocks on the queue filesystem device
tune2fs -l "$(df /var/spool/postfix --output=source | tail -1)" 2>/dev/null | grep -i reserved

How to diagnose it

Run df -i on the queue filesystem. The single most important step. If inode usage is at or near 100%, you have found the cause. Critical threshold: fewer than 10,000 free inodes. Warning: less than 5% free. Healthy: more than 20% free.
Run df -h on the same filesystem. If blocks are also exhausted, you have a combined problem. If blocks are fine but inodes are exhausted, focus on queue file cleanup.
Count files per queue subdirectory. Deferred and bounce directories are the usual suspects. Postfix uses hash subdirectories (for example, deferred/A/queueid, one level deep with the default hash_queue_depth = 1) to keep individual directories small, but the total file count across all subdirectories is what matters for inode consumption. Very large queues also slow down postsuper operations because it processes each message individually.
Check for ext4 reserved blocks. ext4 reserves 5% of blocks for root by default. Postfix runs as a non-root user and cannot write to reserved blocks. On a 20 GB partition, that is 1 GB Postfix cannot touch. The Available column in df subtracts this; Use% can hit 100% before blocks are physically exhausted.
Verify queue_minfree and message_size_limit. If message_size_limit is high (for example, 100 MB), Postfix requires 150 MB free before accepting mail. On small partitions, this rejects all mail even when the filesystem is mostly empty.
Consider XFS-specific behavior. XFS dynamically allocates inodes but has an allocation ceiling (maxpct, default 25% of filesystem space). Inspect it with xfs_db -r -c 'sb 0' -c 'p imax_pct' <device>, and raise it on a mounted filesystem with xfs_growfs -m <percent>. If df -i shows free inodes but writes still fail with ENOSPC, XFS may be refusing to allocate new inodes beyond this internal limit.
Check container mount semantics. If Postfix runs in a container with a bind-mounted queue directory, df inside the container may report the host filesystem’s free space, but the container runtime’s storage driver (overlay2 and others) may impose its own limits. Run df from both inside and outside.

Metrics and signals to monitor

Signal	Why it matters	Warning sign
Queue filesystem inode usage	One file per queued message; exhaustion halts everything	>90% used, or <10,000 free inodes
Queue filesystem disk usage	Large messages or accumulated attachments exhaust blocks	>90% used
Deferred queue file count	Growth is the leading indicator of inode pressure	Sustained growth >100 files/hour
Total queue file count	Direct proxy for inode consumption	>50% of filesystem inode capacity
Postfix ENOSPC log rate	Confirms the failure is active	Any occurrence
Postfix free-space warnings	Precursor to hard failure	Sustained warnings over minutes
ext4 reserved block percentage	Reduces effective writable space for non-root Postfix	>5% on large partitions

Fixes

Inode exhaustion: clear queue files

The immediate goal is to free inodes. Identify the worst offenders first:

for d in maildrop incoming active deferred bounce defer hold; do
  count=$(find /var/spool/postfix/$d -type f 2>/dev/null | wc -l)
  echo "$d: $count"
done

Option 1: Postfix-native cleanup (safe, slow). postsuper -d ALL deletes all queued messages and respects queue consistency. On large queues, this is slow because it processes each message individually.

# Destructive: removes ALL pending mail from every queue
postsuper -d ALL

Option 2: Selective deletion (safer). Delete only messages matching specific patterns, such as MAILER-DAEMON bounces. Hold all mail first to stop delivery attempts and reduce I/O:

# Hold all mail to stop delivery attempts
postsuper -h ALL

# Delete MAILER-DAEMON messages
# postqueue -p appends * (active) or ! (hold) to queue IDs; strip them
postqueue -p | awk '/MAILER-DAEMON/ {gsub(/[*!]/, "", $1); print $1}' | postsuper -d -

Option 3: Direct filesystem deletion (fastest, highest risk). When postsuper is too slow and the situation is critical, deleting files directly bypasses Postfix queue management. This may leave orphaned references in queue index files. Stop Postfix first if possible, use ionice to avoid I/O starvation, and run postsuper -s afterward to repair structure.

# Destructive: bypasses Postfix queue management entirely.
# Stop Postfix first if the situation allows.
postfix stop

# Delete files directly from the worst-affected directories
ionice -c 3 find /var/spool/postfix/maildrop -type f -delete
ionice -c 3 find /var/spool/postfix/deferred -type f -delete
ionice -c 3 find /var/spool/postfix/bounce -type f -delete

# Repair queue structure after direct deletion
postsuper -s

# Restart Postfix
postfix start

Disk space exhaustion: free blocks

If df -h confirms actual disk space exhaustion, find what is consuming storage:

# Largest space consumers in the queue tree
du -sh /var/spool/postfix/* | sort -rh | head -10

# Check if logs share the partition
df -h /var/log /var/spool/postfix

Delete large messages by queue ID with postsuper -d <queue_id>, or free space from other consumers on the same partition.

ext4 reserved blocks: reduce or resize

If df shows available space but Postfix cannot write, ext4 reserved blocks may be the cause. The default 5% reservation wastes significant space on large partitions.

# Check current reserved block percentage
tune2fs -l "$(df /var/spool/postfix --output=source | tail -1)" | grep -i 'reserved'

# Reduce reserved blocks to 1% (takes effect immediately, no unmount required)
tune2fs -m 1 "$(df /var/spool/postfix --output=source | tail -1)"

Reducing reserved blocks below 5% on root or /var filesystems leaves less room for emergency root operations. Consider whether other services on the same filesystem depend on reserved blocks.

queue_minfree or message_size_limit mismatch

If df -i and df -h both show healthy usage but Postfix still refuses mail, check whether message_size_limit is too high for the partition:

# Check current values
postconf -h message_size_limit queue_minfree

# Calculate the 1.5x threshold
limit=$(postconf -h message_size_limit)
echo "1.5x threshold: $((limit * 3 / 2)) bytes ($((limit * 3 / 2 / 1024 / 1024)) MB)"

# Reduce message_size_limit if too high for the partition
postconf -e 'message_size_limit = 25600000'
postfix reload

# Or disable queue_minfree if set too aggressively
postconf -e 'queue_minfree = 0'
postfix reload

Prevention

Monitor inodes, not just disk space. Alert on df -i for the queue filesystem. Critical: fewer than 10,000 free inodes or above 98% used. Warning: above 90% used. Target: above 20% free.
Monitor deferred queue growth rate, not just size. A static queue of 5,000 messages may be normal. A queue growing at 1,000 messages/hour is a crisis. Alert on sustained growth over 100 files/hour.
Size the queue filesystem for inode headroom. If the queue directory is on a small /var partition, consider a dedicated filesystem with high inode density.
Consider XFS for queue directories. XFS dynamically allocates inodes, reducing the risk of exhaustion. ext4 with default inode density is more vulnerable.
Clean deferred queues regularly. Messages that will never deliver (invalid recipients, dead domains) consume inodes indefinitely. Schedule periodic cleanup of old deferred entries.
Watch for backscatter storms. These generate massive numbers of bounce and double-bounce files in minutes. Early detection of bounce rate spikes prevents queue-induced inode exhaustion.
Validate container storage limits. If Postfix runs in a container, verify the storage driver and volume mount do not impose limits lower than the host filesystem reports.

How Netdata helps

Netdata collects filesystem and Postfix metrics per second, so you see inode exhaustion developing in real time rather than at the next cron check.

Disk and inode utilization per mount point. Both bytes and inodes are collected per second. The anomaly detector flags unusual inode consumption rates before static thresholds fire.
Postfix queue depth by subdirectory. The Postfix collector breaks down queue counts by incoming, active, deferred, maildrop, bounce, and hold. Sustained growth in any subdirectory correlates directly with inode consumption.
Correlation between queue growth and filesystem exhaustion. When deferred queue depth rises alongside declining free inodes, the causal relationship is visible on a single dashboard.
Bounce rate and delivery velocity. A spike in bounces or a collapse in delivery rate often precedes queue-induced inode exhaustion. Netdata surfaces these alongside filesystem metrics.
Disk I/O saturation. Postfix queue operations are fsync-heavy. When I/O wait rises alongside queue growth, cleanup operations themselves become slow, extending the recovery window.