$ guides / postfix / postfix-deferred-queue-growing ▌

Operations Guides

Postfix deferred queue growing: why mail piles up and how to drain it

Sustained growth in /var/spool/postfix/deferred/ means delivery failures are outpacing retry successes. The queue is accumulating messages faster than the queue manager can drain them.

The problem compounds in two ways. First, every message that fails retry stays in the deferred queue while fresh mail continues to enter the system. Second, the queue manager uses exponential backoff, so older deferred messages may not be retried for up to maximal_backoff_time (default 4000s, roughly 66 minutes). Even after you fix the root cause, the queue takes hours to drain because most messages are in their backoff cool-off period.

The critical monitoring mistake is alerting on absolute queue size rather than growth rate. A deferred queue of 5,000 messages that is shrinking is healthy. The same 5,000 growing at 1,000 messages per hour is a crisis onset. Rate-of-change is the signal that matters.

What this means

The deferred queue holds messages that failed delivery with temporary errors (4xx SMTP responses). The queue manager (qmgr) retries these messages on a schedule governed by exponential backoff.

Each failed delivery increases the time until the next retry attempt, clamped between minimal_backoff_time (default 300s) and maximal_backoff_time (default 4000s). The queue is scanned for eligible messages every queue_run_delay (default 300s). Messages that remain undeliverable past maximal_queue_lifetime (default 5d) are bounced back to the sender as permanently failed.

The fairness strategy matters for understanding why growth compounds. When the active queue has room, qmgr interleaves messages from the incoming and deferred queues. Deferred retries compete with fresh mail for active queue slots. As the deferred queue grows, a larger fraction of active queue capacity goes to retries, slowing throughput for new mail. If the active queue also saturates (default limit: 20,000 messages via qmgr_message_active_limit), the system enters gridlock: new mail cannot enter active delivery, and the deferred queue grows faster.

flowchart TD
    A["Delivery fails with 4xx"] --> B["Message enters deferred/"]
    B --> C["Backoff timer: 300s to 4000s"]
    C --> D["Retry on next queue scan"]
    D --> E{"Succeed?"}
    E -- No --> B
    E -- Yes --> F["Delivered"]
    B --> G["Deferred queue grows"]
    G --> H["Retries compete with\nnew mail for active slots"]
    H --> I["New mail also defers\nunder congestion"]
    I --> G

Common causes

Cause	What it looks like	First thing to check
Slow destination monopolization	One or few domains dominate deferred entries with “connection timed out” or 4xx rate-limit codes; active queue near limit	Group deferred mail by destination domain
DNS resolver failure	All destinations affected equally; deferrals mention “Host not found” or “Name service error”; active queue low	Test DNS resolution from the MTA host
Content filter or Milter backpressure	Incoming queue grows while active stays small; filter processes slow, unresponsive, or OOM-killed	Filter process health and response time
IP reputation degradation	Major destinations (Gmail, Outlook) return 4xx rate-limit or policy deferrals	External blocklist and reputation checks
TLS certificate or policy failure	Deferrals for specific destinations mention TLS errors or certificate verification failures	Certificate expiration and TLS negotiation logs

Quick checks

All commands below are read-only and safe to run on a production system.

# Current deferred message count (recurses hash subdirectories)
find /var/spool/postfix/deferred -type f | wc -l

# Deferred events in the current minute (instantaneous rate indicator)
# Take two queue-depth samples minutes apart for a reliable growth rate.
# Syslog path varies by distro: /var/log/mail.log (Debian) or /var/log/maillog (RHEL).
grep "$(date '+%b %e %H:%M')" /var/log/mail.log | grep -c 'status=deferred'

# Top deferred destinations by domain
grep 'status=deferred' /var/log/mail.log | awk -F'to=<|>,' '{print $2}' | cut -d@ -f2 | sort | uniq -c | sort -rn | head

# Age distribution of deferred queue
qshape deferred

# Inode usage on queue filesystem (not the same as disk space)
df -i /var/spool/postfix

# Current backoff and lifetime settings
postconf -h minimal_backoff_time maximal_backoff_time queue_run_delay maximal_queue_lifetime

# Active queue size vs configured limit
find /var/spool/postfix/active -type f | wc -l
postconf -h qmgr_message_active_limit

# Queue summary (total count and size)
postqueue -p | tail -1

How to diagnose it

Confirm the queue is actually growing. Take two find /var/spool/postfix/deferred -type f | wc -l measurements a few minutes apart. A single snapshot tells you nothing about trajectory.
Check the deferral reasons. Scan recent mail logs for status=deferred entries and group by reason. The SMTP response text in the deferral tells you the root cause: “Host not found” means DNS, “connection timed out” means network or destination, rate-limit language means throttling.
Group deferred mail by destination domain. If one or few domains dominate, you have a destination-specific problem (monopolization, reputation, greylisting). If all domains are affected, suspect infrastructure-wide issues (DNS, network, TLS).
Check active queue saturation. If the active queue is near qmgr_message_active_limit (default 20,000), the system is in gridlock. The active queue being full means new deliveries cannot be scheduled regardless of destination health. See active queue saturation.
Test DNS independently. Run dig or host queries from the MTA host for domains showing deferrals. If DNS lookups fail or time out, the resolver is the problem, not the destinations.
Check inode pressure. Run df -i /var/spool/postfix. If inodes are above 90%, the queue growth is approaching a hard cliff where Postfix cannot create new queue files. The deferred queue stores each message as a separate file across hash subdirectories (deferred/0/, deferred/1/, and so on), so a large queue can exhaust inodes while disk space looks fine.
Estimate runway. Divide free inodes by the growth rate (files per hour) to estimate how long until exhaustion. This tells you whether you have hours or minutes to fix the root cause before Postfix can no longer accept or queue mail.
Check content filter health if applicable. If your deployment uses a content_filter or Milter, test it directly on its listening port. Filter backpressure typically shows up as incoming queue growth first, but can cascade into deferrals if the filter times out and messages are deferred rather than filtered.

Metrics and signals to monitor

Signal	Why it matters	Warning sign
Deferred queue growth rate	Rate-of-change, not absolute size, indicates accumulation vs drainage	Sustained positive growth over 4+ hours
Deferred queue absolute size	Indicates operational burden and disk/inode risk	Above 10,000 messages; above 50,000 indicates systemic failure
Active queue utilization	Saturation blocks all delivery scheduling	Above 80% of `qmgr_message_active_limit`
Inode usage on queue filesystem	Postfix creates many small files; exhaustion is a cliff-edge failure	Above 90% or fewer than 10,000 free inodes
Deferral reason codes	Identifies whether the problem is DNS, destination, TLS, or policy	Clustering around specific 4xx codes or “Host not found”
Injection vs delivery rate	Divergence confirms the system is accumulating mail	Injection consistently exceeding delivery over 5-minute windows
DNS resolver latency and failure rate	DNS is a single point of failure for Postfix delivery	Elevated latency or SERVFAIL responses

Fixes

When the root cause is fixed: drain safely

Do not flush the deferred queue unless you expect that most of its content has recently become deliverable. If the root cause is still active, flushing generates more failed delivery attempts and makes things worse.

If the root cause is resolved (relayhost back up, DNS fixed, destination accepting mail again):

# Force an immediate queue run
postqueue -f

This schedules immediate delivery attempts for all deferred messages. It does not reset backoff timers, so messages that have not reached their scheduled retry time will be skipped.

Warning: If a host with lots of deferred mail has been down for some time, the entire deferred queue may reach its retry time simultaneously. Flushing in this scenario can cause a thundering herd: the active queue fills completely as thousands of messages become eligible at once, and deliveries to other destinations stall.

Requeuing to reset backoff timers

When messages have been deferred long enough that their backoff timers are near 4000s and you want them retried immediately:

# Requeue all messages with fresh timestamps (I/O intensive on large queues)
postsuper -r ALL

This moves all messages into the maildrop queue for reprocessing by cleanup and re-injection into incoming. Messages get new timestamps, resetting their backoff schedule entirely. This is more aggressive than postqueue -f and generates significant I/O.

For individual messages:

# Schedule immediate delivery of one message
postqueue -i QUEUEID

Holding mail for a specific problem destination

If one destination is monopolizing the queue but others are healthy, selectively hold that destination’s mail:

# Find queue IDs for messages to a problem domain, then hold them.
# Verify the grep matches before running postsuper.
find /var/spool/postfix/deferred -type f -exec grep -l '@problem.domain' {} \; \
  | sed 's|.*/||' | postsuper -h -

The sed extracts the queue ID (filename) from the full path. postsuper -h - reads queue IDs from stdin. Held messages move to the hold queue and are excluded from delivery scheduling.

Release them later with postsuper -H ALL (preserves original timestamps) or postsuper -r ALL (gives new timestamps). The distinction matters for old messages: postsuper -H preserves the original timestamp, so messages near maximal_queue_lifetime may expire and bounce before their next retry. For messages that have been deferred for days, use postsuper -r to give them a fresh start.

Deleting undeliverable mail

When mail is definitively undeliverable (backscatter, spam, mail to recipients that should never have been accepted):

# Delete all mail in the deferred queue (DESTRUCTIVE, irreversible)
postsuper -d ALL deferred

Verify the queue contents before running. Consider holding mail first with postsuper -h ALL if you are unsure whether deletion is appropriate. Deleting large numbers of files frees inodes slowly; if the filesystem is near exhaustion, operations may remain sluggish until the kernel reclaims the freed inodes.

Reducing destination concurrency for a throttling destination

If a destination is rate-limiting your deliveries, reducing concurrency can improve overall throughput by preventing the destination from blocking you entirely:

# Reduce SMTP destination concurrency, then reload
postconf -e 'smtp_destination_concurrency_limit=2'
postfix reload

This sets the limit globally for all smtp-transport destinations. For per-destination tuning, use master.cf overrides or transport maps. The queue manager does not pick up main.cf changes automatically because qmgr is a persistent process. Run postfix reload after any configuration change.

Prevention

Alert on deferred queue growth rate, not absolute size. Any sustained positive growth over 4 hours is abnormal. Set page-level alerts for growth above 1,000 messages per hour with no plateau.
Monitor inodes explicitly on the queue filesystem. Postfix creates many small files. Inode exhaustion manifests as “No space left on device” even when disk blocks are available. Alert below 10,000 free inodes.
Monitor DNS resolver health independently. Postfix depends entirely on DNS for MX lookups. Resolver failure looks like “slow mail” or “deferred to everyone” and is easy to misdiagnose as a destination problem.
Monitor content filter response times, not just process liveness. Filter slowdown causes queue backup long before the filter process crashes. Incoming queue growth with a stable active queue is the tell.
Track active queue utilization against qmgr_message_active_limit. A full active queue means gridlock even when destinations are healthy.
Know your queue manipulation commands before you need them. Understanding when to use postqueue -f versus postsuper -r ALL, and the difference between postsuper -r (new timestamps) and postsuper -H (old timestamps preserved), saves critical minutes during an incident.
Set appropriate maximal_queue_lifetime for your workload. The default 5 days means deferred mail can sit for nearly a week. For transactional mail, shorter lifetimes reduce queue bloat from messages that will never deliver.

How Netdata helps

Per-second queue metrics let you compute deferred queue growth rate in real time. Rate-of-change alerting on the deferred queue catches the crisis onset that absolute-threshold alerts miss.
Inode monitoring alongside disk space surfaces the most common surprise during queue growth: a filesystem with free disk blocks but no inodes. Correlating inode usage with deferred queue growth gives you an automatic runway estimate.
Correlation across logs and metrics helps distinguish DNS failure (all destinations affected equally), destination monopolization (one domain dominating), and filter backpressure (incoming queue growing before deferred).
Active queue saturation detection catches the gridlock state where new mail cannot enter delivery because active slots are consumed by retries to a slow destination.
Anomaly detection on queue growth rates learns normal daily patterns (queues that grow during business hours and drain overnight) and surfaces abnormal growth that static thresholds would miss.