When your content filter (Amavis, Rspamd, or a commercial scanner) slows down or stalls, mail accumulates in Postfix’s incoming queue while the active queue stays small. This is the reverse of the more familiar queue gridlock pattern where a slow destination fills the active queue, and it confuses operators who expect active to be large whenever mail is stuck.

The signature is specific: incoming queue growing, active queue flat or small, deferred queue low, and the filter port (commonly 127.0.0.1:10024 for Amavis) refusing connections or timing out. SMTP clients may experience slow DATA-phase responses. Upstream senders start retrying, which amplifies load on a system that is already falling behind.

What this means

The content_filter parameter in main.cf tells Postfix to redirect all mail through an external scanner before final delivery. A typical Amavis integration:

content_filter = smtp-amavis:[127.0.0.1]:10024

A corresponding smtp-amavis transport in master.cf defines how many concurrent filter deliveries Postfix will attempt. When the filter is healthy, mail flows through the full pipeline: cleanup writes to incoming, the queue manager (qmgr) promotes messages to active, the smtp-amavis transport delivers to port 10024, the filter processes and reinjects via a second smtpd listener (commonly port 10025), and final delivery proceeds.

When the filter slows, the smtp-amavis transport processes fill up to their maxproc limit and stall waiting for responses. The qmgr stops promoting new messages from incoming to active because there is no available delivery capacity for the filter transport. Mail accumulates in incoming.

The active queue stays small because the bottleneck is not general delivery capacity but the specific content_filter transport. The qmgr is behaving correctly: it does not overfill active when the downstream consumer cannot keep up.

flowchart TD
    A[smtpd accepts] --> B[incoming queue]
    B --> C[qmgr promotes to active]
    C --> D[smtp-amavis transport]
    D --> E{filter port 10024}
    E -->|healthy| F[filter reinjects via 10025]
    F --> G[final delivery]
    E -->|slow or refused| H[transport stalls at maxproc]
    H --> I[incoming grows, active stays small]

Note: Rspamd is typically integrated as a milter (before-queue filter) rather than as a content_filter. The milter backpressure pattern looks different: smtpd processes hang during the SMTP transaction itself, and you see connection acceptance slowdowns rather than incoming queue growth. If your Rspamd integration uses a milter and you see the symptoms in this article, check whether you also have a separate content_filter configured.

Common causes

CauseWhat it looks likeFirst thing to check
Filter database unreachableFilter processes alive but slow; filter logs show Redis, ClamAV, or SQL connection errors; response latency climbingTest the filter’s backend connectivity directly from the filter host
Filter OOM on large messageKernel OOM killer logged for filter process; RSS was climbing before the kill; may correlate with a large attachment entering the systemdmesg | grep -i oom and filter memory limits
Filter config reload or restartQueue growth starts at the same timestamp as filter service restart; brief total unavailability of the filter portFilter service logs and restart timestamps in systemd journal
Filter port unreachable from Postfix hostConnection refused or timeout to the filter port; filter may be healthy on its own host but unreachable from Postfix (container networking, firewall, remote partition)nc -zv <filter_host> <filter_port> from the Postfix host

Quick checks

Run these read-only checks to confirm the diagnosis. None of them modify the queue or configuration.

# Compare queue sizes - the signature of filter backpressure.
# Postfix stores queue files in hash subdirectories, so use find -type f.
echo "incoming: $(find /var/spool/postfix/incoming -type f | wc -l)"
echo "active:   $(find /var/spool/postfix/active -type f | wc -l)"
echo "deferred: $(find /var/spool/postfix/deferred -type f | wc -l)"

# Show the configured content_filter and the transport's maxproc
postconf -h content_filter
postconf -M | grep smtp-amavis

# Test the filter port directly
nc -zv 127.0.0.1 10024

# Count recent connection refusals to the filter port
tail -10000 /var/log/mail.log | grep '10024' | grep -c 'Connection refused'

# Check filter process health and memory
systemctl status amavisd 2>/dev/null || ps aux | grep -E 'amavis|rspamd' | grep -v grep

# Check for kernel OOM kills of filter processes
dmesg | grep -i 'out of memory\|oom' | tail -10

# Check disk and inodes on the queue partition
df -h /var/spool/postfix
df -i /var/spool/postfix

If amavisd-nanny is available on your system, it provides a direct health check of Amavis worker processes:

amavisd-nanny 2>&1 | head -20

How to diagnose it

  1. Confirm the queue signature. If incoming is large, active is small, and deferred is low, you are looking at filter backpressure, not a delivery problem. If active is also large, you may instead have a queue gridlock pattern where a slow destination is consuming active queue slots.

  2. Verify the content_filter configuration. Run postconf -h content_filter to confirm the filter is set. The value must reference a transport name that exists in master.cf. A mismatch (for example, content_filter = amavis: when the transport is named smtp-amavis) causes delivery failures with “mail transport unavailable” errors rather than slow buildup, but it is worth ruling out.

  3. Test the filter port. Run nc -zv 127.0.0.1 10024 from the Postfix host. Connection refused means the filter process is not listening. Timeout means the filter is alive but too loaded to accept new connections. Both produce the same queue behavior from Postfix’s perspective.

  4. Check filter logs for backend failures. Look for Redis connection errors (Rspamd), ClamAV socket errors (Amavis), or SQL timeouts (rule storage). These typically appear before the filter stops accepting connections entirely.

  5. Correlate the queue growth start time with filter events. Check journalctl -u amavisd (or the equivalent filter service) for restarts, reloads, or error bursts around the time the incoming queue started growing. A config reload that took longer than expected is a common trigger.

  6. Check the smtp-amavis transport concurrency. Run postconf -M | grep smtp-amavis to see the maxproc column. If it is set to a low value (2 is common), the filter transport can only handle 2 concurrent deliveries. When the filter is slow, those 2 processes stall and no more mail can be promoted from incoming.

  7. Check for I/O contention. A growing incoming queue increases disk write pressure. Under extreme I/O contention, cleanup slows down writing new queue files, which compounds the problem. Check iostat -x 1 5 or vmstat 1 5 for disk saturation.

Metrics and signals to monitor

SignalWhy it mattersWarning sign
Incoming queue sizeFirst indicator of filter backpressure; grows before any other queueSustained growth with active queue flat
Active queue sizeShould be proportional to delivery capacity; stays small in filter backpressureDisproportionately small relative to incoming
Deferred queue sizeShould stay low in pure filter backpressure; if it grows, you may have a compound problemRising along with incoming
Filter port reachabilityDirect test of filter availabilityConnection refused or timeout on port 10024
Filter response latencySlow filter causes smtp-amavis processes to stallp99 above 5 seconds (baseline typically under 1 second)
Filter process count and memoryFilter OOM or exhaustion is a common root causeRSS climbing steadily or process count at limit
smtp-amavis transport utilizationShows whether the filter transport is saturatedAll maxproc processes busy, none idle
Mail flow velocity (injected vs delivered)Confirms delivery has stalled despite continued acceptanceDelivery rate near zero with injection continuing
Queue filesystem inodesIncoming queue growth consumes inodes; secondary failure modeFewer than 10,000 free inodes on the queue partition

Fixes

Emergency bypass: disable content_filter

When the filter is completely unresponsive and the incoming queue is growing without bound, you can bypass the filter to restore mail flow immediately.

# WARNING: This sends all mail unfiltered. Use only when the alternative
# is unbounded queue growth. Know your compliance and security requirements.
postconf -e 'content_filter='
postfix reload

Postfix stops routing mail through the filter. Messages already in incoming begin flowing to active and out to delivery. The filter itself is not affected; it continues running (or not) independently.

To re-enable filtering once the underlying issue is resolved, restore your original content_filter value:

# Replace with your original content_filter setting
postconf -e 'content_filter=smtp-amavis:[127.0.0.1]:10024'
postfix reload

Mail that bypassed the scanner during the outage is delivered unfiltered. This is a deliberate tradeoff: unfiltered mail delivered now versus filtered mail delivered hours late. For most transactional and internal mail systems, delivery timeliness outweighs the filtering gap. For environments with strict content scanning requirements, coordinate with security before bypassing.

Filter database unreachable

If the filter’s backend store (Redis for Rspamd, ClamAV socket for Amavis, SQL for SpamAssassin Bayes) is unreachable:

  1. Verify the backend service is running and accepting connections.
  2. Test connectivity from the filter host, not just from the Postfix host.
  3. Restart the backend if it is hung or unresponsive.
  4. Monitor filter response latency as it recovers. A database that just came back online may have cold caches and respond slowly for several minutes.

If the backend cannot be restored quickly, use the emergency bypass above and schedule filtered delivery for after the repair.

Filter OOM on large message

If the kernel OOM killer is terminating filter processes:

  1. Check dmesg | grep -i oom for the kill events and identify the victim process.
  2. Review the filter’s memory limits: cgroup limits in containers, systemd MemoryMax= (legacy: MemoryLimit=) on bare metal.
  3. Increase the memory allocation if the filter legitimately needs more for your message volume.
  4. Configure the filter to skip scanning for very large messages if your policy allows it. Amavis exposes message size thresholds for skipping checks. Rspamd has configurable message size limits.
  5. Identify the triggering message if possible. A single message with a very large attachment can OOM a filter process that handles smaller messages fine.

Filter reload or restart

If a filter config reload caused temporary unavailability:

  1. Check the filter service logs for reload timestamps.
  2. Verify the filter is now responding: nc -zv 127.0.0.1 10024.
  3. If the filter recovered on its own, the incoming queue should drain naturally. Monitor the drain rate.
  4. A slow drain (incoming shrinking but not emptying) may indicate the filter is still degraded but not fully down. Check response latency to distinguish recovery from persistent slowness.

Prevention

  • Monitor filter response latency, not just process liveness. A filter process that is running but responding in 30 seconds causes the same backpressure as a dead one. Track p99 response time, not just whether the service is up.
  • Alert on incoming queue growth with active queue flat. This specific combination is the earliest reliable signal of filter backpressure. It appears minutes before the deferred queue grows or delivery rate drops.
  • Set queue filesystem inode alerts. Incoming queue growth consumes inodes rapidly because each message is a small file. Monitor df -i alongside df -h.
  • Match filter transport concurrency to actual filter capacity. The smtp-amavis maxproc in master.cf should reflect what the filter can handle without degrading. Too many concurrent deliveries to an overloaded filter makes things worse, not better.
  • Include the filter port in health checks. A simple TCP check on port 10024 catches filter outages before they affect mail flow. Test from the Postfix host, not just locally on the filter host.
  • Rehearse the emergency bypass. When the incoming queue is growing and upstream senders are retrying, operators should not be reading documentation for the first time. Document the exact commands and test them in a staging environment.

How Netdata helps

  • Per-second queue size metrics expose the incoming-versus-active divergence within seconds, before the queue grows large enough to cause disk or inode pressure.
  • Anomaly detection on queue sizes can flag the incoming-growth-with-active-flat pattern even when absolute counts remain below static thresholds.
  • Process resource metrics (RSS, CPU, file descriptors) for filter processes like amavisd or rspamd reveal memory growth or CPU saturation before the filter stops responding.
  • Correlation between Postfix queue depth and filter process health in the same dashboard shortens the diagnostic path: you see the symptom (incoming queue growth) and the cause (filter OOM or high latency) in the same time window.
  • Filesystem inode monitoring on the queue partition gives early warning of the secondary failure mode that follows unbounded incoming queue growth.