RabbitMQ queue with zero consumers: zombie queues that grow until they OOM the node

A queue with zero consumers and a non-zero publish rate is a memory leak with a routing key. Every published message lands in the queue, nothing drains it, and the backlog grows until something stops it: a TTL, a queue length limit, a manual purge, or the node’s memory alarm. If none of the first three happen, you find out when publishers across the whole cluster get blocked.

This failure is invisible in global metrics. Cluster-wide consumer count stays healthy because your other queues have plenty of consumers. Cluster-wide queue depth grows, but if you run batch workloads, growth alone does not page anyone. The only signal that isolates this failure early is the per-queue consumer count.

This guide covers how to confirm a zombie queue, how to distinguish a real consumer outage from an intentional accumulating queue, and how to drain or bound the damage without making things worse.

What this means

When a consumer subscribes with basic.consume, RabbitMQ registers it against the queue and starts delivering messages up to the prefetch limit. When the last consumer disconnects, unsubscribes, or its channel dies, the queue’s consumer count drops to zero. The queue itself does not care. It keeps accepting every message routed to it.

What happens next depends on queue type and version:

  • Classic queues (pre-3.12 behavior): messages sit in RAM until the paging ratio (default 0.5 of the memory watermark) forces them to disk. Deep zero-consumer queues eat memory first, disk second.
  • Lazy and modern classic queues (3.12+): messages go to disk much more aggressively. High depth no longer threatens memory directly; disk free space and disk I/O take the hit instead.
  • Quorum queues: every message is appended to a Raft WAL on each member. A zero-consumer quorum queue grows WAL segments on multiple nodes at once, and disk consumption can outpace snapshot compaction. This is the fastest path to a disk alarm.

Either way, the endgame is a resource alarm. Memory crosses vm_memory_high_watermark (default 0.4 of RAM in 3.x, 0.6 in 4.x) or free disk drops below disk_free_limit (default 50MB), and the alarm blocks all publishers on all nodes. Consumers still work, but nothing new comes in. One forgotten queue can halt ingestion for the entire cluster.

flowchart TD
  A[Consumer app crashes or is undeployed] --> B[Queue consumers = 0]
  B --> C[messages_ready grows with every publish]
  C --> D{Paging / WAL writes to disk}
  D --> E[Memory ratio climbs toward watermark]
  D --> F[Disk free drops toward disk_free_limit]
  E --> G[Memory alarm: all publishers blocked cluster-wide]
  F --> H[Disk alarm: all publishers blocked cluster-wide]
  G --> I[Ingestion halt or OOM kill]
  H --> I

Common causes

CauseWhat it looks likeFirst thing to check
Consumer deployment failureConsumers dropped to zero at a specific time; often coincides with a deploy, crash loop, or scaled-to-zero replica setDeployment and pod/instance history for the consumer app
Queue declared but consumer never deployedNew queue, publish rate > 0 since creation, consumer count has always been 0When the queue was created and by which application
Intentional batch or scheduled-consumer queueDepth grows and drains on a schedule (hourly, nightly); consumer count is 0 between runsHistorical depth pattern: does it ever drain?
Zombie connections hiding a dead consumerConsumer count > 0 but ack rate is 0; looks subscribed but nothing processeslist_consumers and per-queue ack rate
Routing change sends traffic to the wrong queueA previously quiet queue suddenly has publish rate and zero consumersExchange bindings and recent topology changes
Temporary queue that lost its auto-delete triggerNon-auto-delete durable queue left behind by a retired applicationQueue arguments (x-expires, auto-delete flag) and last consumer activity

The distinction that matters operationally: a batch queue that accumulates and drains on a schedule is normal, and alerting on it is noise. A queue whose consumer count silently went to zero and stays there is an incident in progress. Depth history tells you which one you have.

Quick checks

All of these are read-only.

# The core signal: per-queue consumers next to depth
rabbitmqctl list_queues name consumers messages_ready messages_unacknowledged

Any line with consumers = 0 and a positive, growing messages_ready is a zombie queue.

# Same data via the management API, including publish rate per queue
curl -s -u guest:guest http://localhost:15672/api/queues | \
  jq '.[] | select(.consumers == 0 and .messages_ready > 0) |
      {name, vhost, messages_ready, consumers,
       publish_rate: .message_stats.publish_details.rate}'

The publish rate matters: a zero-consumer queue with publish_rate = 0 is a leftover, not a leak. A zero-consumer queue with a positive publish rate is actively filling.

# Check how much runway the backlog is buying you
curl -s -u guest:guest http://localhost:15672/api/nodes | \
  jq '.[] | {name, mem_used, mem_limit,
             ratio: (.mem_used / .mem_limit),
             disk_free, disk_free_limit, mem_alarm, disk_free_alarm}'

# See which exchanges feed the queue, and which policies bound it
rabbitmqctl list_bindings
rabbitmqctl list_policies

# If consumer count is non-zero but you suspect zombies, list actual consumers
rabbitmqctl list_consumers

How to diagnose it

  1. Enumerate zero-consumer queues. Run the list_queues command above and look at messages_ready. The queue with the most messages and zero consumers is your incident.

  2. Check whether it is still being fed. Pull the per-queue publish rate from the management API. If publish rate is zero, the queue is inert: a cleanup task, not an emergency. If publish rate is positive, every minute of delay costs memory or disk.

  3. Decide: outage or by design? Look at depth history. A queue that grows for 23 hours and drains at 01:00 every night is a scheduled batch consumer; the fix is to exclude it from zero-consumer alerting, not to page anyone. A queue whose consumers vanished at deploy time and never came back is an outage. If you have no per-queue history, that is the monitoring gap to close after the incident.

  4. Check for zombie consumers. If the consumer count is non-zero but ack rate is zero, you have a different pattern: consumers connected but not processing. Causes include deadlocked application threads and dead client processes whose TCP connections are still open. Verify consumer-side application health, and check consumer_utilisation for the queue: a value near 0 with a growing backlog confirms consumers are attached but ineffective.

  5. Quantify the blast radius. Compute mem_used / mem_limit and disk_free / disk_free_limit from the node API. Above 0.7 memory ratio, paging is likely active and the alarm is approaching. Below 3x the disk limit, disk is your binding constraint. Estimate runway from the growth rate: (mem_limit - mem_used) / memory_growth_rate or (disk_free - disk_free_limit) / disk_consumption_rate.

  6. Identify the owning application. Bindings tell you which exchange feeds the queue. The queue name, vhost, and the user that declared it usually identify the owning team. If the consumer application was scaled to zero, crashed, or is stuck in a crash loop, restoring it is the fix.

Metrics and signals to monitor

SignalWhy it mattersWarning sign
Per-queue consumer countThe only signal that isolates this failure; global consumer count hides a single queue at zero0 on a queue that should have consumers
Per-queue messages_ready trendTells you how fast the zombie is filling and whether it ever drainsSustained positive growth with consumers = 0
Per-queue publish rateDistinguishes an active leak from an inert leftover queue> 0 on a zero-consumer queue
mem_used / mem_limitRunway to the memory alarm; paging starts at the paging ratio (default 0.5 of the watermark)> 0.7 sustained
disk_free / disk_free_limitRunway to the disk alarm, the usual endgame for lazy and quorum queues< 3.0, and < 1GB absolute
Queue paging countersConfirm the backlog has reached disk; a leading indicator before the alarmAppearing and growing on the affected queue
consumer_utilisationDistinguishes “no consumers” from “consumers attached but stuck”Near 0 with backlog growing and consumers > 0
Ack rate per queueGround truth that processing happens at all0 while consumers > 0 (zombie connections)

For alerting, the composite condition is consumers == 0 AND messages_ready > 0 (or publish rate > 0), sustained long enough to cover scheduled batch windows, with a per-queue allowlist for intentional accumulating queues. If you export via the Prometheus plugin, the equivalent expression is rabbitmq_queue_consumers == 0 and rabbitmq_queue_messages > 0. Either way, the alert must be per-queue; a global consumer gauge will not fire.

Fixes

Restore the consumers

If the consumer application crashed or was undeployed, bringing it back is the correct fix. Once consumers reconnect, the queue drains at the rate the consumers can process. Watch messages_ready trend down and confirm ack rate tracks delivery rate. If the backlog is huge, scale consumers up temporarily, but watch the consumer-side downstream dependencies (database, APIs) that the drain surge will hit.

Purge the queue (destructive)

If the messages are expendable (telemetry, cache-warming jobs, anything idempotent or replayable) and the alarm is close, purging is the fastest relief:

# DESTRUCTIVE: deletes all messages in the queue without delivering them
rabbitmqctl purge_queue <queue_name>

This discards every message permanently. Confirm with the owning team that the messages are replayable or worthless before running it. Do not purge a queue whose owner you cannot identify during a live incident without sign-off.

Delete the queue (destructive)

If the queue is an orphan (no consumers, no owner, no value), deleting it removes the leak entirely. Deleting a queue drops its messages. Verify bindings first so you know what stops receiving copies, and get sign-off from the owning team if one exists.

Do not restart the broker

Restarting does not fix a zero-consumer queue. Durable queues reload from disk on boot, the messages come back, and the memory spike during queue recovery can make things worse. Do not treat a broker restart as a first response to a memory-wall incident.

Bound the queue with policy

For queues that are intentionally accumulating, add guardrails so a consumer outage cannot take the node down:

  • Message TTL (x-message-ttl via policy): messages expire and are dropped or dead-lettered after a fixed age.
  • Queue length limit (max-length or max-length-bytes): oldest messages are dropped (or dead-lettered, if a DLX is configured) once the limit is hit. Prefer max-length-bytes for bounding memory and disk directly.
  • Dead-letter exchange: route expired or overflowed messages somewhere inspectable instead of silently discarding them.

Every bound has a tradeoff: TTL and length limits lose data by design. That is acceptable for telemetry and unacceptable for financial transactions. Set bounds per queue according to what the data is worth.

Prevention

  • Alert per-queue, not globally. The zero-consumer alert (consumers == 0 and messages or publish rate > 0) must evaluate per queue, with a documented allowlist for batch and scheduled-consumer queues. Global consumer count will never catch this.
  • Use auto-delete and exclusive queues for ephemeral work. Queues that exist to serve one consumer should die with that consumer instead of lingering.
  • Set queue expiry for forgotten queues. x-expires deletes a queue after it has been unused (no consumers, no redeclaration) for a period. It is a garbage collector for abandoned topology.
  • Bound every accumulating queue. Any queue designed to hold a backlog should have max-length-bytes or a message TTL and a dead-letter exchange. Unbounded durable queues are how one forgotten consumer becomes a cluster-wide publisher halt.
  • Set heartbeats so dead consumers actually disconnect. Clients killed without a clean shutdown (spot instance termination, OOM kill, kill -9) leave connections that look alive until TCP times out. Heartbeats let the broker detect the dead peer, close the channel, and requeue unacked messages. If you terminate consumer instances aggressively, this is not optional.
  • Watch the paging ratio as the early warning. Memory alarms are cliff-edge. Paging to disk (default at 0.5 of the watermark) is the last graduated signal before the cliff, and disk I/O plus queue paging counters are how you see it.
  • Raise disk_free_limit from the 50MB default. If your zombie-queue endgame is disk exhaustion, the default limit gives you almost no runway. See the related guides on the disk alarm.

How Netdata helps

  • Netdata collects per-queue consumer count, messages_ready, and messages_unacknowledged from the management API, so a queue drifting to zero consumers shows up on the queue’s own charts rather than being averaged away in cluster totals.
  • Correlating per-queue publish rate against consumer count on the same dashboard separates “actively filling zombie” from “inert leftover” without manual API queries.
  • Node-level mem_used / mem_limit and disk free charts next to queue depth charts let you read runway visually: which queue is growing, and how much room is left before the alarm.
  • Paging activity and disk I/O on the data partition surface the graduated warning that precedes the cliff-edge memory alarm.
  • Per-queue alerting lets you page on zero consumers for critical queues while suppressing the alert for known batch queues, instead of choosing between noise and blindness.