RabbitMQ NOT_FOUND - no queue or exchange: publishing to something that does not exist

Your publisher or consumer logs show NOT_FOUND - no exchange 'orders.events' in vhost '/', or the queue variant, NOT_FOUND - no queue 'billing.jobs' in vhost '/'. The RabbitMQ server log has the matching line: operation basic.publish caused a channel exception not_found: no exchange 'orders.events' in vhost '/'. The channel that raised it is now closed, and depending on the client library, your application may be spinning in a declare-fail-reconnect loop.

This is AMQP reply code 404, a channel-level exception. The client tried to use an exchange or queue the broker has no record of: it was never declared, it was deleted, or it deleted itself (auto-delete semantics). The failure is loud, which helps. The harder case to rule out is the quiet cousin: the exchange exists, the publish succeeds, and messages silently vanish because nothing is bound.

This error clusters around deploys. A release that renames an exchange, reorders topology declarations, or changes which service declares what produces a burst of 404s from whichever component starts first or still has the old name in its config.

What this means

In AMQP 0-9-1, exchanges and queues are server-side objects that must exist before they are used. Four operations raise NOT_FOUND against a missing object:

  • basic.publish to an exchange that does not exist
  • basic.consume from a queue that does not exist
  • queue.bind (or exchange.bind) referencing a queue or exchange that does not exist
  • A passive declaration (queue.declare or exchange.declare with passive=true) against a resource that does not exist. Passive declare means “tell me if this exists, do not create it”, and its answer for “no” is closing your channel with 404.

Two failure mechanics matter for diagnosis. First, the channel closure is asynchronous: the offending operation does not fail at the call site; the broker sends a channel.close shortly after. Some client libraries surface this as an exception on the next operation, so the error can appear one statement away from the actual cause. Second, a channel error closes only the channel, but many client libraries respond by tearing down and rebuilding the whole connection. If the application retries the same operation without declaring the missing resource, you get a reconnect loop that shows up as connection churn on the broker.

Do not confuse 404 with its neighbors. 406 PRECONDITION_FAILED means the resource exists but you re-declared it with different properties. 405 RESOURCE_LOCKED means you touched an exclusive queue from a different connection. 403 ACCESS_REFUSED means permissions, not existence. And the case that produces no error at all: publishing to an exchange that exists but has no matching binding, which silently discards non-mandatory messages.

flowchart TD
  A[Channel exception or missing messages] --> B{Broker log shows?}
  B -->|not_found 404| C[Exchange or queue never existed or was deleted]
  B -->|precondition_failed 406| D[Resource exists, declared with mismatched properties]
  B -->|resource_locked 405| E[Exclusive queue accessed from another connection]
  B -->|access_refused 403| F[Permissions problem, not existence]
  B -->|nothing, publish succeeds| G[Silent routing loss: no binding matches]
  C --> H[Check exchange and queue lists, then find who should declare it]
  G --> I[Check bindings and return_unroutable]

Common causes

CauseWhat it looks likeFirst thing to check
Deploy renamed or removed the resource404s start exactly at deploy time; old name in errors, new name in configrabbitmqctl list_exchanges / list_queues for both old and new names
Startup ordering: publisher up before declarer404 storm right after rollout, stops once the consumer service finishes bootingTimestamps: do errors stop shortly after the declaring service starts?
Auto-delete exchange deleted itselfWorked for weeks, broke after the last bound queue disappeared (its consumer disconnected)Was the exchange declared auto-delete? Did a bound queue vanish recently?
Auto-delete or exclusive queue expiredConsumer restart fails with 404 on a queue that “used to be there”Queue durability and exclusivity flags in the declaring code
Passive declare in client setupConsumer fails at startup with 404 even though it “only checks” the queueClient library flags: passive=true in declare calls
Wrong vhostResource exists, client lands in / but queue lives in another vhostVhost in the connection string vs rabbitmqctl list_queues -p <vhost>
Deleted by an operator or cleanup job404s begin outside any deploy windowAudit management API usage and any automation that deletes topology

Quick checks

All read-only. The curl checks assume the management plugin is enabled; the default guest user can only authenticate from localhost.

# Find the exact exception text and which operation triggered it
grep -i "not_found" /var/log/rabbitmq/rabbit@$(hostname).log | tail -20

# Does the exchange exist, and in which vhost?
rabbitmqctl list_exchanges name vhost type durable auto_delete

# Does the queue exist?
rabbitmqctl list_queues name vhost durable auto_delete consumers

# What is bound to the exchange the publisher targets?
rabbitmqctl list_bindings source_name destination_name routing_key

# Check for a reconnect loop caused by the 404 (channel churn climbing)
curl -s -u guest:guest http://localhost:15672/api/overview | jq '.churn_rates | {channel_created, channel_closed, connection_created, connection_closed}'

# Confirm the resource truly is absent via the management API (404 here too, by design)
curl -s -o /dev/null -w "%{http_code}\n" -u guest:guest http://localhost:15672/api/exchanges/%2F/orders.events

Two notes. %2F is the URL-encoded default vhost /; substitute the real vhost name if the client connects elsewhere. And if list_exchanges shows the resource present while the client still gets NOT_FOUND, the mismatch is almost always the vhost or a stale name in client config, not the broker.

How to diagnose it

  1. Capture the exact error. From the broker log, note the operation (basic.publish, basic.consume, queue.declare, queue.bind), the resource name, and the vhost. The operation tells you which side is failing: publish means a publisher problem, consume or passive declare usually means consumer startup.
  2. Verify existence from the broker’s point of view. Run list_exchanges / list_queues for that vhost. If the resource is absent, the question becomes “who was supposed to declare it”. If it is present, the question becomes “why does the client see a different reality”: wrong vhost, wrong name, or (rarely) a cluster node with divergent metadata.
  3. Check for auto-delete semantics. An auto-delete exchange is removed when its last bound queue is unbound or deleted; an auto-delete queue is removed when its last consumer disconnects. The classic cascade: an auto-delete queue’s consumer disconnects, the queue vanishes, the exchange bound only to that queue vanishes too, and every publisher that does not re-declare gets NOT_FOUND on the next publish. Check the declaring code for auto-delete flags and the logs for a consumer disconnect just before the first 404.
  4. Correlate with deploy timing. If 404s began at a rollout, diff the topology declarations between the old and new versions. Renamed exchanges and queues are the usual finding. Also check startup ordering: if the publisher service boots faster than the service that declares the exchange, the publisher 404s until the declarer catches up. Whether the error self-resolves after a few minutes is the tell.
  5. Rule out the silent variant. If the exchange exists and publishes succeed but consumers see nothing, this was never a 404 problem. Check bindings (list_bindings source_name) and the return_unroutable counter. Non-mandatory messages to an exchange with no matching binding are discarded without any error anywhere.
  6. Check client behavior after the error. Because the channel closure is asynchronous, the client stack trace may point at the operation after the one that failed. The operation named in caused a channel exception in the broker log is ground truth.

Metrics and signals to monitor

SignalWhy it mattersWarning sign
not_found lines in broker logThe only direct record of 404s; channel exceptions are not exposed as broker metricsAny occurrence in production outside a deploy window
Channel churn rate (churn_rates.channel_created/closed)A client looping on declare-fail-reconnect produces constant channel churnChannel creation rate sustained above baseline
Connection churn rateSome libraries rebuild the whole connection after a channel exceptionConnection creation spikes correlated with 404 log lines
message_stats.return_unroutableCompanion check for the silent-loss variant (only fires with mandatory=true)Sustained rate > 0
Publish rate vs deliver_get ratePublish healthy, deliver_get zero: messages may be going nowhereSustained divergence with flat queue depth
Consumer count per queueA queue that disappeared takes its consumers to zero; a recreated queue starts with zero consumers until clients reconnectExpected consumers missing on a critical queue
Queue churn (queue_declared/created/deleted)Deletes of queues that should be permanent show up herequeue_deleted events for durable application queues

Fixes

Restore the missing resource now

Re-declare the exchange or queue with the exact properties the applications expect (type, durability, arguments), from the management UI, the HTTP API, or by restarting the service that owns the declaration. Get the properties right: a later re-declaration with mismatched arguments trades your 404 for a 406 PRECONDITION_FAILED. If bindings were also lost, recreate them; an exchange without bindings routes nothing.

Fix auto-delete footguns

If an auto-delete exchange or queue vanished, decide whether auto-delete was ever appropriate. For topology that publishers depend on unconditionally, declare it durable and not auto-delete. If auto-delete is intentional (ephemeral reply queues, per-session exchanges), publishers must tolerate NOT_FOUND by re-declaring before retrying, not by crashing.

Make every service declare what it needs

The durable fix for deploy-ordering 404s is idempotent declaration on startup in every service: each publisher declares the exchanges it publishes to, each consumer declares its queues and bindings. Declarations of an existing resource with identical properties are no-ops, so overlapping declarations are safe. This removes the boot-order dependency entirely. The alternative, a startup dependency forcing publishers to wait for the declarer, works but couples service startup in a way that bites during partial outages.

Handle the channel exception in the client

Treat a channel-level 404 as recoverable: open a new channel, declare the topology, retry. Do not let a single NOT_FOUND tear down the whole connection in a tight loop, and add backoff to reconnect logic. A declare-fail-reconnect loop against a resource that will never appear is indistinguishable from a connection storm at the broker.

If you upgraded RabbitMQ recently

Passive declaration permission rules changed across versions: 3.13 allowed passive declares for users with any vhost permission, and 4.3.1 tightened this to require at least one permission on the target resource itself. After an upgrade, a previously working passive declare can start failing, though that failure surfaces as ACCESS_REFUSED rather than NOT_FOUND. If the error you see is genuinely 404 after an upgrade, suspect changed declaration ordering in new client or broker defaults before suspecting permissions.

Prevention

  • Topology as code. Manage exchanges, queues, and bindings declaratively (definitions export, or client startup declarations under version control). Review renames like schema migrations: old and new names coexist until all clients move.
  • Idempotent declare on startup, everywhere. Every service declares what it uses. This is the single most effective defense against ordering-related 404s.
  • Alert on not_found in broker logs. Channel exceptions are only visible in logs, not in Management API metrics. A log-based alert on channel exception not_found catches the first occurrence instead of the thousandth.
  • Deploy correlation. Tag 404 log alerts with deploy events. A burst that starts at a rollout and self-resolves is ordering; one that persists is a rename or deletion.
  • Avoid auto-delete on shared topology. Reserve auto-delete for resources with a genuine single-owner lifecycle.
  • Set mandatory=true or publisher confirms where loss matters. This does not prevent 404s, but it converts the neighboring silent-routing-loss failure into a visible signal.

How Netdata helps

  • Churn correlation. Netdata collects RabbitMQ connection and channel churn rates from the Management API, so a client looping on declare-fail-reconnect shows up as a visible spike you can correlate with the deploy timeline and broker log 404s.
  • Publish vs deliver divergence. Per-queue and global publish, deliver_get, and ack rates make the silent-routing-loss variant visible: publishing continues, delivery is zero, depth is flat.
  • Return unroutable tracking. The return_unroutable counter is charted, so mandatory-flag publishers get a durable signal when routing breaks, complementing log-based 404 alerts.
  • Consumer count per queue. When a queue disappears and is recreated, its consumers drop to zero until clients recover; per-queue consumer charts make that gap obvious.
  • Object totals. Queue and exchange count changes over time surface deletions of topology that should be permanent.