$ guides / nats / nats-maximum-payload-violation ▌

Operations Guides

NATS Maximum Payload Violation: messages rejected for exceeding max_payload

A publisher calls publish, the call fails, and the NATS server log shows a Maximum Payload Violation. Seconds later the same client reconnects, publishes again, and gets disconnected again. The server is not broken: it is enforcing the configured max_payload limit, which defaults to 1 MB. The application, meanwhile, is in a publish-disconnect-reconnect loop and its messages are not flowing.

The protocol behavior is strict. When a client sends a message whose payload exceeds max_payload, the server responds with -ERR 'Maximum Payload Violation' and closes the connection. Client libraries that auto-reconnect come straight back and repeat the offense, which is why a single oversized publish looks like connection churn rather than a clean, one-time error.

This guide covers how to confirm the violation, identify the offending client and subject, decide whether the payload growth is legitimate, and raise the limit without trading this problem for a memory or slow-consumer problem.

What this means

Every NATS server enforces a maximum payload size per published message. The default is 1 MB. The limit exists because core NATS routes messages through memory in real time: each in-flight message must be read fully from the publisher’s socket into a buffer and fanned out to subscriber write buffers. Unbounded payloads would make per-connection memory unpredictable.

The limit is not advisory. The server tells each client the current value in the max_payload field of the INFO protocol message sent at connect time, and clients are expected to check it before publishing. Clients that skip that check learn about the limit the hard way: one -ERR, one closed connection.

Two consequences matter operationally:

The rejection looks like churn, not like a clean error. The publish fails and the connection drops. Auto-reconnect logic brings the client back. If the application retries the same oversized message, the loop repeats. In /varz this shows up as total_connections climbing while connections stays roughly flat.
The message is never delivered. This is not a queue-and-retry situation in core NATS. The rejected publish is gone. If the publisher does not handle the error, that data is lost.

flowchart LR
  P[Publisher] -->|publish N bytes| S{NATS server}
  S -->|N <= max_payload| R[Route to subscribers]
  S -->|N > max_payload| E[-ERR Maximum Payload Violation]
  E --> D[Connection closed]
  D --> RC[Client auto-reconnects]
  RC -->|retry same message| S

Common causes

Cause	What it looks like	First thing to check
Payloads legitimately outgrew the 1 MB default	Violations start after a data model change, new field, or feature rollout; byte rate climbs while message rate stays flat	Average message size: `in_bytes / in_msgs` trend from `/varz`
A new client publishes a type of message the system never had (reports, exports, file chunks, images)	Violations correlate with one client or one subject; other traffic is fine	Server logs for the violating connection, then that client’s subjects
Client never checks `max_payload` from INFO and blasts large messages on connect	Reconnect loop immediately after each connect; violations in bursts	Client library behavior and publish error handling
Serialized format change (e.g., switching to a verbose encoding, embedding blobs in JSON)	Gradual average-size creep, then a hard wall once messages cross 1 MB	Byte rate vs message rate divergence over weeks
Misconfiguration: limit lowered or left at default intentionally, client team unaware	Violations start right after a config change or reload	Current `max_payload` in server config vs recent config changes
MQTT clients sending large messages	Oversized MQTT publishes; note that some server versions did not enforce `max_payload` on the MQTT path at all	Server version if you run the MQTT endpoint (see Fixes)

Quick checks

All of these are read-only and safe to run on a production server.

# Find payload violations in the server log
grep "Maximum Payload" /var/log/nats/nats-server.log | tail -20

The log line ties the violation to a specific connection, which is how you find the guilty client. Adjust the log path to your deployment.

# Check current throughput: message rate vs byte rate
curl -s http://localhost:8222/varz | jq '{in_msgs, out_msgs, in_bytes, out_bytes}'

Take two snapshots 10 to 60 seconds apart and compute rates. Then compute average message size as in_bytes delta / in_msgs delta. If that average is climbing toward your max_payload, you have found the trajectory before you hit the wall.

# Check connection churn: is total_connections climbing while connections is flat?
curl -s http://localhost:8222/varz | jq '{active: .connections, total: .total_connections}'

A publisher stuck in the publish-violate-disconnect-reconnect loop shows up here as churn.

# See what is currently connected and what it looks like
curl -s "http://localhost:8222/connz?sort=last&limit=10" | jq '.connections[] | {cid, name, ip, subscriptions, last_activity}'

Newly reconnected clients near the top, correlated with log timestamps, usually identify the offender. If your clients set a connection name, this is fast; if not, match by source IP.

# Confirm the configured limit the server is actually running with
grep -i max_payload /etc/nats/nats-server.conf

Also check for account-level overrides. Per-account max_payload limits can be set under account limits in the server config, and an account-scoped limit lower than the server-wide value produces exactly this symptom for only some clients.

How to diagnose it

Confirm the error. Grep the server log for “Maximum Payload”. If it is not there, the publish failures have a different cause (permissions, JetStream limits, slow consumer disconnects) and this guide does not apply.
Identify the connection. The log entry references the offending connection. Cross-reference the timestamp with /connz output or the client IP to find which application it is.
Identify the subject and message type. Once you know the client, determine what it was publishing. A single subject (an export job, a report generator) points to a payload design problem. Every subject from that client points to a serialization or client bug.
Check the trajectory. Compute average message size from /varz (in_bytes / in_msgs over an interval) and compare it to historical values. A byte-rate spike without a message-rate rise means payloads are bloating. This distinguishes “messages grew gradually and crossed the line” from “one new message type was always too big.”
Check for account-scoped limits. If only one team’s clients are affected while others publish similar sizes fine, look for a per-account max_payload that is lower than the server-wide value; account limits have been available since v2.0.0.
Rule out a JetStream limit instead. JetStream streams have their own per-stream MaxMsgSize limit that can be set lower than the server max_payload. A publish rejected by a stream’s MaxMsgSize is a different error path than the core protocol violation. If the publisher is writing to JetStream and the server log does not show the protocol -ERR, check the stream configuration; the JetStream publish acknowledgment carries error 10054, “message size exceeds maximum allowed.”
Decide: legitimate growth or abuse/bug. Legitimate growth (the data model genuinely needs 2 MB messages) is a capacity decision. A bug (a client accidentally embedding a file in a field, or a pathological retry building ever-larger batches) is a code fix. Do not raise the limit to paper over a bug.

Metrics and signals to monitor

Signal	Why it matters	Warning sign
Average message size (`in_bytes` rate / `in_msgs` rate)	The leading indicator. Payloads bloat long before they hit the limit	Steady climb toward `max_payload` over days or weeks
Byte rate vs message rate divergence	Separates “more messages” from “bigger messages”. Only byte rate catches payload bloat	Byte rate spikes while message rate is flat
`total_connections` delta vs `connections`	Violating clients disconnect and reconnect, producing churn with a stable active count	Churn correlated with violation log lines
Server log: Maximum Payload events	The authoritative record of which connection violated and when	Any occurrence in production; repeats from one client
`slow_consumers` and per-connection `pending_bytes`	Large payloads amplify slow-consumer risk: fewer messages fill a subscriber’s write buffer	Slow consumer events appearing after a payload size increase
Server memory (`/varz` `mem`)	Larger payloads mean larger per-message buffers in flight	RSS growth after raising `max_payload` or after average size climbs

Fixes

Fix the client (correct in most cases)

If one client is publishing oversized messages by accident or by a design that violates the system’s contract, fix the producer: chunk large payloads, move bulky data to object storage and send a reference, or compress before publishing. This keeps NATS doing what it is good at: fast routing of small messages. The server-side limit is cheap to raise, but every megabyte you add increases memory-per-in-flight-message and the slow-consumer blast radius for all clients, not just the offender.

Raise max_payload deliberately (when payloads legitimately grew)

If the data model genuinely outgrew 1 MB, raising the limit is supported. Do it with eyes open:

Check the bounds. The configuration parser accepts values up to the signed 32-bit byte limit, but the server warns above 8 MB and the NATS maintainers recommend staying at or under 8 MB. max_payload must also be smaller than or equal to max_pending (default 64 MB); a config that violates this is rejected.
Set it explicitly in the server config, e.g. max_payload: 8MB at the top level, or per-account under that account’s limits if only one tenant needs it.
Reload instead of restarting. max_payload is reloadable: nats-server --signal reload updates server INFO and already-connected clients, and it enforces the new value without dropping connections.
Budget the memory. A larger max_payload increases the buffer the server needs to read each full message from the socket and enqueue it to outbound connections. On a server with many concurrent large publishers this shows up in RSS. Watch /varz mem after the change.
Tell client teams. Clients that read max_payload from the INFO message will pick up the new value on their next connect. Clients that hardcoded assumptions need to reconnect or be updated.

If you use the MQTT endpoint

A server-side bug in v2.12.x (reported on v2.12.1) meant max_payload was not enforced for messages arriving over the MQTT endpoint; oversized messages were accepted and processed. The fix shipped in v2.14.0. If you run MQTT, upgrade to v2.14.0 or later rather than relying on the limit for protection on affected releases.

If it is actually a JetStream MaxMsgSize rejection

If diagnosis showed the limit is the stream’s MaxMsgSize rather than the server max_payload, adjust the stream configuration instead (or fix the publisher). Server-wide max_payload changes will not help a stream-scoped rejection.

Prevention

Watch average message size as a first-class metric. The ratio of byte rate to message rate is the earliest warning that payloads are drifting toward the limit. Alert on the trend, not on violations.
Publish a payload contract. Give client teams a documented maximum message size (typically well under max_payload) and a pattern for bulk data: store the blob elsewhere, send the pointer.
Handle publish errors in clients. The -ERR plus disconnect must not become an invisible retry loop. Clients should check max_payload from INFO at connect time and fail loudly on oversized messages instead of reconnect-and-repeat.
Size changes deliberately. If you raise max_payload, do it per-account where possible, stay within the recommended range, and re-check memory and slow-consumer signals afterward. Large payloads mean a slow consumer’s pending buffer fills with fewer messages, so existing slow-consumer thresholds get more sensitive.

How Netdata helps

Netdata’s NATS collector polls the server’s HTTP monitoring endpoints and surfaces the signals that matter for this failure:

Byte rate and message rate side by side, so a payload-bloat divergence (bytes climbing, messages flat) is visible on one chart before it reaches the limit.
Connection churn: connections against the total_connections delta, which exposes the publish-violate-disconnect-reconnect loop even when the active connection count looks healthy.
Slow consumer counters, which often tick upward after payloads grow, since oversized messages fill subscriber write buffers faster.
Server memory (RSS) trend, to validate that a max_payload increase did not quietly raise per-connection buffer cost.
Long retention, so you can correlate the first violation in the logs with the week-over-week climb in average message size that preceded it.