nginx upstream sent too big header while reading response header from upstream

The error log contains upstream sent too big header while reading response header from upstream. Clients receive 502 Bad Gateway. This is not an upstream crash or network timeout. It is a hard size limit: an upstream server is sending response headers larger than the fixed buffer nginx allocates for reading them, so nginx aborts the request and returns 502.

This failure typically hits specific endpoints rather than the whole site. Login callbacks, OAuth2 redirects, and session initialization paths are common because they inject large Set-Cookie headers, JWTs, or verbose debug information into the response. The error is deterministic: every request that produces an oversized header triggers it. This makes it easy to reproduce, but it also blocks all traffic to the affected endpoint until the buffer is enlarged or the header is shrunk.

What this means

nginx allocates a fixed per-connection buffer for reading upstream response headers. For HTTP proxying, proxy_buffer_size controls this buffer. For FastCGI, the equivalent is fastcgi_buffer_size. The default is typically one memory page. When upstream response headers exceed this buffer, nginx cannot parse the response. It treats the upstream as having sent an invalid reply, logs the error, and returns 502 Bad Gateway to the client.

Because nginx workers are event-driven, a buffer rejection happens immediately. The worker does not wait or retry. It closes the upstream connection and moves on. The 502 appears instantly, with no preceding latency spike. This is a key diagnostic signature.

There is no spillover or dynamic expansion for headers. This is a hard boundary enforced on every proxied request.

Common causes

CauseWhat it looks likeFirst thing to check
Large authentication cookies or JWTs502 on login, OAuth callback, or session refreshDirect curl to upstream for Set-Cookie header size
Multiple stacked Set-Cookie headersBackend issues several session cookies in one responseCount and total byte size of Set-Cookie headers
Verbose upstream debug or trace headers502s begin immediately after an application deployRecent application changelogs or header injection
FastCGI backend with large headersError variant references FastCGI instead of proxyfastcgi_buffer_size and fastcgi_buffers values
Buffer size left at defaultHeader sizes grew organically; default is smallnginx -T output for buffer directives in the relevant context

Quick checks

# Count occurrences in the error log
grep -c "upstream sent too big header" /var/log/nginx/error.log

# View recent occurrences with upstream context
grep -n "upstream sent too big header" /var/log/nginx/error.log | tail -20

# Inspect proxy buffer configuration
nginx -T 2>/dev/null | grep -E 'proxy_buffer_size|proxy_buffers'

# Inspect FastCGI buffer configuration
nginx -T 2>/dev/null | grep -E 'fastcgi_buffer_size|fastcgi_buffers'

# Capture upstream response headers directly to measure size
curl -sSL -D - -o /dev/null http://<upstream_address>/<path> 2>/dev/null | wc -c

How to diagnose it

  1. Confirm the symptom. Verify the error log contains the exact string. Correlate timestamps with 502 entries in the access log to measure user impact.
  2. Identify the affected upstream. The error log includes the upstream server address. Determine whether the issue is isolated to one backend or affects all upstreams behind a given location block.
  3. Check for recent deployments. If the error started after an application deploy, the new version is likely adding headers or cookies. Review changelogs or compare headers against a known-good version. Rolling back to confirm is disruptive; only do so in a controlled maintenance window.
  4. Measure upstream response headers. Bypass nginx and send a direct request to the upstream with curl. Inspect the response headers. Pay special attention to Set-Cookie, Authorization, and any custom headers. Pipe the header block through wc -c to get a byte count.
  5. Check nginx buffer configuration. Run nginx -T 2>/dev/null | grep proxy_buffer_size. If no value is returned, the default applies. Compare the measured header size against this value.
  6. Check for FastCGI. If your stack uses FastCGI, the error may reference fastcgi instead of proxy. Inspect fastcgi_buffer_size and fastcgi_buffers using the same measurement approach.
  7. Validate and reload. After adjusting configuration, run nginx -t to test syntax. Only then reload with nginx -s reload or send HUP to the master process.

Metrics and signals to monitor

SignalWhy it mattersWarning sign
HTTP 502 response rateDirect user impact from header buffer rejectionSustained 502s correlated with error log entries for “too big header”
Upstream header time ($upstream_header_time)Distinguishes slow backends from fast backends with large headersNormal or low header time with rising 502 rate suggests a size issue
Error log rate for “too big header”Early warning before client-facing metrics degradeAny nonzero sustained rate warrants investigation
Connection state Writing vs. WaitingReveals whether workers are stuck waiting for upstream dataNormal Writing state with 502s indicates the upstream responded but was rejected
Upstream response header sizeProactive measure if logged at the application layerGrowth toward 8 KB, 16 KB, or your configured proxy_buffer_size limit

Fixes

Increase proxy_buffer_size

In the relevant server or location block, raise the header buffer:

proxy_buffer_size 16k;
proxy_buffers 4 16k;

proxy_buffer_size must be at least as large as the largest upstream response header you expect. proxy_buffers allocates body buffers.

Tradeoff: These buffers are allocated per connection. Raising proxy_buffer_size from the default to 128k on a server handling 10,000 concurrent proxy connections increases memory usage by roughly 1.2 GB. Size for your peak connection count, not your average.

If the upstream response is otherwise valid, increasing the buffer is the correct operational response. Do not treat this as a permanent workaround for runaway application headers. If your application is sending 64 KB of cookies, the root cause is application design. Treat oversized headers as a bug.

Apply the FastCGI equivalent

For FastCGI upstreams, use the analogous directives:

fastcgi_buffer_size 16k;
fastcgi_buffers 4 16k;

The semantics are identical. fastcgi_buffer_size reads the response header from the FastCGI backend, and fastcgi_buffers handles the body.

Reduce upstream header size

If you control the upstream application, shrink the headers before they reach nginx. Options include compressing JWT claims, reducing cookie attributes, removing debug headers, or moving session state out of cookies entirely.

Tradeoff: This requires an application deployment and may involve security or state-management changes. It is the safest long-term fix because it avoids increasing nginx’s per-connection memory footprint.

Prevention

  • Document your header budget. Know the maximum response header size your application produces and ensure your nginx buffer configuration exceeds it by a safety margin. Revisit this after every authentication or session-management change.
  • Test auth flows in staging. OAuth2, OIDC, and SAML callbacks are frequent triggers. Run complete login flows in a staging environment that uses production nginx buffer settings.
  • Log header sizes upstream. If possible, have your application log total response header size. Alert when it approaches 75% of the configured proxy_buffer_size.
  • Include buffer review in deployment checklists. Any change that adds tracking cookies, CORS headers, or debug information can push responses over the limit. Validate header growth before it reaches production.
  • Scope configurations carefully. Buffer directives inherit down into nested contexts, but an oversized buffer at the http level applies to every connection. Set the minimum viable size at the narrowest location block to limit memory exposure.

How Netdata helps

  • Chart HTTP 502 rate from nginx access logs. A sustained spike that correlates with error log timestamps isolates the failure mode without manual grepping during an incident.
  • Monitor $upstream_header_time alongside 502 rates. Flat or low header time combined with climbing 502s points to a header-size abort, not a slow backend.
  • Compare $upstream_header_time against $upstream_response_time. A tiny header time with a normal body transfer time, combined with 502s, confirms the upstream is generating large headers quickly and then streaming a normal body. This pattern isolates header-size issues before you read the error log.
  • Track active connections and the Writing state breakdown. If Writing remains normal while 502s rise, nginx is not waiting for a slow upstream; it is rejecting the response immediately after reading the headers.
  • Alert on sustained HTTP 502 rate increases to catch header-size regressions within minutes of an application deployment.