NGINX upstream keepalive: eliminating per-request TCP and TLS overhead

Every proxied request that opens a fresh TCP connection burns latency on the handshake and, if the upstream uses HTTPS, on TLS negotiation. At low volume this cost is invisible. At production throughput it becomes a measurable tax on every request, and at extreme scale it can exhaust the kernel’s ephemeral port range and bury the host in TIME_WAIT sockets. For HTTPS upstreams, the CPU cost of TLS handshakes across thousands of requests per second consumes worker cycles that could be spent proxying traffic.

The keepalive directive inside an upstream block caches idle backend connections per worker process. When it works, the next request reuses an existing socket and $upstream_connect_time logs as 0.000. When it is missing or misconfigured, you pay the full connection tax every time, and the symptoms look like upstream latency or connect() failed (99: Cannot assign requested address) errors.

What it is and why it matters

The keepalive directive in an upstream block configures a per-worker cache of idle connections to backend servers. Without it, nginx closes the upstream connection after each response. The next request must open a new TCP connection, complete the three-way handshake, and, for HTTPS upstreams, perform a full TLS negotiation. On a local network the TCP handshake adds roughly one millisecond; a TLS handshake can add tens of milliseconds and significant CPU load. Under high request rates the kernel holds each closed socket in TIME_WAIT state for twice the maximum segment lifetime, typically 60 seconds. This is the primary cause of ephemeral port exhaustion on busy reverse proxies.

Nginx in reverse proxy mode uses two connections per proxied request: one from the client and one to the upstream. Reusing the upstream socket removes per-request overhead and prevents the kernel from churning through the ephemeral port range.

How it works

The keepalive mechanism is straightforward in concept but has specific prerequisites and version-dependent behavior.

Inside an upstream block, the keepalive directive sets the maximum number of idle connections that each worker process will cache for that upstream group. When a worker finishes proxying a request, it keeps the upstream connection open and places it in a pool keyed by the upstream server address. The next time that worker needs to reach the same backend, it pulls the idle connection instead of opening a new one.

The pool is per-worker, not global. If you configure worker_processes 8 and keepalive 32, nginx may hold up to 256 idle connections against the upstream in total. This is important for capacity planning on the backend: the upstream server must tolerate the sum of idle pools from all workers plus any active connections currently in flight.

Since nginx 1.29.7, keepalive to HTTP upstreams is enabled by default with a value of keepalive 32 local. The local keyword means cached connections are isolated to the location block that acquired them; they are not shared across different location blocks that reference the same upstream. Without local, any matching idle connection to the same upstream server is reused regardless of which location triggered it.

Versions prior to 1.29.7 required explicit configuration to enable upstream keepalive. You needed to set proxy_http_version 1.1 and proxy_set_header Connection "" in the location block to prevent nginx from sending Connection: close to the backend. Since 1.29.7, the default proxy_http_version is 1.1 and the Connection header is no longer sent by default, so those two lines are unnecessary for basic keepalive. If you need to force HTTP/1.0 to a specific backend, use proxy_http_version 1.0 and proxy_set_header Connection "close".

Several directives control connection lifecycle:

  • keepalive_requests sets the maximum number of requests that nginx will send over a single cached connection before closing it. The default is 1000 since nginx 1.19.10; before that it was 100.
  • keepalive_timeout sets how long an idle connection stays in the pool. The default is 60 seconds since nginx 1.15.3.
  • keepalive_time sets the maximum time a connection may be kept alive regardless of activity. The default is 1 hour since nginx 1.19.10.

When a request arrives, nginx checks the worker’s pool for an idle connection to the target upstream server. If one exists and has not exceeded these limits, nginx reuses it. The access log variable $upstream_connect_time will show 0.000 because no new TCP or TLS handshake occurred. If no idle connection is available, nginx opens a new one and $upstream_connect_time will be nonzero. Note that DNS resolution time is not included in this metric.

flowchart TD
    Request[New upstream request] --> Check{Idle connection available?}
    Check -->|Yes| Reuse[Reuse connection]
    Check -->|No| Handshake[New TCP plus TLS handshake]
    Reuse --> Proxy[Proxy request]
    Handshake --> Proxy
    Proxy --> Return[Return response]
    Return --> Keepalive{Below keepalive_requests and keepalive_time?}
    Keepalive -->|Yes| Pool[Add to idle pool]
    Keepalive -->|No| Close[Close connection]
    Pool --> Timeout{Idle exceeds keepalive_timeout?}
    Timeout -->|Yes| Close
    Timeout -->|No| Check

FastCGI upstreams require fastcgi_keep_conn on in the location block to use keepalive. SCGI and uwsgi do not support keepalive connections. Memcached upstreams use standard keepalive without extra header manipulation.

Where it shows up in production

The most obvious symptom of missing or broken upstream keepalive is elevated $upstream_connect_time. If your backends are on the same datacenter network, new TCP connections should complete in under a millisecond. Seeing a consistent P95 of 2-10 ms on every request often means you are paying for TLS handshakes that could have been avoided.

At higher scale, the kernel symptom is more severe. Without connection reuse, every proxied request opens a new outbound socket. When the response finishes, that socket enters TIME_WAIT for roughly 60 seconds. If your request rate exceeds the drain rate, TIME_WAIT sockets accumulate until they exhaust the ephemeral port range. The error log fills with connect() failed (99: Cannot assign requested address) and clients receive 500-series errors. Confirm this by checking ss -tan state time-wait | wc -l. The fix is enabling upstream keepalive, not merely widening net.ipv4.ip_local_port_range.

A subtler symptom appears during nginx restarts. After a restart or reload, each worker’s keepalive pool is empty. The first requests to each upstream incur the full handshake penalty, causing a brief latency spike until the pools warm. This is normal, but if you never see $upstream_connect_time drop to 0.000 after the warm-up period, your pool is either too small or the backend is closing connections prematurely.

Keepalive also affects cascade failure patterns. When backends slow down, active upstream connections stay busy longer. If the pool is small and traffic is high, new requests cannot find idle connections and must open new ones. This increases load on the already struggling backend and accelerates the consumption of ephemeral ports.

Tradeoffs and when to use it

For HTTP upstreams, keepalive should be the default. The exceptions are specific and worth understanding.

The keepalive directive caps only idle connections, not total active connections. A setting of keepalive 32 does not mean “at most 32 connections to this upstream.” It means “cache up to 32 idle connections per worker.” Active connections in flight are not counted against this limit. You must size the upstream server to handle active load plus the sum of all worker idle pools.

Because pools are per-worker, load balancers with many workers can open a surprising number of idle connections against a small backend. If your upstream is resource-constrained, you may need to reduce the keepalive count or use the max_conns parameter on the server directive to limit total active connections.

Persistent HTTP/1.1 connections are a prerequisite for HTTP request smuggling. If your backend parses chunked encoding or Content-Length headers differently than nginx, reusing connections across requests can enable desync attacks. In high-security environments with untrusted or ambiguous backends, you may choose to disable keepalive or ensure the backend strictly follows RFC 7230.

The local parameter, now default since 1.29.7, trades efficiency for isolation. Connections are not shared across location blocks, which prevents certain cross-location state issues but increases total idle connections against the upstream. If you have many location blocks hitting the same upstream and backend capacity is tight, you may want to test removing local.

SCGI and uwsgi upstreams cannot use keepalive. FastCGI requires an extra directive. Memcached upstreams use standard keepalive without header manipulation.

Signals to watch in production

SignalWhy it mattersWarning sign
$upstream_connect_timeZero means keepalive reuse; nonzero means new handshakeP95 consistently above 0 for local upstreams
Upstream keepalive hit rateFraction of requests reusing a cached connectionBelow 70 percent indicates significant overhead
TIME_WAIT socket countSockets held after close; accumulate without keepaliveThousands of entries targeting upstream ports
Ephemeral port utilizationExhaustion blocks new upstream connectionsApproaching 80 percent of ip_local_port_range
Active connection utilizationProxy mode uses two slots per requestSustained above 80 percent of worker_connections times worker_processes divided by 2
$upstream_response_timeCascade failures start when backends slow and pools emptyP95 trending up while connect time stays flat

To approximate keepalive hit rate from access logs, compare $upstream_connect_time values. A value of 0.000 strongly indicates reuse. Very fast local connections under one millisecond may occasionally be misclassified, so treat this as an approximation rather than an exact ratio.

How Netdata helps

Netdata collects nginx stub_status metrics and system-level TCP metrics. Use them to detect keepalive problems before they become outages:

  • Correlate nginx active connections with $upstream_connect_time from access logs. Flat or rising connect time while active connections are stable means your keepalive pool is ineffective.
  • Monitor kernel TIME_WAIT sockets and ephemeral port usage on the nginx node. A climbing TIME_WAIT count with steady traffic is a leading indicator that upstream connections are not being reused.
  • Watch the accepts-handled gap from stub_status alongside connection slot utilization. If slots are filling and connect time is nonzero, you may be hitting the proxy connection multiplier limit while also failing to reuse upstream sockets.
  • Alert on sustained nonzero $upstream_connect_time for upstreams that should be local and fast.