$ guides / varnish / varnish-session-close-reasons ▌

Operations Guides

Varnish session close reasons: reading the sc_* counters

The MAIN.sc_* counters in Varnish record why every client session ended. Each close reason is a DIAG-level counter visible through varnishstat. The distribution across these counters is one of the fastest triage signals available: it tells you whether sessions are ending normally or whether the client, the network, or Varnish itself is causing problems.

The sc_* family has grown across versions. Some counters shifted accounting between releases, and several error counters are benign under certain traffic patterns. Treating every nonzero error counter as an incident leads to alert fatigue; treating them all as noise means missing real problems.

The sc_* family

Every Varnish client session ends with a close reason. Varnish records these as counters prefixed MAIN.sc_, each incremented when a session terminates for that cause. The counters are classified at DIAG level:

varnishstat -1 -f 'MAIN.sc_*'

On a healthy system, the vast majority of closes are normal client-initiated disconnects. A shift toward error counters is the signal to investigate.

Normal close reasons

These counters reflect expected session lifecycle and should dominate the distribution.

Counter	Close reason	What it means
`sc_rem_close`	Client closed	Client closed the TCP connection. The most common close reason on any healthy system.
`sc_req_close`	Client requested close	Client sent `Connection: close`, ending the session after the request. Normal for HTTP/1.0 clients or clients that disable keepalive.
`sc_tx_pipe`	Piped transaction	Session ended after a pipe transaction. Normal for traffic handled via `return(pipe)` in VCL.
`sc_tx_eof`	EOF transmission	Session closed after Varnish sent an EOF to the client. Normal for streaming or chunked responses that complete naturally.
`sc_resp_close`	Backend/VCL requested close	Varnish closed the session because the backend or VCL indicated `Connection: close`.

A high count of sc_rem_close is not a problem. Clients close connections when done, especially mobile clients and browser connection pools. What matters is the ratio of normal closes to error closes, and whether previously zero error counters start incrementing.

Close reasons that warrant investigation

These counters indicate something went wrong during the session. Each points to a different layer of the stack.

sc_overload: Varnish ran out of a resource

The official description is “Out of some resource.” This is a Varnish-side problem. The most common trigger is thread pool exhaustion: no worker thread was available to handle the session, and the queue was full.

When sc_overload is nonzero and sustained, correlate with:

MAIN.threads at or near thread_pool_max * thread_pools
MAIN.thread_queue_len greater than zero
MAIN.threads_limited incrementing

If these are elevated, either increase thread_pool_max or address the upstream cause (slow backends holding threads, traffic spike exceeding pool capacity).

sc_rx_bad: malformed request from the client

The client sent a request that Varnish could not parse as valid HTTP. Causes include buggy clients, protocol mismatches, or deliberate attack traffic such as request smuggling, fuzzing, or buffer overflow attempts.

varnishlog -q 'BogoHeader or HttpGarbage'

A low, steady rate is normal internet background noise from scanners and bots. A sudden spike above baseline warrants source IP analysis. Varnish is strict in HTTP parsing compared to some other proxies, so some legitimate but non-compliant clients may trigger this counter.

sc_rx_timeout and sc_rx_close_idle: idle or slow clients

These counters were split in Varnish 6.4. Before 6.4, all idle timeouts were counted as sc_rx_timeout. After 6.4:

sc_rx_close_idle counts sessions where timeout_idle was exceeded while waiting for a client request. This is the idle-keepalive case: the client opened a connection, sent a request, then sat idle past the configured timeout (default 5 seconds).
sc_rx_timeout now counts timeouts during active request receiving. The client started sending a request but stalled partway through.

sc_rx_close_idle is usually the dominant error counter on a healthy system. Clients open keepalive connections and abandon them. This is benign. Increasing timeout_idle reduces sc_rx_close_idle but keeps idle connections alive longer, consuming file descriptors.

sc_rx_timeout in the post-6.4 meaning is more interesting. A client started but did not finish a request. This can indicate slow uploads, broken clients, or slowloris-style attacks. Correlate with timeout_req, the parameter governing how long Varnish waits for a complete request.

sc_tx_error: error transmitting to the client

Varnish encountered an error while writing the response to the client. The most common cause is a broken pipe: the client disconnected while Varnish was still sending the response body.

This is often normal on mobile networks and unreliable connections. A moderate sc_tx_error rate is expected for traffic with a significant mobile client base.

Important version change: starting in Varnish 6.4, send_timeout events are reported as sc_tx_error instead of sc_rem_close. If you upgraded from pre-6.4 and see sc_tx_error increase, check whether send_timeout is appropriate for your workload. The Varnish 6.4 upgrading guide warns about this for HTTP/1 clients with long-running backend fetches: the client connection may time out while Varnish waits for the backend, and the close is now accounted as sc_tx_error.

sc_rapid_reset and sc_bankrupt: HTTP/2 abuse

Counter	What it detects
`sc_rapid_reset`	HTTP/2 Rapid Reset attack pattern (CVE-2023-44487): a client rapidly opens and resets streams to overwhelm the server.
`sc_bankrupt`	HTTP/2 credit bankruptcy: all streams were waiting for flow-control window credits when `h2_window_timeout` triggered. CVE-2024-30156 is the related Broke Window credit-exhaustion attack.

On a healthy system, both should be zero. A sustained nonzero sc_rapid_reset rate indicates a possible DDoS attack against your HTTP/2 endpoint. Version availability differs: VSV00013 added rapid-reset protection in 7.3.1/7.4.2/6.0.12, while the credit-exhaustion mitigation appeared in 7.3.2/7.4.3/6.0.13 and h2_window_timeout was generalized in 7.5.

Other error close counters

Counter	What it means	When to worry
`sc_rx_junk`	Received junk data that could not be parsed	Spike indicates scanning or protocol abuse
`sc_rx_overflow`	Received buffer overflow	Spike indicates oversized requests or attack
`sc_rx_body`	Failure receiving request body	Client disconnected during POST/PUT or network issue
`sc_pipe_overflow`	Session pipe buffer overflow	Piped request exceeded pipe buffer limits
`sc_range_short`	Insufficient data for requested byte range	Client requested a range beyond object size
`sc_req_http10`	Session used HTTP/1.0 protocol	Informational; normal for legacy clients
`sc_req_http20`	HTTP/2 not accepted	Varnish rejected an HTTP/2 preface because HTTP/2 was disabled or not accepted on that listener
`sc_vcl_failure`	VCL failure during session	Correlate with `MAIN.vcl_fail`; indicates VCL runtime error

Version changes that shift counter accounting

Three version transitions materially change how the sc_* distribution looks. If you compare data across these boundaries, the numbers are not directly comparable.

Varnish 6.4:

MAIN.sess_drop was removed entirely. Any monitoring referencing it must be updated.
Idle timeouts split from sc_rx_timeout into the new sc_rx_close_idle. Pre-6.4 idle timeout counts will appear as sc_rx_timeout.
send_timeout events reclassified from sc_rem_close to sc_tx_error. This is the most impactful change for operators who treated sc_rem_close as “all normal.” The sc_tx_error counter will increase after upgrade even if traffic behavior is unchanged.

Varnish 6.6:

Fixed the close reason to properly report sc_resp_close where previously only sc_req_close was reported. After upgrading from pre-6.6, sc_req_close will appear to drop and sc_resp_close will appear to rise. This is the fix taking effect, not a behavior change in your traffic.

Varnish 7.3:

VXID format changed to 64-bit. In-memory and on-disk VSL format is not compatible with previous versions. Log dumps from prior releases are unreadable. This does not affect counter semantics but affects log analysis tooling.

Varnish 7.5:

Added sc_rapid_reset and sc_bankrupt for HTTP/2 attack mitigations.

Reading the distribution as a triage tool

The value of the sc_* family is not in any single counter but in the distribution. When diagnosing client-reported problems or investigating an anomaly, the distribution tells you where to look first.

flowchart TD
    A["Session close distribution"] --> B{"Dominant counters?"}
    B -->|"sc_rem_close, sc_req_close,
sc_tx_pipe, sc_tx_eof"| C["Normal:
client-initiated closes"]
    B -->|"Error counters elevated"| D{"Which error counter?"}
    D -->|"sc_overload"| E["Varnish-side:
check thread pool, queue"]
    D -->|"sc_rx_bad, sc_rx_junk,
sc_rx_overflow"| F["Client-side:
malformed HTTP or attack"]
    D -->|"sc_rx_timeout,
sc_rx_close_idle"| G["Idle/slow clients:
check timeout_idle, timeout_req"]
    D -->|"sc_tx_error"| H["Transmit error:
check send_timeout, mobile baseline"]
    D -->|"sc_rapid_reset,
sc_bankrupt"| I["HTTP/2 abuse:
DDoS or credit exhaustion"]
    D -->|"sc_vcl_failure"| J["VCL error:
check MAIN.vcl_fail"]

Triage logic:

Normal counters dominate, error counters near zero. System is healthy.
sc_overload incrementing. Problem is on the Varnish side. Check thread pool saturation, queue length, and resource limits before looking at client behavior.
sc_rx_bad or sc_rx_junk spikes. Problem is on the client side. Determine whether it is a broken client or deliberate attack by examining source IPs and request patterns in varnishlog.
sc_rx_close_idle dominates error counters. Usually benign idle keepalive behavior. Investigate only if the rate is consuming file descriptors or if timeout_idle is set so high that idle connections accumulate.
sc_tx_error spikes after a pre-6.4 upgrade. Verify that the increase is due to send_timeout reclassification rather than a real network problem. Compare with the pre-upgrade sc_rem_close rate.
sc_rapid_reset nonzero. Treat as a potential security incident and investigate HTTP/2 traffic patterns.
sc_vcl_failure incrementing. Correlate with MAIN.vcl_fail. The compiled VCL hit a runtime error during session processing. Check for a recent VCL reload.

Monitoring sc_* counters with Netdata

Netdata collects all sc_* counters with per-second resolution. This matters during traffic spikes or attacks where the distribution shifts in seconds.

When sc_overload spikes, the Varnish collector shows MAIN.threads, MAIN.thread_queue_len, and MAIN.threads_limited in the same dashboard, confirming whether the overload is thread-pool-driven. Per-second rates mean a brief sc_rapid_reset burst is visible before it becomes sustained. Historical retention lets you compare the distribution before and after a version upgrade, so a reclassification artifact does not look like a phantom regression.

Anomaly detection flags deviations from the learned baseline for each counter, which helps catch gradual increases in sc_rx_bad or sudden spikes in sc_tx_error without manual threshold tuning.