Varnish backend probe configuration: threshold, window, interval, and initial

A Varnish backend probe is a periodic HTTP request that determines whether a backend receives traffic. Its parameters control detection latency, flapping behavior, and false health states.

Tighter windows detect failures faster but flap on network hiccups. Longer intervals reduce probe load but extend the detection blind spot. The default .initial prevents false-sick states at startup, but a freshly loaded VCL marks backends unhealthy until the first real probe succeeds.

What it is and why it matters

The probe is Varnish’s only proactive mechanism for deciding backend health. A director routes traffic only to backends the probe has marked healthy. If all backends in a director are sick, Varnish serves stale content via grace if configured, or returns 503.

Health is not binary per-probe. Varnish maintains a sliding window of recent results and requires a minimum number of successes within that window before marking a backend healthy. This prevents a single packet loss from removing a backend from rotation, but means probe configuration directly determines detection latency: how long a backend can be broken before Varnish stops sending it traffic.

How it works

Varnish tracks probe results in a 64-stage shift register per backend. Each probe result (success or failure) is shifted into the register. The backend is healthy if successes in the last .window entries meet or exceed .threshold.

flowchart TD
    A[VCL loaded] --> B[.initial happy entries inserted into register]
    B --> C[Probe timer fires every .interval]
    C --> D[HTTP request sent to .url or .request]
    D --> E{Status matches .expected_response?}
    E -->|Yes| F[Shift success into register]
    E -->|No or timeout| G[Shift failure into register]
    F --> H{Successes in last .window >= .threshold?}
    G --> H
    H -->|Yes| I[Backend healthy]
    H -->|No| J[Backend sick]
    I --> K[Director routes traffic to backend]
    J --> L[Director skips backend]
    K --> C
    L --> C

Health is evaluated after every probe but depends on history, not just the latest result. With .window=8 and .threshold=3, a healthy backend (register full of successes) tolerates up to 5 consecutive failures before the success count drops below threshold. A sick backend with a register full of failures needs 3 consecutive successes to recover. In intermediate states, recovery depends on how many successes remain in the window.

The probe parameters

.window

The number of recent probe results tracked. Default is 8. The shift register holds up to 64 entries: setting .window above 64 fails VCL compilation (“Window must be 64 or less.”). A larger window smooths transient failures. A smaller window makes health more responsive.

.threshold

The minimum successes within the window to consider the backend healthy. Default is 3. With .window=8 and .threshold=3, the backend stays healthy as long as at least 3 of the last 8 probes succeeded. The ratio threshold / window controls failure tolerance:

• 3/8 (default): tolerates up to 5 consecutive failures. Reasonable for backends with occasional network blips.
• 3/3: requires all recent probes to succeed. Detects failures immediately but flaps on any transient.
• 1/1: a single failed probe marks the backend sick. Combined with a long .interval, creates dangerous blind spots (see below).

.interval

Time between probe requests. Default is 5 seconds. With .interval=5s and .threshold=3, a healthy backend that starts failing all probes takes 3 consecutive failures (15 seconds) to transition to sick. Recovery takes 3 consecutive successes (15 seconds) in the worst case.

.timeout

Maximum time to wait for a probe response. Default is 2 seconds. If the backend does not respond within this window, the probe counts as a failure. .timeout should be shorter than .interval to avoid stretched detection cycles. Probes per backend are strictly serialized: the next probe is scheduled one .interval after the previous probe completes, so a long .timeout cannot create overlapping probe requests — it just delays the next probe.

.initial

The number of successes assumed when VCL loads, before real probes complete. Default is .threshold - 1.

This prevents false-sick states at startup. Without .initial, every backend starts with zero successes in the register, guaranteeing a sick state until enough real probes cross threshold.

The default .threshold - 1 is deliberate: it fills the register with just enough successes to avoid an immediate sick state, but not enough to mark the backend healthy. The backend remains unhealthy until the first real probe succeeds. A freshly loaded VCL does not immediately route traffic to an unverified backend.

For backends created on demand by VMODs (such as vmod_dynamic), this is problematic. A backend created and immediately needed for a request will likely still be unhealthy because the first real probe has not completed. Set .initial equal to .threshold to start the backend as healthy, accepting the risk that no real probe has verified it yet.

.initial above .threshold compiles, but Varnish clamps it to .threshold when the backend is created, so a higher value has no effect.

.expected_response

The HTTP status code that counts as a successful probe. Default is 200. Any other status (301, 302, 404, 500) fails the probe. A common source of false-sick states: if the backend redirects the probe URL (common when the Host header does not match the backend’s virtual host), the probe sees a 301 or 302 and marks the backend sick.

.url vs .request

Two ways to define what the probe sends. .request takes precedence if both are specified:

• .url sends an HTTP GET to the specified path (default “/”). Varnish constructs the full request including the Host header.
• .request sends a raw HTTP request string. Requires Connection: close because connection shutdown is part of the health check.

.expect_close (Varnish 7.3+)

Defaults to true. When enabled, the probe expects the backend to close the connection after the response. Setting false makes the probe task wait until .timeout before inspecting the response, increasing probe latency.

The 1-of-1 problem: how probe configuration creates blind spots

The most dangerous misconfiguration is .threshold=1 with a long .interval. With .window=1, .threshold=1, .interval=30s:

• Varnish sends one probe every 30 seconds.
• A single success marks the backend healthy.
• A single failure marks it sick.

If a backend fails 1 second after a successful probe, Varnish routes traffic to it for up to 29 seconds until the next probe fires and fails. During that window, client requests hit a broken backend and return 503.

The fix is not a shorter interval alone. Use a threshold / window ratio that requires multiple consecutive failures before marking sick, combined with an interval short enough that the worst-case detection window is acceptable.

The static-URL problem: probes that do not reflect real health

A probe hitting “/” or a static health endpoint returning 200 tells Varnish only that the web server process is accepting connections. It does not verify that the application can serve real requests, reach its database, or render without errors.

Common scenarios where a static probe reports healthy while the application is broken:

• Database down, web server up: The application server is running but database connections fail on every real request. The health endpoint, which does not touch the database, returns 200.
• Cached error page: The backend serves a cached error page with HTTP 200 status.
• Load balancer maintenance page: The backend’s own load balancer responds to the probe with a 200 maintenance page while all application instances behind it are down.

Designing a probe that reflects real application health requires balancing:

• Depth: exercise the critical path. If the application depends on a database and a cache, the probe should verify both. A probe that touches the database with a trivial query is more useful than one that returns a static file.
• Load: the probe adds request load to the backend. On a backend already struggling, aggressive probes worsen the situation.

A practical approach: create a dedicated health endpoint that touches the database with a trivial query (such as SELECT 1) and verifies the application can render a minimal response. Keep the response body small. Use .interval=5s with .threshold=3 and .window=8 for reasonable detection latency without excessive load.

Probe-induced cascading failures

Every probe is an HTTP request the backend must handle. In a director with N backends, each backend receives one probe per .interval. With the default .interval=5s, that is 0.2 requests per second per backend.

Tightening to .interval=1s means each backend handles 1 probe per second. With multiple Varnish instances probing the same backends, aggregate probe load adds up. On a backend at capacity, the probe competes with real traffic. If the probe itself starts timing out because the backend is overloaded, the backend is marked sick, traffic concentrates on remaining backends, and the cascade accelerates.

The .expect_close parameter (Varnish 7.3+) interacts here. With the default true, the probe waits for the backend to close the connection. If the backend is slow to close, probe tasks accumulate, consuming Varnish worker resources.

How Varnish 7.6 changed sick-backend behavior

In Varnish 7.6, tasks waiting on a backend that goes sick (via probe failure or backend.set_health) now fail immediately rather than waiting. Two new runtime parameters were added for connection queuing: backend_wait_timeout and backend_wait_limit.

Probe-driven health transitions have more immediate user impact in 7.6+. A backend marked sick causes in-flight waits to fail instantly. Probe configuration that causes flapping (such as threshold=1) now produces immediate client-visible errors rather than buffered delays.

Inspecting probe state

# Show per-backend health with probe details
varnishadm backend.list -p

# Detailed shift register view for one backend
# (4 rows: Good IPv4, Good Xmit, Good Recv, Happy; the old debug.health command no longer exists)
varnishadm backend.list -p <backend_name>

# Watch real-time probe results
varnishlog -g raw -i Backend_health

# Check VBE happy counter (successful probes in window)
varnishstat -1 -f 'VBE.*.happy'

# Count connections NOT attempted because backend is sick
varnishstat -1 -f MAIN.backend_unhealthy

VBE.<backend>.happy shows successful probes in the current window. This is not a boolean. A value of 3 with .threshold=3 means the backend is at the healthy boundary. A value of 0 means every recent probe failed.

backend_unhealthy counts connections Varnish did not attempt because the backend was marked sick. This is distinct from backend_fail (connections attempted and failed) and backend_busy (too many outstanding connections). A backend can be completely offline with backend_fail=0 because Varnish never tries to connect when the probe says sick.

To override probe state manually (the backend.set_health command predates the 6.x line and exists in every supported release):

# Force a backend healthy, sick, or return to probe-driven auto
varnishadm backend.set_health <pattern> healthy
varnishadm backend.set_health <pattern> sick
varnishadm backend.set_health <pattern> auto

auto returns the backend to probe-driven health determination. Useful for temporarily removing a flapping backend while investigating.

Signals to watch in production

How Netdata helps

• Per-backend VBE.*.happy tracking: Netdata collects per-backend happy counters at per-second resolution, making the approach toward the sick threshold visible before the transition happens.
• backend_unhealthy vs backend_fail correlation: Side-by-side charts distinguish “probe says sick” from “connections actually failing,” the first diagnostic fork in any backend health incident.
• Anomaly detection on probe patterns: ML-based anomaly flags on happy counter trajectories catch gradual degradation that static thresholds miss, such as a backend dropping from 8/8 to 5/8 over several minutes.
• Correlation with 503 rate and cache_hit_grace: When backends go sick, correlating happy counter drops with client-facing 503s and grace-served hits shows whether the problem is user-visible or masked by stale content.
• Varnish child restart detection: The initial parameter’s effect is most visible after restarts. Netdata’s MGT.child_* counters paired with happy counter resets show whether the post-restart false-sick window is resolving normally.

Related guides

• Varnish Error 503 Backend fetch failed: what the error page actually means
• Varnish backend is sick: health probes, all-backends-sick, and grace
• Varnish cache hit ratio dropped: hit rate collapse and backend overload
• Varnish cache stampede: a popular object expires and the herd hits the backend
• Varnish ESI errors: broken pages and workspace pressure from Edge Side Includes
• Varnish fetch_failed: backend connected but the fetch broke
• Varnish Guru Meditation: reading the XID and tracing the failing request
• Varnish cache_hitpass / cache_hitmiss climbing: uncacheable content bleeding to the backend
• How Varnish actually works in production: a mental model for operators
• Varnish monitoring checklist: the signals every production cache needs
• Varnish monitoring maturity model: from survival to expert
• Varnish not caching: Set-Cookie, Vary, and Cache-Control killing your hit rate