Varnish cache_hitpass / cache_hitmiss climbing: uncacheable content bleeding to the backend

cache_hitmiss and cache_hitpass are climbing. Backend request rate is growing. Cache hit rate looks stable. This is Varnish’s learned “do not cache” mechanism silently routing traffic past the cache.

When Varnish fetches an object and determines it cannot be cached, it caches that decision itself. For the next 120 seconds (the builtin.vcl default), every request for that URL bypasses the cache and goes straight to the backend. Unlike normal cache misses, hit-for-miss requests are not coalesced: concurrent requests for the same URL each generate an independent backend fetch. The result is silent, compounding backend load that your hit-rate alert probably cannot see.

The problem hides inside hit-rate formulas. When request volume shifts from miss to hitmiss, the hit-rate ratio barely moves because both miss and hitmiss occupy the denominator. A 90% hit rate with 10% hitmiss and a 90% hit rate with 10% miss look identical on a dashboard. But the hitmiss fraction is structurally worse: those requests will never be cached for the TTL window, and each one hits the backend independently without request collapsing.

What this means

Varnish has two mechanisms for caching the decision not to cache. Both create short-lived objects that sit in the cache and short-circuit subsequent requests.

Hit-for-miss is the default behavior in Varnish 5.0+. The built-in VCL in vcl_backend_response creates a hit-for-miss object when a backend response matches any of these conditions:

• beresp.ttl <= 0s
• Set-Cookie header is present
• Cache-Control matches no-cache, no-store, or private
• Surrogate-Control matches no-store
• Vary: *

When any of these match, builtin.vcl sets beresp.uncacheable = true and beresp.ttl = 120s. The object is stored in transient storage with a 120-second TTL. For that window, subsequent requests for the same cache key find the hit-for-miss object and proceed directly to the backend without request coalescing.

Hit-for-miss objects can be replaced. If a cacheable response arrives before the 120-second TTL expires, it replaces the hit-for-miss object and normal caching resumes. A transient backend misconfiguration self-heals after the TTL, but only if a request arrives before the window closes and the response is cacheable.

Hit-for-pass is an older mechanism triggered explicitly in custom VCL via return(pass(DURATION)) in vcl_backend_response. It is not invoked by builtin.vcl in Varnish 5.1+. Hit-for-pass objects persist for their full duration and cannot be replaced by a cacheable response mid-TTL. The only way to end a hit-for-pass object early is req.hash_always_miss = true in vcl_recv or a ban.

flowchart TD
    A["Request for URL"] --> B{"Cache lookup"}
    B -->|"No hit-for-miss object"| C["Normal cache miss"]
    C --> D["Backend fetch
with request coalescing"]
    D --> E{"builtin.vcl finds
uncacheable response?"}
    E -->|"Set-Cookie or
Cache-Control: private/no-cache/no-store
or Vary: * or TTL <= 0"| F["Create hit-for-miss
TTL 120s, uncacheable"]
    E -->|"Response is cacheable"| G["Store in cache
with response TTL"]
    B -->|"Hit-for-miss exists
within 120s"| H["Bypass to backend"]
    H --> I["No coalescing:
each request fetches independently"]

Common causes

Quick checks

# Check current hitpass and hitmiss counters
varnishstat -1 -f MAIN.cache_hitpass -f MAIN.cache_hitmiss

# Check all cache outcome counters to compute hit rate
varnishstat -1 -f MAIN.cache_hit -f MAIN.cache_miss -f MAIN.cache_hitpass -f MAIN.cache_hitmiss

# Check backend request rate (should correlate with hitmiss if the cause)
varnishstat -1 -f MAIN.backend_req

<!-- TODO: verify whether MAIN.beresp_uncacheable is a real counter in any Varnish version -->
# Check if beresp_uncacheable is climbing (counter name may vary by version)
varnishstat -1 -f MAIN.beresp_uncacheable

# Check transient storage growth (hit-for-miss objects live here)
varnishstat -1 -f SMA.Transient.g_bytes

# Find URLs with Set-Cookie in backend responses
varnishtop -i BereqURL -q 'BerespHeader:Set-Cookie'

# Find URLs with Cache-Control directives that prevent caching
varnishtop -i BereqURL -q 'BerespHeader:Cache-Control ~ "private|no-cache|no-store"'

# Check Vary headers on backend responses
varnishtop -i BereqURL -q 'BerespHeader:Vary'

# Verify which VCL is active (recent deploy may have introduced the issue)
varnishadm vcl.list

How to diagnose it

1. Confirm the counters are rising. Take two readings of cache_hitpass and cache_hitmiss 30 seconds apart and compute the delta. A rate proportional to your traffic volume means Varnish is actively short-circuiting a significant fraction of requests.

2. Compute hit rate with and without hitpass and hitmiss. Compare cache_hit / (cache_hit + cache_miss) against cache_hit / (cache_hit + cache_miss + cache_hitpass + cache_hitmiss). A gap of more than a few percentage points means hitpass and hitmiss are masking real cache ineffectiveness.

3. Identify which URLs are triggering the uncacheable decision. Use the varnishtop commands from Quick checks to rank URLs by Set-Cookie, Cache-Control, or Vary header presence in backend responses.

4. Examine the actual response headers. For a specific problematic URL, trace the full transaction:

   varnishlog -g request -q 'ReqURL eq "/problematic/path"'
   

   Look for BerespHeader:Set-Cookie, BerespHeader:Cache-Control, BerespHeader:Vary, and the TTL tag showing how Varnish assigned the TTL.

5. Check for recent application deploys. cache_hitmiss rising after a deploy is the most common pattern. Frameworks like Rails and Django add session cookies by default. A middleware change can add Set-Cookie to previously cacheable endpoints.

6. Review your VCL’s vcl_backend_response. If your custom VCL calls return(pass(DURATION)), you are explicitly creating hit-for-pass objects. Confirm whether this is intentional.

Metrics and signals to monitor

| MAIN.beresp_uncacheable | Backend responses marked uncacheable. Leading indicator before hit-for-miss objects are created, if exposed by your Varnish version. | Climbing trend | | SMA.Transient.g_bytes | Memory used by transient objects including hit-for-miss bodies. | Unbounded growth, potential OOM path |

Fixes

Strip Set-Cookie on cacheable responses

If the application sets session cookies on content that should be publicly cacheable, unset the header in vcl_backend_response before the built-in VCL evaluates cacheability:

sub vcl_backend_response {
    if (bereq.url ~ "^/static/" || bereq.url ~ "^/public-api/") {
        unset beresp.http.Set-Cookie;
    }
    # Fall through to built-in vcl_backend_response
}

Do not call return(deliver) here if you want the built-in VCL to handle TTL and other logic. The built-in runs after your custom code if you do not explicitly return.

Tradeoff: stripping Set-Cookie means the client never receives the cookie for this content. If the cookie is needed for analytics or personalization on these paths, you need a different caching strategy.

Fix Cache-Control headers from the backend

If the backend sends Cache-Control: private or no-store on public content, fix it at the source. Varnish’s builtin.vcl respects these directives, and overriding them in VCL is fragile because it requires re-implementing the cacheability decision logic.

If you cannot fix the backend immediately, you can unset the header in VCL:

sub vcl_backend_response {
    if (bereq.url ~ "^/public-content/") {
        unset beresp.http.Cache-Control;
        set beresp.ttl = 5m;
    }
}

Tradeoff: you are overriding the origin’s explicit caching intent. Document why and revisit when the backend is fixed.

Normalize cookies for Vary: Cookie paths

A Vary: * header makes the response completely uncacheable. Fix this at the backend. For Vary: Cookie, normalize the cookie in vcl_recv before hashing so that only meaningful cookie values affect the cache key:

sub vcl_recv {
    if (req.http.Cookie) {
        # Remove cookies that do not affect the response but inflate the Vary key.
        # The exact cookies to strip are application-specific. Test carefully.
        set req.http.Cookie = regsuball(req.http.Cookie,
            "(^|;\s*)(tracking|analytics|ab_test)=[^;]*", "");
        # Clean up empty segments left behind
        set req.http.Cookie = regsuball(req.http.Cookie, ";\s*;", ";");
        set req.http.Cookie = regsuball(req.http.Cookie, "^[;\s]+|[;\s]+$", "");

        if (req.http.Cookie == "") {
            unset req.http.Cookie;
        }
    }
}

Tradeoff: normalizing cookies is path-specific. Over-aggressive stripping can serve one user’s personalized content to another. Test with real cookie values before deploying.

Clear existing hit-for-miss objects

If the root cause is fixed but hit-for-miss objects persist (up to 120 seconds), ban the affected URLs to force immediate re-evaluation:

# WARNING: this evicts matching objects from cache, causing a temporary
# spike in backend requests until the cache repopulates.
varnishadm ban 'req.url ~ /affected/path/'

For hit-for-pass objects created via return(pass(DURATION)), the pass duration must expire naturally or you must set req.hash_always_miss = true in vcl_recv for the affected requests. Banning may not clear hit-for-pass objects depending on the ban expression and object state.

Prevention

• Monitor hitpass and hitmiss as independent time series. Do not rely on a hit-rate alert to catch their growth. Track the rate (delta per second) and alert on sustained increases above baseline.
• Alert on rate-of-change, not absolute thresholds. A gradual climb from 5% to 15% hitmiss over a week is more actionable than a static threshold that fires only when the problem is severe.
• Track MAIN.beresp_uncacheable if your Varnish version exposes it. It counts backend responses marked uncacheable before they become hit-for-miss objects.
• Audit response headers after every application deploy. A middleware change that adds Set-Cookie to a previously cookie-free endpoint is the most common cause of silent hitmiss growth.
• Review Vary headers in backend responses. Vary: Cookie without cookie normalization in VCL is a common cacheability killer for sites with analytics or A/B testing cookies.
• Watch transient storage. Hit-for-miss and hit-for-pass objects consume transient storage. If SMA.Transient.g_bytes grows monotonically alongside rising hitmiss, you are accumulating memory pressure toward an OOM.

How Netdata helps

• Netdata collects MAIN.cache_hitpass and MAIN.cache_hitmiss at per-second resolution, making gradual climbs visible that would be invisible at 60-second polling intervals.
• Correlating cache_hitmiss rate with MAIN.backend_req rate in the same dashboard reveals whether rising hitmiss is driving backend load growth.
• ML-based anomaly detection flags the rate-of-change in hitmiss counters even when the absolute value remains below a static threshold, catching gradual drift that hit-rate alerts miss.
• The SMA.Transient.g_bytes signal, correlated with hitmiss rate, reveals the memory cost of accumulated uncacheable objects before they cause an OOM.

Related guides

• How Varnish actually works in production: a mental model for operators
• Varnish monitoring checklist: the signals every production cache needs
• Varnish monitoring maturity model: from survival to expert
• Varnish sess_dropped vs req_dropped: HTTP/1 connection drops and HTTP/2 stream drops
• Varnish thread pool exhaustion: workers all busy, queue full, sessions dropped
• Varnish thread_queue_len above zero: requests waiting for a worker
• Varnish threads_failed: the OS refusing to create worker threads
• Varnish threads_limited climbing: hitting thread_pool_max