Varnish transient storage OOM: the unbounded memory path that kills the process

Varnish dies. The OOM killer takes it. No panic, no crash, no VCL error in the logs. Process RSS was far above your configured -s malloc,8G storage, and the kernel reclaimed memory the only way it knows how. This is the transient storage OOM path: the number-one missed signal in Varnish operations.

The problem is structural. Varnish has two storage paths: your configured cache storage (malloc or file, capped at a known size) and transient storage. Transient storage holds objects that will never be cached: pass responses, hit-for-pass, hit-for-miss, and piped traffic. By default, transient storage uses malloc with no upper limit. A pass storm, a batch of large uncacheable responses, or a slow leak over weeks can push process RSS past the configured storage size and into system memory the kernel will not let you keep.

The primary storage counters (SMA.s0.g_bytes, SMA.s0.g_space) show a healthy cache. The transient path is a separate accounting that, on older versions, is invisible in varnishstat output entirely. The cache looks full but fine, and then the process is gone.

What this means

flowchart TD
    A["Client request"] --> B{"VCL: cacheable?"}
    B -->|Yes: hit or miss| C["Primary storage
-s malloc,SIZE"]
    B -->|No: pass or hit-for-miss| D["Transient storage
unbounded malloc"]
    C -->|Full: LRU evict| E["Hit rate drops
backend load up"]
    D -->|Grows without limit| F["RSS climbs
past configured size"]
    F --> G["OOM killer
terminates child"]
    G --> H["Management process
restarts child"]
    H -->|Cache lost| I["Cold cache
backend flood"]

Uncacheable traffic enters transient storage, which has no cap, and RSS climbs until the OOM killer fires. When the management process restarts the child, the cache is empty, sending a flood of misses to backends on top of whatever caused the pass storm.

Transient storage is not a bug. It is how Varnish handles objects that should not enter the normal cache: responses marked uncacheable by VCL (return(pass)), hit-for-pass and hit-for-miss objects (Varnish cached the decision not to cache), and short-lived objects whose TTL is below the shortlived runtime parameter threshold. Each consumes transient storage memory, and none of it counts against your primary storage allocation.

The default transient backend is a malloc stevedore with no size limit. On Varnish 6.1 and later, you can cap it explicitly. Before that, there is no clean way to limit it.

Common causes

Quick checks

# Get the child process PID (higher PID is typically the child)
# On most setups: management starts first, child is forked second
CHILD_PID=$(pgrep -n varnishd)

# Check child RSS in KB
ps -p "$CHILD_PID" -o rss=

# Check transient storage counters if exposed (Varnish 6.1+)
varnishstat -1 -f 'SMA.Transient.*'

# Check pass rate and hit-for-pass/hit-for-miss activity
varnishstat -1 -f MAIN.cache_hitpass -f MAIN.cache_hitmiss

<!-- TODO: verify MAIN.s_pass is a valid counter name in varnishstat -->

# Check what traffic is being passed right now
varnishlog -q 'VCL_call eq "PASS"' -g request

# Check for OOM kills in system logs
dmesg | grep -i oom | grep -i varnish
journalctl -u varnish --since "1 hour ago" | grep -i "child"

# Check child process stability and restart history
varnishstat -1 -f MGT.uptime -f MAIN.uptime

<!-- TODO: verify MGT.child_died and MGT.child_panic are valid counter names -->

# Check storage configuration from process arguments
# Note: -P flag is GNU grep only; use -E on BSD/Alpine
grep -oE '\-s [[:space:]]*\S+' /proc/$CHILD_PID/cmdline 2>/dev/null || tr '\0' ' ' < /proc/$CHILD_PID/cmdline | grep -oE '\-s \S+'

RSS is the most important signal. If RSS is significantly above your configured storage size, the difference is transient storage plus overhead (thread stacks, workspace memory, VSM segment, allocator fragmentation). On pre-6.1 versions where SMA.Transient.* counters are not exposed, RSS is your only direct window into transient consumption.

How to diagnose it

1. Confirm the OOM path. Check dmesg for OOM killer entries targeting the Varnish child process. Check MGT.child_died or MGT.child_panic counters in varnishstat. If MAIN.uptime is much smaller than MGT.uptime, the child has been restarting. An OOM kill shows up as MGT.child_died (child exited on a signal), not MGT.child_panic.

2. Measure RSS vs configured storage. Get the child process RSS and subtract your configured malloc storage size. The remainder is transient storage plus overhead. A well-tuned Varnish should have RSS at roughly configured storage plus 20-30% for thread stacks, workspaces, VSM, and allocator overhead. If the remainder is much larger or growing, transient storage is the problem.

3. Check transient counters. On Varnish 6.1+, SMA.Transient.g_bytes shows bytes currently held in transient storage. On older versions, this counter may not be present. If present and growing, you have confirmed the path. SMA.Transient.g_space reads 0 when transient is unbounded, which is itself diagnostic: it confirms there is no cap.

4. Identify the pass traffic. Run varnishlog -q 'VCL_call eq "PASS"' to see which requests are being passed. Look for patterns: a specific URL prefix, requests with cookies, large Content-Length responses. Correlate with MAIN.cache_hitpass and MAIN.cache_hitmiss rates. If cache_hitpass is climbing, the application is emitting response headers that make content uncacheable.

5. Check for the slow leak. If there is no pass storm but RSS is still growing, the cause may be malloc fragmentation in transient storage or a slow accumulation of hit-for-miss objects. Compare RSS readings over days or weeks. A monotonic upward trend with no corresponding traffic increase points to fragmentation or a leak in the transient object lifecycle.

6. Check VCL for accidental pass. A common mistake is leaving debugging VCL in production that passes traffic under certain conditions. Review vcl_recv and vcl_backend_response for return(pass) calls broader than intended. Also check whether Set-Cookie headers from the backend are causing Varnish to mark responses uncacheable.

Metrics and signals to monitor

Fixes

Cap transient storage (Varnish 6.1+)

The most direct fix. Configure transient storage with a hard limit at startup:

# Add to varnishd startup arguments
-s Transient=malloc,1G

This caps transient storage at 1GB. When transient is full, Varnish cannot allocate space for pass response bodies, producing fetch errors and potentially HTTP 503 responses for pass traffic. It prevents the OOM kill that takes down the entire process along with the full cache.

The tradeoff is explicit: bounded transient storage means pass traffic may fail when the cap is reached. The alternative is an unbounded path that kills the process. Choose the bounded failure mode.

Size the cap based on observed transient usage during normal operation plus headroom for spikes. Monitor SMA.Transient.g_bytes after capping to confirm the cap is adequate.

Reduce pass traffic

Fix the root cause: why is traffic being passed instead of cached?

• Application cookies on cacheable content: Strip cookies in vcl_recv for cacheable URL patterns, or fix the application to not send Set-Cookie on public content. Most common cause of unexpected pass traffic.
• Overly broad Vary headers: A Vary: Cookie or Vary: * header makes responses uncacheable under many conditions. Normalize or remove unnecessary Vary headers at the backend or in VCL.
• Debugging VCL left in production: Remove return(pass) calls added for debugging. Audit every pass path in the VCL to confirm it is intentional and bounded.
• Large objects that should not be cached: Consider whether these need to go through Varnish at all. Pipe mode (return(pipe)) bypasses the object store entirely, though large pipe bodies still consume socket buffer memory.

Address the slow RSS creep

If RSS grows slowly over weeks with no pass storm, the cause is likely malloc fragmentation in the transient allocator or gradual accumulation of short-lived objects not being freed efficiently. Options:

• Cap transient storage as described above. A capped transient backend forces eviction rather than unbounded growth. This is the durable fix.
• Upgrade Varnish if running an older version. The 6.0.x and later series include fixes for transient storage object lifecycle issues.
• Schedule periodic child restarts during low-traffic windows as a last-resort stopgap. This resets transient storage and defragments the allocator but loses the entire cache. Use only while working on the durable fix.

Prevention

• Monitor process RSS as a primary metric. Alert when RSS exceeds configured storage size plus 25-30% overhead. This catches both acute pass storms and slow fragmentation-driven creep.
• Monitor SMA.Transient.g_bytes if your Varnish version exposes it. Alert on monotonic growth or sharp spikes above baseline.
• Monitor MAIN.cache_hitpass and MAIN.cache_hitmiss rates independently. A gradual increase means more traffic is flowing through the transient path. Investigate before it becomes a storage problem.
• Cap transient storage on 6.1+. Even if normal traffic does not cause OOM, a future traffic change or application deploy could trigger a pass storm. A cap converts an unbounded failure into a bounded one.
• Audit VCL regularly for pass paths. Every return(pass) is a transient storage consumer. Ensure each one is intentional and that traffic volume through each pass path is understood.
• Review beresp.grace settings in vcl_backend_response. If grace is set globally before the uncacheable check, pass objects may be retained in transient storage longer than expected. Consider setting beresp.grace = 0s on the uncacheable branch.

How Netdata helps

Netdata’s per-second metrics collection makes the slow RSS creep visible long before it becomes an OOM kill. The signals worth correlating:

• Process RSS collected per second against configured storage size. A monotonic RSS trend over hours or days is the leading indicator, and per-second resolution catches the inflection point where growth accelerates.
• Varnish counters via the built-in collector: SMA.Transient.g_bytes, MAIN.cache_hitpass, MAIN.cache_hitmiss, and MGT.child_* are collected automatically. Correlating a hitpass rate spike with an RSS increase confirms the transient storage path within seconds rather than minutes.
• ML-based anomaly detection on RSS and transient storage counters. The slow weeks-long fragmentation creep has a characteristic signature that static threshold alerts miss but anomaly detection flags early.
• Child process restart detection via MGT.child_start, MGT.child_died, and MGT.child_panic. Correlating these with system-level OOM events and RSS trends distinguishes a transient storage OOM from a VCL panic or VMOD crash.
• Composite dashboards that overlay cache hit rate, pass rate, backend request rate, and process RSS on the same timeline. The pattern is visually obvious: pass rate increases, RSS follows with a lag, then the child restarts and the cycle repeats.

Related guides

• Varnish Error 503 Backend fetch failed: what the error page actually means
• Varnish backend_fail, backend_unhealthy, and backend_busy: three different backend problems
• Varnish backend connection reuse low: keepalive not working and slow TTFB
• Varnish backend probe configuration: threshold, window, interval, and initial
• Varnish backend is sick: health probes, all-backends-sick, and grace
• Varnish cache hit ratio dropped: hit rate collapse and backend overload
• Varnish cache stampede: a popular object expires and the herd hits the backend
• Varnish ESI errors: broken pages and workspace pressure from Edge Side Includes
• Varnish fetch_failed: backend connected but the fetch broke
• Varnish grace masking a backend outage: the ticking-clock incident
• Varnish Guru Meditation: reading the XID and tracing the failing request
• Varnish cache_hitpass / cache_hitmiss climbing: uncacheable content bleeding to the backend