HAProxy SSL session cache misses: full handshakes and lost resumption

Your HAProxy CPU is climbing but connection counts look normal. Idle_pct is dropping, latency on new connections is rising, and existing sessions seem fine. When you pull show info, SslFrontendKeyRate is far above baseline and the SSL session cache numbers tell the story: almost every lookup is a miss.

A high cache miss rate means most TLS clients are paying for a full handshake instead of resuming a previous session. Full handshakes are the most CPU-intensive operation HAProxy performs; resumed handshakes skip most of that cost. When resumption breaks, the load balancer loses a large fraction of its capacity without any change in traffic volume.

What this means

HAProxy maintains an in-process SSL session cache so that a client reconnecting within the session lifetime can resume instead of doing a full key exchange. Two counters in show info track this:

  • SslCacheLookups: how many times clients presented a session ID and HAProxy consulted the cache.
  • SslCacheMisses: how many of those lookups failed to find a usable session.

There is no SslCacheHits field. Hit rate must be computed:

hit_rate = (SslCacheLookups - SslCacheMisses) / SslCacheLookups

Every miss means the client falls back to a full handshake, which shows up directly in SslFrontendKeyRate. The causal chain is: cache miss rate rises, SslFrontendKeyRate rises, Idle_pct falls, and eventually new connections stall while old ones keep working. That asymmetry (existing connections fine, new connections slow) is the signature that separates this from a backend problem. Backends are untouched here: rtime stays normal.

Two things complicate the interpretation:

  1. Reloads flush the cache. The new process starts with an empty cache, so every client does one full handshake after each reload. This is expected. Gate any alerting on Uptime_sec > 600 so the post-reload transient does not page you.
  2. TLS 1.3 changes the accounting. TLS 1.3 resumes via PSK tickets rather than the server-side session ID cache, so the cache counters may show fewer lookups even when resumption is working. A low lookup volume is not the same as a high miss rate.
flowchart TD
  A[Client reconnects with session ticket or ID] --> B{Cache lookup}
  B -->|hit| C[Resumed handshake - cheap]
  B -->|miss| D[Full handshake - expensive]
  D --> E[SslFrontendKeyRate rises]
  E --> F[Idle_pct falls]
  F --> G[New connections stall]
  H[Reload: empty cache] --> B
  I[Cache too small: LRU eviction] --> B
  J[Ticket key rotation] --> B

Common causes

CauseWhat it looks likeFirst thing to check
Post-reload cache flushMiss rate spikes right after a reload, then decays back to baseline within minutesUptime_sec: is the process younger than ~10 minutes?
Cache too small for client diversityPersistently high miss rate at steady state, SslCacheLookups volume hightune.ssl.cachesize vs. your unique-client population and reconnect window
Session lifetime too shortClients that reconnect after the lifetime always misstune.ssl.lifetime (default 300s) vs. typical client reconnect interval
Ticket key rotationMiss spike correlated with a key rotation event, not a reloadRotation schedule, set ssl tls-key usage
HAProxy version bugAll connections show full handshakes regardless of cache configVersion: 2.4.0-2.4.7 broke session ID resumption for TLS 1.2 and below (fixed in 2.4.8)
Genuine client diversityMiss rate always high, never was lowWhether the workload (mobile, CDN, many one-shot clients) ever allowed resumption
TLS 1.3 accountingLookups low, reuse looks bad in cache counters but CPU is fineSslFrontendSessionReuse_pct and actual Idle_pct before concluding anything

Quick checks

All commands are read-only against the runtime socket.

# Pull the SSL cache counters
echo "show info" | socat unix-connect:/var/run/haproxy.sock stdio | grep -E "^(Ssl|Uptime_sec|Idle_pct)"

# Key fields to look at:
#   SslCacheLookups, SslCacheMisses  - compute the hit rate yourself
#   SslFrontendKeyRate               - full handshakes per second
#   SslFrontendSessionReuse_pct      - global reuse percentage
#   Uptime_sec                       - is this a post-reload transient?
#   Idle_pct                         - is the event loop actually saturated?

# Compute the miss rate from two fields in one shot
echo "show info" | socat unix-connect:/var/run/haproxy.sock stdio | \
  awk -F': ' '/^SslCacheLookups:/{l=$2} /^SslCacheMisses:/{m=$2} END {if (l>0) printf "miss rate: %.1f%%\n", m/l*100; else print "no lookups yet"}'

# Check configured cache size and lifetime in the running config
grep -E "tune.ssl.(cachesize|lifetime)" /etc/haproxy/haproxy.cfg

# Confirm whether CPU pressure is real or whether this is only an accounting artifact
echo "show info" | socat unix-connect:/var/run/haproxy.sock stdio | grep -E "^(Idle_pct|SslFrontendKeyRate|SslRate):"

# Verify resumption from the outside (two connections, second should show Reused)
openssl s_client -connect your.frontend:443 -reconnect -no_ticket 2>/dev/null | grep -E "New|Reused"

If every line says “New” and you are on HAProxy 2.4.0 through 2.4.7, you are hitting the known resumption bug fixed in 2.4.8. No configuration change will help; upgrade.

How to diagnose it

  1. Rule out the post-reload transient first. Check Uptime_sec. If the process is under 10 minutes old, the cache was just flushed and misses are expected. Wait and re-measure before changing anything.
  2. Compute the steady-state miss rate. Take two samples of SslCacheLookups and SslCacheMisses a few minutes apart and compute the rate from the deltas. Point-in-time ratios on cumulative counters are distorted by the process’s whole lifetime, including the reload spike.
  3. Check whether CPU is actually suffering. Compare SslFrontendKeyRate against baseline and look at Idle_pct. If the miss rate is high but Idle_pct is comfortable and key rate is within capacity, this is a monitoring observation, not an incident. Note the busy-polling caveat: with busy-polling enabled, Idle_pct is near zero by design, so use show activity run-queue depth instead.
  4. Cross-check with SslFrontendSessionReuse_pct. If cache misses are high but session reuse percentage is healthy, TLS 1.3 PSK resumption is doing the work outside the session ID cache, and the cache counters are misleading you.
  5. Correlate with operational events. Line the miss-rate timeline up with reloads, config pushes, and ticket key rotations (set ssl tls-key). A spike that tracks rotation events is expected invalidation, not a sizing problem.
  6. Verify from the client side. Use openssl s_client -reconnect as above to confirm whether resumption works at all. “Reused” on the second connection means the mechanism works and the misses come from volume, lifetime, or client behavior.
  7. Only then look at sizing. Compare your unique-client population and their reconnect intervals against tune.ssl.cachesize and tune.ssl.lifetime.

Metrics and signals to monitor

SignalWhy it mattersWarning sign
Computed cache hit rate (Lookups - Misses) / LookupsThe direct measure of resumption effectivenessMiss rate > 50% sustained with meaningful lookup volume and Uptime_sec > 600
SslFrontendKeyRateFull handshake rate, the CPU cost driverRising in step with miss rate, trending toward tested capacity
SslFrontendSessionReuse_pctGlobal reuse view that includes paths the cache counters missDropping trend over days or weeks
Idle_pctWhether the extra handshakes actually hurt< 20% sustained; < 5% is saturation. Meaningless with busy-polling
Uptime_secReload detection for gating alerts and interpreting counter resetsAny alert on cache metrics without this gate will false-fire
SslRateOverall TLS session rate for contextDiverging from SslFrontendKeyRate trend indicates resumption loss

Fixes

Post-reload spikes

Do nothing to the cache. This is expected behavior. The fix is in alerting: gate cache-miss and key-rate alerts on Uptime_sec > 600 and require a persistence window. If reloads are frequent (dynamic service discovery), reduce reload frequency, because each one forces a full-handshake wave across your entire client base. Sharing ticket keys across processes via on-disk ticket key files mitigates ticket-based resumption loss across reloads, though the in-process session ID cache still starts empty.

Cache too small or lifetime too short

Raise tune.ssl.cachesize and/or tune.ssl.lifetime in the global section. The cache defaults to 20000 blocks, and sessions with peer certificates consume multiple blocks. The lifetime defaults to 300 seconds, and sessions can be evicted earlier under LRU pressure when the cache fills. Tradeoff: the cache is memory resident, so a large cache is a permanent memory allocation, and a longer lifetime keeps sessions (and their keys) valid longer, which is a security posture decision, not just a performance one. Size for your unique-client population within the reconnect window, not for total connection volume.

Ticket key rotation

Rotation invalidates tickets minted under the old key, so affected clients do one full handshake each. That is the price of rotation and usually worth it. Stagger rotations away from traffic peaks and keep the monitoring gate so the expected spike does not page.

Version bug

If you are on 2.4.0-2.4.7 and openssl s_client -reconnect shows “New” on every connection, upgrade to 2.4.8 or later. No tuning fixes broken resumption code.

Genuine client diversity

If your clients are mostly one-shot (API consumers, webhooks, mobile fleets with long gaps between connections), resumption will never be high and the full-handshake rate is simply your cost of doing business. The fix is capacity: size CPU for SslFrontendKeyRate at peak, prefer ECDHE key exchange over RSA, and do not alert on a miss rate that is normal for this workload.

Prevention

  • Compute the hit rate in your monitoring system. HAProxy does not export hits; derive (SslCacheLookups - SslCacheMisses) / SslCacheLookups from delta rates and alert on it only when Uptime_sec > 600 and lookup volume is meaningful.
  • Trend SslFrontendSessionReuse_pct. A gradual decline is an early warning that CPU cost per connection is silently increasing, weeks before Idle_pct moves.
  • Baseline SslFrontendKeyRate against tested capacity so you know how much headroom a resumption failure would consume.
  • Batch reloads. Fewer reloads means fewer cache flushes and fewer full-handshake waves.
  • Re-verify after upgrades and TLS stack changes. A quick openssl s_client -reconnect check after any HAProxy or SSL library change catches resumption regressions before production traffic does.

How Netdata helps

  • Correlates the causal chain on one dashboard: cache miss rate, SslFrontendKeyRate, and Idle_pct side by side, so you can see in seconds whether misses are actually costing CPU.
  • Detects reloads via Uptime_sec drops and lets you visually separate the expected post-reload miss spike from a steady-state resumption failure.
  • Tracks SslFrontendSessionReuse_pct trends so a slow decline in resumption shows up long before the event loop saturates.
  • Per-second collection catches short spikes that minute-resolution scraping averages away, which matters because post-reload handshake waves are brief.
  • Combines HAProxy TLS signals with system CPU per core, so you can confirm the load is handshake crypto on HAProxy threads and not an unrelated process.