Envoy memory_heap_size vs memory_allocated: tcmalloc fragmentation that fools dashboards

Envoy exposes three memory gauges that operators naturally try to map onto process RSS: server.memory_allocated, server.memory_heap_size, and server.memory_physical_size. On a busy proxy the first two diverge by 2x-4x routinely, and dashboards that graph heap_size look like a slow leak even when nothing is wrong.

The gap is not a bug. Envoy ships with tcmalloc as its default allocator, and tcmalloc is built for allocation latency, not for prompt return of freed memory to the OS. It parks freed pages in per-thread caches, central caches, and the page heap free lists so the next allocation is fast. Those pages stay mapped into the process, counted in memory_heap_size, and very often still resident in RSS.

What it is and why it matters

server.memory_allocated and server.memory_heap_size both come from tcmalloc’s MallocExtension. memory_allocated reflects bytes currently handed out to Envoy’s own data structures: connection buffers, stats storage, config caches, filter state, the deferred-delete list. memory_heap_size reflects the total bytes tcmalloc has reserved from the OS, which includes everything in memory_allocated plus everything tcmalloc has freed at the application layer but is holding onto internally.

The ratio memory_allocated / memory_heap_size is the fragmentation signal. Low ratios are normal. A heap that is several times the size of allocated is not pathology, it is tcmalloc holding the working set hot so it does not have to call back into the kernel on the next burst. The operational signal you want is growth of memory_allocated itself without a corresponding traffic change. That, not the ratio, is what indicates a leak or unbounded accumulation.

server.memory_physical_size exists to give operators a number that is closer to actual RSS. It maps to tcmalloc’s generic.total_physical_bytes and includes allocator overhead the kernel still counts as resident. Even memory_physical_size is not a perfect RSS proxy because some allocation paths in Envoy bypass tcmalloc. For process RSS itself, read /proc/<pid>/status or cgroup memory.current.

How tcmalloc holds freed memory

When Envoy frees an object, the bytes do not go back to the kernel. They walk through several layers of tcmalloc internal caches first, and each layer has different implications for RSS.

flowchart TD
    APP[Envoy frees object] --> TC[Per-thread cache]
    TC --> CC[Central cache]
    CC --> PHF[Page heap free list: mapped, in RSS]
    PHF -. madvise DONTNEED .-> PHU[Page heap unmapped: virtual only]
    PHU -. slow release .-> OS[Returned to OS]

• Per-thread caches: each worker thread has its own fast bins, and most frees return here. Bytes in thread caches are still mapped and still in RSS.
• Central cache: when a thread cache fills, it spills into a process-wide central cache that other threads can pull from. Still mapped, still resident.
• Page heap free list: when spans of pages are fully free, they sit in the page heap free lists, classified by size class. Mapped, resident, ready for reuse.
• Page heap unmapped: eventually tcmalloc may madvise(MADV_DONTNEED) a span. The virtual range stays reserved but the kernel is free to reclaim the physical pages.
• Returned to OS: the final step, which only the unmapped path leads to.

Only the last step reduces RSS, and tcmalloc does it on its own schedule. The practical effect is that a workload which allocates a burst of memory (loading many routes, a large config push, a body-buffering event) and then frees it will see memory_allocated drop, memory_heap_size stay flat, and RSS drift down slowly for minutes afterward.

The /memory admin endpoint shows the breakdown directly:

# Inspect tcmalloc internal breakdown
curl -s http://localhost:9901/memory | jq .

The fields returned are:

pageheap_free + total_thread_cache + allocated is roughly the gap between heap_size and what tcmalloc could plausibly return. Watching those three fields while you load and unload clusters tells you whether the gap is mostly thread caches (will drift down over time) or pageheap_free (will linger).

Where it shows up in production

The fragmentation gap is most visible in deployment shapes that produce allocation spikes followed by quiet periods.

Sidecar meshes with frequent xDS churn. Each cluster add or remove drives a burst of small allocations: cluster metadata, host state, per-worker connection pool state, route table entries. Removing the cluster frees the objects, but the spans they lived in are not necessarily returned. Operators running Istio at scale with services deploying and removing continuously see heap_size march upward over hours or days even though memory_allocated tracks traffic. This is the workload shape most likely to fool a dashboard.

Edge proxies handling large buffered bodies. A buffer filter, or an ext_authz configuration that buffers the full request body, allocates a large contiguous region for the duration of the request. Under burst traffic many such regions exist simultaneously. When the requests complete the memory is freed, but heap_size and RSS will reflect the high-water mark for a while.

Config reloads and hot restart. Loading a new config, even one that ends up smaller, allocates the new graph before the old one is released. Hot restart runs two Envoy processes side by side. During the handover window RSS roughly doubles as both processes hold their heaps. This is expected, but if your alert is a fixed threshold on RSS it will fire every deploy.

Lua and Wasm filters. Lua extensions in particular may manage some allocations through their own runtime rather than tcmalloc , so memory used by Lua may appear in process RSS without being fully reflected in memory_allocated or memory_heap_size. If RSS is climbing and Envoy’s memory stats are flat, look at filter-level allocations before assuming the stat is wrong.

Common misuses

Alerting on memory_heap_size directly. This is the mistake the title is about. A threshold like “page if heap_size > 1GB” will fire on a perfectly healthy proxy that has seen a normal burst in the last hour. The signal you want is memory_allocated growth without traffic growth.

Graphing heap_size and allocated on the same panel without context. Two lines diverging by 3x looks alarming to anyone who has not seen tcmalloc behavior before. Either graph the ratio (allocated / heap_size) alongside the raw bytes, or graph allocated and memory_physical_size, and document that heap_size is allocator working set, not memory pressure.

Treating shrink_heap as a fix. The overload manager exposes a shrink_heap action that calls into tcmalloc’s release path . It is worth enabling as a soft pressure-relief mechanism, but it is not guaranteed to reduce RSS . Treat it as best-effort, not as a way to lower your alert thresholds.

Assuming memory_physical_size is RSS. It is closer than heap_size, but it still reflects tcmalloc-tracked allocations. Filter runtimes, segment allocations from other allocators, and any third-party allocator linked into the process are not in it. For cgroup-bounded workloads, watch memory.current on the cgroup, not Envoy’s internal stat.

Blaming Envoy for sidecar OOMs without per-process breakdown. In a sidecar pod the kernel kills the whole pod when memory.current hits the limit. The OOM is often blamed on Envoy because heap_size looks high, while the real consumer was the application. Always look at per-process RSS inside the pod before assigning blame.

Signals to watch in production

The two signals worth alerting on are server.memory_allocated sustained growth against traffic, and memory_physical_size (or cgroup memory.current) approaching the limit. The ratio is a dashboard annotation, not an alert.

Working with the allocator, not against it

If the fragmentation gap is genuinely causing operational pain (RSS so high you cannot pack pods the way you want), there are a few real levers.

Configure the overload manager deliberately. The overload manager uses the envoy.resource_monitors.fixed_heap resource monitor, which depends on max_heap_size_bytes. If it is unset or set too high, the overload manager never triggers and Envoy goes straight from “fine” to OOM with no graceful degradation. Setting max_heap_size_bytes deliberately, with shrink_heap at the lower threshold and stop_accepting_requests at the upper threshold, gives Envoy a self-protection path. This is a capacity decision, not a fragmentation fix.

Investigate jemalloc as an alternative allocator. Operators running Envoy builds that link jemalloc instead of tcmalloc have reported lower fragmentation and lower RSS for the same workload, particularly on workloads with heavy allocation churn. This is a build-time decision; you cannot swap allocators at runtime.

Reduce churn at the source. The biggest lever is usually reducing how often you allocate and free large graphs of small objects. For xDS-driven deployments this means less frequent, larger config pushes instead of many small pushes. For body-buffering workloads it means streaming instead of buffering where possible.

Force kernel reclaim under cgroups. On cgroup v2 you can write a byte count to memory.reclaim to nudge the kernel into reclaiming pages. This is a workaround, not a fix, and it does not help if the pages are still mapped in tcmalloc’s pageheap_free.

How Netdata helps

• Netdata’s per-second collection of server.memory_allocated, server.memory_heap_size, and server.memory_physical_size makes the fragmentation ratio visible as its own chart. Plotting the ratio next to the raw bytes makes “low ratio is normal” obvious to anyone reading the dashboard.
• ML anomaly detection on memory_allocated separates genuine growth (the leak signal) from the normal sawtooth of allocator behavior. Alerts fire on the in-use signal, not on heap_size.
• Correlating memory_allocated with downstream_cx_active, upstream_cx_active, and total stat count tells you whether growth is connection-driven, traffic-driven, or cardinality-driven. Three different root causes, three different fixes.
• The overload manager action gauges appear alongside the memory stats. When shrink_heap or stop_accepting_requests flips active, the memory chart is one click away.
• Cgroup-level memory.current and memory.max from the same Netdata node let you compare Envoy’s self-reported memory to what the kernel will actually OOM on. In sidecar deployments this is the comparison that matters.
• Hot restart epoch and server.state transitions are collected per second, so memory spikes during drain are easy to identify and exclude from baseline calculations.

Related guides

• Envoy 502 and upstream resets: rx_reset, tx_reset, and mid-response failures
• Envoy 503 with response flag UO: a tripped circuit breaker, not a dead backend
• Envoy 504 upstream timeout: upstream_rq_timeout, per-try timeouts, and the UT flag
• Envoy circuit breaker open: cx_open, rq_pending_open, and fast-failed requests
• Envoy clusters stuck warming: warming_clusters non-zero and routes returning 503
• Envoy connection pool exhaustion: a slow upstream that fills the pool
• Envoy control_plane.connected_state = 0: running on stale xDS config
• Envoy downstream_rq_time high: client-observed latency and proxy overhead
• Envoy file descriptor exhaustion: the FD cliff that refuses every new connection
• Envoy health checks vs outlier detection: two systems that eject hosts differently
• How Envoy actually works in production: a mental model for operators
• Envoy listener_create_failure: a listener config Envoy could not apply