vSphere memory compression: the reclamation tier between balloon and swap

Memory compression is the third tier in ESXi’s four-tier memory reclamation hierarchy. It sits between ballooning, which asks the guest OS to return pages, and host-level swapping, which writes pages to .vswp files on a datastore and destroys latency. Compression is ESXi buying time: keep the page in RAM but in a smaller form, at the cost of CPU cycles and added access latency.

The key point up front: compression is not a tuning exercise. Any sustained non-zero compression rate means the host has exhausted ballooning and is in the zone between degraded and catastrophic.

What it is and why it matters

ESXi overcommits memory through a four-tier reclamation hierarchy, invoked in order of increasing desperation:

  1. Transparent Page Sharing (TPS). Deduplicates identical pages across VMs. Largely disabled by default since vSphere 6.0 for security reasons, so in most modern deployments this tier is effectively absent.
  2. Ballooning. The vmmemctl driver inside the guest is inflated, forcing the guest OS to page internally using its own swap or pagefile. This is the first tier of active reclamation.
  3. Compression. Pages that ballooning could not reclaim are compressed into a per-VM cache that lives inside the VM’s own memory space.
  4. Host-level swapping. VM memory pages are written to .vswp files on the datastore. This is catastrophically slow because the .vswp file competes with VM disk I/O on shared storage.

Compression exists to delay tier 4. Each compressed page avoids a disk I/O but consumes CPU cycles on both compress and decompress, and adds roughly 2-10x DRAM access latency when the guest touches that page. That is a painful tradeoff, but it is orders of magnitude better than touching a .vswp file on shared storage.

The reason operators need to treat compression as a distinct tier rather than background noise is that it is the earliest hypervisor-visible signal that memory pressure has crossed from manageable to dangerous. Ballooning can run for long periods with modest guest impact. Compression cannot. By the time you see sustained ZIP/s in esxtop, the host is already in trouble.

How it works

When ballooning has reached its limit, or cannot run because VMware Tools is absent, the VMkernel begins compressing memory pages instead of swapping them immediately. Each 4 KB page is evaluated for compressibility. Pages that compress well are stored in the compression cache. Pages that fail to compress meaningfully are swapped to disk directly.

flowchart LR
    A["Memory pressure"] --> B["Tier 1: TPS
dedup, off by default"]
    B --> C["Tier 2: Balloon
guest pages internally"]
    C --> D["Tier 3: Compression
per-VM cache"]
    D --> E["Tier 4: Swap
.vswp on datastore"]
    D -.-> F["CPU cost
2-10x DRAM latency"]
    E -.-> G["Disk-speed RAM
catastrophic"]

The compression cache is per-VM and lives inside the VM’s own allocated memory, not in separate host memory. The default maximum cache size is 10% of the VM’s configured memory, controlled by the advanced setting Mem.MemZipMaxPct (range 5-100). Because the cache occupies guest-visible address space, raising Mem.MemZipMaxPct to very high values can paradoxically increase memory pressure by consuming more of the VM’s own RAM for compressed storage rather than freeing host capacity.

When the compression cache fills, ESXi does not grow it dynamically. It evicts the least-recently-used compressed page, decompresses it, and writes it to the .vswp file. At that point, the host has crossed from compression into active swapping. Compression was only a buffer, and the buffer is now full.

Compression can be toggled with the advanced setting Mem.MemZipEnable (1 = enabled, 0 = disabled). It is enabled by default on ESXi hosts from vSphere 4.1 through current releases. Disabling it on a production host is disruptive: it removes the buffer between ballooning and swap entirely, so any pressure that ballooning cannot absorb goes directly to .vswp I/O with no intermediate step.

Where it shows up in production

The compression rate signals are visible in two places:

  • esxtop: the ZIP/s column (compression rate) and UNZIP/s column (decompression rate), per VM in the memory view.
# On the ESXi host (via SSH or DCUI), check per-VM compression rates
esxtop
# press 'm' for memory view
# press 'f' to add ZIP/s and UNZIP/s fields if they are not visible by default
# ZIP/s = pages compressed per second, UNZIP/s = pages decompressed per second
  • vCenter API: mem.compressionRate.average (KBps), mem.decompressionRate.average (KBps), and mem.compressed.average (KB total currently held in cache).

A sustained non-zero ZIP/s means the host is actively compressing pages because ballooning was insufficient. This is a TICKET-level condition: any sustained rate indicates memory pressure beyond what ballooning can handle, and the host is in the zone between degraded and catastrophic.

UNZIP/s (decompression rate) is equally important. Active decompression means the guest is touching pages that were previously compressed. Every decompress costs CPU and adds latency to that memory access. High UNZIP/s alongside high ZIP/s is a particularly bad pattern: the host is compressing and decompressing the same working set repeatedly, burning CPU on both sides without making net progress. This is the compression equivalent of swap thrashing.

The balloon-without-compression trap

If you see swap or compression activity with zero or near-zero balloon, the balloon driver is not functioning. VMware Tools may be absent, the balloon driver may be disabled, or the guest may be unresponsive to inflation requests. The host skipped tier 2 and went straight to tier 3 or 4. This is worse than the raw numbers suggest because ballooning is the gentlest reclamation mechanism. Without it, every pressure spike goes directly to compression and swap.

Poor compression candidates

Compression effectiveness depends on the data itself. Encrypted or highly randomized data compresses poorly. The specific production pattern to watch for: databases with Transparent Data Encryption (TDE), encrypted VMs, or in-memory databases with random data. The VMkernel spends CPU cycles attempting to compress these pages, fails to meet the compression threshold, and swaps them anyway. The compression step was pure overhead.

In environments running TDE databases under memory pressure, you may see elevated host CPU with no corresponding reduction in swap rate. The compression cache is churning through incompressible pages. If your workload is predominantly encrypted or random data, compression provides little benefit and disabling it with Mem.MemZipEnable = 0 removes the wasted CPU, accepting that the host will swap directly when pressure exceeds what ballooning can absorb.

Tradeoffs and when this matters

Compression is a brake, not a solution. It delays swapping but does not prevent it. The cache has a fixed ceiling, and once it fills, swap begins regardless. The operational question is never “should we tune compression” but “why is the host under enough pressure to need it at all.”

FactorImplication
CPU costEvery compress and decompress consumes pCPU cycles. On an already CPU-constrained host, compression adds a secondary tax.
Access latencyCompressed pages cost roughly 2-10x DRAM latency on access due to decompression. Faster than disk swap, slower than uncompressed RAM.
Cache ceilingDefault 10% of VM memory. When full, LRU eviction decompresses and swaps. Compression only delays swap.
Incompressible dataEncrypted, random, or already-compressed data wastes CPU before swapping anyway.
Guest memory footprintThe cache lives inside the VM’s own memory space. Raising Mem.MemZipMaxPct consumes more guest-visible RAM.

Correlating with the rest of the cascade

Compression never appears in isolation. Its diagnostic value comes from correlating it with the adjacent tiers:

  • Balloon should be at or near maximum when compression activates. If balloon (mem.vmmemctl.average, or MCTLSZ in esxtop) is low or zero but compression is active, the balloon driver is not functioning. This is the Tools-absent scenario described above.
  • Swap follows if compression is insufficient. Watch mem.swapinRate.average and mem.swapoutRate.average alongside compression. The moment swap-in rate goes above zero, the compression cache has filled and pages are being recalled from disk. This is the cliff edge.
  • Host CPU rises with compression activity. Compression and decompression consume pCPU. If you see host CPU climbing alongside ZIP/s and UNZIP/s, some of that CPU is the compression tax, not application demand. This matters when deciding whether to migrate VMs or add host memory: you may be closer to CPU contention than the guest-level numbers suggest.

The most dangerous misread is treating compression as normal background activity that a well-overcommitted host just does. A host that sustains compression for minutes is a host that is one cache-fill away from swapping, and swapping is where the death spiral begins.

Signals to watch in production

SignalWhy it mattersWarning sign
mem.compressionRate.average (ZIP/s)Active compression means pressure beyond ballooningAny sustained non-zero rate
mem.decompressionRate.average (UNZIP/s)Guest touching compressed pages; CPU and latency costHigh UNZIP/s alongside high ZIP/s means churning working set
mem.compressed.averageTotal pages currently held in the compression cacheApproaching 10% of VM memory means cache near full, swap imminent
mem.vmmemctl.average (balloon)Must be near max when compression activatesLow or zero balloon with active compression means Tools absent
mem.swapinRate.averagePages recalled from .vswp means compression failed to absorbAny sustained non-zero rate is an active performance emergency
Host CPU utilizationCompression consumes pCPU cyclesCPU climbing with no application change may be the compression tax

How Netdata helps

  • Per-second ZIP/s and UNZIP/s catches the transition from transient spike to sustained pressure before the cache fills and swap begins. Five-minute vCenter rollups average away the exact moment the cascade escalates.
  • Single-view correlation of balloon, compression, and swap rates shows whether balloon is at max, whether compression is absorbing the overflow, and whether swap has started despite compression.
  • Anomaly detection on compression rate flags the first sustained departure from zero, the earliest hypervisor-visible signal that a host has crossed from manageable pressure into the danger zone.
  • Host CPU alongside compression metrics separates the compression tax from genuine application demand, which matters when deciding whether to migrate VMs or add host memory.
  • Composite alerting across the memory cascade, from balloon through compression to swap, avoids the common failure of alerting only on swap, which fires after the damage is already done.