Memcached evicted_unfetched and expired_unfetched: caching data nobody ever reads

Memcached exposes two counters that most operators never look at, and both describe the same failure: the application is storing data that nobody ever reads. evicted_unfetched counts items evicted from the LRU before they were ever touched by a read. expired_unfetched counts items that lived their full TTL and expired without ever being read. Both are efficiency signals, not reliability signals. They will not tell you the cache is down. They tell you the cache is being used as a write-only buffer, and that write-only data is displacing items that might actually be read.

Under memory pressure, every eviction is a choice about what to discard. If a large fraction of evicted items were never read, the cache threw away memory it could have spent on items that would have been hit. Those now-missing items fall through to the backend.

For the broader model of how memcached manages memory and the slab allocator, see the how memcached works in production guide.

What these counters measure

Both counters are cumulative, monotonic, and available since memcached 1.4.8. They appear in the global stats output and per-slab in stats items.

• evicted_unfetched: items forcibly evicted from the LRU to make room for new sets, where the item was never touched by get, incr, append, or similar read operations after it was stored.
• expired_unfetched: items that reached their TTL and were reclaimed, where the item was never touched by a read operation after it was stored.

The defining word is “unfetched.” A normal cached item is stored, read one or more times, then either evicted or expired. An unfetched item skips the read step entirely: SET, wait, discard. It consumed slab memory for its entire lifetime and returned no value to any client.

Neither counter is a problem indicator on its own. They are meaningful only in ratio to the operations that produce them, and, for evicted_unfetched, only when evictions are actually occurring.

# Read global cache-waste counters
echo "stats" | nc -q1 localhost 11211 | grep -E "STAT (evicted|expired)_unfetched"

How an item becomes unfetched

Every SET inserts an item into the HOT tier of its slab class’s segmented LRU. The HOT/WARM/COLD segmentation has been the default since 1.5.0; the optional TEMP tier is off by default. A background LRU maintainer thread moves items between the active tiers based on access. From insertion, the item follows one of three paths.

1. It is read (GET, incr, append, touch, and so on) at least once before it leaves the cache. It is not unfetched. It contributed to cache effectiveness.
2. It is evicted from the COLD tier before its TTL expires, and it was never read. evicted_unfetched increments.
3. Its TTL expires first (either lazily on access or proactively by the LRU crawler), and it was never read. expired_unfetched increments.

flowchart TD
    SET[SET inserts item into HOT tier] --> READ{Read by GET/incr/append?}
    READ -->|Yes, at least once| NORMAL[Normal cached item]
    READ -->|Never read| COLD[Ages to COLD tier]
    COLD --> TTL{TTL expires first?}
    TTL -->|Yes| EU[expired_unfetched increments]
    TTL -->|No, evicted for space| EVU[evicted_unfetched increments]

The LRU crawler, a background thread refined through 1.5.0, walks the queues and proactively reclaims expired items rather than leaving them to be discovered lazily during SET operations. On builds with the crawler active, expired_unfetched can be substantially higher than on older flat-LRU builds, because the crawler finds and reclaims expired items that would otherwise have sat unread. A high expired_unfetched on a modern build does not by itself indicate a problem. It often just means the crawler is doing its job on a short-TTL workload.

Why this matters, and when it does not

These are efficiency signals. They describe wasted memory, not danger to the service.

evicted_unfetched is the more actionable of the two. When the cache is under memory pressure and evicting, every eviction is a choice about what to discard. If a large fraction of evicted items were never read, the application is storing write-only data that displaces items that might have been hit. The useful items that got evicted to make room are now misses, and those misses fall through to the backend.

expired_unfetched is weaker as an alert. A short-TTL workload (sessions with 60-second TTLs, rate-limit counters, transient dedup keys) will naturally produce a high expired_unfetched count with zero evictions and zero memory pressure. Items expire before anyone asks for them. That can be entirely healthy, or it can be a sign that the application is caching data with no read demand. You need the surrounding context, not the absolute number, to tell which.

Both counters are only meaningful relative to the operations that drive them.

The 0.3 to 0.5 range for evicted_unfetched / evictions is operational experience, not an official memcached threshold. There is no documented number. Treat it as a heuristic: above it, write-only data is a meaningful share of what the cache is throwing away.

Reading the ratio correctly

The single most common mistake is reading the absolute counter. evicted_unfetched is cumulative since process start. A large number on a long-running process means nothing on its own. Compute the ratio against evictions over a recent window, and compute it per-slab before concluding the whole cache is wasteful.

# Global ratio requires two samples
echo "stats" | nc -q1 localhost 11211 | grep -E "STAT (evictions|evicted_unfetched)"
# sample again after a fixed interval, then compute:
#   ratio = delta(evicted_unfetched) / delta(evictions)

# Per-slab cache waste: find which slab class drives the ratio
echo "stats items" | nc -q1 localhost 11211 \
  | grep -E "items:[0-9]+:(evicted|evicted_unfetched|expired_unfetched)"

Two edge cases are worth knowing.

Ratio equals 1.0. If every eviction is of an unfetched item, the cache is storing nothing but write-only data in that slab class. This is the worst case: 100% of evicted memory was waste. Real production examples exist where evicted_unfetched exactly equals evictions for a sustained period.

High evicted_unfetched with zero evictions. The ratio is undefined when the denominator is zero. This can appear transiently as counters lag, or when evictions stop. Do not alert on evicted_unfetched alone. This counter is only meaningful when evictions are occurring.

Per-slab granularity matters because a high global ratio can be driven by one slab class. A slab class holding session blobs that are written and never re-read can produce the entire global evicted_unfetched count, while other classes evict healthy, frequently-read items. Fix the slab class, not the cache. This is the same slab-trap pattern that makes global memory utilization misleading: the aggregate hides the imbalance.

A related trap is alerting on the raw expired_unfetched rate. Short-TTL workloads produce large absolute counts of expired items by design. A session cache with a 60-second TTL and 50,000 writes per second will show a high expired_unfetched rate forever, and most of those expirations are correct behavior. Correlate against hit ratio and backend load before treating it as a defect.

Where this shows up in production

The write-only pattern has a few common origins. Recognizing them is faster than re-deriving the cause from first principles.

Over-eager cache warming. A startup or deploy-time job pre-populates keys that steady-state traffic never requests. The warmed data sits in COLD until it is evicted or expires, having never been read. This is common after deploys that warm a larger key set than production actually touches.

Cache-everything cache-aside. The application writes to cache after every backend read, regardless of whether that key will be read again. For low-cardinality or one-shot data, the cache write is pure overhead. The item lives its TTL or gets evicted, never read. This pattern produces a high cmd_set to cmd_get ratio alongside the unfetched counters.

Session data stored but never retrieved. Some session patterns write to memcached on every request, for audit, for redundancy, or for a fallback path that is rarely exercised, but only read on a cold branch. Most of those writes are unfetched.

Short-TTL write-heavy workloads. Counters, dedup keys, and idempotency tokens with short TTLs are often written and never re-read by design. High expired_unfetched here is expected, not a defect. The signal to watch is whether these items are also being evicted before they expire, which would indicate the slab class is undersized for the write rate.

Large-object slab classes. When a slab class holds very few pages (for example, a class for 394KB items with only a handful of pages assigned), a burst of writes can exhaust the slab before eviction frees enough space. The result is SERVER_ERROR out of memory storing object responses and store_no_memory increments when -M is set, alongside high evicted_unfetched, because items are being evicted faster than they can be read.

Signals to watch

How Netdata helps

Netdata surfaces these counters as per-second metrics, so you see the waste pattern as it develops rather than as a cumulative number you have to difference by hand.

• Per-second evicted_unfetched and expired_unfetched rates against evictions and per-slab item stats let you compute the ratio in context and identify which slab class drives the waste.
• Correlate against hit ratio and backend load to determine whether the write-only pattern is actually hurting effectiveness, or is just noise on a short-TTL workload.
• Anomaly detection on the ratio flags a sudden shift in eviction quality, for example a deploy that introduces a new write-only key pattern, even when the absolute eviction rate is unchanged.
• Alert on cmd_flush increments in the same view, since an accidental flush produces a cold cache that can masquerade as a sudden spike in unfetched items during the re-warm.

Related guides

• Memcached connection refused: telling a dead process from a hung or full one
• How Memcached actually works in production: a mental model for operators
• Memcached monitoring checklist: the signals every production cache needs
• Memcached monitoring maturity model: from survival to expert
• Memcached alive but not responding: the silent process hang