Redis monitoring maturity model: from survival to expert

Redis can return PONG while fragmentation doubles RSS, a replica falls behind, or a slow command wedges the event loop. This guide maps four cumulative operator levels derived from production runbooks. Level 1 is liveness and memory limits. Level 2 is workload anomalies and capacity pressure. Level 3 is leading indicators for composite failures. Level 4 is forensic depth. Do not skip Level 1 because you are collecting allocator statistics.

Use this as a coverage checklist for existing deployments or a roadmap for new ones.

flowchart TD
    L1[Level 1: survival - liveness and memory limits]
    L2[Level 2: operational - throughput and eviction rates]
    L3[Level 3: mature - latency events and replication quality]
    L4[Level 4: expert - allocator and key-space forensics]
    L1 --> L2
    L2 --> L3
    L3 --> L4

Level 1: survival

Binary signals. If any fail, you have an immediate incident.

If PING times out, check INFO server before assuming the process is gone; a blocked event loop may still answer metadata commands. If maxmemory is 0, Redis grows until the OOM killer terminates it. If loading stays 1 longer than your baseline, investigate disk I/O or dataset corruption. Correlate rdb_last_bgsave_status:err with rdb_last_save_time; a status of ok with a stale timestamp means saves are not triggering.

Level 2: operational

Rate-based signals that distinguish “Redis is up” from “Redis is keeping up.”

Calculate cache hit rate as keyspace_hits / (keyspace_hits + keyspace_misses). A rate below your baseline after warmup indicates evictions or a pattern shift. Watch evicted_keys as a rate, not an absolute. Cache workloads tolerate some eviction; persistent workloads treat any eviction as data loss. When maxmemory-policy is noeviction, watch total_error_replies instead: writes fail with OOM errors while evicted_keys stays flat. A simultaneous climb in evicted_keys, keyspace_misses, and instantaneous_ops_per_sec signals a memory pressure spiral.

Level 3: mature

Internal latency decomposition, replication quality, and security signals. These catch composite patterns such as fork-induced cascades, backlog overflow loops, and memory pressure spirals.

LATENCY LATEST requires latency-monitor-threshold > 0; the default is 0, which disables the subsystem. On replicas, master_link_status:up does not mean the replica is caught up: check offset lag. In Cluster mode, cluster_state:ok with cluster-require-full-coverage no masks partial failures, so monitor cluster_slots_fail and cluster_slots_pfail directly. If blocked_clients climbs, identify blocking consumers with CLIENT LIST and check producer health.

WARNING: On instances with thousands of connections, CLIENT LIST can stall the event loop.

Level 4: expert

Forensic signals that separate allocator noise from real leaks, identify resource-heavy keys, and catch encoding downgrades that silently inflate memory.

WARNING: redis-cli --bigkeys scans the full keyspace and can impact production latency. Run it against a replica or during low-traffic windows. MEMORY USAGE on large aggregate keys can block; test on a replica first.

Detect encoding downgrades by sampling with OBJECT ENCODING. A ziplist that transitions to hashtable for small hashes is a common memory surprise. Set vm.overcommit_memory to 1 so fork() does not fail, and keep vm.swappiness near 0. Disable Transparent Huge Pages; they are the most common cause of fork latency spikes.

How Netdata helps

Netdata derives rates from cumulative counters such as evicted_keys, sync_full, and total_commands_processed, preventing the common mistake of graphing raw absolutes. It charts used_memory, used_memory_rss, and mem_fragmentation_ratio together, making swap and fragmentation visible before the OOM killer fires.

It tracks latest_fork_usec alongside rdb_bgsave_in_progress and used_memory_rss to surface fork-induced latency cascades as composite events rather than isolated spikes. It tracks replication offset lag per replica and warns when lag approaches the configured repl-backlog-size, giving runway before a full resync triggers.

It surfaces slowlog entries and LATENCY LATEST events next to throughput drops so you can distinguish event loop blocking from traffic spikes. It ingests CLIENT LIST to visualize per-client output buffer memory, catching slow Pub/Sub subscribers and forgotten MONITOR sessions before they OOM the instance.