Redis Operations Guides on Netdata

How Redis actually works in production: a mental model for operators

Mon, 01 Jan 0001 00:00:00 +0000

How Redis actually works in production: a mental model for operators

Production incidents do not come from the Redis API. They come from invisible internals: a fork that doubles memory usage, a replication backlog that wraps around, a single slow command that freezes every client. You cannot debug a cascading failure at 3 a.m. without knowing which abstractions compete for which resources.

What it is and why it matters

Redis is a single-threaded event loop around an in-memory dataset. Every incident traces back to resource competition inside one process: memory consumed by the dataset, client buffers, replication backlogs, and allocator fragmentation; CPU consumed by command execution, active expiry, and defragmentation; disk I/O consumed by AOF fsync and RDB snapshots; and network bandwidth consumed by replication and Pub/Sub fan-out.

MISCONF Redis is configured to save RDB snapshots - what it means and how to fix it

Mon, 01 Jan 0001 00:00:00 +0000

MISCONF Redis is configured to save RDB snapshots - what it means and how to fix it

Applications see MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk on every write. Reads still work, but SET, HSET, LPUSH, and all mutating commands are rejected.

Redis makes itself read-only when the last background save failed and stop-writes-on-bgsave-error is yes (the default). The error persists until a subsequent BGSAVE succeeds. Retrying writes will not help. The instance is protecting you from accepting writes that can never be persisted. To recover, fix the underlying persistence failure and clear the error state with a successful save.

Redis and Transparent Huge Pages: why THP must be disabled

Mon, 01 Jan 0001 00:00:00 +0000

Redis and Transparent Huge Pages: why THP must be disabled

Redis latency spikes during background saves, AOF rewrites, or replica full resyncs often trace to a frozen main thread. Clients time out. Replicas disconnect. If the write rate is high, the next reconnection triggers another fork, and the cascade repeats. One common root cause is Transparent Huge Pages (THP), enabled by default on most Linux distributions. Redis detects THP at startup and logs a warning, but provisioning automation often buries it. The impact is severe: THP can increase fork latency by 10 to 100 times by amplifying copy-on-write memory traffic.

Redis aof_last_write_status:err: AOF write failures and recovery

Mon, 01 Jan 0001 00:00:00 +0000

Redis aof_last_write_status:err: AOF write failures and recovery

INFO persistence showing aof_last_write_status:err means Redis failed to flush its Append-Only File buffer to disk on the last attempt. If you depend on AOF for durability, the instance is no longer persisting writes. With the default appendfsync everysec, Redis logs the failed fsync and retries, but once aof_last_write_status is err and stop-writes-on-bgsave-error is enabled, the server rejects mutations. The error returned to clients references RDB snapshots even when AOF is the actual failure, which often misleads first-line diagnosis.

Redis appendfsync always latency: durability vs throughput trade-offs

Mon, 01 Jan 0001 00:00:00 +0000

Redis appendfsync always latency: durability vs throughput trade-offs

Redis AOF persistence logs every write to disk, but the appendfsync policy controls how aggressively Redis forces that log to physical storage. That choice defines whether a crash loses zero, one, or sixty seconds of data, and whether a single slow disk seek can freeze the entire event loop.

appendfsync always on slow storage degrades throughput more severely than dataset growth. appendfsync no on a critical ledger silently exposes you to massive data loss.

Redis big keys: finding the giant key that blocks the event loop

Mon, 01 Jan 0001 00:00:00 +0000

Redis big keys: finding the giant key that blocks the event loop

Application latency spikes while redis-cli PING still returns PONG. Simple GET commands take hundreds of milliseconds. Aggregate used_memory looks stable, instantaneous_ops_per_sec drops, and the slowlog grows. The culprit is often a single oversized key: a sorted set with millions of elements, a hash with millions of fields, or a list fetched with an unbounded range. Redis executes commands sequentially on one main thread; an O(N) command on a giant key blocks every other client until it completes. This guide shows how to find that key and fix it without restarting Redis.

Redis blocked_clients growing: dead consumers vs healthy queues

Mon, 01 Jan 0001 00:00:00 +0000

Redis blocked_clients growing: dead consumers vs healthy queues

blocked_clients in INFO clients is climbing. In queue-based architectures this is often normal: workers call BLPOP, BRPOP, or XREAD BLOCK and wait for producers to push work. When blocked_clients grows while queue depth also grows, consumers are no longer consuming. They may have crashed, been OOM-killed, or stalled on replication lag via WAIT.

blocked_clients counts only clients waiting on explicit blocking commands. It does not capture clients stalled by slow commands like KEYS * or large SMEMBERS. A high value is either a healthy signal of an active queue pattern or a pathological signal of dead connections holding slots open forever, especially with timeout 0.

Redis BUSY Redis is busy running a script: blocking Lua and how to recover

Mon, 01 Jan 0001 00:00:00 +0000

Redis BUSY Redis is busy running a script: blocking Lua and how to recover

redis-cli returns (error) BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE. Normal commands stall. Redis is not down, but it might as well be: a Lua script is holding the single event loop hostage and will not yield until it finishes or you intervene.

Redis executes EVAL and EVALSHA atomically. While a script runs, no other command processes. Once execution exceeds lua-time-limit (default 5000 ms), Redis replies with BUSY to other clients. The script itself continues until it finishes, is killed, or the server shuts down. Whether the script has already performed writes determines whether you can kill it safely or must choose between waiting and a hard shutdown.

Redis Can't save in background: fork: Cannot allocate memory - diagnosis and fix

Mon, 01 Jan 0001 00:00:00 +0000

Redis Can’t save in background: fork: Cannot allocate memory - diagnosis and fix

Redis logs Can't save in background: fork: Cannot allocate memory. free -h shows plenty of free RAM, yet BGSAVE or BGREWRITEAOF fails. If stop-writes-on-bgsave-error is yes (default), writes fail too. The gap between free RAM and fork failure is the key.

This is not a simple OOM. It is a kernel commit charge failure. Linux fork() must account for the worst case where every copy-on-write page is modified. With vm.overcommit_memory=0 (the default), the kernel enforces a heuristic commit limit. When Redis RSS is large, that limit blocks fork() even with free physical memory. The fix is usually one sysctl, but THP, container limits, and actual RAM headroom determine whether it holds.

Redis client output buffer overflow: slow consumers and client-output-buffer-limit

Mon, 01 Jan 0001 00:00:00 +0000

Redis client output buffer overflow: slow consumers and client-output-buffer-limit

Redis memory climbs faster than the dataset justifies. used_memory approaches maxmemory, keys evict, or the process is OOM-killed, yet the keyspace has not grown. Logs show “scheduled to be closed ASAP for overcoming of output buffer limits,” or clients vanish and reconnect. The culprit is usually a slow consumer that cannot drain its output buffer as fast as Redis fills it. A forgotten MONITOR session or an application that left a socket open but stopped reading are the textbook cases. Redis allocates client output buffers from the main heap; unread response data counts against maxmemory. The default client-output-buffer-limit normal 0 0 0 leaves normal clients unbounded, turning one slow reader into a memory leak that can kill the instance.

Redis cluster bus port blocked: the port+10000 firewall gotcha

Mon, 01 Jan 0001 00:00:00 +0000

Redis cluster bus port blocked: the port+10000 firewall gotcha

CLUSTER INFO reports cluster_state:fail. Nodes show non-zero cluster_slots_pfail. Clients receive CLUSTERDOWN. Yet redis-cli -p 6379 PING returns PONG on every node, application connections are still accepted, and the client port shows no obvious network outage. The cluster behaves like it is partitioned, but only the bus is broken. Port 16379, or your configured client port plus 10000, is missing from a firewall rule, security group, or container port mapping. The cluster bus carries gossip, failure detection, and node discovery over this separate TCP port. When the bus is unreachable, nodes cannot synchronize the cluster map, so they mark peers as failed and withdraw slot coverage even though the data port stays healthy. Because firewall rules often cover the client port but omit the bus port, this failure mode is common after infrastructure changes, node replacements, or environment migrations.

Redis cluster_slots_pfail > 0: impending node failure in a cluster

Mon, 01 Jan 0001 00:00:00 +0000

Redis cluster_slots_pfail > 0: impending node failure in a cluster

cluster_slots_pfail > 0 means at least one hash slot is mapped to a node that a peer suspects is down. In Redis Cluster, PFAIL is unilateral: any node raises it when another stops answering gossip PINGs for longer than cluster-node-timeout. Slots continue to serve traffic; the cluster has not yet agreed the node is dead.

Brief spikes are expected during background saves, AOF rewrites, or any main-thread freeze. Sustained non-zero values indicate a real problem: network partition, node crash, or overload. If the majority of masters confirm the suspicion within twice cluster-node-timeout, PFAIL escalates to FAIL. The affected slots become unavailable until a replica wins election. In a three-master cluster, losing two primaries leaves the survivor without quorum. The cluster enters a zombie state where no failover can proceed. Investigate PFAIL while you still have quorum and before automatic escalation.

Redis CLUSTERDOWN / cluster_state:fail: slot coverage and recovery

Mon, 01 Jan 0001 00:00:00 +0000

Redis CLUSTERDOWN / cluster_state:fail: slot coverage and recovery

CLUSTERDOWN The cluster is down means at least one of the 16384 hash slots lacks a healthy master. With cluster-require-full-coverage yes (the default), a single missing slot blocks all writes. This guide covers diagnosing the root cause, recovering safely, and preventing recurrence.

What this means

Redis Cluster shards the keyspace across 16384 hash slots. Each slot must be assigned to a master node that is reachable and healthy to count toward cluster_slots_ok. When cluster_slots_assigned drops below 16384, or cluster_slots_fail becomes non-zero because a node has been marked FAIL by quorum, the cluster transitions to cluster_state:fail. Clients receive CLUSTERDOWN for operations hashing to affected slots.

Redis connected_clients climbing: connection leak detection

Mon, 01 Jan 0001 00:00:00 +0000

Redis connected_clients climbing: connection leak detection

A sustained climb in connected_clients over hours or days while application traffic is flat is a classic Redis connection leak. Each connection costs roughly 10 KB of server-side memory. A thousand leaked connections consume ~100 MB independent of your dataset. If the instance is near maxmemory, that overhead can push Redis into eviction or OOM territory.

The default timeout is 0, so idle connections are never closed. Missing close() calls, connection pool misconfiguration, or unsubscribed pub/sub listeners all accumulate forever.

Redis connected_slaves dropped: detecting replica disconnects on the primary

Mon, 01 Jan 0001 00:00:00 +0000

Redis connected_slaves dropped: detecting replica disconnects on the primary

If INFO replication on a Redis primary shows connected_slaves lower than expected, the missing replica shrinks read capacity, makes a full resync likely, and widens the data-loss window during failover. The primary does not keep a tombstone: it decrements the counter and drops the corresponding slaveN line from the next INFO sample. You need to determine whether the replica crashed, the network partitioned, or Sentinel promoted the replica and the old primary has not caught up.

Redis connection exhaustion: leaks, pools, and the retry storm

Mon, 01 Jan 0001 00:00:00 +0000

Redis connection exhaustion: leaks, pools, and the retry storm

Application logs show connection timeouts and Redis returns ERR max number of clients reached. Downstream services fail because they cannot reach the cache. INFO clients shows connected_clients at the hard limit even though traffic has not increased. This is connection exhaustion. The most dangerous response is a retry storm that turns a small leak into a site-wide cascade.

Redis enforces a hard upper bound on connections via maxclients. When the sum of connected_clients, connected_slaves, and cluster_connections reaches that limit, Redis rejects every new TCP connection. Applications that retry immediately without backoff create a feedback loop: existing connections age out slowly while new attempts pile up, keeping the server pinned at the limit even after the original leak stops growing.

Redis CPU saturation: hitting the single-core throughput ceiling

Mon, 01 Jan 0001 00:00:00 +0000

Redis CPU saturation: hitting the single-core throughput ceiling

Redis latency climbs. PING returns PONG, but simple GETs take milliseconds instead of microseconds. Host CPU looks moderate - perhaps 25% across eight cores - yet commands queue. The likely cause is main-thread CPU saturation. Redis executes all commands on a single event-loop thread. Once that thread saturates one core, latency rises linearly with queue depth. There is no performance cliff - only a steady ramp that eventually drives client timeouts. On multi-core hosts, aggregate process CPU hides this bottleneck because background children, I/O threads, and system accounting spread usage across cores.

Redis event loop blocked: when one slow command freezes everything

Mon, 01 Jan 0001 00:00:00 +0000

Redis event loop blocked: when one slow command freezes everything

Redis processes every command on a single main thread. Even with Redis 6.0 and later offloading network I/O to threads, command execution itself remains strictly sequential. When one command takes too long, everything behind it waits. You will see clients still connected, but PING stalls and throughput collapses to zero. This is the Slow Command Snowball pattern: a single expensive operation blocks the event loop, the client queue backs up, and latency compounds across every connected application.

Redis eviction policy tuning: allkeys-lru vs volatile-ttl vs noeviction

Mon, 01 Jan 0001 00:00:00 +0000

Redis eviction policy tuning: allkeys-lru vs volatile-ttl vs noeviction

When Redis reaches maxmemory, it must either reject new writes or delete existing keys. The maxmemory-policy directive decides which path it takes, yet many production instances run with a policy that mismatches the workload. A cache running noeviction returns OOM errors to clients. A database running allkeys-lru silently deletes committed data. A session store running volatile-ttl suddenly rejects writes the moment an application bug omits a TTL.

Redis exposed without authentication: the CONFIG SET dir crontab attack

Mon, 01 Jan 0001 00:00:00 +0000

Redis exposed without authentication: the CONFIG SET dir crontab attack

An unauthenticated Redis instance on a public interface is remote code execution. The classic attack chains four commands: CONFIG SET dir to a cron folder, CONFIG SET dbfilename to a valid cron file, SET a malicious payload, and SAVE. If Redis runs as root, the host is compromised immediately. If unprivileged, attackers pivot via SSH keys or systemd timers.

This guide is an operational audit and lockdown. It covers how the file-write attack works, how to detect exposure, how to check for active compromise, and how to harden the instance. It also covers the follow-on risk: once authenticated access is gained, recently disclosed authenticated RCE bugs can escalate to full system control.

Redis FLUSHALL ran in production: detection, prevention, and recovery

Mon, 01 Jan 0001 00:00:00 +0000

Redis FLUSHALL ran in production: detection, prevention, and recovery

FLUSHALL deletes every key in every logical database. Under default save policies, the server typically triggers a background save of the empty keyspace within seconds, overwriting the RDB snapshot. There is no undo. If you are responding to an active incident, your priorities are: confirm the scope, isolate unaffected replicas before they process the flush, recover from the freshest intact persistence source, and harden the instance.

Redis fork/COW memory storm: why persistence doubles RSS and OOM-kills the box

Mon, 01 Jan 0001 00:00:00 +0000

Redis fork/COW memory storm: why persistence doubles RSS and OOM-kills the box

Redis disappeared from your container with only an OOMKilled status and a metrics gap that aligns with an RDB snapshot or AOF rewrite. The dataset was under its memory limit moments ago, but during persistence the reported RSS doubled and the kernel killed the process.

This is the Redis fork/copy-on-write memory storm. Redis calls fork() to spawn a child process for background RDB snapshots, AOF rewrites, and full replication syncs. After the fork, parent and child share pages through copy-on-write. Pages stay read-only until one process writes. If the parent continues serving writes, every modified page is copied. On a write-heavy instance, this can duplicate the entire dataset, pushing RSS to roughly twice the logical data size. Containers with tight memory limits do not see used_memory; they see RSS. When RSS hits the cgroup ceiling, the OOM killer fires, both processes die, and the instance restarts cold.

Redis KEYS command blocking production: why to replace it with SCAN

Mon, 01 Jan 0001 00:00:00 +0000

Redis KEYS command blocking production: why to replace it with SCAN

Redis P99 latency jumps from sub-millisecond to seconds. Clients time out. instantaneous_ops_per_sec drops to near zero while connected_clients stays high. The likely culprit is a single slow command monopolizing the event loop, and KEYS is the classic offender.

Redis runs all client commands on one main thread. KEYS scans the entire keyspace synchronously to match a pattern. Time complexity is O(N) where N is the total number of keys. During the scan, nothing else executes. Every client, including replication streams, health checks, and monitoring probes, waits.

Redis keyspace growing unbounded: keys without TTL and memory leaks

Mon, 01 Jan 0001 00:00:00 +0000

Redis keyspace growing unbounded: keys without TTL and memory leaks

Redis memory climbs. used_memory approaches maxmemory, or the OOM killer intervenes. In INFO keyspace, keys rises while expires stays flat. Or both rise, but used_memory_rss outpaces the dataset. Unbounded keyspace growth is a symptom, not a root cause: missing TTLs, application bugs, expiration backlog, or leaks in client buffers and allocators.

What this means

INFO keyspace reports db{N}:keys=X,expires=Y,avg_ttl=Z. The keys field counts every key; expires counts only those with a TTL. When keys grows and expires does not, new keys are being created without expiration. Even when expires tracks keys, memory can still grow if the active expiration cycle cannot delete keys as fast as they are created.

Redis latency spikes: diagnosis with the LATENCY subsystem

Mon, 01 Jan 0001 00:00:00 +0000

Redis latency spikes: diagnosis with the LATENCY subsystem

Your application P99 latency just jumped and clients are timing out. The infrastructure dashboard shows the Redis host is up, memory is not exhausted, and ops per second look normal, yet something is blocking the main thread. It could be a single KEYS * freezing the event loop, a fork() duplicating page tables for an RDB save, an AOF fsync stalling on saturated disk, or the active expire cycle burning CPU. Standard INFO counters will not tell you which one.

Redis latest_fork_usec too high: THP, NUMA, and fork latency

Mon, 01 Jan 0001 00:00:00 +0000

Redis latest_fork_usec too high: THP, NUMA, and fork latency

INFO stats shows latest_fork_usec in the hundreds of milliseconds. Every fork() blocks the single event loop, so during that window no commands are processed. Clients time out, replicas disconnect, and a full resync can trigger another fork, creating a loop of latency and reconnection storms. A normal fork costs roughly 10-20ms per gigabyte of resident memory with Transparent Huge Pages disabled. If you are seeing 10-100x that, the culprit is usually THP, NUMA, or memory overcommit policy.

Redis Linux kernel tuning: vm.overcommit_memory, swappiness, and NUMA

Mon, 01 Jan 0001 00:00:00 +0000

Redis Linux kernel tuning: vm.overcommit_memory, swappiness, and NUMA

Redis uses fork-based copy-on-write for background saves, AOF rewrites, and full replication resyncs. Linux defaults for memory overcommit, swap, page size, and socket queuing suit general-purpose workloads, not an in-memory store that clones multi-gigabyte address spaces. Left unchanged, they produce intermittent Cannot allocate memory errors during BGSAVE, 10-100x latency spikes during AOF rewrite, and silent OOM kills. This guide covers the five host-level tunables, the failure mode each prevents, and the production signals that expose a misconfigured host.

Redis LOADING Redis is loading the dataset in memory - why and how long

Mon, 01 Jan 0001 00:00:00 +0000

Redis LOADING Redis is loading the dataset in memory - why and how long

What this means

When INFO persistence returns loading:1, Redis is reading an RDB dump or replaying an AOF log into memory. This happens after any restart where persistence files are present. redis-cli PING returns PONG, but all data commands such as GET, SET, and HGETALL are rejected with a -LOADING error. The loading_loaded_perc, loading_loaded_bytes, and loading_eta_seconds fields expose real-time progress. While loading is active, latency, hit rate, and eviction metrics are meaningless. Load duration ranges from seconds for small RDB snapshots to tens of minutes, or even hours, for large AOF files on slow disks.

Redis low keyspace hit rate: cache effectiveness and cold-start recovery

Mon, 01 Jan 0001 00:00:00 +0000

Redis low keyspace hit rate: cache effectiveness and cold-start recovery

A low keyspace hit rate turns Redis into a pass-through to the backend. The metric is keyspace_hits / (keyspace_hits + keyspace_misses), but interpreting it is not simple. A restarted instance shows 0% for minutes. Workloads heavy on EXISTS naturally miss. A cache that evicts faster than it is hit degrades silently until backend load spikes. This guide shows how to distinguish real cache degradation from false alarms and recover.

Redis mass key expiration spike: TTL jitter and the active expiry cycle

Mon, 01 Jan 0001 00:00:00 +0000

Redis mass key expiration spike: TTL jitter and the active expiry cycle

Your application latency graph just spiked. Redis instantaneous_ops_per_sec dropped. keyspace_misses jumped. INFO stats shows expired_keys climbing by thousands per second. The cause is likely not a traffic surge, but a wave of keys hitting TTL at the same moment. When millions of keys share an identical expiration time, Redis’s active expiry cycle cannot sample and delete them fast enough. The main thread spends increasing time on expiration cleanup, blocking client commands and triggering a cache stampede as expired keys return nil.

Redis MASTERDOWN / master_link_status:down: replication link broken

Mon, 01 Jan 0001 00:00:00 +0000

Redis MASTERDOWN / master_link_status:down: replication link broken

You see MASTERDOWN errors from client libraries, or monitoring shows master_link_status:down on a replica. The replica is still accepting connections and serving reads, but every response is increasingly stale. The primary continues to take writes, so the gap widens. Determine whether this is a transient resync or a real partition, and fix it without forcing an expensive full resync that freezes the primary with a fork.

Redis max number of clients reached: maxclients and rejected_connections

Mon, 01 Jan 0001 00:00:00 +0000

Redis max number of clients reached: maxclients and rejected_connections

ERR max number of clients reached in application logs, or a rising rejected_connections metric, means new TCP sockets are refused while existing connections work normally. PING still returns PONG. The failure is a hard cliff: once the total connection count hits maxclients, every new connection is rejected immediately. Because rejected_connections is cumulative, a flat line is healthy and any upward slope is an active incident.

Redis maxmemory not set: why every production instance needs a memory limit

Mon, 01 Jan 0001 00:00:00 +0000

Redis maxmemory not set: why every production instance needs a memory limit

On 64-bit builds, Redis defaults maxmemory to 0. In production, this is an incident waiting to happen. A value of 0 means Redis tracks used_memory but enforces no limit. The process grows until the host OS OOM killer intervenes, or until a container memory limit triggers a SIGKILL. Redis dies without warning, restarts into a cold cache, and faces a thundering herd of client reconnections that immediately reapply the same write pressure. The cycle repeats until the configuration is fixed.

Redis mem_fragmentation_ratio below 1.0: detecting swap death

Mon, 01 Jan 0001 00:00:00 +0000

Redis mem_fragmentation_ratio below 1.0: detecting swap death

A mem_fragmentation_ratio of 0.72 on a substantial dataset means the operating system has swapped out Redis memory pages. Redis stays alive and responds to PING, but every command touching a swapped key blocks the single event loop on disk I/O. Because Redis logs nothing about swap, the resulting latency catastrophe looks like a mystery.

mem_fragmentation_ratio equals used_memory_rss / used_memory. When the ratio drops below 1.0, the resident set size is smaller than the memory Redis requested from its allocator. The missing bytes are in swap. Operators typically watch for ratios above 1.5, so a low ratio is misread as “good fragmentation” when it is actually the worst memory-related failure mode short of an OOM kill.

Redis mem_fragmentation_ratio high: jemalloc fragmentation and active defrag

Mon, 01 Jan 0001 00:00:00 +0000

Redis mem_fragmentation_ratio high: jemalloc fragmentation and active defrag

A mem_fragmentation_ratio sustained above 1.5 on a production instance means Redis holds significantly more physical memory (RSS) than its logical dataset size (used_memory), wasting RAM that could hold data or absorb spikes. This is not a memory leak. Redis uses jemalloc by default, which does not return freed pages to the OS eagerly. Deleted or resized keys leave holes in allocator arenas, inflating RSS while used_memory stays flat or drops.

Redis memory pressure spiral: eviction thrashing and how to break it

Mon, 01 Jan 0001 00:00:00 +0000

Redis memory pressure spiral: eviction thrashing and how to break it

Redis latency climbs, CPU saturates, and cache hit rate falls. evicted_keys rises while application writes increase. The backend database gets hammered. This is not a simple capacity shortage; it is a memory pressure spiral. Redis has reached maxmemory and started evicting keys. The application responds to cache misses by re-fetching from the origin and writing back to Redis. Those writes trigger more evictions, which cause more misses, which cause more writes. Redis does maximum work for minimum value.

Redis MONITOR left running: the output-buffer OOM footgun

Mon, 01 Jan 0001 00:00:00 +0000

Redis MONITOR left running: the output-buffer OOM footgun

A forgotten MONITOR session typically produces this pattern:

used_memory and used_memory_rss climb steadily.
If maxmemory is set and an eviction policy is active, keys disappear; otherwise the kernel OOM killer may terminate the process.
Application latency spikes.
Network output roughly doubles network input.
No large keys, no persistence fork, and no replication backlog overflow.

MONITOR streams a serialized copy of every executed command into the requesting client’s output buffer. That buffer is allocated on the main heap and counts in used_memory, which contributes to maxmemory pressure and RSS growth . Redis classifies MONITOR clients as normal clients, and the default client-output-buffer-limit normal 0 0 0 places no bound on normal client output buffers. Under production load, the buffer can grow by multiple gigabytes per minute. The monitor client also consumes roughly one extra copy of outbound network traffic, so output bandwidth approximately doubles.

Redis monitoring checklist: the signals every production instance needs

Mon, 01 Jan 0001 00:00:00 +0000

Redis monitoring checklist: the signals every production instance needs

Redis can return PONG while replicating hours behind, during an OOM kill in a background save, or while a KEYS command wedges the event loop. This checklist structures monitoring into four maturity levels. Level 1 is the survival floor. Level 2 adds workload and resource awareness. Level 3 introduces leading indicators that catch degradation before it becomes an incident. Level 4 exposes allocator and encoding internals for granular diagnostics.

Redis monitoring maturity model: from survival to expert

Mon, 01 Jan 0001 00:00:00 +0000

Redis monitoring maturity model: from survival to expert

Redis can return PONG while fragmentation doubles RSS, a replica falls behind, or a slow command wedges the event loop. This guide maps four cumulative operator levels derived from production runbooks. Level 1 is liveness and memory limits. Level 2 is workload anomalies and capacity pressure. Level 3 is leading indicators for composite failures. Level 4 is forensic depth. Do not skip Level 1 because you are collecting allocator statistics.

Redis NOAUTH / WRONGPASS authentication failures: ACL LOG and credential drift

Mon, 01 Jan 0001 00:00:00 +0000

Redis NOAUTH / WRONGPASS authentication failures: ACL LOG and credential drift

Application logs report (error) NOAUTH Authentication required or (error) WRONGPASS invalid username-password pair or user is disabled. Connections drop, transactions abort, and previously working requests are rejected. Redis does not expose an authentication failure counter in INFO stats; on Redis 6.0 and later, ACL LOG is the only structured source. On earlier versions, scrape the log file. Correlate the failure source, username, and timing to distinguish credential drift, misconfiguration, and brute-force probes.

Redis OOM command not allowed when used memory > 'maxmemory' - causes and fixes

Mon, 01 Jan 0001 00:00:00 +0000

Redis OOM command not allowed when used memory > ‘maxmemory’ - causes and fixes

Redis returns (error) OOM command not allowed when used memory > 'maxmemory'. The server stays online; reads succeed, writes fail. If the client library suppresses errors, the first symptom may be missing data or backend load spikes. This occurs when used_memory reaches maxmemory and the eviction policy cannot free space. Under noeviction, Redis rejects every write and keeps all keys. Under volatile-*, the same happens when no keys carry a TTL. Monitoring often misses this because evicted_keys stays at zero while used_memory sits just below the limit.

Redis OOM-killed by the kernel: RSS, overcommit, and recovery

Mon, 01 Jan 0001 00:00:00 +0000

Redis OOM-killed by the kernel: RSS, overcommit, and recovery

Redis reports used_memory at 60% of maxmemory, then disappears. The container status is OOMKilled, or dmesg shows the kernel OOM killer selected redis-server. The kernel enforces resident memory (RSS), while used_memory and maxmemory track logical allocator state. Fragmentation, copy-on-write pages during persistence, and client buffers inflate RSS above the logical figure most operators monitor. When RSS hits the host or cgroup memory ceiling, the kernel terminates the process even though Redis believes it is within limits.

Redis Pub/Sub pattern overhead: PSUBSCRIBE scaling and slow subscribers

Mon, 01 Jan 0001 00:00:00 +0000

Redis Pub/Sub pattern overhead: PSUBSCRIBE scaling and slow subscribers

You see Redis latency spikes that align with PUBLISH bursts. Subscribers disconnect with output buffer limit errors. In cluster mode, CPU and network climb linearly with node count even though PUBLISH volume stays flat. Two mechanisms drive this: PSUBSCRIBE pattern matching adds O(N) work to every PUBLISH, and slow subscribers accumulate per-client output memory until Redis cuts them off. This guide shows how to confirm the bottleneck, apply safe fixes, and decide when to move to sharded Pub/Sub.

Redis rdb_last_bgsave_status:err: diagnosing failed background saves

Mon, 01 Jan 0001 00:00:00 +0000

Redis rdb_last_bgsave_status:err: diagnosing failed background saves

INFO persistence returning rdb_last_bgsave_status:err means the last background save failed. The flag is sticky: it remains err until a subsequent BGSAVE succeeds, so the failure may be hours old. If stop-writes-on-bgsave-error is enabled (the default), Redis rejects writes and your application sees MISCONF errors. If the setting is disabled, writes continue but durability is broken; the exposure window grows with every update.

The failure modes are a narrow set: fork failure, disk full, filesystem write rejection, or child death before completion. Follow this sequence to separate them without restarting Redis.

Redis READONLY You can't write against a read only replica - causes and fixes

Mon, 01 Jan 0001 00:00:00 +0000

Redis READONLY You can’t write against a read only replica - causes and fixes

Your application hits (error) READONLY You can't write against a read only replica. Writes fail; reads work. The connected Redis instance thinks it is a replica, so it rejects mutating commands. The replica is behaving correctly. The problem is a write-capable client routed to a node that is not the current primary. This typically happens in three situations: a routing bug that sends writes to a replica endpoint, stale client topology after a failover or upgrade, or an instance that was accidentally demoted at runtime.

Redis replication backlog overflow: full-resync storms and the 1MB default

Mon, 01 Jan 0001 00:00:00 +0000

Redis replication backlog overflow: full-resync storms and the 1MB default

Replicas drop and reconnect, but each reconnection triggers a full resync instead of a partial sync. The primary forks for an RDB dump, latency spikes, and other replicas fall behind. Before recovery, another replica exceeds the backlog window and the cycle repeats. The default repl-backlog-size of 1 MB triggers this cascade in most production workloads.

The backlog is a fixed-size circular buffer of recent writes that lets a disconnected replica catch up without a full resync. When writes during a blip exceed the 1 MB default, the replica’s offset falls outside the window. Recovery requires a full resync, which forks the primary and turns a brief disconnect into a site-wide latency event.

Redis replication lag: detection, diagnosis, and fixes

Mon, 01 Jan 0001 00:00:00 +0000

Redis replication lag: detection, diagnosis, and fixes

Reads from Redis replicas returning stale data indicate replication lag. Your monitoring shows a growing gap between the primary’s replication offset and what the replica has acknowledged. During failover, every byte of that gap is potential data loss.

Replication lag in Redis is measured in bytes: the difference between master_repl_offset on the primary and slave_repl_offset on the replica. Small, stable lag is normal in asynchronous replication, but lag that grows continuously or exceeds repl-backlog-size signals a bottleneck that can cascade into full resync storms.

Redis Sentinel triggering unnecessary failovers: quorum and split-brain

Mon, 01 Jan 0001 00:00:00 +0000

Redis Sentinel triggering unnecessary failovers: quorum and split-brain

Your logs show a failover, but the old master never restarted. Or failovers flap: one node is promoted, then another, and clients see MASTERDOWN and READONLY while Sentinels disagree. The root cause is usually not the Redis data node. It is the Sentinel control plane: quorum too low, down-after-milliseconds too aggressive, or a network partition that leaves Sentinels on the wrong side of the split.

Redis slowlog filling up: finding and fixing the slow commands

Mon, 01 Jan 0001 00:00:00 +0000

Redis slowlog filling up: finding and fixing the slow commands

Clients report timeouts and latency has a new plateau. Check SLOWLOG LEN: if it is climbing or already at 128, the slowlog is filling. The slowlog is a circular buffer that logs commands whose execution exceeds slowlog-log-slower-than (default 10 ms). Rapid rotation means entries evict before you inspect them, and every entry marks a blocked single-threaded event loop. Extract the culprits before the evidence disappears, distinguish execution time from queue wait, and stop the bleed.

Redis stale replica promotion: silent data loss at failover

Mon, 01 Jan 0001 00:00:00 +0000

Redis stale replica promotion: silent data loss at failover

After a Redis failover, the new primary accepts writes immediately and clients reconnect without errors. Writes that were acknowledged by the old primary but not yet replicated are gone. This is stale replica promotion.

Redis replication is asynchronous by default. The primary persists a write locally, replies OK to the client, then streams the change to replicas. If the primary fails before a replica receives the outstanding writes, that replica never sees them. Sentinel or Redis Cluster promotes the best available replica. If that replica is lagging, the writes in the gap are permanently lost. The client receives no error and no log warns you. The data is simply missing.

Redis Stream consumer group lag: pending entries and dead consumers

Mon, 01 Jan 0001 00:00:00 +0000

Redis Stream consumer group lag: pending entries and dead consumers

Stream processing falls behind. An alert fires on used_memory, or your application dashboard shows lag. You run XINFO GROUPS and see lag or pending climbing. One means consumers cannot keep up. The other means they are not acknowledging messages. Both increase memory pressure, but they have different fixes. This guide shows how to tell them apart, find dead consumers, and clean up the Pending Entry List before it becomes a memory incident.

Redis sync_full incrementing: diagnosing full resync events

Mon, 01 Jan 0001 00:00:00 +0000

Redis sync_full incrementing: diagnosing full resync events

Your Redis primary’s sync_full counter is climbing. That means replicas are performing full resyncs instead of partial ones. Each full resync forces the primary to fork, write an RDB snapshot, and push it to the replica, which then wipes its own dataset and reloads from scratch. One full resync is a heavy operation. Several in succession, or multiple at once, can freeze the primary’s event loop, spike memory via copy-on-write, and trigger a cascade where more replicas fall behind and also need full resyncs.