Kafka UnderMinIsrPartitionCount: confirming the write path is blocked

UnderMinIsrPartitionCount is the Kafka metric that confirms active write-path impact. When UnderReplicatedPartitions (URP) is nonzero, the cluster’s durability window is open but writes may still be succeeding. When UnderMinIsrPartitionCount is nonzero, producers using acks=all are actively receiving NotEnoughReplicasException and their produce requests are being rejected at the broker.

This distinction matters for alerting. URP fires constantly during normal operations: rolling restarts, partition reassignment, transient GC pauses, brief network blips. Most of these are not worth paging someone at 3 a.m. UnderMinIsrPartitionCount only fires when ISR membership has dropped below min.insync.replicas, meaning the write durability guarantee you configured cannot be met. That is direct, measurable user impact.

This article covers how the metric works, how to distinguish it from URP, and how to confirm that producers are actually blocked. It walks through the per-partition CLI and JMX signals needed to enumerate affected topics, and the correlated metrics that tell you whether the situation is worsening or recovering.

What it is and why it matters

The MBean kafka.server:type=ReplicaManager,name=UnderMinIsrPartitionCount reports a per-broker gauge: the count of leader partitions on this broker where the current ISR size is less than the topic’s configured min.insync.replicas. The attribute is Value.

In steady state, this value must be 0. Any nonzero value means the partition cannot satisfy the durability requirement for acks=all producers. Those producers receive NotEnoughReplicasException, or in some cases NotEnoughReplicasAfterAppendException. The broker rejects the produce request rather than accepting data that would be under-replicated relative to the configured durability standard.

The severity model for this metric is PAGE, with conditions. The playbook specifies: page if nonzero for more than 2 minutes, no broker has uptime less than 600 seconds (to suppress cold-start false positives during restarts), and no partition reassignment is in progress. The 2-minute threshold filters out transient blips during leader elections or brief GC pauses. The uptime gate prevents pages during planned maintenance.

The operational value is that this metric is the confirmation signal. URP tells you the system is degraded. UnderMinIsrPartitionCount tells you writes are being rejected. That difference determines whether you have a durability risk or an active outage.

How it works

Every partition has one leader and N-1 followers, determined by replication.factor. Followers run fetcher threads that pull data from the leader. The leader tracks which followers are caught up within replica.lag.time.max.ms (default 30 seconds, changed from 10 seconds in Kafka 2.5.0). Followers within this window remain in the ISR. Followers that fall behind are removed by the leader; when they catch up, they are added back.

When a producer sends with acks=all, the leader does not acknowledge the write until all ISR members have replicated it. This is the mechanism that provides Kafka’s strongest durability guarantee. But if ISR membership drops below min.insync.replicas, the partition can no longer meet this guarantee. The broker rejects the produce request instead.

The default for min.insync.replicas is 1. With this default, UnderMinIsrPartitionCount fires only when a partition has zero ISR members. Many operators set min.insync.replicas to replication.factor - 1 (typically 2 for RF=3) so that the metric fires earlier, before the partition reaches the cliff edge of total unavailability.

The metric is unaffected by whether the cluster runs in ZooKeeper mode or KRaft mode. It lives in ReplicaManager, which is part of the broker data plane, not the consensus layer. ZooKeeper mode is removed entirely in Kafka 4.0, but the metric’s semantics and MBean name remain the same.

flowchart TD
    A["Follower exceeds replica.lag.time.max.ms"] --> B["Leader removes follower from ISR"]
    B --> C{"ISR below min.insync.replicas?"}
    C -- No --> D["URP fires, writes still succeed"]
    C -- Yes --> E["UnderMinIsrPartitionCount fires"]
    E --> F["acks=all producers rejected"]
    F --> G["FailedProduceRequestsPerSec rises"]

Distinguishing UnderMinIsr from UnderReplicatedPartitions

URP counts partitions where the ISR size is less than the total number of configured replicas. UnderMinIsrPartitionCount counts partitions where the ISR size is less than min.insync.replicas. UnderMinIsr is a strict subset of URP.

Consider a topic with replication.factor=3 and min.insync.replicas=2. If one follower falls behind and is removed from ISR, the ISR shrinks from three members to two. Now |ISR|=2, which is less than |replicas|=3, so URP increments. But |ISR|=2 is not less than minIsr=2, so UnderMinIsrPartitionCount stays at 0. Writes with acks=all still succeed because two replicas (leader plus one follower) have acknowledged.

If a second follower also falls behind, ISR drops to one member (the leader alone). Now |ISR|=1 < minIsr=2, and UnderMinIsrPartitionCount increments. acks=all producers start receiving NotEnoughReplicasException.

This is why the playbook treats URP as the leading indicator and UnderMinIsrPartitionCount as the impact confirmation. A cluster can have URP without having any under-min-ISR partitions. Both metrics must be checked independently. Alerting on URP alone means you are paging on conditions that may not affect any producer. Alerting on UnderMinIsrPartitionCount alone means you miss the early warning that ISR is degrading toward the threshold.

Where it shows up in production

Broker failure or restart. When a broker goes down, all partitions where it was a follower lose a replica from ISR. If min.insync.replicas is set to rf - 1, losing a single broker can immediately push partitions below the threshold. During rolling restarts, expect transient under-min-ISR conditions that should resolve within 1-2 times replica.lag.time.max.ms after the broker returns.

Disk degradation on a follower. The follower’s write path slows, it falls behind replica.lag.time.max.ms, and the leader removes it from ISR. If enough followers are affected, the partition drops below minIsr. Disk I/O latency (iostat await) on the follower is the first place to look.

GC pauses on a follower broker. A Full GC pause lasting longer than replica.lag.time.max.ms causes the leader to consider the follower dead. The follower is removed from ISR. GC logs (-Xlog:gc*) and CollectionTime on the old generation collector confirm the pause.

Network partition between leader and followers. Followers cannot fetch, fall behind, and are removed from ISR. TCP retransmit rate on the affected link correlates with the ISR shrink.

Leader-local counting. UnderMinIsrPartitionCount is reported per-broker and counts only partitions where this broker is the leader. A broker that is a follower for an under-min-ISR partition reports 0 for that partition. If you are looking at broker B and see UnderMinIsrPartitionCount=0, but broker A is the leader for a partition below minIsr, you will miss the signal. Always aggregate across all brokers, or use the cluster-wide CLI tools.

The min.insync.replicas=1 masking problem. With the default min.insync.replicas=1, a partition stays “healthy” in this metric even when only the leader is alive and zero followers are in ISR. The metric cannot fire because minIsr=1 is satisfied by the leader alone. This masks a dangerous state: one more failure and the partition goes offline with no under-min-ISR warning. Setting min.insync.replicas to rf - 1 trades earlier visibility for earlier write rejection.

Confirming the write path is blocked

Enumerate affected partitions. The fastest way to identify which partitions are below minIsr is the CLI flag introduced by KIP-351:

# List all partitions where ISR is below min.insync.replicas
kafka-topics.sh --bootstrap-server localhost:9092 --describe --under-min-isr-partitions

On older Kafka versions that predate this flag, you can query the per-partition MBean introduced alongside the aggregate metric in KIP-164 (Kafka 1.0.0):

kafka.cluster:type=Partition,topic={topic},name=UnderMinIsr,partition={partition}

This returns 1 when the broker is leader and |ISR| < minIsr, otherwise 0. Iterate across partitions to enumerate the affected set.

Confirm producer impact. Pair UnderMinIsrPartitionCount with FailedProduceRequestsPerSec from kafka.server:type=BrokerTopicMetrics,name=FailedProduceRequestsPerSec. If both are elevated, producers are actively experiencing rejections. If UnderMinIsrPartitionCount is nonzero but FailedProduceRequestsPerSec is zero, either no producers are currently writing to the affected partitions, or producers are using acks=0 or acks=1 (which bypass the minIsr gate).

Identify the degraded broker. Once you know which partitions are affected, cross-reference with these signals to find the root cause:

• IsrShrinksPerSec (kafka.server:type=ReplicaManager,name=IsrShrinksPerSec). Elevated rate means replicas are actively falling out of sync. Sustained shrinks without matching expands indicate a worsening condition.
• IsrExpandsPerSec. If shrinks are followed by expands, the ISR is flapping: an intermittent problem such as periodic GC pauses or bursty traffic pushing followers temporarily out of sync.
• Per-broker disk I/O latency (iostat -xz 1) on follower brokers. Slow disk writes prevent followers from keeping up with the leader’s append rate.
• JVM GC metrics on follower brokers. Full GC pauses exceeding replica.lag.time.max.ms directly cause ISR removal.
• FetchFollower request latency on the leader. If the leader is slow to serve follower fetches, followers fall behind through no fault of their own. Check kafka.network:type=RequestMetrics,name=TotalTimeMs,request=FetchFollower.

Signals to watch

How Netdata helps

• Per-second collection of UnderMinIsrPartitionCount across every broker lets you see the exact onset and recovery of the condition, not just a point-in-time sample. JMX percentile windows are 30 seconds by default; per-second collection catches spikes that minute-level polling misses.
• Correlating UnderMinIsrPartitionCount with UnderReplicatedPartitions on the same chart makes the leading-indicator-to-impact-confirmation relationship visible. You can see URP rise first, then UnderMinIsr follow as ISR crosses the minIsr threshold.
• Layering FailedProduceRequestsPerSec alongside ISR metrics confirms whether producers are actually experiencing rejections or whether the under-min-ISR partitions are idle.
• Disk I/O latency and JVM GC metrics on the same dashboard let you jump from “ISR is shrinking” to “which broker’s disk or GC is causing it” without switching tools.
• ML-based anomaly detection on IsrShrinksPerSec and IsrExpandsPerSec can surface ISR flapping patterns that raw threshold alerts miss, flagging intermittent follower degradation before it cascades to under-min-ISR.

Related guides

• How Kafka actually works in production: a mental model for operators
• Kafka Broker May Not Be Available: How To Fix It
• Kafka Broker Out Of Disk: How To Fix It
• Kafka network egress saturation: BytesOutPerSec, replication amplification, and fan-out
• Kafka consumer group lag growing: detection, lag-as-time, and root causes
• Kafka authentication failures: SASL/mTLS errors, credential rotation, and brute force
• Kafka authorization failures: ACL denials, wrong-topic clients, and audit trails
• Kafka enable.auto.commit data loss: committed offsets that outrun processing
• Kafka CommitFailedException: rebalanced-out consumers and poll loop timeouts
• Kafka connection storms: connection-count spikes, FD pressure, and network threads
• Kafka consumer group stuck Empty or Dead: no members consuming
• Kafka consumer group rebalancing too often: heartbeats, session timeout, and assignors