https://www.netdata.cloud/tags/kubernetes-api-server/Recent content in Kubernetes API Server on NetdataHugoen-usSat, 22 Aug 2026 05:09:03 +0300https://www.netdata.cloud/solutions/technologies/kubernetes-monitoring/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/technologies/kubernetes-monitoring/Transform Kubernetes observability with Netdata&rsquo;s edge-native architecture. Get per-second visibility, ML anomaly detection on every metric, and AI-powered root cause analysis - all at 90% lower cost than traditional solutions.https://www.netdata.cloud/guides/kubernetes/kubernetes-api-server-etcd-latency/Mon, 01 Jan 0001 00:00:00 +0000https://www.netdata.cloud/guides/kubernetes/kubernetes-api-server-etcd-latency/<p>When etcd slows down, the entire control plane slows with it. A few extra milliseconds on disk fsync turns into hung kubectl commands, backed-up controller queues, and eventually a cluster that cannot schedule pods or update endpoints. Detect the etcd latency cascade, confirm whether storage is the root cause, and break the feedback loop before the cluster becomes effectively read-only.</p> <h2 id="what-this-means">What This Means</h2> <p>etcd serializes every Kubernetes mutation. Every API server write becomes a Raft proposal that must fsync to the WAL before etcd acknowledges it. When the disk under etcd is slow, every fsync waits longer. The API server holds mutating requests open until etcd responds. Requests pile up in the inflight queue. Once the queue hits the limit, the API server returns 429 Too Many Requests. Controllers that depend on writes (scheduler, replica set controller, and others) fall behind and retry. Retries generate more write load. The result is a feedback loop: slow disk -&gt; slow etcd -&gt; slow API server -&gt; retry storm -&gt; amplified etcd load.</p>https://www.netdata.cloud/guides/kubernetes/kubernetes-api-server-rate-limited/Mon, 01 Jan 0001 00:00:00 +0000https://www.netdata.cloud/guides/kubernetes/kubernetes-api-server-rate-limited/<p>Your API server is running. <code>/healthz</code> returns 200. <code>/readyz</code> passes. Yet nodes drop to <code>NotReady</code>, the scheduler stops placing pods, and controller logs fill with <code>context deadline exceeded</code>. The cluster is not down, but it is frozen. This pattern often points to API Priority and Fairness (APF) starvation: low-priority traffic consumes the API server&rsquo;s concurrency budget, and critical control plane requests queue or get rejected.</p> <p>APF is enabled by default in Kubernetes 1.20+. It classifies every API request into a priority level via FlowSchema rules, then schedules requests against a per-level concurrency limit. When a priority level exhausts its seats, requests queue. If the queue fills, the server returns HTTP 429. When the queue grows in <code>system</code> or <code>leader-election</code>, kubelets cannot renew leases, controllers cannot write status, and the cluster degrades from the inside out. This guide shows how to confirm APF starvation, identify the culprit, and fix the allocation without turning the API server into a free-for-all.</p>https://www.netdata.cloud/guides/kubernetes/kubernetes-api-server-slow/Mon, 01 Jan 0001 00:00:00 +0000https://www.netdata.cloud/guides/kubernetes/kubernetes-api-server-slow/<p>When <code>kubectl</code> hangs, controllers log <code>context deadline exceeded</code>, and deployments stall, the Kubernetes API server is usually the bottleneck. It is the single funnel for every read and write to cluster state. Slowness propagates to scheduling, pod lifecycle, service discovery, and external automation.</p> <p>This article covers operational causes and gives a step-by-step diagnostic flow to run during an incident. Use it to distinguish etcd latency, <a >admission webhook</a> stalls, request saturation, and memory pressure.</p>https://www.netdata.cloud/guides/kubernetes/kubernetes-monitoring-checklist/Mon, 01 Jan 0001 00:00:00 +0000https://www.netdata.cloud/guides/kubernetes/kubernetes-monitoring-checklist/<p>This article is a reference checklist for senior engineers who are wiring up, auditing, or hardening monitoring for a production Kubernetes cluster. It assumes you already understand the control plane architecture and focuses on what to collect, where to find it, and which symptoms matter. Use it during greenfield instrumentation, post-incident gap analysis, or routine health audits.</p> <p>The signals are grouped by domain. Each entry leads with a short noun phrase, followed by one sentence explaining why it matters, and a concrete warning sign to alert on. Thresholds are drawn from upstream SLOs, kubelet defaults, and etcd operational limits documented in the Kubernetes source and production playbooks. If you run a managed service such as EKS, GKE, or AKS, treat control-plane metrics as provider-mediated; many etcd and API server internals are opaque in those environments.</p>https://www.netdata.cloud/guides/kubernetes/Mon, 01 Jan 0001 00:00:00 +0000https://www.netdata.cloud/guides/kubernetes/Learn what actually breaks in production Kubernetes clusters, how to diagnose it, and what to monitor so the next incident is shorter than the last one.https://www.netdata.cloud/guides/kubernetes/kubernetes-pod-stuck-terminating/Mon, 01 Jan 0001 00:00:00 +0000https://www.netdata.cloud/guides/kubernetes/kubernetes-pod-stuck-terminating/<p>A pod stuck in <code>Terminating</code> stays visible in the <a href="https://www.netdata.cloud/guides/kubernetes/kubernetes-api-server-slow/">API server</a> after <code>kubectl delete</code>, sometimes for minutes or hours. Usually a finalizer blocks removal, a CSI volume is still attached, or the container is ignoring SIGTERM. Force deleting without diagnosis orphans containers and can violate StatefulSet guarantees. Check the signals first, then decide whether to wait, patch a finalizer, or force delete.</p> <h2 id="what-this-means">What This Means</h2> <p>When you delete a pod, the API server sets <code>metadata.deletionTimestamp</code> and starts a graceful deletion window controlled by <code>terminationGracePeriodSeconds</code> (default 30). The kubelet executes any <code>preStop</code> hooks, sends SIGTERM to all containers, and waits up to the grace period before sending SIGKILL.</p>https://www.netdata.cloud/guides/kubernetes/kubernetes-scheduler-not-scheduling/Mon, 01 Jan 0001 00:00:00 +0000https://www.netdata.cloud/guides/kubernetes/kubernetes-scheduler-not-scheduling/<p>Pods stay Pending for many reasons, but the scheduler process being down is rarely one. More often, pods accumulate in internal queues because the cluster is out of capacity, a control plane dependency stalls the binding cycle, or a filter plugin rejects every candidate node. Distinguishing &ldquo;unschedulable&rdquo; (no node fits) from &ldquo;not scheduling&rdquo; (the scheduler cannot keep up or the binding cycle is failing) prevents wasted node scaling when the real problem is an <a href="https://www.netdata.cloud/guides/kubernetes/kubernetes-api-server-etcd-latency/">etcd latency</a> spike or a volume affinity conflict.</p>