containerd Containers on Netdata

Docker daemon not responding: how to troubleshoot a hung dockerd

Mon, 01 Jan 0001 00:00:00 +0000

A hung Docker daemon is one of the more disorienting production failures you can face. The process is still running, your containers are still serving traffic, but every docker command hangs. You cannot inspect, stop, or create containers. Deployments stall. Automation times out.

This guide covers the diagnostic ladder from socket probe to storage driver investigation, explains when to wait versus when to restart, and describes what to avoid when you are not sure what is wrong.

Kubernetes node NotReady: kubelet, runtime, and network diagnosis

Mon, 01 Jan 0001 00:00:00 +0000

When a Kubernetes node becomes NotReady, existing containers usually keep running, but the cluster stops scheduling new pods, removes endpoints from Services, and eventually evicts workloads after the pod eviction timeout. Root causes fall into three domains: kubelet health, container runtime responsiveness, and CNI or control plane connectivity.

What This Means

Kubernetes marks a node NotReady when the kubelet Ready condition is False, or when the node controller has not received a heartbeat within --node-monitor-grace-period (default 40 seconds). The node receives the node.kubernetes.io/not-ready:NoSchedule taint. If the condition persists longer than the pod eviction timeout (default 5 minutes), the controller manager marks pods on the node for rescheduling.

Kubernetes node PIDPressure: detection and remediation

Mon, 01 Jan 0001 00:00:00 +0000

PID exhaustion is a cliff-edge failure: once the kernel cannot fork, containers fail to start, health checks fail, and ssh to the node may hang. Kubernetes surfaces this through the PIDPressure node condition, but many clusters ship without PID-based eviction thresholds. Without them, the first symptom is usually EAGAIN or ENOMEM from fork failures, not a kubelet eviction.

This guide shows how to detect PIDPressure before it triggers an outage, distinguish between application leaks, runtime shim accumulation, and kernel limits, and remediate the root cause. You will correlate node-level PID utilization with specific pods, validate kubelet cgroup enforcement, and configure thresholds that provide lead time.