Exit Codes on Netdata

Docker container memory leak: how to find one and prove it

Mon, 01 Jan 0001 00:00:00 +0000

Memory that only ever climbs is easy to spot. The harder problem is proving whether the growth is a leak, unbounded caching, or a limit set below the working set. During an incident, operators need to decide in minutes whether to page an on-call developer or bump a cgroup limit. This guide shows how to use cgroup memory.stat, process-level RSS, and container restart patterns to build a defensible diagnosis. You will be able to separate anonymous memory growth from reclaimable cache, identify whether the leak lives in application heap or runtime overhead, and present evidence that justifies either a code fix or a capacity change.

Docker exit code 1: application errors and how to find them

Mon, 01 Jan 0001 00:00:00 +0000

A container exiting immediately with code 1 is a common production incident. Unlike exit code 137 (OOM killer) or code 125 (Docker daemon error), code 1 means the application inside the container called exit(1). Docker is only reporting what PID 1 did.

The challenge is that code 1 is a catch-all. It can mask an unhandled JavaScript exception, a Python import error, a missing configuration file, a shell script failing under set -e, or a Go binary that cannot reach its database. Time spent checking Docker daemon health or host memory is wasted when the real failure is an application-level error written to stdout or stderr.

Docker exit code 137: OOMKilled or SIGKILL?

Mon, 01 Jan 0001 00:00:00 +0000

A container exits with code 137. Docker restarts it, or it stays down, and you need to know why. The number itself only tells you that the process received SIGKILL. What matters for your next step is whether the kernel’s cgroup OOM killer fired because the container exceeded its memory limit, or whether an external actor sent the signal. The remediation for an undersized memory limit is completely different from fixing a misconfigured stop timeout or an orchestrator sending a premature kill. This guide shows how to classify the cause in under a minute using only the Docker CLI and cgroup files.

Docker exit code 143: SIGTERM and graceful shutdown failures

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing container exit codes after a deployment or node drain and see 143. If your monitoring alerts on it, you might think something failed. Exit code 143 is not an error. It is 128 plus signal 15 (SIGTERM), and it means the container’s PID 1 process received SIGTERM and exited voluntarily. This is exactly what docker stop is designed to do.

The operational problem is not the code itself, but what happens around it. If your application never receives the signal, it gets SIGKILL after the timeout and exits 137, potentially dropping connections and corrupting state. If your orchestrator pages you for 143 during a normal rolling restart, your alerts are noisy. If your stop timeout is too short for your database to finish checkpointing, you risk data inconsistency.

Kubernetes kubelet memory leak: detection and OOM cycle

Mon, 01 Jan 0001 00:00:00 +0000

Kubelet memory growth ends one of two ways: the process hits its cgroup limit or the node runs out of memory. The kernel OOM killer sends SIGKILL. Systemd restarts kubelet, but the new process has cold caches and immediately runs a full reconciliation pass: relisting all containers, re-syncing every pod status, and re-attaching every volume. On a busy node, that burst spikes CPU and memory, which can push the fresh kubelet back over the edge and create a Ready/NotReady flap cycle.