PHP-FPM workers OOM-killed: "child N exited on signal 9 (SIGKILL)" and the memory cliff

The PHP-FPM error log shows the same line: WARNING: [pool www] child 12345 exited on signal 9 (SIGKILL). No segfault, no stack trace, no “core dumped”. Just signal 9. Users see 502s, requests fail in batches, and reloading PHP-FPM makes it go away for a while. Hours or days later, it returns.

Signal 9 is not a PHP crash. It is the kernel or cgroup OOM killer terminating the worker because the process exceeded available memory. The master sees the worker die, logs the SIGKILL, and forks a replacement. It has no idea why the worker was killed. The log line is a consequence, not a cause.

The pattern is the “memory cliff”: per-worker RSS accumulates slowly over many requests, then total worker memory crosses the host or cgroup limit and the kills cascade. Nothing in the PHP-FPM status page or error log warns you before it starts.

What this means

A SIGKILL on a PHP-FPM worker almost always means one of two things:

1. The kernel OOM killer selected the worker because the host ran out of reclaimable memory.
2. A cgroup memory limit (systemd MemoryMax, container runtime limit) was exceeded and the cgroup OOM killer fired.

Both leave the same trace in the FPM error log. They differ only in where you find the evidence.

Kernel OOM kills are logged in dmesg and /var/log/kern.log (or journalctl -k) with Out of memory: Kill process. The entry includes the PID, oom_score, and a memory accounting line. If the FPM SIGKILL line matches an Out of memory: Kill process <pid> entry in dmesg with the same PID, the kernel OOM killer did it.

Cgroup v2 OOM kills behave differently. When a systemd unit has MemoryMax= set (or a container runtime enforces a limit), the kill is logged by systemd, not the kernel. There is no Out of memory: line in dmesg. The unit journal records:

php-fpm.service: A process of this unit has been killed by the OOM killer.

This is the most common operator trap. If you only grep -i oom /var/log/kern.log, you see nothing and conclude it was not OOM. Check the service journal as well.

emergency_restart_threshold will not save you here. It only counts SIGSEGV and SIGBUS, not SIGKILL. A cascade of OOM kills will not trigger a master restart. The master keeps forking replacements, which keep getting killed, until you intervene or the master itself is OOM-killed.

flowchart TD
    A[Workers spawn at baseline RSS] --> B[Each request leaves memory behind]
    B --> C[pm.max_requests = 0: no recycling]
    C --> D[Per-worker RSS climbs over hours or days]
    D --> E{Total worker memory}
    E -->|Host RAM exhausted| F[Kernel OOM killer fires]
    E -->|Cgroup memory.max hit| G[Cgroup OOM killer fires]
    F --> H["child N exited on signal 9 (SIGKILL)"]
    G --> H
    H --> I[Master forks replacement]
    I --> D

The cycle in the diagram is the spiral. Restarting PHP-FPM resets every worker to baseline RSS and makes the problem disappear, which is why a bare restart looks like a fix. It is not. The leak is still there. The cliff just moved.

Common causes

Quick checks

These are read-only and safe to run during an incident.

# Confirm the SIGKILL pattern in the FPM error log
grep "exited on signal 9" /var/log/php-fpm/error.log | tail -20

# Look for the kernel OOM kill signature (host RAM exhaustion)
dmesg -T | grep -E "Out of memory|Killed process" | tail -20

# Check the systemd journal for cgroup OOM kills (invisible in dmesg)
journalctl -u php-fpm --since "1 hour ago" | grep -i "killed by the OOM killer"

# Verify pm.max_requests is set (0 = unlimited, the default)
# Debian/Ubuntu: /etc/php/*/fpm/pool.d/  RHEL/CentOS: /etc/php-fpm.d/
grep -R "pm.max_requests" /etc/php/*/fpm/pool.d/

# Current per-worker RSS, sorted descending
ps -eo pid,rss,cmd --sort=-rss | grep '[p]hp-fpm' | grep -v master | head -20

# Total RSS across all workers (naive sum, overstated by shared pages)
ps -eo rss,cmd | grep '[p]hp-fpm' | grep -v master | awk '{sum+=$1} END {printf "Total: %.0f MB\n", sum/1024}'

# Accurate per-worker memory using PSS (accounts for shared opcache pages)
smem -P php-fpm -c 'pid pss rss' -s pss 2>/dev/null | tail -20

# Check for a systemd memory cap on the unit
systemctl show php-fpm -p MemoryMax -p MemoryHigh 2>/dev/null

# Swap usage per worker (thrash precedes the cliff)
for pid in $(pgrep -f "php-fpm: pool"); do echo -n "$pid "; awk '/VmSwap/{print $2" kB"}' /proc/$pid/status; done

How to diagnose it

1. Confirm the kill is OOM. Match the PID from the FPM log line to a kernel or cgroup OOM entry. If neither dmesg nor the unit journal shows an OOM kill for that PID, the SIGKILL came from something else: a manual kill -9, systemd KillMode, or an OOM killer in a parent cgroup. Do not assume OOM without the matching evidence.
2. Determine the limiting boundary. Is the constraint host RAM, a systemd MemoryMax, or a container limit? Run free -m for the host view and systemctl show php-fpm -p MemoryMax for the unit view. In containers, check the orchestrator’s memory limit. The fix depends on which boundary is firing.
3. Sample per-worker RSS over time. A single snapshot tells you nothing. You need the slope. Poll ps or smem every few minutes and record the max and average. Monotonic growth between request cycles is the leak signature. A plateau means the baseline is just high.
4. Compute the memory ceiling. Formula: (total_RAM * 0.7 - OS_overhead) / avg_worker_PSS. Use PSS, not RSS, for the average. If your configured max_children times peak PSS already approaches the limit, the cliff is structural, not a leak. Lowering max_children is the fix.
5. Identify the leaky endpoint, if there is one. Use the full status page (?full) to correlate high-RSS workers with their script and request URI. Enable the slow log (request_slowlog_timeout) if it is not already on. Workers whose RSS is multiples of the p50 are handling a pathological request.
6. Check the recycling rate. If pm.max_requests is set, workers should exit with code 0 after the limit and the master spawns a replacement. If you see workers with thousands of requests served despite a low max_requests, recycling is broken or the config is not applied.
7. Use PSS, not RSS, for capacity math. RSS sums overstate actual usage by 30-50% because forked workers share read-only pages including the opcache segment. Use PSS from smem or /proc/<pid>/smaps_rollup.

Metrics and signals to monitor

Fixes

Set pm.max_requests

This is the immediate mitigation. 500-1000 is the standard range. The worker finishes its current request and delivers the response before exiting, so there is no mid-request interruption. Set it in the pool config:

pm.max_requests = 500

Reload with SIGUSR2. The leak does not go away, but each worker is recycled before its RSS grows enough to matter.

The fork cost is negligible at these request counts. Even if your codebase has no leaks today, a future code path or extension upgrade can introduce one, and without pm.max_requests you have no safety net. Leaving it at 0 is the single most common PHP-FPM misconfiguration.

Size max_children by memory, not CPU

The common mistake is sizing max_children by CPU cores. PHP-FPM workers spend most of their time waiting on I/O, not computing. The binding constraint is memory.

Compute the ceiling before raising max_children:

memory_safe_max_children = (total_RAM * 0.7 - OS_overhead) / avg_worker_PSS

A 60MB RSS worker might have a PSS of 35MB. If the configured max_children already exceeds the safe number, lowering it is the fix. Adding more workers without memory headroom makes the cliff arrive sooner.

Add memory or raise the cgroup limit

If the application legitimately needs the workers and the per-worker memory is not a leak, the constraint is capacity. Options:

• Add host RAM.
• Raise MemoryMax on the systemd unit, and confirm the physical headroom supports it.
• Raise the container memory limit in the orchestrator.

None of these fix a leak. They buy runway. If per-worker RSS is still climbing, pm.max_requests is still the primary defense.

Address extension-specific bloat

Some extensions allocate outside the PHP heap and are not bounded by memory_limit. ImageMagick with OpenMP is the classic case: a single worker doing image resizing can spawn multiple threads, each with its own glibc malloc arena, ballooning VSZ and RSS. The mitigation is to cap thread and arena counts in the pool config:

env[OMP_NUM_THREADS] = 1
env[MAGICK_THREAD_LIMIT] = 1
env[MALLOC_ARENA_MAX] = 2

This caps the per-worker blast radius for memory-heavy extensions. The application still works; it just does not parallelize inside a single request.

Prevention

• Set pm.max_requests on every production pool. 500 is a reasonable starting point.
• Monitor per-worker RSS as a trend, not a snapshot. Alert on monotonic growth between request cycles, not on a single threshold.
• Use PSS for capacity math. smem or /proc/<pid>/smaps_rollup gives the accurate number.
• Re-run the max_children formula whenever the application baseline changes: new framework, new extension, heavier endpoints.
• Track cgroup memory separately from host memory in containers. memory.current, memory.max, and memory.events.oom_kill tell you what the kernel actually enforces, which is not the same as host free RAM.
• Do not rely on emergency_restart_threshold for OOM cascades. It only counts SIGSEGV and SIGBUS.

How Netdata helps

• Per-second process RSS collection lets you see the leak slope that a 10-second poll interval misses. The cliff forms over hours, but the moment of saturation unfolds in seconds.
• cgroup memory metrics (memory.current, memory.max, memory.events.oom_kill) are collected automatically for systemd units and containers, so the cgroup OOM boundary is visible without manual journalctl digging.
• Correlation between FPM worker counts, per-worker RSS, and system memory shortens diagnosis: you see total memory climbing as worker RSS climbs, and the OOM kill lands at the intersection.
• Swap monitoring catches the thrash window that precedes the cliff.
• ML anomaly detection on per-worker RSS flags gradual upward drift before it crosses a static threshold, which catches slow leaks earlier than fixed alerting.

Related guides

• PHP-FPM sizing pm.max_children: by memory, not by CPU cores
• PHP-FPM active processes near max_children: reading pool utilization
• PHP-FPM monitoring checklist: the signals every production pool needs
• PHP-FPM monitoring maturity model: from survival to expert
• How PHP-FPM actually works in production: a mental model for operators
• PHP-FPM slow log: turning on request_slowlog_timeout to see what is slow
• PHP-FPM slow request cascade: one slow dependency drains the whole pool