Users see 504 Gateway Timeout. You check PHP-FPM and the pool is not saturated, or it is saturated but the active workers are running requests that should have finished minutes ago. The nginx error log shows upstream timeouts. The FPM error log shows nothing unusual.
nginx and PHP-FPM each have their own notion of how long a request may run. When those notions disagree, you get two failure modes: phantom workers (nginx gave up, FPM kept going) and confusing 502/504 patterns (FPM killed a worker nginx was still waiting on). The configuration is incoherent across the request path.
A related mismatch: nginx worker_connections is typically far higher than FPM pm.max_children. nginx accepts traffic bursts that FPM cannot serve, so nginx’s connection acceptance rate has no relationship to FPM’s actual processing capacity.
What this means
PHP-FPM workers handle exactly one request at a time. When nginx forwards a request to FPM via FastCGI, two independent clocks start:
- nginx starts the
fastcgi_read_timeoutclock (default 60s). This is the time between successive read operations from the FastCGI upstream, not the total response time. - FPM starts the
request_terminate_timeoutclock (default 0, meaning disabled). When set, the worker is killed after this wall-clock duration.
If these clocks disagree, one side gives up before the other. The side that gives up first determines what the user sees.
Phantom workers: nginx times out first (fastcgi_read_timeout < request_terminate_timeout, or request_terminate_timeout is 0). nginx returns 504 and closes its side of the FastCGI connection. The FPM worker does not know the client is gone. It keeps running the request, holding a worker slot for a response nobody will read. If request_terminate_timeout is 0 (the default), that worker runs until the script finishes naturally, which could be never (infinite loop, hung database call, deadlocked external API).
Confusing 502s: FPM times out first (request_terminate_timeout < fastcgi_read_timeout). FPM kills the worker. nginx is still waiting on the FastCGI socket and gets an unexpected connection close. nginx logs “upstream prematurely closed connection” and returns 502. The user sees a 502, not a 504, even though the root cause is a timeout.
A third clock exists but is less useful as a safety net: max_execution_time (PHP ini, default 30s). On Linux, this timer uses ITIMER_PROF, which counts CPU time, not wall-clock time. A script blocked on I/O (database, HTTP, sleep) does not accumulate CPU time and will not trigger max_execution_time. This is why request_terminate_timeout (wall-clock) is the more reliable backstop for stuck workers.
flowchart TD
A["Request arrives at nginx"] --> B["nginx forwards to FPM worker"]
B --> C{"Which timeout fires first?"}
C -->|"nginx fastcgi_read_timeout"| D["nginx returns 504 to client"]
D --> E["FPM worker still running"]
E --> F["Phantom worker holds slot"]
C -->|"FPM request_terminate_timeout"| G["FPM kills worker"]
G --> H["nginx still waiting on socket"]
H --> I["nginx logs upstream closed, returns 502"]Common causes
| Cause | What it looks like | First thing to check |
|---|---|---|
request_terminate_timeout = 0 (default) | Workers stuck in Running state for minutes; nginx returns 504 but FPM shows active workers with no errors | grep request_terminate_timeout in pool config |
fastcgi_read_timeout left at default 60s | nginx returns 504 at 60s even though FPM would finish at 65s | nginx -T 2>/dev/null | grep fastcgi_read_timeout |
| nginx timeout much higher than FPM timeout | nginx logs “upstream prematurely closed”; users see 502 not 504 | Compare both timeout values directly |
worker_connections » max_children with no rate limiting | nginx accepts burst, FPM listen queue fills, 502s appear under load | Compare worker_connections against pm.max_children |
| Streaming responses with long pauses | nginx fires fastcgi_read_timeout between chunks even though total time is under limit | Check if application sends output in chunks with gaps |
Quick checks
# Check FPM request_terminate_timeout for each pool
grep -r "request_terminate_timeout" /etc/php/*/fpm/pool.d/
# Check nginx fastcgi timeouts
nginx -T 2>/dev/null | grep "fastcgi_.*timeout"
# Compare worker_connections vs max_children
nginx -T 2>/dev/null | grep worker_connections
grep -r "pm.max_children" /etc/php/*/fpm/pool.d/
# Look for phantom workers: Running workers with very long durations.
# NOTE: the status path (here /fpm-status) must match pm.status_path in your FPM pool config.
curl -s "http://127.0.0.1/fpm-status?json&full" | python3 -c "
import sys, json
data = json.load(sys.stdin)
for p in data['processes']:
if p['state'] == 'Running' and p['request duration'] > 30000000:
print(f\"PID {p['pid']}: {p['request duration']/1e6:.1f}s - {p['request uri']}\")"
# jq equivalent (if python3 is unavailable):
# curl -s "http://127.0.0.1/fpm-status?json&full" | jq -r \
# '.processes[] | select(.state=="Running" and .["request duration"]>30000000) \
# | "PID \(.pid): \(.["request duration"]/1e6|round)s - \(.["request uri"])"'
# Check nginx for upstream timeout and premature close errors
grep "upstream timed out\|upstream prematurely closed" /var/log/nginx/error.log | tail -20
# Check FPM listen queue (are bursts already queuing?)
curl -s http://127.0.0.1/fpm-status | grep "listen queue"
How to diagnose it
Pull both timeout values. Get
request_terminate_timeoutfrom every FPM pool config andfastcgi_read_timeoutfrom nginx config. If either is at its default (0 for FPM, 60s for nginx), that is likely the mismatch.Determine which side gives up first. If
request_terminate_timeoutis 0 or greater thanfastcgi_read_timeout, nginx gives up first and you will see phantom workers. Ifrequest_terminate_timeoutis set and less thanfastcgi_read_timeout, FPM gives up first and you will see 502s with “upstream prematurely closed” in nginx logs.Check for phantom workers. Pull the full FPM status page and look for workers in Running state with
request durationexceeding your nginx timeout. These are workers processing responses that nginx has already abandoned. Therequest durationfield is in microseconds: 60,000,000 is 60 seconds.Correlate nginx 502/504 timestamps with FPM worker state. If nginx logs a 504 at time T, check whether FPM still shows that request running at time T+5s. If yes, it is a phantom worker. If nginx logs a 502 with “upstream prematurely closed” at time T, check whether FPM logged a worker kill around the same time.
Check the listen queue and max_children. If
worker_connectionsis 1024 andmax_childrenis 20, nginx can accept 50x more concurrent connections than FPM can process. Under burst, the listen queue fills and connections are dropped at the kernel level. Checkss -xlnp | grep php(Unix socket) orss -tlnp | grep 9000(TCP) for Recv-Q approaching the backlog limit.Verify the streaming edge case. nginx’s
fastcgi_read_timeoutis between successive read operations, not total response time. A script that sends a byte every 59 seconds can run for hours without triggering a 60sfastcgi_read_timeout, butrequest_terminate_timeoutwould kill it at its configured limit. If your application streams output with pauses, this asymmetry matters.
Metrics and signals to monitor
| Signal | Why it matters | Warning sign |
|---|---|---|
| FPM active processes with long request duration | Identifies phantom workers still running after nginx gave up | Workers in Running state with duration > fastcgi_read_timeout |
| nginx 504 rate | nginx gave up waiting for FPM | Sustained non-zero rate during normal traffic |
| nginx 502 rate with “upstream prematurely closed” | FPM killed a worker nginx was still reading | Correlates with request_terminate_timeout firing |
| FPM listen queue depth | Bursts are queuing because max_children is too low relative to nginx acceptance | Sustained non-zero value |
| FPM max children reached counter | Pool hit its ceiling under burst | Counter incrementing during normal traffic |
| Kernel ListenOverflows / ListenDrops | Connections dropped at kernel level when backlog is full | Counters increasing in /proc/net/netstat |
Fixes
Make the timeout chain coherent
The goal is for both sides to agree on when a request should end. There are two coherent strategies:
Strategy A: nginx gives up first, FPM cleans up. Set fastcgi_read_timeout slightly less than request_terminate_timeout. For example, fastcgi_read_timeout 55s and request_terminate_timeout = 60s. nginx returns a clean 504 at 55s. FPM kills the worker at 60s, closing the phantom worker window to 5 seconds. This is the common operator preference because users get a predictable 504 and phantom workers are short-lived.
Strategy B: FPM gives up first, nginx reports it. Set request_terminate_timeout less than fastcgi_read_timeout. For example, request_terminate_timeout = 55s and fastcgi_read_timeout 60s. FPM kills the worker at 55s. nginx sees the connection close and returns 502. Users see a 502, not a 504. This is less common because 502 is a worse user experience, but it ensures FPM is the authority on request lifetime.
Either strategy requires request_terminate_timeout to be set (not 0). With the default of 0, there is no FPM-side cleanup at all.
Set request_terminate_timeout_track_finished on PHP 7.3+
request_terminate_timeout_track_finished (default “no”) controls whether the timeout applies after fastcgi_finish_request() or during shutdown functions. If your application calls fastcgi_finish_request() to send a response early and then does background work, the default “no” means the timeout does not cover that background work. Set it to “yes” if you want the timeout to cover the full worker lifecycle.
Address the worker_connections mismatch
nginx worker_connections is per worker. Effective max connections is worker_connections * worker_processes, which is typically 2048-8192 in production. FPM max_children is typically 20-50. Under burst, nginx accepts connections that FPM cannot serve, filling the listen queue and eventually triggering kernel-level drops.
Options:
- Add rate limiting at the nginx layer (
limit_req) to cap the request rate to what FPM can handle. - Increase
pm.max_childrenif memory allows. A rough budget:avg_worker_RSS * new_max_children + OS_overheadshould be less than ~70% of total RAM. - Use
pm = staticto eliminate dynamic scaling lag under bursts.
Kill existing phantom workers
If phantom workers have accumulated and are holding slots, you can target them individually:
# Identify phantom workers (Running with duration > fastcgi_read_timeout)
curl -s "http://127.0.0.1/fpm-status?json&full" | python3 -c "
import sys, json
data = json.load(sys.stdin)
for p in data['processes']:
if p['state'] == 'Running' and p['request duration'] > 60000000:
print(p['pid'])"
SIGQUIT requests graceful termination of a single worker. The master process will respawn a replacement.
kill -SIGQUIT <pid>
Do not use kill -9 or kill -SIGKILL unless the worker is completely unresponsive. Restarting the entire FPM pool terminates all workers including phantoms, but causes a brief no-worker window during the reload. Use systemctl reload <php-fpm-service> (service name varies by distribution and PHP version: php8.1-fpm, php-fpm, etc.) or send SIGUSR2 to the FPM master process (kill -SIGUSR2 <master-pid>).
Prevention
- Set
request_terminate_timeouton every pool. The default of 0 means a single stuck request permanently removes a worker. A value of 30-60 seconds (application-dependent) is the safety net. - Document the timeout chain. Record the relationship between
fastcgi_read_timeout,request_terminate_timeout,max_execution_time, and any upstream API timeouts. Outer layers should have longer timeouts than inner layers, and the gap betweenfastcgi_read_timeoutandrequest_terminate_timeoutshould be small enough to avoid phantom workers. - Alert on phantom workers. Alert on workers in Running state with durations exceeding
fastcgi_read_timeout. That means the timeout chain is broken. - Review after every config change. Timeout mismatches are often introduced when one team changes nginx config and another changes FPM config without coordination.
- Enable the slow log. Set
request_slowlog_timeout(e.g., 5s). The slow log captures stack traces that show where workers are blocked, which is essential for distinguishing “stuck because of a timeout mismatch” from “stuck because of a slow database query.”
How Netdata helps
- Per-second FPM status polling catches phantom workers as they form, including transient ones in the narrow window between nginx timeout and FPM cleanup. Coarse polling intervals (10s or higher) can miss short-lived phantoms, though persistent phantoms (from
request_terminate_timeout = 0) are visible at any interval. - Active processes with request duration lets you correlate worker duration against your configured
fastcgi_read_timeout. Workers exceeding that threshold are phantoms. - nginx 502 and 504 rate metrics alongside FPM active/idle process counts let you see whether nginx is giving up before or after FPM.
- Listen queue depth from the FPM status page, correlated with nginx connection rates, shows whether the
worker_connections»max_childrengap is causing kernel-level drops under burst. - ML anomaly detection on active process count and request duration surfaces the slow drift toward phantom worker accumulation before it triggers
max_childrenexhaustion.
Related guides
- PHP-FPM 504 Gateway Timeout: requests accepted but never finishing in time
- PHP-FPM active processes near max_children: reading pool utilization
- PHP-FPM in containers: cgroup limits and the silent OOM kill
- PHP-FPM “child N exited on signal 11 (SIGSEGV)”: worker segfaults
- PHP-FPM crash loop and fork storm: workers dying faster than they serve
- PHP-FPM dynamic mode scaling lag: why the pool cannot keep up with bursts
- PHP-FPM emergency restart: “failed processes threshold reached, initiating reload”
- PHP-FPM graceful reload: the brief no-worker window on SIGUSR2
- How PHP-FPM actually works in production: a mental model for operators
- PHP-FPM idle processes at zero: no burst headroom left
- PHP-FPM listen queue growing: the earliest signal of saturation
- PHP-FPM “server reached pm.max_children setting (N), consider raising it”






