Users see 504 Gateway Timeout. You check PHP-FPM and the pool is not saturated, or it is saturated but the active workers are running requests that should have finished minutes ago. The nginx error log shows upstream timeouts. The FPM error log shows nothing unusual.

nginx and PHP-FPM each have their own notion of how long a request may run. When those notions disagree, you get two failure modes: phantom workers (nginx gave up, FPM kept going) and confusing 502/504 patterns (FPM killed a worker nginx was still waiting on). The configuration is incoherent across the request path.

A related mismatch: nginx worker_connections is typically far higher than FPM pm.max_children. nginx accepts traffic bursts that FPM cannot serve, so nginx’s connection acceptance rate has no relationship to FPM’s actual processing capacity.

What this means

PHP-FPM workers handle exactly one request at a time. When nginx forwards a request to FPM via FastCGI, two independent clocks start:

  • nginx starts the fastcgi_read_timeout clock (default 60s). This is the time between successive read operations from the FastCGI upstream, not the total response time.
  • FPM starts the request_terminate_timeout clock (default 0, meaning disabled). When set, the worker is killed after this wall-clock duration.

If these clocks disagree, one side gives up before the other. The side that gives up first determines what the user sees.

Phantom workers: nginx times out first (fastcgi_read_timeout < request_terminate_timeout, or request_terminate_timeout is 0). nginx returns 504 and closes its side of the FastCGI connection. The FPM worker does not know the client is gone. It keeps running the request, holding a worker slot for a response nobody will read. If request_terminate_timeout is 0 (the default), that worker runs until the script finishes naturally, which could be never (infinite loop, hung database call, deadlocked external API).

Confusing 502s: FPM times out first (request_terminate_timeout < fastcgi_read_timeout). FPM kills the worker. nginx is still waiting on the FastCGI socket and gets an unexpected connection close. nginx logs “upstream prematurely closed connection” and returns 502. The user sees a 502, not a 504, even though the root cause is a timeout.

A third clock exists but is less useful as a safety net: max_execution_time (PHP ini, default 30s). On Linux, this timer uses ITIMER_PROF, which counts CPU time, not wall-clock time. A script blocked on I/O (database, HTTP, sleep) does not accumulate CPU time and will not trigger max_execution_time. This is why request_terminate_timeout (wall-clock) is the more reliable backstop for stuck workers.

flowchart TD
    A["Request arrives at nginx"] --> B["nginx forwards to FPM worker"]
    B --> C{"Which timeout fires first?"}
    C -->|"nginx fastcgi_read_timeout"| D["nginx returns 504 to client"]
    D --> E["FPM worker still running"]
    E --> F["Phantom worker holds slot"]
    C -->|"FPM request_terminate_timeout"| G["FPM kills worker"]
    G --> H["nginx still waiting on socket"]
    H --> I["nginx logs upstream closed, returns 502"]

Common causes

CauseWhat it looks likeFirst thing to check
request_terminate_timeout = 0 (default)Workers stuck in Running state for minutes; nginx returns 504 but FPM shows active workers with no errorsgrep request_terminate_timeout in pool config
fastcgi_read_timeout left at default 60snginx returns 504 at 60s even though FPM would finish at 65snginx -T 2>/dev/null | grep fastcgi_read_timeout
nginx timeout much higher than FPM timeoutnginx logs “upstream prematurely closed”; users see 502 not 504Compare both timeout values directly
worker_connections » max_children with no rate limitingnginx accepts burst, FPM listen queue fills, 502s appear under loadCompare worker_connections against pm.max_children
Streaming responses with long pausesnginx fires fastcgi_read_timeout between chunks even though total time is under limitCheck if application sends output in chunks with gaps

Quick checks

# Check FPM request_terminate_timeout for each pool
grep -r "request_terminate_timeout" /etc/php/*/fpm/pool.d/

# Check nginx fastcgi timeouts
nginx -T 2>/dev/null | grep "fastcgi_.*timeout"

# Compare worker_connections vs max_children
nginx -T 2>/dev/null | grep worker_connections
grep -r "pm.max_children" /etc/php/*/fpm/pool.d/

# Look for phantom workers: Running workers with very long durations.
# NOTE: the status path (here /fpm-status) must match pm.status_path in your FPM pool config.
curl -s "http://127.0.0.1/fpm-status?json&full" | python3 -c "
import sys, json
data = json.load(sys.stdin)
for p in data['processes']:
    if p['state'] == 'Running' and p['request duration'] > 30000000:
        print(f\"PID {p['pid']}: {p['request duration']/1e6:.1f}s - {p['request uri']}\")"

# jq equivalent (if python3 is unavailable):
# curl -s "http://127.0.0.1/fpm-status?json&full" | jq -r \
#   '.processes[] | select(.state=="Running" and .["request duration"]>30000000) \
#   | "PID \(.pid): \(.["request duration"]/1e6|round)s - \(.["request uri"])"'

# Check nginx for upstream timeout and premature close errors
grep "upstream timed out\|upstream prematurely closed" /var/log/nginx/error.log | tail -20

# Check FPM listen queue (are bursts already queuing?)
curl -s http://127.0.0.1/fpm-status | grep "listen queue"

How to diagnose it

  1. Pull both timeout values. Get request_terminate_timeout from every FPM pool config and fastcgi_read_timeout from nginx config. If either is at its default (0 for FPM, 60s for nginx), that is likely the mismatch.

  2. Determine which side gives up first. If request_terminate_timeout is 0 or greater than fastcgi_read_timeout, nginx gives up first and you will see phantom workers. If request_terminate_timeout is set and less than fastcgi_read_timeout, FPM gives up first and you will see 502s with “upstream prematurely closed” in nginx logs.

  3. Check for phantom workers. Pull the full FPM status page and look for workers in Running state with request duration exceeding your nginx timeout. These are workers processing responses that nginx has already abandoned. The request duration field is in microseconds: 60,000,000 is 60 seconds.

  4. Correlate nginx 502/504 timestamps with FPM worker state. If nginx logs a 504 at time T, check whether FPM still shows that request running at time T+5s. If yes, it is a phantom worker. If nginx logs a 502 with “upstream prematurely closed” at time T, check whether FPM logged a worker kill around the same time.

  5. Check the listen queue and max_children. If worker_connections is 1024 and max_children is 20, nginx can accept 50x more concurrent connections than FPM can process. Under burst, the listen queue fills and connections are dropped at the kernel level. Check ss -xlnp | grep php (Unix socket) or ss -tlnp | grep 9000 (TCP) for Recv-Q approaching the backlog limit.

  6. Verify the streaming edge case. nginx’s fastcgi_read_timeout is between successive read operations, not total response time. A script that sends a byte every 59 seconds can run for hours without triggering a 60s fastcgi_read_timeout, but request_terminate_timeout would kill it at its configured limit. If your application streams output with pauses, this asymmetry matters.

Metrics and signals to monitor

SignalWhy it mattersWarning sign
FPM active processes with long request durationIdentifies phantom workers still running after nginx gave upWorkers in Running state with duration > fastcgi_read_timeout
nginx 504 ratenginx gave up waiting for FPMSustained non-zero rate during normal traffic
nginx 502 rate with “upstream prematurely closed”FPM killed a worker nginx was still readingCorrelates with request_terminate_timeout firing
FPM listen queue depthBursts are queuing because max_children is too low relative to nginx acceptanceSustained non-zero value
FPM max children reached counterPool hit its ceiling under burstCounter incrementing during normal traffic
Kernel ListenOverflows / ListenDropsConnections dropped at kernel level when backlog is fullCounters increasing in /proc/net/netstat

Fixes

Make the timeout chain coherent

The goal is for both sides to agree on when a request should end. There are two coherent strategies:

Strategy A: nginx gives up first, FPM cleans up. Set fastcgi_read_timeout slightly less than request_terminate_timeout. For example, fastcgi_read_timeout 55s and request_terminate_timeout = 60s. nginx returns a clean 504 at 55s. FPM kills the worker at 60s, closing the phantom worker window to 5 seconds. This is the common operator preference because users get a predictable 504 and phantom workers are short-lived.

Strategy B: FPM gives up first, nginx reports it. Set request_terminate_timeout less than fastcgi_read_timeout. For example, request_terminate_timeout = 55s and fastcgi_read_timeout 60s. FPM kills the worker at 55s. nginx sees the connection close and returns 502. Users see a 502, not a 504. This is less common because 502 is a worse user experience, but it ensures FPM is the authority on request lifetime.

Either strategy requires request_terminate_timeout to be set (not 0). With the default of 0, there is no FPM-side cleanup at all.

Set request_terminate_timeout_track_finished on PHP 7.3+

request_terminate_timeout_track_finished (default “no”) controls whether the timeout applies after fastcgi_finish_request() or during shutdown functions. If your application calls fastcgi_finish_request() to send a response early and then does background work, the default “no” means the timeout does not cover that background work. Set it to “yes” if you want the timeout to cover the full worker lifecycle.

Address the worker_connections mismatch

nginx worker_connections is per worker. Effective max connections is worker_connections * worker_processes, which is typically 2048-8192 in production. FPM max_children is typically 20-50. Under burst, nginx accepts connections that FPM cannot serve, filling the listen queue and eventually triggering kernel-level drops.

Options:

  • Add rate limiting at the nginx layer (limit_req) to cap the request rate to what FPM can handle.
  • Increase pm.max_children if memory allows. A rough budget: avg_worker_RSS * new_max_children + OS_overhead should be less than ~70% of total RAM.
  • Use pm = static to eliminate dynamic scaling lag under bursts.

Kill existing phantom workers

If phantom workers have accumulated and are holding slots, you can target them individually:

# Identify phantom workers (Running with duration > fastcgi_read_timeout)
curl -s "http://127.0.0.1/fpm-status?json&full" | python3 -c "
import sys, json
data = json.load(sys.stdin)
for p in data['processes']:
    if p['state'] == 'Running' and p['request duration'] > 60000000:
        print(p['pid'])"

SIGQUIT requests graceful termination of a single worker. The master process will respawn a replacement.

kill -SIGQUIT <pid>

Do not use kill -9 or kill -SIGKILL unless the worker is completely unresponsive. Restarting the entire FPM pool terminates all workers including phantoms, but causes a brief no-worker window during the reload. Use systemctl reload <php-fpm-service> (service name varies by distribution and PHP version: php8.1-fpm, php-fpm, etc.) or send SIGUSR2 to the FPM master process (kill -SIGUSR2 <master-pid>).

Prevention

  • Set request_terminate_timeout on every pool. The default of 0 means a single stuck request permanently removes a worker. A value of 30-60 seconds (application-dependent) is the safety net.
  • Document the timeout chain. Record the relationship between fastcgi_read_timeout, request_terminate_timeout, max_execution_time, and any upstream API timeouts. Outer layers should have longer timeouts than inner layers, and the gap between fastcgi_read_timeout and request_terminate_timeout should be small enough to avoid phantom workers.
  • Alert on phantom workers. Alert on workers in Running state with durations exceeding fastcgi_read_timeout. That means the timeout chain is broken.
  • Review after every config change. Timeout mismatches are often introduced when one team changes nginx config and another changes FPM config without coordination.
  • Enable the slow log. Set request_slowlog_timeout (e.g., 5s). The slow log captures stack traces that show where workers are blocked, which is essential for distinguishing “stuck because of a timeout mismatch” from “stuck because of a slow database query.”

How Netdata helps

  • Per-second FPM status polling catches phantom workers as they form, including transient ones in the narrow window between nginx timeout and FPM cleanup. Coarse polling intervals (10s or higher) can miss short-lived phantoms, though persistent phantoms (from request_terminate_timeout = 0) are visible at any interval.
  • Active processes with request duration lets you correlate worker duration against your configured fastcgi_read_timeout. Workers exceeding that threshold are phantoms.
  • nginx 502 and 504 rate metrics alongside FPM active/idle process counts let you see whether nginx is giving up before or after FPM.
  • Listen queue depth from the FPM status page, correlated with nginx connection rates, shows whether the worker_connections » max_children gap is causing kernel-level drops under burst.
  • ML anomaly detection on active process count and request duration surfaces the slow drift toward phantom worker accumulation before it triggers max_children exhaustion.