A 504 from nginx means the FastCGI connection was established, the request was handed to a worker, and the worker never produced a response before nginx gave up. This is a different failure from a 502, where nginx could not connect at all (socket missing, backlog full, or master dead). The FastCGI channel was healthy. The work was not.

The default fastcgi_read_timeout in nginx is 60 seconds, measured between successive read operations on the socket, not for the response as a whole. Once it fires, nginx closes its side and returns 504. PHP-FPM has no idea this happened. The worker keeps executing, builds the full response, and writes it to a socket nobody is reading. This is the phantom worker problem, and it is the central operational trap of the 504 symptom.

The fix is never just “raise the timeout.” The timeout is doing its job: it is telling you that a subset of requests exceeds your user-facing latency budget. The actual work is to identify which requests are slow, why they are slow, and align the three timeout layers (PHP max_execution_time, FPM request_terminate_timeout, nginx fastcgi_read_timeout) so stuck workers are reclaimed instead of left running on abandoned requests.

What this means

The worker pool may not even be saturated. A single endpoint that hangs for 90 seconds produces 504s for the users hitting it while the rest of the site stays fast. The signature is:

  • nginx error log shows upstream timed out lines referencing the FastCGI socket.
  • PHP-FPM ping answers. The status page renders.
  • active processes is nowhere near max_children.
  • The slow log (if configured) shows the same endpoints repeatedly.

The failure mechanism is a timeout cascade across three independent clocks that almost never start aligned.

flowchart TD
    A[Request enters FPM worker] --> B{Work is CPU-bound?}
    B -->|Yes| C[max_execution_time 30s kills it]
    B -->|No, I/O blocked| D[max_execution_time does NOT fire]
    D --> E{request_terminate_timeout set?}
    E -->|0, disabled| F[Worker runs until script ends]
    E -->|e.g. 30s| G[Master kills worker]
    F --> H{nginx fastcgi_read_timeout fires first?}
    H -->|Yes| I[nginx returns 504 to client]
    I --> J[Worker keeps running: phantom worker]
    H -->|No| K[Response delivered to client]

The three clocks that matter:

  • PHP max_execution_time (default 30s). Counts CPU time, not wall-clock, on Linux/Unix. sleep(), network I/O, database queries, and system calls do not count toward it. A script that opens a socket and blocks for 120 seconds never trips max_execution_time. This is the most common misunderstanding operators bring to 504 diagnosis.
  • PHP-FPM request_terminate_timeout (default 0, disabled). When set, the FPM master kills the worker after the configured wall-clock duration. Enforced at the master level: ini_set() and set_time_limit() cannot extend it. If it is 0, nothing in PHP-FPM will ever kill a stuck worker.
  • nginx fastcgi_read_timeout (default 60s). Measured between successive reads, not for the total response. A script that trickles partial output can keep resetting this timer and run far past the nominal 60 seconds.

The recommended ordering, from lowest to highest, is max_execution_time < request_terminate_timeout < fastcgi_read_timeout. If fastcgi_read_timeout is lower than request_terminate_timeout, nginx returns 504 while the FPM worker keeps running: the phantom worker scenario.

Common causes

CauseWhat it looks likeFirst thing to check
Slow upstream dependency (database, external API, Redis, NFS)Slow log stack traces show PDO::*, curl_exec, Redis::*, or flock at the top; per-worker request duration is bimodalThe dependency’s own metrics: DB slow query log, API latency, Redis latency
request_terminate_timeout = 0 with a stuck requestactive processes stays flat, one or two workers show request duration climbing into minutes with no corresponding CPUPool config: the request_terminate_timeout line
fastcgi_read_timeout lower than request_terminate_timeoutnginx returns 504 but FPM status shows the worker still Running the same URI seconds laternginx fastcgi_read_timeout vs pool request_terminate_timeout
max_execution_time treated as wall-clockScript blocks on network I/O, never hits the 30s limit, runs until nginx times outphp.ini max_execution_time and where the script actually spends time (slow log)
Session lock contention (file-based sessions)Slow log shows session_start near the top; affects only same-session concurrent requestslsof on session files; session_write_close() placement in code

Quick checks

Read-only and safe to run during an incident. Adjust paths and status URLs to match your distribution and configuration.

# Confirm 504s are FastCGI timeouts, not proxy timeouts
grep "upstream timed out" /var/log/nginx/error.log | tail -20

# nginx: 504 count in the current hour
grep "$(date '+%d/%b/%Y:%H')" /var/log/nginx/access.log | grep -c " 504 "

# FPM: is the master alive and the socket listening?
pgrep -f "php-fpm: master" && ss -lxn | grep php

# FPM: pool saturation snapshot
curl -s http://127.0.0.1/fpm-status

# FPM: per-worker detail, including request duration and script
curl -s 'http://127.0.0.1/fpm-status?full' | grep -E "pid|state|request duration|request URI|script"

# FPM: slow log tail (only useful if request_slowlog_timeout is set)
tail -100 /var/log/php-fpm/slow.log

# FPM: verify pool-level timeout configuration
php-fpm -tt 2>&1 | grep -E "request_terminate_timeout|request_slowlog_timeout"

# PHP: max_execution_time from the FPM SAPI (CLI defaults to 0/unlimited, do not use php -i)
grep -r max_execution_time /etc/php/*/fpm/ 2>/dev/null

If request_slowlog_timeout is 0 or absent from the php-fpm -tt output, the slow log is disabled. You will not be able to localize the slow path from PHP-FPM alone until you enable it.

How to diagnose it

  1. Confirm the 504 is a FastCGI timeout, not a proxy timeout. The nginx error line should reference the FastCGI socket or the fastcgi_pass upstream, not proxy_pass. If you see upstream timed out against a proxy_pass upstream, this article is not the right diagnostic path.

  2. Capture the slow path before it moves. Pull the full status page and look for workers in Running state with request duration values above your p99. Note the script and request URI fields. The duration field is in microseconds.

  3. Read the slow log. If request_slowlog_timeout is set (for example, 5s), every entry shows the script path and a PHP backtrace captured when the master sent SIGSTOP to the worker. The frame at the top of the trace is where the worker was blocked. Common signatures: PDO::query or PDO::prepare (database), curl_exec or file_get_contents on an HTTP URL (external API), Redis::* (cache), flock or session_start (session lock).

  4. Check for phantom workers. Compare the timestamp of a recent nginx 504 with the FPM full status. If the same worker is still Running the same URI after nginx has already returned 504, you have phantom workers. This confirms a timeout ordering problem: fastcgi_read_timeout is firing before request_terminate_timeout (or before the script finishes, if request_terminate_timeout is 0).

  5. Verify the three timeout layers and their ordering. Check max_execution_time in php.ini, request_terminate_timeout in the pool config, and fastcgi_read_timeout in nginx. They should be ordered max_execution_time < request_terminate_timeout < fastcgi_read_timeout. Any other ordering produces either premature 504s or phantom workers.

  6. Correlate with the dependency. If the slow log points at the database, check the database slow query log and connection count at the same timestamp. If it points at an external API, check the API’s latency from the app host. The goal is to confirm the slow path is upstream of PHP, not inside PHP itself.

Metrics and signals to monitor

SignalWhy it mattersWarning sign
nginx 504 rateThe user-facing symptom itselfAny sustained non-zero rate on PHP endpoints
Per-worker request duration (full status)Localizes which workers are stuck and on which scriptWorkers with durations more than 10x the p99 baseline
Slow log entry rateTells you what is slow, not just that something is slowSudden increase above baseline; concentration on specific scripts
active processes / max_children ratioRules worker exhaustion in or out as a compounding factorSustained above 80% means slow requests are also consuming pool capacity
request_terminate_timeout configurationDetermines whether stuck workers are ever reclaimedValue of 0 means stuck workers run until the script ends or the pool restarts
Dependency latency (DB, API, cache)The root cause in most 504 incidentsSpikes that line up with the nginx 504 rate

Fixes

Align the three timeout layers

The ordering must be max_execution_time < request_terminate_timeout < fastcgi_read_timeout. A common working configuration:

  • max_execution_time = 30 (php.ini). CPU-bound work is killed with a fatal error that produces a trace.
  • request_terminate_timeout = 30 (pool config). Wall-clock safety net. The master kills the worker, so there is no PHP-level trace, but the worker is reclaimed. Cannot be overridden by ini_set() or set_time_limit().
  • fastcgi_read_timeout = 60 (nginx). Gives nginx a window longer than FPM’s terminate timeout, so FPM reclaims the worker before nginx gives up. This avoids phantom workers.

Some operators set request_terminate_timeout slightly higher than max_execution_time to give CPU-bound scripts a chance to hit max_execution_time first, since the fatal error produces a more useful trace than a master kill. The exact values are application-dependent. The ordering is not.

If you have long-running endpoints (webhooks, synchronous report generation, exports), exclude them from the global timeout policy. The right answer is usually a job queue, not a global timeout high enough to accommodate the slowest endpoint.

Enable the slow log

In the pool config:

request_slowlog_timeout = 5
slowlog = /var/log/php-fpm/slow.log

Set the threshold low enough to catch the requests causing 504s but high enough that normal traffic does not flood the log. For an application with a p99 around 200ms, a threshold of 5 seconds catches only pathological requests. For a heavier application, 10 seconds may be more appropriate. The slow log mechanism uses SIGSTOP/SIGCONT to capture the trace, which adds a small delay to the already-slow request. Under extreme conditions (thousands of slow requests per second), the logging I/O itself can compound the problem.

Address the actual slow path

The timeout alignment stops the bleeding. The slow log tells you where to look. Common fixes:

  • Database: add missing indexes, fix lock contention, or move the query behind a connection pool (pgbouncer, ProxySQL) if the bottleneck is connection acquisition rather than query execution.
  • External API: set an explicit, low timeout in the HTTP client. PHP’s cURL default has no timeout (infinite) unless your application sets one. Add a circuit breaker so a failing API does not convert every FPM worker into a waiting thread.
  • Session locks: call session_write_close() as early as possible in the request lifecycle, or switch to Redis or Memcached session handlers, which have different locking semantics.
  • NFS or shared filesystem: check for stall or high latency on the mount. Consider local caching of the files in question.

Reclaim phantom workers during the incident

If you are mid-incident and workers are stuck on abandoned requests, you can reclaim individual workers:

# Identify long-running workers from full status, then send SIGQUIT (graceful)
kill -SIGQUIT <worker_pid>

SIGQUIT tells the worker to finish the current request and exit. For a worker already running a phantom request that nobody will read, this ends the useless work. The master respawns a replacement. Avoid SIGKILL on individual workers unless the worker is not responding to signals. Note: if the worker is blocked in a system call (network I/O, disk I/O), the signal handler will not run until the call returns or is interrupted.

Prevention

  • Enable request_slowlog_timeout on every production pool. It is the single most direct signal for localizing slow work. Without it, 504 diagnosis degrades into correlating nginx error timestamps with APM traces and database logs by hand.
  • Set request_terminate_timeout to a finite value. A value of 0 means any stuck request (deadlock, infinite loop, hung socket) permanently occupies a worker until the next FPM restart.
  • Order the three timeouts correctly and document why. max_execution_time < request_terminate_timeout < fastcgi_read_timeout. Record the chosen values and reasoning so the next operator does not undo the alignment.
  • Monitor the slow log entry rate, not just the counter. The counter resets on restart. Alert on rate of change and on concentration against specific scripts.
  • Route long-running endpoints through a job queue or a dedicated nginx location. Do not pollute the global timeout policy with per-endpoint exceptions.
  • On PHP 7.3+, consider request_terminate_timeout_track_finished. Defaults to no, which means the terminate timeout does not apply after fastcgi_finish_request() or during shutdown functions. If your application does long work in shutdown handlers, enabling it prevents those paths from escaping the timeout.

How Netdata helps

  • Per-second collection of active processes, idle processes, listen queue, and max children reached from the FPM status page lets you see the saturation signature of a slow-request cascade as it forms, not minutes later.
  • The PHP-FPM collector surfaces the slow requests counter as a rate, so a spike in slow log entries is visible alongside the nginx 504 rate without manual log correlation.
  • Correlating nginx upstream error rate with FPM pool saturation in a single view confirms whether a 504 is a slow-worker problem (this article) or a connection-refused problem (worker exhaustion or listen backlog overflow).
  • ML-based anomaly detection on active process count and average request duration can surface the bimodal latency distribution that precedes user-visible 504s.