PHP-FPM OPcache out of memory: oom_restarts and the recompile cliff

Hours or days of normal operation. Then CPU spikes across every PHP-FPM worker at once, request latency jumps uniformly, and throughput drops. When you check OPcache status, oom_restarts has incremented and free_memory is near zero. The shared memory segment filled, OPcache force-cleared the entire cache, and every worker is now recompiling PHP from disk simultaneously.

This is the recompile cliff. OPcache has no LRU eviction. When it runs out of space it either restarts (clearing everything) or silently stops caching new scripts.

There is a second, subtler failure mode. When the cache is full but wasted_memory is below opcache.max_wasted_percentage (default 5%), OPcache does not restart. New scripts that are not yet cached are left out. They get recompiled on every request as if OPcache were disabled for them. oom_restarts stays at zero. The only visible symptoms are a collapsing hit rate and rising CPU. This is the silent no-restart trap, and it is the harder one to catch.

What this means

OPcache allocates a single shared memory segment at PHP startup via mmap(MAP_SHARED), sized by opcache.memory_consumption (default 128 MB). All workers in a pool share this segment. When a worker compiles a PHP script, the bytecode is stored here. Subsequent requests for the same script hit the cache instead of recompiling.

When the segment fills, two counters track what happened:

• oom_restarts: increments when OPcache ran out of shared memory and triggered a full cache restart. The entire cache is cleared. Every worker must recompile every script it touches until the cache warms again. This produces a CPU stampede.
• hash_restarts: increments when the hash table (sized by opcache.max_accelerated_files) is full. Same outcome: full cache clear, mass recompilation.

The restart trigger is not simply “free memory is low.” A restart only fires when free memory is exhausted AND wasted_memory exceeds opcache.max_wasted_percentage of the total segment. If the cache is full but wasted memory is below that threshold, no restart occurs and new scripts are silently left uncached.

flowchart TD
    A["OPcache segment fills"] --> B{"Wasted above threshold?"}
    B -->|Yes| C["oom_restarts increments"]
    C --> D["Full cache clear"]
    D --> E["All workers recompile everything"]
    B -->|No| F["Silent: no restart"]
    F --> G["Only new scripts recompiled"]
    E --> H["CPU spikes, hit rate drops"]
    G --> H

wasted_memory grows when scripts are invalidated (file changes on disk) but the old bytecode cannot be freed due to fragmentation. Only a full restart clears it. After a deploy that changes many files, old and new bytecode coexist until a restart, causing a wasted memory spike.

Common causes

Quick checks

Get OPcache memory and restart status. The false argument returns summary without per-script details. Query through FPM, not CLI, because CLI has a separate OPcache instance:

# Requires a web-accessible script restricted to localhost or monitoring network:
# <?php echo json_encode(opcache_get_status(false)); ?>
curl -s http://127.0.0.1/opcache-status.php | python3 -c "
import sys, json
d = json.load(sys.stdin)
m = d['memory_usage']; s = d['opcache_statistics']
total = m['used_memory'] + m['free_memory'] + m['wasted_memory']
print(f\"Used: {m['used_memory']/1048576:.0f}MB  Free: {m['free_memory']/1048576:.0f}MB  Wasted: {m['wasted_memory']/1048576:.0f}MB\")
print(f\"Wasted pct: {m['wasted_memory']/total*100:.1f}%\")
print(f\"OOM restarts: {s['oom_restarts']}  Hash restarts: {s.get('hash_restarts', 'n/a')}\")
print(f\"Scripts: {s['num_cached_scripts']}  Hit rate: {s['opcache_hit_rate']:.1f}%\")"

Count PHP files in the codebase. This determines whether max_accelerated_files is adequate:

# Count all PHP files including vendor dependencies
find /path/to/app -name '*.php' | wc -l

Check effective max_accelerated_files. PHP selects the first value from a fixed prime set that is greater than or equal to the configured value. Configuring 10000 yields an effective 16229 slots:

# Show configured value from CLI SAPI (effective value is the next prime >= this).
# FPM may use a different php.ini; check /etc/php/*/fpm/php.ini specifically.
php -r 'echo ini_get("opcache.max_accelerated_files") . "\n";'

The prime set is: {223, 463, 983, 1979, 3907, 7963, 16229, 32531, 65407, 130987, 262237, 524521, 1048793}. Valid range is 200 to 1000000.

Check interned strings buffer. This allocation comes out of the same memory_consumption budget:

# CLI values may differ from FPM; cross-check with the FPM php.ini
php -r 'echo "interned_strings_buffer: " . ini_get("opcache.interned_strings_buffer") . "MB\n";'
php -r 'echo "memory_consumption: " . ini_get("opcache.memory_consumption") . "MB\n";'

For example, interned_strings_buffer=64 with memory_consumption=128 leaves only 64 MB for bytecode.

Verify settings location. OPcache memory directives set in PHP-FPM pool config (php_admin_value[opcache.memory_consumption]) do not take effect. The shared memory segment is allocated before pool config is applied. These directives must be in php.ini:

# Search all PHP config locations, not just the CLI default
grep -r 'opcache.memory_consumption' /etc/php/

How to diagnose it

1. Distinguish the visible cliff from the silent trap. Poll opcache_get_status() twice, 60 seconds apart. If oom_restarts incremented, you are hitting the visible cliff. If oom_restarts is zero but free_memory is near zero and hit rate is below 95%, you are in the silent no-restart trap.

2. Measure warm steady-state memory. After a restart and 10 to 15 minutes of normal traffic, record used_memory. This is your codebase’s actual OPcache footprint. If used exceeds 70% of memory_consumption, the segment is undersized.

3. Check num_cached_scripts against the effective file limit. If num_cached_scripts has plateaued at the effective max_accelerated_files, the hash table is full. New scripts are not cached regardless of free memory. Increase max_accelerated_files.

4. Assess wasted_memory. If wasted_memory exceeds 5% of total (the default max_wasted_percentage), the segment is fragmented. This is expected after deploys. If it grows without deploys, check whether opcache.validate_timestamps = 1 is causing frequent file invalidations in production.

5. Review interned strings allocation. If interned_strings_buffer is a large fraction of memory_consumption, bytecode has less room than you think.

Metrics and signals to monitor

Fixes

Size opcache.memory_consumption for the codebase

Measure used_memory after full warmup under production traffic. Set memory_consumption to at least 1.3x that value, giving 30% headroom. Account for growth: each deploy that adds files increases the footprint.

Large frameworks with extensive vendor trees can easily exceed the 128 MB default. Measure rather than guess. Remember that interned_strings_buffer comes out of this budget. If you increase interned_strings_buffer, increase memory_consumption by the same amount.

This directive must be set in php.ini, not in pool config. Restart PHP-FPM for the change to take effect.

Size opcache.max_accelerated_files

Count PHP files including vendor dependencies:

find /path/to/app -name '*.php' | wc -l

Set max_accelerated_files to at least 1.5x that count. The margin covers files added between sizing checks and dynamically generated PHP (template caches, container dumps). Files loaded through different symlink paths resolve to different real paths and get separate cache entries, consuming hash table slots faster than expected. The default of 10000 (effective 16229 after prime rounding) is often too low for applications with large vendor trees.

Clear the cache on deploy

After deploying new code, old bytecode occupies wasted_memory until a full restart clears it. Either call opcache_reset() from a web request during deploy, or reload PHP-FPM with kill -USR2 <master_pid>. Without this, wasted memory accumulates with each deploy until it triggers the restart threshold or fills the segment.

Address the silent no-restart trap

If cache_full is true but oom_restarts stays at zero, the segment is full but wasted memory is below max_wasted_percentage. Scripts already in the cache are still served from cache. Only uncached scripts get recompiled on every request.

Two options:

• Increase opcache.memory_consumption so all scripts fit. This is the preferred fix.
• Lower opcache.max_wasted_percentage so a restart fires sooner when the cache fills, clearing it and allowing a fresh warmup. The trade-off: a restart clears everything (brief spike, then improvement), while the silent trap only affects uncached scripts (sustained partial degradation). If most scripts are cached and only a small set is uncached, the silent trap may cause less total recompilation than frequent full restarts. Evaluate based on your miss rate.

Set validate_timestamps to 0 in production

With opcache.validate_timestamps = 1 (the default), PHP checks file modification times on every request, subject to opcache.revalidate_freq. In production where files do not change between deploys, this adds unnecessary stat() calls and contributes to wasted memory growth if files are touched outside of deploys. Set validate_timestamps = 0 and explicitly clear the cache on deploy.

Prevention

• Measure, do not guess. After warmup, record used_memory and num_cached_scripts. Size both memory_consumption and max_accelerated_files from measured data with 30% headroom.
• Count files including vendor. The vendor directory often contains more PHP files than the application itself.
• Clear on every deploy. Build opcache_reset() or a PHP-FPM reload into the deploy pipeline.
• Verify settings location. Memory directives in pool config silently fail. Confirm they are in php.ini.
• Monitor the delta of misses, not just hit rate. Hit rate is cumulative since pool start and masks current problems on long-running pools. Track misses per second.

How Netdata helps

• Per-second OPcache metrics. Netdata collects used_memory, free_memory, wasted_memory, oom_restarts, and hit rate at 1-second resolution. The cliff-edge nature of OPcache exhaustion means slower polling can miss the transition entirely.
• Correlation with worker CPU. When OPcache force-clears, CPU spikes across all workers simultaneously. Seeing OPcache memory exhaustion and per-worker CPU in the same view confirms the recompile cliff without guesswork.
• Anomaly detection on hit rate and memory. Anomaly flags highlight the gradual decline in hit rate that precedes the cliff, and the sudden drop when a restart fires.
• Alerts on oom_restarts increments. Any increment of oom_restarts during normal traffic is actionable. Configure alerts on the rate of change of this counter.
• Wasted memory tracking. Fragmentation growth after deploys is visible as a trend, making it easier to correlate performance degradation with a recent deploy.

Related guides

• PHP-FPM 504 Gateway Timeout: requests accepted but never finishing in time
• PHP-FPM active processes near max_children: reading pool utilization
• PHP-FPM in containers: cgroup limits and the silent OOM kill
• PHP-FPM “child N exited on signal 11 (SIGSEGV)”: worker segfaults
• PHP-FPM crash loop and fork storm: workers dying faster than they serve
• PHP-FPM dynamic mode scaling lag: why the pool cannot keep up with bursts
• PHP-FPM emergency restart: “failed processes threshold reached, initiating reload”
• PHP-FPM graceful reload: the brief no-worker window on SIGUSR2
• How PHP-FPM actually works in production: a mental model for operators
• PHP-FPM idle processes at zero: no burst headroom left
• PHP-FPM listen queue growing: the earliest signal of saturation
• PHP-FPM “server reached pm.max_children setting (N), consider raising it”