You see respawn_count climbing across your uWSGI workers. The question is whether workers are crashing or whether uWSGI is doing what you configured it to do.

respawn_count is a single monotonic counter per worker slot that increments for every reason a worker can die and come back: max-requests recycling, reload-on-rss memory recycling, harakiri timeout kills, segfaults, OOM kills, and manual kill -9. The raw number tells you workers are churning. It does not tell you why.

Without knowing your max-requests configuration and subtracting the harakiri delta, the respawn signal is uninterpretable noise. This article walks through the arithmetic and the timing patterns that separate healthy recycling from real failures.

What this means

respawn_count lives on the worker slot struct, not on the process. It persists through respawns and is never reset. The stats server exposes it as workers[].respawn_count (per-worker, monotonic). There is no separate counter for crashes versus intentional recycling. You must infer the cause by combining three signals:

  1. Your max-requests configuration. If recycling is configured, some respawns are expected. If it is not, every respawn is abnormal.
  2. The harakiri_count delta. Every harakiri kill increments respawn_count. Subtract the harakiri rate to see what remains.
  3. The timing pattern. Staggered, periodic respawns at a predictable cadence indicate healthy recycling. A simultaneous spike across all workers indicates a mass-kill event: a deploy, an OOM-killer sweep, or all workers hitting max-requests at once without a delta offset.

The diagnostic arithmetic is straightforward. Compute the expected recycling rate from your traffic volume and max-requests setting, then subtract the harakiri delta from the measured respawn rate. What remains above the expected recycling baseline is crash-driven or reload-on-rss-driven churn.

Common causes

CauseWhat it looks likeFirst thing to check
max-requests recyclingSteady, periodic respawns at predictable cadence. harakiri_count flat. Workers exit gracefully (finish current request, then exit).Compute expected rate: total_rps / max_requests. Compare to measured respawn delta.
reload-on-rss recyclingPeriodic respawns correlated with RSS crossing the threshold. Sawtooth RSS pattern per worker. harakiri_count flat.Per-worker RSS trend versus configured reload-on-rss value.
Harakiri killsRespawn delta tracks harakiri_count delta closely (approximately 1:1). Workers die mid-request. avg_rt approaching harakiri timeout.harakiri_count delta. Downstream dependency health. See uWSGI harakiri death spiral.
Application crash (segfault, SIGABRT)Respawn rate exceeds expected recycling after subtracting harakiri delta. uWSGI log shows DAMN ! worker N died, killed by signal 11 (or 6).uWSGI log for the signal number. Application logs for tracebacks.
OOM killWorkers killed by kernel. No graceful exit preceding respawn. dmesg shows oom-killer entries.dmesg -T | grep -iE "out of memory|killed process" or journalctl -k | grep -iE "out of memory|killed process". Compare per-worker RSS to available memory.
Mass-kill (deploy, OOM sweep)All workers respawn simultaneously. Sudden spike across every worker slot at the same timestamp.Deployment logs, process manager events, dmesg for kernel-level kills.

Quick checks

# Total respawn count and harakiri count across all workers
uwsgi --connect-and-read 127.0.0.1:9191 | jq '{total_respawns: ([.workers[].respawn_count] | add), total_harakiri: ([.workers[].harakiri_count] | add)}'

# Per-worker breakdown: respawn_count, harakiri_count, requests, RSS
uwsgi --connect-and-read 127.0.0.1:9191 | jq '.workers[] | select(.pid > 0) | {id: .id, respawns: .respawn_count, harakiri: .harakiri_count, requests: .requests, rss_mb: (.rss / 1048576)}'

# Total request count (for computing expected recycling rate)
uwsgi --connect-and-read 127.0.0.1:9191 | jq '[.workers[].requests] | add'

# Check for OOM kills in kernel log
dmesg -T | grep -iE "out of memory|killed process" | tail -20

# Check uWSGI log for crash patterns (violent deaths vs graceful exits)
grep -E "DAMN ! worker|killed by signal|NO MERCY" /var/log/uwsgi/*.log | tail -20

Adjust the stats socket address (127.0.0.1:9191) to match your deployment. If your stats server uses a UNIX socket, substitute uwsgi --connect-and-read /path/to/stats.sock. If you enabled --stats-http, curl http://127.0.0.1:9191 also works.

How to diagnose it

Step 1: Establish the expected recycling rate

If max-requests is configured, compute the expected recycling rate:

expected_total_recycling_rate = total_requests_per_second / max_requests

For example, with 100 req/s fleet-wide and max-requests = 1000, expect approximately 0.1 respawns per second (one recycling event every 10 seconds). This assumes traffic is evenly distributed across workers.

Step 2: Measure the actual respawn delta

Poll respawn_count twice with a known interval (60 seconds is a good starting point) and compute the delta. Compare the measured rate to the expected recycling rate from Step 1.

Step 3: Subtract the harakiri delta

Pull harakiri_count over the same interval. Every harakiri kill increments respawn_count, so subtract the harakiri delta from the respawn delta:

non_harakiri_respawns = respawn_delta - harakiri_delta

Compare non_harakiri_respawns to the expected recycling rate from Step 1. If they are close, your respawns are healthy max-requests or reload-on-rss recycling. If non_harakiri_respawns significantly exceeds the expected recycling rate, you have crash-driven respawns.

Step 4: Classify the remainder

The following decision tree summarizes the classification logic:

flowchart TD
    A["respawn_count delta rising"] --> B{"max-requests configured?"}
    B -- No --> C["All respawns are abnormal.
Investigate crashes and OOM."]
    B -- Yes --> D["Compute expected rate:
total_rps / max_requests"]
    D --> E{"Measured rate close to expected
AND harakiri delta is zero?"}
    E -- Yes --> F["Healthy max-requests recycling.
No action needed."]
    E -- No --> G["Subtract harakiri delta
from respawn delta"]
    G --> H{"Remaining respawns > 0?"}
    H -- No --> I["Harakiri-driven respawns.
Check downstream dependencies."]
    H -- Yes --> J["Crash or mass-kill event.
Check logs and dmesg."]
    J --> K{"Simultaneous across
all worker slots?"}
    K -- Yes --> L["Mass-kill: deploy, OOM sweep,
or max-requests without delta."]
    K -- No --> M["Individual crashes.
Check signal number in uWSGI log."]

Step 5: Correlate with logs and system signals

For crash-driven respawns, the uWSGI log distinguishes violent from graceful deaths:

  • Graceful exit (max-requests, reload-on-rss): worker finishes its current request, exits cleanly, master forks a replacement. Log shows Respawned uWSGI worker N (new pid: XXXX) without a preceding death message.
  • Harakiri kill: worker exceeds the timeout, master sends SIGKILL. Log shows HARAKIRI ON WORKER N (and a traceback if harakiri-verbose is enabled). See uWSGI harakiri-verbose diagnosis.
  • Crash (segfault, abort): log shows DAMN ! worker N died, killed by signal 11 (SIGSEGV) or signal 6 (SIGABRT). Always investigate these.
  • Reload mercy timeout: worker took too long to die during a graceful reload (exceeding worker-reload-mercy). Log shows NO MERCY !!! as uWSGI escalates to SIGKILL.

For OOM-driven respawns, check dmesg or journalctl -k for kernel oom-killer entries targeting worker PIDs. The OOM-killer targets the highest-RSS process first, so with many uWSGI workers it picks one off at a time, creating a slow churn that looks like random respawns without any application-level signal.

Metrics and signals to monitor

SignalWhy it mattersWarning sign
respawn_count delta (summed across workers)Primary churn indicator. Must be interpreted against expected recycling rate, not in isolation.Rate significantly exceeds total_rps / max_requests.
harakiri_count delta (summed across workers)Every harakiri is also a respawn. Subtract this from respawn delta to isolate non-harakiri causes.Non-zero delta in a normally-zero-harakiri deployment.
Per-worker requests (rate of change)Needed to compute expected recycling rate. Also reveals stuck workers (frozen request count while others progress).One worker with zero request growth while others are active.
Per-worker RSSDetects reload-on-rss triggers and memory leaks driving recycling.Steady linear growth across all workers (memory leak sawtooth).
Application exception rateCrashes are sometimes preceded by a rising exception rate. Correlating exceptions with respawns narrows the code path.Sustained increase from baseline.
dmesg OOM eventsKernel-level kills are invisible in uWSGI stats. The only evidence is in the kernel log.Any Out of memory or Killed process entry targeting a worker PID.
last_spawn timestampsShows whether respawns are staggered (healthy) or simultaneous (mass-kill).All workers with nearly identical spawn timestamps.

Fixes

Respawn rate is at the expected recycling rate

No fix needed. This is max-requests or reload-on-rss working as designed. If the recycling cadence is too aggressive (capacity dips during peak traffic), raise max-requests or increase the reload-on-rss threshold. Each respawn briefly reduces capacity by one worker, so high recycling rates under heavy load can cause intermittent latency spikes.

Respawn rate matches harakiri delta 1:1

Workers are being killed by the timeout watchdog, not crashing. The root cause is downstream: requests are hanging on a database, external API, or lock. Do not disable harakiri to silence the respawns. Investigate the blocked syscall with harakiri-verbose and fix the downstream dependency. See uWSGI harakiri death spiral and uWSGI HARAKIRI ON WORKER.

Non-harakiri respawns exceed expected recycling rate

Workers are crashing. Check the signal number in the uWSGI log. Signal 11 (SIGSEGV) or signal 6 (SIGABRT) in production points to a C extension bug, a memory corruption issue, or a deserialization crash. Common culprits: numpy operations on malformed arrays, lxml parsing of adversarial XML, database driver segfaults on connection pool exhaustion, and pickle/yaml deserialization of malicious payloads.

If crashes started after a deployment, roll back and test in staging with the same input that triggered the crash.

All workers respawn simultaneously

This is a mass-kill event. Three common causes:

  1. Deploy or reload without chain reload. A graceful SIGHUP kills all workers at once. Use --chain-reload to cycle workers one at a time and maintain capacity during deploys.
  2. OOM-killer sweep. Total worker RSS exceeded available memory. The kernel kills multiple processes in rapid succession. Lower max-requests or reload-on-rss to recycle workers before they collectively exhaust memory. Check that total worker RSS stays below 70% of system RAM.
  3. All workers hit max-requests simultaneously. Without max-requests-delta, workers that start together and receive balanced load all hit the limit at the same time, causing a brief total outage during respawn. This is a known problem documented in uWSGI issue #648.

Staggering max-requests recycling

max-requests-delta adds (worker_id * delta) to each worker’s max-requests value, staggering restarts so they do not all recycle at once. For example, with max-requests = 1000 and max-requests-delta = 100, worker 1 recycles at 1100 requests, worker 2 at 1200, and so on.

`max-requests-delta` may not be recognized when `strict = true` is set in uWSGI 2.0.20 and possibly other 2.0.x versions. The option exists in the source parser but is missing from the strict-mode whitelist, producing an "unknown config directive" error. If you use strict mode, verify whether your uWSGI build accepts this directive before relying on it. The `min-worker-lifetime` directive (default 60 seconds in the source code, though some distribution builds show a default of 10) prevents workers from being recycled before they have run for a minimum period. Verify the default on your installed version with `uwsgi --help | grep min-worker-lifetime`.

Prevention

  • Correlate respawns with harakiri count. If you only alert on raw respawn rate, you will page on healthy recycling and miss crash loops that happen to produce a similar count. The subtraction (respawn_delta - harakiri_delta) is the minimum viable correlation.
  • Compute and document the expected recycling rate for each service based on its max-requests setting and typical traffic. Store it alongside the alert threshold so on-call engineers can distinguish expected from unexpected without doing arithmetic at 3 a.m.
  • Stagger recycling with max-requests-delta to prevent simultaneous mass-recycle events. Verify compatibility with your uWSGI version and strict-mode configuration first.
  • Monitor dmesg for OOM events. The kernel oom-killer is invisible in uWSGI stats. A slow OOM-kill cycle produces mysterious respawns with no application-level signal.
  • Use --chain-reload for deployments to avoid mass-kill events from simultaneous worker replacement.

How Netdata helps

  • Per-second respawn_count collection makes the exact timing of respawn events visible. Simultaneous spikes across all workers (mass-kill) are immediately distinguishable from staggered periodic patterns (healthy recycling).
  • Correlating respawn_count with harakiri_count in the same dashboard makes the subtraction visual. If the two lines track each other, respawns are harakiri-driven. If respawn_count rises independently of harakiri_count, workers are crashing or hitting reload-on-rss.
  • Per-worker RSS charts show the sawtooth pattern of memory-triggered recycling and reveal whether reload-on-rss is the driver.
  • Request throughput alongside respawn rate lets you compute the expected recycling rate in real time and spot when actual respawns deviate from it.
  • Anomaly detection on respawn_count can flag unusual churn patterns even when the absolute rate looks normal.
  • Kernel memory metrics from the same agent let you correlate uWSGI-level respawns with system-level memory pressure and OOM conditions without switching to dmesg.