Two uWSGI options recycle workers when their resident set size crosses a threshold: reload-on-rss and evil-reload-on-rss. Both produce the same effect on a monitoring dashboard. The respawn_count counter increments, RSS drops to baseline, a new worker is forked. The memory sawtooth looks healthy. From the master process perspective, the two are indistinguishable.

The difference is in what happens to the in-flight request when the kill occurs.

reload-on-rss is graceful. The master signals the worker to shut down. The worker finishes its current request, delivers the full response, and exits. The replacement worker picks up the next connection from the listen queue. No client sees anything unusual.

evil-reload-on-rss is immediate. The master sends SIGKILL to the worker, even if it is mid-response. The client’s connection breaks. The response is truncated or never sent. The write_errors counter increments. But the respawn_count ticks up exactly as it would for a graceful recycle, and the RSS drops exactly as it would for a graceful recycle.

This is the trap: operators see memory management working while clients see intermittent 502s, broken responses, or connection resets. The write_errors counter in the stats server is the signal that separates the two cases.

What it is and why it matters

Both options accept a single integer value in megabytes. The threshold is per-worker, not a global pool limit. With processes = 10 and reload-on-rss = 512, each worker is independently recycled when its own RSS exceeds 512 MB. One worker hitting 513 MB triggers a respawn of that worker only; the other nine continue serving requests.

The total memory ceiling is approximately reload-on-rss * processes. Ten workers at a 512 MB threshold means the system can hold up to 5 GB of worker RSS before recycling kicks in, assuming all workers grow at the same rate. If they do not (one worker leaks faster), that worker recycles more frequently while the others remain stable.

The companion options reload-on-as and evil-reload-on-as work identically but check virtual set size (address space) instead of RSS. RSS is the more useful signal in practice because it reflects physical memory pressure, which is what leads to swapping and OOM kills. VSZ growth without RSS growth is typically benign.

Both options remain in the current uWSGI 2.0.x stable line. Neither has been deprecated.

How it works

The master process periodically checks each worker’s RSS. When a worker crosses the configured threshold, both paths fork a replacement and increment that worker slot’s respawn_count. The divergence is in the kill itself.

flowchart TD
    R[Worker RSS exceeds threshold] --> GO{Which option?}
    GO -->|reload-on-rss| G1[Master signals graceful exit]
    G1 --> G2[Worker finishes current request]
    G2 --> G3[Client gets full response]
    G3 --> G4[Worker exits, respawn_count +1]
    GO -->|evil-reload-on-rss| E1[Master sends SIGKILL]
    E1 --> E2[Worker dies mid-request]
    E2 --> E3[write_errors +1, client gets broken response]
    E3 --> E4[Worker respawns, respawn_count +1]
    G1 -.->|request exceeds worker-reload-mercy| E1

The dashed line captures a critical edge case. Even reload-on-rss can degrade to a mid-request kill if the current request takes longer than worker-reload-mercy (default 60 seconds). The master signals a graceful exit, starts the mercy timer, and if the worker has not exited when the timer expires, force-kills it. A worker processing a slow upload or a long-running API call at the moment RSS crosses the threshold can still produce a broken response under reload-on-rss.

The mem_collector thread

Before the mem_collector thread existed, evil reload checks ran synchronously in the master’s main loop. The master could miss runaway workers if it was busy with other lifecycle work, and operators reported workers consuming unlimited memory despite having evil-reload-on-rss configured.

The evil memory monitors (evil-reload-on-rss and evil-reload-on-as) are now managed by a dedicated mem_collector thread that asynchronously checks RSS. The companion option mem-collector-freq controls the polling frequency when evil reloads are enabled. Verify your uWSGI version before relying on evil reload timing.

Where it shows up in production

The most common scenario is a slow memory leak in a Python application. Workers grow steadily as the allocator fragments memory or global caches accumulate objects. Without any recycling configured, RSS climbs until the Linux OOM killer terminates a worker, which may target the master process instead of a worker.

Teams add reload-on-rss as a defense. Workers recycle gracefully before hitting the OOM danger zone. The RSS chart shows a clean sawtooth: linear growth, sharp drop at recycle, repeat. Response times stay flat because the graceful exit does not interrupt clients. This is the healthy pattern.

The problem appears when someone also adds evil-reload-on-rss as a “safety net,” or when reload-on-rss is set so close to the OOM limit that workers do not finish their requests before memory pressure becomes critical. In both cases, the dashboard looks identical to the healthy pattern. The respawn_count goes up. The RSS drops. Memory management appears to be working.

But clients are seeing broken responses. To detect this, correlate two signals:

# Check respawn rate (delta over polling intervals)
uwsgi --connect-and-read 127.0.0.1:9191 | jq '[.workers[].respawn_count] | add'

# Check write_errors (sum across all workers and cores)
uwsgi --connect-and-read 127.0.0.1:9191 | jq '[.workers[].cores[].write_errors] | add'

If write_errors spikes in lockstep with respawn_count deltas, workers are being killed mid-response. If write_errors stays flat while respawn_count increments, the recycles are graceful and clients are unaffected.

Tradeoffs and when to use it

reload-on-rss: the primary defense

Use reload-on-rss as your standard memory recycling mechanism. It is predictable, does not interrupt clients, and integrates cleanly with the worker lifecycle. Pair it with max-requests for a layered defense: max-requests recycles workers after N requests regardless of RSS (catching leaks that do not manifest as RSS growth), while reload-on-rss catches memory-specific growth.

evil-reload-on-rss: the emergency backstop

The case for evil-reload-on-rss is narrow. It exists for situations where a worker’s RSS is growing so fast that waiting for request completion would risk OOM-killing the entire system. If a single worker leaks 500 MB in seconds, the graceful path may not be fast enough.

The case against it is stronger. The SIGKILL is indiscriminate: it hits regardless of where the worker is in the request lifecycle. The client has no way to distinguish a mid-response kill from a network failure or a server crash. If your reverse proxy retries the request, the retry may land on the same recycling worker pool and fail again. At scale, this creates a thundering herd of retries on top of the memory pressure that triggered the kills.

Most operators avoid evil-reload-on-rss entirely and let the Linux OOM killer handle the catastrophic case. The OOM killer is at least system-aware: it considers total memory pressure, not just a single worker’s RSS, and it logs its actions to the kernel ring buffer where you can audit them.

Complementary options

OptionWhat it doesHow it complements reload-on-rss
max-requestsWorker self-exits after N requests (graceful)Catches leaks that do not manifest as RSS growth. Predictable cadence.
cheaper-rss-limit-softPrevents the cheaper subsystem from spawning new workers when total RSS exceeds a thresholdStops the problem from getting worse by adding more memory-hungry workers. Does not recycle existing workers.
cheaper-rss-limit-hardHard RSS limit for the cheaper subsystemTriggers worker reduction when total RSS exceeds this limit. Stricter than soft.
worker-reload-mercyCaps graceful shutdown time before force-kill (default 60s)If set too low, even reload-on-rss can produce mid-request kills.
limit-asCaps virtual address spaceDifferent mechanism: the worker hits MemoryError on allocation failure rather than being recycled. May leave the worker unable to handle requests. Not a substitute for reload-on-rss.

Setting the threshold

The threshold needs to sit below the OOM danger zone with enough margin that the graceful exit can complete before system memory is exhausted.

  1. Calculate the per-worker memory budget. Total system memory minus overhead for the OS, page cache, and non-uWSGI processes, divided by worker count. Total worker RSS (adjusted for copy-on-write sharing) should stay below 70% of available system memory.

  2. Set reload-on-rss below that per-worker budget. The gap between the threshold and the OOM limit needs to accommodate the time for graceful exit, which is bounded by worker-reload-mercy.

  3. Monitor the RSS growth rate. If workers grow at R MB per hour and the threshold is T MB above baseline, each worker has approximately T/R hours before recycling. If this interval is too short (workers recycling every few minutes), the leak needs fixing, not a higher threshold.

  4. Account for copy-on-write. Linux COW means rss over-reports per-worker usage because shared pages (shared libraries, pre-fork application code) are counted fully for each process. The sum of worker RSS values is higher than actual memory consumption. Use /proc/<pid>/smaps_rollup for PSS if you need precise per-worker accounting on 2.0.x.

A common mistake is setting the threshold too close to normal peak RSS. If workers normally peak at 450 MB and the threshold is 512 MB, a small traffic spike that pushes RSS to 513 MB triggers a recycle that would not have happened with the threshold at 640 MB. The threshold should reflect the OOM danger zone, not the normal operating range.

Signals to watch in production

SignalWhy it mattersWarning sign
Worker RSS (workers[].rss)Shows which workers are approaching the thresholdAny worker within 10% of the configured limit is about to be recycled
Respawn rate (workers[].respawn_count delta)Indicates recycling frequency and causeRate exceeding what max-requests alone would produce signals memory-driven recycling
Write errors (workers[].cores[].write_errors delta)The key discriminator between graceful and mid-request killsSpikes that track respawn deltas 1:1 indicate evil-reload-on-rss or mercy timeout kills
Harakiri count (workers[].harakiri_count delta)Separates timeout kills from memory killsHarakiri increasing alongside respawns means the worker was stuck, not leaking
Delta requests (workers[].delta_requests)Resets on respawn, confirming the recycling happenedSudden reset across multiple workers indicates a mass recycling event

The most important correlation is between respawn rate and write errors. Under reload-on-rss with healthy request durations, respawns happen but write errors stay flat. Under evil-reload-on-rss, or when worker-reload-mercy expires during a graceful exit, write errors spike in lockstep with respawns. Build an alert that fires when write_error deltas are non-zero during respawn events.

Also subtract the harakiri rate from the respawn rate to isolate memory-driven recycling from timeout-driven recycling. Every harakiri kill increments respawn_count, so without this subtraction, a harakiri storm looks identical to aggressive memory recycling.

How Netdata helps

Netdata collects uWSGI stats server output at per-second resolution and surfaces the signals that distinguish graceful recycling from mid-request kills:

  • Per-worker RSS trended continuously, so the sawtooth pattern is visible and the growth rate between recycles is measurable.
  • Respawn rate computed as a delta, not a cumulative total, so individual recycling events appear as discrete events rather than a monotonically climbing counter.
  • Write errors per core, correlated in time with respawn events. A write_error spike that coincides with a respawn delta is the signature of a mid-request kill from evil-reload-on-rss or a worker-reload-mercy timeout.
  • Harakiri count tracked as a separate delta, so memory-driven kills are distinguishable from timeout-driven kills without manual subtraction.
  • ML-based anomaly detection on the write_errors rate, catching the early signs of mid-request kills before the error rate becomes user-visible at scale.