$ guides / uwsgi / uwsgi-gevent-async-monitoring ▌

Operations Guides

uWSGI in gevent/async mode: why worker busy ratio stops meaning anything

Switching uWSGI from pre-fork to gevent mode lets a single worker multiplex dozens or hundreds of concurrent requests on an event loop. The tradeoff: the worker busy ratio, the primary capacity metric in pre-fork mode, becomes unreliable. A worker reports “busy” whenever its event loop is running, which is nearly always, regardless of actual request load.

Teams that keep alerting on busy ratio after switching to gevent either see perpetual 100% utilization alarms or, worse, silence while real problems go undetected. The cheaper_busyness algorithm has a known incompatibility with gevent that causes workers to spawn under load but never scale back down.

What changes when you enable gevent mode

The --gevent N option enables the gevent loop engine with N async cores per worker. Each async core manages a single in-flight request. With --gevent 100, one worker process handles up to 100 concurrent requests by switching between greenlets on the event loop.

This is fundamentally different from pre-fork mode, where one worker handles one request at a time:

Mode	What “busy” means	Idle state	Useful capacity signal
Pre-fork	Processing one request synchronously	Worker waiting for accept()	Busy ratio reflects true utilization
Threaded	At least one thread active	All threads idle	Need per-thread visibility
Gevent	Event loop is running	Almost never idle	Busy ratio is near-useless

In gevent mode, the worker’s event loop is always running. It accepts greenlet switches, manages I/O, and reports as “busy” nearly 100% of the time regardless of actual request load. The busy ratio cannot tell you whether the worker is serving 1 request or 100.

How the event loop changes the meaning of worker status

In pre-fork mode, the worker state machine is binary: idle when waiting for accept(), busy when processing a request. The ratio of busy workers to total workers directly reflects capacity utilization.

In gevent mode, the worker process hosts an event loop that manages N async cores. The worker-level status field reports “busy” whenever the loop is active, which is the normal operating state. There is no meaningful “idle” state for a gevent worker.

flowchart TD
    subgraph sync_mode ["Pre-fork mode"]
        direction LR
        S1["Worker: idle"] -->|"accept()"| S2["Worker: busy"]
        S2 -->|"response sent"| S1
    end
    subgraph gevent_mode ["Gevent mode: --gevent N"]
        direction LR
        G1["Worker: loop running"] -->|"accept()"| G2["Worker: loop running, N greenlets"]
        G2 -->|"greenlet completes"| G2
        G2 -->|"all greenlets idle"| G1
    end

The true concurrency indicator is per-core in_request. The stats server exposes a cores[] array per worker, and each core has an in_request field set to 0 or 1. Counting cores where in_request == 1 across all workers gives you the actual number of requests being processed simultaneously.

# Count active requests across all async cores
# Requires: --stats 127.0.0.1:9191 on the uWSGI process
uwsgi --connect-and-read 127.0.0.1:9191 | jq '[.workers[] | select(.pid > 0) | .cores[] | select(.in_request == 1)] | length'

Compare this count against your configured async core total (workers x N) to understand real utilization. This replaces busy ratio as your primary capacity signal.

Other metrics that change meaning

Several standard uWSGI metrics shift in interpretation under gevent mode.

Request throughput: A single worker’s requests counter includes all greenlets it has served. Per-worker request counts can be very high because one worker multiplexes many concurrent requests. Do not compare per-worker throughput between pre-fork and gevent deployments without accounting for the concurrency multiplier.

Average response time (avg_rt): avg_rt always reflects wall-clock time, including I/O wait, not CPU time. In gevent mode this matters more: I/O-bound workloads show high avg_rt because requests spend real time waiting on backends. avg_rt is an exponential moving average computed as (old + new) / 2, giving roughly 50% weight to the most recent request. A single slow request shifts it significantly. See uWSGI avg_rt is not a real average for details on this formula.

Worker running_time: Accumulates wall-clock time spent in request handling. In gevent mode, running_time includes time waiting on I/O because event loop idle time between greenlet switches is difficult to separate from active processing. The derived metric running_time / requests gives you the wall-clock average per request, which includes I/O wait.

The cheaper_busyness trap

The cheaper_busyness plugin has a known incompatibility with gevent async mode. The busyness algorithm uses a global per-worker value that reports the worker as busy whenever the event loop is running, regardless of actual request activity. It does not account for async cores.

The consequence: with cheaper-algo = busyness and gevent enabled, workers spawn under load but never get collected, even after hours of zero requests. This is tracked in uWSGI GitHub issue #880 (reported against 2.0.x) and remains open.

If you run gevent mode, avoid cheaper-algo = busyness. The alternatives are:

cheaper-algo = spare (default): Scales based on spare worker count. Does not depend on the busyness calculation.
cheaper-algo = backlog: Scales based on the Linux TCP listen queue depth (Linux only). Note that the backlog algorithm reads the same kernel-measured backlog value that feeds the listen_queue stats field, so its signal is only as reliable as that measurement.
Fixed worker count: Skip the cheaper subsystem entirely. Handle horizontal scaling at the deployment layer (more containers, more instances).

Blocking calls: the real failure mode in gevent

The most dangerous failure mode in gevent mode is a blocking syscall that gevent has not patched. If any code path makes a blocking call (a C extension that blocks, a database driver without gevent support, a file operation that bypasses the patched stdlib), the entire worker’s event loop stalls.

When the event loop stalls, every greenlet in that worker freezes. All in-flight requests on that worker hang simultaneously. The worker still reports “busy” because the loop is technically running, just blocked on a syscall. No per-core signal updates because the loop is not switching between greenlets. From the outside, the worker looks active but serves no requests.

This is why monkey-patching matters. The --gevent-monkey-patch option calls gevent.monkey.patch_all() at worker loop startup (the option exists in the 1.9 series; verified in 1.9.21). With the default eager app loading, module-level imports run before the worker loop starts, so blocking I/O during import can execute before the patch. Use --gevent-early-monkey-patch (backported in 2.0.14) to patch before the application is loaded, or call gevent.monkey.patch_all() at the very top of your application entry point, before any other imports that might trigger blocking I/O:

# Must be first, before any other imports
from gevent import monkey
monkey.patch_all()

# Now safe to import application modules
import your_app

Constraints when running gevent

--gevent 1: creates a single async core per worker (one in-flight request at a time on an event loop). No minimum is enforced in the uWSGI source, so the instance starts normally; it just has no concurrency.
--threads is incompatible with gevent. Monkey-patched threads inside your application code work (they become greenlets), but uWSGI’s --threads option cannot be combined with --gevent.
Do not mix uWSGI’s Async API with gevent. Calls like uwsgi.wait_fd_read() and uwsgi.suspend() are not compatible with gevent primitives.

Signals to watch in gevent mode

Signal	Why it matters	Warning sign
Per-core `in_request` count	True concurrency indicator. Replaces busy ratio.	Count approaching workers x N
Per-core `req_info.request_start` age	Detects stuck requests on specific cores. A greenlet blocked on an unpatched call will show growing age.	Request age approaching harakiri timeout
Request throughput (delta of `requests`)	Throughput drop with stable incoming traffic indicates greenlets stalling on I/O or blocking calls.	Sudden drop below baseline
Harakiri count (delta)	If harakiri fires, a greenlet exceeded the timeout, likely blocked on an unpatched call.	Any sustained non-zero rate
Worker RSS	Memory pressure triggers GC pauses that stall all greenlets in the worker.	Steady growth toward `reload-on-rss` threshold
avg_rt trend	Wall-clock response time including I/O wait. Volatile due to EMA formula.	Sustained increase above 2x baseline
Exception count (delta)	Unhandled exceptions in any greenlet increment this counter.	Sustained non-zero rate

Checking per-core request ages

# Find in-flight requests older than 10 seconds
uwsgi --connect-and-read 127.0.0.1:9191 | jq --argjson now "$(date +%s)" '
  [.workers[] | select(.pid > 0) | .id as $wid | .cores[] | select(.in_request == 1) |
   {worker: $wid, core: .id, age_seconds: ($now - .req_info.request_start)}] |
  map(select(.age_seconds > 10))'

In current 2.0.x releases the field is cores[].req_info.request_start, a UNIX timestamp in seconds that is present only while in_request == 1 (and suppressed by --stats-no-cores).

The cores[] array is suppressed if uWSGI is started with --stats-no-cores. Ensure that flag is not set if you depend on per-core visibility.

Using uwsgitop for async core inspection

uwsgitop can display async core statistics. Press a while uwsgitop is running to cycle the core-level statistics display (documented in the uwsgitop README); f toggles fast refresh and q quits.

How Netdata helps

When busy ratio is no longer meaningful, Netdata surfaces the signals that are:

Per-core in_request: Reading the cores[] array from the stats server gives you true concurrency count at per-second collection rate, catching brief stalls that coarser polling misses.
Throughput anomaly detection: Per-second request throughput deltas distinguish between a traffic decrease (normal) and a greenlet stall (throughput drops while incoming traffic stays constant). Correlating throughput with harakiri count isolates blocked greenlets from reduced demand.
ML anomaly detection on utilization patterns: Because busy ratio is perpetually near 100% in gevent mode, static threshold alerts are useless. Netdata learns the normal pattern of per-core utilization and flags deviations that threshold-based monitoring cannot distinguish.
Memory and GC correlation: Worker RSS growth and system-level memory pressure can stall the event loop through GC pauses. Correlating memory metrics with per-core request latency identifies when memory pressure degrades greenlet throughput.