Apache BusyWorkers and IdleWorkers: reading worker utilization from mod_status

BusyWorkers and IdleWorkers are the two most-quoted numbers from Apache’s mod_status output, and the two most frequently misread. Together they are the primary saturation gauge for the server: how much of the worker pool is currently occupied. Read them wrong and you either miss the onset of worker exhaustion or page someone for healthy autoscaling churn.

This guide covers what the two counters actually count, how to turn them into a utilization ratio you can alert on, why Apache degrades at a cliff edge rather than gradually, and how to distinguish the two very different situations that both show IdleWorkers: 0.

It assumes Apache 2.4.x with mod_status enabled. The ?auto counters are always present in the machine-readable output; ExtendedStatus On adds per-request detail to the full HTML status page. Its base default is Off, but loading mod_status changes the default to On. Everything here applies across the prefork, worker, and event MPMs, with MPM-specific differences called out where they matter.

What the two counters actually count

Apache tracks every worker slot in a shared-memory structure called the scoreboard. Each slot is in one of a small set of states: waiting for a connection (_), starting up (S), reading a request (R), sending a reply (W), keepalive read (K), DNS lookup (D), closing (C), logging (L), gracefully finishing (G), idle cleanup (I), or open slot with no process (.). mod_status reads the scoreboard and summarizes it.

The two summary counters are:

• BusyWorkers: active request-processing slots—reading (R), writing (W), keepalive (K), DNS lookup (D), closing (C), and logging (L). Starting (S), gracefully finishing (G), idle-cleanup (I), and open/dead (.) slots are excluded.
• GracefulWorkers: slots in G, reported separately from BusyWorkers.
• IdleWorkers: ready slots in _ from the current MPM generation.

Two consequences follow.

First, “busy” is broader than “serving a request right now.” A worker blocked writing an access log line or stuck in a DNS lookup inflates BusyWorkers; draining workers appear in GracefulWorkers and scoreboard G, not BusyWorkers. When BusyWorkers climbs, the scoreboard state distribution is how you find out which kind of busy you have.

Second, G, I, and open slots (.) are in neither BusyWorkers nor IdleWorkers; BusyWorkers + IdleWorkers therefore does not include draining or cleanup workers. The spawned-worker count can be below the configured maximum because Apache scales the pool between its spare-worker thresholds.

Reading utilization from mod_status

The machine-readable endpoint is http://localhost/server-status?auto:

# Pull the worker counters
curl -s http://localhost/server-status?auto | grep -E "BusyWorkers|IdleWorkers"

Typical output:

BusyWorkers: 47
IdleWorkers: 18

Two ratios are worth computing, and they answer different questions.

Pool utilization: BusyWorkers / (BusyWorkers + IdleWorkers) tells you how much of the currently spawned pool is occupied. This is what moves second to second and what you watch for saturation onset.

# Compute pool utilization from mod_status
curl -s http://localhost/server-status?auto | \
  awk -F': ' '/^BusyWorkers:/{b=$2} /^IdleWorkers:/{i=$2} END {
    if (b+i > 0) printf "pool utilization: %.1f%% (%d busy, %d idle)\n", b/(b+i)*100, b, i
  }'

Utilization against the ceiling: BusyWorkers / MaxRequestWorkers tells you how close you are to the hard limit. This is the number that predicts queuing. The catch, covered below, is that mod_status does not expose MaxRequestWorkers, so you have to supply it from the MPM configuration yourself.

A single snapshot is a point in time. Both ratios fluctuate with traffic bursts and with Apache’s own child management. Trends and sustained values are what matter; a one-second spike to high utilization that recovers is normal burst absorption.

The cliff edge: why 100% busy means instant queuing

Worker capacity in Apache does not degrade gracefully. Up to full utilization, requests are served normally. The moment every worker is occupied, new connections go to the kernel’s TCP listen backlog. When the backlog fills (default ListenBacklog is 511), new connections are refused outright.

So the degradation curve is a cliff: normal service, then queuing, then connection refusal, with almost no middle ground. The practical implications:

• Latency dashboards look fine until they don’t. Queued connections show up as client-side connect timeouts, not as slow responses in your access log, because queued requests never reach a worker to be logged.
• The listen backlog is the only buffer. Watch Recv-Q on the listening sockets; it is the earliest visible symptom of saturation, appearing before users see failures:

# Check listen backlog depth on Apache's sockets
ss -ltn | grep -E ':80\s|:443\s'

• Alerting on “latency is up” will page you late. Alerting on worker utilization approaching the ceiling, corroborated by backlog depth, pages you early.

Reading IdleWorkers = 0

Zero idle workers is the most alarming-looking value in the output, and it has two completely different causes. Telling them apart requires exactly one piece of outside information: MaxRequestWorkers.

flowchart TD
    A[IdleWorkers = 0] --> B{BusyWorkers near MaxRequestWorkers?}
    B -->|Yes| C[True saturation]
    B -->|No| D[Ramp lag]
    C --> E[Check Recv-Q, MPM worker-limit code, 503s]
    D --> F[Check MinSpareServers or MinSpareThreads]
    E --> G[Page: raise capacity or find what holds workers]
    F --> H[Tune spare minimums, not a capacity problem]

True saturation: IdleWorkers = 0 with BusyWorkers at or near MaxRequestWorkers. Every slot is occupied and Apache cannot spawn more. New connections queue in the listen backlog. This is the cliff edge, and it is page-worthy when sustained. Corroborate with a non-zero Recv-Q, the MPM worker-limit code in the error log (for example AH00484), or 503 responses appearing.

Ramp lag: IdleWorkers = 0 with BusyWorkers well below MaxRequestWorkers. Apache still has room to grow but has not spawned workers fast enough to keep a spare ready. Arriving requests wait for a worker to be created. This is not a capacity problem; it means MinSpareServers (prefork) or MinSpareThreads (worker/event) is too low for your ramp rate. The fix is tuning the spare minimums, not raising the ceiling.

Both produce user-visible latency. Only the first is fixed by raising MaxRequestWorkers. If you raise the ceiling in response to ramp lag, you add memory pressure without fixing the spawn rate.

One more case to keep separate: IdleWorkers fluctuating rapidly between zero and small positive values during traffic bursts is healthy autoscaling. Apache creates and destroys workers between its MinSpare and MaxSpare thresholds, and the count oscillates. Brief dips to zero that recover within seconds are normal; sustained zero is the signal.

What mod_status does not tell you

mod_status is a point-in-time snapshot of the scoreboard, and several things you need for correct interpretation live outside it:

• MaxRequestWorkers is not exposed. Read it from the MPM configuration (mpm_event.conf, mpm_worker.conf, or mpm_prefork.conf depending on MPM and distribution). There is no way to compute ceiling utilization from mod_status output alone.
• ServerLimit can silently cap the real maximum. If your MaxRequestWorkers would require more child processes than ServerLimit allows, Apache reduces it and logs a warning at startup. The ceiling you configured may not be the ceiling you have.
• The numbers lie during graceful restarts. Old-generation children linger while finishing requests, so the spawned worker count can temporarily exceed MaxRequestWorkers, while GracefulWorkers and scoreboard G states identify that overlap. Do not alert on snapshots taken during a restart window.
• BusyWorkers has no sub-state detail. A worker in W could be writing to a client, waiting on a backend, or processing. The scoreboard line breaks this down:

# Summarize the scoreboard by state
curl -s http://localhost/server-status?auto | grep "Scoreboard:" | \
  awk '{print $2}' | fold -w1 | sort | uniq -c | sort -nr

On the event MPM, keepalive connections are normally handled by the listener thread and tracked in ConnsAsyncKeepAlive, so idle keepalives do not hold worker slots. A high sustained K count on event is worth investigating, especially for connection filters incompatible with event. On prefork and worker MPMs, K states consume real worker slots, and a high count means keepalive connections are hoarding capacity.

Thresholds that work in production

These thresholds are layered so that only corroborated saturation pages anyone.

• Headroom target: keep at least 25% of the pool idle at your highest normal traffic period. Below that, a routine burst can push you to the cliff.
• Ticket threshold: sustained utilization above 80% of MaxRequestWorkers for more than 10 minutes. Headroom is thin; investigate before it becomes an incident.
• Page criteria: all of the following, sustained for more than 2 minutes: BusyWorkers/MaxRequestWorkers above 0.95, IdleWorkers at 0, ServerUptimeSeconds above 600 (filters out cold-start churn), and at least one corroborating signal: Recv-Q above 0 sustained, the MPM worker-limit code in the error log, or 503 responses appearing.

The corroboration requirement matters. A ratio alone can be fooled by a graceful restart, a spawn burst, or a snapshot artifact. A full worker pool plus a growing kernel backlog or an explicit MaxRequestWorkers message is unambiguous.

For capacity planning, plot peak BusyWorkers day over day. If the peaks trend toward the ceiling, extrapolate the intersection date. If peaks correlate with backend response time rather than request rate, fixing the backend buys more headroom than raising the limit.

Signals to correlate with worker utilization

The combination of high BusyWorkers with a normal request rate is the classic slow-backend cascade: workers are stuck waiting on a proxied backend, not serving traffic. Check backend health before touching Apache’s limits.

How Netdata helps

• Netdata collects the mod_status counters continuously, turning point-in-time BusyWorkers and IdleWorkers snapshots into a time series so you can see sustained saturation versus momentary burst absorption.
• Pool utilization is charted as a ratio, so you can watch the approach to the ceiling instead of reacting after connections start queuing.
• Scoreboard state distribution is tracked per state over time, which is how you distinguish “busy waiting on a backend” from “busy serving clients” without shell access during an incident.
• Correlating worker utilization with request completion rate, 5xx responses, and listen socket metrics on one dashboard shortens the path from “utilization is high” to the actual cause.
• Uptime tracking alongside utilization filters out cold-start noise, matching the uptime gate in the page criteria above.

Netdata’s Apache HTTP Server monitoring with Netdata brings these signals together with per-second metrics and ML anomaly detection.

Related guides

• How Apache HTTPD actually works in production: a mental model for operators
• Apache AH00484: server reached MaxRequestWorkers setting - worker pool exhausted
• Apache HTTPD monitoring checklist: the signals every production web server needs
• Apache HTTPD monitoring maturity model: from survival to expert