BIND RecursClients climbing toward the limit: reading the recursive saturation gauge

RecursClients is the number of queries currently in flight, each waiting for an upstream nameserver to respond. When this gauge climbs toward the configured recursive-clients limit, BIND is running out of slots to start new recursive lookups. Past the hard limit, every new recursive query receives SERVFAIL.

The metric is a gauge, not a rate. The absolute number means little without the configured ceiling next to it: 300 is comfortable against a limit of 1000 and dangerous against a limit of 350. Track RecursClients as a percentage of recursive-clients, not as a raw count, and watch the daily peak for runway estimation.

The failure curve is cliff-edge. BIND has no graceful degradation between stressed and broken. At the soft quota (90% by default), it starts aborting the oldest pending queries to make room for new ones. At the hard limit, new recursive queries fail immediately with SERVFAIL.

What RecursClients actually measures

Every time BIND receives a query that requires recursion (a cache miss for a domain it is configured to resolve), it allocates a recursive client slot. That slot stays occupied until the upstream nameserver responds, the query times out, or the slot is forcibly reclaimed. RecursClients is the instantaneous count of occupied slots.

The value goes up when new recursive queries arrive faster than old ones complete, and down when upstream responses arrive and slots are freed. On an idle server or an authoritative-only server with recursion disabled, RecursClients sits at or near zero. On a busy recursive resolver, it fluctuates with query volume and upstream latency.

Collect it from the statistics channel:

# Check current recursive client count
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); \
  print('RecursClients:', d.get('nsstats',{}).get('RecursClients', 'N/A'))"

The port (8653 in this example) depends on your statistics-channels configuration. The counter lives under the nsstats key in the JSON output and is an absolute gauge value, not a cumulative counter. Zero-valued counters are omitted by default in non-verbose statistics channel output, so a missing key on an idle server usually means zero.

For a detailed view of what is in flight, rndc recursing lists each active recursive query with the upstream nameserver it is waiting on. It also reports statistics on active, allowed, and dropped fetch counts due to fetches-per-server and fetches-per-zone quotas. This is the most valuable diagnostic command during an incident because it tells you not just how many slots are occupied but which upstream is causing the pile-up.

The recursive-clients limit and quota mechanics

The recursive-clients option in named.conf caps the number of concurrent recursive queries BIND will process. The default is 1000. BIND enforces this limit in two tiers:

Soft quota. When recursive-clients is 1000 or less (the default), the soft quota is 90% of the configured value: 900 at the default. When recursive-clients is set above 1000, the soft quota is recursive-clients - 100 (or minus the number of worker threads, whichever is greater). When RecursClients exceeds the soft quota, BIND logs recursive-clients soft limit exceeded, aborting oldest query and drops the oldest pending recursive client to make room for each new incoming request. This is backpressure, not a crash.

Hard quota. When RecursClients reaches the configured recursive-clients value, BIND drops both a pending request and the new inbound query. The new query receives SERVFAIL.

flowchart TD
    A[Upstream nameserver
slows or goes silent] --> B[Each in-flight query
holds a slot for
timeout duration]
    B --> C[Slots accumulate
faster than they drain]
    C --> D[50% of limit:
ticket-level alert]
    D --> E[90% soft quota:
oldest queries aborted
to make room]
    E --> F[100% hard limit:
all new recursive
queries get SERVFAIL]

The transition from 90% to 100% is not gradual. Once the hard limit is hit, resolution for every new recursive query fails, including queries targeting upstream nameservers that are healthy.

Reading the saturation gauge

Operational thresholds expressed as a percentage of the configured recursive-clients limit:

Two rules govern how to use this gauge operationally:

Daily peak should stay under 50% of limit. This provides headroom for traffic spikes, upstream slowdowns, and attack absorption. If your daily peak regularly exceeds 50%, you are operating without a safety margin.

Track the daily peak as a trend for runway estimation. If the daily peak is growing by X% per week, time-to-saturation is approximately (100% - current_peak%) / X% weeks.

Two gating conditions prevent false readings:

• Idle or authoritative-only servers sit near zero because no recursive queries are in flight. This is correct, not a monitoring gap.
• Cold start produces a temporary spike because cache hit rate is 0% after restart and every query triggers recursion. An uptime gate (typically 300 seconds) suppresses alerts during this cache-warming window.

What a rising count tells you

RecursClients climbing means fetches are piling up waiting on upstream. Each in-flight query occupies a slot until the upstream responds or the resolver-query-timeout fires. Under normal conditions, upstream nameservers respond in tens to hundreds of milliseconds, and slots cycle quickly. When upstream degrades, each slot is held longer.

The resolver-query-timeout option controls the total time BIND will spend on a single recursive resolution before abandoning it. When multiple queries target a slow upstream simultaneously, slots fill rapidly.

During an active climb, rndc recursing is the diagnostic command:

# Identify which upstream nameservers are causing the pile-up
rndc recursing | awk '{print $NF}' | sort | uniq -c | sort -rn | head -10

If most in-flight queries target the same upstream nameserver, the problem is localized. If they are spread across many upstreams, the problem is broader: network path failure, DNSSEC validation overhead, or volume overload.

CPU may not be elevated during a recursive pile-up. Worker threads are blocked waiting for network responses, not computing. A resolver with RecursClients at 85% and low CPU is under upstream latency pressure, not compute pressure. Adding CPU does not help; failing faster or routing around the slow upstream does.

Tuning the limit

The default recursive-clients value of 1000 is adequate for small or medium deployments but too low for busy resolvers. Raising it is a valid response to sustained high utilization, but each slot consumes downstream resources:

• File descriptors. Each recursive query uses one or more file descriptors for outbound sockets. If recursive-clients is set high relative to the FD limit, FD exhaustion occurs before the recursive client limit is reached.
• Memory. Each recursing client uses approximately 20 KB of memory. At the default 1000, that is roughly 20 MB. At 10000, approximately 200 MB.
• clients-per-query and max-clients-per-query. These options (defaults 10 and 100 respectively) control how many identical concurrent recursive queries BIND processes versus deduplicating. Higher clients-per-query means more clients share a single outstanding fetch, reducing slot pressure for popular domains.

Check the currently configured value:

# Show the configured recursive-clients value
named-checkconf -p /etc/named.conf | grep recursive-clients

If no value is printed, BIND is using the default of 1000.

Signals to correlate with RecursClients

RecursClients is the hub of a correlated signal set. Interpret it alongside:

A rising count with a falling cache hit ratio and rising QueryTimeout is the recursive resolution cascade: upstream slowness is filling slots. A rising count with stable cache hit ratio and no upstream signal suggests a volume increase that may warrant capacity expansion.

Version-specific counter reliability

The RecursClients counter has had bugs that corrupt its value:

• BIND before 9.18.1 / 9.16.27: The counter could be miscalculated in certain resolution scenarios, potentially dropping below zero.
• BIND before 9.18.9 / 9.16.36: The counter could overflow in certain resolution scenarios, producing implausibly large values.

If your RecursClients value appears negative or implausibly large, and you are running a version older than 9.18.9 or 9.16.36, the counter itself may be corrupt. Any percentage-of-limit alert built on a corrupt counter will fire spuriously or miss real saturation. Upgrade to at least 9.18.9 or 9.16.36 before trusting the metric.

The JSON key RecursClients under nsstats is stable across BIND 9.16 and later.

How Netdata helps

Netdata collects RecursClients as an absolute gauge, so per-second visualization reflects real-time saturation state without rate conversion.

• Percentage-of-limit alerting. Netdata alerts on RecursClients as a percentage of the configured recursive-clients value, applying the 50% warning and 90% critical thresholds automatically. This normalizes across instances with different configured limits.
• Correlation with upstream signals. Netdata dashboards place RecursClients alongside QueryTimeout rates, upstream RTT distributions, and SERVFAIL rates, so cause and effect are visible in one view.
• Cache hit ratio correlation. A declining cache hit ratio is often the earliest predictor of RecursClients pressure. Both metrics share a timeline.
• Uptime gating. Alert rules can incorporate process uptime to suppress false positives during cold-start cache warming.
• Daily peak tracking. Long-term storage retains the daily peak pattern for runway estimation.

Related guides

• How BIND actually works in production: a mental model for operators
• BIND monitoring checklist: the signals every production resolver and authoritative server needs
• BIND monitoring maturity model: from survival to expert
• BIND NXDOMAIN spike: DGA malware, water torture, and Windows suffix search lists
• BIND REFUSED responses: ACL denials, recursion policy, and clients that get locked out
• BIND SERVFAIL responses: what a DNS SERVFAIL actually means and how to trace the cause