BIND monitoring maturity model: from survival to expert

Most BIND deployments catch complete outages but miss the slow-burn failures that cause real incidents: cache pressure spirals, recursive client exhaustion, DNSSEC signature expiry, and kernel-level UDP drops that BIND itself never sees.

This article maps four monitoring maturity levels, from Survival to Expert. Each level adds signals that catch failure patterns invisible to the previous one. Use this as an inventory checklist: identify your current level, then decide which signals to add next based on whether you run a recursive resolver, an authoritative-only server, or a mixed-role deployment.

Signal names come from BIND’s statistics channel (JSON at /json/v1/*, XML at /xml/v3/*), OS-level sources like /proc/net/snmp, and rndc commands. Verify counter names against the statistics output for the exact BIND release in use.

flowchart TD
    L1["L1 Survival
named alive, canary, RSS"]
    L2["L2 Operational
rate, SERVFAIL, cache, resources"]
    L3["L3 Mature
RTT, kernel drops, leading indicators"]
    L4["L4 Expert
per-thread, signing, entropy, NTP"]
    L1 --> L2
    L2 --> L3
    L3 --> L4

Level 1: survival

Level 1 answers one question: is named running and answering queries? You will miss every degradation condition, but you will know about total outages.

Use pgrep -x named (exact match), not -f, to avoid false positives from named-checkzone or named-checkconf. The canary query must be role-correct: a recursive resolver should probe a domain requiring recursion; an authoritative server should probe +norecurse against a locally served zone. A REFUSED response is not a dead service. It means the ACL denied the query, which may be correct behavior.

What Level 1 cannot detect: rising latency, declining cache hit ratio, increasing SERVFAIL, approaching resource limits, kernel packet drops, zone transfer failures, or DNSSEC validation problems. A BIND server can be slowly dying for days while passing every Level 1 check.

Level 2: operational

Level 2 answers: is BIND working correctly under production load, and are resources healthy?

The most important transition from Level 1 to Level 2 is monitoring RecursClients as a percentage of the configured limit. The recursive-clients option (default 1000) is BIND’s circuit breaker. A soft quota is 90% for a configured limit of 1000 or less (900), or the limit minus 100 when larger; exceeding it causes BIND to abort its oldest query. At the hard limit, BIND also aborts its oldest query and the new recursive query fails with SERVFAIL. This is how a single slow upstream nameserver cascades into a local resolver outage: the count climbs from stressed to broken with no graceful degradation in between.

Express SERVFAIL rate as a ratio: QrySERVFAIL divided by the sum of classified responses (QrySuccess + QrySERVFAIL + QryNXDOMAIN + QryFORMERR + QryNxrrset + QryReferral). Normal is near 0%. Above 0.1% warrants investigation. Above 1% indicates a systemic problem. BIND caches SERVFAIL according to servfail-ttl (default 1 second, maximum 30 seconds), so a momentary upstream outage can briefly sustain SERVFAIL after recovery. Fixing upstream does not instantly fix the metric.

named starts successfully even if individual zones fail to load. Process liveness checks do not cover this. After every rndc reload, verify zone count via rndc status and scan logs for load failures.

File descriptor limits are OS-controlled. The BIND files option is deprecated in 9.18 and removed in 9.20. Default ulimit -n is often 1024, which is dangerously low for a busy resolver. BIND logs “too many open files” and silently drops queries when the limit is hit.

Level 3: mature

Level 3 answers: is BIND degrading, and are there invisible failures the previous levels missed? This is where leading indicators and kernel-level signals enter.

The most important addition at Level 3 is UdpRcvbufErrors from /proc/net/snmp. BIND’s statistics only count queries it successfully read from the socket. Packets dropped by the kernel due to receive buffer overflow produce no log entry, no counter, nothing in BIND’s own accounting. Teams typically discover this during an incident where “BIND is fine but queries are disappearing.” The metric is invisible to BIND and must be collected from the OS.

The second key addition is the SOA expire countdown. Serial mismatch tells you a transfer failed, but the real danger is how much time remains before the secondary stops serving the zone entirely. Once expired, the secondary returns SERVFAIL or REFUSED for that zone. rndc zonestatus <zone> shows the expire time directly. Alert when the runway drops below 50% of the SOA expire value. Escalate at 25% or 24 hours, whichever is shorter. This is a ticking bomb, not a self-healing condition.

Resolver RTT buckets measure outbound recursive query response times to upstream nameservers, not end-to-end client-perceived latency. BIND has no native inbound latency histogram. A shift toward higher RTT buckets indicates upstream degradation, which drives cache-miss latency and recursive client accumulation. These are per-view counters, so in split-horizon deployments, check each view separately.

Cache eviction counters (DeleteLRU, DeleteTTL) reveal whether the cache is at capacity and evicting entries before TTL expiry. Rapid DeleteLRU increase correlates with undersized max-cache-size and declining hit ratio.

Level 4: expert

Level 4 answers: is the resolver secure, correlated across system layers, and ahead of every known failure mode? These are the signals experienced operators add after major incidents.

Per-thread CPU matters because BIND 9.16+ uses one worker thread per CPU core by default (-n flag to override). Aggregate CPU can look moderate while a single thread is saturated, creating a bottleneck invisible in process-level metrics. Use pidstat -t or thread-level views to see per-thread distribution. Per-core CPU saturation and IRQ/softirq imbalance compound this: if network interrupts are not balanced across cores, one core handles all packet processing while others sit idle.

RRSIG expiry monitoring is essential for authoritative zones with inline signing. Auto-signing can silently fail when key files are missing, permissions are wrong, or disk is full. No log error is produced, no .signed.jnl file is created, and the zone continues being served with expiring signatures because the authoritative server does not validate its own signatures. Validating resolvers worldwide reject the zone when signatures expire, producing hours-long outages invisible from the authoritative operator’s perspective. The only reliable detection is checking actual RRSIG validity windows and .signed.jnl file presence.

NTP offset tracking matters because DNSSEC depends on accurate time. Even 5 minutes of clock drift can cause validation failures. Correlating NTP offset with ValFail counters catches this before it manifests as broad SERVFAIL for signed domains.

Source port entropy is the last line of defense against cache poisoning. BIND randomizes source ports for outbound recursive queries. NAT devices, firewalls, or misconfiguration can constrain this entropy. The porttest.dns-oarc.net probe reports the quality of randomization. Anything below “GREAT” warrants investigation.

rndc recursing sampling reveals which upstream nameservers are causing pile-up during recursive resolution cascades. During incidents, this distinguishes a single bad upstream from a broad network issue. It is invaluable during incidents but almost never collected proactively.

Cross-level failure patterns

Three composite failure patterns span multiple levels and require cross-level signal correlation to diagnose correctly:

• Recursive resolution cascade: RecursClients climbing toward limit (Level 2) plus QueryTimeout increasing per view (Level 3) plus rndc recursing showing pile-up on specific upstreams (Level 4). CPU may remain moderate because threads are blocked waiting, not computing.
• DNSSEC time bomb: ValFail spiking (Level 2) plus SERVFAIL for signed domains while unsigned domains work normally (Level 3) plus NTP offset drift (Level 4). The +cd flag in dig bypasses validation and confirms DNSSEC as the cause.
• Zone staleness cascade: SOA serial mismatch between primary and secondary (Level 2) plus expire countdown shrinking (Level 3) plus DNSSEC signing freshness check for the zone (Level 4).

How Netdata helps

Netdata’s BIND collector ingests statistics channel data at per-second granularity, which matters for several reasons specific to this maturity model:

• Rate computation is automatic. BIND’s counters are cumulative since process start. Netdata computes deltas and rates, so you see queries-per-second and SERVFAIL-per-second without manual sampling intervals.
• Kernel metrics sit alongside BIND metrics. UdpRcvbufErrors from /proc/net/snmp appears on the same timeline as BIND’s query counters, making the Level 3 kernel-drop gap immediately visible without a separate dashboard.
• Per-view breakdowns surface cache hit ratio, resolver failure counters (QueryTimeout, Lame), and RTT distributions in context, supporting split-horizon debugging.
• Process-level metrics (RSS, FD count, per-thread CPU) are collected natively alongside the BIND collector, so Level 4 signals do not require separate tooling or manual correlation.

Related guides

• BIND monitoring checklist: the signals every production resolver and authoritative server needs
• How BIND actually works in production: a mental model for operators
• BIND SERVFAIL responses: what a DNS SERVFAIL actually means and how to trace the cause
• BIND REFUSED responses: ACL denials, recursion policy, and clients that get locked out
• named not responding on port 53: total outage versus UDP-works-TCP-fails
• BIND NXDOMAIN spike: DGA malware, water torture, and Windows suffix search lists
• rndc not responding: control-plane failure while queries still work
• BIND ’no more recursive clients: quota reached’: the recursive-clients circuit breaker
• BIND RecursClients climbing toward the limit: reading the recursive saturation gauge
• BIND recursive resolution cascade: one slow upstream taking down all resolution
• BIND forwarding loops: recursion that never terminates and burns recursive slots
• BIND resolver NumFetch per view: per-view recursive pressure in split-horizon setups