BIND DNSSEC failing from clock drift: NTP, RRSIG inception/expiry windows, and SERVFAIL

Signed domains start returning SERVFAIL while unsigned domains resolve normally. The named process is running, CPU is moderate, and the cache hit ratio has not collapsed. There was no recent rndc reload or configuration change. The failure appeared gradually over minutes or hours, not all at once.

This is the fingerprint of a DNSSEC validation failure caused by clock drift. Every RRSIG record carries an inception timestamp (when the signature becomes valid) and an expiry timestamp (when it stops being valid). BIND compares these against the local system clock. If the clock drifts far enough outside the RRSIG validity window, valid signatures appear expired or not-yet-valid, and BIND returns SERVFAIL for the entire domain.

Most major internet domains are DNSSEC-signed, so they fail. Unsigned zones continue to work, which can mislead operators into thinking the problem is upstream or zone-specific rather than local. The longer NTP has been broken, the wider the drift, and the more domains are affected.

What this means

DNSSEC validation is a time-dependent cryptographic check. BIND retrieves a DNSKEY and RRSIG chain from upstream, then verifies that the signatures were generated by the key holder and that the signature’s validity window encompasses the current time. The RRSIG record contains two timestamps:

• Inception: the earliest time the signature is valid. For zones signed by BIND with default settings, the inception is set to one hour before the signing time to tolerate modest clock skew. For zones signed by other operators, the inception window depends on their signing practices and may be tighter.
• Expiry: the latest time the signature is valid. The default sig-validity-interval is 30 days.

When the local clock is ahead of real time, signatures that are still valid appear to have expired. When the clock is behind, signatures may not have reached their inception yet. Either condition causes BIND to mark the response as bogus and return SERVFAIL.

The validation failure is then cached. BIND stores DNSSEC validation failures in its bad cache for at least 30 seconds and up to lame-ttl (default 10 minutes). Even after you fix the clock, stale SERVFAIL responses persist until the bad cache entries expire or are explicitly cleared.

flowchart TD
    A["NTP daemon fails or stops"] --> B["System clock drifts"]
    B --> C["Clock outside RRSIG validity window"]
    C --> D{"Clock direction"}
    D -->|"Ahead"| E["Signatures appear expired"]
    D -->|"Behind"| F["Signatures not yet valid"]
    E --> G["ValFail counter rising"]
    F --> G
    G --> H["SERVFAIL for signed domains"]
    H --> I["Unsigned domains still resolve"]
    I --> J["Bad cache extends SERVFAIL after fix"]
    J --> K["Chrony cannot resolve NTP servers if they use signed domains"]
    K --> A

The diagram shows the circular dependency that can trap operators. When the clock drifts far enough, DNSSEC validation fails. If chronyd uses hostname-based NTP server addresses, those hostnames may live in DNSSEC-signed zones, and resolution fails. Chrony cannot correct the clock because it cannot resolve the NTP server. The system stays broken until the loop is broken manually.

Common causes

Quick checks

Run these read-only commands to confirm the diagnosis before making changes.

# Check system clock sync status
timedatectl status

# Check NTP offset and stratum
chronyc tracking

# Check NTP source reachability
chronyc sources

# Confirm DNSSEC is the problem: validation-enabled query should fail
dig +time=2 +tries=1 @127.0.0.1 google.com A

# Bypass validation: this should succeed if DNSSEC validation is the root cause
dig +time=2 +tries=1 @127.0.0.1 google.com A +cd

# Check DNSSEC validation failure counters per view
# Adjust the port to match your statistics-channel configuration
curl -s http://localhost:8053/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); \
  [print(f'{v}: {k}={s}') for v,vd in d.get('views',{}).items() \
  for k,s in vd.get('resolver',{}).get('stats',{}).items() if k.startswith('Val')]"

# Check trust anchor state (BIND 9.11+)
rndc managed-keys status

# Look for RRSIG-specific log messages
journalctl -u named --since "1 hour ago" | grep -i "RRSIG\|validat\|dnssec"

The critical test is comparing dig ... A versus dig ... A +cd. If the first returns SERVFAIL and the second returns a valid answer, DNSSEC validation is confirmed as the cause. The +cd flag sets the Checking Disabled bit, telling BIND to skip validation and return the upstream answer as-is.

If timedatectl status shows “System clock synchronized: no” or chronyc tracking shows a large offset (anything beyond a few hundred milliseconds is concerning, beyond 30 seconds is critical for DNSSEC), clock drift is the likely root cause.

How to diagnose it

1. Confirm DNSSEC is the failure layer. Run dig @127.0.0.1 google.com A and dig @127.0.0.1 google.com A +cd. If the first returns SERVFAIL and the second succeeds, validation is the problem. If both fail, the issue is elsewhere (upstream connectivity, recursion failure, zone load).

2. Check the clock. Run timedatectl status and chronyc tracking. Look at the System time and Last offset fields. Any offset larger than a few seconds is suspicious. More than 30 seconds is a definitive cause of DNSSEC failures.

3. Determine direction of drift. If the clock is ahead, BIND logs messages like “RRSIG has expired” because valid signatures appear past their expiry. If the clock is behind, BIND logs “RRSIG validity period has not begun” because signatures have not reached inception. Check BIND logs for these exact strings.

4. Check NTP source health. Run chronyc sources. If all sources show reachability of 0 or are marked as unreachable, chronyd has lost contact with its upstream time servers.

5. Check for the circular dependency. If chronyd cannot reach NTP servers configured by hostname (not IP), attempt to resolve them: dig @127.0.0.1 pool.ntp.org A +cd. If this fails without +cd but works with it, chrony is trapped in the DNSSEC bootstrap loop. The clock cannot be corrected because DNSSEC validation fails, and DNSSEC validation fails because the clock is wrong.

6. Verify trust anchors if clock is correct. If chronyc tracking shows a normal offset but ValFail is still rising, check rndc managed-keys status. A corrupted or stale managed-keys database can cause broad validation failures independent of clock state.

7. Check for systemd-resolved interference. On systems using systemd-resolved as a stub resolver (listening on 127.0.0.53), validation may also fail there due to the same clock skew. Use resolvectl status to check DNSSEC state.

Metrics and signals to monitor

Fixes

Fix the clock

The primary fix is to restore accurate system time. If chronyd is running but cannot reach sources, check network connectivity to the NTP servers. If chronyd is stopped, start it: systemctl start chronyd (or ntpd if using legacy NTP).

If the clock is dramatically wrong (hours or days off), chronyd by default will not step the clock in one jump. You may need to force a manual correction:

# Force chrony to step the clock immediately (disruptive: may affect TLS, databases, logging)
sudo chronyc -a makestep

This applies the offset as an immediate step rather than a gradual slew. It is safe for DNS purposes but may affect other time-sensitive applications on the host. Run it during a maintenance window if possible.

Break the chrony DNSSEC bootstrap loop

If chrony cannot resolve NTP server hostnames because DNSSEC validation is failing, break the circular dependency:

1. Configure NTP servers by IP address in /etc/chrony/chrony.conf (or /etc/chrony.conf). IP-based configuration bypasses DNS resolution entirely. This is the most reliable fix.

2. Temporarily disable DNSSEC validation on the local resolver, allow chrony to sync, then re-enable it. Run rndc flush after re-enabling to clear stale entries. This is a temporary workaround, not a permanent fix.

3. Use an external resolver for chrony by pointing chronyd at a public resolver IP directly, if your network policy allows it.

Clear the bad cache after fixing the clock

After the clock is corrected, stale SERVFAIL responses persist in BIND’s bad cache. The bad cache stores DNSSEC validation failures for at least 30 seconds and up to lame-ttl (default 10 minutes). Clear it explicitly:

# Clear the entire cache (including bad cache entries)
rndc flush

# Or clear entries for a specific domain
rndc flushname google.com

rndc flushtree may not clear bad cache entries. Use rndc flush, rndc flushname, or a named restart to clear them reliably. If you used rndc flushtree and are still seeing SERVFAIL after fixing the clock, this is why.

Repair corrupted managed-keys

If the clock is correct but ValFail persists, the managed-keys database may be corrupted. Check the state:

rndc managed-keys status

If the trust anchor is not in a trusted state, you may need to reinitialize it. Stop named, remove the managed-keys bind file (location varies by distribution, commonly in the working directory specified in named.conf), and restart named to rebuild it from the built-in root trust anchor. Do this only after confirming the clock is accurate, as a wrong clock during trust anchor initialization can cause the same problem to recur.

Prevention

• Monitor NTP offset as a first-class signal. Alert on any sustained offset exceeding 30 seconds. This precedes DNSSEC validation failure by minutes to hours.
• Configure NTP servers by IP address in chrony.conf to avoid the DNSSEC bootstrap loop. If you must use hostnames, ensure a fallback IP-based source exists.
• Run NTP on every DNS resolver host. Do not rely on the hypervisor or container runtime to provide accurate time. VMs and containers are susceptible to clock drift, especially under CPU pressure or during live migration.
• Alert on loss of NTP source reachability. chronyc sources showing all sources unreachable is an early warning that drift will begin. This alert should fire before the offset grows large enough to affect DNSSEC.
• Include +cd in your diagnostic toolkit. When SERVFAIL appears, the +cd test immediately distinguishes DNSSEC validation failures from other causes.
• Test your NTP failover. If your primary NTP source becomes unreachable, does the secondary work? Does chrony step the clock automatically, or does it require manual intervention? Test this before you need it.
• Monitor ValFail continuously. A non-zero ValFail rate that was previously zero is an early signal, even before users report problems. Correlate ValFail with clock offset to catch the pattern early.

How Netdata helps

Netdata’s per-second metrics collection is useful for DNSSEC clock drift incidents because the failure develops over minutes, and the correlation between clock offset and ValFail is the diagnostic key.

• NTP offset is collected per-second from chrony or ntpd, showing the exact moment NTP synchronization was lost and the offset growth toward the DNSSEC-breaking threshold.
• ValFail and ValAttempt are collected from the BIND statistics channel, allowing you to overlay validation failures against clock offset on the same dashboard. The temporal correlation (offset rises first, ValFail follows) confirms the causal chain.
• SERVFAIL rate is tracked as a percentage of total responses, showing user-visible impact alongside root cause signals.
• Anomaly detection on NTP offset can alert on drift before it reaches the DNSSEC-breaking threshold.
• Composite alerting across NTP offset, ValFail rate, and SERVFAIL rate lets you define a multi-signal page that fires only when clock drift is actually causing validation failures.

Related guides

• BIND cache eviction storms: DeleteLRU, an undersized max-cache-size, and the pressure spiral
• BIND cache hit ratio dropping: the leading edge of recursive pain
• BIND clients-per-query and max-clients-per-query: duplicate recursion for popular names
• BIND cold cache after restart: the warming storm and elevated upstream load
• BIND forwarding loops: recursion that never terminates and burns recursive slots
• How BIND actually works in production: a mental model for operators
• BIND journal (.jnl) corruption: dynamic-update and IXFR failures that block zone load
• BIND lame delegations: ’lame server resolving’ and nameservers that are not authoritative
• BIND max-cache-size: sizing the resolver cache without triggering the OOM killer
• BIND monitoring checklist: the signals every production resolver and authoritative server needs
• BIND monitoring maturity model: from survival to expert
• BIND ’no more recursive clients: quota reached’: the recursive-clients circuit breaker