BIND SOA serial mismatch: a secondary serving stale data behind the primary

The secondary’s SOA serial is behind the primary. Clients querying the secondary get stale A values, missing new entries, deleted hosts still resolving. The server responds NOERROR because its zone data is internally consistent. It is not current.

This is the precursor to zone expiry, and it is silent. The secondary serves stale but functional answers for the entire SOA expire period, typically 1 to 4 weeks. Monitoring that checks “can I resolve this zone?” passes. Health checks on port 53 pass. Everything looks fine except the data is wrong. The only signal is the serial number gap between primary and secondary, and BIND’s statistics channel does not expose it. You must probe externally.

The mismatch becomes actionable when it persists beyond the zone’s SOA refresh interval. At that point, transfers are failing, not delayed. The expire countdown is running. This article covers how to detect the gap, diagnose why transfers are failing, and fix it before the secondary stops serving the zone entirely.

What this means

A BIND secondary keeps zone data current through polling and NOTIFY:

1. The primary sends NOTIFY over UDP when the zone changes.
2. The secondary schedules a refresh. It queries its configured primaries list, not the NOTIFY sender directly.
3. For each primary, the secondary queries the SOA and compares serials using RFC 1982 arithmetic.
4. If the primary’s serial is greater, the secondary initiates AXFR or IXFR over TCP/53.
5. If the serials match or the primary is unreachable, the secondary moves to the next primary.
6. If no primary offers a newer serial, the secondary keeps its current data and waits for the next refresh interval.

Persistent failure here means the secondary falls behind. The serial mismatch is the symptom. The cause is almost always one of: primary unreachable, transfer denied (ACL or TSIG), network path broken, or serial arithmetic confusion.

RFC 1982 defines serial comparison on a 32-bit sequence space. Serials wrap at 2^32. A serial of 4294967290 followed by 5 is valid: the primary incremented past the boundary. The maximum defined increment is 2^31 - 1 (2147483647). Pairs exactly 2^31 apart are undefined. In practice, serial wraparound is rare and usually signals an operational mistake.

The expire timer runs independently of the refresh mechanism. If the secondary has not completed a successful transfer within the SOA expire interval (the sixth field in the SOA record), it stops serving the zone entirely. Serial mismatch is the warning. Expiry is the cliff.

flowchart TD
    A["Serial mismatch detected"] --> B{"Primary SOA query works?"}
    B -- No --> C["Primary down or unreachable"]
    B -- Yes --> D{"AXFR from primary succeeds?"}
    D -- No --> E["allow-transfer, TSIG, or TCP/53"]
    D -- Yes --> F["Check RFC 1982 arithmetic"]
    E --> G["Fix root cause, then rndc retransfer"]
    F --> G
    G --> H{"Serial converges?"}
    H -- No --> I["Check BIND version, min-transfer-rate-in, journal"]
    H -- Yes --> J["Fixed. Track expire runway."]

Common causes

Quick checks

# Compare SOA serials: third field is the serial number
dig @primary-ip example.com SOA +short
dig @secondary-ip example.com SOA +short

# Check expire runway and transfer state on the secondary
rndc zonestatus example.com

# Check transfer counters from statistics channel
# Note: replace 8653 with your configured statistics-channels port
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); \
  [print(f'{k}: {v}') for k,v in d.get('zonestats',{}).items() if 'Xfr' in k]"

# Look for transfer failures in logs
journalctl -u named --since "1 hour ago" | grep -iE "transfer|xfr|notify|denied"

# Check allow-transfer configuration on the primary
named-checkconf -p | grep -i "allow-transfer"

# Test AXFR from the primary (run from the secondary host)
dig @primary-ip example.com AXFR +time=5

# Check BIND version
rndc status | head -1

Zero-valued counters are omitted from statistics channel output by default. If XfrFail does not appear in the output, the counter is zero.

How to diagnose it

1. Confirm the mismatch is real and persistent. Compare serials from an external probe. If the gap appeared moments ago, it may be a normal in-flight transfer. If it persists beyond the SOA refresh interval, transfers are failing.

2. Check the expire runway. Run rndc zonestatus example.com on the secondary. The expire field tells you how long until the zone is removed. If the runway is below 25% of the total expire value or below 24 hours, this is urgent.

3. Verify primary reachability from the secondary host. Run a SOA query directly against the primary’s IP from the secondary. If this fails, the problem is network path or primary health, not transfer configuration.

4. Test whether transfers are authorized. Attempt an AXFR from the secondary host against the primary. If it fails with REFUSED, the allow-transfer ACL is denying the secondary. If it fails with NOTAUTH, TSIG is the problem.

5. Check transfer logs on both servers. On the primary, look in the security and xfer-out log categories for denied or failed transfers. On the secondary, look in xfer-in for transfer failures, timeouts, or TSIG errors.

6. Check for BIND version-specific default changes. If either server was recently upgraded, allow-transfer may now default to none. Any zone that previously relied on the implicit default will silently stop transferring. Also check for removed options: alt-transfer-source, alt-transfer-source-v6, and use-alt-transfer-source are obsolete and will cause config errors if still present.

7. Consider serial arithmetic and refresh internals. BIND caches unreachable primaries and retries at intervals. A NOTIFY from a primary causes earlier removal from that cache, so NOTIFY can accelerate recovery even if the regular refresh cycle has not fired yet. BIND allows one active refresh and one queued refresh per zone. Additional NOTIFY messages arriving while a refresh is already queued are discarded.

8. Force a retransfer and observe. Run rndc retransfer example.com on the secondary. This forces a full AXFR regardless of serial comparison. If it succeeds, the serial converges and the transfer path works. If it fails, the failure mode is visible in the logs.

Metrics and signals to monitor

Fixes

Fix the allow-transfer ACL

On the primary, explicitly authorize the secondary:

zone "example.com" {
    type primary;
    allow-transfer { <secondary-ip>; };
};

Open TCP/53 between primary and secondary

Zone transfers use TCP port 53. If the primary responds to SOA queries over UDP but transfers fail, the firewall is likely blocking TCP/53 in one direction. This is common in environments where UDP/53 was opened but TCP/53 was assumed unnecessary. NOTIFY messages use UDP/53, so NOTIFY may succeed while transfers fail.

Resync TSIG keys

If transfers fail with NOTAUTH or BADKEY, the TSIG key on the primary and secondary has drifted. Compare the key name, algorithm, and secret on both servers. Regenerate and redistribute if the source of the drift is unclear. Verify that the server statement on the secondary references the correct key for the primary, and that allow-transfer on the primary requires the matching key.

Recover from serial number errors

If the primary’s serial was accidentally set lower than the secondary’s (for example, a zone file restored from an old backup), BIND will not accept a downgrade. The two-step recovery:

1. Set the primary serial to the secondary’s current serial plus 2147483647 (2^31 - 1). This value is guaranteed to be greater by RFC 1982 arithmetic. Reload the primary and let the secondary transfer.
2. After the transfer succeeds, set the primary serial to the desired correct value. Reload again. The secondary transfers again and converges.

Avoid setting the serial to zero. Some DNS implementations treat zero as special and BIND may exhibit unexpected behavior.

Handle slow-transfer termination

BIND can terminate inbound transfers that fall below a minimum rate (min-transfer-rate-in). On large zones transferred over slow or congested links, this can cause repeated transfer failures with no obvious ACL or TSIG error. If transfers start but never complete, check for transfer-rate termination in the xfer-in logs.

Handle journal corruption

If IXFR transfers repeatedly fail and fall back to AXFR (visible in xfer-in logs), the journal file on the primary may be corrupt.

WARNING: rndc sync -clean writes the journal to the zone file and deletes the journal. Run this only on the primary, and ensure no dynamic DNS updates are in flight.

# On the primary: write journal to zone file, remove journal
rndc sync -clean example.com
rndc reload example.com

This commits current journal contents to the zone file and removes the journal. Dynamic updates since the last sync are preserved in the written zone file.

Force an immediate transfer

After fixing the root cause, force convergence on the secondary:

# Forces a full AXFR regardless of serial comparison
rndc retransfer example.com

Use this only after resolving the underlying issue. If the root cause is not fixed, the retransfer will fail the same way. For zones in views, specify the view: rndc retransfer example.com IN external.

Prevention

• Monitor SOA serial consistency externally. Probe both primary and secondary with dig SOA at regular intervals. Alert on any mismatch persisting beyond the SOA refresh interval. This signal is not available in the statistics channel.
• Track the expire runway. Use rndc zonestatus output to compute remaining time before expiry. Alert when runway drops below 50% of the total expire value.
• Audit allow-transfer after BIND upgrades. If the default changed to none, any upgrade requires an explicit audit of all transfer ACLs before the upgrade is considered complete.
• Verify transfer health after every config change. After rndc reload, confirm that the secondary’s serial converges. Do not assume transfers work because the reload command succeeded.
• Monitor XfrFail and XfrSuccess counters. A trending XfrFail or flatlined XfrSuccess while the primary serial advances indicates a transfer problem before the serial gap is large enough to matter.
• Keep NOTIFY paths open. NOTIFY uses UDP/53. If NOTIFY is blocked, the secondary only learns about changes on its next scheduled refresh, adding unnecessary latency to convergence.

How Netdata helps

• Netdata collects XfrFail and XfrSuccess from the BIND statistics channel. A rising XfrFail trend or flatlined XfrSuccess while the primary is known to be updating is the earliest statistics-channel signal that transfers are broken.
• QrySERVFAIL collected per second lets you see the exact moment a zone starts failing on the secondary. If SERVFAIL for a specific zone appears after a sustained period of transfer failures, the causal chain from broken transfer to zone expiry is confirmed.
• Per-second collection means you can correlate a transfer failure timestamp with other events on the same host: BIND restarts, config reloads, network interface flaps, or CPU spikes on the primary that made it too slow to respond.
• The serial mismatch itself requires external SOA probing, which is outside the statistics channel. Pair the transfer counter trends from Netdata with an external serial check for complete coverage of the failure-to-expiry cascade.
• Netdata collects transfer counters alongside cache, recursive client, and query rate metrics on the same timeline, so you can distinguish a transfer-specific problem from general BIND degradation.

Related guides

• BIND cache eviction storms: DeleteLRU, an undersized max-cache-size, and the pressure spiral
• BIND cache hit ratio dropping: the leading edge of recursive pain
• BIND clients-per-query and max-clients-per-query: duplicate recursion for popular names
• BIND forwarding loops: recursion that never terminates and burns recursive slots
• How BIND actually works in production: a mental model for operators
• BIND lame delegations: ’lame server resolving’ and nameservers that are not authoritative
• BIND max-cache-size: sizing the resolver cache without triggering the OOM killer
• BIND monitoring checklist: the signals every production resolver and authoritative server needs
• BIND monitoring maturity model: from survival to expert
• BIND ’no more recursive clients: quota reached’: the recursive-clients circuit breaker
• named not responding on port 53: total outage versus UDP-works-TCP-fails
• BIND resolver NumFetch per view: per-view recursive pressure in split-horizon setups