$ guides / coredns / coredns-response-truncation-tcp-fallback ▌

Operations Guides

CoreDNS response truncation: the TC bit, EDNS bufsize, and TCP fallback

Most DNS traffic fits in a single UDP datagram. When it does not, the protocol has an escape hatch: the server sets the TC (truncated) bit in the response, and the client retries the same query over TCP. CoreDNS implements this correctly in the common case, but the edges are where production incidents live: firewalls that silently block TCP/53, legacy clients that do not speak EDNS0, and known CoreDNS bugs where the TC flag never reaches the client at all.

This guide covers how truncation works in CoreDNS, which plugin controls the buffer size, and how to detect truncation when there is no dedicated metric for it. The target symptom: intermittent resolution failures on large TXT records, SRV-heavy headless services, or ANY-style responses, where small lookups succeed and large ones fail.

What truncation is and why it matters

Without EDNS0, a DNS response over UDP is limited to 512 bytes. With EDNS0, the client advertises a buffer size in its query and the server may send UDP responses up to that size. When the full answer does not fit, the server must not fragment or silently clip it. It sends a response with the TC bit set and typically an empty or partial answer section: this answer is bigger than UDP allows, ask again over TCP.

Operationally, three properties make this a debugging trap:

Truncation is normal protocol behavior, not an error. Nothing is logged by default, no error metric increments, and well-behaved clients only see a small latency bump from the TCP round trip.
TCP fallback is a hard dependency. If TCP/53 is blocked anywhere on the path between client and CoreDNS (or between CoreDNS and an upstream), the retry fails. From the application’s perspective, DNS just does not work, and nothing on either side reports why.
Not all clients and paths handle TC correctly. Non-EDNS0 clients, minimal stub resolvers, and specific CoreDNS plugin combinations have known bugs where truncated answers are delivered as if they were complete.

The responses that cross the threshold in practice are the big ones: headless Services with many endpoints, TXT-heavy records, SRV record sets, and ANY-style queries. As clusters grow, responses that used to fit start truncating, and a latent TCP/53 problem suddenly becomes visible.

How it works

The truncation decision is made per query, based on the buffer size the client advertised. Three numbers matter:

512 bytes: the limit for clients that do not send an EDNS0 OPT record.
The client’s advertised bufsize: for EDNS0 clients, whatever the client put in its OPT record.
1232 bytes: the default applied by CoreDNS’s bufsize plugin, aligned with the DNS Flag Day 2020 recommendation. 1232 was chosen to keep UDP responses below the size where IP fragmentation becomes a risk on nearly all current networks.

The bufsize plugin (available since CoreDNS 1.6.6) caps the client’s advertised bufsize downward. It accepts a value in the range 512 to 4096 and defaults to 1232 when enabled without an argument. Two behaviors to know precisely:

It only reduces the advertised size. A client advertising 512 is not upgraded to 1232.
It only affects EDNS0 queries, because the buffer size lives in the OPT record. Non-EDNS0 clients keep the 512-byte ceiling.

When the serialized answer exceeds the applicable limit, CoreDNS sets the TC bit and sends the truncated response. A compliant stub resolver (glibc, musl, the Go resolver, systemd-resolved) then re-issues the identical query over TCP, and CoreDNS answers on TCP/53 with the full response.

sequenceDiagram
  participant C as Client
  participant CD as CoreDNS
  participant U as Upstream
  C->>CD: UDP query (EDNS0, bufsize 1232)
  CD->>U: forward query
  U-->>CD: answer (1800 bytes)
  CD-->>C: UDP response, TC=1 (truncated)
  C->>CD: TCP query (same question)
  CD-->>C: TCP response, full answer

The same logic applies one hop up. When CoreDNS forwards to an upstream and the upstream’s response is itself truncated, the forward plugin’s prefer_udp option controls behavior: it tries UDP first even for queries that arrived over TCP, and if the upstream response comes back truncated, it retries the upstream over TCP. The force_tcp option skips UDP upstream entirely and takes precedence over prefer_udp.

Where it shows up in production

Firewall blocks TCP/53

The most common failure mode, and one DNS Flag Day 2020 explicitly calls out: many firewalls and security appliances allow UDP/53 but block TCP/53, on the outdated assumption that DNS is UDP-only. When that happens:

The client receives a TC=1 response.
The client opens a TCP connection to the resolver on port 53.
The connection is dropped or times out.
Resolution fails silently, with no error on either side that names the cause.

The signature is that small lookups work and large lookups fail. A record with two answers resolves; the same record after the endpoint count grows does not. Because TCP/53 is rarely exercised, this can sit latent for years until some response crosses the size threshold. Verify both directions: client to CoreDNS, and CoreDNS to upstreams.

Non-EDNS0 clients and the dropped TC flag (issue #4183)

Older or minimal clients (embedded resolvers, busybox-style tools) send plain DNS without an OPT record. CoreDNS issue #4183, affecting CoreDNS 1.7.x and earlier: when the forward plugin received a truncated response from an upstream and neither force_tcp nor prefer_udp was configured, CoreDNS neither retried the upstream over TCP nor set the TC flag in its response to the client. The client received an incomplete answer that looked complete. That is the worst outcome: no error, just wrong data. Fixed in CoreDNS 1.8.0, but worth checking if you run older CoreDNS builds or unusual client populations.

bufsize plugin and non-EDNS0 clients (issue #5366)

A second known issue in the opposite direction: with the bufsize plugin enabled, clients that do not use EDNS0 could receive a reply containing an OPT RR (which RFC 6891 says they should not) and potentially responses larger than 512 bytes. Naive stub resolvers, including busybox nslookup, can fail on such replies. This affected versions from bufsize’s introduction in 1.6.6 through 1.9.3; fixed in CoreDNS 1.9.4.

Malformed oversized upstream responses (fixed in 1.11.x)

Some upstreams (notably certain Windows AD DNS servers) send UDP responses larger than the requested bufsize without setting the TC bit, which is a protocol violation. Before the fix in CoreDNS 1.11.2/1.11.3, CoreDNS failed to parse these (“overflowing header size”), retried over UDP for several seconds, and eventually returned SERVFAIL. Since 1.11.3, the forward plugin instead sends the client an empty response with the TC bit set, prompting a clean TCP retry. If you forward to AD-integrated DNS and see periodic SERVFAIL on large responses, check your CoreDNS version first.

ExternalName services (issue #4684)

There is a reported case where CoreDNS fails to set the TC flag for ExternalName services with large payloads; the AA flag being set appears to prevent clients from retrying over TCP. If you use ExternalName services with large associated answers, validate truncation behavior explicitly rather than assuming it.

Tuning and tradeoffs

Raising the bufsize above 1232. The bufsize plugin allows up to 4096. A larger bufsize means fewer truncations and fewer TCP round trips, at the cost of reintroducing the IP fragmentation risk that Flag Day 2020 set out to eliminate. Fragmented UDP responses are dropped by many middleboxes and are a known amplification and reliability hazard. Unless you control the entire network path and have measured fragmentation behavior, keep 1232.

Lowering the bufsize. Some operators deliberately set a small bufsize to force TCP fallback early, trading latency for predictability in environments where fragmented UDP is unreliable. Legitimate, but it increases TCP connection load on CoreDNS.

prefer_udp vs force_tcp upstream. With prefer_udp, CoreDNS retries truncated upstream responses over TCP automatically. With force_tcp, every upstream query uses TCP, eliminating truncation on that hop at the cost of connection overhead per query. For upstreams known to return large answers (or known to violate the protocol, as above), force_tcp removes a whole class of failure.

Fix the data. If a specific record set routinely exceeds 1232 bytes, the cheapest fix is often to shrink the answer: split oversized TXT records, reduce endpoint counts behind a single headless service, or review whatever is generating ANY-style traffic.

Signals to watch in production

The single most important fact: there is no Prometheus metric for truncation. No counter increments when CoreDNS sets the TC bit. You have to infer truncation from adjacent signals:

Signal	Why it matters	Warning sign
`coredns_dns_requests_total{proto="tcp"}` share	Clients retrying over TCP show up here; TCP share is your only metric proxy for truncation rate	Sustained rise in TCP share without a config change, or TCP share near zero when large responses are common
`coredns_dns_response_size_bytes`	Histogram of response sizes with a `proto` label; shows how much traffic sits near or above 512/1232	A growing mass of responses above 1232 bytes, or a sudden distribution shift
Client-side resolution failures	The actual symptom when TCP fallback fails	Lookups failing only for large record sets; intermittent failures that correlate with endpoint count growth
dnstap or packet capture	The only direct observation of TC=1 responses and TCP retries	TC responses with no corresponding TCP retry from the same client indicates blocked TCP/53

A practical baseline: compute the TCP share of coredns_dns_requests_total per server block and record it. When that share moves, either your response sizes moved or something started (or stopped) blocking the retry path. When the share is suspiciously zero in an environment with large headless services, suspect that truncation is happening and the TCP retry is failing before it ever reaches CoreDNS.

If you need ground truth, enable the dnstap plugin or capture traffic on port 53 and look for responses with the TC flag. That is the only way to count truncation events exactly.

How Netdata helps

Netdata shortens this diagnosis by putting the proxy signals on one timeline:

Per-second proto breakdown of DNS requests, so a rising TCP share is visible as it happens rather than in a weekly review.
Response size distribution alongside request volume, so you can correlate “responses got bigger” with “TCP share went up” directly.
Cache hit ratio and upstream latency in the same view, since TCP retries change both the latency profile and the effective query load on upstreams.
Long retention on the TCP-share baseline, which makes “this changed after the endpoint count grew” a measurable statement instead of a guess.
Node-level network metrics alongside CoreDNS, which matters when the real question is whether TCP/53 connections are being established at all.