$ guides / traefik / traefik-tls-certs-not-after ▌

Operations Guides

Traefik TLS certificate expiry monitoring: reading traefik_tls_certs_not_after

traefik_tls_certs_not_after is the only signal Traefik gives you about certificate expiry. There is no metric for ACME renewal failures, no counter for rate limit rejections, no gauge for “the challenge did not complete.” You get a Unix timestamp per certificate, and everything else has to be inferred from it or pulled from logs.

Most teams wire it to a pager with a threshold like “alert at 7 days.” Then the first ACME renewal happens, the old certificate series stays in the output, and the pager fires for a certificate that is no longer serving anything. Or the alert fires for a staging certificate, or for a dormant cert left in the store from a decommissioned hostname, and the on-call learns to ignore it. This guide covers reading the metric correctly and building an alert chain that only wakes someone up when a certificate that is actually serving production traffic is about to expire.

What the metric actually reports

traefik_tls_certs_not_after is a gauge. Its value is the Unix epoch timestamp of the certificate’s notAfter field, and it carries three labels:

Label	Content
`cn`	The certificate’s Common Name
`sans`	Subject Alternative Names
`serial`	The certificate serial number

A typical exposition looks like:

traefik_tls_certs_not_after{cn="example.com",sans="example.com,www.example.com",serial="03ab..."} 1.7935e+09

To get days remaining, subtract time() and divide by 86400. The important property is what the metric counts: every certificate in Traefik’s certificate store. That includes certificates currently terminating traffic, certificates that were renewed and replaced, certificates for routers that no longer exist, and certificates issued against the Let’s Encrypt staging CA. The store is a cache, not a live view of what is on the wire.

Two prerequisites before you see anything at all:

Prometheus metrics must be enabled (--metrics.prometheus=true, typically scraped from a dedicated metrics entrypoint).
The metric only appears once Traefik has at least one TLS-terminating entrypoint and certificates loaded. If your entrypoints are plain HTTP, the series is absent entirely. Absence of the metric is not “no certs expiring soon”; it is “nothing to report” or “not configured,” and those are very different states.

The three traps

1. The store outlives the certificate’s usefulness

A cert for a hostname you migrated off Traefik six weeks ago still emits a series, still ticks toward expiry, and still crosses your alert threshold. There is no label that says “this cert is currently serving traffic,” and no metric that maps a certificate to an active router. This is the fundamental reason paging on any single cert’s expiry false-fires.

2. Stale serials after renewal

There is a long-standing open bug (traefik/traefik#8606): when ACME renews a certificate, the old certificate’s series is not removed from the Prometheus output. You end up with multiple series for the same cn/sans pair, differing only by serial. The old series keeps counting down toward its original expiry date. With a 90-day Let’s Encrypt certificate renewed at 30 days remaining, a 7-day alert fires about 23 days after renewal (a 30-day plan threshold can fire immediately), and the stale series remains until the process restarts. Users have confirmed this persists in recent v3 releases. A proposed fix (setting stale series to 0) stalled because maintainers considered the sentinel value approach unsatisfactory, so plan on working around it in your queries, not waiting for it to disappear.

3. Staging certificates look valid to the metric

If your caServer points at the Let’s Encrypt staging endpoint (acme-staging-v02.api.letsencrypt.org), Traefik issues certificates signed by “Fake LE Intermediate X1” and reports them in traefik_tls_certs_not_after like any other valid cert. Traefik does not distinguish staging from production ACME in its metrics. Browsers reject them. So the metric can show a healthy 80 days remaining while every client that matters is throwing TLS errors. This comes up constantly when a staging config leaks into production, or when someone flips the CA server to debug rate limits and forgets to flip it back.

Alert design that does not false-fire

A three-tier design maps cleanly onto the failure modes above. The tiers differ in who gets woken up and why:

PLAN at 30 days. Let’s Encrypt certificates live 90 days and Traefik attempts renewal 30 days before expiry. A cert inside 30 days with no renewal yet means the renewal window just opened. No alert fatigue risk here; this is a ticket or a dashboard annotation telling you to verify the automation is alive.
TICKET at 7 to 14 days. A cert 14 days from expiry means renewal has been failing for roughly 16 days. At 7 days, it has been failing for about 53. This is a genuine signal that something in the ACME pipeline (challenge reachability, DNS provider credentials, rate limits, acme.json permissions) is broken, but you still cannot tell from the metric alone whether the expiring cert serves real traffic.
PAGE only with an external synthetic probe. The metric cannot tell you which certificate is on the wire for your production hostname. Pair the internal signal with an external TLS probe (blackbox-style) that connects to the real production hostname, completes a handshake, and checks the served chain’s validity and expiry. When the probe confirms the actively-serving certificate for a known production name is about to expire, that is page-worthy. Until then, it is a ticket.

flowchart TD
  A["traefik_tls_certs_not_after series"] --> B{Days remaining?}
  B -->|"> 30d"| C["No action"]
  B -->|"14-30d"| D["PLAN: verify renewal automation"]
  B -->|"7-14d"| E["TICKET: investigate ACME pipeline"]
  B -->|"< 7d"| F{"External probe confirms production hostname serving this cert?"}
  F -->|"Yes"| G["PAGE"]
  F -->|"No / dormant cert"| H["TICKET: clean up store, fix renewal"]

Collapsing the stale serials

Because of the stale-serial bug, group away the serial label before thresholding. The community-standard pattern from Awesome Prometheus Alerts uses min by (instance, sans):

# Critical: any cert under 7 days, collapsing duplicate serials per SAN set
min by (instance, sans) (
  last_over_time(traefik_tls_certs_not_after[5m]) - time()
) / 86400 < 7

# Warning: same expression with a 14-day threshold
min by (instance, sans) (
  last_over_time(traefik_tls_certs_not_after[5m]) - time()
) / 86400 < 14

A caveat worth knowing: min by (sans) takes the earliest expiry among duplicate serials, which means a stale series from a just-renewed cert can still drag the group below threshold for a while. An alternative is max by (instance, sans) (...) on the expiry expression, which selects the largest expiry timestamp—the freshest renewed certificate—in each group. It masks stale serials in the query; it does not delete them. A Traefik restart clears its process-local stale gauge state, but the series can reappear after subsequent renewals if upstream does not change collector cleanup.

Neither workaround fixes staging certs. If you run a shared Traefik for staging and production, split staging onto its own instance so the stores never mix. The certificate metric exposes only cn, sans, and serial; it has no issuer or CA label, so there is no reliable label pattern for filtering “Fake LE Intermediate” certificates.

When the metric lies about the filesystem

Two adjacent failure modes matter for operators who provide certs by file rather than ACME:

On some network filesystems (the reported cases involve GlusterFS and NFS), Traefik does not detect certificate file changes and keeps serving the old cert. The metric reflects the old cert’s expiry even though the new file is on disk. A config reload or restart picks it up. If your certs live on a network mount, verify a rotation actually changes the metric value before relying on it.
acme.json must be readable and writable with mode 600. If permissions change (volume remount, restore from backup), Traefik may silently stop persisting renewals. The metric keeps counting down with no error surfaced anywhere except logs. Traefik hard-fails at startup on a corrupted acme.json, but mid-operation corruption fails silently while in-memory certs keep serving.

Version notes

The Prometheus metric name traefik_tls_certs_not_after is stable across v2 and v3. It first appeared in v2.5.
In v3.5.4, the OpenTelemetry variant was renamed from traefik_tls_certs_not_after_milliseconds to traefik_tls_certs_not_after_seconds to match its actual unit. The migration note documents a rename, not a dual-emission grace period; verify the exact name emitted by your exporter after upgrading.
In HA deployments, each instance reports its own store. If ACME storage is not shared (or the distributed lock in Consul/etcd is stuck because an instance died holding it), instances can show divergent expiry values for the same hostname. Compare the metric per instance, and watch logs for lock acquisition failures.

Signals to correlate

Certificate expiry never exists in isolation. These signals tell you whether “cert expiring” is “renewal automation hiccup” or “production outage in 6 days”:

Signal	Why it matters	Warning sign
`traefik_tls_certs_not_after`	The countdown itself; the only cert metric that exists	Any series < 14 days, or a production-name series < 7 days
Traefik logs (ACME errors)	The only place renewal failure causes appear; there is no failure metric	“Error renewing certificate”, challenge failures, rate limit responses
External synthetic TLS probe	Confirms which cert is actually served on the production hostname	Probe sees the near-expiry or staging-signed chain
`traefik_config_last_reload_success`	A frozen config can mean cert updates are not being applied either	Timestamp not advancing while renewals should be occurring
`traefik_entrypoint_requests_tls_total`	TLS version/cipher distribution; drops can indicate clients rejecting handshakes	Sudden change in handshake mix near expiry
Per-instance cert values (HA)	Detects store divergence and stuck ACME locks	Same hostname showing different `not_after` across replicas

For Let’s Encrypt specifically, keep the rate limits in mind when diagnosing: 50 certificates per registered domain per week, 5 duplicate certificates per week. Hitting the duplicate limit is a common cause of “renewal silently stopped” in crash-looping or HA-without-shared-storage deployments, and the metric alone will not tell you that is what happened.

How Netdata helps

Netdata charts traefik_tls_certs_not_after per certificate with its cn/sans/serial labels, so you can see the full store at a glance and spot dormant or duplicate-serial series before they page you.
Per-second collection catches the moment a renewal lands: a new series appears with a fresh expiry while the stale one keeps counting down, which is exactly the pattern the alert queries need to handle.
Correlating the cert countdown with traefik_config_last_reload_success on the same dashboard separates “renewal failing” from “config frozen, nothing is being applied,” two causes with very different fixes.
Comparing the metric across HA instances side by side surfaces store divergence and stuck ACME locks without per-instance manual checks.
Pairing Netdata’s internal view with an external synthetic probe of the production hostname closes the gap the metric cannot: knowing whether the expiring cert is the one on the wire.