A broken DNSSEC rollover took down .AL. Now 1.1.1.1 tells you when validation is bypassed
A failed DNSSEC key rollover at Albania’s `.AL` registry caused validating resolvers to return errors for every `.AL` domain. Cloudflare temporarily installed a Negative Trust Anchor (NTA) to restore access, suspending DNSSEC validation while the registry fixed the issue. To make this bypass visible, 1.1.1.1 began returning a new Extended DNS Error (EDE) code alongside affected responses. ## What Happened to `.AL` - Around 14:15 UTC on July 3, the registry published a new DNSKEY and stopped serving the old key. - The root zone’s DS record still referenced the old key (`id=26319`), breaking the DNSSEC chain of trust. - Around 17:00 UTC, the new DNSKEY was also removed, leaving `.AL` with no DNSKEY records. - At approximately 19:15 UTC, the DS record was removed from the root zone, restoring normal resolution but leaving the entire TLD unsigned. - The outage affected government services, banks, media, and any other `.AL` domain accessed through DNSSEC-validating resolvers. ## Why Negative Trust Anchors Were Used - An NTA, defined by RFC 7646, tells recursive resolvers to treat a zone as unsigned and skip DNSSEC validation. - Cloudflare applied an NTA to `.AL` at 17:15 UTC, restoring resolution for 1.1.1.1 users. - The measure was considered acceptable because the failure was public, confirmed, and affected validating resolvers broadly. - Communication with the registry was difficult because its contact addresses were themselves hosted under `.AL`. - The NTA was removed the following day after the DS record had been removed from the root zone. ## The Security Tradeoff - NTAs prevent widespread `SERVFAIL` responses but remove cryptographic protection against DNS spoofing. - Previously, clients could not distinguish an NTA-served response from a fully DNSSEC-validated response. - Public status pages provide disclosure, but applications, monitoring systems, and users cannot reliably discover the bypass from DNS responses alone. ## EDE-Based Transparency - Extended DNS Errors, defined in RFC 8914, let resolvers attach explanatory information to DNS responses. - Cloudflare implemented a proposed EDE code, “Negative Trust Anchor,” developed with Quad9 contributors. - During the incident, `.AL` responses included: - EDE 9: `DNSKEY Missing`, explaining the broken DNSSEC chain. - EDE 33: `Negative Trust Anchor`, explicitly stating that validation had been bypassed. - A query such as `google.al` could therefore return a successful answer while clearly indicating that it was not DNSSEC-validated. Resolvers and applications should use EDE information where possible, while DNS operators should disclose NTAs and remove them promptly once the underlying DNSSEC problem is resolved.
Read original(opens in new tab)