Hetzner DNS unable to pass LetsEncrypt validation?

[g-a-c]

I recently moved some domains to Hetzner DNS, reconfigured my Traefik instance (which uses lego internally) with the new key and restarted. Today, it was unable to renew the certificate, which has now expired.

• mydomain.com is my domain
• I am requesting a certificate for h.mydomain.com with a SAN of *.h.mydomain.com
• I can see two TXT records being created at _acme-challenge.h.mydomain.com in Hetzner's DNS control panel so the API token is valid
• When this happens, I can do a dig @1.1.1.1 -t TXT _acme-challenge.h.mydomain.com and get two results (as expected, for the two names)
• However the lego binary is not working correctly and simply spits out

2024/06/12 17:59:09 [INFO] [h.mydomain.com] acme: Waiting for DNS record propagation.
2024/06/12 17:59:19 [INFO] [h.mydomain.com] acme: Waiting for DNS record propagation.
2024/06/12 17:59:29 [INFO] [h.mydomain.com] acme: Waiting for DNS record propagation.
2024/06/12 17:59:39 [INFO] [h.mydomain.com] acme: Cleaning DNS-01 challenge
2024/06/12 17:59:41 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/<removed>
2024/06/12 17:59:41 Could not obtain certificates:
	error: one or more domains had a problem:
[h.mydomain.com] propagation: time limit exceeded: last error: [zone=mydomain.com.] could not determine authoritative nameservers

The authoritative nameservers are correctly configured for mydomain.com at my registrar (verified with dig as above, and with https://dnsviz.net), I do have some internal DNS resolution set up on my router for name resolution of internal services, so when using Lego and Traefik I set the --dns.resolvers 1.1.1.1:53 option to make Lego check the external view of DNS for this domain. This "split view" DNS has been in place for years and never seemed to cause a problem previously so I don't think it's to blame here.

The OPNsense firewall also has its own certificate for the same name/SAN (so that its web UI is TLS-protected internally), the OPNsense implementation uses acme.sh and this was able to successfully renew its certificate using the same Hetzner API token and the same zone/CN/SAN properties on the certificate.

I'd love to know if other people have seen the same issue and have a solution - so far the only other discussion I've found was someone who was NAT-redirecting all external DNS queries back to their pfSense firewall - I do not have one of these rules, so anything which explicitly queries an external DNS server is able to. I've verified using tcpdump that I see these queries being sent to (and responded to by) the correct server (in this case Cloudflare's public 1.1.1.1 resolver) on the WAN interface.

My environment:
Traefik version: 2.11 and 3.0.2
Lego version: 4.17.3 (installed from source with go install github.com/go-acme/lego/v4/lego/[email protected]
Hetzner DNS

[g-a-c]

Never mind, I may have solved my own problem.

There are NS records in the parent zone for mydomain.com, which means DNS resolution is working. There were not NS records inside the mydomain.com zone, referring to its own nameservers. I can only assume that my previous provider added these records silently, where Hetzner do not. I can only also assume that acme.sh doesn't care about what the authoritative nameservers are, and skips this check.

Adding these records has allowed me to retrieve a LetsEncrypt staging certificate successfully, so I suspect retrieving a real one will also work when I've removed the staging config.