Weird DynDNS issue

LydolfSleipner · December 3, 2025, 3:11pm

I have this weird issue with DynDNS.

This is an excerpt from my terminal while running host git.mydomain.tld every few seconds.

# 2025-11-20 17:54:38
~$ host  git.mydomain.tld
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 212.x.y.z

# 2025-11-20 17:54:41 
~$ host  git.mydomain.tld
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c

$ cat /etc/resolv.conf
nameserver 168.63.129.16
search .

Obviously, the actual addresses are replaced to avoid doxxing myself.

The point is, it seems this dynamic DNS address for core.dyn.mydomain.tld keeps changing while the actual machine does NOT have the one of those address.

So it works like this:

It core.dyn.mydomain.tld will resolve to 89.a.b.c something like 99% of the time (or more), but then suddenly, it will resolve to 212.x.y.z. Maybe once or maybe a few times, then it will go back to 89.a.b.c.

I have been scratching my head for a long time about this. I have not been able to find the root cause.

I though it was impossible for DynDNS to assign an address that is not the source address of the client.

I would really like to hear from all you experts if you have some ideas on what might be going on and how to troubleshoot it further.

LydolfSleipner · December 3, 2025, 3:13pm

I just did a quick test and was able to catch one of these:

$ while true; do date; host git.mydomain.tld ; sleep 0.5; done
Wed Dec  3 15:09:05 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:05 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:06 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:06 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:07 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:08 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:08 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:09 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 212.x.y.z
Wed Dec  3 15:09:10 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:10 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:11 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:11 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:12 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:13 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:13 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:14 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:14 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:15 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c

See Wed Dec 3 15:09:09 UTC 2025

LydolfSleipner · December 3, 2025, 3:15pm

The reason I care about these one-off errors is that my VCS is running on that address, and these hiccups cause some jobs to occasionally fail.

peter · December 3, 2025, 4:01pm

Without any further information, I would expect a DNS caching issue, in case that other IP address used to be configured for this name.

But, without any further information, it’s not really possible to say more. Please use your real domain, or contact support if you don’t want to do that in public.

Stay secure,
Peter

fiwswe · December 4, 2025, 2:28am

Another thing to check would be the URL used for your DynDNS updates. Does it rely on implicit determination of your IPv4 address or does it pass the address explicitly in myip, myipv4 or ip? If the latter, from where does the IP get determined?

Maybe check the logs of your DynDNS client.

See Determine IP Address(es)

Basically you may be dealing with:

A bad source for the IP and your client using that to set the A RRset;
A problem resolving the correct IP (caching issue or some DNS forwarder injecting a wrong answer if DNSSEC is not validated);
Another DynDNS client updating the A RRset (either due to a mistake or maliciously);
Something weird like your host actually getting these different IPs through DHCP.

HTH
fiwswe

LydolfSleipner · December 4, 2025, 11:04am

Thank you for your response and for helping me investigate this mystery!

I simply run this at regular intervals:

curl --ipv4 "https://update.dedyn.io/?hostname=${hostname}" --header "Authorization: Token ${token_usage}"
# sleep to avoid rate limiting
sleep 60;
curl "https://update6.dedyn.io/?hostname=${hostname}&myipv4=preserve" --header "Authorization: Token ${token_usage}"

But I do not think this problem is caused by the client setting the IP Address wrong. Because look at the timings:

LydolfSleipner:

Wed Dec  3 15:09:08 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c
Wed Dec  3 15:09:09 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 212.x.y.z
Wed Dec  3 15:09:10 UTC 2025
git.mydomain.tld is an alias for core.mydomain.tld.
core.mydomain.tld is an alias for core.dyn.mydomain.tld.
core.dyn.mydomain.tld has address 89.a.b.c

The wrong IP is resolved in literally less than 2 seconds.

I am not even sure it is possible with desec dyndns to change the IP address of a dyndns record and then change it back so fast due to rate limiting.

I think I can eliminate #3 on your list due to there not being anyone else having access to my token credentials.

I did another experiment. I let a job do periodic DNS lookups over time.

During that time, it resolved to the correct address 45366 times and to the wrong address 136 times. So it fails at a rate of about 0.3 %

I also learned that when it resolved to the wrong address, it is always the same wrong address.

My dyndns client is on a residential ISP.

The correct and incorrect IP Addresses belong to the same residential ISP. (now, this is part of the biggest ISP in the country, so this might be a coincidence)

KapernMagIchNicht · December 4, 2025, 2:28pm

Could be CG-NAT.
What ISP do you have?

Also if you want to use a more “intelligent” solution than just running these two curls, here is my shell script so you don’t request an update every minute:

GenericUser · December 4, 2025, 3:59pm

Not likely. The records don’t change from the right address to the wrong address in one second and back to the right address in the next second. This looks like a resolver issue, probably some round-robin resolver cluster where one resolver has outdated data. I’d try to get more verbose output to see information about the server which answered your query. If these are the ISP’s resolvers, I’d also try other resolvers to see if the issue can be replicated then. I don’t think this has anything to do with Desec at all, because no resolver would query the authoritative servers that often.

LydolfSleipner · December 4, 2025, 4:12pm

I do not have CGNAT.

ISP is Hiper (part of Danish major telco TDC)

LydolfSleipner · December 4, 2025, 4:14pm

Yeah, it is weird. The strange thing is: I have experienced this with multiple different resolvers. So as can be seen above, this server uses 168.63.129.16 which is owned by Microsoft (Azure).

I have another app running at a completely hosting provider that also experiences this.

fiwswe · December 4, 2025, 4:40pm

After an update it will take a bit for the new IPs to propagate to the deSEC frontend servers. (Currently this delay is fairly long, but deSEC seems to be working on mitigations.) Then there is the TTL of 60. So a valid test should occur no sooner than ≈3 min. after the update. Testing every second is bound to get strange results.

Theoretically it is. The limit is 2/120s. But TTLs and propagation delays will get in the way.

The rest of your observations seem to point to a DNS caching issue, more than anything else. You are using a Microsoft Azure DNS resolver apparently. Have you tried either using Google (8.8.8.8)/Cloudflare (1.1.1.1) or hosting your own DNSSEC validating resolver, such as unbound? The latter has the advantage that you can be sure that the answers you get are DNSSEC secured, whereas any remote resolvers involve a bit of risk that their answers could have been manipulated on the way to your client, even if you trust the resolver itself.

LydolfSleipner · December 4, 2025, 5:51pm

I know. I am testing every second to provoke the issue.

The rest of your observations seem to point to a DNS caching issue, more than anything else. You are using a Microsoft Azure DNS resolver apparently. Have you tried either using Google (8.8.8.8)/Cloudflare (1.1.1.1) or hosting your own DNSSEC validating resolver, such as unbound? The latter has the advantage that you can be sure that the answers you get are DNSSEC secured, whereas any remote resolvers involve a bit of risk that their answers could have been manipulated on the way to your client, even if you trust the resolver itself.

I just made a test using 8.8.8.8 as DNS server. I just did 5 manual runs in my terminal, and 2 of them returned the wrong IP address.

If this is some sort of round robin issue, wouldn’t I see all the addresses in the RR? Just the order would change?

I then tried using 1.1.1.1 and did a random 10 runs in the terminal, all resolved to correct address.

I then went back and tried 8.8.8.8 and it returned the wrong address the first two times I tried! Then went back to returning the correct address.

fiwswe · December 4, 2025, 6:13pm

Hmm, note that I am only recommending this for a short test to keep server load down, but what results do the official authoritative NS yield?

I.e.: dig +short @ns1.desec.io yourhostname a and dig +short @ns2.desec.org yourhostname a

This test would cut out any strange effects caused by the DNS infrastructures MS/Google/Cloudflare are using.

LydolfSleipner · December 4, 2025, 7:15pm

OMG! This is gold!

When I run this command against ns1, I get the right result every time. When I run it against ns2.desec.io, I get the wrong result every time.

So I think we are very close to the root cause here!

I have no idea why this happens though?

GenericUser · December 4, 2025, 7:27pm

Please notify Desec support and tell them the affected domain. The servers should have the same zone data. Any difference that isn’t just temporary when records are being changed is an error. (Note though that the servers are ns1.desec.io and ns2.desec.org, not ns2.desec.io. Make sure that was just a typo in your comment.)

LydolfSleipner · December 9, 2025, 6:32pm

OK, så I was in contact with Desec support whom were extremely helpful and quick to respond!

I will try my best to summarize the issue.

At some point several years back, I actually had the 212.* IP address…

At that time, I added a core.dyn A record on my domain.

And I also set up the DDNS.

Now, this is a kinda broken config. But it worked because they were both pointing to the same IP address.

The core.dyn record should never be returned, because the DDNS configures a more specific record which should be returned instead.

But since the two records pointed to the same IP address, this never gave me any problems.

Later I moved and was no longer on that IP address.

Then this started to become a problem.

It appears that ns1 and ns2 uses two different DNS server software. ns2 uses Knot DNS.

Not this situation seems to have hit a quite unusual edge case and revealed a bug in Knot DNS.

I am not knowledgeable enough on DNS to explain the bug correctly.

Anyway, I believe the bug has now been fixed.

I have subsequently removed the unnecessary record as well.

Thank you all for helping!