← Go back to NixNet Status

DNS Postmortem

November 2, 2021 at 11:18 AM UTC

BigBlueButton IRC Misskey Mumble Pleroma XMPP Etherpad Framadate Hedgedoc Mobilizon Bibliogram Nitter RSS-Bridge TinyTinyRSS fosspay Gitea linx MediaWiki PrivateBin searx Wallabag xBrowserSync

Resolved after 14h 18m of downtime. November 3, 2021 at 1:37 AM UTC

At 11:00 UTC on November 2nd, I became aware of issues with DNS resolution. These issues were with our authoritative DNS servers which have been run by a third party for the last 2 years. This is the second time their service has failed in a catastrophic way and seriously affected our users. At the time of writing, their issues are still not fixed.

We have therefore chosen to migrate to our own authoritative DNS servers — a process that took a considerable amount of time.

To help mitigate the chances of such a failure occurring again, we have provisioned an additional two authoritative DNS servers to act as secondaries. We are also coordinating with the admins of snopyta.org and linux.pizza to create a “DNS ring”. Should all of our DNS servers go down, theirs will act as further fallbacks.

Note that the DNS servers I just set up are not going to be part of NixNet. They are paid for and operated by Alder IT, a for-profit version of NixNet I’m in the process of creating. This will not affect the free nature of NixNet in any way, however; NixNet will continue running exactly as it has for the last 3 years. Alder IT will be providing authoritative DNS to NixNet for free.

We apologise for any disruptions. As always, let us know if there are further issues!

Amolith

Last updated: November 3, 2021 at 3:13 AM UTC