nsd users - Jan 2020 - Seeking advice for deploying an anycast cluster

First, the advice from Ond?ej Caletka is worth heeding.

On 3 Jan 2020, at 17:04, Daniel Corbe via nsd-users wrote:
> The main issue I'm running into is I want to keep the primary's
> interface to the world as simple as possible.
Fair enough.
> At maximum, two hosts to communicate with.
If this is a limit you have freely chosen, I suspect you may be
importing inapplicable experience from another problem domain,
and would advise reviewing your choice.

NSD is well capable of, and easily configured for, handling much
greater fanout than two. In case the anycast cloud is really
enormous, it may be sensible to use some intermediate distribution
servers, as described below using the term "hidden slave".
> So for that to work, I'd need to somehow
> cluster my NSD instances together or I'd need some sort of proxy
> server that can listen for incoming NOTIFYs and then distribute them
> to the rest of the constellation.
Any of the authoritative name-server codes is designed to do exactly
this kind of proxying. The old-fashioned terminology for a server
which does only this, and which is not announced to the world in an
NS record, is "hidden slave". Setting one of these up is particularly
easy to do with (perhaps another instance of) NSD. Other choices might
be BIND (named), Knot, or PowerDNS.

An NSD-based hidden slave needs to refer to the upstream master
in 'allow-notify' and 'request-xfr' configuration directives and
to
refer to each dependent downstream server in 'notify' and
'provide-xfr'
directives. Of course, the downstream servers must be configured
correspondingly to accept NOTIFY and request a zone transfer when
appropriate.
> I don't think any of the usual
> suspects (nginx, haproxy, etc) have that capability out of the box.
> Do they?
I don't know: modules of all kinds abound; but it would be really
extraordinary to use such a web proxy as part of a DNS infrastructure.
> On Fri, Jan 3, 2020 at 3:49 AM Ond?ej Caletka via nsd-users
> <nsd-users at lists.nlnetlabs.nl> wrote:
  [...]
>>
>> This is usually solved by sending NOTIFY messages not to the 
>> anycasted
>> cluster address but to all the unique address of each cluster node 
>> instead.
I may have mentioned already that this advice from Ond?ej Caletka
is worth heeding.

Niall O'Reilly

Thank you for taking the effort to type this up.   I think you've hit
on a solution here.   At least I believe you've got me thinking in the
right direction now.

On Fri, Jan 3, 2020 at 3:32 PM Niall O'Reilly <niall.oreilly at
ucd.ie> wrote:
> NSD is well capable of, and easily configured for, handling much
> greater fanout than two. In case the anycast cloud is really
> enormous, it may be sensible to use some intermediate distribution
> servers, as described below using the term "hidden slave".
It is.  38 servers and counting.  Spread out over the globe.
>
> > So for that to work, I'd need to somehow
> > cluster my NSD instances together or I'd need some sort of proxy
> > server that can listen for incoming NOTIFYs and then distribute them
> > to the rest of the constellation.
>
> Any of the authoritative name-server codes is designed to do exactly
> this kind of proxying. The old-fashioned terminology for a server
> which does only this, and which is not announced to the world in an
> NS record, is "hidden slave". Setting one of these up is
particularly
> easy to do with (perhaps another instance of) NSD. Other choices might
> be BIND (named), Knot, or PowerDNS.
I've been thinking about using PowerDNS anyways to front a web
interface for my customers.  Seems like the shortest path to market
considering it supports pgsql out of the box.
>
> An NSD-based hidden slave needs to refer to the upstream master
> in 'allow-notify' and 'request-xfr' configuration
directives and to
> refer to each dependent downstream server in 'notify' and
'provide-xfr'
> directives. Of course, the downstream servers must be configured
> correspondingly to accept NOTIFY and request a zone transfer when
> appropriate.
So I'm worried about nsd getting the right information from the right
source.   If I put my nsd instances in a "full mesh" so to speak,
where they are all XFR and NOTIFY each other, my "A" and "B"
PowerDNS
servers would still need to be the authoritative source for
information.  I'm assuming that's where the serial number of the zone
comes into play?   Highest provided serial number wins, right?