Well done!
That all looks quite sensible. Perhaps write it up as a wiki page?
Glad things worked out OK. The --rpc-server-ip bit will force updates
over a protocol that is less reliant on DNS working, and that will help
a lot (which is why it was implemented, it is also in there
automatically as a fallback).
Andrew Bartlett
On Tue, 2021-06-22 at 09:43 +0000, Chris Puttick via samba
wrote:> Ok, so we're not sure whether running the samba_upgradedns was part
> of the fix, but for those who encounter similar scenarios, these
> steps did seem to resolve it (over time):
>
> Recreate the zone with
>
> samba-tool dns zonecreate A-DC domain.local -U admin.username
>
> (where in this organisation's case A-DC was what we think of in old-
> fashioned terms as the PDC e.g. has the FSMO roles - may or may not
> be relevant)
>
> The zone starts to populate itself fairly quickly with stuff from the
> LAN.
>
> On the DCs restart samba
>
> (service samba-ad-dc restart)
>
> That gets the zone recreated on the DCs, and a little while later the
> DCs appear in as records in the zone, checking with
>
> samba-tool dns query this-DC domain.local @ ALL -U admin.username
>
> Somehow or other, probably with some variant of samba_dnsupdate we
> managed to get one of the remote DCs A record to get updated more
> quickly. Waiting (impatiently) for the others took an age (maybe 15
> minutes ;) ) to appear.
>
> The one that got forced had the command run *before* samba was
> restarted, with the form that resulted in most apparent activity (but
> without the domain zone being recreated on the DC) including said DC
> appearing as a record.
>
> samba_dnsupdate --verbose --rpc-server-ip ip.of.a-.dc --all-names
>
> Again noting this may or may not be a factor, as other DCs which
> didn't have any samba_dnsupdate commands run eventually appeared in a
> seemingly random length of time (where a DC on each remote site was
> shutdown for safety, and booted up about the same time, with 10+
> minutes difference in when their A records got created.
>
> So while the idea that the AD zone should not be deletable without
> some complex, preferably CLI only, process is a great one, recovery
> from the deletion seems relatively painless.
>
> Cheers
>
> Chris
>
> ----- Original Message -----
> From: "Andrew Bartlett" <abartlet at samba.org>
> To: "Chris Puttick" <chris.puttick at cp1associates.net>
> Cc: "samba" <samba at lists.samba.org>
> Sent: Tuesday, 22 June, 2021 08:40:45
> Subject: Re: [Samba] Accidental zone deletion
>
> On Tue, 2021-06-22 at 07:16 +0000, Chris Puttick via samba wrote:
> > Thanks for the response, about what we feared. For interest on one
> > of
> > the DCs we tried
> >
> > samba_upgradedns --dns-backend=SAMBA_INTERNAL --migrate=no
> >
> > and got the response
> >
> > # samba_upgradedns --dns-backend=SAMBA_INTERNAL --migrate=no
> > Reading domain information
> > DNS accounts already exist
> > No zone file /var/lib/samba/private/dns/OXARCH.LOCAL.zone
> > DNS records will be automatically created
> > DNS partitions already exist
> > Finished upgrading DNS
>
> This may have rebuilt a very minimal DNS, which may be a good thing.
>
> > Found the comment about "no zone file" interesting because
that
> > directory doesn't exist and AFAIK has never existed (Ubuntu 18.04
> > running Samba 4.7.6-Ubuntu).
>
> Correct. That tool does a number of different things, but one thing
> it
> was built for originally was upgrading from a file-based zone into
> the
> in-directory zone, hence that check and message.
>
> Andrew Bartlett
>
>
> > ----- Original Message -----
> > From: "Andrew Bartlett" <abartlet at samba.org>
> > To: "Chris Puttick" <chris.puttick at
cp1associates.net>, "samba" <
> > samba at lists.samba.org>
> > Sent: Tuesday, 22 June, 2021 06:47:29
> > Subject: Re: [Samba] Accidental zone deletion
> >
> > On Tue, 2021-06-22 at 05:29 +0000, Chris Puttick via samba wrote:
> > > Hi
> > >
> > > We have a situation where an MS admin used the AD utilities to
> > > tidy
> > > up an neighbouring (MS-based) domain but was attached to the
> > > wrong
> > > DC
> > > and deleted the wrongdomain.local zone file (which is apparently
> > > a
> > > bit of a thing in MS circles); by the time said admin realised
> > > the
> > > deletion had replicated across DCs on all sites. How do we
> > > recreate
> > > it, in partiular the contents? Hoping the answer is "just
> > > manually
> > > create the zone and it'll repopulate".
> > >
> > > Any suggestions welcomed...
> >
> > I assume of course you mean the zone in a Samba AD DC, not a simple
> > .zone file.
> >
> > This has happened, and yes, I do think we should prevent it at the
> > database level, as nobody ever really means to do that. Last time
> > that
> > happened we helped a client jury-rig up a backup of the sam.ldb
> > into
> > BIND9-DLZ (so only DNS used the old data), allowing service to
> > somewhat
> > continue while things were fixed back up.
> >
> > However, I'm sorry to say it won't just be regenerated, while
Samba
> > will try and re-register itself every now and then, I wouldn't
> > count
> > on
> > it getting back the way you found it fast.
> >
> > How are your backups?
> >
> > Andrew Bartlett
> >
> > --
> > Andrew Bartlett (he/him) https://samba.org/~abartlet/
> > Samba Team Member (since 2001) https://samba.org
> > Samba Team Lead, Catalyst IT
> > https://catalyst.net.nz/services/samba
> >
> > Samba Development and Support, Catalyst IT - Expert Open Source
> > Solutions
> >
> --
> Andrew Bartlett (he/him) https://samba.org/~abartlet/
> Samba Team Member (since 2001) https://samba.org
> Samba Team Lead, Catalyst IT https://catalyst.net.nz/services/samba
>
> Samba Development and Support, Catalyst IT - Expert Open Source
> Solutions
>
--
Andrew Bartlett (he/him) https://samba.org/~abartlet/
Samba Team Member (since 2001) https://samba.org
Samba Team Lead, Catalyst IT https://catalyst.net.nz/services/samba
Samba Development and Support, Catalyst IT - Expert Open Source
Solutions