samba - Aug 2016 - Horrible BIND9_DLZ DNS breakage after DC replaced and samba-tool domain demote --remove-other-dead-server

On Mon, 15 Aug 2016 16:02:38 +0100
Rowland Penny via samba <samba at lists.samba.org> wrote:
> 
> OK, this has nothing to do with the classicupgrade, I have setup a
> couple of VMs and provisioned a test DC in one and joined another DC
> in the other.
> 
> I am now at the point the OP is at, samba_dnsupdate cannot add the
> required records, all I get in log.samba is this multiple times:
> 
> [2016/08/15 15:57:23.949917,
> 0] ../lib/util/util_runcmd.c:328(samba_runcmd_io_handler)
/usr/local/samba/sbin/samba_dnsupdate:
> update failed: NOTAUTH
> 
> and it ends with this:
> 
> [2016/08/15 15:57:23.975421,
> 0] ../source4/dsdb/dns/dns_update.c:295(dnsupdate_nameupdate_done)
../source4/dsdb/dns/dns_update.c:295:
> Failed DNS update - NT_STATUS_UNSUCCESSFUL
> 
> Now to try and find the cause and fix it.
> 
> Rowland
> 
OK, I think I have sorted this, I added some lines to samba_dnsupdate
to print out why it didn't work and got this:

Could not obtain Kerberos ticket for DNS/devdc1.example.com as DEVDC2$
response to GSS-TSIG query was unsuccessful
....
response to GSS-TSIG query was unsuccessful
Failed update of 24 entries

So, I thought, no SOA records for DEVDC2

Added them:

samba-tool dns add 127.0.0.1 example.com devdc2 A 192.168.0.251 -Uadministrator

samba-tool dns add 127.0.0.1 example.com @ NS devdc2.example.com -Uadministrator

samba-tool dns add 127.0.0.1 _msdcs.example.com @ NS devdc2.example.com
-Uadministrator

and then ran samba_dnsupdate again and this time it didn't print
anything, so I tried this:

root at devdc2:~# host -t SRV _ldap._tcp.example.com.

and got this:

_ldap._tcp.example.com has SRV record 0 100 389 devdc1.example.com.
_ldap._tcp.example.com has SRV record 0 100 389 devdc2.example.com.

I think all the records are now there.

So, as the OP said, this is a bit of a chicken and egg situation, you
need the SOA records to add the SOA records via samba_dnsupdate.

Rowland

On Mon, 15 Aug 2016 19:59:56 +0100
Rowland Penny via samba <samba at lists.samba.org> wrote:
> On Mon, 15 Aug 2016 16:02:38 +0100
> Rowland Penny via samba <samba at lists.samba.org> wrote:
> 
> 
> So, as the OP said, this is a bit of a chicken and egg situation, you
> need the SOA records to add the SOA records via samba_dnsupdate.
> 
> Rowland
> 
> 
And after further testing, but this time using the internal DNS server,
the problem doesn't exist, so it is a 'using Bind9 with Samba
problem'

Rowland

On Tue, 16 Aug 2016 09:20:56 +0100
Rowland Penny via samba <samba at lists.samba.org> wrote:
> On Mon, 15 Aug 2016 19:59:56 +0100
> Rowland Penny via samba <samba at lists.samba.org> wrote:
> 
> > On Mon, 15 Aug 2016 16:02:38 +0100
> > Rowland Penny via samba <samba at lists.samba.org> wrote:
> > 
> > 
> > So, as the OP said, this is a bit of a chicken and egg situation,
> > you need the SOA records to add the SOA records via samba_dnsupdate.
> > 
> > Rowland
> > 
> > 
> 
> And after further testing, but this time using the internal DNS
> server, the problem doesn't exist, so it is a 'using Bind9 with
Samba
> problem'
> 
> Rowland
> 
After much further testing, I 'think' I have the magic incantation to
get this working ;-)

Install samba and Bind9 as normal on the second DC.
Edit /etc/resolv.conf so that the nameserver points to the first DC. 
Now join the computer as a DC, once the join is finalised and before
you start bind9 or Samba, edit /etc/resolv.conf again, but this time,
point the nameserver at the new DCs ipaddress or 127.0.0.1 i.e. itself.

Start bind9 and then samba, this should run samba_dnsupdate and add all
the missing records. You can check this with:

host -t SRV _ldap._tcp.example.com.

You should get a result similar to this:

_ldap._tcp.example.com has SRV record 0 100 389 devdc1.example.com.
_ldap._tcp.example.com has SRV record 0 100 389 devdc2.example.com.

edit /etc/resolv.conf on both DCs to use the other as a nameserver and
then itself:

DC1:

search example.com
nameserver 192.168.0.251
nameserver 127.0.0.1

DC2:

search example.com
nameserver 192.168.0.250
nameserver 127.0.0.1

Finally, restart samba on both DCs

Rowland

Hai eveyone.

 

I know about the dns "things" in the past. DNS Islanding problems etc.

This one is a bit hijacking the subject :

“Horrible BIND9_DLZ DNS breakage after DC replaced and samba-tool domain demote
--remove-other-dead-server”

 

 

I would like to suggest a smale change in how we suggest to setup samba ADDC dns
things,

and i do think this help in the setup of the AD DC, and reduce change on errors.

 

So this is what i suggest, and i explain why, so yeah.. long email again, sorry
about that.

 

The loopback address ip should be configured only as a secondary or tertiary DNS
server on a domain controller.

but in my opionion should be avoided in all times. 

I’ll address 2 things here. Resolving (orders) and ipv4/ipv6 preferences.

 

---------------------

In a single ADDC server setup, resolv.conf suggestions. 

 

search ad-dc-subdom.domain.tld ( and maybe others to search.) 

nameserver IP_OF_DC_AND_NOT_127.0.0.1

 

Only now a localhost ip is optional here but i dont suggest it, 

when you later add a DC and you move the FSMO roles, this can a problem. 

 

Why, simple we forget to change it when needed if we add a dc, 

or change FSMO roles to other servers. 

At least this happens, you reboot and you have a dns problem. 

 

 

---------------------

In a 2 server ADDC server setup

First Server. ( ADDC with fsmo roles and primary dns zones )

 

search ad-dc-subdom.domain.tld ( and maybe others to search.) 

nameserver IP_OF_DC1_AND_NOT_127.0.0.1

( and later (optional) add DC2 ip. ) 

 

DONT CHANGE THE ORDER HERE. First DC1 then DC2. 

Note : any server should always resolv first to the ADDC dns which contains 

domain controller locator CNAME record for all the other domain controllers in
the root.

 

Second ADDC Server.

search ad-dc-subdom.domain.tld ( and maybe others to search.) 

nameserver IP_OF_DC1_AND_NOT_127.0.0.1

nameserver IP_OF_DC2_AND_NOT_127.0.0.1

 

---------------------

In a 3 DC server setup, or more. 

First Server. ( primary with fsmo roles )

search ad-dc-subdom.domain.tld ( and maybe others to search.) 

nameserver IP_OF_DC1_AND_NOT_127.0.0.1

( optional add DC2 and/or DC3 IP) 

 

Second ADDC Server.

search ad-dc-subdom.domain.tld ( and maybe others to search.) 

nameserver IP_OF_DC1_AND_NOT_127.0.0.1

nameserver IP_OF_DC3_AND_NOT_127.0.0.1

(optional nameserver IP_OF_THIS_ADDC_OR_127.0.0.1)

 

Third ADDC Server.

search ad-dc-subdom.domain.tld ( and maybe others to search.) 

nameserver IP_OF_DC1

nameserver IP_OF_DC2

(optional nameserver IP_OF_THIS_ADDC_OR_127.0.0.1)

 

IF you have the room for it, 3 DC setup is the best. 

For the clients, point to DC2 and DC3, or depending on load of the servers.

 

And for all servers above, NEVER add the own ip of a ADDC AND 127.0.0.1 in
resolv.conf.

But that should be obvious. 

 

---------------------------------

Since MS is change-ing a lot in security and i see lots it pointing to FQDN 

and not single names like it used to before, so looks to me using ip/hostname
with FQDN, more correct, better resolving, less problems in the future.

Latest security fixed, badlock things, GPO security fixes changed a lot to FQDN
for authentication things (etc).

 

 

And i think this is one of the best tips for today..

Also setup what you preffer IPV4 over IPV6, etc, the clients (win7 and win10) 

ALWAYS prefferer ipv6 over ipv4. thanks to MS. 

So i can suggest setup a COMPUTER GPO and setup your preferences for the resolve
order.

I disabled all IPv6 components on my clients since i dont use it in my lan. 

Look here howto setup.  ( preffered ) 

http://social.technet.microsoft.com/wiki/contents/articles/5927.how-to-disable-ipv6-through-group-policy.aspx

 

Or use : https://support.microsoft.com/en-us/kb/929852 

 

Last to know, above avoids DNS islanding in all cases. 

 

Tell us your thoughts.... 

 

Greetz, 

 

Louis

 

p.s. 

 

source reverals : 

https://technet.microsoft.com/en-us/library/ff807362(v=ws.10).aspx 

https://support.microsoft.com/en-us/kb/275278 

http://social.technet.microsoft.com/wiki/contents/articles/5927.how-to-disable-ipv6-through-group-policy.aspx