samba - Sep 2016 - dnsupdate_nameupdate_done - Failed DNS update

Thank you Denis and Rowland - I didn't realise this was the script, makes
sense now.

I've run it (on dc2) and it gets as far as:

need update: SRV _ldap._tcp.mysite._sites.ForestDnsZones.mydomain.org.uk
dc2.mydomain.org.uk 389
[lots of updates needed]
10 DNS updates and 0 DNS deletes needed
Successfully obtained Kerberos ticket to DNS/dc1.mydomain.org.uk as DC2$

and then it fails here:
update(nsupdate): SRV _ldap._tcp.mysite._
sites.ForestDnsZones.mydomain.org.uk dc2.mydomain.org.uk 389
Calling nsupdate for SRV _ldap._tcp.mysite._
sites.ForestDnsZones.mydomain.org.uk dc2.mydomain.org.uk 389 (add)
Failed nsupdate: SRV _ldap._tcp.mysite._sites.ForestDnsZones.mydomain.org.uk
dc2.mydomain.org.uk 389 : [Errno 2] No such file or directory

which I assume is due to a needed file not being in $PATH, or similar.

I'll have a poke about using strace and see if I can spot exactly what
it's
trying to run, and where it might be..

Thanks, both - I'm much further forward now!

Cheers

Jonathan

On 23 September 2016 at 16:43, Denis Cardon <
denis.cardon at tranquil-it-systems.fr> wrote:
> Hi Jonathan,
>
> All 3 of my DCs regularly display an error in syslog almost exactly every
>> 10 minutes. They have been doing this for quite some time, and I have
so
>> far ignored the message as everything else DNS-wise seemed to mostly be
>> working - but I figured it was worth getting to the bottom of it if I
can.
>> So this isn't new at all but rather something that has been present
for
>> some time.
>>
>> I am using the internal Samba DNS server, currently with Samba 4.5.0.
The
>> message is as follows, every 10 minutes (I have pasted in from all 3
DCs
>> here):
>>
>> Sep 23 13:03:54 dc1 samba[13117]: [2016/09/23 13:03:54.867360,  0]
>> ../source4/dsdb/dns/dns_update.c:290(dnsupdate_nameupdate_done)
>> Sep 23 13:03:54 dc1 samba[13117]:  
../source4/dsdb/dns/dns_update.c:290:
>> Failed DNS update - with error code 5
>>
>
> could you please try to run samba_dnsupdate --verbose from the command
> line on your three DC. When a DC starts, it will try to update its own DNS
> fields, but it your DNS zones is missing some stuff, it may not be able to
> do it.
>
> Cheers,
>
> Denis
>
>
>
>> Sep 23 13:00:11 dc2 samba[901]: [2016/09/23 13:00:11.584679,  0]
>> ../source4/dsdb/dns/dns_update.c:290(dnsupdate_nameupdate_done)
>> Sep 23 13:00:11 dc2 samba[901]:   ../source4/dsdb/dns/dns_update.c:290:
>> Failed DNS update - with error code 10
>>
>> Sep 23 13:05:28 dc3 samba[897]: [2016/09/23 13:05:28.800364,  0]
>> ../source4/dsdb/dns/dns_update.c:290(dnsupdate_nameupdate_done)
>> Sep 23 13:05:28 dc3 samba[897]:   ../source4/dsdb/dns/dns_update.c:290:
>> Failed DNS update - with error code 1
>>
>> The precise error codes vary (I have had 1, 6, 10, 110 recently) but I
do
>> get some sort of message every 10 minutes, and the error code usually
>> stays
>> the same on a particular DC. If it makes a difference, DC1 and DC2 are
in
>> site A, and DC3 is at site B, there is full connectivity between them
all
>> (or at least, there should be).
>>
>> I've tried tcpdump and wireshark to figure out what's going on,
but I
>> can't
>> seem to spot any form of DNS request coming in that would be an update.
>> The
>> most I can see via tcpdump at any time I've looked are some queries
that
>> return NXDOMAIN - e.g. there are frequent ones from an VMWare ESXi
server
>> querying for _kerberos-master.udp.MYDOMAIN.ORG.UK as per
>> https://communities.vmware.com/thread/491621 and getting NXDOMAIN - but
I
>> wouldn't have thought that these queries would constitute a
"DNS update"
>> that would fail?
>>
>> My debugging method so far has been to run tcpdump against port 53 -
but
>> either I am somehow managing to not see the failing DNS packet when I
look
>> at the results, or the DNS update arrives at the DC some other way.
>> Looking
>> at the code in dns_update.c it looks like there may be some form of
>> regular
>> DNS check, that is failing in my case?
>>
>> Does anybody know
>>   - if I can turn debugging on for just this DNS functionality? I
expect
>> the log file here to be massive as a DC is also a DNS server.. but
>> hopefully that will give me more of a clue as to what
"update" is failing?
>>   - if there is some other way I might be able to capture / check this
>> traffic?
>>   - what else I should maybe be looking for in my packet dumps or
>> elsewhere?
>>
>> Are the error codes regular UNIX values, in which case I believe
>> 1 = EPERM (Operation not permitted)
>> 6 = ENXIO (No such device or address)
>> 10 = ECHILD (No child processes)
>> 110 = ETIMEDOUT (Connection timed out)
>> This would explain what the errors mean; but I don't know why they
are
>> occurring, and so regularly..
>>
>> Thank you for any pointers! :)
>>
>> Jonathan
>>
>>
> --
> Denis Cardon
> Tranquil IT Systems
> Les Espaces Jules Verne, bâtiment A
> 12 avenue Jules Verne
> 44230 Saint Sébastien sur Loire
> tel : +33 (0) 2.40.97.57.55
> http://www.tranquil-it-systems.fr
>
>

-- 
"If we knew what it was we were doing, it would not be called research,
would it?"
      - Albert Einstein

OK, the code from samba_dnsupdate :
nsupdate_cmd = lp.get('nsupdate command')
[....]
        cmd = nsupdate_cmd[:]
        cmd.append(tmpfile)
        ret = subprocess.call(cmd, shell=False, env=env)

And
$ sudo /usr/local/samba/bin/samba-tool testparm --parameter-name=nsupdate\
command
/usr/bin/nsupdate -g

But, I don't have anything called nsupdate anywhere on my machine, which I
guess is the problem.
$ find / -name nsupdate 2>/dev/null
$

I thought that nsupdate was something for using BIND, not internal DNS;
maybe I'm wrong there.

My smb.conf is very simple; do I need anything else in there?

Cheers,

J

[global]
        workgroup = MYDOMAIN
        realm = mydomain.org.uk
        netbios name = DC2
        server role = active directory domain controller
        dns forwarder = 1.2.3.4 10.11.12.13
        # Need NTLM Auth for radius
        ntlm auth = yes

[netlogon]
        path = /usr/local/samba/var/locks/sysvol/mydomain.org.uk/scripts
        read only = No

[sysvol]
        path = /usr/local/samba/var/locks/sysvol
        read only = No

On 23 September 2016 at 17:21, Jonathan Hunter <jmhunter1 at gmail.com>
wrote:
> Thank you Denis and Rowland - I didn't realise this was the script,
makes
> sense now.
>
> I've run it (on dc2) and it gets as far as:
>
> need update: SRV _ldap._tcp.mysite._sites.ForestDnsZones.mydomain.org.uk
> dc2.mydomain.org.uk 389
> [lots of updates needed]
> 10 DNS updates and 0 DNS deletes needed
> Successfully obtained Kerberos ticket to DNS/dc1.mydomain.org.uk as DC2$
>
> and then it fails here:
> update(nsupdate): SRV _ldap._tcp.mysite._sites.
> ForestDnsZones.mydomain.org.uk dc2.mydomain.org.uk 389
> Calling nsupdate for SRV _ldap._tcp.mysite._sites.
> ForestDnsZones.mydomain.org.uk dc2.mydomain.org.uk 389 (add)
> Failed nsupdate: SRV _ldap._tcp.mysite._sites.
> ForestDnsZones.mydomain.org.uk dc2.mydomain.org.uk 389 : [Errno 2] No
> such file or directory
>
> which I assume is due to a needed file not being in $PATH, or similar.
>
> I'll have a poke about using strace and see if I can spot exactly what
> it's trying to run, and where it might be..
>
> Thanks, both - I'm much further forward now!
>
> Cheers
>
> Jonathan
>
>
> On 23 September 2016 at 16:43, Denis Cardon <denis.cardon at
tranquil-it-
> systems.fr> wrote:
>
>> Hi Jonathan,
>>
>> All 3 of my DCs regularly display an error in syslog almost exactly
every
>>> 10 minutes. They have been doing this for quite some time, and I
have so
>>> far ignored the message as everything else DNS-wise seemed to
mostly be
>>> working - but I figured it was worth getting to the bottom of it if
I
>>> can.
>>> So this isn't new at all but rather something that has been
present for
>>> some time.
>>>
>>> I am using the internal Samba DNS server, currently with Samba
4.5.0. The
>>> message is as follows, every 10 minutes (I have pasted in from all
3 DCs
>>> here):
>>>
>>> Sep 23 13:03:54 dc1 samba[13117]: [2016/09/23 13:03:54.867360,  0]
>>> ../source4/dsdb/dns/dns_update.c:290(dnsupdate_nameupdate_done)
>>> Sep 23 13:03:54 dc1 samba[13117]:   ../source4/dsdb/dns/dns_updat
>>> e.c:290:
>>> Failed DNS update - with error code 5
>>>
>>
>> could you please try to run samba_dnsupdate --verbose from the command
>> line on your three DC. When a DC starts, it will try to update its own
DNS
>> fields, but it your DNS zones is missing some stuff, it may not be able
to
>> do it.
>>
>> Cheers,
>>
>> Denis
>>
>>
>>
>>> Sep 23 13:00:11 dc2 samba[901]: [2016/09/23 13:00:11.584679,  0]
>>> ../source4/dsdb/dns/dns_update.c:290(dnsupdate_nameupdate_done)
>>> Sep 23 13:00:11 dc2 samba[901]:  
../source4/dsdb/dns/dns_update.c:290:
>>> Failed DNS update - with error code 10
>>>
>>> Sep 23 13:05:28 dc3 samba[897]: [2016/09/23 13:05:28.800364,  0]
>>> ../source4/dsdb/dns/dns_update.c:290(dnsupdate_nameupdate_done)
>>> Sep 23 13:05:28 dc3 samba[897]:  
../source4/dsdb/dns/dns_update.c:290:
>>> Failed DNS update - with error code 1
>>>
>>> The precise error codes vary (I have had 1, 6, 10, 110 recently)
but I do
>>> get some sort of message every 10 minutes, and the error code
usually
>>> stays
>>> the same on a particular DC. If it makes a difference, DC1 and DC2
are in
>>> site A, and DC3 is at site B, there is full connectivity between
them all
>>> (or at least, there should be).
>>>
>>> I've tried tcpdump and wireshark to figure out what's going
on, but I
>>> can't
>>> seem to spot any form of DNS request coming in that would be an
update.
>>> The
>>> most I can see via tcpdump at any time I've looked are some
queries that
>>> return NXDOMAIN - e.g. there are frequent ones from an VMWare ESXi
server
>>> querying for _kerberos-master.udp.MYDOMAIN.ORG.UK as per
>>> https://communities.vmware.com/thread/491621 and getting NXDOMAIN -
but
>>> I
>>> wouldn't have thought that these queries would constitute a
"DNS update"
>>> that would fail?
>>>
>>> My debugging method so far has been to run tcpdump against port 53
- but
>>> either I am somehow managing to not see the failing DNS packet when
I
>>> look
>>> at the results, or the DNS update arrives at the DC some other way.
>>> Looking
>>> at the code in dns_update.c it looks like there may be some form of
>>> regular
>>> DNS check, that is failing in my case?
>>>
>>> Does anybody know
>>>   - if I can turn debugging on for just this DNS functionality? I
expect
>>> the log file here to be massive as a DC is also a DNS server.. but
>>> hopefully that will give me more of a clue as to what
"update" is
>>> failing?
>>>   - if there is some other way I might be able to capture / check
this
>>> traffic?
>>>   - what else I should maybe be looking for in my packet dumps or
>>> elsewhere?
>>>
>>> Are the error codes regular UNIX values, in which case I believe
>>> 1 = EPERM (Operation not permitted)
>>> 6 = ENXIO (No such device or address)
>>> 10 = ECHILD (No child processes)
>>> 110 = ETIMEDOUT (Connection timed out)
>>> This would explain what the errors mean; but I don't know why
they are
>>> occurring, and so regularly..
>>>
>>> Thank you for any pointers! :)
>>>
>>> Jonathan
>>>
>>>
>> --
>> Denis Cardon
>> Tranquil IT Systems
>> Les Espaces Jules Verne, bâtiment A
>> 12 avenue Jules Verne
>> 44230 Saint Sébastien sur Loire
>> tel : +33 (0) 2.40.97.57.55
>> http://www.tranquil-it-systems.fr
>>
>>
>
>
> --
> "If we knew what it was we were doing, it would not be called
research,
> would it?"
>       - Albert Einstein
>


-- 
"If we knew what it was we were doing, it would not be called research,
would it?"
      - Albert Einstein

On Fri, 23 Sep 2016 17:21:53 +0100
Jonathan Hunter via samba <samba at lists.samba.org> wrote:
> Thank you Denis and Rowland - I didn't realise this was the script,
> makes sense now.
> 
> I've run it (on dc2) and it gets as far as:
> 
> need update: SRV
> _ldap._tcp.mysite._sites.ForestDnsZones.mydomain.org.uk
> dc2.mydomain.org.uk 389 [lots of updates needed]
> 10 DNS updates and 0 DNS deletes needed
> Successfully obtained Kerberos ticket to DNS/dc1.mydomain.org.uk as
> DC2$
> 
> and then it fails here:
> update(nsupdate): SRV _ldap._tcp.mysite._
> sites.ForestDnsZones.mydomain.org.uk dc2.mydomain.org.uk 389
> Calling nsupdate for SRV _ldap._tcp.mysite._
> sites.ForestDnsZones.mydomain.org.uk dc2.mydomain.org.uk 389 (add)
> Failed nsupdate: SRV
> _ldap._tcp.mysite._sites.ForestDnsZones.mydomain.org.uk
> dc2.mydomain.org.uk 389 : [Errno 2] No such file or directory
> 
> which I assume is due to a needed file not being in $PATH, or similar.
> 
> I'll have a poke about using strace and see if I can spot exactly
> what it's trying to run, and where it might be..
> 
> Thanks, both - I'm much further forward now!
> 
Get a bit further by reading this:

https://wiki.samba.org/index.php/Verifying_and_Creating_a_DC_DNS_Record

Rowland

On Fri, 23 Sep 2016 17:38:35 +0100
Jonathan Hunter via samba <samba at lists.samba.org> wrote:
> OK, the code from samba_dnsupdate :
> nsupdate_cmd = lp.get('nsupdate command')
> [....]
>         cmd = nsupdate_cmd[:]
>         cmd.append(tmpfile)
>         ret = subprocess.call(cmd, shell=False, env=env)
> 
> And
> $ sudo /usr/local/samba/bin/samba-tool testparm
> --parameter-name=nsupdate\ command
> /usr/bin/nsupdate -g
> 
> But, I don't have anything called nsupdate anywhere on my machine,
> which I guess is the problem.
> $ find / -name nsupdate 2>/dev/null
> $
> 
> I thought that nsupdate was something for using BIND, not internal
> DNS; maybe I'm wrong there.
> 
Well, wrong and right ;-)
Wrong, in that you need it for the updates and right that it comes from
Bind.

You need to install it, it is called bind9utils on debian.

Rowland