samba - Mar 2018 - Error joining Samba 4.7.4 DC to existing Win2008R2 domain

Thanks for your attention
> You are always receiving these:
>
> Adding DNS A record SRVAD-NEW.SAMDOM.LOCAL for IPv4 IP: 10.0.3.100
> Join failed - cleaning up
Yes, but the DNS record is created and it persists after the failure.
Another thing I've noticed using RSAT "Active Directory Users and 
Computers" is that the new DC computer account SRVAD-NEW$@SAMDOM.LOCAL 
is created at the start of "samba-tool join" run (under "Domain 
Controllers" folder), it persists till the end (it runs about 15 seconds 
before failure) then it's removed upon failure.
> Questions:
>
> 1) Prior to the join, dos a kinit -V5 ADMINISTRATOR at SAMDOM.LOCAL works?
Yes, it does. Here's the log:

root at srvad-new:~# kinit -V5 ADMINISTRATOR at SAMDOM.LOCAL
Using default cache: /tmp/krb5cc_0
Using principal: ADMINISTRATOR at SAMDOM.LOCAL
Password for ADMINISTRATOR at SAMDOM.LOCAL:
Authenticated to Kerberos v5

root at srvad-new:~# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: ADMINISTRATOR at SAMDOM.LOCAL

Valid starting       Expires              Service principal
03/02/2018 08:56:52  03/02/2018 18:56:52 krbtgt/SAMDOM.LOCAL at SAMDOM.LOCAL
         renew until 03/03/2018 08:56:47
> 2) Can you create DNS entries without issues with your administrator 
> account?
If you mean create them with samba-tool yes I can, no errors:

root at srvad-new:~# samba-tool dns add srvad-old.samdom.local samdom.local 
foo A 1.2.3.4
GENSEC backend 'gssapi_spnego' registered
GENSEC backend 'gssapi_krb5' registered
GENSEC backend 'gssapi_krb5_sasl' registered
GENSEC backend 'spnego' registered
GENSEC backend 'schannel' registered
GENSEC backend 'naclrpc_as_system' registered
GENSEC backend 'sasl-EXTERNAL' registered
GENSEC backend 'ntlmssp' registered
GENSEC backend 'ntlmssp_resume_ccache' registered
GENSEC backend 'http_basic' registered
GENSEC backend 'http_ntlm' registered
GENSEC backend 'krb5' registered
GENSEC backend 'fake_gssapi_krb5' registered
Using binding ncacn_ip_tcp:srvad-old.samdom.local[,sign]
resolve_lmhosts: Attempting lmhosts lookup for name 
srvad-old.samdom.local<0x20>
resolve_lmhosts: Attempting lmhosts lookup for name 
srvad-old.samdom.local<0x20>
Record added successfully

The new DNS record is visible with RSAT on SRVAD-OLD.
> 3) Can you do a test and join your samba server as a normal computer? 
> Does it work?
Yes it does, it joins immediately, no errors (thanks to VBox virtual 
machines I can easily go back to snapshots).
This was one of the test I've already did but didn't mentioned here to 
avoid confusion.

I'm still focusing on log lines after the failure:

--- no SRVAD-OLD address in /etc/hosts ---
Join failed - cleaning up
ldb_wrap open of secrets.ldb
Could not find machine account in secrets database: Failed to fetch 
machine account password for SAMDOM from both secrets.ldb (Could not 
find entry to match filter: 
'(&(flatname=SAMDOM)(objectclass=primaryDomain))' base:
'cn=Primary
Domains': No such object: dsdb_search at 
../source4/dsdb/common/util.c:4636) and from 
/var/lib/samba/private/secrets.tdb: NT_STATUS_CANT_ACCESS_DOMAIN_INFO
---

--- SRVAD-OLD address in /etc/hosts ---
Join failed - cleaning up
ldb_wrap open of secrets.ldb
resolve_lmhosts: Attempting lmhosts lookup for name 
SRVAD-OLD.SAMDOM.LOCAL<0x20>
Wrong username or password: kinit for SRVAD-NEW$@SAMDOM.LOCAL failed 
(Preauthentication failed)
SPNEGO(gssapi_krb5) creating NEG_TOKEN_INIT for 
ldap/SRVAD-OLD.SAMDOM.LOCAL failed (next[ntlmssp]): NT_STATUS_LOGON_FAILURE
Got challenge flags:
Got NTLMSSP neg_flags=0x62898235
NTLMSSP: Set final flags:
Got NTLMSSP neg_flags=0x62088235
NTLMSSP Sign/Seal - Initialising with flags:
Got NTLMSSP neg_flags=0x62088235
Failed to bind - LDAP error 49 LDAP_INVALID_CREDENTIALS - <8009030C: 
LdapErr: DSID-0C0904D0, comment: AcceptSecurityContext error, data 52e, 
v1db0> <>
Failed to connect to 'ldap://SRVAD-OLD.SAMDOM.LOCAL' with backend 
'ldap': LDAP error 49 LDAP_INVALID_CREDENTIALS - <8009030C: LdapErr: 
DSID-0C0904D0, comment: AcceptSecurityContext error, data 52e, v1db0>
<>
---

Don't know how an authentication error could occur after being able to 
create DNS records, DC computer account...

> Em 01/03/2018 10:05, Claudio Nicora via samba escreveu:
>> It seems I'm talking to myself... anyway another test here:
>>
>> Added the existing DC IP config to /etc/hosts and the join now shows 
>> a more explicit LDAP error:
>>
>> ---
>> Wrong username or password: kinit for SRVAD-NEW$@SAMDOM.LOCAL failed 
>> (Preauthentication failed)
>> SPNEGO(gssapi_krb5) creating NEG_TOKEN_INIT for 
>> ldap/SRVAD-OLD.SAMDOM.LOCAL failed (next[ntlmssp]): 
>> NT_STATUS_LOGON_FAILURE
>> Got challenge flags:
>> Got NTLMSSP neg_flags=0x62898235
>> NTLMSSP: Set final flags:
>> Got NTLMSSP neg_flags=0x62088235
>> NTLMSSP Sign/Seal - Initialising with flags:
>> Got NTLMSSP neg_flags=0x62088235
>> Failed to bind - LDAP error 49 LDAP_INVALID_CREDENTIALS - <8009030C:
>> LdapErr: DSID-0C0904D0, comment: AcceptSecurityContext error, data 
>> 52e, v1db0> <>
>> Failed to connect to 'ldap://SRVAD-OLD.SAMDOM.LOCAL' with
backend
>> 'ldap': LDAP error 49 LDAP_INVALID_CREDENTIALS - <8009030C:
LdapErr:
>> DSID-0C0904D0, comment: AcceptSecurityContext error, data 52e,
v1db0> <>
>>

On Fri, 2018-03-02 at 09:19 +0100, Ing. Claudio Nicora via samba
wrote:
> 
> I'm still focusing on log lines after the failure:
> 
> --- no SRVAD-OLD address in /etc/hosts ---
> Join failed - cleaning up
> ldb_wrap open of secrets.ldb
> Could not find machine account in secrets database: Failed to fetch 
> machine account password for SAMDOM from both secrets.ldb (Could not 
> find entry to match filter: 
> '(&(flatname=SAMDOM)(objectclass=primaryDomain))' base:
'cn=Primary
> Domains': No such object: dsdb_search at 
> ../source4/dsdb/common/util.c:4636) and from 
> /var/lib/samba/private/secrets.tdb: NT_STATUS_CANT_ACCESS_DOMAIN_INFO
> ---
This isn't it.  The things after the failure are not the issue, they
are happening in the unwind.  

You can see the real failure in the backtrace, where it fails to find
the records using our DNS client library for LDAP (yes, a very strange
thing). 

As Garming said, the issue is that Samba can't find the DNS records on
your AD DC over LDAP, having just added them via RPC.

This code is a bit tricky, and I thought I had it right, but clearly
that isn't the case.  Garming asked you to see if you could locate
where the records got put the records by hand.  

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba

> Garming asked you to see if you could locate
 > where the records got put the records by hand

Sorry, I can't understand what you mean with "if you could locate where
the records got put"...
Are you're asking me to create the DNS record by hand with RSAT on 
SRVAD_OLD, then run samba-tool join again?
If so, yes I've tried to create the record manually and re-run 
samba-tool with the same error.

Sorry for the misunderstand
>> I'm still focusing on log lines after the failure:
>>
>> --- no SRVAD-OLD address in /etc/hosts ---
>> Join failed - cleaning up
>> ldb_wrap open of secrets.ldb
>> Could not find machine account in secrets database: Failed to fetch
>> machine account password for SAMDOM from both secrets.ldb (Could not
>> find entry to match filter:
>> '(&(flatname=SAMDOM)(objectclass=primaryDomain))' base:
'cn=Primary
>> Domains': No such object: dsdb_search at
>> ../source4/dsdb/common/util.c:4636) and from
>> /var/lib/samba/private/secrets.tdb: NT_STATUS_CANT_ACCESS_DOMAIN_INFO
>> ---
> This isn't it.  The things after the failure are not the issue, they
> are happening in the unwind.
>
> You can see the real failure in the backtrace, where it fails to find
> the records using our DNS client library for LDAP (yes, a very strange
> thing).
>
> As Garming said, the issue is that Samba can't find the DNS records on
> your AD DC over LDAP, having just added them via RPC.
>
> This code is a bit tricky, and I thought I had it right, but clearly
> that isn't the case.  Garming asked you to see if you could locate
> where the records got put the records by hand.
>
> Thanks,
>
> Andrew Bartlett
>

On Fri, 2018-03-02 at 09:19 +0100, Ing. Claudio Nicora via samba
wrote:
> Failed to bind - LDAP error 49 LDAP_INVALID_CREDENTIALS - <8009030C: 
> LdapErr: DSID-0C0904D0, comment: AcceptSecurityContext error, data 52e, 
> v1db0> <>
> Failed to connect to 'ldap://SRVAD-OLD.SAMDOM.LOCAL' with backend 
> 'ldap': LDAP error 49 LDAP_INVALID_CREDENTIALS - <8009030C:
LdapErr:
> DSID-0C0904D0, comment: AcceptSecurityContext error, data 52e, v1db0>
<>
> ---
> 
> Don't know how an authentication error could occur after being able to 
> create DNS records, DC computer account...
This is in part of the code that tries to avoid 'cleaning up' an
account that we are actually fully joined with.  It produces a bit of
noise but is otherwise harmless.

It was added after a user re-joined their fully working DC to the
domain, causing trouble as it re-synced a large domain.  That same code
runs in the clenaup mode you see here.

Sorry for the further red herrings. 

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba