samba - Oct 2017 - Made a join with a netbios name, which already existed, now replication errors

On Wed, 2 Aug 2017 10:48:50 +0200
gizmo via samba <samba at lists.samba.org> wrote:
> > No you cannot delete something that is already deleted, but then
> > deleted objects should be ignored and I think this is fixed in later
> > versions.
> > 
> > Does your Samba version have 'samba-tool domain tombstones
> > expunge' ? if it does, you can set the
'--tombstone-lifetime' to 1
> > day and then wait, all the 'OADEL' objects should disappear.
> 
> no, 4.3.11 (SerNet) doesnt have this option yet. I have to wait then.
> Because I wont risc an upgrade before I can join a new DC.
> What's the default time for keeping deleted objects ?
> 
You are possibly going to have a long wait, it is 180 days

Rowland

> On 2017.08.02, at 4:22 AM, Rowland Penny via samba <samba at
lists.samba.org> wrote:
> 
> On Wed, 2 Aug 2017 10:48:50 +0200
> gizmo via samba <samba at lists.samba.org> wrote:
> 
>>> No you cannot delete something that is already deleted, but then
>>> deleted objects should be ignored and I think this is fixed in
later
>>> versions.
>>> 
>>> Does your Samba version have 'samba-tool domain tombstones
>>> expunge' ? if it does, you can set the
'--tombstone-lifetime' to 1
>>> day and then wait, all the 'OADEL' objects should
disappear.
>> 
>> no, 4.3.11 (SerNet) doesnt have this option yet. I have to wait then.
>> Because I wont risc an upgrade before I can join a new DC.
>> What's the default time for keeping deleted objects ?
>> 
> 
> You are possibly going to have a long wait, it is 180 days
> 
> Rowland
I’m having a similar problem. I just fixed a bad member of my samba domain - an
samba AD DC that wasn’t working. I demoted it, uninstalled Samba and
reinstalled, then rejoined the domain.

Everything's replicating nicely. All my users can authenticate. But my samba
AD DCs are all on 4.4.16, and I want to be on 4.7.

So, I set up a new server to act as my 4.7. My plan: Join it to the domain, move
the FSMO role to this new server, then one-by-one replace my old DCs with new
ones running Samba 4.7.

I go to get the new 4.7 samba machine joined and here’s what happens:

-----

samba-tool domain join mydomain.net DC -Uadministrator --realm=mydomain.net
--dns-backend=BIND9_DLZ
Finding a writeable DC for domain 'mydomain.net'
Found DC rhea.mydomain.net
Password for [mydomain\administrator]:
workgroup is mydomain
realm is mydomain.net
Adding CN=UMBRIEL,OU=Domain Controllers,DC=mydomain,DC=net
Adding
CN=UMBRIEL,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=net
Adding CN=NTDS
Settings,CN=UMBRIEL,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=net
Adding SPNs to CN=UMBRIEL,OU=Domain Controllers,DC=mydomain,DC=net
Setting account password for UMBRIEL$
Enabling account
Adding DNS account CN=dns-UMBRIEL,CN=Users,DC=mydomain,DC=net with dns/ SPN
Setting account password for dns-UMBRIEL
Calling bare provision
Looking up IPv4 addresses
Looking up IPv6 addresses
No IPv6 address will be assigned
Setting up secrets.ldb
Setting up the registry
Setting up the privileges database
Setting up idmap db
Setting up SAM db
Setting up sam.ldb partitions and settings
Setting up sam.ldb rootDSE
Pre-loading the Samba 4 and AD schema
A Kerberos configuration suitable for Samba AD has been generated at
/var/lib/samba/private/krb5.conf
Provision OK for domain DN DC=mydomain,DC=net
Starting replication
Schema-DN[CN=Schema,CN=Configuration,DC=mydomain,DC=net] objects[402/1578]
linked_values[0/0]
Schema-DN[CN=Schema,CN=Configuration,DC=mydomain,DC=net] objects[804/1578]
linked_values[0/0]
Schema-DN[CN=Schema,CN=Configuration,DC=mydomain,DC=net] objects[1206/1578]
linked_values[0/0]
Schema-DN[CN=Schema,CN=Configuration,DC=mydomain,DC=net] objects[1578/1578]
linked_values[0/0]
Analyze and apply schema objects
Partition[CN=Configuration,DC=mydomain,DC=net] objects[402/1636]
linked_values[0/0]
Partition[CN=Configuration,DC=mydomain,DC=net] objects[804/1636]
linked_values[0/0]
Partition[CN=Configuration,DC=mydomain,DC=net] objects[1206/1636]
linked_values[0/0]
Partition[CN=Configuration,DC=mydomain,DC=net] objects[1608/1636]
linked_values[0/0]
Partition[CN=Configuration,DC=mydomain,DC=net] objects[1636/1636]
linked_values[47/0]
Unxpectedly got mismatching RDN values when checking RDN against name of CN=NTDS
Settings,CN=GANYMEDE\0ADEL:9646252c-8e4d-447f-90fa-3a51355276ac,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=netFailed
to convert object CN=NTDS
Settings,CN=GANYMEDE\0ADEL:9646252c-8e4d-447f-90fa-3a51355276ac,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=net:
WERR_GEN_FAILURE
Failed to convert objects: WERR_GEN_FAILURE
Join failed - cleaning up
Deleted CN=UMBRIEL,OU=Domain Controllers,DC=mydomain,DC=net
Deleted CN=dns-UMBRIEL,CN=Users,DC=mydomain,DC=net
Deleted CN=NTDS
Settings,CN=UMBRIEL,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=net
Deleted
CN=UMBRIEL,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=net
ERROR(runtime): uncaught exception - (31, "Failed to process
'chunk' of DRS replicated objects: WERR_GEN_FAILURE")
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/__init__.py",
line 176, in _run
    return self.run(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/domain.py", line
661, in run
    machinepass=machinepass, use_ntvfs=use_ntvfs, dns_backend=dns_backend)
  File "/usr/lib/python2.7/dist-packages/samba/join.py", line 1474, in
join_DC
    ctx.do_join()
  File "/usr/lib/python2.7/dist-packages/samba/join.py", line 1377, in
do_join
    ctx.join_replicate()
  File "/usr/lib/python2.7/dist-packages/samba/join.py", line 924, in
join_replicate
    replica_flags=ctx.replica_flags)
  File "/usr/lib/python2.7/dist-packages/samba/drs_utils.py", line
295, in replicate
    schema=schema, req_level=req_level, req=req)

-----

("Ganymede" is the server I just demoted and re-promoted.)

By your thread with gizmo, I take it that my new samba AD DC doesn’t like this
deleted record:

-----

sudo ldbsearch --cross-ncs --show-deleted -H /var/lib/samba/private/sam.ldb
"distinguishedName=CN=GANYMEDE\0ADEL:9646252c-8e4d-447f-90fa-3a51355276ac,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=net"
[sudo] password for svr.matthew.delfino: 
# record 1
dn:
CN=GANYMEDE\0ADEL:9646252c-8e4d-447f-90fa-3a51355276ac,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=net
objectClass: top
objectClass: server
instanceType: 4
whenCreated: 20151103020735.0Z
uSNCreated: 20599
objectGUID: 9646252c-8e4d-447f-90fa-3a51355276ac
systemFlags: 1375731712
dNSHostName: GANYMEDE.mydomain.net
isDeleted: TRUE
lastKnownParent: CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configurati
 on,DC=mydomain,DC=net
isRecycled: TRUE
cn:: R0FOWU1FREUKREVMOjk2NDYyNTJjLThlNGQtNDQ3Zi05MGZhLTNhNTEzNTUyNzZhYw=name::
R0FOWU1FREUKREVMOjk2NDYyNTJjLThlNGQtNDQ3Zi05MGZhLTNhNTEzNTUyNzZhYw=whenChanged:
20171030231808.0Z
uSNChanged: 17728815
distinguishedName: CN=GANYMEDE\0ADEL:9646252c-8e4d-447f-90fa-3a51355276ac,CN=S
 ervers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=lo
 c

# returned 1 records
# 1 entries
# 0 referrals

-----

If I understand you correspondence above, this "tombstone" record
needs to be expunged. But, since my version, (4.4.16), has a samba-tool that
appears to not be able to do "samba-tool domain tombstones…." I have
to wait 180 days for that record to automatically go away and the mismatch to go
away in kind? Do I have this right?

Do I have any options other than waiting 179 more days? I mean, besides a
DeLorean with a Flux Capacitor, or cryogenic stasis… or (gulp) patience?

Thanks,
Matthew

©2017 KNOCK, inc. All rights reserved. KNOCK is a registered trademark of KNOCK,
inc. This message and any attachments contain information, which is confidential
and/or privileged. If you are not the intended recipient, please refrain from
any disclosure, copying, distribution or use of this information. Please be
aware that such actions are prohibited. If you have received this transmission
in error, kindly notify the sender by e-mail. Your cooperation is appreciated.

On Tue, 2017-10-31 at 17:37 -0500, Matthew Delfino via samba
wrote:
> > 
> 
> I’m having a similar problem. I just fixed a bad member of my samba
> domain - an samba AD DC that wasn’t working. I demoted it,
> uninstalled Samba and reinstalled, then rejoined the domain.
> 
> Everything's replicating nicely. All my users can authenticate. But
> my samba AD DCs are all on 4.4.16, and I want to be on 4.7.
> 
> So, I set up a new server to act as my 4.7. My plan: Join it to the
> domain, move the FSMO role to this new server, then one-by-one
> replace my old DCs with new ones running Samba 4.7.
> 
> I go to get the new 4.7 samba machine joined and here’s what happens:
> 
> -----
> Partition[CN=Configuration,DC=mydomain,DC=net] objects[402/1636]
linked_values[0/0]
> Partition[CN=Configuration,DC=mydomain,DC=net] objects[804/1636]
linked_values[0/0]
> Partition[CN=Configuration,DC=mydomain,DC=net] objects[1206/1636]
linked_values[0/0]
> Partition[CN=Configuration,DC=mydomain,DC=net] objects[1608/1636]
linked_values[0/0]
> Partition[CN=Configuration,DC=mydomain,DC=net] objects[1636/1636]
linked_values[47/0]
> Unxpectedly got mismatching RDN values when checking RDN against name of
CN=NTDS
Settings,CN=GANYMEDE\0ADEL:9646252c-8e4d-447f-90fa-3a51355276ac,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=netFailed
to convert object CN=NTDS
Settings,CN=GANYMEDE\0ADEL:9646252c-8e4d-447f-90fa-3a51355276ac,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=net:
WERR_GEN_FAILURE
> Failed to convert objects: WERR_GEN_FAILURE
> Join failed - cleaning up
This is interesting.  Sadly the code checking this doesn't print the
RDN value and name that it dislikes for comparison, this really wasn't
expected to be seen in the field. 

What does dbcheck say?  Once you back it up and fix it on 4.4, if you
copy the DB to a 4.7 host, does it give any more errors regarding this
object?
> -----
> 
> ("Ganymede" is the server I just demoted and re-promoted.)
> 
> By your thread with gizmo, I take it that my new samba AD DC doesn’t like
this deleted record:
> 
> -----
> 
> sudo ldbsearch --cross-ncs --show-deleted -H /var/lib/samba/private/sam.ldb
"distinguishedName=CN=GANYMEDE\0ADEL:9646252c-8e4d-447f-90fa-3a51355276ac,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=net"
> [sudo] password for svr.matthew.delfino: 
> # record 1
> dn:
CN=GANYMEDE\0ADEL:9646252c-8e4d-447f-90fa-3a51355276ac,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=net
> 
> lastKnownParent:
CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configurati
>  on,DC=mydomain,DC=net
> isRecycled: TRUE
> cn::
R0FOWU1FREUKREVMOjk2NDYyNTJjLThlNGQtNDQ3Zi05MGZhLTNhNTEzNTUyNzZhYw=> name::
R0FOWU1FREUKREVMOjk2NDYyNTJjLThlNGQtNDQ3Zi05MGZhLTNhNTEzNTUyNzZhYw=>
whenChanged: 20171030231808.0Z
> uSNChanged: 17728815
> distinguishedName:
CN=GANYMEDE\0ADEL:9646252c-8e4d-447f-90fa-3a51355276ac,CN=S
> 
ervers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mydomain,DC=lo
>  c
Yes and no.  This looks normal enough, it actually doesn't like the
CN=NTDS Settings child of this object.  Can you show that?
> If I understand you correspondence above, this "tombstone" record
> needs to be expunged. But, since my version, (4.4.16), has a samba-
> tool that appears to not be able to do "samba-tool domain
> tombstones…." I have to wait 180 days for that record to
> automatically go away and the mismatch to go away in kind? Do I have
> this right?
You could upgrade the domain in-place and use the modern tools, or on a
new host that you will give the same name as the old one (we are not
fussy about the surrounding OS, just the hostname and to a lesser
extent the IP). 
> Do I have any options other than waiting 179 more days? I mean, besides a
DeLorean with a Flux Capacitor, or cryogenic stasis… or (gulp) patience?
You can change the tombstoneLifetime, but please turn it back up once
you are done. 

Andrew Bartlett
-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba