samba - Sep 2017 - Server GC/name.dom/dom is not registered with our KDC: Miscellaneous failure (see text): Server (GC/name/dom@DOM) unknown

> Keytabs look reasonable, as far as I can see, but why does 
> graz-dc-sem have the same SPN output as graz-dc-1b in 
> addition to its own?
A snapshotted server/cloned server? I dont know but thats not correct.

I suggest, cleanup the DS with FSMO roles. 
Then remove a failty server and re-add it as a new installed DC.

( the good DS with FSMO) 
First backup: /var/lib/samba/private/secrets.keytab 
Remove the incorrect entries from keytab file with ktutil 
rkt /var/lib/samba/private/secrets.keytab
list -e -t

Check if dates here are related to other work you/someone did?

Now you can remove the failty one from the domain and re-add it (with
provisioning)
Backup and cleanup
/etc/samba/smb.conf  (rename) 
/var/cache/samba	   ( remove all files from folder) 
/var/lib/samba	   ( remove all files and directories from folder) 

Now re-provision and you should have correct working DC's again. 

! Before re-provisioning, make sure all OLD records dns and AD are gone. 



Greetz, 

Louis
> -----Oorspronkelijk bericht-----
> Van: Sven Schwedas [mailto:sven.schwedas at tao.at] 
> Verzonden: dinsdag 5 september 2017 15:34
> Aan: L.P.H. van Belle; samba at lists.samba.org
> Onderwerp: Re: [Samba] Server GC/name.dom/dom is not 
> registered with our KDC: Miscellaneous failure (see text): 
> Server (GC/name/dom at DOM) unknown
> 
> On 2017-09-05 14:40, L.P.H. van Belle wrote:
> > Ah.. I had a "member break down" ..  
> > 
> > Out of the blue,.. Kerberos problem, but pretty simple to fix. 
> > 
> > kinit Administrator
> 
> Works on all DCs.
> 
> > Check your spn of the ad server with :  
> > samba-tool spn list DC_HOSTNAME$
> >
> > Check keytab
> > klist -ke /var/lib/samba/private/secrets.keytab
> 
> Outputs attached. graz-dc-1b is the one making trouble, 
> graz-dc-sem is the FSMO role holder.
> 
> Keytabs look reasonable, as far as I can see, but why does 
> graz-dc-sem have the same SPN output as graz-dc-1b in 
> addition to its own?
> 
> > Can you check this. 
> > 
> > Greetz,
> > 
> > Louis
> > 
> > 
> >> -----Oorspronkelijk bericht-----
> >> Van: samba [mailto:samba-bounces at lists.samba.org] Namens Sven 
> >> Schwedas via samba
> >> Verzonden: dinsdag 5 september 2017 14:28
> >> Aan: samba at lists.samba.org
> >> Onderwerp: [Samba] Server GC/name.dom/dom is not 
> registered with our 
> >> KDC: Miscellaneous failure (see text): Server
> >> (GC/name/dom at DOM) unknown
> >>
> >> Today's episode of "why is AD break", brought to you
by:
> >>
> >>> [2017/09/05 10:17:06.015617,  3]
> >> ../source4/auth/gensec/gensec_gssapi.c:613(gensec_gssapi_update)
> >>>   Server GC/graz-dc-1b.ad.tao.at/ad.tao.at is not
> >> registered with our
> >>> KDC:  Miscellaneous failure (see text): Server
> >>> (GC/graz-dc-1b.ad.tao.at/ad.tao.at at AD.TAO.AT) unknown
> >>> [2017/09/05 10:17:06.015717,  0]
> >> ../source4/librpc/rpc/dcerpc_util.c:745(dcerpc_pipe_auth_recv)
> >>>   Failed to bind to uuid e3514235-4b06-11d1-ab04-00c04fc2dcd2
for
> >>>
> >> 
> ncacn_ip_tcp:192.168.17.66[1024,seal,krb5,target_hostname=bcffbad8-1a
> >> d
> >>>
> >> 
> d-46b9-bf69-90e52c0f09ea._msdcs.ad.tao.at,target_principal=GC/graz-dc
> >> -
> >>>
> >> 
> 1b.ad.tao.at/ad.tao.at,abstract_syntax=e3514235-4b06-11d1-ab04-00c04f
> >> c
> >>> 2dcd2/0x00000004,localaddress=192.168.16.213]
> >>> NT_STATUS_INVALID_PARAMETER
> >>> [2017/09/05 10:17:06.015869,  4]
> >> 
> ../source4/dsdb/repl/drepl_notify.c:196(dreplsrv_notify_op_callback)
> >>>   dreplsrv_notify: Failed to send DsReplicaSync to 
> >>> bcffbad8-1add-46b9-bf69-90e52c0f09ea._msdcs.ad.tao.at for 
> >>> DC=ad,DC=tao,DC=at - NT_STATUS_INVALID_PARAMETER :
> >> WERR_INVALID_PARAM
> >>
> >> The few google results for this seem to indicate DNS 
> issues, but I'm 
> >> not sure where those should come from. The servers in question 
> >> resolve graz-dc-1b.ad.tao.at as well as 
> >> bcffbad8-1add-46b9-bf69-90e52c0f09ea._msdcs.ad.tao.at to 
> the correct 
> >> IP.
> >> Same goes for _kerberos.* and the other SRV records in _msdcs. and
> >> the AD domain itself.
> >>
> >> Any ideas where else to look?
> >>
> >> --
> >> Mit freundlichen Grüßen, / Best Regards, Sven Schwedas, 
> >> Systemadministrator Mail/XMPP sven.schwedas at tao.at | Skype 
> >> sven.schwedas TAO Digital | Lendplatz 45 | A8020 Graz 
> >> https://www.tao-digital.at | Tel +43 680 301 7167
> >>
> >> --
> >> To unsubscribe from this list go to the following URL and read the
> >> instructions:  https://lists.samba.org/mailman/options/samba
> >>
> > 
> 
> --
> Mit freundlichen Grüßen, / Best Regards, Sven Schwedas, 
> Systemadministrator Mail/XMPP sven.schwedas at tao.at | Skype 
> sven.schwedas TAO Digital | Lendplatz 45 | A8020 Graz 
> https://www.tao-digital.at | Tel +43 680 301 7167
>

On 2017-09-05 16:21, L.P.H. van Belle wrote:
>> Keytabs look reasonable, as far as I can see, but why does 
>> graz-dc-sem have the same SPN output as graz-dc-1b in 
>> addition to its own?
> A snapshotted server/cloned server? I dont know but thats not correct.
Nope, both were created clean. There used to be a graz-dc-bis, but
removing and re-adding it completely broke replication, so I nuked it
and created 1b to replace it. That odyssey is in the list archives
somewhere…
> I suggest, cleanup the DS with FSMO roles. 
Clean up as in move FSMO roles to a clean server (leaves only
villach-dc-*) ?
> Then remove a failty server and re-add it as a new installed DC.
> ( the good DS with FSMO) 
> First backup: /var/lib/samba/private/secrets.keytab 
> Remove the incorrect entries from keytab file with ktutil 
> rkt /var/lib/samba/private/secrets.keytab
> list -e -t
Might as well just nuke graz-dc-sem and add a complete new DC from
scratch, no?
> Check if dates here are related to other work you/someone did?
> 
> Now you can remove the failty one from the domain and re-add it (with
provisioning)
> Backup and cleanup
> /etc/samba/smb.conf  (rename) 
> /var/cache/samba	   ( remove all files from folder) 
> /var/lib/samba	   ( remove all files and directories from folder) 
> 
> Now re-provision and you should have correct working DC's again. 
> 
> ! Before re-provisioning, make sure all OLD records dns and AD are gone. 
I still have undeleteable replication records from the last time I had
to nuke a DC, nobody replied to my emails on that issue.
> 
> 
> 
> Greetz, 
> 
> Louis
> 
>> -----Oorspronkelijk bericht-----
>> Van: Sven Schwedas [mailto:sven.schwedas at tao.at] 
>> Verzonden: dinsdag 5 september 2017 15:34
>> Aan: L.P.H. van Belle; samba at lists.samba.org
>> Onderwerp: Re: [Samba] Server GC/name.dom/dom is not 
>> registered with our KDC: Miscellaneous failure (see text): 
>> Server (GC/name/dom at DOM) unknown
>>
>> On 2017-09-05 14:40, L.P.H. van Belle wrote:
>>> Ah.. I had a "member break down" ..  
>>>
>>> Out of the blue,.. Kerberos problem, but pretty simple to fix. 
>>>
>>> kinit Administrator
>>
>> Works on all DCs.
>>
>>> Check your spn of the ad server with :  
>>> samba-tool spn list DC_HOSTNAME$
>>>
>>> Check keytab
>>> klist -ke /var/lib/samba/private/secrets.keytab
>>
>> Outputs attached. graz-dc-1b is the one making trouble, 
>> graz-dc-sem is the FSMO role holder.
>>
>> Keytabs look reasonable, as far as I can see, but why does 
>> graz-dc-sem have the same SPN output as graz-dc-1b in 
>> addition to its own?
>>
>>> Can you check this. 
>>>
>>> Greetz,
>>>
>>> Louis
>>>
>>>
>>>> -----Oorspronkelijk bericht-----
>>>> Van: samba [mailto:samba-bounces at lists.samba.org] Namens
Sven
>>>> Schwedas via samba
>>>> Verzonden: dinsdag 5 september 2017 14:28
>>>> Aan: samba at lists.samba.org
>>>> Onderwerp: [Samba] Server GC/name.dom/dom is not 
>> registered with our 
>>>> KDC: Miscellaneous failure (see text): Server
>>>> (GC/name/dom at DOM) unknown
>>>>
>>>> Today's episode of "why is AD break", brought to
you by:
>>>>
>>>>> [2017/09/05 10:17:06.015617,  3]
>>>>
../source4/auth/gensec/gensec_gssapi.c:613(gensec_gssapi_update)
>>>>>   Server GC/graz-dc-1b.ad.tao.at/ad.tao.at is not
>>>> registered with our
>>>>> KDC:  Miscellaneous failure (see text): Server
>>>>> (GC/graz-dc-1b.ad.tao.at/ad.tao.at at AD.TAO.AT) unknown
>>>>> [2017/09/05 10:17:06.015717,  0]
>>>> ../source4/librpc/rpc/dcerpc_util.c:745(dcerpc_pipe_auth_recv)
>>>>>   Failed to bind to uuid
e3514235-4b06-11d1-ab04-00c04fc2dcd2 for
>>>>>
>>>>
>> ncacn_ip_tcp:192.168.17.66[1024,seal,krb5,target_hostname=bcffbad8-1a
>>>> d
>>>>>
>>>>
>> d-46b9-bf69-90e52c0f09ea._msdcs.ad.tao.at,target_principal=GC/graz-dc
>>>> -
>>>>>
>>>>
>> 1b.ad.tao.at/ad.tao.at,abstract_syntax=e3514235-4b06-11d1-ab04-00c04f
>>>> c
>>>>> 2dcd2/0x00000004,localaddress=192.168.16.213]
>>>>> NT_STATUS_INVALID_PARAMETER
>>>>> [2017/09/05 10:17:06.015869,  4]
>>>>
>> ../source4/dsdb/repl/drepl_notify.c:196(dreplsrv_notify_op_callback)
>>>>>   dreplsrv_notify: Failed to send DsReplicaSync to 
>>>>> bcffbad8-1add-46b9-bf69-90e52c0f09ea._msdcs.ad.tao.at for 
>>>>> DC=ad,DC=tao,DC=at - NT_STATUS_INVALID_PARAMETER :
>>>> WERR_INVALID_PARAM
>>>>
>>>> The few google results for this seem to indicate DNS 
>> issues, but I'm 
>>>> not sure where those should come from. The servers in question 
>>>> resolve graz-dc-1b.ad.tao.at as well as 
>>>> bcffbad8-1add-46b9-bf69-90e52c0f09ea._msdcs.ad.tao.at to 
>> the correct 
>>>> IP.
>>>> Same goes for _kerberos.* and the other SRV records in _msdcs.
and
>>>> the AD domain itself.
>>>>
>>>> Any ideas where else to look?
>>>>
>>>> --
>>>> Mit freundlichen Grüßen, / Best Regards, Sven Schwedas, 
>>>> Systemadministrator Mail/XMPP sven.schwedas at tao.at | Skype 
>>>> sven.schwedas TAO Digital | Lendplatz 45 | A8020 Graz 
>>>> https://www.tao-digital.at | Tel +43 680 301 7167
>>>>
>>>> --
>>>> To unsubscribe from this list go to the following URL and read
the
>>>> instructions:  https://lists.samba.org/mailman/options/samba
>>>>
>>>
>>
>> --
>> Mit freundlichen Grüßen, / Best Regards, Sven Schwedas, 
>> Systemadministrator Mail/XMPP sven.schwedas at tao.at | Skype 
>> sven.schwedas TAO Digital | Lendplatz 45 | A8020 Graz 
>> https://www.tao-digital.at | Tel +43 680 301 7167
>>
> 
-- 
Mit freundlichen Grüßen, / Best Regards,
Sven Schwedas, Systemadministrator
Mail/XMPP sven.schwedas at tao.at | Skype sven.schwedas
TAO Digital | Lendplatz 45 | A8020 Graz
https://www.tao-digital.at | Tel +43 680 301 7167

Yes, if you flexible with reinstalling, you could..
(more below) 
> -----Oorspronkelijk bericht-----
> Van: Sven Schwedas [mailto:sven.schwedas at tao.at] 
> Verzonden: dinsdag 5 september 2017 16:32
> Aan: L.P.H. van Belle; samba at lists.samba.org
> Onderwerp: Re: [Samba] Server GC/name.dom/dom is not 
> registered with our KDC: Miscellaneous failure (see text): 
> Server (GC/name/dom at DOM) unknown
> 
> On 2017-09-05 16:21, L.P.H. van Belle wrote:
> >> Keytabs look reasonable, as far as I can see, but why does 
> >> graz-dc-sem have the same SPN output as graz-dc-1b in 
> addition to its 
> >> own?
> > A snapshotted server/cloned server? I dont know but thats 
> not correct.
> 
> Nope, both were created clean. There used to be a 
> graz-dc-bis, but removing and re-adding it completely broke 
> replication, so I nuked it and created 1b to replace it. That 
> odyssey is in the list archives somewhere…
Very strange then if they where all created clean. 
removing and re-adding is possible, but not without rist. 
> 
> > I suggest, cleanup the DS with FSMO roles. 
> 
> Clean up as in move FSMO roles to a clean server (leaves only
> villach-dc-*) ?
Yes and no.  ;-) 

I suggest the following, move fsmo roles to villach-dc and check database
replications.

Remove the most faulty one first, graz-dc-1b, from the domain. ( check and
cleanup DNS and AD! Very important )

You dont have to reinstall the complete os, just cleanup as told, and
reprovisioning that server again.
Reboot and then wait, and check database replication again. 
! Do reboot ! 

And repeat for all servers you dont trust. 

That should bring you network back as it should be. 

> 
> > Then remove a failty server and re-add it as a new installed DC.
> > ( the good DS with FSMO)
> > First backup: /var/lib/samba/private/secrets.keytab
> > Remove the incorrect entries from keytab file with ktutil rkt 
> > /var/lib/samba/private/secrets.keytab
> > list -e -t
> 
> Might as well just nuke graz-dc-sem and add a complete new DC 
> from scratch, no?
No, and yes, but i preffer no, not needed (yet). 
Start with the keytab cleanup 
Check the dns record if the uuid A PTR and hostnames resolve to the correct
server.
If thats the case, then no, cleanup of keytab is, i think, sufficient. 

Yes, if its really a mess. ;-) 
Then, first a an new DC, then remove, just make sure you always have 2 dc's
up and running (correctly)

> 
> > Check if dates here are related to other work you/someone did?
> > 
> > Now you can remove the failty one from the domain and 
> re-add it (with 
> > provisioning) Backup and cleanup /etc/samba/smb.conf  (rename)
> > /var/cache/samba	   ( remove all files from folder) 
> > /var/lib/samba	   ( remove all files and directories 
> from folder) 
> > 
> > Now re-provision and you should have correct working DC's again. 
> > 
> > ! Before re-provisioning, make sure all OLD records dns and 
> AD are gone. 
> 
> I still have undeleteable replication records from the last 
> time I had to nuke a DC, nobody replied to my emails on that issue.
Ok, now, im out of office in about 10 min, but mail that subject for me again. 
I'll have a look. 
Own and if you dont use it, ApacheDirectoryStudio can help a lot with cleanup of
these kind of things.
But just make sure you know what you delete, for you mess up the AD even more. 


Greetz, 

Louis

On 2017-09-05 16:52, L.P.H. van Belle wrote:
> Yes, if you flexible with reinstalling, you could..
I don't want another quick and dirty solution that turns out to break
half a year down the line, I'm fine with nuking half my DCs if that
means getting to a clean state.

Besides, recreating containers is faster than manually messing around in
/var/lib on each one of them.
> I suggest the following, move fsmo roles to villach-dc and check database
replications.
DB replication is already spewing errors, what am I to look out for?
> Remove the most faulty one first, graz-dc-1b, from the domain. ( check and
cleanup DNS and AD! Very important )
What to check for? What to clean up?
> You dont have to reinstall the complete os, just cleanup as told, and
reprovisioning that server again.
Adding a new DC with the same hostname as the old DC is what got me into
trouble last time. I'll pass up on that offer.
>>> Then remove a failty server and re-add it as a new installed DC.
>>> ( the good DS with FSMO)
>>> First backup: /var/lib/samba/private/secrets.keytab
>>> Remove the incorrect entries from keytab file with ktutil rkt 
>>> /var/lib/samba/private/secrets.keytab
>>> list -e -t
>>
>> Might as well just nuke graz-dc-sem and add a complete new DC 
>> from scratch, no?
> No, and yes, but i preffer no, not needed (yet). 
> Start with the keytab cleanup 
> Check the dns record if the uuid A PTR and hostnames resolve to the correct
server.
> If thats the case, then no, cleanup of keytab is, i think, sufficient. 
Just to confirm the order: Clean up the keytab, if that doesn't work,
start removing servers?
> Yes, if its really a mess. ;-) 
> Then, first a an new DC, then remove, just make sure you always have 2
dc's up and running (correctly)
Servers in Villach seem to run fine, thank $DEITY, so I'll leave them
alone for now.
>>> Now re-provision and you should have correct working DC's
again.
>>>
>>> ! Before re-provisioning, make sure all OLD records dns and 
>> AD are gone. 
>>
>> I still have undeleteable replication records from the last 
>> time I had to nuke a DC, nobody replied to my emails on that issue.
> 
> Ok, now, im out of office in about 10 min, but mail that subject for me
again> I'll have a look.
First message on that topic:
https://lists.samba.org/archive/samba/2017-March/207429.html

Last message, where I mentioned the replication bug:
https://lists.samba.org/archive/samba/2017-April/207918.html
> Own and if you dont use it, ApacheDirectoryStudio can help a lot with
cleanup of these kind of things.
Currently I'm using the ADSI MMC snap-in, any downsides compared to ADS?
> But just make sure you know what you delete, for you mess up the AD even
more.
That why I'm not touching anything without a full list.

-- 
Mit freundlichen Grüßen, / Best Regards,
Sven Schwedas, Systemadministrator
Mail/XMPP sven.schwedas at tao.at | Skype sven.schwedas
TAO Digital | Lendplatz 45 | A8020 Graz
https://www.tao-digital.at | Tel +43 680 301 7167

Hai Sven,  
> -----Oorspronkelijk bericht-----
> Van: Sven Schwedas [mailto:sven.schwedas at tao.at] 
> Verzonden: dinsdag 5 september 2017 17:13
> Aan: L.P.H. van Belle
> CC: samba at lists.samba.org
> Onderwerp: Re: [Samba] Server GC/name.dom/dom is not 
> registered with our KDC: Miscellaneous failure (see text): 
> Server (GC/name/dom at DOM) unknown
> 
> On 2017-09-05 16:52, L.P.H. van Belle wrote:
> > Yes, if you flexible with reinstalling, you could..
> 
> I don't want another quick and dirty solution that turns out 
> to break half a year down the line, I'm fine with nuking half 
> my DCs if that means getting to a clean state.
No,  i dont want a quick and dirty solution for you. You need to get a good fix.
> 
> Besides, recreating containers is faster than manually 
> messing around in /var/lib on each one of them.
> 
> > I suggest the following, move fsmo roles to villach-dc and 
> check database replications.
> 
> DB replication is already spewing errors, what am I to look out for?
Ok, get my check db script, run it from any dc. And post me the output. 
https://github.com/thctlo/samba4/blob/master/samba-check-db-repl.sh 
With the output, we should be able to see which servers have the best replicated
database.
( the script uses the standaard samba tools, but all in one go) 
> 
> > Remove the most faulty one first, graz-dc-1b, from the 
> domain. ( check 
> > and cleanup DNS and AD! Very important )
> 
> What to check for? What to clean up?
Ah, thats hard to tell, this depends a bit on the errors.
I search/look for the left overs, first with RSAT tools and samba-tool, then
with ApacheStudio.
I look for hostnames/UUID/ things like that, but this is only done if all other
options did not work.
But it depends on the errors/warnings i see/get. 
> 
> > You dont have to reinstall the complete os, just cleanup as 
> told, and reprovisioning that server again. 
> 
> Adding a new DC with the same hostname as the old DC is what 
> got me into trouble last time. I'll pass up on that offer.
Ok, but i know the correct steps to do this, its all in the correct order and
when to remove where/what.
I can save you the time to reinstall the OS, you can re-use the os, just dont
reuse the same hostname.
But, if its not an option for you anymore, thats ok, that what you want. 
> 
> >>> Then remove a failty server and re-add it as a new installed
DC.
> >>> ( the good DS with FSMO)
> >>> First backup: /var/lib/samba/private/secrets.keytab
> >>> Remove the incorrect entries from keytab file with ktutil rkt 
> >>> /var/lib/samba/private/secrets.keytab
> >>> list -e -t
> >>
> >> Might as well just nuke graz-dc-sem and add a complete new DC from
> >> scratch, no?
> > No, and yes, but i preffer no, not needed (yet). 
> > Start with the keytab cleanup
> > Check the dns record if the uuid A PTR and hostnames 
> resolve to the correct server. 
> > If thats the case, then no, cleanup of keytab is, i think, 
> sufficient. 
> 
> Just to confirm the order: Clean up the keytab, if that 
> doesn't work, start removing servers?
Almost. Backup then ...   Cleanup keytab of the server. 
> 
> > Yes, if its really a mess. ;-)
> > Then, first a an new DC, then remove, just make sure you 
> always have 2 
> > dc's up and running (correctly)
> 
> Servers in Villach seem to run fine, thank $DEITY, so I'll 
> leave them alone for now.
Ok, thats good, run the check-db script and post the complete output for me. 
> 
> >>> Now re-provision and you should have correct working DC's
again.
> >>>
> >>> ! Before re-provisioning, make sure all OLD records dns and
> >> AD are gone. 
> >>
> >> I still have undeleteable replication records from the last time I
> >> had to nuke a DC, nobody replied to my emails on that issue.
> > 
> > Ok, now, im out of office in about 10 min, but mail that 
> subject for me again> I'll have a look.
> 
> First message on that topic:
> https://lists.samba.org/archive/samba/2017-March/207429.html
Ok, this one, track down both uuid's, checkout which which hostname belongs
with these.
Basicly https://wiki.samba.org/index.php/Demoting_a_Samba_AD_DC 
In based on the Demoteing wiki example. 
I do the same steps as shown there. Now the last three pictures on the site
shows where to look.
At the site and Service, go through very folder, and check if it is as it should
be.

https://lists.samba.org/archive/samba/2017-March/207460.html
Between 2 and 3, there the problem starts
>>  – I noticed a typo in the server's `netbios name` setting,
corrected it, and restarted the DC.
3. Yes, for the new hostname, the old hostname is as left over in the ADDC DB
and/or DNS.

This name GRAZ-DC-BIS, and the name with the typo. The GUID for these is where
to look info.

Step 6. ah the point of origin of you problems with of the current post? 

7. dnsmasq? Ok, i just hope these are not running on the DC's. 
> Last message, where I mentioned the replication bug:
> https://lists.samba.org/archive/samba/2017-April/207918.html
> 
> > Own and if you dont use it, ApacheDirectoryStudio can help 
> a lot with cleanup of these kind of things. 
> 
> Currently I'm using the ADSI MMC snap-in, any downsides 
> compared to ADS?
I dont know, never used ADS :-/ 

Track done these : GUID's, the hostname's, ipnumbers A and PTR records 
7e4973ba-4093-4523-a70f-7caa4845e34d 
d613fa11-064b-4bcc-ab01-20264f870f47
(how, see Verifying and Creating the objectGUID Record 
https://wiki.samba.org/index.php/Verifying_and_Creating_a_DC_DNS_Record)
And as suggested, do this first through the RSAT tools or samba-tools, try
removing them with these tools first.
(how https://wiki.samba.org/index.php/Demoting_a_Samba_AD_DC) 
Open the Active Directory Users and Computers application and set it to advanced
view.

Now, i think the last left overs can be removed with : Active Directory Sites
and Services and DNS manager.
And now check them all, every folder, take your time. 

Then when its done, i run samba-tool dbcheck again per
server.
> 
> > But just make sure you know what you delete, for you mess 
> up the AD even more. 
> 
> That why I'm not touching anything without a full list.
Yes, good, im pro that, the more info we get the better we can help you. 


Greetz, 

Louis