samba - Sep 2017 - Server GC/name.dom/dom is not registered with our KDC: Miscellaneous failure (see text): Server (GC/name/dom@DOM) unknown

> -----Oorspronkelijk bericht-----
> Van: Sven Schwedas [mailto:sven.schwedas at tao.at] 
> Verzonden: vrijdag 8 september 2017 14:40
> Aan: L.P.H. van Belle; samba at lists.samba.org
> Onderwerp: Re: [Samba] Server GC/name.dom/dom is not 
> registered with our KDC: Miscellaneous failure (see text): 
> Server (GC/name/dom at DOM) unknown
> 
> On 2017-09-08 10:37, L.P.H. van Belle wrote:
> > Hai Sven,
> > 
> > Sorry for the long wait, i had some things todo first here.
> 
> 's fine. It's not like I don't have other stuff to worry about
either.
Pfew, im thinging your just waiting for me ;-) 
> 
> >> -----Oorspronkelijk bericht-----
> >>>>> I suggest the following, move fsmo roles to villach-dc
and
> >>>> check database replications.
> >>>>
> >>>> DB replication is already spewing errors, what am I to
look out
> >>>> for?
> >>> Ok, get my check db script, run it from any dc. And post me
the
> >>> output.
> >>
> >> Output attached. Seems like all servers but graz-dc-sem have some 
> >> stuff wrong, just wonderful.
> > Ok base on the log,
> > graz-dc-sem.ad.tao.at is the most complete/correct server. 
> 
> And also the one with the keytab problems. :(
Ok, yes, but wait, read on... 
> 
> >>>>>>> Then remove a failty server and re-add it as a
new
> installed DC.
> >>>>>>> ( the good DS with FSMO)
> >>>>>>> First backup:
/var/lib/samba/private/secrets.keytab
> >>>>>>> Remove the incorrect entries from keytab file
with ktutil rkt
> >>>>>>> /var/lib/samba/private/secrets.keytab
> >>>>>>> list -e -t
> >>>>>>
> >>>>> Start with the keytab cleanup
> >>>>> Check the dns record if the uuid A PTR and hostnames
resolve to
> >>>>> the correct server.
> >>>>> If thats the case, then no, cleanup of keytab is, i
think,
> >>>>> sufficient.
> >>>>
> >>>> Just to confirm the order: Clean up the keytab, if that
doesn't
> >>>> work, start removing servers?
> >>> Almost. Backup then ...   Cleanup keytab of the server. 
> >>
> >> I'll try that next, unless you find something shady in the
report.
> 
> While going through the records with Directory Studio I 
> noticed that the duplicate keytab entries are mirrored by 
> servicePrincipalName entries in CN=GRAZ-DC-SEM,OU=Domain 
> Controllers,DC=ad,DC=tao,DC=at
> 
> Which should I clean up, and in what order? LDAP first, then keytab?
> Just LDAP?
Now, only remove the incorrect ones from : servicePrincipalName 
You can do this also very easy from withing RSAT "User and Domain
manager"
Enable advanced view. 
Right klik on the computer object, properties goto tab: Attribute Editor

servicePrincipalName
edit and remove the failty ones 
And ok, exit. 

! Dont add there. (not now, first fix everything. ) 
> 
> >>>> First message on that topic:
> >>>>
https://lists.samba.org/archive/samba/2017-March/207429.html
> >>> Ok, this one, track down both uuid's, checkout which 
> which hostname 
> >>> belongs with these.
> >>
> >> Going by the output of `samba-tool drs showrepl` and sam.ldb, the 
> >> GUIDs are as follows:
> >>
> >> graz-dc-sem	160f5a53-5c29-4a83-aeee-6cb1dbabeed7
> >> graz-dc-bis	7e4973ba-4093-4523-a70f-7caa4845e34d 
> (not in sam.ldb)
> >> graz-dc-1b	bcffbad8-1add-46b9-bf69-90e52c0f09ea
> >> villach-dc-sem	eb5f9772-cd8f-4cde-9629-f1898c94aedd
> >> villach-dc-bis	e1569c90-50f9-4bb5-bd85-79145e3ff6fd
> >>
> >> villach-sem/bis want to replicate to the dead and removed 
> graz-dc-bis 
> >> for some reason, but don't have its GUID in their sam.ldb
either.
> > Ok, can you do a manualy check on graz-dc-sem and 
> villach-dc-sem Both 
> > command on each server.
> > samba-tool dbcheck
> > And
> > samba-tool dbcheck --cross-nc
> > 
> > Error, post them. 
> 
> About that? villach-dc-sem is down because its VM host went 
> poof yesterday. :/ Gonna get it repaired at some point next week.
> 
Option. 
Well this is you change... Get it, setup a clean server. 
Join the domain, Seize the FSMO roles after AD syncs are done. 
Push this AD DB to the other servers. s
> On graz-dc-sem, samba-tool dbcheck shows zero errors. 
> --cross-nc gives:
> 
> > Error: governsID 
> > CN=ucsUser,CN=Schema,CN=Configuration,DC=ad,DC=tao,DC=at on 
> > 1.3.6.1.4.1.19414.3.2.2 already exists as an attributeId or 
> governsId
> > Error: governsID 
> > CN=taoSharedFolder,CN=Schema,CN=Configuration,DC=ad,DC=tao,DC=at on 
> > 1.3.6.1.4.1.19414.3.2.4 already exists as an attributeId or 
> governsId
> > Error: governsID 
> > CN=taoMailingList,CN=Schema,CN=Configuration,DC=ad,DC=tao,DC=at on 
> > 1.3.6.1.4.1.19414.3.2.3 already exists as an attributeId or 
> governsId
> 
> Those are presumably from me fucking around with custom LDAP 
> attributes years ago and don't seem to be related. I hope.
I hope so to, i dont know that yet, but are these sill used somewere? 
If not, run safely : samba-tool dbcheck --cross-nc --fix 
If they are used, well, run it also, but prepair for extra (possible) problems.
> 
> >>>>>  ? I noticed a typo in the server's `netbios name`
> >> setting, corrected it, and restarted the DC. 
> >>> 3. Yes, for the new hostname, the old hostname is as left
> >> over in the ADDC DB and/or DNS. 
> >>>
> >>> This name GRAZ-DC-BIS, and the name with the typo. The GUID
> >> for these is where to look info. 
> >>
> >> Yeah, but where?
> > Now this is here ApacheStudio is very handy, but again, 
> dont rush things here. 
> > With ApacheStu.. Connect to the AD, and search for the faulty 
> > GUID/UUID's Same for the hostname.
> > 
> > Ow did i tell you to make a backup first..  ;-) make it 2. ;-)
> 
> No results for either the old name, nor the new name, nor the 
> GUID, on any server.
> 
> How is `drs showrepl` giving me the UUID for something that's 
> nowhere in LDAP?
Huh, yeah thats very strange, then, its still there, 
You should find it somewhere, i think your search is off, you did set the BASE
DN also?

And i forgot this one : samba-tool domain tombstones expunge
That cleanup the AD also from deleted items. 

> 
> > Now, last, which is an option. 
> > If one server hold a good and correct database, you can 
> sync that one to the other servers. 
> > But the database must be correct. ( and backuped 2 x )
> 
> So
> 
> 1. Back up graz-dc-sem
> 2. purge the duplicate keytab/SPN magic voodoo somehow (see 
> above) 
> 3. Force sync?
Now since im having a hardtime myself here with a keytab problem.  
I think a bug due GnPG2 still investigating it.  ( kerberos nfs and AES key
errors )
I suggest, we ask a dev the best way to cleanup the keytab file. 
I have several ways, but i want to know what the devs say also about this. 
And fideling around with keytabs on a member server is imo more easy then the AD
DC.


The "force sync" 
samba-tool drs replicate DEST SOURCE DN --full-sync is what i have in mind. 
So you can since the best database to the other servers. 

> 
> > samba-tool dbcheck --help gives also few extra options but 
> i never used them so i cant tell much of these are usefull. 
> >   --reindex             force database re-index 
> >   --reset-well-known-acls
> >   --cross-ncs
> 
> --
> Mit freundlichen Grüßen, / Best Regards, Sven Schwedas, 
> Systemadministrator Mail/XMPP sven.schwedas at tao.at | Skype 
> sven.schwedas TAO Digital | Lendplatz 45 | A8020 Graz 
> https://www.tao-digital.at | Tel +43 680 301 7167

Greetz, 

Louis

Okay, time to bring some order into this.


== Keytab and replication =
Editing servicePrincipalName cleaned up the keytab. Some things are even
replicating correctly (modulo the missing host), others aren't.

DNS replication (DC=ForestDnsZones and DC=DomainDnsZones) is still not
working, at least one host is trying to connect to deleted servers for
this. The other I cannot confirm because it's still offline.

I'll check again once the remaining DC is back online.

In the meantime it'd be nice to know where exactly Samba ADDCs store
their replication config so I can see why they have outdated garbage in
them, even though the RSAT tools claim everything is working and
configured correctly.


== governsID errors during dbcheck =
--fix doesn't do anything; the error still remains. How do I fix it
manually?


On 2017-09-08 16:34, L.P.H. van Belle via samba wrote:
> 
> 
>> -----Oorspronkelijk bericht----- Van: Sven Schwedas
>> [mailto:sven.schwedas at tao.at] Verzonden: vrijdag 8 september 2017
>> 14:40 Aan: L.P.H. van Belle; samba at lists.samba.org Onderwerp: Re:
>> [Samba] Server GC/name.dom/dom is not registered with our KDC:
>> Miscellaneous failure (see text): Server (GC/name/dom at DOM) unknown
>> 
>> On 2017-09-08 10:37, L.P.H. van Belle wrote:
>>> Hai Sven,
>>> 
>>> Sorry for the long wait, i had some things todo first here.
>> 
>> 's fine. It's not like I don't have other stuff to worry
about
>> either.
> Pfew, im thinging your just waiting for me ;-)
> 
>> 
>>>> -----Oorspronkelijk bericht-----
>>>>>>> I suggest the following, move fsmo roles to
villach-dc
>>>>>>> and
>>>>>> check database replications.
>>>>>> 
>>>>>> DB replication is already spewing errors, what am I to
look
>>>>>> out for?
>>>>> Ok, get my check db script, run it from any dc. And post me
>>>>> the output.
>>>> 
>>>> Output attached. Seems like all servers but graz-dc-sem have
>>>> some stuff wrong, just wonderful.
>>> Ok base on the log, graz-dc-sem.ad.tao.at is the most
>>> complete/correct server.
>> 
>> And also the one with the keytab problems. :(
> Ok, yes, but wait, read on...
> 
>> 
>>>>>>>>> Then remove a failty server and re-add it
as a new
>> installed DC.
>>>>>>>>> ( the good DS with FSMO) First backup:
>>>>>>>>> /var/lib/samba/private/secrets.keytab
Remove the
>>>>>>>>> incorrect entries from keytab file with
ktutil rkt
>>>>>>>>> /var/lib/samba/private/secrets.keytab list
-e -t
>>>>>>>> 
>>>>>>> Start with the keytab cleanup Check the dns record
if the
>>>>>>> uuid A PTR and hostnames resolve to the correct
server.
>>>>>>> If thats the case, then no, cleanup of keytab is, i
>>>>>>> think, sufficient.
>>>>>> 
>>>>>> Just to confirm the order: Clean up the keytab, if that
>>>>>> doesn't work, start removing servers?
>>>>> Almost. Backup then ...   Cleanup keytab of the server.
>>>> 
>>>> I'll try that next, unless you find something shady in the
>>>> report.
>> 
>> While going through the records with Directory Studio I noticed
>> that the duplicate keytab entries are mirrored by 
>> servicePrincipalName entries in CN=GRAZ-DC-SEM,OU=Domain 
>> Controllers,DC=ad,DC=tao,DC=at
>> 
>> Which should I clean up, and in what order? LDAP first, then
>> keytab? Just LDAP?
> Now, only remove the incorrect ones from : servicePrincipalName You
> can do this also very easy from withing RSAT "User and Domain
> manager" Enable advanced view. Right klik on the computer object,
> properties goto tab: Attribute Editor
> 
> servicePrincipalName edit and remove the failty ones And ok, exit.
> 
> ! Dont add there. (not now, first fix everything. )
> 
>> 
>>>>>> First message on that topic: 
>>>>>>
https://lists.samba.org/archive/samba/2017-March/207429.html
>>>>>
>>>>>> 
Ok, this one, track down both uuid's, checkout which
>> which hostname
>>>>> belongs with these.
>>>> 
>>>> Going by the output of `samba-tool drs showrepl` and sam.ldb,
>>>> the GUIDs are as follows:
>>>> 
>>>> graz-dc-sem	160f5a53-5c29-4a83-aeee-6cb1dbabeed7 graz-dc-bis
>>>> 7e4973ba-4093-4523-a70f-7caa4845e34d
>> (not in sam.ldb)
>>>> graz-dc-1b	bcffbad8-1add-46b9-bf69-90e52c0f09ea villach-dc-sem
>>>> eb5f9772-cd8f-4cde-9629-f1898c94aedd villach-dc-bis
>>>> e1569c90-50f9-4bb5-bd85-79145e3ff6fd
>>>> 
>>>> villach-sem/bis want to replicate to the dead and removed
>> graz-dc-bis
>>>> for some reason, but don't have its GUID in their sam.ldb
>>>> either.
>>> Ok, can you do a manualy check on graz-dc-sem and
>> villach-dc-sem Both
>>> command on each server. samba-tool dbcheck And samba-tool dbcheck
>>> --cross-nc
>>> 
>>> Error, post them.
>> 
>> About that? villach-dc-sem is down because its VM host went poof
>> yesterday. :/ Gonna get it repaired at some point next week.
>> 
> Option. Well this is you change... Get it, setup a clean server. Join
> the domain, Seize the FSMO roles after AD syncs are done. Push this
> AD DB to the other servers. s
> 
>> On graz-dc-sem, samba-tool dbcheck shows zero errors. --cross-nc
>> gives:
>> 
>>> Error: governsID 
>>> CN=ucsUser,CN=Schema,CN=Configuration,DC=ad,DC=tao,DC=at on 
>>> 1.3.6.1.4.1.19414.3.2.2 already exists as an attributeId or
>> governsId
>>> Error: governsID 
>>> CN=taoSharedFolder,CN=Schema,CN=Configuration,DC=ad,DC=tao,DC=at
>>> on 1.3.6.1.4.1.19414.3.2.4 already exists as an attributeId or
>> governsId
>>> Error: governsID 
>>> CN=taoMailingList,CN=Schema,CN=Configuration,DC=ad,DC=tao,DC=at
>>> on 1.3.6.1.4.1.19414.3.2.3 already exists as an attributeId or
>> governsId
>> 
>> Those are presumably from me fucking around with custom LDAP 
>> attributes years ago and don't seem to be related. I hope.
> I hope so to, i dont know that yet, but are these sill used somewere?
>  If not, run safely : samba-tool dbcheck --cross-nc --fix If they are
> used, well, run it also, but prepair for extra (possible) problems.
> 
>> 
>>>>>>> ? I noticed a typo in the server's `netbios
name`
>>>> setting, corrected it, and restarted the DC.
>>>>> 3. Yes, for the new hostname, the old hostname is as left
>>>> over in the ADDC DB and/or DNS.
>>>>> 
>>>>> This name GRAZ-DC-BIS, and the name with the typo. The GUID
>>>> for these is where to look info.
>>>> 
>>>> Yeah, but where?
>>> Now this is here ApacheStudio is very handy, but again,
>> dont rush things here.
>>> With ApacheStu.. Connect to the AD, and search for the faulty 
>>> GUID/UUID's Same for the hostname.
>>> 
>>> Ow did i tell you to make a backup first..  ;-) make it 2. ;-)
>> 
>> No results for either the old name, nor the new name, nor the GUID,
>> on any server.
>> 
>> How is `drs showrepl` giving me the UUID for something that's 
>> nowhere in LDAP?
> 
> Huh, yeah thats very strange, then, its still there, You should find
> it somewhere, i think your search is off, you did set the BASE DN
> also?
> 
> And i forgot this one : samba-tool domain tombstones expunge That
> cleanup the AD also from deleted items.
> 
> 
>> 
>>> Now, last, which is an option. If one server hold a good and
>>> correct database, you can
>> sync that one to the other servers.
>>> But the database must be correct. ( and backuped 2 x )
>> 
>> So
>> 
>> 1. Back up graz-dc-sem 2. purge the duplicate keytab/SPN magic
>> voodoo somehow (see above) 3. Force sync?
> 
> Now since im having a hardtime myself here with a keytab problem. I
> think a bug due GnPG2 still investigating it.  ( kerberos nfs and AES
> key errors ) I suggest, we ask a dev the best way to cleanup the
> keytab file. I have several ways, but i want to know what the devs
> say also about this. And fideling around with keytabs on a member
> server is imo more easy then the AD DC.
> 
> 
> The "force sync" samba-tool drs replicate DEST SOURCE DN
--full-sync
> is what i have in mind. So you can since the best database to the
> other servers.
> 
> 
>> 
>>> samba-tool dbcheck --help gives also few extra options but
>> i never used them so i cant tell much of these are usefull.
>>> --reindex             force database re-index 
>>> --reset-well-known-acls --cross-ncs
>> 
>> -- Mit freundlichen Grüßen, / Best Regards, Sven Schwedas, 
>> Systemadministrator Mail/XMPP sven.schwedas at tao.at | Skype 
>> sven.schwedas TAO Digital | Lendplatz 45 | A8020 Graz 
>> https://www.tao-digital.at | Tel +43 680 301 7167
> 
> 
> Greetz,
> 
> Louis
> 
> 
-- 
Mit freundlichen Grüßen, / Best Regards,
Sven Schwedas, Systemadministrator
Mail/XMPP sven.schwedas at tao.at | Skype sven.schwedas
TAO Digital | Lendplatz 45 | A8020 Graz
https://www.tao-digital.at | Tel +43 680 301 7167

On Mon, 11 Sep 2017 14:32:53 +0200
Sven Schwedas via samba <samba at lists.samba.org> wrote:
> Okay, time to bring some order into this.
> 
> 
> == Keytab and replication => 
> Editing servicePrincipalName cleaned up the keytab. Some things are
> even replicating correctly (modulo the missing host), others aren't.
> 
> DNS replication (DC=ForestDnsZones and DC=DomainDnsZones) is still not
> working, at least one host is trying to connect to deleted servers for
> this. The other I cannot confirm because it's still offline.
> 
> I'll check again once the remaining DC is back online.
> 
> In the meantime it'd be nice to know where exactly Samba ADDCs store
> their replication config so I can see why they have outdated garbage
> in them, even though the RSAT tools claim everything is working and
> configured correctly.
> 
> 
Everything is stored in sam.ldb, but by default, most records are not
shown by ldbsearch etc
To show all the records, use '--cross-ncs'

Rowland