samba - Feb 2023 - Replication between Samba DCs (on different sites)?

07.02.2023 21:44, Rowland Penny via samba ?????:
..
> If DC and dc3 are working correctly, I would demote DC2 (forcibly if
necessary) then seize the FSMO roles to one of the good DC's. Once you are
sure
> that your domain is working correctly, find anything to do with the dead DC
in AD and remove it. Now add a new DC (I would use a new name and IP),
> hopefully everything should be okay after all that.
Rowland, this happened at least 3 times already, all after upgrade.
I really regret that I followed your suggestion and did that force-thing,
but at that time I didn't know it will be a common problem.  At least I
were able to try to figure out what's going on. Yes I need help from
someone who understands this part of samba and/or protocols involved,
but I sure am able to perform quite some debugging without requiring
to be babysitted with every step. Now once I "fixed" my situation with
force, I don't have that issue anymore and can't find out what was
actually wrong.

The problem here is common and it will be much more common once people
start upgrading. Maybe there's a bug in debian packaging, maybe it is
a prob with samba code, maybe something  else, I dunno.

Lorenzo seems to be able to do some basic debugging too.. maybe we
can use this opportunity and try to understand what is going on,
instead of using the force?

Thanks,

/mjt

> Lorenzo seems to be able to do some basic debugging too.. maybe we
> can use this opportunity and try to understand what is going on,
> instead of using the force?
Inspired by those kind words (:D), I tried investigating further... Indeed,
I'd rather find a solution than wiping dc2. At the moment users are
authenticating correctly, I have some time. I asked to avoid any administration
task on the domain in the meantime.


root at dc2:~# ldbsearch --cross-ncs -H /var/lib/samba/private/sam.ldb -b
DC=DomainDnsZones,DC=wdc,DC=domain,DC=it '(cn=Infrastructure)'
# record 1
dn: CN=Infrastructure,DC=DomainDnsZones,DC=wdc,DC=domain,DC=it
objectClass: top
objectClass: infrastructureUpdate
cn: Infrastructure
instanceType: 4
whenCreated: 20200723054831.0Z
uSNCreated: 5712
showInAdvancedViewOnly: TRUE
name: Infrastructure
objectGUID: 21b49376-474e-481b-b18d-0062e980b1f3
systemFlags: -1946157056
objectCategory:
CN=Infrastructure-Update,CN=Schema,CN=Configuration,DC=wdc,DC=domain,DC=it
isCriticalSystemObject: TRUE
whenChanged: 20230207165009.0Z
uSNChanged: 6269
fSMORoleOwner: CN=NTDS
Settings,CN=DC2,CN=Servers,CN=ARUBA-DataCenter1,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
distinguishedName: CN=Infrastructure,DC=DomainDnsZones,DC=wdc,DC=domain,DC=it

# returned 1 records
# 1 entries
# 0 referrals

Same output is returned on DC1:
root at dc1:~# ldbsearch --cross-ncs -H /var/lib/samba/private/sam.ldb -b
DC=DomainDnsZones,DC=wdc,DC=domain,DC=it '(cn=Infrastructure)'
# record 1
dn: CN=Infrastructure,DC=DomainDnsZones,DC=wdc,DC=domain,DC=it
objectClass: top
objectClass: infrastructureUpdate
cn: Infrastructure
instanceType: 4
whenCreated: 20200723054831.0Z
whenChanged: 20230207165009.0Z
uSNCreated: 5834
uSNChanged: 5834
showInAdvancedViewOnly: TRUE
name: Infrastructure
objectGUID: 21b49376-474e-481b-b18d-0062e980b1f3
fSMORoleOwner: CN=NTDS
Settings,CN=DC2,CN=Servers,CN=ARUBA-DataCenter1,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
systemFlags: -1946157056
objectCategory:
CN=Infrastructure-Update,CN=Schema,CN=Configuration,DC=wdc,DC=domain,DC=it
isCriticalSystemObject: TRUE
distinguishedName: CN=Infrastructure,DC=DomainDnsZones,DC=wdc,DC=domain,DC=it

# returned 1 records
# 1 entries
# 0 referrals

I found this[1] thread from last month, unfortunately without a solution, where
Rowland talks about possible issues with DC's site. So I moved dc2 to
"Default-First-Site-Name" and attempted roles transfer, and IT WORKED:

root at dc1:~# samba-tool fsmo transfer --role=all -U administrator
This DC already has the 'rid' FSMO role
This DC already has the 'pdc' FSMO role
This DC already has the 'naming' FSMO role
This DC already has the 'infrastructure' FSMO role
This DC already has the 'schema' FSMO role
Password for [WDC\administrator]:
FSMO transfer of 'domaindns' role successful
FSMO transfer of 'forestdns' role successful

root at dc1:~# samba-tool fsmo show
SchemaMasterRole owner: CN=NTDS
Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
InfrastructureMasterRole owner: CN=NTDS
Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
RidAllocationMasterRole owner: CN=NTDS
Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
PdcEmulationMasterRole owner: CN=NTDS
Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
DomainNamingMasterRole owner: CN=NTDS
Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
DomainDnsZonesMasterRole owner: CN=NTDS
Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
ForestDnsZonesMasterRole owner: CN=NTDS
Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it


Now all DCs reports the roles managed by dc1.

Replication is still not working on dc2:
root at dc2~# samba-tool drs replicate dc2 dc1
DC=ForestDnsZones,DC=wdc,DC=domain,DC=it
ERROR(<class 'samba.drs_utils.drsException'>): DsReplicaSync
failed - drsException: DsReplicaSync failed (2, 'WERR_FILE_NOT_FOUND')
  File "/usr/lib/python3/dist-packages/samba/netcmd/drs.py", line 570,
in run
    drs_utils.sendDsReplicaSync(server_bind, server_bind_handle,
source_dsa_guid, NC, req_options)
  File "/usr/lib/python3/dist-packages/samba/drs_utils.py", line 100,
in sendDsReplicaSync
    raise drsException("DsReplicaSync failed %s" % estr)


So I should now have a sane domain, correct me if I'm wrong.
I just need to figure out what to do with dc2. I can hold until tomorrow, if it
can be useful for debugging, but my abilities are limited on this.


[1] https://lists.samba.org/archive/samba/2023-January/243664.html
-- 
Lorenzo Milesi - lorenzo.milesi at yetopen.com 
CTO @ YetOpen Srl

Corso Martiri della Liberazione 114 - 23900 Lecco - ITALY - | 4801 Glenwood
Avenue - Suite 200 - Raleigh, NC 27612 - USA -
Tel +39 0341 220 205 - info.it at yetopen.com  | Phone +1 919-817-8106 - info.us
at yetopen.com

Think green - Non stampare questa e-mail se non necessario / Don't print
this email unless necessary

-------- D.Lgs. 196/2003 e GDPR 679/2016 --------
Tutte le informazioni contenute in questo messaggio sono riservate ed a uso
esclusivo del destinatario.
Tutte le informazioni ivi contenute, compresi eventuali allegati, sono da
ritenere confidenziali e riservate secondo i termini
del vigente D.Lgs. 196/2003 in materia di privacy e del Regolamento europeo
679/2016 - GDPR - e quindi ne e' proibita l'utilizzazione ulteriore non
autorizzata.
Nel caso in cui questo messaggio Le fosse pervenuto per errore, La invitiamo ad
eliminarlo senza copiarlo, stamparlo, a non inoltrarlo a terzi e ad avvertirci
non appena possibile.
Grazie.

Confidentiality notice: this email message including any attachment is for the
sole use of the intended recipient and may contain confidential and privileged
information;
pursuant to Legislative Decree 196/2003 and the European General Data Protection
Regulation 679/2016 - GDPR - any unauthorized review, use, disclosure or
distribution
is prohibited. If you are not the intended recepient please delete this message
without copying, printing or forwarding it to others, and alert us as soon as
possible.
Thank you.

On 07/02/2023 22:47, Michael Tokarev via samba wrote:
> 07.02.2023 21:44, Rowland Penny via samba ?????:
> ..
> 
>> If DC and dc3 are working correctly, I would demote DC2 (forcibly if 
>> necessary) then seize the FSMO roles to one of the good DC's. Once
you
>> are sure that your domain is working correctly, find anything to do 
>> with the dead DC in AD and remove it. Now add a new DC (I would use a 
>> new name and IP), hopefully everything should be okay after all that.
> 
> Rowland, this happened at least 3 times already, all after upgrade.
> I really regret that I followed your suggestion and did that force-thing,
> but at that time I didn't know it will be a common problem.? At least I
> were able to try to figure out what's going on. Yes I need help from
> someone who understands this part of samba and/or protocols involved,
> but I sure am able to perform quite some debugging without requiring
> to be babysitted with every step. Now once I "fixed" my situation
with
> force, I don't have that issue anymore and can't find out what was
> actually wrong.
> 
> The problem here is common and it will be much more common once people
> start upgrading. Maybe there's a bug in debian packaging, maybe it is
> a prob with samba code, maybe something? else, I dunno.
> 
> Lorenzo seems to be able to do some basic debugging too.. maybe we
> can use this opportunity and try to understand what is going on,
> instead of using the force?
> 
> Thanks,
> 
> /mjt
> 
Anything is possible and yes, if there is a bug, it needs fixing. It is 
just that because of the nature of AD, it is sometimes quicker to just 
remove a faulty DC and add a new one.

Rowland