Lorenzo Milesi
2023-Mar-08 11:07 UTC
[Samba] Unable to "rejoin" existing DC after upgrade (infamous WERR_FILE_NOT_FOUND)
Hi. As happened some weeks ago, here I am again updating an old Samba 4.14.x network to a current version. The server hosting the FSMO roles is a Debian10 with 4.14.14, while the third node is a Ubuntu 18 running LinuxSchools build 4.14.8. We started from a Ubuntu 20.04 server running Louis builds. We demoted the node and joined it back to the domain with 4.17.5 from Michael. Although on the node itself everything seemed ok, the DC didn't appear in DNS, while visible in Sites and ADUC. Replication is reported as ALL GOOD on the upgraded node, but the remote ones are in error with WERR_FILE_NOT_FOUND. We enabled drs_repl log on the 4.14.8, pasting below. It seems failing because it cannot find the DNS records, which it cannot have because replication is not working. If I run dig 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan @zimbraip it returns the correct value, while the same command against any of the two other DCs fails. But maybe this is not the root cause of the problem. samba_dnsupdate ran without errors on the 4.17 node, but the other DCs never received those DNS records. What else can we check? thanks # conf on 4.14.8 [global] netbios name = ZIMBRA realm = DOMAIN.LAN server role = active directory domain controller workgroup = DOM server services = -dns allow dns updates = disabled interfaces = tun0 lo log level = 1 drs_repl:10 [sysvol] path = /var/lib/samba/sysvol read only = No [netlogon] path = /var/lib/samba/sysvol/domain.lan/scripts read only = No # conf on newly upgraded 4.17.5 [global] interfaces = tun0 lo netbios name = NEXTCLOUD realm = DOMAIN.LAN server role = active directory domain controller workgroup = DOM log level = 1 drs_repl:10 [sysvol] path = /var/lib/samba/sysvol read only = No [netlogon] path = /var/lib/samba/sysvol/domain.lan/scripts read only = No # log excerpt from 4.14.8 - zimbra node [2023/03/08 11:53:48.764405, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:391(dreplsrv_notify_check) dreplsrv_notify_check: queued DsReplicaSync for DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan (urgent=true) uSN=0:27662 [2023/03/08 11:53:48.764675, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:391(dreplsrv_notify_check) dreplsrv_notify_check: queued DsReplicaSync for CN=Schema,CN=Configuration,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan (urgent=false) uSN=0:26357 [2023/03/08 11:53:48.764748, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:247(dreplsrv_notify_run_ops) dreplsrv_notify_run_ops: started DsReplicaSync for DC=DomainDnsZones,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan [2023/03/08 11:53:48.764766, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:467(dreplsrv_notify_schedule) dreplsrv_notify_schedule: dreplsrv_notify_schedule(5) scheduled for: Wed Mar 8 11:53:54 2023 CET [2023/03/08 11:53:48.809115, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:199(dreplsrv_notify_op_callback) dreplsrv_notify_op_callback: dreplsrv_notify: Failed to send DsReplicaSync to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan for DC=DomainDnsZones,DC=domain,DC=lan - NT_STATUS_OBJECT_NAME_NOT_FOUND : WERR_FILE_NOT_FOUND [2023/03/08 11:53:48.813994, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:247(dreplsrv_notify_run_ops) dreplsrv_notify_run_ops: started DsReplicaSync for CN=Configuration,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan [2023/03/08 11:53:48.850673, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:199(dreplsrv_notify_op_callback) dreplsrv_notify_op_callback: dreplsrv_notify: Failed to send DsReplicaSync to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan for CN=Configuration,DC=domain,DC=lan - NT_STATUS_OBJECT_NAME_NOT_FOUND : WERR_FILE_NOT_FOUND [2023/03/08 11:53:48.855475, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:247(dreplsrv_notify_run_ops) dreplsrv_notify_run_ops: started DsReplicaSync for DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan [2023/03/08 11:53:48.893889, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:199(dreplsrv_notify_op_callback) dreplsrv_notify_op_callback: dreplsrv_notify: Failed to send DsReplicaSync to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan for DC=domain,DC=lan - NT_STATUS_OBJECT_NAME_NOT_FOUND : WERR_FILE_NOT_FOUND [2023/03/08 11:53:48.899875, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:247(dreplsrv_notify_run_ops) dreplsrv_notify_run_ops: started DsReplicaSync for CN=Schema,CN=Configuration,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan [2023/03/08 11:53:48.938349, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:199(dreplsrv_notify_op_callback) dreplsrv_notify_op_callback: dreplsrv_notify: Failed to send DsReplicaSync to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan for CN=Schema,CN=Configuration,DC=domain,DC=lan - NT_STATUS_OBJECT_NAME_NOT_FOUND : WERR_FILE_NOT_FOUND [2023/03/08 11:53:53.771482, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:391(dreplsrv_notify_check) dreplsrv_notify_check: queued DsReplicaSync for DC=DomainDnsZones,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan (urgent=false) uSN=0:27611 [2023/03/08 11:53:53.771874, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:391(dreplsrv_notify_check) dreplsrv_notify_check: queued DsReplicaSync for CN=Configuration,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan (urgent=true) uSN=0:27663 [2023/03/08 11:53:53.772186, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:391(dreplsrv_notify_check) dreplsrv_notify_check: queued DsReplicaSync for DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan (urgent=true) uSN=0:27662 [2023/03/08 11:53:53.772458, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:391(dreplsrv_notify_check) dreplsrv_notify_check: queued DsReplicaSync for CN=Schema,CN=Configuration,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan (urgent=false) uSN=0:26357 [2023/03/08 11:53:53.772526, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:247(dreplsrv_notify_run_ops) dreplsrv_notify_run_ops: started DsReplicaSync for DC=DomainDnsZones,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan [2023/03/08 11:53:53.772544, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:467(dreplsrv_notify_schedule) dreplsrv_notify_schedule: dreplsrv_notify_schedule(5) scheduled for: Wed Mar 8 11:53:59 2023 CET [2023/03/08 11:53:53.811887, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:199(dreplsrv_notify_op_callback) dreplsrv_notify_op_callback: dreplsrv_notify: Failed to send DsReplicaSync to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan for DC=DomainDnsZones,DC=domain,DC=lan - NT_STATUS_OBJECT_NAME_NOT_FOUND : WERR_FILE_NOT_FOUND [2023/03/08 11:53:53.817975, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:247(dreplsrv_notify_run_ops) dreplsrv_notify_run_ops: started DsReplicaSync for CN=Configuration,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan [2023/03/08 11:53:53.856681, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:199(dreplsrv_notify_op_callback) dreplsrv_notify_op_callback: dreplsrv_notify: Failed to send DsReplicaSync to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan for CN=Configuration,DC=domain,DC=lan - NT_STATUS_OBJECT_NAME_NOT_FOUND : WERR_FILE_NOT_FOUND [2023/03/08 11:53:53.862628, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:247(dreplsrv_notify_run_ops) dreplsrv_notify_run_ops: started DsReplicaSync for DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan [2023/03/08 11:53:53.901447, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:199(dreplsrv_notify_op_callback) dreplsrv_notify_op_callback: dreplsrv_notify: Failed to send DsReplicaSync to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan for DC=domain,DC=lan - NT_STATUS_OBJECT_NAME_NOT_FOUND : WERR_FILE_NOT_FOUND [2023/03/08 11:53:53.909327, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:247(dreplsrv_notify_run_ops) dreplsrv_notify_run_ops: started DsReplicaSync for CN=Schema,CN=Configuration,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan [2023/03/08 11:53:53.958106, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:199(dreplsrv_notify_op_callback) dreplsrv_notify_op_callback: dreplsrv_notify: Failed to send DsReplicaSync to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan for CN=Schema,CN=Configuration,DC=domain,DC=lan - NT_STATUS_OBJECT_NAME_NOT_FOUND : WERR_FILE_NOT_FOUND [2023/03/08 11:53:58.779502, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:391(dreplsrv_notify_check) dreplsrv_notify_check: queued DsReplicaSync for DC=DomainDnsZones,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan (urgent=false) uSN=0:27611 [2023/03/08 11:53:58.779901, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:391(dreplsrv_notify_check) dreplsrv_notify_check: queued DsReplicaSync for CN=Configuration,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan (urgent=true) uSN=0:27663 [2023/03/08 11:53:58.780205, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:391(dreplsrv_notify_check) dreplsrv_notify_check: queued DsReplicaSync for DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan (urgent=true) uSN=0:27662 [2023/03/08 11:53:58.780522, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:391(dreplsrv_notify_check) dreplsrv_notify_check: queued DsReplicaSync for CN=Schema,CN=Configuration,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan (urgent=false) uSN=0:26357 [2023/03/08 11:53:58.780594, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:247(dreplsrv_notify_run_ops) dreplsrv_notify_run_ops: started DsReplicaSync for DC=DomainDnsZones,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan [2023/03/08 11:53:58.780658, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:467(dreplsrv_notify_schedule) dreplsrv_notify_schedule: dreplsrv_notify_schedule(5) scheduled for: Wed Mar 8 11:54:04 2023 CET [2023/03/08 11:53:58.823771, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:199(dreplsrv_notify_op_callback) dreplsrv_notify_op_callback: dreplsrv_notify: Failed to send DsReplicaSync to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan for DC=DomainDnsZones,DC=domain,DC=lan - NT_STATUS_OBJECT_NAME_NOT_FOUND : WERR_FILE_NOT_FOUND [2023/03/08 11:53:58.829798, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:247(dreplsrv_notify_run_ops) dreplsrv_notify_run_ops: started DsReplicaSync for CN=Configuration,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan [2023/03/08 11:53:58.874287, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:199(dreplsrv_notify_op_callback) dreplsrv_notify_op_callback: dreplsrv_notify: Failed to send DsReplicaSync to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan for CN=Configuration,DC=domain,DC=lan - NT_STATUS_OBJECT_NAME_NOT_FOUND : WERR_FILE_NOT_FOUND [2023/03/08 11:53:58.879552, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:247(dreplsrv_notify_run_ops) dreplsrv_notify_run_ops: started DsReplicaSync for DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan [2023/03/08 11:53:58.918491, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:199(dreplsrv_notify_op_callback) dreplsrv_notify_op_callback: dreplsrv_notify: Failed to send DsReplicaSync to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan for DC=domain,DC=lan - NT_STATUS_OBJECT_NAME_NOT_FOUND : WERR_FILE_NOT_FOUND [2023/03/08 11:53:58.927320, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:247(dreplsrv_notify_run_ops) dreplsrv_notify_run_ops: started DsReplicaSync for CN=Schema,CN=Configuration,DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan [2023/03/08 11:53:58.965872, 5, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:199(dreplsrv_notify_op_callback) dreplsrv_notify_op_callback: dreplsrv_notify: Failed to send DsReplicaSync to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan for CN=Schema,CN=Configuration,DC=domain,DC=lan - NT_STATUS_OBJECT_NAME_NOT_FOUND : WERR_FILE_NOT_FOUND -- Lorenzo Milesi - lorenzo.milesi at yetopen.com CTO @ YetOpen Srl Corso Martiri della Liberazione 114 - 23900 Lecco - ITALY - | 4801 Glenwood Avenue - Suite 200 - Raleigh, NC 27612 - USA - Tel +39 0341 220 205 - info.it at yetopen.com | Phone +1 919-817-8106 - info.us at yetopen.com Think green - Non stampare questa e-mail se non necessario / Don't print this email unless necessary -------- D.Lgs. 196/2003 e GDPR 679/2016 -------- Tutte le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario. Tutte le informazioni ivi contenute, compresi eventuali allegati, sono da ritenere confidenziali e riservate secondo i termini del vigente D.Lgs. 196/2003 in materia di privacy e del Regolamento europeo 679/2016 - GDPR - e quindi ne e' proibita l'utilizzazione ulteriore non autorizzata. Nel caso in cui questo messaggio Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo, stamparlo, a non inoltrarlo a terzi e ad avvertirci non appena possibile. Grazie. Confidentiality notice: this email message including any attachment is for the sole use of the intended recipient and may contain confidential and privileged information; pursuant to Legislative Decree 196/2003 and the European General Data Protection Regulation 679/2016 - GDPR - any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recepient please delete this message without copying, printing or forwarding it to others, and alert us as soon as possible. Thank you.
Rowland Penny
2023-Mar-08 11:56 UTC
[Samba] Unable to "rejoin" existing DC after upgrade (infamous WERR_FILE_NOT_FOUND)
On 08/03/2023 11:07, Lorenzo Milesi via samba wrote:> Hi. > As happened some weeks ago, here I am again updating an old Samba 4.14.x network to a current version. The server hosting the FSMO roles is a Debian10 with 4.14.14, while the third node is a Ubuntu 18 running LinuxSchools build 4.14.8. > > We started from a Ubuntu 20.04 server running Louis builds. We demoted the node and joined it back to the domain with 4.17.5 from Michael. Although on the node itself everything seemed ok, the DC didn't appear in DNS, while visible in Sites and ADUC. > Replication is reported as ALL GOOD on the upgraded node, but the remote ones are in error with WERR_FILE_NOT_FOUND. > > We enabled drs_repl log on the 4.14.8, pasting below. > It seems failing because it cannot find the DNS records, which it cannot have because replication is not working. If I run > dig 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan @zimbraip > it returns the correct value, while the same command against any of the two other DCs fails. But maybe this is not the root cause of the problem. > > samba_dnsupdate ran without errors on the 4.17 node, but the other DCs never received those DNS records. > > What else can we check? > thanks > > > # conf on 4.14.8 > [global] > netbios name = ZIMBRA > realm = DOMAIN.LAN > server role = active directory domain controller > workgroup = DOM > server services = -dns > allow dns updates = disabled > interfaces = tun0 lo > log level = 1 drs_repl:10 > > [sysvol] > path = /var/lib/samba/sysvol > read only = No > > [netlogon] > path = /var/lib/samba/sysvol/domain.lan/scripts > read only = No > > # conf on newly upgraded 4.17.5 > [global] > interfaces = tun0 lo > netbios name = NEXTCLOUD > realm = DOMAIN.LAN > server role = active directory domain controller > workgroup = DOM > > log level = 1 drs_repl:10 > [sysvol] > path = /var/lib/samba/sysvol > read only = No > > [netlogon] > path = /var/lib/samba/sysvol/domain.lan/scripts > read only = No > > # log excerpt from 4.14.8 - zimbra node > [2023/03/08 11:53:48.764405, 10, pid=4709, effective(0, 0), real(0, 0), class=drs_repl] ../../source4/dsdb/repl/drepl_notify.c:391(dreplsrv_notify_check) > dreplsrv_notify_check: queued DsReplicaSync for DC=domain,DC=lan to 3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan (urgent=true) uSN=0:27662It looks like it is trying to replicate to the GUID representation of a DC, if you check the DC objects you should find SPN's like this: servicePrincipalName: ldap/3fa4ff9a-7fdc-4912-ad73-08b98f6bf347._msdcs.domain.lan Each DC should have a similar SPN, but with a different GUID and the GUID will probably be a part of other SPN's in each DC object. Do any of your current DC's use that GUID (3fa4ff9a-7fdc-4912-ad73-08b98f6bf347) ? Rowland
Possibly Parallel Threads
- Unable to "rejoin" existing DC after upgrade (infamous WERR_FILE_NOT_FOUND)
- Unable to "rejoin" existing DC after upgrade (infamous WERR_FILE_NOT_FOUND)
- Unable to "rejoin" existing DC after upgrade (infamous WERR_FILE_NOT_FOUND)
- Problems with W2K8R2 <-> S4 replication
- Unable to "rejoin" existing DC after upgrade (infamous WERR_FILE_NOT_FOUND)