mj
2020-Nov-16 12:56 UTC
[Samba] changes on DC not replicated, while showrepl reports no issues
Hi all, We are running a three DC samba AD, using 4.12.8 sernet packages. Very stable for years. Today at 12:30 my colleague moved two users from * CN=Users,DC=samba,DC=company,DC=com to * OU=disabled,DC=samba,DC=company,DC=com This change was done on the DC4 at 12:30 using LAM (ldap-account-manager version 7.3) Ever since that, my automated samba-tool ldapcmp scripts started reporting ldapcmp discrepancies between the DCs, like:> * DNs found only in ldap://dc4.samba.company.com: > CN=USER1,OU=DISABLED,DC=SAMBA,DC=COMPANY,DC=COM > CN=USER2,OU=DISABLED,DC=SAMBA,DC=COMPANY,DC=COM > > * DNs found only in ldap://dc3.samba.company.com: > CN=USER1,CN=USERS,DC=SAMBA,DC=COMPANY,DC=COM > CN=USER2,CN=USERS,DC=SAMBA,DC=COMPANY,DC=COMIt seems DC2 & DC3 are still in sync (both having the two users in CN=USERS) and only DC4 has the user now in OU=DISABLED. And now the worrying part: "samba-tool drs showrepl" still shows success on all DCs! Recent timestamps (long after 12:30) on inbound replication, outbound replication also success (but without timestamps), and every DC replicates to both other DCs for all partitions. The only reason we actually noticed that this issue is occuring, is because we run automated ldapcmp between the DC's, otherwise we would not have known. samba-tool dbcheck --cross-ncs reports 0 errors on 5413 objects on all three DCs. Of course we could do try to re-replicate "samba-tool drs replicate" etc, but should the above not be impossible to happen? What could cause it? MJ
Rowland penny
2020-Nov-16 13:07 UTC
[Samba] changes on DC not replicated, while showrepl reports no issues
On 16/11/2020 12:56, mj via samba wrote:> Hi all, > > We are running a three DC samba AD, using 4.12.8 sernet packages. Very > stable for years. > > Today at 12:30 my colleague moved two users from > * CN=Users,DC=samba,DC=company,DC=com > to > * OU=disabled,DC=samba,DC=company,DC=com > > This change was done on the DC4 at 12:30 using LAM > (ldap-account-manager version 7.3) > > Ever since that, my automated samba-tool ldapcmp scripts started > reporting ldapcmp discrepancies between the DCs, like: > >> * DNs found only in ldap://dc4.samba.company.com: >> ??? CN=USER1,OU=DISABLED,DC=SAMBA,DC=COMPANY,DC=COM >> ??? CN=USER2,OU=DISABLED,DC=SAMBA,DC=COMPANY,DC=COM >> >> * DNs found only in ldap://dc3.samba.company.com: >> ??? CN=USER1,CN=USERS,DC=SAMBA,DC=COMPANY,DC=COM >> ??? CN=USER2,CN=USERS,DC=SAMBA,DC=COMPANY,DC=COM > > It seems DC2 & DC3 are still in sync (both having the two users in > CN=USERS) and only DC4 has the user now in OU=DISABLED. > > And now the worrying part: > > "samba-tool drs showrepl" still shows success on all DCs! Recent > timestamps (long after 12:30) on inbound replication, outbound > replication also success (but without timestamps), and every DC > replicates to both other DCs for all partitions. > > The only reason we actually noticed that this issue is occuring, is > because we run automated ldapcmp between the DC's, otherwise we would > not have known. > > samba-tool dbcheck --cross-ncs reports 0 errors on 5413 objects on all > three DCs. > > Of course we could do try to re-replicate "samba-tool drs replicate" > etc, but should the above not be impossible to happen? What could > cause it? > > MJ >My first thought is time, is it the same on all DC's ? Rowland
mj
2020-Nov-16 13:14 UTC
[Samba] changes on DC not replicated, while showrepl reports no issues
Hi Rowland, I think so, see:> root at dc2:~# date > Mon 16 Nov 2020 02:11:27 PM CET > > root at dc3:~# date > Mon 16 Nov 2020 02:11:36 PM CET > > root at dc4:~# date > Mon 16 Nov 2020 02:11:45 PM CETTen seconds apart, because it requires around 10 sec to logon to each DC over ssh. MJ On 11/16/20 2:07 PM, Rowland penny via samba wrote:> On 16/11/2020 12:56, mj via samba wrote: >> Hi all, >> >> We are running a three DC samba AD, using 4.12.8 sernet packages. Very >> stable for years. >> >> Today at 12:30 my colleague moved two users from >> * CN=Users,DC=samba,DC=company,DC=com >> to >> * OU=disabled,DC=samba,DC=company,DC=com >> >> This change was done on the DC4 at 12:30 using LAM >> (ldap-account-manager version 7.3) >> >> Ever since that, my automated samba-tool ldapcmp scripts started >> reporting ldapcmp discrepancies between the DCs, like: >> >>> * DNs found only in ldap://dc4.samba.company.com: >>> ??? CN=USER1,OU=DISABLED,DC=SAMBA,DC=COMPANY,DC=COM >>> ??? CN=USER2,OU=DISABLED,DC=SAMBA,DC=COMPANY,DC=COM >>> >>> * DNs found only in ldap://dc3.samba.company.com: >>> ??? CN=USER1,CN=USERS,DC=SAMBA,DC=COMPANY,DC=COM >>> ??? CN=USER2,CN=USERS,DC=SAMBA,DC=COMPANY,DC=COM >> >> It seems DC2 & DC3 are still in sync (both having the two users in >> CN=USERS) and only DC4 has the user now in OU=DISABLED. >> >> And now the worrying part: >> >> "samba-tool drs showrepl" still shows success on all DCs! Recent >> timestamps (long after 12:30) on inbound replication, outbound >> replication also success (but without timestamps), and every DC >> replicates to both other DCs for all partitions. >> >> The only reason we actually noticed that this issue is occuring, is >> because we run automated ldapcmp between the DC's, otherwise we would >> not have known. >> >> samba-tool dbcheck --cross-ncs reports 0 errors on 5413 objects on all >> three DCs. >> >> Of course we could do try to re-replicate "samba-tool drs replicate" >> etc, but should the above not be impossible to happen? What could >> cause it? >> >> MJ >> > My first thought is time, is it the same on all DC's ? > > Rowland > > >
mj
2020-Nov-17 11:54 UTC
[Samba] changes on DC not replicated, while showrepl reports no issues
Hi, Checking a bit more today, as the two changes on DC4 have *still* not propagated to DC2/DC3, I discovered the samba-tool drs uptodateness command. I gives the following output, not sure what "unknown invocation ID" means. I hope someone here is able to explain..? It's output:> root at dc2:~# samba-tool drs uptodateness > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc > Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 > Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 > Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 > Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 > DOMAIN maximum: 1 median: 0.0 failure: 0 > CONFIGURATION maximum: 1 median: 1.0 failure: 0 > SCHEMA maximum: 1 median: 1.0 failure: 0 > DNSDOMAIN maximum: 1 median: 1.0 failure: 0 > DNSFOREST maximum: 1 median: 1.0 failure: 0Any idea what this means? On DC3, it lists the same Unknown invocation ID, but ends differently:> DOMAIN maximum: 0 median: 0.0 failure: 0 > CONFIGURATION maximum: 0 median: 0.0 failure: 0 > SCHEMA maximum: 0 median: 0.0 failure: 0 > DNSDOMAIN maximum: 0 median: 0.0 failure: 0 > DNSFOREST maximum: 0 median: 0.0 failure: 0Output on DC4 (the one where we made the change yesterday) looks like DC3 above. Not sure what this means. And worried that this samba-tool drs showrepl still shows nothing wrong at all, though there clearly IS an issue. Warning to everyone: use the samba-tool ldapcmp command to verify LDAP contents between your DCs! Don't just trust your showrepl output..! MJ
Rowland penny
2020-Nov-17 12:11 UTC
[Samba] changes on DC not replicated, while showrepl reports no issues
On 17/11/2020 11:54, mj via samba wrote:> Hi, > > Checking a bit more today, as the two changes on DC4 have *still* not > propagated? to DC2/DC3, I discovered the > ?samba-tool drs uptodateness > command. > > I gives the following output, not sure what "unknown invocation ID" > means. I hope someone here is able to explain..? > > It's output: > >> root at dc2:~# samba-tool drs uptodateness >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> Unknown invocation ID 556b2cb4-e576-48e2-bb7c-7f62caee84fc >> Unknown invocation ID 7281ff0f-5af2-4ded-bdaf-362f1545d221 >> Unknown invocation ID 82b24c02-16cb-4ed1-b68b-a047c26886a9 >> Unknown invocation ID b97cc603-6d03-4ad0-8629-be408fed8d69 >> Unknown invocation ID e50c4440-f3d2-429e-a19d-c11bc72f5565 >> DOMAIN????????? maximum: 1? median: 0.0? failure: 0 >> CONFIGURATION?? maximum: 1? median: 1.0? failure: 0 >> SCHEMA????????? maximum: 1? median: 1.0? failure: 0 >> DNSDOMAIN?????? maximum: 1? median: 1.0? failure: 0 >> DNSFOREST?????? maximum: 1? median: 1.0? failure: 0 > > Any idea what this means? > > On DC3, it lists the same Unknown invocation ID, but ends differently: >> DOMAIN????????? maximum: 0? median: 0.0 failure: 0 >> CONFIGURATION?? maximum: 0? median: 0.0? failure: 0 >> SCHEMA????????? maximum: 0? median: 0.0? failure: 0 >> DNSDOMAIN?????? maximum: 0? median: 0.0? failure: 0 >> DNSFOREST?????? maximum: 0? median: 0.0? failure: 0 > > Output on DC4 (the one where we made the change yesterday) looks like > DC3 above. > > Not sure what this means. > > And worried that this samba-tool drs showrepl still shows nothing > wrong at all, though there clearly IS an issue. > > Warning to everyone: use the samba-tool ldapcmp command to verify LDAP > contents between your DCs! Don't just trust your showrepl output..! > > MJ >Try reading this: http://www.frickelsoft.net/blog/?p=148 Rowland
mj
2020-Nov-17 12:20 UTC
[Samba] changes on DC not replicated, while showrepl reports no issues
Hi, Again more data: The command samba-tool visualize reps seems to agree with the observed lack of replication from DC4 to DC3 & DC2:> RepsTo objects for DOMAIN > destination > ,--- CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=samba,DC=company,DC=com > |,-- CN=DC3,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=samba,DC=company,DC=com > source ||,- CN=DC4,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=samba,DC=company,DC=com > CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=samba,DC=company,DC=com 011 > CN=DC3,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=samba,DC=company,DC=com 101 > CN=DC4,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=samba,DC=company,DC=com --0 > > Data can get from source to destination in the indicated number of steps. > 0 means zero steps (it is the same DC) > 1 means a direct link > 2 means a transitive link involving two steps (i.e. one intermediate DC) > - means there is no connection, even through other DCsI hope formatting survives. ABove it says: "no connection" for DOMAIN from DC4 to DC3 / DC2. Even though there is nothing in between our DCs. No firewalling, all in the same subnet, etc. Anyone can explain more..? MJ
mj
2020-Nov-26 08:10 UTC
[Samba] changes on DC not replicated, while showrepl reports no issues
Hi, Just to follow-up this post. We are now working with sernet support to get this resolved. It seems the reason this occured is unclear, but the most likely cause is: We are using ldap-account-manager (LAM) to manage our samba AD through LDAP access. We access samba LDAPs over an HAProxy server that has the three samba DCs configured as backend servers. (so in LAM we only configured one haproxy source that holds all DCs) However: The default load balancing mechanism of HAProxy is round-robin, so haproxy talks to a different backend server for each request. This can potentially cause problems when writing/editing, and HAProxy switches backend servers but replication has not yet been completed. We have now switched the haproxy load balancing method to 'source', where a connection will 'stick' to the same backend server, as long as the backend server is available. That should work better. Time will tell. MJ On 11/16/20 1:56 PM, mj via samba wrote:> Hi all, > > We are running a three DC samba AD, using 4.12.8 sernet packages. Very > stable for years. > > Today at 12:30 my colleague moved two users from > * CN=Users,DC=samba,DC=company,DC=com > to > * OU=disabled,DC=samba,DC=company,DC=com > > This change was done on the DC4 at 12:30 using LAM (ldap-account-manager > version 7.3) > > Ever since that, my automated samba-tool ldapcmp scripts started > reporting ldapcmp discrepancies between the DCs, like: > >> * DNs found only in ldap://dc4.samba.company.com: >> ??? CN=USER1,OU=DISABLED,DC=SAMBA,DC=COMPANY,DC=COM >> ??? CN=USER2,OU=DISABLED,DC=SAMBA,DC=COMPANY,DC=COM >> >> * DNs found only in ldap://dc3.samba.company.com: >> ??? CN=USER1,CN=USERS,DC=SAMBA,DC=COMPANY,DC=COM >> ??? CN=USER2,CN=USERS,DC=SAMBA,DC=COMPANY,DC=COM > > It seems DC2 & DC3 are still in sync (both having the two users in > CN=USERS) and only DC4 has the user now in OU=DISABLED. > > And now the worrying part: > > "samba-tool drs showrepl" still shows success on all DCs! Recent > timestamps (long after 12:30) on inbound replication, outbound > replication also success (but without timestamps), and every DC > replicates to both other DCs for all partitions. > > The only reason we actually noticed that this issue is occuring, is > because we run automated ldapcmp between the DC's, otherwise we would > not have known. > > samba-tool dbcheck --cross-ncs reports 0 errors on 5413 objects on all > three DCs. > > Of course we could do try to re-replicate "samba-tool drs replicate" > etc, but should the above not be impossible to happen? What could cause it? > > MJ >