lingpanda101 at gmail.com
2016-Sep-22 12:59 UTC
[Samba] replPropertyMetaData & KCC issues after updating to Samba 4.5.0
On 9/21/2016 11:32 PM, Garming Sam wrote:> I would have assumed that deleting all the NTDS connections for all the > DCs would have remedied this issue. I'll write a patch to try to avoid > this error in any case. > > Ignoring this error and answering your earlier question, it's often hard > to tell if the KCC is doing what is expected or not. To figure that out, > you need a lot more information about the network topology > configuration, which site links are defined, who belongs to which site > in the database and probably some of the debug logs (running samba_kcc > --debug manually). It also takes a little while for everything to settle > down, until everyone is completely aware of who is online. > > > > Cheers, > > Garming > > On 22/09/16 00:32, lingpanda101 at gmail.com wrote: >> I'm getting several KCC errors in each of my DC's. They are as follows. >> >> [2016/09/21 08:06:12.364447, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: AttributeError: 'NoneType' object >> has no attribute 'size' >> [2016/09/21 08:06:12.381710, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../source4/dsdb/kcc/kcc_periodic.c:646(samba_kcc_done) >> ../source4/dsdb/kcc/kcc_periodic.c:646: Failed samba_kcc - >> NT_STATUS_ACCESS_DENIED >> [2016/09/21 08:11:12.870383, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: Traceback (most recent call last): >> [2016/09/21 08:11:12.870528, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: File >> "/usr/local/samba/sbin/samba_kcc", line 337, in <module> >> [2016/09/21 08:11:12.870588, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: >> attempt_live_connections=opts.attempt_live_connections) >> [2016/09/21 08:11:12.870639, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: File >> "/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", >> line 2644, in run >> [2016/09/21 08:11:12.870994, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: all_connected >> self.intersite(ping) >> [2016/09/21 08:11:12.871046, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: File >> "/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", >> line 1883, in intersite >> [2016/09/21 08:11:12.871338, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: all_connected >> self.create_intersite_connections() >> [2016/09/21 08:11:12.871398, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: File >> "/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", >> line 1817, in create_intersite_connections >> [2016/09/21 08:11:12.871676, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: part, True) >> [2016/09/21 08:11:12.871724, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: File >> "/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", >> line 1769, in create_connections >> [2016/09/21 08:11:12.871999, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: partial_ok, detect_failed) >> [2016/09/21 08:11:12.872048, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: File >> "/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", >> line 1419, in create_connection >> [2016/09/21 08:11:12.872272, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: not >> cn.is_equivalent_schedule(link_sched))): >> [2016/09/21 08:11:12.872321, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: File >> "/usr/local/samba/lib/python2.7/site-packages/samba/kcc/kcc_utils.py", >> line 1223, in is_equivalent_schedule >> [2016/09/21 08:11:12.872513, 0, pid=1087, effective(0, 0), real(0, >> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler) >> /usr/local/samba/sbin/samba_kcc: if ((self.schedule.size !>> sched.size or >> >> Replication appears to report no errors. Running a KCC check I get the >> following. >> >> samba-tool drs kcc >> ERROR(runtime): DsExecuteKCC failed - (-1073610699, 'The operation >> cannot be performed.') >> >> Switching back to the old KCC clears the errors up. >>For clarification I'll add a few things. I initially deleted all the NTDS site links for each site and allowed the new KCC to create them. However it did not create them I believe correctly. By that I mean it defined what appeared to be a bridgehead server at each site. So I disabled the new KCC 'kccsrv:samba_kcc=false' in my smb.conf and allowed the full mesh to be used again. After all site links were recreated. I then switched the 'kccsrv:samba_kcc=true' in my smb.conf and that's what prompted the following errors above. To further expand on my Topology, I have 3 sites. I'll call them A,B and C. Each site contains 2 DC's. Sites use different subnets and are connected via. fiber. Sites B and C should not be replication partners. They should only replicate with Site A(Default-First-Site-Name). With the new KCC after deleting all the NTDS links, Sites B and C Domain Controller #1 becomes the bridgehead server for that site. Domain Controller #2 at sites B and C only replicates with Domain Controller #1 at it's respective site. So if the bridgehead server goes down, Domain Controller #2 at sites B and C will no longer receive changes. The new KCC does prevent sites B and C from replicating with each other. That is correct. This isn't a huge issue for me. I can continue using the old KCC for now. The full mesh isn't detrimental to my network. Don't want to take up too much of your time. Thanks -- -James
Garming Sam
2016-Sep-22 22:31 UTC
[Samba] replPropertyMetaData & KCC issues after updating to Samba 4.5.0
On 23/09/16 00:59, lingpanda101 at gmail.com wrote:> For clarification I'll add a few things. > > I initially deleted all the NTDS site links for each site and allowed > the new KCC to create them. However it did not create them I believe > correctly. By that I mean it defined what appeared to be a bridgehead > server at each site. So I disabled the new KCC > 'kccsrv:samba_kcc=false' in my smb.conf and allowed the full mesh to > be used again. After all site links were recreated. I then switched > the 'kccsrv:samba_kcc=true' in my smb.conf and that's what prompted > the following errors above. > > To further expand on my Topology, I have 3 sites. I'll call them A,B > and C. Each site contains 2 DC's. Sites use different subnets and are > connected via. fiber. Sites B and C should not be replication > partners. They should only replicate with Site > A(Default-First-Site-Name). With the new KCC after deleting all the > NTDS links, Sites B and C Domain Controller #1 becomes the bridgehead > server for that site. Domain Controller #2 at sites B and C only > replicates with Domain Controller #1 at it's respective site. So if > the bridgehead server goes down, Domain Controller #2 at sites B and C > will no longer receive changes. > > The new KCC does prevent sites B and C from replicating with each > other. That is correct. This isn't a huge issue for me. I can continue > using the old KCC for now. The full mesh isn't detrimental to my > network. Don't want to take up too much of your time. Thanks > >The KCC has been my pet project for the last little bit, so I am very interested in how it functions in general. But as far as I can tell, the KCC is doing what is expected of it. What should happen, and I say should, is that if the bridgehead server dies, the bridgehead server role will transfer to the other DC. There might be a brief period of time before the KCC re-runs where the sites are disconnected, but in general, the failovers should be relatively stable. With only a small number of sites (and DCs), this might be more trouble than it's worth, like you say. In either case, I appreciate your input. Thanks, Garming
lingpanda101 at gmail.com
2016-Sep-23 18:53 UTC
[Samba] replPropertyMetaData & KCC issues after updating to Samba 4.5.0
On 9/22/2016 6:31 PM, Garming Sam wrote:> On 23/09/16 00:59, lingpanda101 at gmail.com wrote: >> For clarification I'll add a few things. >> >> I initially deleted all the NTDS site links for each site and allowed >> the new KCC to create them. However it did not create them I believe >> correctly. By that I mean it defined what appeared to be a bridgehead >> server at each site. So I disabled the new KCC >> 'kccsrv:samba_kcc=false' in my smb.conf and allowed the full mesh to >> be used again. After all site links were recreated. I then switched >> the 'kccsrv:samba_kcc=true' in my smb.conf and that's what prompted >> the following errors above. >> >> To further expand on my Topology, I have 3 sites. I'll call them A,B >> and C. Each site contains 2 DC's. Sites use different subnets and are >> connected via. fiber. Sites B and C should not be replication >> partners. They should only replicate with Site >> A(Default-First-Site-Name). With the new KCC after deleting all the >> NTDS links, Sites B and C Domain Controller #1 becomes the bridgehead >> server for that site. Domain Controller #2 at sites B and C only >> replicates with Domain Controller #1 at it's respective site. So if >> the bridgehead server goes down, Domain Controller #2 at sites B and C >> will no longer receive changes. >> >> The new KCC does prevent sites B and C from replicating with each >> other. That is correct. This isn't a huge issue for me. I can continue >> using the old KCC for now. The full mesh isn't detrimental to my >> network. Don't want to take up too much of your time. Thanks >> >> > The KCC has been my pet project for the last little bit, so I am very > interested in how it functions in general. But as far as I can tell, the > KCC is doing what is expected of it. What should happen, and I say > should, is that if the bridgehead server dies, the bridgehead server > role will transfer to the other DC. There might be a brief period of > time before the KCC re-runs where the sites are disconnected, but in > general, the failovers should be relatively stable. With only a small > number of sites (and DCs), this might be more trouble than it's worth, > like you say. In either case, I appreciate your input. > > > Thanks, > > GarmingI went ahead and enabled the new KCC. I deleted all the automatically generated NTDS links and let Samba create them. I did this through the Microsoft Active Directory Sites and Services tool. I didn't see the option to delete with 'samba-tool drs options --help'. I did run 'samba-tool drs kcc' to force the check and not wait. I see all the automatically generated site links are created as you say they should. I shutdown one of the bridgehead servers in a site (killall samba). In my case it's SOLDC1 in Site B. I ran 'samba-tool drs kcc' on all DC's to see if a new KCC connection would be created on SOLDC2 in site B. It never was. So I restarted SOLDC2 in site B and no connection was ever created. This is all with SOLDC1 in site B still down. This tells me SOLDC2 becomes an island without anyway to replicate. One strange thing is 'samba-tool drs showrepl' begs to differ. root at soldc2:~# samba-tool drs showrepl site-b\SOLDC2 DSA Options: 0x00000001 DSA object GUID: 25055641-49e7-4b3f-a7e3-9d206375306c DSA invocationId: d11890e8-6b90-4e94-aca4-6d7a940f47b5 ==== INBOUND NEIGHBORS === CN=Configuration,DC=domain,DC=local site-b\SOLDC1 via RPC DSA object GUID: 55e069f5-4f47-415b-8fa4-a398948235aa Last attempt @ Fri Sep 23 14:40:18 2016 EDT was successful 0 consecutive failure(s). Last success @ Fri Sep 23 14:40:18 2016 EDT DC=DomainDnsZones,DC=domain,DC=local site-b\SOLDC1 via RPC DSA object GUID: 55e069f5-4f47-415b-8fa4-a398948235aa Last attempt @ Fri Sep 23 14:42:24 2016 EDT was successful 0 consecutive failure(s). Last success @ Fri Sep 23 14:42:24 2016 EDT DC=DomainDnsZones,DC=domain,DC=local Default-First-Site-Name\PFDC2 via RPC DSA object GUID: e6284e90-f964-4643-b6a6-5baafdd7ba36 Last attempt @ Fri Sep 23 14:42:34 2016 EDT was successful 0 consecutive failure(s). Last success @ Fri Sep 23 14:42:34 2016 EDT DC=DomainDnsZones,DC=domain,DC=local site-c\DUNDC1 via RPC DSA object GUID: a216e718-488f-4821-8d9c-a399e6789222 Last attempt @ Fri Sep 23 14:42:32 2016 EDT was successful 0 consecutive failure(s). Last success @ Fri Sep 23 14:42:32 2016 EDT DC=DomainDnsZones,DC=domain,DC=local site-c\DUNDC2 via RPC DSA object GUID: 3c08db42-9416-40df-99ad-6d0c0ec554a6 Last attempt @ Fri Sep 23 14:41:00 2016 EDT was successful 0 consecutive failure(s). Last success @ Fri Sep 23 14:41:00 2016 EDT DC=DomainDnsZones,DC=domain,DC=local Default-First-Site-Name\PFDC1 via RPC DSA object GUID: acc2392f-9567-450f-bcb3-4fb1034b8753 Last attempt @ Fri Sep 23 14:40:58 2016 EDT was successful 0 consecutive failure(s). Last success @ Fri Sep 23 14:40:58 2016 EDT CN=Schema,CN=Configuration,DC=domain,DC=local site-b\SOLDC1 via RPC DSA object GUID: 55e069f5-4f47-415b-8fa4-a398948235aa Last attempt @ Fri Sep 23 14:40:19 2016 EDT was successful 0 consecutive failure(s). Last success @ Fri Sep 23 14:40:19 2016 EDT DC=domain,DC=local site-b\SOLDC1 via RPC DSA object GUID: 55e069f5-4f47-415b-8fa4-a398948235aa Last attempt @ Fri Sep 23 14:40:20 2016 EDT was successful 0 consecutive failure(s). Last success @ Fri Sep 23 14:40:20 2016 EDT DC=ForestDnsZones,DC=domain,DC=local site-b\SOLDC1 via RPC DSA object GUID: 55e069f5-4f47-415b-8fa4-a398948235aa Last attempt @ Fri Sep 23 14:40:18 2016 EDT was successful 0 consecutive failure(s). Last success @ Fri Sep 23 14:40:18 2016 EDT ==== OUTBOUND NEIGHBORS === CN=Configuration,DC=domain,DC=local site-c\DUNDC2 via RPC DSA object GUID: 3c08db42-9416-40df-99ad-6d0c0ec554a6 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) CN=Configuration,DC=domain,DC=local Default-First-Site-Name\PFDC1 via RPC DSA object GUID: acc2392f-9567-450f-bcb3-4fb1034b8753 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) CN=Configuration,DC=domain,DC=local site-b\SOLDC1 via RPC DSA object GUID: 55e069f5-4f47-415b-8fa4-a398948235aa Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) CN=Configuration,DC=domain,DC=local Default-First-Site-Name\PFDC2 via RPC DSA object GUID: e6284e90-f964-4643-b6a6-5baafdd7ba36 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) CN=Configuration,DC=domain,DC=local site-c\DUNDC1 via RPC DSA object GUID: a216e718-488f-4821-8d9c-a399e6789222 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=DomainDnsZones,DC=domain,DC=local site-c\DUNDC2 via RPC DSA object GUID: 3c08db42-9416-40df-99ad-6d0c0ec554a6 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=DomainDnsZones,DC=domain,DC=local Default-First-Site-Name\PFDC1 via RPC DSA object GUID: acc2392f-9567-450f-bcb3-4fb1034b8753 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=DomainDnsZones,DC=domain,DC=local Default-First-Site-Name\PFDC2 via RPC DSA object GUID: e6284e90-f964-4643-b6a6-5baafdd7ba36 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=DomainDnsZones,DC=domain,DC=local site-b\SOLDC1 via RPC DSA object GUID: 55e069f5-4f47-415b-8fa4-a398948235aa Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=DomainDnsZones,DC=domain,DC=local site-c\DUNDC1 via RPC DSA object GUID: a216e718-488f-4821-8d9c-a399e6789222 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) CN=Schema,CN=Configuration,DC=domain,DC=local site-c\DUNDC2 via RPC DSA object GUID: 3c08db42-9416-40df-99ad-6d0c0ec554a6 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) CN=Schema,CN=Configuration,DC=domain,DC=local Default-First-Site-Name\PFDC1 via RPC DSA object GUID: acc2392f-9567-450f-bcb3-4fb1034b8753 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) CN=Schema,CN=Configuration,DC=domain,DC=local site-b\SOLDC1 via RPC DSA object GUID: 55e069f5-4f47-415b-8fa4-a398948235aa Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) CN=Schema,CN=Configuration,DC=domain,DC=local Default-First-Site-Name\PFDC2 via RPC DSA object GUID: e6284e90-f964-4643-b6a6-5baafdd7ba36 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) CN=Schema,CN=Configuration,DC=domain,DC=local site-c\DUNDC1 via RPC DSA object GUID: a216e718-488f-4821-8d9c-a399e6789222 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=domain,DC=local Default-First-Site-Name\PFDC1 via RPC DSA object GUID: acc2392f-9567-450f-bcb3-4fb1034b8753 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=domain,DC=local site-c\DUNDC2 via RPC DSA object GUID: 3c08db42-9416-40df-99ad-6d0c0ec554a6 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=domain,DC=local site-b\SOLDC1 via RPC DSA object GUID: 55e069f5-4f47-415b-8fa4-a398948235aa Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=domain,DC=local Default-First-Site-Name\PFDC2 via RPC DSA object GUID: e6284e90-f964-4643-b6a6-5baafdd7ba36 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=domain,DC=local site-c\DUNDC1 via RPC DSA object GUID: a216e718-488f-4821-8d9c-a399e6789222 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=ForestDnsZones,DC=domain,DC=local site-c\DUNDC2 via RPC DSA object GUID: 3c08db42-9416-40df-99ad-6d0c0ec554a6 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=ForestDnsZones,DC=domain,DC=local Default-First-Site-Name\PFDC1 via RPC DSA object GUID: acc2392f-9567-450f-bcb3-4fb1034b8753 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=ForestDnsZones,DC=domain,DC=local site-b\SOLDC1 via RPC DSA object GUID: 55e069f5-4f47-415b-8fa4-a398948235aa Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=ForestDnsZones,DC=domain,DC=local Default-First-Site-Name\PFDC2 via RPC DSA object GUID: e6284e90-f964-4643-b6a6-5baafdd7ba36 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) DC=ForestDnsZones,DC=domain,DC=local site-c\DUNDC1 via RPC DSA object GUID: a216e718-488f-4821-8d9c-a399e6789222 Last attempt @ NTTIME(0) was successful 0 consecutive failure(s). Last success @ NTTIME(0) ==== KCC CONNECTION OBJECTS === Connection -- Connection name: 7b7ddab7-4377-44f4-9831-8fe7feb55115 Enabled : TRUE Server DNS name : SOLDC1.domain.local Server DN name : CN=NTDS Settings,CN=SOLDC1,CN=Servers,CN=site-b,CN=Sites,CN=Configuration,DC=domain,DC=local TransportType: RPC options: 0x00000001 Warning: No NC replicated for Connection! I have what appears to still be a full mesh replication. Shouldn't the outbound and inbound neighbors be reflective of the KCC connection objects? I would expect to find only inbound and outbound connections for SOLDC1. Maybe I'm completely misinterpreting the intended behavior. -- -James
Reasonably Related Threads
- replPropertyMetaData & KCC issues after updating to Samba 4.5.0
- replPropertyMetaData & KCC issues after updating to Samba 4.5.0
- updates of repsFrom/repsTo attributes (was : Re: replPropertyMetaData & KCC issues after updating to Samba 4.5.0)
- replPropertyMetaData & KCC issues after updating to Samba 4.5.0
- Samba DNS fails when queried with nslookup commands