samba - Sep 2016 - replPropertyMetaData & KCC issues after updating to Samba 4.5.0

On 9/18/2016 6:23 PM, Garming Sam wrote:
> Hi,
>
> For the unsorted attributeID values errors, can you first try:
>
> samba-tool dbcheck --cross-ncs --fix --yes
'fix_replmetadata_unsorted_attid'
>
> There's too much going on, and it does look like it might be bailing
> out. Running it with 'fix_replmetadata_unsorted_attid' should fix
those
> first errors, then it will probably be easier to figure out what is
> happening. The 'ERROR: incorrect GUID component for member in
object'
> should be completely harmless (and due to objects which have been
> recycled) and there's likely a fix to get rid of them to come. However,
> it seems there is something else occurring which we may need to look at
> in more detail.
>
>
>
> As for the KCC, it looks like those are probably stale links from the
> old KCC which connected every DC. The KCC is supposed to delete extra
> connections, but this doesn't always occur (or does not occur
> immediately). Simply deleting those connections should allow the new KCC
> to follow all the site requirements.
>
> If you find that DNS zones are not working correctly, this is probably
> related to the failing dbcheck, and so you may want to also run:
>
> samba-tool dbcheck --cross-ncs --fix --yes 'fix_replica_locations'
>
>
> Hopefully that helps some of your issues.
>
> Cheers,
>
> Garming
>
>
> On 13/09/16 05:12, lingpanda101--- via samba wrote:
>> Hello,
>>
>>      Updated samba from 4.4.5 to 4.5.0. All DC's are Ubuntu 12.04.5
>> LTS. I install samba from source(./configure,make,make install).
>> Looking at the release notes I see the section on
>> "replPropertyMetaData Chnages".  I run 'samba-tool
dbcheck --cross-ncs
>> --fix --yes' and see the errors and samba attempts to fix.
>>
>> ERROR: unsorted attributeID values in replPropertyMetaData on
>> CN=BOOPTI760-7,OU=Computers,OU=BO Staff,OU=BO
>> Office,OU=PF,DC=domain,DC=local
>>
>> Fix replPropertyMetaData on CN=BOOPTI760-7,OU=Computers,OU=BO
>> Staff,OU=BO,OU=PF,DC=domain,DC=local by sorting the attribute list?
[YES]
>> Fixed attribute 'replPropertyMetaData' of
>> 'CN=BOOPTI760-7,OU=Computers,OU=BO
Staff,OU=BO,OU=PF,DC=domain,DC=local'
>>
>> If I run the same command again 'samba-tool dbcheck --cross-ncs
--fix
>> --yes'.  I appear to see the same errors all over again. It's
as if
>> they don't really get corrected.
>>
>> I also see several of these new errors.
>>
>> ERROR: incorrect GUID component for member in object CN=IMG P
>> Share,CN=Users,DC=domain,DC=local -
>>
<GUID=6357f99052feb942af868a84a4d5dd78>;<RMD_ADDTIME=130647328190000000>;<RMD_CHANGETIME=130650285980000000>;<RMD_FLAGS=1>;<RMD_INVOCID=194264d3cddbff43815e8850f94192e1>;<RMD_LOCAL_USN=360361>;<RMD_ORIGINATING_USN=478913>;<RMD_VERSION=3>;<SID=010500000000000515000000730d083801679a88e52f2fc7360c0000>;CN=Test
>> User,CN=Users,DC=domain,DC=local
>> unable to find object for DN CN=Test User,CN=Users,DC=domain,DC=local
>> - (No such Base DN: CN=Test User,CN=Users,DC=domain,DC=local)
>> Not removing dangling forward link
>> ERROR: incorrect DN string component for member in object CN=IMG P
>> Share,CN=Users,DC=domain,DC=local -
>>
<GUID=f192ae2cf2a55342818fe1b4a45d5396>;<RMD_ADDTIME=130649535030000000>;<RMD_CHANGETIME=130649601110000000>;<RMD_FLAGS=1>;<RMD_INVOCID=194264d3cddbff43815e8850f94192e1>;<RMD_LOCAL_USN=360194>;<RMD_ORIGINATING_USN=478611>;<RMD_VERSION=1>;<SID=010500000000000515000000730d083801679a88e52f2fc7110e0000>;CN=Demo
>> User,OU=Users,OU=IT Department,OU=Prince Frederick,DC=domain,DC=local
>> Change DN to
>>
<GUID=2cae92f1-a5f2-4253-818f-e1b4a45d5396>;<SID=S-1-5-21-940051827-2291820289-3341758437-3601>;CN=Demo
>> User,OU=Users,OU=PF MA,OU=MA,OU=PF,DC=domain,DC=local? [YES]
>> ERROR: Failed to fix incorrect DN string on attribute member : (53,
>> 'Attribute member already deleted for target GUID
>> 2cae92f1-a5f2-4253-818f-e1b4a45d5396')
>>
>> The second issue has to do with the new KCC. I had this same issue
>> when I tested out the 'kccsrv:samba_kcc=true' feature in prior
builds.
>> See the duplicate connections for 'PFDC2.domain.local' below. I
have
>> the same issue on another DC, although for a different DC connection.
>> Site links are also not being adhered to.
>>
>> ==== KCC CONNECTION OBJECTS ===>>
>> Connection --
>>          Connection name: 042e3f91-6f91-4e3d-ab58-4b9fea0c4b81
>>          Enabled        : TRUE
>>          Server DNS name : PFDC2.domain.local
>>          Server DN name  : CN=NTDS
>>
Settings,CN=PFDC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=domain,DC=local
>>                  TransportType: RPC
>>                  options: 0x00000001
>> Warning: No NC replicated for Connection!
>> Connection --
>>          Connection name: 1244834d-74e3-4a5a-981e-88367d7f1a36
>>          Enabled        : TRUE
>>          Server DNS name : pfdc1.domain.local
>>          Server DN name  : CN=NTDS
>>
Settings,CN=PFDC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=domain,DC=local
>>                  TransportType: RPC
>>                  options: 0x00000001
>> Warning: No NC replicated for Connection!
>> Connection --
>>          Connection name: 26508262-933f-4fd3-bc2c-c236e050bfb0
>>          Enabled        : TRUE
>>          Server DNS name : SOLDC2.domain.local
>>          Server DN name  : CN=NTDS
>>
Settings,CN=SOLDC2,CN=Servers,CN=Solomons,CN=Sites,CN=Configuration,DC=domain,DC=local
>>                  TransportType: RPC
>>                  options: 0x00000001
>> Warning: No NC replicated for Connection!
>> Connection --
>>          Connection name: 5ef1d75c-2977-435c-8b90-a94886d3b92d
>>          Enabled        : TRUE
>>          Server DNS name : DUNDC2.domain.local
>>          Server DN name  : CN=NTDS
>>
Settings,CN=DUNDC2,CN=Servers,CN=Dunkirk,CN=Sites,CN=Configuration,DC=domain,DC=local
>>                  TransportType: RPC
>>                  options: 0x00000001
>> Warning: No NC replicated for Connection!
>> Connection --
>>          Connection name: 6743a36d-2401-4ecb-9f05-565a4528f7c6
>>          Enabled        : TRUE
>>          Server DNS name : SOLDC1.domain.local
>>          Server DN name  : CN=NTDS
>>
Settings,CN=SOLDC1,CN=Servers,CN=Solomons,CN=Sites,CN=Configuration,DC=domain,DC=local
>>                  TransportType: RPC
>>                  options: 0x00000001
>> Warning: No NC replicated for Connection!
>> Connection --
>>          Connection name: 865908ee-2f8b-456c-841e-7f54e3e93835
>>          Enabled        : TRUE
>>          Server DNS name : PFDC2.domain.local
>>          Server DN name  : CN=NTDS
>>
Settings,CN=PFDC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=domain,DC=local
>>                  TransportType: RPC
>>                  options: 0x00000001
>> Warning: No NC replicated for Connection!
>>
>>
>> Smb.conf is similar among all DC's. See below.
>>
>> # Global parameters
>> [global]
>>          workgroup = DOMAIN
>>          realm = domain.local
>>          netbios name = DUNDC1
>>          server role = active directory domain controller
>>          dns forwarder = 8.8.8.8
>>          idmap_ldb:use rfc2307 = yes
>>
>>          # Debug Logging Information
>>          log file = /usr/local/samba/var/log.%U
>>          max log size = 5000
>>          log level = 1
>>          logging = syslog at 2 file
>>          debug timestamp = Yes
>>          debug uid = Yes
>>          debug pid = Yes
>>
>>          allow dns updates = secure
>>
>>          # Disable Cups Printing
>>          load printers = No
>>          printcap name = /dev/null
>>          disable spoolss = Yes
>>
>>          ldap server require strong auth = No
>>
>> [netlogon]
>>          path = /usr/local/samba/var/locks/sysvol/domain.local/scripts
>>          read only = No
>>
>>
>> [sysvol]
>>          path = /usr/local/samba/var/locks/sysvol
>>          read only = No
>>
>> Thanks for any guidance.
>>
I'm getting several KCC errors in each of my DC's. They are as follows.

[2016/09/21 08:06:12.364447,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc: AttributeError: 'NoneType' object 
has no attribute 'size'
[2016/09/21 08:06:12.381710,  0, pid=1087, effective(0, 0), real(0, 0)] 
../source4/dsdb/kcc/kcc_periodic.c:646(samba_kcc_done)
   ../source4/dsdb/kcc/kcc_periodic.c:646: Failed samba_kcc - 
NT_STATUS_ACCESS_DENIED
[2016/09/21 08:11:12.870383,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc: Traceback (most recent call last):
[2016/09/21 08:11:12.870528,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:   File 
"/usr/local/samba/sbin/samba_kcc", line 337, in <module>
[2016/09/21 08:11:12.870588,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc: 
attempt_live_connections=opts.attempt_live_connections)
[2016/09/21 08:11:12.870639,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:   File 
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", 
line 2644, in run
[2016/09/21 08:11:12.870994,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:     all_connected = self.intersite(ping)
[2016/09/21 08:11:12.871046,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:   File 
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", 
line 1883, in intersite
[2016/09/21 08:11:12.871338,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:     all_connected = 
self.create_intersite_connections()
[2016/09/21 08:11:12.871398,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:   File 
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", 
line 1817, in create_intersite_connections
[2016/09/21 08:11:12.871676,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:     part, True)
[2016/09/21 08:11:12.871724,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:   File 
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", 
line 1769, in create_connections
[2016/09/21 08:11:12.871999,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:     partial_ok, detect_failed)
[2016/09/21 08:11:12.872048,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:   File 
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", 
line 1419, in create_connection
[2016/09/21 08:11:12.872272,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:     not 
cn.is_equivalent_schedule(link_sched))):
[2016/09/21 08:11:12.872321,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:   File 
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/kcc_utils.py",
line 1223, in is_equivalent_schedule
[2016/09/21 08:11:12.872513,  0, pid=1087, effective(0, 0), real(0, 0)] 
../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
   /usr/local/samba/sbin/samba_kcc:     if ((self.schedule.size != 
sched.size or

Replication appears to report no errors. Running a KCC check I get the 
following.

samba-tool drs kcc
ERROR(runtime): DsExecuteKCC failed - (-1073610699, 'The operation 
cannot be performed.')

Switching back to the old KCC clears the errors up.

-- 
-James

I would have assumed that deleting all the NTDS connections for all the
DCs would have remedied this issue. I'll write a patch to try to avoid
this error in any case.

Ignoring this error and answering your earlier question, it's often hard
to tell if the KCC is doing what is expected or not. To figure that out,
you need a lot more information about the network topology
configuration, which site links are defined, who belongs to which site
in the database and probably some of the debug logs (running samba_kcc
--debug manually). It also takes a little while for everything to settle
down, until everyone is completely aware of who is online.



Cheers,

Garming

On 22/09/16 00:32, lingpanda101 at gmail.com wrote:
>
> I'm getting several KCC errors in each of my DC's. They are as
follows.
>
> [2016/09/21 08:06:12.364447,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc: AttributeError: 'NoneType'
object
> has no attribute 'size'
> [2016/09/21 08:06:12.381710,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../source4/dsdb/kcc/kcc_periodic.c:646(samba_kcc_done)
>   ../source4/dsdb/kcc/kcc_periodic.c:646: Failed samba_kcc -
> NT_STATUS_ACCESS_DENIED
> [2016/09/21 08:11:12.870383,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc: Traceback (most recent call last):
> [2016/09/21 08:11:12.870528,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:   File
> "/usr/local/samba/sbin/samba_kcc", line 337, in <module>
> [2016/09/21 08:11:12.870588,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:
> attempt_live_connections=opts.attempt_live_connections)
> [2016/09/21 08:11:12.870639,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:   File
>
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py",
> line 2644, in run
> [2016/09/21 08:11:12.870994,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:     all_connected >
self.intersite(ping)
> [2016/09/21 08:11:12.871046,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:   File
>
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py",
> line 1883, in intersite
> [2016/09/21 08:11:12.871338,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:     all_connected >
self.create_intersite_connections()
> [2016/09/21 08:11:12.871398,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:   File
>
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py",
> line 1817, in create_intersite_connections
> [2016/09/21 08:11:12.871676,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:     part, True)
> [2016/09/21 08:11:12.871724,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:   File
>
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py",
> line 1769, in create_connections
> [2016/09/21 08:11:12.871999,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:     partial_ok, detect_failed)
> [2016/09/21 08:11:12.872048,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:   File
>
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py",
> line 1419, in create_connection
> [2016/09/21 08:11:12.872272,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:     not
> cn.is_equivalent_schedule(link_sched))):
> [2016/09/21 08:11:12.872321,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:   File
>
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/kcc_utils.py",
> line 1223, in is_equivalent_schedule
> [2016/09/21 08:11:12.872513,  0, pid=1087, effective(0, 0), real(0,
> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>   /usr/local/samba/sbin/samba_kcc:     if ((self.schedule.size !>
sched.size or
>
> Replication appears to report no errors. Running a KCC check I get the
> following.
>
> samba-tool drs kcc
> ERROR(runtime): DsExecuteKCC failed - (-1073610699, 'The operation
> cannot be performed.')
>
> Switching back to the old KCC clears the errors up.
>

On 9/21/2016 11:32 PM, Garming Sam wrote:
> I would have assumed that deleting all the NTDS connections for all the
> DCs would have remedied this issue. I'll write a patch to try to avoid
> this error in any case.
>
> Ignoring this error and answering your earlier question, it's often
hard
> to tell if the KCC is doing what is expected or not. To figure that out,
> you need a lot more information about the network topology
> configuration, which site links are defined, who belongs to which site
> in the database and probably some of the debug logs (running samba_kcc
> --debug manually). It also takes a little while for everything to settle
> down, until everyone is completely aware of who is online.
>
>
>
> Cheers,
>
> Garming
>
> On 22/09/16 00:32, lingpanda101 at gmail.com wrote:
>> I'm getting several KCC errors in each of my DC's. They are as
follows.
>>
>> [2016/09/21 08:06:12.364447,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc: AttributeError: 'NoneType'
object
>> has no attribute 'size'
>> [2016/09/21 08:06:12.381710,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../source4/dsdb/kcc/kcc_periodic.c:646(samba_kcc_done)
>>    ../source4/dsdb/kcc/kcc_periodic.c:646: Failed samba_kcc -
>> NT_STATUS_ACCESS_DENIED
>> [2016/09/21 08:11:12.870383,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc: Traceback (most recent call last):
>> [2016/09/21 08:11:12.870528,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:   File
>> "/usr/local/samba/sbin/samba_kcc", line 337, in
<module>
>> [2016/09/21 08:11:12.870588,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:
>> attempt_live_connections=opts.attempt_live_connections)
>> [2016/09/21 08:11:12.870639,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:   File
>>
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py",
>> line 2644, in run
>> [2016/09/21 08:11:12.870994,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:     all_connected >>
self.intersite(ping)
>> [2016/09/21 08:11:12.871046,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:   File
>>
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py",
>> line 1883, in intersite
>> [2016/09/21 08:11:12.871338,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:     all_connected >>
self.create_intersite_connections()
>> [2016/09/21 08:11:12.871398,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:   File
>>
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py",
>> line 1817, in create_intersite_connections
>> [2016/09/21 08:11:12.871676,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:     part, True)
>> [2016/09/21 08:11:12.871724,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:   File
>>
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py",
>> line 1769, in create_connections
>> [2016/09/21 08:11:12.871999,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:     partial_ok, detect_failed)
>> [2016/09/21 08:11:12.872048,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:   File
>>
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py",
>> line 1419, in create_connection
>> [2016/09/21 08:11:12.872272,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:     not
>> cn.is_equivalent_schedule(link_sched))):
>> [2016/09/21 08:11:12.872321,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:   File
>>
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/kcc_utils.py",
>> line 1223, in is_equivalent_schedule
>> [2016/09/21 08:11:12.872513,  0, pid=1087, effective(0, 0), real(0,
>> 0)] ../lib/util/util_runcmd.c:316(samba_runcmd_io_handler)
>>    /usr/local/samba/sbin/samba_kcc:     if ((self.schedule.size
!>> sched.size or
>>
>> Replication appears to report no errors. Running a KCC check I get the
>> following.
>>
>> samba-tool drs kcc
>> ERROR(runtime): DsExecuteKCC failed - (-1073610699, 'The operation
>> cannot be performed.')
>>
>> Switching back to the old KCC clears the errors up.
>>
For clarification I'll add a few things.

I initially deleted all the NTDS site links for each site and allowed 
the new KCC to create them. However it did not create them I believe 
correctly. By that I mean it defined what appeared to be a bridgehead 
server at each site. So I disabled the new KCC 'kccsrv:samba_kcc=false' 
in my smb.conf and allowed the full mesh to be used again. After all 
site links were recreated. I then switched the 'kccsrv:samba_kcc=true' 
in my smb.conf and that's what prompted the following errors above.

To further expand on my Topology, I have 3 sites. I'll call them A,B and 
C. Each site contains 2 DC's. Sites use different subnets and are 
connected via. fiber. Sites B and C should not be replication partners. 
They should only replicate with Site A(Default-First-Site-Name). With 
the new KCC after deleting all the NTDS links, Sites B and C Domain 
Controller #1 becomes the bridgehead server for that site. Domain 
Controller #2 at sites B and C only replicates with Domain Controller #1 
at it's respective site. So if the bridgehead server goes down, Domain 
Controller #2 at sites B and C will no longer receive changes.

The new KCC does prevent sites B and C from replicating with each other. 
That is correct. This isn't a huge issue for me. I can continue using 
the old KCC for now. The full mesh isn't detrimental to my network. 
Don't want to take up too much of your time. Thanks


-- 
-James