samba - May 2016 - error during DRS repl ADD: No rDN found in replPropertyMetaData

The system described by 
https://lists.samba.org/archive/samba/2016-May/199829.html (Invalid data 
for index DN=@INDEX:OBJECTCLASS:DNSNODE) now appears to perform DNS 
updates correctly, all systems are 4.2.10-Debian, and we've been able to 
add a user and a new DC. (Thanks for the help!)

Synchronisation between v-ward (the new local DC), and empire isn't 
entirely working, though.

 > root at v-ward:/home/abc# /usr/bin/samba-tool drs replicate 
v-ward.chester-dc.example.com empire.chester-dc.example.com 
DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com
 > ERROR(<class 'samba.drs_utils.drsException'>): DsReplicaSync
failed -
drsException: DsReplicaSync failed (58, 'WERR_BAD_NET_RESP')
 >   File "/usr/lib/python2.7/dist-packages/samba/netcmd/drs.py",
line
345, in run
 >     drs_utils.sendDsReplicaSync(self.drsuapi, self.drsuapi_handle, 
source_dsa_guid, NC, req_options)
 >   File "/usr/lib/python2.7/dist-packages/samba/drs_utils.py",
line
83, in sendDsReplicaSync
 >     raise drsException("DsReplicaSync failed %s" % estr)

Looking in the log file, I see:

 > [2016/05/19 13:41:52.219968,  0] 
../source4/dsdb/repl/replicated_objects.c:783(dsdb_replicated_objects_commit)
 >   Failed to apply records: replmd_replicated_apply_add: error during 
DRS repl ADD: No rDN found in replPropertyMetaData for 
DC=DEEL032,DC=chester-dc.example.com,CN=MicrosoftDNS,DC=DomainDnsZones,DC=chester-dc,example!
 >   : Constraint violation
 > [2016/05/19 13:41:52.223745,  0] 
../source4/dsdb/repl/drepl_out_helpers.c:770(dreplsrv_op_pull_source_apply_changes_trigger)
 >   Failed to commit objects: 
WERR_GENERAL_FAILURE/NT_STATUS_INVALID_NETWORK_RESPONSE

I've found a similar case ( 
https://lists.samba.org/archive/samba/2014-September/185225.html ), but 
it doesn't seem like there was a good resolution.

We are considering simply deleting the 
DC=DEEL032,DC=chester-dc.example.com,CN=MicrosoftDNS,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com
object. Would this be unwise?

I don't know if this is relevant ( Andrew Bartlett suggested "It is
also
harmless, so I wouldn't worry too much until we can fix up
dbcheck" in a similar case - 
https://lists.samba.org/archive/samba/2014-October/186439.html ), but we 
also the the following when running dbcheck --cross-ncs

 > 0 root at empire:/home/abc[0] samba-tool dbcheck --cross-ncs --fix
 > Checking 18686 objects
 > ERROR: wrong dn[DC=DEELR013,CN=Deleted 
Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com] dc='DEELR013'
name='DEELR013\nDEL:1fa8058d-c987-4518-958d-10352c93c28a' 
new_dn[DC=DEELR013\0ADEL:1fa8058d-c987-4518-958d-10352c93c28a,CN=Deleted 
Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com]
 > Rename DC=DEELR013,CN=Deleted 
Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com to 
DC=DEELR013\0ADEL:1fa8058d-c987-4518-958d-10352c93c28a,CN=Deleted 
Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com? [y/N/all/none] y
 > Failed to rename object DC=DEELR013,CN=Deleted 
Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com into 
DC=DEELR013\0ADEL:1fa8058d-c987-4518-958d-10352c93c28a,CN=Deleted 
Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com : (64, 
'objectclass: structural objectClass dnsNode is not a valid child class 
for CN=Deleted Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com')
 > Checked 18686 objects (1 errors)

Can anybody recommend our next course of action?

Thanks

An awful response: change tombstoneLifetime : )

When deleting objects they go to recycle bin then to deleted objects then
are deleted. This if you have some recycle bin working which is not the
case by default I reckon.

tombstoneLifetime is the number of days AD has to keep deleted objects
before the real deletion. If you use the recycle bin this very same
tombstoneLifetime is also used to define how long objects will stay into
the recycle bin. There is a parameter named
"msDS-deletedObjectLifetime"
which is meant to define a different delay to keep data into recycle bin
which is not defined by default I think (samba does not activate by default
recycle bin so no need of that parameter).

So tombstoneLifetime = 1 and your deleted objects should be fully removed
in few days (one if my understanding is correct, but I bet on 2).

2016-05-19 14:58 GMT+02:00 ash-samba at comtek.co.uk <ash-samba at
comtek.co.uk>:
> The system described by
> https://lists.samba.org/archive/samba/2016-May/199829.html (Invalid data
> for index DN=@INDEX:OBJECTCLASS:DNSNODE) now appears to perform DNS updates
> correctly, all systems are 4.2.10-Debian, and we've been able to add a
user
> and a new DC. (Thanks for the help!)
>
> Synchronisation between v-ward (the new local DC), and empire isn't
> entirely working, though.
>
> > root at v-ward:/home/abc# /usr/bin/samba-tool drs replicate
> v-ward.chester-dc.example.com empire.chester-dc.example.com
> DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com
> > ERROR(<class 'samba.drs_utils.drsException'>):
DsReplicaSync failed -
> drsException: DsReplicaSync failed (58, 'WERR_BAD_NET_RESP')
> >   File
"/usr/lib/python2.7/dist-packages/samba/netcmd/drs.py", line 345,
> in run
> >     drs_utils.sendDsReplicaSync(self.drsuapi, self.drsuapi_handle,
> source_dsa_guid, NC, req_options)
> >   File
"/usr/lib/python2.7/dist-packages/samba/drs_utils.py", line 83,
> in sendDsReplicaSync
> >     raise drsException("DsReplicaSync failed %s" % estr)
>
> Looking in the log file, I see:
>
> > [2016/05/19 13:41:52.219968,  0]
>
../source4/dsdb/repl/replicated_objects.c:783(dsdb_replicated_objects_commit)
> >   Failed to apply records: replmd_replicated_apply_add: error during
DRS
> repl ADD: No rDN found in replPropertyMetaData for DC=DEEL032,DC>
chester-dc.example.com
> ,CN=MicrosoftDNS,DC=DomainDnsZones,DC=chester-dc,example!
> >   : Constraint violation
> > [2016/05/19 13:41:52.223745,  0]
>
../source4/dsdb/repl/drepl_out_helpers.c:770(dreplsrv_op_pull_source_apply_changes_trigger)
> >   Failed to commit objects:
> WERR_GENERAL_FAILURE/NT_STATUS_INVALID_NETWORK_RESPONSE
>
> I've found a similar case (
> https://lists.samba.org/archive/samba/2014-September/185225.html ), but
> it doesn't seem like there was a good resolution.
>
> We are considering simply deleting the DC=DEEL032,DC>
chester-dc.example.com,CN=MicrosoftDNS,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com
> object. Would this be unwise?
>
> I don't know if this is relevant ( Andrew Bartlett suggested "It
is also
> harmless, so I wouldn't worry too much until we can fix up
> dbcheck" in a similar case -
> https://lists.samba.org/archive/samba/2014-October/186439.html ), but we
> also the the following when running dbcheck --cross-ncs
>
> > 0 root at empire:/home/abc[0] samba-tool dbcheck --cross-ncs --fix
> > Checking 18686 objects
> > ERROR: wrong dn[DC=DEELR013,CN=Deleted
> Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com]
dc='DEELR013'
> name='DEELR013\nDEL:1fa8058d-c987-4518-958d-10352c93c28a'
> new_dn[DC=DEELR013\0ADEL:1fa8058d-c987-4518-958d-10352c93c28a,CN=Deleted
> Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com]
> > Rename DC=DEELR013,CN=Deleted
> Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com to
> DC=DEELR013\0ADEL:1fa8058d-c987-4518-958d-10352c93c28a,CN=Deleted
> Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com? [y/N/all/none] y
> > Failed to rename object DC=DEELR013,CN=Deleted
> Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com into
> DC=DEELR013\0ADEL:1fa8058d-c987-4518-958d-10352c93c28a,CN=Deleted
> Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com : (64,
> 'objectclass: structural objectClass dnsNode is not a valid child class
for
> CN=Deleted Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com')
> > Checked 18686 objects (1 errors)
>
> Can anybody recommend our next course of action?
>
> Thanks
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions:  https://lists.samba.org/mailman/options/samba
>

On 23/05/16 13:26, mathias dufresne wrote:
> An awful response: change tombstoneLifetime : )
>
> When deleting objects they go to recycle bin then to deleted objects then
> are deleted. This if you have some recycle bin working which is not the
> case by default I reckon.
>
> tombstoneLifetime is the number of days AD has to keep deleted objects
> before the real deletion. If you use the recycle bin this very same
> tombstoneLifetime is also used to define how long objects will stay into
> the recycle bin. There is a parameter named
"msDS-deletedObjectLifetime"
> which is meant to define a different delay to keep data into recycle bin
> which is not defined by default I think (samba does not activate by default
> recycle bin so no need of that parameter).
>
> So tombstoneLifetime = 1 and your deleted objects should be fully removed
> in few days (one if my understanding is correct, but I bet on 2).
>
That seems like a reasonable solution to delete the old deleted objects 
(which I don't think we need anyway), Andrew Bartlett was suggesting in 
https://lists.samba.org/archive/samba/2014-October/186439.html that the 
error harmless, though.

Is the Deleted Objects error even relevant to the DsReplicaSync failed 
(58, 'WERR_BAD_NET_RESP') ?

I'm happy to let the deleted objects expire naturally if we can just 
make replication work

On Thu, 2016-05-19 at 13:58 +0100, ash-samba at comtek.co.uk
wrote:
> I don't know if this is relevant ( Andrew Bartlett suggested "It
is
> also 
> harmless, so I wouldn't worry too much until we can fix up
> dbcheck" in a similar case - 
> https://lists.samba.org/archive/samba/2014-October/186439.html ), but
> we 
> also the the following when running dbcheck --cross-ncs
> 
>  > 0 root at empire:/home/abc[0] samba-tool dbcheck --cross-ncs --fix
>  > Checking 18686 objects
>  > ERROR: wrong dn[DC=DEELR013,CN=Deleted 
> Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com]
> dc='DEELR013' 
> name='DEELR013\nDEL:1fa8058d-c987-4518-958d-10352c93c28a' 
> new_dn[DC=DEELR013\0ADEL:1fa8058d-c987-4518-958d
> -10352c93c28a,CN=Deleted 
> Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com]
>  > Rename DC=DEELR013,CN=Deleted 
> Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com to 
> DC=DEELR013\0ADEL:1fa8058d-c987-4518-958d-10352c93c28a,CN=Deleted 
> Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com?
> [y/N/all/none] y
>  > Failed to rename object DC=DEELR013,CN=Deleted 
> Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com into 
> DC=DEELR013\0ADEL:1fa8058d-c987-4518-958d-10352c93c28a,CN=Deleted 
> Objects,DC=DomainDnsZones,DC=chester-dc,DC=example,DC=com : (64, 
> 'objectclass: structural objectClass dnsNode is not a valid child
> class 
> for CN=Deleted Objects,DC=DomainDnsZones,DC=chester
> -dc,DC=example,DC=com')
>  > Checked 18686 objects (1 errors)
> 
> Can anybody recommend our next course of action?
Please file a bug on this one.  I have a set of patches for this, and it is
always good to have a real-world user bug to attach to it.

As mentioned previously, it is harmless, but now that we can get it right, we
should.  The patches may not be backported, but it will help tracking if we do.

Thanks!

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba