samba - Oct 2014 - Multi domain controller environment Ubuntu 12.04, replication and DNS updates broken

Hi all,

Am posting this again with a more helpful subject line...

My 5 DC production domain (4.1.7 Ubuntu 12.04) is in a bit of a state.

I attempted an upgrade from 4.1.5 to 4.1.7 which appeared to work, but now
we have replication errors and am unable to add any new DNS entries. I am
now certain that we've fallen foul of the DomainDnsZones DeletedObjects
problem that I've been reading about in various posts on the lists.

My DC=DOMAINDNSZONES,DC=EXAMPLE,DC=INTERNAL,DC=COM.ldb files are now
between 3 and 4GB on each of the DC's. Doing an ldapsearch ( ldbsearch -H
DC=DOMAINDNSZONES,DC=ESSENCE,DC=INTERNAL,DC=COM.ldb 'isDeleted=TRUE' dn
)on
each DC returns a different number of objects ranging from 387000 down to
88000 on the FSMO DC. Almost all of these are stale isDeleted entries.

I have lowered the tombstoneLifetime setting as suggested by other posters
on the lists and this appears to be slowly (very slowly) lowering the
number of records within the ldb domaindnszones file, my hope is that they
will lower sufficiently so that I can join a new working 4.1.12 DC to
domain.

I am currently attempting a Bind migration on a test DC as this is toted as
a possible fix (any successes out there with this?).

A matter of note for the lists: When I originally provisioned my domain
(classic upgrade from Samba3) I created a new OU for Groups and moved all
groups into it, this is a mistake if you want to migrate to Bind as the
migration script needs CN=DnsAdmins to be in Users OU, if it isn't the
script errors. I moved DnsAdmins back to Users to get the script to
complete.

At present I'm holding the domain together with bits of string and sticky
tape - having to reboot one of my DC's every 30 mins just to keep things
ticking over.

I have tried many variations of joining a new DC to the domain but that has
failed, so my current plan is to create a test version of my FSMO DC using
BIND_DLZ (using a current snapshot of the FSMO DC) and get things to a
working state there, and then replace this on the production site and
re-join new DC's to rebuild things. Obviously, not best practice but I
can't think of any other way of getting things stable again.

I have tried manually editing the .ldb files but they are so inflated now
that any vim edits just time out and error.

Thanks,
Chris.

-- 
ACS (Alavoine Computer Services Ltd)
Chris Alavoine
mob +44 (0)7724 710 730
www.alavoinecs.co.uk
http://twitter.com/#!/alavoinecs
http://www.linkedin.com/pub/chris-alavoine/39/606/192

ah.. 

 DeletedObjects ... and replication errors. 

This is a known samba 4 bug. 
see also :  https://bugzilla.samba.org/show_bug.cgi?id=10398 
Look at the post : No objectClass found in replPropertyMetaData *(was thread
:replication issues solved by adding GUID name ... )
by me. ;-)  today an old e-mail entered the mailing list, which involves the
problem you discribe.

I dont know it the fix in in the latest samba release yet.
maybe someone of samba knows. 

Karolin can you answhere this? or pass this to someone who knows. 


Louis

>-----Oorspronkelijk bericht-----
>Van: chrisa at acs-info.co.uk 
>[mailto:samba-bounces at lists.samba.org] Namens Chris Alavoine
>Verzonden: woensdag 1 oktober 2014 9:31
>Aan: samba at lists.samba.org
>Onderwerp: [Samba] Multi domain controller environment Ubuntu 
>12.04, replication and DNS updates broken
>
>Hi all,
>
>Am posting this again with a more helpful subject line...
>
>My 5 DC production domain (4.1.7 Ubuntu 12.04) is in a bit of a state.
>
>I attempted an upgrade from 4.1.5 to 4.1.7 which appeared to 
>work, but now
>we have replication errors and am unable to add any new DNS 
>entries. I am
>now certain that we've fallen foul of the DomainDnsZones DeletedObjects
>problem that I've been reading about in various posts on the lists.
>
>My DC=DOMAINDNSZONES,DC=EXAMPLE,DC=INTERNAL,DC=COM.ldb files are now
>between 3 and 4GB on each of the DC's. Doing an ldapsearch ( 
>ldbsearch -H
>DC=DOMAINDNSZONES,DC=ESSENCE,DC=INTERNAL,DC=COM.ldb 
>'isDeleted=TRUE' dn )on
>each DC returns a different number of objects ranging from 
>387000 down to
>88000 on the FSMO DC. Almost all of these are stale isDeleted entries.
>
>I have lowered the tombstoneLifetime setting as suggested by 
>other posters
>on the lists and this appears to be slowly (very slowly) lowering the
>number of records within the ldb domaindnszones file, my hope 
>is that they
>will lower sufficiently so that I can join a new working 4.1.12 DC to
>domain.
>
>I am currently attempting a Bind migration on a test DC as 
>this is toted as
>a possible fix (any successes out there with this?).
>
>A matter of note for the lists: When I originally provisioned my domain
>(classic upgrade from Samba3) I created a new OU for Groups 
>and moved all
>groups into it, this is a mistake if you want to migrate to Bind as the
>migration script needs CN=DnsAdmins to be in Users OU, if it isn't the
>script errors. I moved DnsAdmins back to Users to get the script to
>complete.
>
>At present I'm holding the domain together with bits of string 
>and sticky
>tape - having to reboot one of my DC's every 30 mins just to 
>keep things
>ticking over.
>
>I have tried many variations of joining a new DC to the domain 
>but that has
>failed, so my current plan is to create a test version of my 
>FSMO DC using
>BIND_DLZ (using a current snapshot of the FSMO DC) and get things to a
>working state there, and then replace this on the production site and
>re-join new DC's to rebuild things. Obviously, not best practice but I
>can't think of any other way of getting things stable again.
>
>I have tried manually editing the .ldb files but they are so 
>inflated now
>that any vim edits just time out and error.
>
>Thanks,
>Chris.
>
>-- 
>ACS (Alavoine Computer Services Ltd)
>Chris Alavoine
>mob +44 (0)7724 710 730
>www.alavoinecs.co.uk
>http://twitter.com/#!/alavoinecs
>http://www.linkedin.com/pub/chris-alavoine/39/606/192
>-- 
>To unsubscribe from this list go to the following URL and read the
>instructions:  https://lists.samba.org/mailman/options/samba
>
>