samba - Nov 2015 - Domain join failure - error during DRS repl ADD: No objectClass found in replPropertyMetaData

If you plan to keep 4.3.x you can use --enable-fhs from configure script's
options (and the two others proposed by configure --help associated to
--enable-fhs). Doing that you should have most of your directories reused
(/etc/samba, /var/lib/samba...)

That worked for me as I don't care polluting /usr and others (I use VMs and
destroy them without a tear) on Centos 7 when I switched from Sernet
version to compiled version. No idea regarding Debian 'cause I'm not
allowed to used that system here :/

Cheers


2015-11-18 11:25 GMT+01:00 Andrew Bartlett <abartlet at samba.org>:
> On Mon, 2015-11-16 at 07:12 -0600, Matthew Delfino wrote:
> > On 2015.11.16, at 2:53 AM, Andrew Bartlett <abartlet at
samba.org>
> > wrote:
> >
> > > On Sun, 2015-11-15 at 20:36 -0600, Matthew Delfino wrote:
> > > > Hello Colleagues and Mentors,
> > > >
> > > > I'm attempting to join a Samba AD DC that I compiled
with samba
> > > > 4.3.1
> > > > on Ubuntu 14.04.3 to a group of three AD DCs, also running
Samba
> > > > on
> > > > Ubuntu 14.04.3, but each of them is running Canonical's
pre
> > > > -compiled
> > > > Samba package, v4.1.6.
> > >
> > > > It appears to me that this initial replication is choking
here:
> > > >
> > > >  No objectClass found in replPropertyMetaData for
> > > >
CN=kerio_emailgroup,OU=Services,OU=Groups,OU=knock,DC=mydomain,DC
> > > > =lan
> > > > !
> > > >
> > > > This makes me think something about my addition of
specialized
> > > > schema
> > > > has triggered, or tripped on, a bug somewhere downstream. I
> > > > searched
> > > > for strings on the internet with similar warnings and found
this
> > > > conversation between Rowland Penny and Luke Bigum:
> > > >
> > > >  https://lists.samba.org/archive/samba/2015-June/192516.html
> > > >
> > > > I'm wondering if I'm in a similar pickle. Could this
be the bug
> > > > I'm
> > > > hitting?
> > > >
> > > >  https://bugzilla.samba.org/show_bug.cgi?id=10973#c8
> > > >
> > > > Any advice on how to get myself out of this, via
work-arounds or
> > > > whatever, would be greatly appreciated. Thank you in
advance!
> > >
> > > Yes, this is the same issue.  You need to upgrade to Samba 4.3 on
> > > the
> > > source DC, run dbcheck, fix the issues, and then you can join
> > > another
> > > DC to the domain.
> > >
> > > Andrew Bartlett
> > >
> > > --
> > > Andrew Bartlett                       http://samba.org/~abartlet/
> > > Authentication Developer, Samba Team  http://samba.org
> > > Samba Developer, Catalyst IT
> > > http://catalyst.net.nz/services/samba
> >
> > I’m breathing a sigh of relief this morning in rainy Minneapolis
> > because this is very encouraging to hear. Thank you!
> >
> > HOWEVER… this opens a door to another room I’ve never been in because
> > I know the Debian/Ubuntu version of Samba 4.1.6 has been configured
> > with some tweaks to install it differently (sbin and bin binaries
> > installed into existing directories, conf file installed under
> > /etc/samba/, etc.).
> >
> > How would an expert do this? Just apt-get remove samba and install
> > the latest version from source? Any configure tweaks? Where would I
> > move the existing databases and other files? What do I need to keep?
> > What can I leave behind? I’m not expecting you to answer all of these
> > questions, but perhaps you know of a helpful tutorial online? I can’t
> > seem to find a good one…
>
> Do you intend to keep the 4.1 server after fixing it?
>
> If not, then you can install Samba from source, then run dbcheck from
> the 4.3 install pointed at the 4.1 sam.ldb file with samba-tool dbcheck
> -H /var/lib/samba/private/sam.ldb
>
> That will fix the DB, and allow you to join the domain.  The DB will
> only corrupt again when modifications are made to objects using
> additional schema - so just don't change those until you remove the 4.1
> server finally.
>
> Andrew Bartlett
>
> --
> Andrew Bartlett                       http://samba.org/~abartlet/
> Authentication Developer, Samba Team  http://samba.org
> Samba Developer, Catalyst IT
> http://catalyst.net.nz/services/samba
>
>
>
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions:  https://lists.samba.org/mailman/options/samba
>

On 2015.11.19, at 4:26 AM, mathias dufresne <infractory at gmail.com>
wrote:
> If you plan to keep 4.3.x you can use --enable-fhs from configure
script's options (and the two others proposed by configure --help associated
to --enable-fhs). Doing that you should have most of your directories reused
(/etc/samba, /var/lib/samba...)
> 
> That worked for me as I don't care polluting /usr and others (I use VMs
and destroy them without a tear) on Centos 7 when I switched from Sernet version
to compiled version. No idea regarding Debian 'cause I'm not allowed to
used that system here :/
> 
> Cheers
> 
> 
> 2015-11-18 11:25 GMT+01:00 Andrew Bartlett <abartlet at samba.org>:
> On Mon, 2015-11-16 at 07:12 -0600, Matthew Delfino wrote:
> > On 2015.11.16, at 2:53 AM, Andrew Bartlett <abartlet at
samba.org>
> > wrote:
> >
> > > On Sun, 2015-11-15 at 20:36 -0600, Matthew Delfino wrote:
> > > > Hello Colleagues and Mentors,
> > > >
> > > > I'm attempting to join a Samba AD DC that I compiled
with samba
> > > > 4.3.1
> > > > on Ubuntu 14.04.3 to a group of three AD DCs, also running
Samba
> > > > on
> > > > Ubuntu 14.04.3, but each of them is running Canonical's
pre
> > > > -compiled
> > > > Samba package, v4.1.6.
> > >
> > > > It appears to me that this initial replication is choking
here:
> > > >
> > > >  No objectClass found in replPropertyMetaData for
> > > >
CN=kerio_emailgroup,OU=Services,OU=Groups,OU=knock,DC=mydomain,DC
> > > > =lan
> > > > !
> > > >
> > > > This makes me think something about my addition of
specialized
> > > > schema
> > > > has triggered, or tripped on, a bug somewhere downstream. I
> > > > searched
> > > > for strings on the internet with similar warnings and found
this
> > > > conversation between Rowland Penny and Luke Bigum:
> > > >
> > > >  https://lists.samba.org/archive/samba/2015-June/192516.html
> > > >
> > > > I'm wondering if I'm in a similar pickle. Could this
be the bug
> > > > I'm
> > > > hitting?
> > > >
> > > >  https://bugzilla.samba.org/show_bug.cgi?id=10973#c8
> > > >
> > > > Any advice on how to get myself out of this, via
work-arounds or
> > > > whatever, would be greatly appreciated. Thank you in
advance!
> > >
> > > Yes, this is the same issue.  You need to upgrade to Samba 4.3 on
> > > the
> > > source DC, run dbcheck, fix the issues, and then you can join
> > > another
> > > DC to the domain.
> > >
> > > Andrew Bartlett
> > >
> > > --
> > > Andrew Bartlett                       http://samba.org/~abartlet/
> > > Authentication Developer, Samba Team  http://samba.org
> > > Samba Developer, Catalyst IT
> > > http://catalyst.net.nz/services/samba
> >
> > I’m breathing a sigh of relief this morning in rainy Minneapolis
> > because this is very encouraging to hear. Thank you!
> >
> > HOWEVER… this opens a door to another room I’ve never been in because
> > I know the Debian/Ubuntu version of Samba 4.1.6 has been configured
> > with some tweaks to install it differently (sbin and bin binaries
> > installed into existing directories, conf file installed under
> > /etc/samba/, etc.).
> >
> > How would an expert do this? Just apt-get remove samba and install
> > the latest version from source? Any configure tweaks? Where would I
> > move the existing databases and other files? What do I need to keep?
> > What can I leave behind? I’m not expecting you to answer all of these
> > questions, but perhaps you know of a helpful tutorial online? I can’t
> > seem to find a good one…
> 
> Do you intend to keep the 4.1 server after fixing it?
> 
> If not, then you can install Samba from source, then run dbcheck from
> the 4.3 install pointed at the 4.1 sam.ldb file with samba-tool dbcheck
> -H /var/lib/samba/private/sam.ldb
> 
> That will fix the DB, and allow you to join the domain.  The DB will
> only corrupt again when modifications are made to objects using
> additional schema - so just don't change those until you remove the 4.1
> server finally.
> 
> Andrew Bartlett
Andrew and Mathias, thank you, these are great options. I’m weighing moving to
an all-source option versus waiting for 4.3 to work its way through the pipeline
for packaging with the rest of what the Debian-based world offers through
aptitude. There’s really no telling how long the latter will take, and who knows
when I’m going to find myself in another tight spot where the powers that be put
the stops on releasing Samba updates for reasons that don’t matter to me.

And, you know, there’s always CentOS or SUSE. I really must learn to sever my
irrational emotional attachments to Ubuntu…

In any case, I have some good ideas that I can move forward with during
maintenance off hours. You guys are great. :-)

Thanks,
Matthew

©2015 KNOCK, inc.  All rights reserved. KNOCK is a registered trademark of
KNOCK, inc. This message and any attachments contain information, which is
confidential and/or privileged.  If you are not the intended recipient, please
refrain from any disclosure, copying, distribution or use of this information.
 Please be aware that such actions are prohibited.  If you have received this
transmission in error, kindly notify the sender by e-mail.  Your cooperation is
appreciated.

On 2015.11.19, at 9:59 AM, Matthew Delfino <mdelfino.list.samba at
KNOCKinc.com> wrote:
> On 2015.11.19, at 4:26 AM, mathias dufresne <infractory at gmail.com>
wrote:
> 
>> If you plan to keep 4.3.x you can use --enable-fhs from configure
script's options (and the two others proposed by configure --help associated
to --enable-fhs). Doing that you should have most of your directories reused
(/etc/samba, /var/lib/samba...)
>> 
>> That worked for me as I don't care polluting /usr and others (I use
VMs and destroy them without a tear) on Centos 7 when I switched from Sernet
version to compiled version. No idea regarding Debian 'cause I'm not
allowed to used that system here :/
>> 
>> Cheers
>> 
>> 
>> 2015-11-18 11:25 GMT+01:00 Andrew Bartlett <abartlet at
samba.org>:
>> On Mon, 2015-11-16 at 07:12 -0600, Matthew Delfino wrote:
>>> On 2015.11.16, at 2:53 AM, Andrew Bartlett <abartlet at
samba.org>
>>> wrote:
>>> 
>>>> On Sun, 2015-11-15 at 20:36 -0600, Matthew Delfino wrote:
>>>>> Hello Colleagues and Mentors,
>>>>> 
>>>>> I'm attempting to join a Samba AD DC that I compiled
with samba
>>>>> 4.3.1
>>>>> on Ubuntu 14.04.3 to a group of three AD DCs, also running
Samba
>>>>> on
>>>>> Ubuntu 14.04.3, but each of them is running Canonical's
pre
>>>>> -compiled
>>>>> Samba package, v4.1.6.
>>>> 
>>>>> It appears to me that this initial replication is choking
here:
>>>>> 
>>>>> No objectClass found in replPropertyMetaData for
>>>>>
CN=kerio_emailgroup,OU=Services,OU=Groups,OU=knock,DC=mydomain,DC
>>>>> =lan
>>>>> !
>>>>> 
>>>>> This makes me think something about my addition of
specialized
>>>>> schema
>>>>> has triggered, or tripped on, a bug somewhere downstream. I
>>>>> searched
>>>>> for strings on the internet with similar warnings and found
this
>>>>> conversation between Rowland Penny and Luke Bigum:
>>>>> 
>>>>> https://lists.samba.org/archive/samba/2015-June/192516.html
>>>>> 
>>>>> I'm wondering if I'm in a similar pickle. Could
this be the bug
>>>>> I'm
>>>>> hitting?
>>>>> 
>>>>> https://bugzilla.samba.org/show_bug.cgi?id=10973#c8
>>>>> 
>>>>> Any advice on how to get myself out of this, via
work-arounds or
>>>>> whatever, would be greatly appreciated. Thank you in
advance!
>>>> 
>>>> Yes, this is the same issue.  You need to upgrade to Samba 4.3
on
>>>> the
>>>> source DC, run dbcheck, fix the issues, and then you can join
>>>> another
>>>> DC to the domain.
>>>> 
>>>> Andrew Bartlett
>>>> 
>>>> --
>>>> Andrew Bartlett                      
http://samba.org/~abartlet/
>>>> Authentication Developer, Samba Team  http://samba.org
>>>> Samba Developer, Catalyst IT
>>>> http://catalyst.net.nz/services/samba
>>> 
>>> I’m breathing a sigh of relief this morning in rainy Minneapolis
>>> because this is very encouraging to hear. Thank you!
>>> 
>>> HOWEVER… this opens a door to another room I’ve never been in
because
>>> I know the Debian/Ubuntu version of Samba 4.1.6 has been configured
>>> with some tweaks to install it differently (sbin and bin binaries
>>> installed into existing directories, conf file installed under
>>> /etc/samba/, etc.).
>>> 
>>> How would an expert do this? Just apt-get remove samba and install
>>> the latest version from source? Any configure tweaks? Where would I
>>> move the existing databases and other files? What do I need to
keep?
>>> What can I leave behind? I’m not expecting you to answer all of
these
>>> questions, but perhaps you know of a helpful tutorial online? I
can’t
>>> seem to find a good one…
>> 
>> Do you intend to keep the 4.1 server after fixing it?
>> 
>> If not, then you can install Samba from source, then run dbcheck from
>> the 4.3 install pointed at the 4.1 sam.ldb file with samba-tool dbcheck
>> -H /var/lib/samba/private/sam.ldb
>> 
>> That will fix the DB, and allow you to join the domain.  The DB will
>> only corrupt again when modifications are made to objects using
>> additional schema - so just don't change those until you remove the
4.1
>> server finally.
>> 
>> Andrew Bartlett
> 
> Andrew and Mathias, thank you, these are great options. I’m weighing moving
to an all-source option versus waiting for 4.3 to work its way through the
pipeline for packaging with the rest of what the Debian-based world offers
through aptitude. There’s really no telling how long the latter will take, and
who knows when I’m going to find myself in another tight spot where the powers
that be put the stops on releasing Samba updates for reasons that don’t matter
to me.
> 
> And, you know, there’s always CentOS or SUSE. I really must learn to sever
my irrational emotional attachments to Ubuntu…
> 
> In any case, I have some good ideas that I can move forward with during
maintenance off hours. You guys are great. :-)
> 
> Thanks,
> Matthew
I have 3 domain controllers, running Samba 4.1.6 on Ubuntu 14.04.3 LTS in a
VMware virtual machine (part of the package install available from the
"apt-get install samba" command). My approach was to do a non-FSMO
first, the other non-FSMO second, then the FSMO last.

I started by shutting down all of these three VMs and doing a snapshot.

Next, I needed to backup the files and purge the system of all traces of Samba:

	sudo -s
	service samba stop
	service samba-ad-dc stop
	service nmbd stop

	cp -Rp /etc/samba/* /root/backup_queue/2015.11.22-SAMBA/root/etc/samba/.
	cp -Rp /var/lib/samba/*
/root/backup_queue/2015.11.22-SAMBA/root/var/lib/samba/.

	apt-get purge ^samba.*

Then, I download all the important packages for compiling from source on Ubuntu:

	apt-get install acl attr autoconf bison build-essential debhelper dnsutils
docbook-xml docbook-xsl flex gdb krb5-user libacl1-dev libaio-dev libattr1-dev
libblkid-dev libbsd-dev libcap-dev libgnutls28-dev libjson-perl libldap2-dev
libncurses5-dev libpam0g-dev libparse-yapp-perl libpopt-dev libreadline-dev perl
perl-modules pkg-config python-all-dev python-dev python-dnspython python-crypto
xsltproc zlib1g-dev

And don’t forget to exit because you don’t want to be root for the next
commands:

	exit

Prepare for, download and unpack the source:

	mkdir source
	cd source
	wget https://download.samba.org/pub/samba/stable/samba-4.3.1.tar.gz
	tar -zxf samba-4.3.1.tar.gz

Configure and compile (compiles always took, like 15 minutes on my system):

	cd samba-4.3.1/
	./configure --enable-fhs --prefix=/usr --sysconfdir=/etc --localstatedir=/var
--enable-debug
	make

Install it:

	sudo -s
	make install

Recover from those backups:

	cp -Rp /root/backup_queue/2015.11.22-SAMBA/root/etc/samba/* /etc/samba/.
	cp -Rp /root/backup_queue/2015.11.22-SAMBA/root/var/lib/samba/*
/var/lib/samba/.

Download the upstart conf and init script from samba wiki (as per
https://wiki.samba.org/index.php/Samba4/InitScript):

	wget -O /etc/init/samba-ad-dc.conf
'http://anonscm.debian.org/gitweb/?p=pkg-samba/samba.git;a=blob_plain;f=debian/samba-ad-dc.upstart;hb=HEAD'
	wget
"http://anonscm.debian.org/gitweb/?p=pkg-samba/samba.git;a=blob_plain;f=debian/samba.samba-ad-dc.init;h=3132d2e367675f822342a5b7bc2e50c046aa3b8f;hb=HEAD"
-O /etc/init.d/samba-ad-dc
	chmod 755 /etc/init.d/samba-ad-dc
	update-rc.d samba-ad-dc defaults

Restart:

	shutdown -r now

Log back in at restart. Make sure it’s running:

	service samba-ad-dc status

In my case, it was running each time (samba-ad-dc start/running, process X). Now
it’s time to fix stuff:

	sudo samba-tool dbcheck —fix

It found hundreds of issues and fixed them all. Lastly, I went on to check all
my work:

	sudo samba-tool drs showrepl

Says inbound and outbound updates are successful, so looked good….

	samba-tool ldapcmp ldap://dc00 ldap://dc01 -Uadministrator

Everything checked out EXCEPT "whenChanged," which was off on scores
of records. So, to make myself feel better, I did this:

	samba-tool ldapcmp --filter="whenChanged" ldap://dc01 ldap://dc00
-Uadministrator

I have tested the authentication systems on a few services that were bound to
these DCs and they all work - so far so good on that front.

I repeated the above steps for the second non-FSMO DC, then the FSMO DC. Every
one of them is now on Samba 4.3.1.

But the final test I did, after all the upgrades, was this:

	sudo samba-tool fsmo show

ERROR(<type 'exceptions.KeyError'>): uncaught exception - 'No
such element'
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/__init__.py",
line 175, in _run
    return self.run(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/fsmo.py", line
395, in run
    domaindnszonesMaster = get_fsmo_roleowner(samdb, domaindns_dn)
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/fsmo.py", line
42, in get_fsmo_roleowner
    master_owner = res[0]["fSMORoleOwner"][0]

Which concerns me. What could I do here to get me out of this pickle? Do I have
to (re)seize FSMO responsibilities? I did run this on the DC that was the FSMO
before all these shenanigans:

	samba-tool fsmo seize --role=all

This DC already has the 'rid' FSMO role
This DC already has the 'pdc' FSMO role
This DC already has the 'naming' FSMO role
This DC already has the 'infrastructure' FSMO role
This DC already has the 'schema' FSMO role
ERROR(<type 'exceptions.KeyError'>): uncaught exception - 'No
such element'
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/__init__.py",
line 175, in _run
    return self.run(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/fsmo.py", line
345, in run
    versionopts, force)
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/fsmo.py", line
301, in seize_dns_role
    master_owner = get_fsmo_roleowner(samdb, m.dn)
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/fsmo.py", line
42, in get_fsmo_roleowner
    master_owner = res[0]["fSMORoleOwner"][0]

The 'domaindns' and 'forestdns' roles seem to be choking it. Any
ideas?

The only other curiosity I have right now is, why are all the
"whenChanged" attributes off between the DCs? Is that normal after a
certain version of Samba, post v4.1.6?

One last thing: Thanks to the kind folks who sent me in the right direction with
this (Andrew, Rowland, Louis and Mathias). It seemed like a daunting task, but I
think I might be very close to nailing it...

Thanks,
Matthew

©2015 KNOCK, inc.  All rights reserved. KNOCK is a registered trademark of
KNOCK, inc. This message and any attachments contain information, which is
confidential and/or privileged.  If you are not the intended recipient, please
refrain from any disclosure, copying, distribution or use of this information.
 Please be aware that such actions are prohibited.  If you have received this
transmission in error, kindly notify the sender by e-mail.  Your cooperation is
appreciated.