samba - Apr 2016 - Failed to re-index objectSid after botched DLZ back-end update

Alright, I'm taking the plunge: We're switching our three AD DCs from
Samba internal to BIND_DLZ back end.

I needed a version of BIND with DLZ, as it appears support for that is not so
ubiquitous.

I went here first:
https://wiki.samba.org/index.php/Using_BIND_DLZ_backend_with_secured_/_signed_DNS_updates

We use Ubuntu 14.04 here, and the Debian/Ubuntu instructions fail on apt-get
installing "libpcap2-dev". And, unsurprisingly, the "dget -x
http://ftp.de.debian.org/debian/pool/main/b/bind9/bind9_9.9.5.dfsg-7.dsc"
command is out of date, so I went into that FTP server to find the source, but
found myself too trepidatious to continue without the "libpcap2-dev"
library installed, so I looked for someone else's instructions.

I found this:
http://askubuntu.com/questions/630875/how-to-install-bind9-with-dlz-unbuntu-server-14-04

These instructions were more helpful, especially when combined with some of the
info about options included on
"Using_BIND_DLZ_backend_with_secured_/_signed_DNS_updates."

I got some deb packages compiled (v9.9.5) and brought them to one of my DCs.

I shut that DC down and snapshot it (I'm using vSphere here) and then
proceeded to attempt to switch it to DLZ backend.

It seemed to work, but later in the process I started having issues which
prompted me to rewind my snapshot.

Now, no matter what, every time I try to move forward again, I get this:

# sudo samba_upgradedns --dns-backend=BIND9_DLZ
Reading domain information
DNS accounts already exist
No zone file /var/lib/samba/private/dns/knockinc.loc.zone
DNS records will be automatically created
DNS partitions already exist
Adding dns-rhea account
Traceback (most recent call last):
  File "/usr/sbin/samba_upgradedns", line 438, in <module>
    "DNSNAME" : dnsname }
  File "/usr/lib/python2.7/dist-packages/samba/provision/common.py",
line 55, in setup_add_ldif
    ldb.add_ldif(data, controls)
  File "/usr/lib/python2.7/dist-packages/samba/__init__.py", line 225,
in add_ldif
    self.add(msg, controls)
_ldb.LdbError: (68, '../lib/ldb/ldb_tdb/ldb_index.c:1216: Failed to re-index
objectSid in CN=dns-rhea,CN=Users,DC=knockinc,DC=loc -
../lib/ldb/ldb_tdb/ldb_index.c:1148: unique index violation on objectSid in
CN=dns-rhea,CN=Users,DC=knockinc,DC=loc')

As best I can tell, a "dns-rhea" user (Rhea is the DC I'm trying
to upgrade) got made, the ojectSid ID got ticked up, the other DCs have that
number, but my rewound DC doesn't know it was ever made and I'm stumped.

What can I do to get out of this mess?

Thanks,
Matthew

©2016 KNOCK, inc.  All rights reserved. KNOCK is a registered trademark of
KNOCK, inc. This message and any attachments contain information, which is
confidential and/or privileged.  If you are not the intended recipient, please
refrain from any disclosure, copying, distribution or use of this information.
 Please be aware that such actions are prohibited.  If you have received this
transmission in error, kindly notify the sender by e-mail.  Your cooperation is
appreciated.

On 12/04/16 23:08, Matthew Delfino wrote:
> Alright, I'm taking the plunge: We're switching our three AD DCs
from Samba internal to BIND_DLZ back end.
>
> I needed a version of BIND with DLZ, as it appears support for that is not
so ubiquitous.
>
> I went here first:
https://wiki.samba.org/index.php/Using_BIND_DLZ_backend_with_secured_/_signed_DNS_updates
>
> We use Ubuntu 14.04 here, and the Debian/Ubuntu instructions fail on
apt-get installing "libpcap2-dev". And, unsurprisingly, the "dget
-x
http://ftp.de.debian.org/debian/pool/main/b/bind9/bind9_9.9.5.dfsg-7.dsc"
command is out of date, so I went into that FTP server to find the source, but
found myself too trepidatious to continue without the "libpcap2-dev"
library installed, so I looked for someone else's instructions.
>
> I found this:
http://askubuntu.com/questions/630875/how-to-install-bind9-with-dlz-unbuntu-server-14-04
>
> These instructions were more helpful, especially when combined with some of
the info about options included on
"Using_BIND_DLZ_backend_with_secured_/_signed_DNS_updates."
>
> I got some deb packages compiled (v9.9.5) and brought them to one of my
DCs.
>
> I shut that DC down and snapshot it (I'm using vSphere here) and then
proceeded to attempt to switch it to DLZ backend.
>
> It seemed to work, but later in the process I started having issues which
prompted me to rewind my snapshot.
>
> Now, no matter what, every time I try to move forward again, I get this:
>
> # sudo samba_upgradedns --dns-backend=BIND9_DLZ
> Reading domain information
> DNS accounts already exist
> No zone file /var/lib/samba/private/dns/knockinc.loc.zone
> DNS records will be automatically created
> DNS partitions already exist
> Adding dns-rhea account
> Traceback (most recent call last):
>    File "/usr/sbin/samba_upgradedns", line 438, in <module>
>      "DNSNAME" : dnsname }
>    File
"/usr/lib/python2.7/dist-packages/samba/provision/common.py", line 55,
in setup_add_ldif
>      ldb.add_ldif(data, controls)
>    File "/usr/lib/python2.7/dist-packages/samba/__init__.py",
line 225, in add_ldif
>      self.add(msg, controls)
> _ldb.LdbError: (68, '../lib/ldb/ldb_tdb/ldb_index.c:1216: Failed to
re-index objectSid in CN=dns-rhea,CN=Users,DC=knockinc,DC=loc -
../lib/ldb/ldb_tdb/ldb_index.c:1148: unique index violation on objectSid in
CN=dns-rhea,CN=Users,DC=knockinc,DC=loc')
>
> As best I can tell, a "dns-rhea" user (Rhea is the DC I'm
trying to upgrade) got made, the ojectSid ID got ticked up, the other DCs have
that number, but my rewound DC doesn't know it was ever made and I'm
stumped.
>
> What can I do to get out of this mess?
>
> Thanks,
> Matthew
>
> ©2016 KNOCK, inc.  All rights reserved. KNOCK is a registered trademark of
KNOCK, inc. This message and any attachments contain information, which is
confidential and/or privileged.  If you are not the intended recipient, please
refrain from any disclosure, copying, distribution or use of this information. 
Please be aware that such actions are prohibited.  If you have received this
transmission in error, kindly notify the sender by e-mail.  Your cooperation is
appreciated.
>
>
I understand this is a known problem and the fix is, change to the 
internal dns first, then change to Bind9 again.

Rowland

On Tue, 2016-04-12 at 17:08 -0500, Matthew Delfino
wrote:
> Alright, I'm taking the plunge: We're switching our three AD DCs
from
> Samba internal to BIND_DLZ back end.
> 
> I needed a version of BIND with DLZ, as it appears support for that
> is not so ubiquitous.
> 
> I went here first: https://wiki.samba.org/index.php/Using_BIND_DLZ_ba
> ckend_with_secured_/_signed_DNS_updates
> 
> We use Ubuntu 14.04 here, and the Debian/Ubuntu instructions fail on
> apt-get installing "libpcap2-dev". And, unsurprisingly, the
"dget -x
> http://ftp.de.debian.org/debian/pool/main/b/bind9/bind9_9.9.5.dfsg-7.
> dsc" command is out of date, so I went into that FTP server to find
> the source, but found myself too trepidatious to continue without the
> "libpcap2-dev" library installed, so I looked for someone
else's
> instructions.
> 
> I found this: http://askubuntu.com/questions/630875/how-to-install-bi
> nd9-with-dlz-unbuntu-server-14-04
> 
> These instructions were more helpful, especially when combined with
> some of the info about options included on
> "Using_BIND_DLZ_backend_with_secured_/_signed_DNS_updates."
> 
> I got some deb packages compiled (v9.9.5) and brought them to one of
> my DCs.
> 
> I shut that DC down and snapshot it (I'm using vSphere here) and then
> proceeded to attempt to switch it to DLZ backend.
> 
> It seemed to work, but later in the process I started having issues
> which prompted me to rewind my snapshot.
This appears to have been your issue.
> Now, no matter what, every time I try to move forward again, I get
> this:
> 
> # sudo samba_upgradedns --dns-backend=BIND9_DLZ
> Reading domain information
> DNS accounts already exist
> No zone file /var/lib/samba/private/dns/knockinc.loc.zone
> DNS records will be automatically created
> DNS partitions already exist
> Adding dns-rhea account
> Traceback (most recent call last):
>   File "/usr/sbin/samba_upgradedns", line 438, in <module>
>     "DNSNAME" : dnsname }
>   File
"/usr/lib/python2.7/dist-packages/samba/provision/common.py",
> line 55, in setup_add_ldif
>     ldb.add_ldif(data, controls)
>   File "/usr/lib/python2.7/dist-packages/samba/__init__.py", line
> 225, in add_ldif
>     self.add(msg, controls)
> _ldb.LdbError: (68, '../lib/ldb/ldb_tdb/ldb_index.c:1216: Failed to
> re-index objectSid in CN=dns-rhea,CN=Users,DC=knockinc,DC=loc -
> ../lib/ldb/ldb_tdb/ldb_index.c:1148: unique index violation on
> objectSid in CN=dns-rhea,CN=Users,DC=knockinc,DC=loc')
> 
> As best I can tell, a "dns-rhea" user (Rhea is the DC I'm
trying to
> upgrade) got made, the ojectSid ID got ticked up, the other DCs have
> that number, but my rewound DC doesn't know it was ever made and
I'm
> stumped.
> 
> What can I do to get out of this mess?
I take it you have another DC.  I suggest re-replicating from that as a new
join, because you have corrupted the replication state by restoring to the
previous snapshot and then re-using a RID.

That is my best guess anyway - that error shouldn't be possible, it means
that a SID has been issued twice despite the RID pools.

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba