samba - May 2016 - Duplicate ObjectSid values

> This certainly sounds stressful.
Yes!
> Another way to (on a backup, particularly given your history above) remove
the index is with samba-tool dbcheck --reindex.
Re-indexing...
completed re-index OK

0 root at empire:~[0] samba-tool  dns add empire chester-dc.example.com 
p-cats A 10.4.4.142  -U ash
Password for [CHESTER-DC\ash]:
Record added successfully

Thanks!
> The missing ntSecurityDescriptor is a curious issue.  Can you check if
> it or the whole record is really missing?  I'm guessing it is another
> index issue, stopping us finding the record rather than the record not
> being there.  Look over an ldbdump of the backend DB in sam.ldb.d/ if
> you have to, to confirm that.
> Andrew Bartlett
I haven't actually got ldbdump on the machine, and I can't see it in the
Debian packages. That said, I do appear to be able to add DNS records 
now, so I'm assuming it was the index. If you particularly want me to 
find out then I'll try to get a dump, but as long as its working I'm 
happy to leave it be!

Ash

>> Andrew Bartlett
> I haven't actually got ldbdump on the machine, and I can't see it
in
> the Debian packages. That said, I do appear to be able to add DNS 
> records now, so I'm assuming it was the index. If you particularly 
> want me to find out then I'll try to get a dump, but as long as its 
> working I'm happy to leave it be!
>
> Ash
Well, I will try to obtain that ldbdump

samba-tool dbcheck --reindex doesn't seem to have entirely worked. While 
we can add DNS records we can't add users. For example:

 > /usr/bin/samba-tool user add test.user --uid=test.user 
--random-password --uid-number=10226 --surname=user --given-name=test 
--job-title=Storekeeper --department=Repairs 
--mail-address=test.user at example.com --telephone-number=01244123456 
--gid-number=513
 > ERROR(ldb): Failed to add user 'test.user':  - 
../ldb_tdb/ldb_index.c:1216: Failed to re-index objectSid in CN=test 
user,CN=Users,DC=chester-dc,DC=example,DC=com - 
../ldb_tdb/ldb_index.c:1148: unique index violation on objectSid in 
CN=test user,CN=Users,DC=chester-dc,DC=example,DC=com

We also can't add a DC:

 > samba-tool domain join chester-dc.example.com DC -Uash 
--realm=CHESTER-DC.EXAMPLE.COM
 > Finding a writeable DC for domain 'chester-dc.example.com'
 > Found DC empire.chester-dc.example.com
 > Password for [CHESTER-DC\ash]:
 > workgroup is CHESTER-DC
 > realm is chester-dc.example.com
 > checking sAMAccountName
 > Adding CN=V-WARD,OU=Domain Controllers,DC=chester-dc,DC=example,DC=com
 > Join failed - cleaning up
 > checking sAMAccountName
 > ERROR(ldb): uncaught exception - LDAP error 68 
LDAP_ENTRY_ALREADY_EXISTS -  <00002071: ../ldb_tdb/ldb_index.c:1216: 
Failed to re-index objectSid in CN=V-WARD,OU=Domain 
Controllers,DC=chester-dc,DC=example,DC=com - 
../ldb_tdb/ldb_index.c:1148: unique index violation on objectSid in 
CN=V-WARD,OU=Domain Controllers,DC=chester-dc,DC=example,DC=com> <>
 >   File
"/usr/lib/python2.7/dist-packages/samba/netcmd/__init__.py",
line 175, in _run
 >     return self.run(*args, **kwargs)
 >   File
"/usr/lib/python2.7/dist-packages/samba/netcmd/domain.py",
line 555, in run
 >     machinepass=machinepass, use_ntvfs=use_ntvfs, 
dns_backend=dns_backend)
 >   File "/usr/lib/python2.7/dist-packages/samba/join.py", line
1172,
in join_DC
 >     ctx.do_join()
 >   File "/usr/lib/python2.7/dist-packages/samba/join.py", line
1075,
in do_join
 >     ctx.join_add_objects()
 >   File "/usr/lib/python2.7/dist-packages/samba/join.py", line
515, in
join_add_objects
 >     ctx.samdb.add(rec)

Or add a member:

 > root at p-bats:/etc/samba# net ads join -Uash
 > Enter ash's password:
 > Failed to join domain: failed to join domain
'CHESTER-DC.EXAMPLE.COM'
over rpc: None of the information to be translated has been translated.

On 16/05/16 16:41, ash-samba at comtek.co.uk wrote:
>
>>> Andrew Bartlett
>> I haven't actually got ldbdump on the machine, and I can't see
it in
>> the Debian packages. That said, I do appear to be able to add DNS 
>> records now, so I'm assuming it was the index. If you particularly 
>> want me to find out then I'll try to get a dump, but as long as its
>> working I'm happy to leave it be!
>>
>> Ash
>
Okay, I've managed to compile ldbdump, and doing  ./ldbdump 
/var/lib/samba/private/sam.ldb.d/DC%3DDOMAINDNSZONES,DC%3DCHESTER-DC,DC%3DEXAMPLE,DC%3DCOM.ldb
now only shows CN=Deleted Objects for the PC record which we considered 
suspect. Each entry does have an ntSecurityDescriptor, but the original 
object doesn't seem to be listed.

I believe that DNS is fine now, but the inability to add user records or 
join machines to the domain is a bigger problem!

Ash

On Mon, 2016-05-16 at 16:41 +0100, ash-samba at comtek.co.uk
wrote:
> > > Andrew Bartlett
> > I haven't actually got ldbdump on the machine, and I can't see
it
> > in 
> > the Debian packages. That said, I do appear to be able to add DNS 
> > records now, so I'm assuming it was the index. If you particularly
> > want me to find out then I'll try to get a dump, but as long as
its
> > working I'm happy to leave it be!
> > 
> > Ash
> 
> Well, I will try to obtain that ldbdump
> 
> samba-tool dbcheck --reindex doesn't seem to have entirely worked.
> While 
> we can add DNS records we can't add users. For example:
> 
>  > /usr/bin/samba-tool user add test.user --uid=test.user 
> --random-password --uid-number=10226 --surname=user --given-name=test
> --job-title=Storekeeper --department=Repairs 
> --mail-address=test.user at example.com --telephone-number=01244123456 
> --gid-number=513
>  > ERROR(ldb): Failed to add user 'test.user':  - 
> ../ldb_tdb/ldb_index.c:1216: Failed to re-index objectSid in CN=test 
> user,CN=Users,DC=chester-dc,DC=example,DC=com - 
> ../ldb_tdb/ldb_index.c:1148: unique index violation on objectSid in 
> CN=test user,CN=Users,DC=chester-dc,DC=example,DC=com
G'Day,

This is a serious situation.  What it means is that the nextRid value for that
DC points at a user account that already exists, so when we go to create it, the
create fails.

That, and the other issue, suggests you have had some serious DB corruption, and
this may not be the only issues.  Does a full dbcheck pass? (Not just the
reindex).

Is there another DC that still works, that you can replicate from? (but you
suggested other issues I think).

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba