samba - Dec 2010 - Samba4 LDAP LDB Indexes

I have a system running Samba 4 Alpha 11, and I seem to have a corrupted LDB
file in my directory. (Probably the result of taking a backup without using
tdbbackup). Right now, running tdbbackup on the file produces an error
message similar to the following:

Failed to insert into DC=WWW,DC=EXAMPLE,DC=COM.ldb.bak.tmp
failed to copy DC=WWW,DC=EXAMPLE,DC=COM.ldb

If I run 

ldbsearch -H "DC=WWW,DC=EXAMPLE,DC=COM.ldb" -a >
www.example.com.ldif

I get a text file containing a text dump of my directory. I can reload this
into a new tdb file using ldbadd, but only if I remove the one object called
@INDEXLIST. Without removing this object, ldbadd simply hangs, and then
eventually dies with the following error:

ltdb: tdb(newfile.ldb): tdb_transaction_cancel: no transaction

Failed to commit transaction: Failed to store index records in transaction
commit: Other

So, my question is: am I on the right track in rebuilding this LDB file? I
know I should have kept a good backup, but right now I would really like to
recover the file, and it seems that all (or most) of my data is still there,
if it can only be made accessible. It seems that this should work, but it
doesn't seem to be importing any indexes, and if the file is not indexed I
fear that performance will not be acceptable (if it even works at all).

Any suggestions would be appreciated!

~Daniel
-- 
View this message in context:
http://samba.2283325.n4.nabble.com/Samba4-LDAP-LDB-Indexes-tp3075866p3075866.html
Sent from the Samba - General mailing list archive at Nabble.com.

As a quick followup--it seems that the ldbadd command WILL work, if I isolate
the @INDEXLIST object and add it first to the newly created ldb file, and
then load the remaining ldif.
-- 
View this message in context:
http://samba.2283325.n4.nabble.com/Samba4-LDAP-LDB-Indexes-tp3075866p3075890.html
Sent from the Samba - General mailing list archive at Nabble.com.

Hi

This sounds like a question for the samba-technical list, which is
still the best place to ask about issues with samba 4.  I have copied
my reply there.

On 7 December 2010 08:49, dmcfeeters <domains at fiforms.org>
wrote:
>
> I have a system running Samba 4 Alpha 11, and I seem to have a corrupted
LDB
> file in my directory. (Probably the result of taking a backup without using
> tdbbackup). Right now, running tdbbackup on the file produces an error
> message similar to the following:
>
> Failed to insert into DC=WWW,DC=EXAMPLE,DC=COM.ldb.bak.tmp
> failed to copy DC=WWW,DC=EXAMPLE,DC=COM.ldb
>
> If I run
>
> ldbsearch -H "DC=WWW,DC=EXAMPLE,DC=COM.ldb" -a >
www.example.com.ldif
>
> I get a text file containing a text dump of my directory. I can reload this
> into a new tdb file using ldbadd, but only if I remove the one object
called
> @INDEXLIST. Without removing this object, ldbadd simply hangs, and then
> eventually dies with the following error:
>
> ltdb: tdb(newfile.ldb): tdb_transaction_cancel: no transaction
>
> Failed to commit transaction: Failed to store index records in transaction
> commit: Other
>
> So, my question is: am I on the right track in rebuilding this LDB file? I
> know I should have kept a good backup, but right now I would really like to
> recover the file, and it seems that all (or most) of my data is still
there,
> if it can only be made accessible. It seems that this should work, but it
> doesn't seem to be importing any indexes, and if the file is not
indexed I
> fear that performance will not be acceptable (if it even works at all).
>
> Any suggestions would be appreciated!
>
> ~Daniel
-- 
Michael Wood <esiotrot at gmail.com>

On 07/12/2010 09:49, dmcfeeters wrote:
> I have a system running Samba 4 Alpha 11, and I seem to have a corrupted
LDB
> file in my directory. (Probably the result of taking a backup without using
> tdbbackup). Right now, running tdbbackup on the file produces an error
> message similar to the following:
>
Won't be a too bad idea to move to at least alpha13 or even better the 
latest git revision ...
> Failed to insert into DC=WWW,DC=EXAMPLE,DC=COM.ldb.bak.tmp
> failed to copy DC=WWW,DC=EXAMPLE,DC=COM.ldb
>
> If I run
>
> ldbsearch -H "DC=WWW,DC=EXAMPLE,DC=COM.ldb" -a> 
www.example.com.ldif
>
> I get a text file containing a text dump of my directory. I can reload this
> into a new tdb file using ldbadd, but only if I remove the one object
called
> @INDEXLIST. Without removing this object, ldbadd simply hangs, and then
> eventually dies with the following error:
> ltdb: tdb(newfile.ldb): tdb_transaction_cancel: no transaction
>
> Failed to commit transaction: Failed to store index records in transaction
> commit: Other
Well that's not a good idea to rebuild index out of the control of samba 
so basically doing ldbadd like this won't work very
well
> So, my question is: am I on the right track in rebuilding this LDB file? I
> know I should have kept a good backup, but right now I would really like to
> recover the file, and it seems that all (or most) of my data is still
there,
> if it can only be made accessible. It seems that this should work, but it
> doesn't seem to be importing any indexes, and if the file is not
indexed I
> fear that performance will not be acceptable (if it even works at all).
>
No it's not the good way to do. A possible way is to remove all the 
@INDEX entry of the ldbsearch dump, add an empty @INDEXLIST
or with one entry ie:

dn: @INDEXLIST
@IDXATTR: sn
@IDXONE: 1
distinguishedName: @INDEXLIST


Normally at this moment you should have a valid tdb file but with broken 
indexes, verify that you can tdbdump it.
If so then edit the sam.ldb like this:

ldbedit -H sam.ldb -o modules:

locate the @INDEXLIST entry and remove almost every thing so that it 
looks like


dn: @INDEXLIST
@IDXATTR: sn
@IDXONE: 1
distinguishedName: @INDEXLIST

then do ldbsearch -H sam.ldb, got take a coffee if you provision is big 
(lot of entries ) if not it should be quick (~1 minute), by altering the 
indexlist in the sam.ldb we force samba4 to reindex the whole ldb.

You can check that your sam.ldb + DC=domain, DC=tld.ldb have been 
reindexed by doing:

ldbsearch -H sam.ldb -o modules:

And look at the @INDEXLIST entry it should be "big" (> 20 lines,
when
just before it was just 4 lines), doing the same search on DC=domain, 
DC=tld.ldb should also give you a "big" list.


Then you will repeat: I will use samba_backup (in source4/scripting/bin) 
to make my backup ! (please check that the backup made are really 
working ...).

Good luck.

Matthieu.

-- 
Matthieu Patou
Samba Team        http://samba.org
Private repo      http://git.samba.org/?p=mat/samba.git;a=summary