Xapian discuss - Jan 2010 - postlist: Tag containing meta information is corrupt.

Greetings,

Using latest svn.

I've noticed the following error when performing index merging:

postlist:
baseB blocksize=8K items=33962 lastblock=534 revision=1 levels=2 root=459
B-tree checked okay
Tag containing meta information is corrupt.
postlist table errors found: 1

I can still search on this index (I've only checked very small indexes),
but merging is now a problem since I check the return code when merging
batches (this obviously returns code 1).

Is this something that I may have caused?

Thanks
Henry

On Mon, Jan 18, 2010 at 10:07:24AM +0200, Henry wrote:
> Using latest svn.
> 
> I've noticed the following error when performing index merging:
> 
> postlist:
> baseB blocksize=8K items=33962 lastblock=534 revision=1 levels=2 root=459
> B-tree checked okay
> Tag containing meta information is corrupt.
> postlist table errors found: 1
Um, that looks like the output from xapian-check, which checks for database
format errors - xapian-compact does merging.

Perhaps you need to explain exactly what you did to get this error, and also
which backend you're using.

Cheers,
    Olly

Found the reason:  one of the cluster nodes wasn't up-to-date - it was
using an older revision to index.

Sorry for spurious post.

Some more information:

xapian-compact = xapian-compact-1.1
xapian-check   = xapian-check-1.1
"batch" means a batch of previously indexed websites (performed across
a
cluster).

Merging a batch into DB.a completes OK (sometimes with "Tag containing
meta information is corrupt." reported by xapian-check).  This index can
be searched on and results are returned as normal.

Merging DB.a (and possibly DB.n) into empty foldertest/ completes OK, but
xapian-check on foldertest/ goes ape with:

postlist:
baseB blocksize=8K items=18252 lastblock=133 revision=1 levels=1 root=2
B-tree checked okay
Tag containing meta information is corrupt.
termfreq 2 != # of entries 1
collfreq 4 != sum wdf 3
termfreq 0 != # of entries 1
collfreq 0 != sum wdf 1
termfreq 5 != # of entries 4
collfreq 169 != sum wdf 163
...
First posting list chunk for term `00' follows previous chunk for the same
term
termfreq 0 != # of entries 32
collfreq 0 != sum wdf 1682
First chunk for term `' is a continuation chunk
...

This index cannot be searched on; it errors with: "RangeError: Value in
posting list too large."

I have no idea what's going on here :|