Xapian discuss - Dec 2008 - Chert status?

Hi.

I have seen some writing about the Chert backend. Generally we use
sorting and ranges quite a lot, since we basically are indexing a
Relational Database. So these speedups are really interesting.

Are there any improvements in indexing/merge speed?

But what are peoples view on the maturity? Whats missing?

We're currently in "production" with a flint backend and since
it'll
cause a complete reindex (takes a few weeks with flint), It isn't 
something I'm going to "just do" if waiting a few weeks/months are
recommended.

Thanks.

-- 
Jesper

On Tue, Dec 09, 2008 at 08:47:34PM +0100, Jesper Krogh
wrote:
> I have seen some writing about the Chert backend. Generally we use
> sorting and ranges quite a lot, since we basically are indexing a
> Relational Database. So these speedups are really interesting.
> 
> Are there any improvements in indexing/merge speed?
There's nothing explicitly aimed at that, but the changes to document
length storage mean there's less data to write to disk, so it will
probably be faster.  I've not benchmarked this though.
> But what are peoples view on the maturity? Whats missing?
I think it is in pretty good shape.

The main thing is that there's a "missing piece" in the new value
handling.  There's an internal API which delivers a stream of
values by ascending docid in a particular slot, but the code which
does sorting and collapsing in the matcher doesn't make use of this
yet.
> We're currently in "production" with a flint backend and
since it'll
> cause a complete reindex (takes a few weeks with flint), It isn't 
> something I'm going to "just do" if waiting a few
weeks/months are
> recommended.
You can probably convert faster using copydatabase:

XAPIAN_PREFER_CHERT=1 copydatabase FLINT.DB CHERT.DB

But you might want to wait for 1.1.0.

Cheers,
    Olly