Xapian discuss - Apr 2007 - A perhaps obvious speed tip

Hey folks,

So, this is probably obvious to some of you, but it wasn't obvious to me:

Put a bunch of adds() and removes() in a begin_transaction() and
commit_transaction() block and you'll drastically (!) increase the
speed at which you can add your documents into your database.

It wasn't clear to be that Xapian had transactions of the sort, and I
believe I tried using them in an older version and they didn't work.
It seems they don't work in >= 0.9.9 but not in 0.9.6, at least when I
tried using them from the python bindings.

Transactions rock!

--Philip Neustrom

On Thu, Apr 05, 2007 at 07:13:44PM -0700, Philip Neustrom
wrote:
> So, this is probably obvious to some of you, but it wasn't obvious to
me:
> 
> Put a bunch of adds() and removes() in a begin_transaction() and
> commit_transaction() block and you'll drastically (!) increase the
> speed at which you can add your documents into your database.
Hmm, how many operations are you performing per transaction?

If it's lots (more than 1000), then this is the same effect you'd get by
setting XAPIAN_FLUSH_THRESHOLD.  If not, then I find it hard to guess
why putting them in a transaction would help.
> It wasn't clear to be that Xapian had transactions of the sort, and I
> believe I tried using them in an older version and they didn't work.
> It seems they don't work in >= 0.9.9 but not in 0.9.6, at least when
I
> tried using them from the python bindings.
They were implemented in 0.9.7.

Cheers,
    Olly