Hi, I've recently been running some performance tests on Xapian, using scriptindex to index Reuters news stories. I was surprised to get slower indexing with Xapian compiled for DANGEROUS database updates (143 docs/s vs. 169 docs/s) using the Flint back-end. This was with a database size of 790,000 docs, created from scratch, running on a generic Linux box and flushing every 10,000 docs. What are other people's experiences of DANGEROUS? Any ideas why I'm getting this result? cheers, Tom
On Fri, Sep 22, 2006 at 10:58:32AM +0100, Tom Mortimer wrote:> What are other people's experiences of DANGEROUS? Any ideas why I'm > getting this result?It was definitely faster for me when I did timings building the gmane index. Not sure if that was with quartz or flint, but it shouldn't make much difference that I can see. The main difference there is that I flush much less often. So perhaps flushing is more expensive for some reason in dangerous mode? Perhaps try increasing the flush threshold and see what happens... Hmm, also do you use "UNIQUE" in your index script? Cheers, Olly