I've finally managed to sort out the problems with my patch to eliminate
QuartzBufferedTable and checked it in. I'm currently building a large
database to test this. The rate was quite a bit faster initially but
after around 200K documents it has settled down to being very slightly
faster than before.
Even if the eventual rate is much the same, this at least improves the
time taken to build smaller databases and reduces the complexity of the
code in the quartz backend.
Once this test has completed, I'll try upping the number of documents
per batch and see if that improves the rate.
Cheers,
Olly
On Wed, Aug 11, 2004 at 10:00:20PM +0100, Olly Betts wrote:> Once this test has completed, I'll try upping the number of documents > per batch and see if that improves the rate.Upping the number of documents per batch to 2000 increases the throughput quite dramatically - once things get going the gain is 50-60%, and it appears to be doing relatively better as the database size increases (currently this run has done about 700K documents). Cheers, Olly