Ferret talk - Sep 2006 - RangeFilter performance

I''m using a RangeFilter to limit a search to only the most recently 
added documents. My index is about 150 000 articles and the RangeFilter 
typically selects about 1000 of them to run a batch of searches against.

The performance is great as long as my index is newly optimized. As soon 
as add a few new documents the average search time for a batch of 
searches using the rangefilter increases about 10 times. And if i keep 
adding a few documents the search time increases even more. Until I 
optimize the index again, and performance is back to superfast. I can''t
do optimize as often as I would like since it takes a long time.

Have anyone excperienced anything similiar?

Thanks/David





-- 
Posted via http://www.ruby-forum.com/.

On 9/27/06, David Wennergren <david.wennergren at gmail.com>
wrote:
> I''m using a RangeFilter to limit a search to only the most
recently
> added documents. My index is about 150 000 articles and the RangeFilter
> typically selects about 1000 of them to run a batch of searches against.
>
> The performance is great as long as my index is newly optimized. As soon
> as add a few new documents the average search time for a batch of
> searches using the rangefilter increases about 10 times. And if i keep
> adding a few documents the search time increases even more. Until I
> optimize the index again, and performance is back to superfast. I
can''t
> do optimize as often as I would like since it takes a long time.
>
> Have anyone excperienced anything similiar?
>
> Thanks/David
>
>
Hi David,

I know about this problem. It also occurs when sorting large indexes.
I''m currently trying to work on a solution. I''ll keep you
updated.

Cheers,
Dave