Thanks for the response Jens.
Indeed I am sorting by something other than relevancy, so that would
explain it. Optimized, it''s extremely fast and handles a good load,
but new records kill it until I optimize.
I haven''t tried :merge_factor as I wasn''t aware of it.
I''m not sure
it will help given the above.
Regarding the re-index locking code, it''s not 100% threadsafe or
necessarily the desired behavior (calling rebuild_index when the last
rebuild isn''t finished won''t result in a new index,) but it
works for
me. I just added this to the beginning of rebuild_index:
if @rebuilding
@logger.warn "already rebuilding index - will lock until
rebuilding is complete"
while @rebuilding
sleep 1
end
return
end
@rebuilding = true
The big problem was in new_index_for, which always uses the same
path, and causes the multiple rebuilds to be stomping on each other.
It''s not necessary to fix this after adding the above though.
Thanks again.
--
Alex Neth
Liivid Inc / cribQ
www.liivid.com / www.cribq.com
On Jan 28, 2008, at 3:45 PM, ferret-talk-request at rubyforge.org wrote:
> Date: Sat, 26 Jan 2008 10:53:22 +0100
> From: Jens Kraemer <jk at jkraemer.net>
> Subject: Re: [Ferret-talk] ferret success stories?
> To: ferret-talk at rubyforge.org
> Message-ID: <20080126095322.GC31262 at thunder.jkraemer.net>
> Content-Type: text/plain; charset=iso-8859-1
>
> Hi Alex,
>
> sorry about not responding to that mail, been pretty busy with other
> stuff at that time.
>
> I just dugg it out of my mail archive now and yes, I''d like to
have a
> look at your locking code to prevent parallel index rebuilds :-)
>
> Regarding your optimization troubles - I''m afraid I don''t
know what''s
> going wrong there with your index being slow when you don''t
> optimize it
> (unless you''re sorting your search results by something else than
> relevancy - that is indeed known to be slow when the index isn''t
> optimized).
>
> Did you try to tweak some of Ferret''s more obscure indexing
parameters
> like :merge_factor (lowering it (the Ferret shortcut suggests a
> value of
> 2 or 3 for better search performance) will let Ferret merge segments
> more frequently so on average there are less files in the index which
> should improve search performance)?
>
>
> Cheers,
> Jens
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/ferret-talk/attachments/20080128/63a02a87/attachment.html
