Rails - Mar 2009 - scaling full text indexing(ferret vs solr vs hyperstraier)

Hi,

Does any have experience scaling full text search in RoR?

Right now our project is running a simple setup with ferret and
acts_as_ferret. We are thinking about deploying a feature that would
send 50x more search requests.

So we probably have to rethink our solution. How do services like
search.twitter.com (the former Summize) use?

Or in what direction should I look?

--
Thanks,
M.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Marcelo Barbudas wrote:
> Right now our project is running a simple setup with ferret and
> acts_as_ferret. We are thinking about deploying a feature that would
> send 50x more search requests.
With Ferret you can scale reads horizontally: you can have multiple read 
servers on a single index. You can only have one write server on a 
single index or you''ll risk data corruption.

Another strategy is partitioning: having separate indices for buckets of 
data. Each index could run on it''s own server or cluster of servers.

--
Roderick van Domburg
http://www.nedforce.com
-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

> With Ferret you can scale reads horizontally: you can have multiple read
> servers on a single index. You can only have one write server on a
> single index or you''ll risk data corruption.
>
> Another strategy is partitioning: having separate indices for buckets of
> data. Each index could run on it''s own server or cluster of
servers.
>
Would it be easier to scale with hyperestraier or something else?

--
M.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

On Sun, Mar 8, 2009 at 1:14 PM, Marcelo Barbudas
<nostef-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
> Does any have experience scaling full text search in RoR?
One option that worked very well for me is ultrasphinx.

  http://blog.evanweaver.com/files/doc/fauna/ultrasphinx/files/README.html

IIRC, 2 limitations of ultrasphinx are:

* new entries can only be found after reindexing
  (full reindexing or delta indexing)
* you need a separate sphinx process somewhere on a server
  (if you run a shared hosting system, this may be an issue)

If you can live with those 2 limitations, ultrasphinx is a very good candidate.

HTH,

Peter

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---