hello, I am actually indexing thousands of 1ko text documents using ferret and acts_as_ferret, and i face performance problems. I takes me hours to index 20 000 1ko text documents. Methology used : I create and object, fulfill it with the text, and save it. So it is automaticly indexed. Is there a way to make it faster ? ( remove the auto optimize option somewhere ?) Thank you if you have any ideas... Regards -- Posted via http://www.ruby-forum.com/.
On Fri, Aug 31, 2007 at 06:24:35PM +0200, Hakita Hakita wrote:> hello, > > I am actually indexing thousands of 1ko text documents using ferret and > acts_as_ferret, and i face performance problems. > I takes me hours to index 20 000 1ko text documents. > > Methology used : > > I create and object, fulfill it with the text, and save it. So it is > automaticly indexed. > > Is there a way to make it faster ? ( remove the auto optimize option > somewhere ?)You should disable Ferret indexing before you start creating your records, then create them, enable Ferret again and index them as a whole: Model.disable_ferret # create records here, collect ids in id_array Model.enable_ferret Model.bulk_index(id_array) bulk_index temporarily turns off auto_flush, and optimizes the index after finishing. I just committed these functions to trunk, so let us know how it works ;-) Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database
Good idea Once again, Thank you Jens -- Posted via http://www.ruby-forum.com/.
This is going really faster now ! As i am indexing thousands of rss feeds, it now takes one hour to retrieve and index it all ( several hours before ) -- Posted via http://www.ruby-forum.com/.
On Sun, Sep 02, 2007 at 02:10:32AM +0200, Hakita Hakita wrote:> This is going really faster now ! > As i am indexing thousands of rss feeds, it now takes one hour to > retrieve and index it all ( several hours before )glad to hear that :-) Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database