similar to: Ferret-talk Digest, Vol 25, Issue 3

> From: Jens Kraemer <jk at jkraemer.net> > Subject: Re: [Ferret-talk] Performance before and after optimization > On Sat, Nov 03, 2007 at 08:49:17PM +0800, Alex Neth wrote: > [..] >> 2) Can I keep a second index so that it doesn''t get locked during >> optimization and then switch to the optimized index? Perhaps the >> index >> is not really

I was under the impression that Model.rebuild_index will automatically call optimize in the end. But that doesn''t seem to be the case. After running Model.rebuild_index how do I optimize the index. I am using aaf most stable and also ferret most stable. def rebuild_index(*models) models << aaf_configuration[:class_name] unless

hi, i''m indexing a really large db table (~4.2 million rows). i''ve noticed that after ~2M records, index performance decreases by almost an order of magnitude. full dataset graph here: http://i122.photobucket.com/albums/o244/spokeo/indexer-data.jpg here''s a couple best-fit lines that represent the data points: 0-2M : y = 78.65655x + 144237.5 2.5M+ : y = 10.79832x +

I have a column of words, for example "DOG" "DOOG" "GOD" "GOOD" "DOOR" ... and I am interested in creating a matrix that contains the string edit distances between each pair of words. I am this close -> ' ' <- to writing the algorithm myself (which will allow for different variations on the string edit rules, indels,

I''m running: ferret (0.10.9) ruby 1.8.5 (2006-08-25) [i386-mswin32] on Windows XP(SP2) When I create an index as follows: field_infos = FieldInfos.new(:store => :yes, :term_vector => :no, :index => :yes) field_infos.add_field(:id, :index => :untokenized) field_infos.add_field(:subject) field_infos.add_field(:author) field_infos.add_field(:tags, :store => :no) index =

Hi guys! Been reading this list for a while. I have two repeatable Ferret crash bugs, both seg faults. 1. The first bug appears to seg fault Ferret when you use quotes in a search argument (eg ''file_name:"file name"'') 2. The second bug appears to seg fault Ferret when you attempt to index text with very long tokens (above 256 chars). It may have something to do with

Hey Dave, I just contributed $100 to the ferret donation box. My project is earning no money yet (but hopefully will), for now I hope this helps you out and covers me for asking stupid questions ;). To get a distance sorted output, I am passing an array of the id field from a ferret search through to mysql in a custom select statement. SELECT ... id IN (#{ids.join(",")}) This has

The following script creates a search index and then searches it. I get no results? Where am I going wrong? Thanks. -----------BEGIN SCRIPT---------------- require ''rubygems'' require ''ferret'' include Ferret path = ''/tmp/myindex'' field_infos = Ferret::Index::FieldInfos.new() field_infos.add_field(:name, :store => :yes, :index => :yes)

Hi, Using the 0.10.4 gem under ruby 1.8.5 (2006-08-25) [i686-linux], I get different results with a TermQuery and a search string. Namely, using a search string seems to always work whereas using a TermQuery often doesn''t return any entries. For example: > x=@i[450][:message_id] => "9e7db9110509070759732b21c4 at mail.gmail.com" >

I''ve been messing around with Ferret (no punn intended). After spending some time testing it out (indexing to file), I decided to index about 10% of the data I want to eventually index. It took several hours to complete the index on my local machine, but it was created without any problems and after optimising it the searches returned results at the sort of speed I was expecting. I

Thanks for the response Jens. Indeed I am sorting by something other than relevancy, so that would explain it. Optimized, it''s extremely fast and handles a good load, but new records kill it until I optimize. I haven''t tried :merge_factor as I wasn''t aware of it. I''m not sure it will help given the above. Regarding the re-index locking code,

Hi, I''m adding some news articles to a keyed Ferret 0.10.14 index and encountering quite serious instability when concurrently reading and writing to the index, even though with just 1 writer and 1 reader process. If I recreate the index without a key, concurrent reading and writing seem to work fine (and indexing is about 10 times quicker :) I''m testing by running my indexing

The included test script turned up the following anomolies (run against Ferret 0.10.3, but had same problems with 0.10.2): 1. When the content word is not in the index the inclusion of a wildcard file term causes search_each to throw a segmentation fault. $ ./test.rb zzz file:*.txt query: +content:zzz +file:*.txt ./test.rb:28: [BUG] Segmentation fault ruby 1.8.4 (2005-12-24)

Hi, Just downloaded the new ferret 0.11. I''m on OSX btw. I get this error everytime I run my unit tests: Loaded suite ferret_updater_unit_test Started E/usr/local/lib/ruby/1.8/erb.rb:504: [BUG] Segmentation fault ruby 1.8.4 (2005-12-24) [i686-darwin8.7.1] Abort trap When I revert back to 10.14 I dont get this error. When I comment out the line: Ferret::Index::Index.new({:path =>

This program reliably crashes for me (usually a segfault): require ''rubygems'' require ''ferret'' reader=Ferret::Index::IndexReader.new ARGV fields=reader.field_infos.fields reader.max_doc.times{|n| fields.each{|field| reader.term_vector(n,field) } unless reader.deleted?(n) print "."; STDOUT.flush } As you can see, it just goes through

Hi, I''m new to using ferret (and fairly new to ruby/rails) and I''m having a problem I can''t fathom. Sorry for the long post ... I have a test which passes require ''rubygems'' require ''ferret'' include Ferret require ''test/unit'' class CompanyTest < Test::Unit::TestCase def test_index puts ''running

Hi, I''m new to using ferret (and fairly new to ruby/rails) and I''m having a problem I can''t fathom. Sorry for the long post ... I have a test which passes require ''rubygems'' require ''ferret'' include Ferret require ''test/unit'' class CompanyTest < Test::Unit::TestCase def test_index puts ''running

Hi all, first off, it''s 2AM and I''m not thinking properly, so please forgive me if this one''s easy, but I just need to get this going. First problem, using 0.9.6 on all of our development machines, works great, then we move it to a server running x86_64 linux and it segfaults as soon as it tries to create an Index. I''ve tried rebuilding with different

Hi .. I tried to exclude certain objects from my search, by adding appropriate term queries .. i = Ferret::Index::Index.new i.field_infos.add_field(:type, :index => :untokenized, :term_vector => :no) i << {:type => "Movie", :name => "Indiana" } i << {:type => "Movie", :name => "Forrest" } i << {:type =>

Hi there, Is highlighting for range queries supposed to work ? It doesn''t work here. here is an non-working example: (highlighting works when q="test:2007*") require ''ferret'' include Ferret index = Index::Index.new() #index.field_infos.add_field(:test, :store => :yes, :index => :untokenized) i=1 for a in [ "20070505", "20071230",