search for: doc_freq

I need to get a set of terms being indexed using Ferret. I used IndexReader.terms and it returns a list of TermEnum nicely. The only problem is that my analyzer includes a stemming filter. So now, the terms I''m getting back are all stemmed. Is there anyway to get the original unstemmed terms back from the index somehow? Thanks. -- Posted via http://www.ruby-forum.com/.

[Replying to xapian-devel, as I think a wider audience would be useful] On Mon, Oct 21, 2013 at 11:24:51PM +0800, jiangwen jiang wrote: > yes, it's less efficient. Lucene database has multiple segments, each > segment can treat as a independent database. The same term may exists in >= > 1 segments. Sorry for taking a while to respond - I've been both busy and mulling this

....9.0/lib/ferret/index/term_infos_io.rb:263:in `get_index_offset'' from c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/term_infos_io.rb:162:in `get_term_info'' from c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/index/segment_reader.rb:176:in `doc_freq'' from c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/search/index_searcher.rb:47:in `doc_freq'' from c:/ruby/lib/ruby/gems/1.8/gems/ferret-0.9.0/lib/ferret/search/term_query.rb:13:in `initialize'' from c:/ruby/lib/ruby/gems/1.8/gems/ferret-...

...change the rounding of my score? I dunno. Basque chicken scored 1.0 3291.866 = product of: 6583.732 = sum of: 674.6201 = weight(ingredients_without_brackets:chicken in 12670), product of: 0.2805449 = query_weight(ingredients_without_brackets:chicken), product of: 3.13109 = idf(doc_freq=2145) 0.08959977 = query_norm 2404.677 = field_weight(ingredients_without_brackets:chicken in 12670), product of: 1.0 = tf(term_freq(ingredients_without_brackets:chicken)=1) 3.13109 = idf(doc_freq=2145) 768.0 = field_norm(field=ingredients_without_brackets, doc...

...from /usr/lib/ruby/site_ruby/1.8/ferret/index/ term_infos_io.rb:285:in `scan_for_term_info'' from /usr/lib/ruby/site_ruby/1.8/ferret/index/ term_infos_io.rb:163:in `get_term_info'' from /usr/lib/ruby/site_ruby/1.8/ferret/index/ segment_reader.rb:176:in `doc_freq'' from /usr/lib/ruby/site_ruby/1.8/ferret/search/ index_searcher.rb:47:in `doc_freq'' from /usr/lib/ruby/site_ruby/1.8/ferret/search/term_query.rb: 13:in `initialize'' from /usr/lib/ruby/site_ruby/1.8/ferret/search/term_query.rb: 99:in `new'...

...he following explanation (via code a added to acts_as_ferret for debugging): QUERY: id:building name:building EXPLANATION of building: 8.438619e-42 = product of: 1.687724e-41 = weight(name:building in 3), product of: 0.6125279 = query_weight(name:building), product of: 2.386294 = idf(doc_freq=1) 0.2566858 = query_norm 2.755373e-41 = field_weight(name:building in 3), product of: 1.0 = tf(term_freq(name:building)=1) 2.386294 = idf(doc_freq=1) 1.15467e-41 = field_norm(field=name, doc=3) 0.5 = coord(1/2) Note the tiny score of field_norm which is throwing the...

...ield_value} fvh = field_index.highlight(query, 0, options.merge({ :field => :keywords }) ) return fvh rescue Exception => e puts "** IndexSearch.highlight(''#{query}''): #{e.message}" # << e.backtrace.join("\n") return field_value end def doc_freq(field, term) self.searcher.doc_freq(field, term) end ############################# # Self Util methods def reader self.searcher.reader end def size if reader total_size = reader.num_docs else total_size = 0 self.sub_searchers.each do |sub_searcher| total_size += sub_se...

Hello, I need to be able to count the occurences of certain terms in the reults. Currently my setup is Ferret 0.10.1 aaf bleeding edge. results = VoObject.find_by_contents(query,:offset=>page, :limit=> 20,:sort => sort_fields) I use results.total_hits for pagination. This all works really nicely. However i need to be able to know how many occurences of certain predefined terms occur

I''ve been seeing some core dumps coming from ferret_server: acts_as_ferret/lib/more_like_this.rb:170: [BUG] Segmentation fault ruby 1.8.6 (2007-03-13) [i386-freebsd6] I''m running the latest build of ferret (0.11.4-rc5). Line 170 in more_like_this.rb is: freq = reader.doc_freq(field_name, word) which is calling into the ferret C code (if I''m reading this correctly). Is there anything I can do to get you more information, or help track down this problem? Thanks. -- Peter Jones pmade inc. - http://pmade.com

Hi all, Having problems trying to get Ferret to read an index generated by Lucene. Am I right in thinking Ferret should be able to read a Lucene generated index no problem? Using the code snippets detailed in http://www.ruby-forum.com/topic/64099#new Any advice gratefully received. Many Thanks, Steven -- Posted via http://www.ruby-forum.com/.

Is there a fast way to determine how many documents a term occurs in, besides iterating through every document with TermDocEnum? -- Best regards, Stian Gryt?yr