Ferret talk - Jul 2007 - Strange search result with conditions in find_by_contents

Hi, guys:
Strange search result with conditions in find_by_contents!
first of all, i''ve installed the acts_as_ferret to my project vender
folder by ''ruby script/plugin install
svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret''

in my SearchController
def searchforum
if !params[:doSearch].nil?
if params[:searchTerms].nil? || params[:searchTerms] == ""
flash[:notice] = ''Please enter some words to search on.''
else
@conditions = " 1 = 1";
if !params[:dateRange].nil? && params[:dateRange] != ""
@conditions += " and creationDate >= " + params[:dateRange]
end
if !params[:forumID].nil? && params[:forumID] != ""
@conditions += " and forum_id = " + params[:forumID]
end

 # @total, @topics = Topic.full_text_search(params[:searchTerms], :page
=> (params[:page]||1))
@total, @topics = Topic.full_text_search(params[:searchTerms], {:page =>
(params[:page]||1)}, {:conditions => @conditions})
@pages = pages_for(@total)
end
end
end
in my model Topic:
def self.full_text_search(q, options = {}, find_options = {})
   return nil if q.nil? or q==""
   default_options = {:limit => 10, :page => 1}
   options = default_options.merge options

   # get the offset based on what page we''re on
   options[:offset] = options[:limit] * (options.delete(:page).to_i-1)

   # now do the query with our options
   results = Topic.find_by_contents(q, options, find_options)
   puts options
   puts find_options
   return [results.total_hits, results]
end
--------------------------------
i display 10 results in one page. In SearchController, if i run this
code:
@total, @topics = Topic.full_text_search(params[:searchTerms], :page =>
(params[:page]||1))
and input a search term, it will get 16 result messages. when i click to
the next page, it display the next 6 results, and the total_hits is
still 16.

but if i run this code in SearchController:
@total, @topics = Topic.full_text_search(params[:searchTerms], {:page =>
(params[:page]||1)}, {:conditions => @conditions})
and input the same search term, it will also get 16 result messages. but
when i click to the next page, it display the next 6 results, and the
total_hits changes to 6 messages, and only display one page.

speak in shortly: searching with conditions in find_by_contents, it got
16 total_hits, it display the first 10 result messages in first page,
when i click to the next page, it display the next 6 results, but the
total_hits changes to 6, and only got one page to display.

i don''t know why.

-- 
Posted via http://www.ruby-forum.com/.

and in the ApplicationController
def pages_for(size, options = {})
  default_options = {:per_page => 10}
  options = default_options.merge options
  pages = Paginator.new self, size, options[:per_page], 
(params[:page]||1)
  return pages
end

thanks in advance!

-- 
Posted via http://www.ruby-forum.com/.

Hi!
On Wed, Jul 18, 2007 at 05:41:46PM +0200, Chengcai He wrote:
[..]
> 
> speak in shortly: searching with conditions in find_by_contents, it got
> 16 total_hits, it display the first 10 result messages in first page,
> when i click to the next page, it display the next 6 results, but the
> total_hits changes to 6, and only got one page to display.
The reason for the odd behaviour you experience is as follows:
Aaf first queries ferret for your search results, it gets a
number of ids matching your query (and your paging). Now it uses this
set of ids to run a query against the DB, combining it with your own
conditions.

So in the end there is no simple way for aaf to determine the total
number of hits (without running the query more than once). 

Additionally it might happen that you get less results than expected,
since Ferret is delivering :limit hits, but these might well be cut down
to less than :limit by your active record conditions.


There are several ways to work around this:

- Get the total_hits separately by calling Model.find_by_contents
  without the paging options. Doesn''t solve the second problem I
  mentioned above.

- Use :limit and :offset as part of your find_options, so the paging
  takes place in the database. Might work okay, didn''t ever try it.

- Put the additional data in your index, too, and do not use 
  active_record conditions at all. Just append your additional query
  criteria to the query entered by the user.

- If you find a way how aaf could deliver consistent results in this
  scenario with any other method I can''t think of right now, submit a
  patch :-)


Btw, do never ever directly append request parameters to conditions 
like this:
> @conditions = " 1 = 1";
> if !params[:dateRange].nil? && params[:dateRange] != ""
> @conditions += " and creationDate >= " + params[:dateRange]
> end
> if !params[:forumID].nil? && params[:forumID] != ""
> @conditions += " and forum_id = " + params[:forumID]
> end
but use something like

conditions = [ ''creationDate >= ? AND forum_id = ?'', 
               params[:dateRange], forum_id  ]

to avoid exploitation of your app via sql injection. In newer Rails
there''s a Hash based notation for the conditions, too (that might be
easier to assemble in your case).

cheers,
Jens



-- 
Jens Kr?mer
http://www.jkraemer.net/ - Blog
http://www.omdb.org/     - The new free film database

one more question:
in the Model
acts_as_ferret({ :fields => {:username => {:store => :yes, :boost =>
30}, :subject => {:store => :yes, :boost => 20}, :body => {:store
=>
:yes, :boost => 10}}, :remote => true }, { :analyzer => 
Ferret::Analysis::RegExpAnalyzer.new(/./, false) })

def username
return self.user.login
end

search as:
@total, @topics = Topic.full_text_search(params[:searchTerms], {:page => 
(params[:page]||1)}, {:conditions => @conditions})

if i input a query term to search, i will got the result: @total = 14, 
@topics.length = 12
but when i input the user login to search the topics posted by the user, 
it got:
@total = 14, @topics.length = 0. It just display the paginate things, 
but none of the result topic displayed!

I''m totally lost!

-- 
Posted via http://www.ruby-forum.com/.

search terms   @total    @topics.length
????         12        12
??             14        12
hecc           16        0

--------------------------------------
''hecc'' is the user login name.

-- 
Posted via http://www.ruby-forum.com/.

On Thu, Jul 19, 2007 at 05:40:45AM +0200, Chengcai He
wrote:
> one more question:
> in the Model
> acts_as_ferret({ :fields => {:username => {:store => :yes, :boost
=>
> 30}, :subject => {:store => :yes, :boost => 20}, :body =>
{:store =>
> :yes, :boost => 10}}, :remote => true }, { :analyzer => 
> Ferret::Analysis::RegExpAnalyzer.new(/./, false) })
> 
> def username
> return self.user.login
> end
> 
> search as:
> @total, @topics = Topic.full_text_search(params[:searchTerms], {:page =>
> (params[:page]||1)}, {:conditions => @conditions})
> 
> if i input a query term to search, i will got the result: @total = 14, 
> @topics.length = 12
> but when i input the user login to search the topics posted by the user, 
> it got:
> @total = 14, @topics.length = 0. It just display the paginate things, 
> but none of the result topic displayed!
> 
> I''m totally lost!
Look at your log files, see what queries ferret runs against the index,
and see what sql statements are issued to your DB. I guess without the
conditions everything is ok?

cheers,
Jens

-- 
Jens Kr?mer
webit! Gesellschaft f?r neue Medien mbH
Schnorrstra?e 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
kraemer at webit.de | www.webit.de
 
Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa