Ferret talk - Dec 2006 - Sorting/Ordering Search Results

Hello All,

I am having an issue with AAF and sorting results of a search. Right
now, I have results being split onto pages of 10. The results are being
sorted alphabetically, but not across multiple pages - it''s just
sorting
the 10 it pulls down on each page. I noticed another post from April
regarding this same issue (http://www.ruby-forum.com/topic/62993#66934)
where the issue was claimed to be resolved - so I figure I''m doing
something wrong - but I just can''t figure it out. Any help anyone can
give would be appreciated. Thanks!


Member.rb (model)
def self.full_text_search(q, options = {}, find_options = {})
  return nil if q.nil? or q == ""

  default_options = {:limit => 10, :page => 1}
  options = default_options.merge options
  options[:offset] = options[:limit] * (options.delete(:page).to_i-1)

  results = Member.find_by_contents(q, options, find_options)
  return [results.total_hits, results]
end


(controller)
sortable_fields = []
sortable_fields <<
Ferret::Search::SortField.new(''company_name'')

@total, @members = Member.full_text_search("Restaurants", {:page =>
(params[:page]||1), :sort => sortable_fields}, {:include =>
[:categories, :discounts], :order => ["members.company_name ASC"]})

-- 
Posted via http://www.ruby-forum.com/.

OK. I''m trying to start this fresh, but I guess I was never sorting the
results via AAF to begin with. Does anyone know how to sort the ferret
results? Here are some things I have tried (console output)
>> members = Member.find_by_contents("bank", {:sort =>
"company_name", :limit => 50}, {})
... [members array displayed]...
>> members.each {|member| puts member.company_name}
SouthTrust Bank
Colonial Bank
SunTrust Bank
Citrus & Chemical Bank
Bank of America
Citizens Bank & Trust
Platinum Bank
CenterState Bank of Florida
Riverside National Bank
AmSouth Bank
Community National Bank
Washington Mutual Bank
Providence Bank
Olde Cypress Community Bank
>> sort_fields = []
=> []
>> sort_fields =
Ferret::Search::SortField.new(''company_name'')
=> company_name:<auto>
>> members = Member.find_by_contents("bank", {:sort =>
sort_fields, :limit => 50}, {})
>> members.each {|member| puts member.company_name}
SouthTrust Bank
Colonial Bank
SunTrust Bank
Citrus & Chemical Bank
Bank of America
Citizens Bank & Trust
Platinum Bank
CenterState Bank of Florida
Riverside National Bank
AmSouth Bank
Community National Bank
Washington Mutual Bank
Providence Bank
Olde Cypress Community Bank
>> members = Member.find_by_contents("bank", {:limit => 50},
{:order => "company_name"})
>> members.each {|member| puts member.company_name}
AmSouth Bank
Bank of America
CenterState Bank of Florida
Citizens Bank & Trust
Citrus & Chemical Bank
Colonial Bank
Community National Bank
Olde Cypress Community Bank
Platinum Bank
Providence Bank
Riverside National Bank
SouthTrust Bank
SunTrust Bank
Washington Mutual Bank

(Good, except for the fact that I only want 10 at a time)...
>> members = Member.find_by_contents("bank", {:limit => 10},
{:order => "company_name"})
>> members.each {|member| puts member.company_name}
AmSouth Bank
Bank of America
CenterState Bank of Florida
Citizens Bank & Trust
Citrus & Chemical Bank
Colonial Bank
Platinum Bank
Riverside National Bank
SouthTrust Bank
SunTrust Bank

(I have also tried the line below this with :sort => sort_fields AND
:sort => "company_name" with no luck!)
>> members = Member.find_by_contents("bank", {:limit => 10,
:offset => 10}, {:order => "company_name"})
>> members.each {|member| puts member.company_name}
Community National Bank
Olde Cypress Community Bank
Providence Bank
Washington Mutual Bank

Now I know thats the same problem I described above, but I will settle
for any help I can get !!! I''ve tried
Member.ferret_index.search("bank",
:sort => .... also tried Member.ferret_index.search_each("bank",
:sort
=> ... Is this even possible to do what I need?

I cant pass the :order => "company_name ASC" as a find_option
because by
the time it goes to AR to get the records, it has already returned the
IDs for the 10 records so I''m only ordering those 10 records and not
the
entire set of 14. PLEASE HELP!!! Thanks!

-- 
Posted via http://www.ruby-forum.com/.

Hey..

I''ve got the same problem, sorting isn''t working the way you
would
think.. I guess thats
a bug in ferret, nothing AAF can do about.. there''s a ticket and I
guess
David is aware
of that.. lets see when a new version of ferret will be released.

Ben

Thanks Ben. Wish I knew it was a bug, otherwise I wouldn''t have wasted 
everyone''s time reading over all that console output!

Benjamin Krause wrote:
> Hey..
> 
> I''ve got the same problem, sorting isn''t working the way
you would
> think.. I guess thats
> a bug in ferret, nothing AAF can do about.. there''s a ticket and I
guess
> David is aware
> of that.. lets see when a new version of ferret will be released.
> 
> Ben

-- 
Posted via http://www.ruby-forum.com/.

I''m also facing this problem here. It''s been driving me crazy
since
yesterday. I also hope it''s a bug and will be fixed soon.

Vin?cius

Benjamin Krause wrote:
> Hey..
> 
> I''ve got the same problem, sorting isn''t working the way
you would
> think.. I guess thats
> a bug in ferret, nothing AAF can do about.. there''s a ticket and I
guess
> David is aware
> of that.. lets see when a new version of ferret will be released.
> 
> Ben
-- 
Posted via http://www.ruby-forum.com/.

Another question regarding the same subject.
How do you sort by relevance? Or is that the default behavior?  Speaking 
of that, what is the default behavior if you don''t give any order 
preference?
Thanks,
Ray

-- 
Posted via http://www.ruby-forum.com/.

Raymond O''connor schrieb:
> Another question regarding the same subject.
> How do you sort by relevance? Or is that the default behavior?  Speaking 
> of that, what is the default behavior if you don''t give any order 
> preference?
>   
hey ..

as you write.. sorting by relevance is the default behaviour.. btw..
there are no bugs when
sorting the results on integer values, the problem seems to be purely
text-sorting related.

so, ordering by relevance is working ..

Ben

Hey...

just checked back with david about that sorting issue.. The problem  
on my index was, that you cannot sort by fields that you''ve indexed.  
you need to store the fields untokenized if you want to sort be them.  
maybe that''ll fix your problem as well?

Ben

sorting by a column, or by score:

     if (''date'' == sort)
       options.merge!({ :sort => Ferret::Search::SortField.new 
(''search_date'', :reverse => :true) })
     else
       options.merge!({ :sort => Ferret::Search::SortField::SCORE })
     end

hope this helps.

John

On Jan 17, 2007, at 3:42 PM, John Bachir wrote:
> sorting by a column, or by score:
>
>     if (''date'' == sort)
>       options.merge!({ :sort => Ferret::Search::SortField.new 
> (''search_date'', :reverse => :true) })
>     else
>       options.merge!({ :sort => Ferret::Search::SortField::SCORE })
>     end
>
> hope this helps.
>
> John

Also, if you want to sort by a column, it can''t be tokenized

  acts_as_ferret :fields => { ............
                              :search_date => {:term_vectors  
=> :no, :index => :untokenized }
...........
},

Benjamin Krause wrote:
> Hey...
> 
> just checked back with david about that sorting issue.. The problem
> on my index was, that you cannot sort by fields that you''ve
indexed.
> you need to store the fields untokenized if you want to sort be them.
> maybe that''ll fix your problem as well?
> 
> Ben
Yeah that did it! Thanks for all the help!

-- 
Posted via http://www.ruby-forum.com/.

Sean Osh wrote:
> Benjamin Krause wrote:
>> Hey...
>> 
>> just checked back with david about that sorting issue.. The problem
>> on my index was, that you cannot sort by fields that you''ve
indexed.
>> you need to store the fields untokenized if you want to sort be them.
>> maybe that''ll fix your problem as well?
>> 
>> Ben
> 
> Yeah that did it! Thanks for all the help!
Hi,
I have a field named ''title''.
If I tokenize this field then I''m able to search it by keywords but not
able to sort it.   However if I untokenize so that sorting works, then 
the search fails.

Its a dilemma but I''m sure there must be some solution to this problem,
right?

It can''t be that for any particular field I have to choose between 
whether I want to search it or sort by it.

Thanks.
Jen

-- 
Posted via http://www.ruby-forum.com/.

On Feb 9, 2007, at 6:28 PM, jen wrote:
> I have a field named ''title''.
> If I tokenize this field then I''m able to search it by keywords
but
> not
> able to sort it.   However if I untokenize so that sorting works, then
> the search fails.
>
> Its a dilemma but I''m sure there must be some solution to this  
> problem,
> right?
>
> It can''t be that for any particular field I have to choose between
> whether I want to search it or sort by it.
What about

FieldInfo.new(:title, :index => :yes, :store => yes)

This should store the field in it''s original format while indexing it  
tokenized.

If this doesn''t work, I''d consider it a bug since the
documentation
suggests that :store and :index are independent options. At least, it  
doesn''t state otherwise. Setting a certain value for :store should  
not conflict or interfere with any value for :index.

If this is indeed not working as expected, you could use two fields,  
one which you store untokenized (for sorting) and one which you don''t  
store but index tokenized. That''d wouldn''t be an elegant
solution but
a feasible one.

Cheers,
Andreas

On Fri, Feb 09, 2007 at 06:57:56PM +0100, Andreas Korth
wrote:
> 
> On Feb 9, 2007, at 6:28 PM, jen wrote:
> 
> > I have a field named ''title''.
> > If I tokenize this field then I''m able to search it by
keywords but
> > not
> > able to sort it.   However if I untokenize so that sorting works, then
> > the search fails.
> >
> > Its a dilemma but I''m sure there must be some solution to
this
> > problem,
> > right?
> >
> > It can''t be that for any particular field I have to choose
between
> > whether I want to search it or sort by it.
> 
> What about
> 
> FieldInfo.new(:title, :index => :yes, :store => yes)
> 
> This should store the field in it''s original format while indexing
it
> tokenized.
> 
> If this doesn''t work, I''d consider it a bug since the
documentation
> suggests that :store and :index are independent options. At least, it  
> doesn''t state otherwise. Setting a certain value for :store should
> not conflict or interfere with any value for :index.
:store does not interfere with the value given for :index, but as I
understand the docs, storing a field''s contents doesn''t help
with
sorting either. From the docs only the :index option influences the
ability to sort.

Imho storing a field''s content is completely independent from
(un)tokenized indexing, and the sorting is done on the indexed values,
not on the stored ones.
> If this is indeed not working as expected, you could use two fields,  
> one which you store untokenized (for sorting) and one which you
don''t
> store but index tokenized. That''d wouldn''t be an elegant
solution but
> a feasible one.
That''s the common solution. There''s no need to use :store
=> :yes for
any of these fields, unless you have any use for the stored original
field content.

so you''d have:
  # the field for search
  FieldInfo.new(:title, :index => :yes, :store => whatever you want)
  # the field for sorting, leaving out any info not needed for sorting
  FieldInfo.new(:sortable_title, :index => :untokenized, :store => :no,
:term_vector => :no)

Jens

-- 
webit! Gesellschaft f?r neue Medien mbH          www.webit.de
Dipl.-Wirtschaftsingenieur Jens Kr?mer       kraemer at webit.de
Schnorrstra?e 76                         Tel +49 351 46766  0
D-01069 Dresden                          Fax +49 351 46766 66