Xapian discuss - Jun 2006 - Effects of passing a RSet to Enquire::get_mset()

Hi All,

 From my brief testing it appears that passing an RSet filled with  
documents to Enquire::get_mset() does not make much of a difference  
to the returned results.

I have a search returning about 1000 results, when I add an RSet  
containing about 75 documents that appeared towards the end of the   
result list (about position 600 on average with a relevancy of  >  
80), there was no change.  Is this to be expected? The understanding  
I got from the documentation is that by adding documents to the RSet  
their terms would be considered more relevant, and query results  
would change.

Did I misunderstand the purpose of the relevancy set with regard to  
query evaluation?  Or may there be a bug lurking somewhere or does  
the passed relevancy set need to be bigger to have an effect?

Cheers,

Rusty
--
Rusty Conover
InfoGears Inc.
Web: http://www.infogears.com

On Mon, Jun 05, 2006 at 11:37:14PM -0600, Rusty Conover
wrote:
> From my brief testing it appears that passing an RSet filled with  
> documents to Enquire::get_mset() does not make much of a difference  
> to the returned results.
This is often the case.  The only effect that passing an RSet has in
get_mset() is to adjust the importance of the terms in the query.  If you
have a large query (eg, more than 10 terms), this can have a big effect on
the ranking on the documents.  However, if you have a small query, the
effect is very limited: in fact, if you have only 1 term in the query, the
weight of the term will be adjusted, but since all documents must already
match that term, the ranking will not change at all.

If you want to have a more pronounced effect, you need to expand the query,
and re-run an expanded one.  In other words, use get_eset() to get a set of
terms to add to the query (based on your supplied RSet), and then run
get_mset() using your initial query combined (with an OR operator) with the
additional terms.  You should still pass your RSet to the call to
get_mset() with the expanded query; it will have more effect in this
situation.

-- 
Richard

On Mon, Jun 05, 2006 at 11:37:14PM -0600, Rusty Conover
wrote:
> The understanding I got from the documentation is that by adding
> documents to the RSet their terms would be considered more relevant,
> and query results would change.
Richard has explained this very well, but can you remember where in
the documentation the misleading description is?  We ought to improve
it.

Cheers,
    Olly

On Jun 6, 2006, at 10:47 AM, Olly Betts wrote:
> On Mon, Jun 05, 2006 at 11:37:14PM -0600, Rusty Conover wrote:
>> The understanding I got from the documentation is that by adding
>> documents to the RSet their terms would be considered more relevant,
>> and query results would change.
>
> Richard has explained this very well, but can you remember where in
> the documentation the misleading description is?  We ought to improve
> it.
Mostly I got the understanding from http://www.xapian.org/docs/ 
intro_ir.html.

It was probably the section "The progress of a query" that piqued the
idea that the R set could be useful for refinement.

Rusty
--
Rusty Conover
InfoGears Inc.
Web: http://www.infogears.com