Xapian discuss - Jan 2008 - Document and folder suggestions

Hello,
I'm working on a document manager using Xapian. If you are familiar  
with, it will hopefully be something similar to Devonthink on Mac.
There are two crucial features, finding similar documents and  
suggesting an existing folder for a document. Xapian website claims to  
do that:
"Relevance feedback - given one or more documents, Xapian can suggest  
the most relevant index terms to expand a query, suggest related  
documents, categorise documents, etc."

Currently to find related documents I get a document, create a one  
item rset, get eset of max size 100 of it and search those terms to  
get a list of documents. Here is the code:

Xapian::RSet rSelection=Xapian::RSet();
rSelection.add_document(currentDocID);
Xapian::ESet eset = enquire.get_eset(100, rSelection);
Xapian::Query query(Xapian::Query::OP_OR, eset.begin(), eset.end());
enquire.set_query(query);
Xapian::MSet matches = enquire.get_mset(0, 10);

and here are my questions:
1) Is there a better way to get similar documents for a given document?
2) Is there way to suggest a folder for a given document to be  
classified in?

Thank you very much in advance.
--
Serkan Cabi
MIT Center for Theoretical Physics

On Sun, Jan 27, 2008 at 04:01:54PM -0500, Serkan Cabi
wrote:
> Currently to find related documents I get a document, create a one  
> item rset, get eset of max size 100 of it and search those terms to  
> get a list of documents. Here is the code:
I suspect 100 is too many.  Omega uses 40 for this (raised from 6 after
someone reported that gave better results), but it's certainly worth
experimenting.
> 1) Is there a better way to get similar documents for a given document?
You could take all the terms from the given document and combine them
with OP_ELITE_SET to select the best discriminators and run a query with
those.  I'm not sure which would give better results, but they're likely
to involve a similar amount of work.
> 2) Is there way to suggest a folder for a given document to be  
> classified in?
Assuming you add a boolean term for the folder to each document in the
database, run the given document as a query, and mark the top few
results as relevant, then expand selecting only the folder terms.

Cheers,
    Olly