Every query seems to be returning 100% relevant for every result, all the way down to the last one returned. That is if I search for something like "treatment OR centre", it returns 16758 responses, all with the same relevancy, being 100%. And get_weight returns 0 as well. There are only 581 documents containing "treatment AND centre", so surely some of those should have a higher relevancy than others? This is with Xapian 0.8.1, using either the PHP or the Perl bindings. It's for TheyWorkForYou.com, but my examples above are from a command line query tool I have for testing (the site itself always does AND). We noticed the problem because people found they weren't getting relevant search results. Everything appears to come vaguely in date order because of the order stuff is loaded into the database with, but only vaguely and it's quite confusing - you can certainly try that for yourself searching on the live site. I'm pretty sure I've broken something. Any suggestions how I can find out what? Thanks very much for any help. Francis -- Give your MP a job performance review --- http://www.theyworkforyou.com
On Tue, Aug 17, 2004 at 08:31:35AM +0100, Francis Irving wrote:> Every query seems to be returning 100% relevant for every result, all > the way down to the last one returned. > > That is if I search for something like "treatment OR centre", it > returns 16758 responses, all with the same relevancy, being 100%. And > get_weight returns 0 as well. > > There are only 581 documents containing "treatment AND centre", so > surely some of those should have a higher relevancy than others?Definitely. It sounds like the query is being run with boolean weights.> This is with Xapian 0.8.1, using either the PHP or the Perl bindings. > It's for TheyWorkForYou.com, but my examples above are from a command > line query tool I have for testing (the site itself always does AND). > > We noticed the problem because people found they weren't getting > relevant search results. Everything appears to come vaguely in date > order because of the order stuff is loaded into the database with, but > only vaguely and it's quite confusing - you can certainly try that for > yourself searching on the live site.With boolean weighting documents will be ordered with lower docids first.> I'm pretty sure I've broken something. Any suggestions how I can find > out what? Thanks very much for any help.Probably the first thing to try is "make check" on the xapian-core tarball. If that fails, it'll give us somewhere to look. If it doesn't, then the xapian library is probably OK - if get_weight() always returned zero, several tests would fail. Cheers, Olly
Hello all, I'm having this problem still, and unpredictably on different servers. It's very depressing, as it means the search on TheyWorkForYou.com is still broken. On Tue, Aug 17, 2004 at 08:31:35AM +0100, Francis Irving wrote:> Every query seems to be returning 100% relevant for every result, all > the way down to the last one returned. > > That is if I search for something like "treatment OR centre", it > returns 16758 responses, all with the same relevancy, being 100%. And > get_weight returns 0 as well.Olly suggested this a while ago: On Sun, Aug 22, 2004 at 12:56:19PM +0100, Olly Betts wrote:> > Any other suggestions for debugging this? > > If it's not the above, I'd suggest building a debug version of Xapian > (configure with --enable-debug). That will enable various consistency > checks which might flag something up.I've done that. How should I expect the consistency checks to occur? Any other suggestions for debugging this? Francis