Xapian devel - Apr 2017 - Omega: Missing support for newer weighting schemes

> It may be worth splitting that part of the $set documentation out into its
> own section somehow, because it's getting a bit long -
Undoubtedly; $set command has the longest section on the documentation page :)
But it would be hard splitting that up because the documentation is organised
in a way that each command is really contained in its own specific section.
> and the details of which weighting schemes were available in which version
> isn't a key part of the $set command itself.
Do you suggest dropping that piece of information out? Since the reason behind
adding that is unclear to me so I'm not entirely sure if it would be a good
idea.

Thanks,
Vivek

On 8 Apr 2017, at 19:15, Vivek Pal <vivekpal.dtu at gmail.com> wrote:
>> and the details of which weighting schemes were available in which
version
>> isn't a key part of the $set command itself.
> 
> Do you suggest dropping that piece of information out? Since the reason
behind
> adding that is unclear to me so I'm not entirely sure if it would be a
good
> idea.
No, we shouldn't lose it completely. But I think that omegascript should
document the language and the functions, and the omega documentation should
cover things like exactly which weighting schemes are available. Right now,
omega doesn't have particularly coherent documentation, and I think finding
a better home for the weighting scheme details might help.

(Or it might not. But it's difficult to get into omega properly at the
moment.)

J

-- 
 James Aylett
 devfort.com — spacelog.org — tartarus.org/james/

On Sat, Apr 08, 2017 at 09:11:22PM +0100, James Aylett
wrote:
> On 8 Apr 2017, at 19:15, Vivek Pal <vivekpal.dtu at gmail.com> wrote:
> 
> >> and the details of which weighting schemes were available in which
version
> >> isn't a key part of the $set command itself.
> > 
> > Do you suggest dropping that piece of information out? Since the
reason behind
> > adding that is unclear to me so I'm not entirely sure if it would
be a good
> > idea.
> 
> No, we shouldn't lose it completely. But I think that omegascript
should
> document the language and the functions, and the omega documentation
> should cover things like exactly which weighting schemes are available.
> Right now, omega doesn't have particularly coherent documentation, and
I
> think finding a better home for the weighting scheme details might help.
We actually talked about adding a mechanism so Weight classes could
provide a scheme-specific parsing of parameters from a human-readable
text string.

Currently there's code in omega to handle this, and also in
examples/quest.cc and a third version in the evaluation module - if
we've implemented this 3 times ourselves in modules we've built on
the C++ API that seems a strong hint that this functionality might belong
in the API instead.

Each scheme already has a human-readable name, and Xapian::Registry
can map that to an "examplar" object of the right type, so we
could take a string like "bm25 1 0.8", see the first word is
"bm25"
and get a BM25Weight object, then call parse_params("1 0.8") on it to
create the correct Weight object (broadly similar to how unserialise()
is handled).

Then we can document the available schemes and the parameters they
take in one place and refer to that from omega, quest and the evaluation
module.

Cheers,
    Olly