Hello everyone, I have been working on adding support for BM25+ weighting function from the last couple of weeks. Initially, I considered modifying bm25weight.cc to add support for BM25+ function without disturbing functionalities of BM25. But that didn't work out very well. A day or two was spent trying to refactor and debug the same code. Later, I took another approach following the suggestions from James and implemented a new sub class (BM25PlusWeight) for BM25+ weighting function. All tests are passing. I've have pushed the changes to update the pull request: https://github.com/xapian/xapian/pull/104 Next, I'll be refactoring the code, improving test coverage in bm25plusweight.cc. and writing the documentation for BM25+. By the mid-end of the next week, I'll begin working on the implementation next weighting function as per project timeline i.e. PL2+ weighting function. Regards, Vivek -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20160611/e2c56921/attachment.html>
^I'm now looking to start writing documentation for BM25+ weighting function. Would it be a good idea to start already and alongside make changes to the PR as required after its review? Thanks, Vivek On Jun 11, 2016 3:07 AM, "Vivek Pal" <vivekpal.dtu at gmail.com> wrote:> Hello everyone, > > I have been working on adding support for BM25+ weighting function from > the last couple of weeks. Initially, I considered modifying bm25weight.cc > to add support for BM25+ function without disturbing functionalities of > BM25. But that didn't work out very well. A day or two was spent trying to > refactor and debug the same code. > > Later, I took another approach following the suggestions from James and > implemented a new sub class (BM25PlusWeight) for BM25+ weighting function. > All tests are passing. I've have pushed the changes to update the pull > request: https://github.com/xapian/xapian/pull/104 > > Next, I'll be refactoring the code, improving test coverage in > bm25plusweight.cc. and writing the documentation for BM25+. By the mid-end > of the next week, I'll begin working on the implementation next weighting > function as per project timeline i.e. PL2+ weighting function. > > Regards, > Vivek >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20160611/9742eab4/attachment.html>
On Sat, Jun 11, 2016 at 12:40:37PM +0530, Vivek Pal wrote:> I'm now looking to start writing documentation for BM25+ weighting > function. Would it be a good idea to start already and alongside make > changes to the PR as required after its review?I'm happy for you to add documentation changes on top of the PR if we haven't merged it by then. Before proceeding with another weighting scheme, I think it'd be a good idea to look at evaluating BM25+. Olly and I were talking about this yesterday, and it'd be helpful to know how it does against plain BM25. Guarav's previous evaluation work isn't up to date with master (in particular it references some work we never merged, since it didn't perform well enough to justify the complexity). However it's a good starting point, and it would be good to pull it into the main repository in any case. J -- James Aylett, occasional trouble-maker xapian.org