Weighting Schemes: Implementing Piv+ Normalization

Vivek Pal
Wed, 27 Jul 2016 18:08:58 -0000

Hi,

I have added support for Piv normalization in Tf-Idf weighting scheme as a
intermediate step to implementing the support for Piv+ normalization. All
tests pass.

But I'm running into some issues with Piv+ normalization. In the Piv+
formula  , there are two parameters (s and delta) that control the weight
assigned. I think the way I'm serialising and unserialising these
parameters has some issues which is causing a few errors,
http://pastebin.com/yCyuvF1X.  Any suggestion is appreciated!

Also, I was wondering if we want to have pivoted normalization weighting as
a separate weighting scheme in Xapian as it will make a way to add support
of different variations of pivoted normalization in the future?

Piv+ weighting formula:
https://trac.xapian.org/attachment/wiki/GSoC2016/Weighting/ProjectPlan/Piv%2B.png
)
Commit of Piv changes:
https://github.com/ivmarkp/xapian/commit/a02942fe3c3cf6cfbdd3c14685c2ff5f8a6b8b7c
Commit Piv+ changes:
https://github.com/ivmarkp/xapian/commit/e31a46e5a5ee1bdd3931de25e25722e46df056fe

Thanks,
Vivek
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.xapian.org/pipermail/xapian-devel/attachments/20160727/a2395be3/attachment.html>