similar to: Major Mistake in pL2 tests in the pull request

Hello guys. I just ran the updated tests for PL2 and they are not giving the mset order I expect.Now,the thing is, dfr's behavior is a bit hard to predict and so even if I expect a particular order ,it may give another order and still be correct.So,the only way to write correct tests for PL2 is to manually calculate the weight of the documents to decide the expected order.For that,I need to

Hello guys.I have added various tests to the PL2 pull request.They are working fine. Have also added PL2 to the registry and to the java and csharp makefiles.Please do let me know what you'll think.Other than the collection frequency problem discussed on IRC, it is ready.Am now beginning work on adding code and tests for DPH to the same branch. -Regards -Aarsh -------------- next part

Hello guys.I am working on implementing the PL2 weighting scheme of the DFR framework by Gianni Amati. It uses the Poisson approximation of the Binomial as the probabilistic model (P), the Laplace law of succession to calculate the after effect of sampling or the risk gain (L) and within document frequency normalization H2(2) (as proposed by Amati in his PHD thesis). The formula for w(t,d) in

Hi olly, I rewrote the test for PL2 after adding code to deal with negative weights. It passes on all backends other than inmemory . I see a different value of weight for inmemory backend. The code to calculate the lower bound is implemented in init().Please can I get some help with this ? -Regards -Aarsh -------------- next part -------------- An HTML attachment was scrubbed... URL:

Hi Parth, I implemented DFR algorithms in Xapian as a part of GSOC last year under the mentorship of Olly. This year, I want to work on analyzing and optimizing the performance of the DFR algorithms and comparing them with BM25.I also want to work on profiling the query expansion schemes and test the relevance(precision and recall) / speed(time taken) of the

Hi all, I have evaluated new weighting schemes along with their existing counterparts in Xapian to compare and see which one does better job. Also, I have put together all the results files for easy access here: https://github.com/ivmarkp/xapian-evaluation/tree/evaluation/run and a README for getting started with xapian-evaluation module. Hopefully, it might be of help to those who are new to

Hey guys,hi.:) I've finished implementing the PL2 scheme . The bounds I have implemented for it are as good as I could, given the nature of the scheme and my mathematical skills.However,tight bounds for other named DFR schemes will be easier to implement because their forumlas are quite simpler compared to PL2 . Will send in a pull request in a couple of days once I'm done with the tests

Hello guys, as discussed on IRC, I have written some code for better parsing of BM25 parameters in Omega. If no parameters are specified ,it defaults all of them. However, if there some are specified and some are not or if the invalid values are given for any of them,it throws an error. https://github.com/aarshkshah1992/xapian/commit/ac0a11f5d8ff975fad1e96e63764eab9b04dfcfb -Regards -Aarsh

Hello Guys. I have updated the code,tests,documentation,makefile entries and the registry entry of the* *TfIdf patch as per the feedback.Please do let me know if any additional changes are required before the patch can be merged, -Regards -Aarsh On Sun, Mar 3, 2013 at 2:50 PM, aarsh shah <aarshkshah1992 at gmail.com> wrote: > Hello guys.I have sent a pull request for the code and

Hello guys.I have sent a pull request for the code and tests of the Tf-Idf weighting scheme. Please do let me know if any changes are required.Meanwhile,Ill begin working on implementing normalizations which require additional statistics and on the DFR schemes. https://github.com/xapian/xapian/pull/6 On Tue, Feb 26, 2013 at 5:30 PM, <xapian-devel-request at lists.xapian.org>wrote: >

Hi everyone, I am thinking of working on the following ideas for my GSOC proposal based on my discussions with Olly and my own understanding. Rather than focusing on an entire perftest module, I have decided to focus on implementing performance tests for weighting schemes based on a wikipedia dump and in addition to that, build a framework to measure the

Hello guys.I just read up about tf-idf schemes and want to implement it in Xapian (with some frequently used normalizations) as it will also give me a good hang of implementing a weighting scheme before I start working on implementing DFR schemes. I read the following as references and I think Ive understood it well and can write the hack :- 1.)

Hello guys, you'll will be happy to know that the current codebase now includes the divergence from randomness weighting schemes which are known to outperform a lot of known weighting schemes such as BM25. Thanks to the amazing mentorship of Olly Betts and Dan Colish, our search results will now be better than before and Xapian will be more preferred in the research community than it was

Hello guys, you'll will be happy to know that the current codebase now includes the divergence from randomness weighting schemes which are known to outperform a lot of known weighting schemes such as BM25. Thanks to the amazing mentorship of Olly Betts and Dan Colish, our search results will now be better than before and Xapian will be more preferred in the research community than it was

> Each scheme already has a human-readable name, and Xapian::Registry > can map that to an "examplar" object of the right type, so we > could take a string like "bm25 1 0.8", see the first word is "bm25" > and get a BM25Weight object, then call parse_params("1 0.8") on it to > create the correct Weight object (broadly similar to how

Hey guys ,Hi :) I am working on a hack which will allow the user to specify a weighing scheme (along with the parameters , if he does not not want to use the default values) to build the Eset (rather than using the hard coded TradWeight scheme with default k=1 ) as Olly had suggested that we can probably get better terms (a more relevant Eset) for query expansion if we use say something

Hello guys.As discussed on IRC,I have sent a pull request for a test for testing TradWeight with an Rset. On Fri, Mar 1, 2013 at 5:30 PM, <xapian-devel-request at lists.xapian.org>wrote: > Send Xapian-devel mailing list submissions to > xapian-devel at lists.xapian.org > > To subscribe or unsubscribe via the World Wide Web, visit >

On Sun, Apr 09, 2017 at 11:34:07PM +0530, Vivek Pal wrote: > > Each scheme already has a human-readable name, and Xapian::Registry > > can map that to an "examplar" object of the right type, so we > > could take a string like "bm25 1 0.8", see the first word is "bm25" > > and get a BM25Weight object, then call parse_params("1 0.8") on

Hello guys :) I have sent a pull request for the Tf-Idf Weighting scheme incorporating as many normalizations as I could with the help of statistics currently available from Xapian::Weight . Please let me know what you'll think about it. I used the weighting scheme in a simple searcher and it did a fine job with it. I have no experience with writing tests for features like this.Please give me

Hello guys, as discussed on IRC, I've added an example of a weighting scheme that requires statistics to the howto manual of our Getting Started guide.Please do let me know if any modifications are required. The pull request is here : https://github.com/jaylett/xapian-docsprint/pull/3 -Regards -Aarsh. -------------- next part -------------- An HTML attachment was scrubbed... URL: