similar to: GSoC 2015 - Weighting Schemes

Hi all, I was trying to generate lcov test coverage reports for xapian-core but got an out of memory error: $ lcov --capture --directory . --output-file xapian-core.info Capturing coverage data from . Found gcov version: 4.7.3 Scanning . for .gcda files ... Found 270 data files in . Processing bin/xapian-progsrv.gcda Out of memory! These are the steps I followed in xapian-core directory

Hello everyone, My name is Ayush Tomar. I'll be working on Learning to Rank stabilisation project over the summers. Here are a few things that I plan to do in coming few days: 1. Revise the timeline. There are some portions that I had kept for the first and second week of coding which have already been done (except writing tests). So, I'd like to adjust the timeline according to it. 2.

Hello, I'm Ayush from New Delhi, India. I am interested in Letor Stabilisation project for GSoC. I have a good background in machine learning. Sorry for getting in so late, university exams were holding me back. I'll try to cover as much as I can in the coming week. I am following the plan of attack suggested on the project page. Following are the things that I have completed: 1.

Hello James, Parth, Following our discussion on IRC and on code review, the way FeatureVector class works needs some discussion. Presently, the FeatureVector class is defined as follows, with a fixed number of feature count (19): class FeatureVector::Internal : public Xapian::Internal::intrusive_base{ friend class FeatureVector; double label; double score;

Ah. If FIRE doesn't have something that can show this suitably, then > maybe Parth can advise on access to TREC, as I know he's used some of > them in the past. > ?I can say FIRE is also a reliable source but INEX/TREC are better. INEX can give you free access and TREC is not freely available. I had used INEX for xapian in the past and some details are here:

Hello James, I am working on breaking down Features into sub-classes. Should each of the features get their own sub-class, or should the grouping be done according to type? i.e. query-document pair dependent, query-dependent and document dependent sub-classes. Using this approach makes more sense if we plan to add support for user to include query-dependent and document-dependent features in

> Hi, Vivek — there isn't any particular reason that I'm aware of. It's > probably worth pointing (in the omegascript documentation) to the part of > the getting started guide which talks about the different weighting schemes If there isn't any reason then I'd like to send in a patch adding support for those weighting schemes in weight.cc and I agree omegascript

Hi all, While building from git with xapian-letor not ignored in bootstrap, I am getting the following make error: In function `main': /home/ayush/Desktop/xapian/xapian-letor/bin/xapian-letor-update.cc:98: undefined reference to `Xapian::Internal::str(unsigned int)' /home/ayush/Desktop/xapian/xapian-letor/bin/xapian-letor-update.cc:99: undefined reference to

On Sun, Apr 09, 2017 at 11:34:07PM +0530, Vivek Pal wrote: > > Each scheme already has a human-readable name, and Xapian::Registry > > can map that to an "examplar" object of the right type, so we > > could take a string like "bm25 1 0.8", see the first word is "bm25" > > and get a BM25Weight object, then call parse_params("1 0.8") on

Hi, In my explorations of Omega codebase, I have found that Omega is currently missing support for newer weighting schemes added in 1.4.1 (BM25+, PL2+, Dir+). I'd submit a PR addressing that but as I think I might be missing something so just wanted to check if there's a particular reason for that? P.S. Finally back after a long week. Been eagerly waiting for a weekend since the

Hello everyone, I have completed introducing some code from v-hasu's branch into mine, mainly for Features, FeatureVector and FeatureManager classes. I have pushed the changes to https://github.com/ayshtmr/xapian/tree/letor-update. I am now proceeding to write unit tests for feature modules. There are a few things that I wanted to clarify: 1. I have introduced a lot of code in a single

Hi all, I have evaluated new weighting schemes along with their existing counterparts in Xapian to compare and see which one does better job. Also, I have put together all the results files for easy access here: https://github.com/ivmarkp/xapian-evaluation/tree/evaluation/run and a README for getting started with xapian-evaluation module. Hopefully, it might be of help to those who are new to

> > > > Actually, you're doing something slightly unusual there: making the > internal member public. Protected would be better, and private is I think > most usual; library clients aren't going to have access to the Internal > class declaration, so they can't call things on it. This means it's > actually difficult right now to subclass Feature. > > I

Hello, I wanted to decide the dataset that should be used for Letor stabilisation project. I think 2009 INEX Wikipedia Collection <http://www.mpi-inf.mpg.de/departments/databases-and-information-systems/software/inex/> should work fine. It's a collection of 2,666,190 XML articles, 115 topics <http://inex.mmci.uni-saarland.de/protected/adhoc/2009-topics.zip>, 50,275 qrel

Hi James, > We probably don't want them committed in git where they're evaluation > runs (because we can recreate them); a gist might be more appropriate. Sorry, I have moved results files over to gist for each individual weighting scheme. Link: https://gist.github.com/ivmarkp/secret > I can't tell, but are some of those files from FIRE? If so, they > shouldn't be

Hi, I have submitted a draft proposal on the GSoC website but I also wanted to share my draft proposal through Dropbox to get your feedback quickly through it. Please review it and let me know your feedbacks as soon as possible. I haven't written much about automated testing as I haven't written any test yet ( I will learn writing automated tests before April end ). Link to my

Hi all, I am trying to call the R script/ R Program via my Shell Script. I need to pass a file to the R script as an argument. When I am trying to do that I am getting error and only first line of the file is being read. Can you tell me how should I do that so that it reads the entire file in a R object so that later I am able to process the matrix operation on that file. It is a tab delimited

> > > I'd prefer to avoid adding things to the public API that don't get > used by end users. However because LTR is outside the Xapian build > tree, we can't easily give it privileged access to Xapian internals. > Sorry for a delayed response. The way I was thinking of performing reranking with updated weights was to add a class MSetRanker (basically containing a

Hi, I'm Ayush an undergraduate Computer Science student from Thapar university, India. I was fiddling with xapian since the morning and trying to understand the code and internals of Xapian. I tried implementing the bite sized project idea posted here: https://trac.xapian.org/wiki/ProjectIdeas#AddnewOmegaScriptcommandtodoasubstringsearch but could not understand what needs to be returned when

I've gone through the link that you sent me and I currently understand how this helps and works to some extent, but I am not too sure of how I should start with converting the current interface to PIMPL design. I'm not used to this design pattern so its taking some time to sink in :) Say I start with the Clusterer class, I create a ClustererImpl class which is the internal class that