similar to: xapian-letor: FeatureVector discussion

> > > > The approach I was thinking would look something like this: > > * instead of Features, which is really a namespace implemented as a > class, we separate out the calculation of the different features > into distinct subclasses of Feature, whose only job is to calculate > a single feature. Currently the FeatureManager calls these (via >

Hello everyone, I have completed introducing some code from v-hasu's branch into mine, mainly for Features, FeatureVector and FeatureManager classes. I have pushed the changes to https://github.com/ayshtmr/xapian/tree/letor-update. I am now proceeding to write unit tests for feature modules. There are a few things that I wanted to clarify: 1. I have introduced a lot of code in a single

Hello everyone, My name is Ayush Tomar. I'll be working on Learning to Rank stabilisation project over the summers. Here are a few things that I plan to do in coming few days: 1. Revise the timeline. There are some portions that I had kept for the first and second week of coding which have already been done (except writing tests). So, I'd like to adjust the timeline according to it. 2.

Hello James, I am working on breaking down Features into sub-classes. Should each of the features get their own sub-class, or should the grouping be done according to type? i.e. query-document pair dependent, query-dependent and document dependent sub-classes. Using this approach makes more sense if we plan to add support for user to include query-dependent and document-dependent features in

Hello, I wanted to decide the dataset that should be used for Letor stabilisation project. I think 2009 INEX Wikipedia Collection <http://www.mpi-inf.mpg.de/departments/databases-and-information-systems/software/inex/> should work fine. It's a collection of 2,666,190 XML articles, 115 topics <http://inex.mmci.uni-saarland.de/protected/adhoc/2009-topics.zip>, 50,275 qrel

Hi all, Thank you for giving me the opportunity to work with Xapian :) I am Jiarong Wei, a third year undergraduate student in Zhejiang University, China. In GSoC 2014, I will work on Letor module with Hanxiao Sun. Here are some questions I encountered these days, 1. In letor.cc, we have two parts of functions: the training part and the ranking part. I?ll use SVMRanker as an example. The

Hi, Wanted to know if other normalization techniques like normalization by standard-deviation have been tried to normalize the Feature-list in Letor. Regards, Ayush Pandey.

Hello, I'm Ayush from New Delhi, India. I am interested in Letor Stabilisation project for GSoC. I have a good background in machine learning. Sorry for getting in so late, university exams were holding me back. I'll try to cover as much as I can in the coming week. I am following the plan of attack suggested on the project page. Following are the things that I have completed: 1.

Hi Rishabh, I think its better not to expose RankiList to Letor.h and make it better user friendly. So my suggestion is to convert RankList to the following statement in this method. std::map<Xapian::docid, double> letor_score(const Xapian::MSet & mset); So just convert the RankList in std::map<Xapian::docid, double> format in the methods where you need to return. Parth. On

Hello everyone! I'm Ayush Tomar, junior undergrad in Computer Science from New Delhi, India. I love C++ coding and working on machine learning and information retrieval project. I was exploring the GSoC ideas for Xapian and the project on "Adding Weighting Schemes" looked really interesting to me. I wanted to work on text mining/IR this summer and this idea seems perfect! I have

Hi, I would like to know how to capture the console output from running an algorithm for further analysis. I can capture this using capture.output() but that yields a character vector. I would like to extract the actual numeric values. Here is an example of what I am trying to do. fr <- function(x) { ## Rosenbrock Banana function on.exit(print(f)) x1 <- x[1] x2 <- x[2]

Hi all, While building from git with xapian-letor not ignored in bootstrap, I am getting the following make error: In function `main': /home/ayush/Desktop/xapian/xapian-letor/bin/xapian-letor-update.cc:98: undefined reference to `Xapian::Internal::str(unsigned int)' /home/ayush/Desktop/xapian/xapian-letor/bin/xapian-letor-update.cc:99: undefined reference to

Hello Rishabh, As per our previous discussion, we will talk each week now about the project. Please update the timeline/todo/journal accordingly so that we can check the status on this saturday for this week. Please remember that this half of the programme is smaller and hence we would need to be very punctual about the goals. Regards, Parth. -------------- next part -------------- An HTML

Hello Rishabh, The attached diff file should help you to refactor the existing letor code and plug in your code easily. I have defined the header files with the necessary flow. I have also specified the ranker class, which should be implemented by a new LTR models, ListNet and ListMLE in your case. The evaluation file should be used for validating the performance of the algorithm in the training

I've gone through the link that you sent me and I currently understand how this helps and works to some extent, but I am not too sure of how I should start with converting the current interface to PIMPL design. I'm not used to this design pattern so its taking some time to sink in :) Say I start with the Clusterer class, I create a ClustererImpl class which is the internal class that

Thanks for your reply! For the third question: In https://inex.mmci.uni-saarland.de/data/documentcollection.jsp, I can find inex2010-article.qrels in 2010 assessment, but can?t find query files. Could you send me the link? I have registered on INEX website. And I also need to download ``INEX 2009 collection without annotation tags: (unofficial)`` on

Ah. If FIRE doesn't have something that can show this suitably, then > maybe Parth can advise on access to TREC, as I know he's used some of > them in the past. > ?I can say FIRE is also a reliable source but INEX/TREC are better. INEX can give you free access and TREC is not freely available. I had used INEX for xapian in the past and some details are here:

Hi Rishabh, I took some time out in the recent days to work on the refactoring. And now it looks like I have included all the files necessary for the complete code. I am sure if it will be quite easy for you now to start with the new code. I would like you to complete it by implementing (copy/paste/little modification) some of the methods. https://github.com/parthg/xapian To make it easier,

> > > > Actually, you're doing something slightly unusual there: making the > internal member public. Protected would be better, and private is I think > most usual; library clients aren't going to have access to the Internal > class declaration, so they can't call things on it. This means it's > actually difficult right now to subclass Feature. > > I

Hi Parth, I have a few questions- 1. Could you please provide me with the link for query-file, qrel-file for the dataset available at http://www.mpi-inf.mpg.de/departments/d5/software/inex/ . 2. I wanted to know how automated testing would be implemented. Will there be test cases like a test query must match this particular N results and this particular ranking. Or will it be in terms of