similar to: Letor re-factored code

Hi, I have several questions regarding the letor module,I looked at the framework of learning to rank in xapian http://rishabhmehrotra.com/gsoc/17.png, I am a little confused. Why using deep learning to find unsupervised features in test data? Since in my understanding, learning to rank model usually learn features from the training data then apply the model to the test data? Why test set and

Hello Rishabh, As per our previous discussion, we will talk each week now about the project. Please update the timeline/todo/journal accordingly so that we can check the status on this saturday for this week. Please remember that this half of the programme is smaller and hence we would need to be very punctual about the goals. Regards, Parth. -------------- next part -------------- An HTML

Hi Rishabh, I think its better not to expose RankiList to Letor.h and make it better user friendly. So my suggestion is to convert RankList to the following statement in this method. std::map<Xapian::docid, double> letor_score(const Xapian::MSet & mset); So just convert the RankList in std::map<Xapian::docid, double> format in the methods where you need to return. Parth. On

1.Could you explain why are these libraries included in all the xapian-letor headers? #include<xapian/intrusive_ptr.h> #include<xapian/types.h> #include<xapian/visibility.h> Or just provide me with the documentation of these header. I looked into these header files but couldn't anything substantial. 2.

Thanks for your reply! For the third question: In https://inex.mmci.uni-saarland.de/data/documentcollection.jsp, I can find inex2010-article.qrels in 2010 assessment, but can?t find query files. Could you send me the link? I have registered on INEX website. And I also need to download ``INEX 2009 collection without annotation tags: (unofficial)`` on

Hi Rishabh, I took some time out in the recent days to work on the refactoring. And now it looks like I have included all the files necessary for the complete code. I am sure if it will be quite easy for you now to start with the new code. I would like you to complete it by implementing (copy/paste/little modification) some of the methods. https://github.com/parthg/xapian To make it easier,

Hi, I've read the code of letor module. And I have some questions about it: 1. In https://github.com/rishabhmehrotra/xapian/blob/master/xapian-letor/letor_internal.cc#L299, there is a write_to_file method, which save RankList into ?train.txt?. But the format for ?train.txt? is different from the one mentioned in http://trac.xapian.org/wiki/GSoC2011/LTR/Notes#QueryLevelNorm. And in

Hello Everyone, I am looking forward to contribute to Xapian and also apply as a Google Summer of Code student. I would like to start by introducing myself. I am a final year M.Sc.(H) Chemistry and B.E. (H) Electronics and Instrumentation student at BITS - Pilani, Goa. I am interested in Machine Learning and presently pursuing my thesis on the same. I have been selected for Google Summer of Code

Hi everyone, My name is Shreedhar Pawar. I have already introduced myself on Xapian-discuss... I feel that the Xapian Search/Letor Algorithm, can speed up using Parallel computing. Techniques like Map-reduce, 'compact n split', radix sort, scan, parallel hashing n much more can be used to speed up the learning algorithms as well as the search... support vector machines in the Letor

Hello, I wanted to decide the dataset that should be used for Letor stabilisation project. I think 2009 INEX Wikipedia Collection <http://www.mpi-inf.mpg.de/departments/databases-and-information-systems/software/inex/> should work fine. It's a collection of 2,666,190 XML articles, 115 topics <http://inex.mmci.uni-saarland.de/protected/adhoc/2009-topics.zip>, 50,275 qrel

Hi all, Thank you for giving me the opportunity to work with Xapian :) I am Jiarong Wei, a third year undergraduate student in Zhejiang University, China. In GSoC 2014, I will work on Letor module with Hanxiao Sun. Here are some questions I encountered these days, 1. In letor.cc, we have two parts of functions: the training part and the ranking part. I?ll use SVMRanker as an example. The

> > > > The approach I was thinking would look something like this: > > * instead of Features, which is really a namespace implemented as a > class, we separate out the calculation of the different features > into distinct subclasses of Feature, whose only job is to calculate > a single feature. Currently the FeatureManager calls these (via >

Hi Parth, I have a few questions- 1. Could you please provide me with the link for query-file, qrel-file for the dataset available at http://www.mpi-inf.mpg.de/departments/d5/software/inex/ . 2. I wanted to know how automated testing would be implemented. Will there be test cases like a test query must match this particular N results and this particular ranking. Or will it be in terms of

Hello, I would like to work with Orange as part of GSoC 2012(and continue henceforth). Apologies for joining in a bit late- i was waiting to get a proper grasp of things before discussing it here. Currently I am a Masters students in Mathematics with my bachelors in Computer Science[integrated dual degree]. Over the last year and a half, I have worked on a few ML projects and have a couple of

Hi Rishabh, As per our last progress meeting, I am off for some days and as now its the time to mid-term evaluation, it would be better to generate a progress report. For that, you should first commit the code as it is and then write 2 to 3 page summary explaining the deliverables so far and then the future plan. Tomorrow is the last day, so better by tomorrow morning, send this across and by

Dear Sir, I am Pankaj Singhal from Jaipur, India. I am very much interested and strongly looking forward in getting involved in this project Learning-to-Rank. My previous experience in this field is good. Last semester I did a similar job of ranking the URLs of the given huge dataset based on their attribute values. The dataset consisted hundreds of thousands of URLs and each url

Hello, I applied to letor stabilisation project for gsoc. I'd like to use coming weeks to improve the workability of xapian-letor. For that, I'm planning to refactor code in current master and begin writing some tests for it. Before adding tests, I think it would be better if xapian-letor could be made consistent with how xapian-core is written. For that, I'd first like to

Hi, I am Jiarong Wei (irc: VcamX). I?m a 3rd year computer science student at Zhejiang University, China. I?m very willing to contribute to Xapian as part of GSoC 2014. Now I?m at Simon Fraser University, Canada, as an exchange student. I?ll go back to China on the end of April. I think it doesn?t matter I?ll change the time zone :) From the list of project?s ideas, Learning to Rank interests me

Hi Parth, I?ve implemented SVMRanker class and also sorted out most of current Letor APIs. Now I?m trying to use INEX dataset to verify my implement. But I stuck in the indexing part. You said in the documentation that we have to add prefix when indexing. Also I notice that you set some metadata in omindex.cc of your version. But the omindex.cc has changed since 2011. I think that?s why my result

Hi, Wanted to know if other normalization techniques like normalization by standard-deviation have been tried to normalize the Feature-list in Letor. Regards, Ayush Pandey.