similar to: xapian-letor: FeatureVector discussion

Hi all, Thank you for giving me the opportunity to work with Xapian :) I am Jiarong Wei, a third year undergraduate student in Zhejiang University, China. In GSoC 2014, I will work on Letor module with Hanxiao Sun. Here are some questions I encountered these days, 1. In letor.cc, we have two parts of functions: the training part and the ranking part. I?ll use SVMRanker as an example. The

Hello James, Parth, Following our discussion on IRC and on code review, the way FeatureVector class works needs some discussion. Presently, the FeatureVector class is defined as follows, with a fixed number of feature count (19): class FeatureVector::Internal : public Xapian::Internal::intrusive_base{ friend class FeatureVector; double label; double score;

Hi Rishabh, I think its better not to expose RankiList to Letor.h and make it better user friendly. So my suggestion is to convert RankList to the following statement in this method. std::map<Xapian::docid, double> letor_score(const Xapian::MSet & mset); So just convert the RankList in std::map<Xapian::docid, double> format in the methods where you need to return. Parth. On

On Thu, Dec 04, 2014 at 08:28:32PM -0500, Manu Gupta wrote: > I was trying to build Han's code and was stuck here. It seems to me that > his code is trying to generate pdf for them. > > Is it possible to modify the make files to stop building the documentation. You can run the top-level configure with --disable-documentation. Cheers, Olly

Hello everyone, I have completed introducing some code from v-hasu's branch into mine, mainly for Features, FeatureVector and FeatureManager classes. I have pushed the changes to https://github.com/ayshtmr/xapian/tree/letor-update. I am now proceeding to write unit tests for feature modules. There are a few things that I wanted to clarify: 1. I have introduced a lot of code in a single

Hi~ Manu I was glad to see you were interested in Xapian, especially the learning to rank project. There are two branch in my repo. The one named gsoc2014 is followed the thought of Parth's work in which we not assign the test set for each ranking and in another one gsoc2014-evaluation we assign the training set and test set explicitly so that we can get the evaluation score of each ranking

As Olly has already pointed out the 2012 branch is not merged. I think there are some compilation errors in the branch. The code in branch is better refactored. The Ranker and FeatureManager classes are well defined and implemented. Parth. On Wed, Sep 25, 2013 at 9:02 AM, Olly Betts <olly at survex.com> wrote: > On Tue, Sep 24, 2013 at 08:34:10PM +0800, jiangwen jiang wrote: >

Hello everyone, My name is Ayush Tomar. I'll be working on Learning to Rank stabilisation project over the summers. Here are a few things that I plan to do in coming few days: 1. Revise the timeline. There are some portions that I had kept for the first and second week of coding which have already been done (except writing tests). So, I'd like to adjust the timeline according to it. 2.

Hello, I applied to letor stabilisation project for gsoc. I'd like to use coming weeks to improve the workability of xapian-letor. For that, I'm planning to refactor code in current master and begin writing some tests for it. Before adding tests, I think it would be better if xapian-letor could be made consistent with how xapian-core is written. For that, I'd first like to

> > > > Actually, you're doing something slightly unusual there: making the > internal member public. Protected would be better, and private is I think > most usual; library clients aren't going to have access to the Internal > class declaration, so they can't call things on it. This means it's > actually difficult right now to subclass Feature. > > I

Hi, I've read the code of letor module. And I have some questions about it: 1. In https://github.com/rishabhmehrotra/xapian/blob/master/xapian-letor/letor_internal.cc#L299, there is a write_to_file method, which save RankList into ?train.txt?. But the format for ?train.txt? is different from the one mentioned in http://trac.xapian.org/wiki/GSoC2011/LTR/Notes#QueryLevelNorm. And in

Thanks for your reply! For the third question: In https://inex.mmci.uni-saarland.de/data/documentcollection.jsp, I can find inex2010-article.qrels in 2010 assessment, but can?t find query files. Could you send me the link? I have registered on INEX website. And I also need to download ``INEX 2009 collection without annotation tags: (unofficial)`` on

Hi guys, I want to transfer the following code from R into .Call compatible form. How can i do that? Thanks!!! INT sim; for(i in 1:sim){ if(i>2) genemat <- genemat[,sample(1:ncol(genemat))] ranklist[,1] <- apply(genemat, 1, function(x){ (mean(x[cols]) - mean(x[-cols]))/sd(x)}) ranklist <- ranklist[order(ranklist[,1]),]

HI, Parth: Thanks for your reply. There's xapian-letor folder in current master trunk, is this code merged in 2011? the letor code of 2012 is not merged to the trunk? Regards 2013/9/23 Parth Gupta <pargup8 at gmail.com> > Dear Jiangwen Jiang, > > The current state of the project can be reached from the repo: > http://trac.xapian.org/wiki/GSoC2012/LTR > > I would

Hello all, Here goes one of my first functions. I want to make a nonparametric multiple sample comparison with unequal sample sizes (see Zar?s Biostatistical Analysis, 3rd. Ed., pg. 201 Example 10.11, pg. 288 Example 11.10). In the real world, I want to compare samples of fish length captured with different fishing gears. After using the Kruskal-Wallis test I want to check the differences

On Tuesday 08 of December 2015, Gerhard Wiesinger wrote: > On 07.12.2015 20:13, Timo Sirainen wrote: > > http://dovecot.org/releases/2.2/dovecot-2.2.20.tar.gz > > http://dovecot.org/releases/2.2/dovecot-2.2.20.tar.gz.sig > > > > This could be (one of) the last v2.2.x release. We're starting v2.3 > > development soon. > > Great! > > What's on

Hi, Before starting my proposal, I wanted to know what is the expected output of Letor module. Is it for transfer learning (i.e you learn from one dataset and leverage it to predict the rankings of other dataset) or is it for supervised learning? For instance - Xapian currently powers the Gmane search which is by default based on BM25 weighting scheme and now suppose we want to use LETOR to rank

Hello All, This time we are working on a weighting scheme "Learning to Rank" which involves machine learning and its a supervised ranking scheme unlike unsupervised schemes like BM25 under GSoC project. This mail intends to discuss the framework of the Learning to Rank in Xapian as a whole. I have thought of the following framework, pour in your insights or issues for the same. This is

Hi I am on Debian verson 7.7, GCC 4.7.2 and make 3.8.1. I cloned the code : git clone https://github.com/v-hasu/xapian git checkout gsoc2014-evaluation rm xapian-letor/nobootstrap ./bootstrap ./configure --no-documentation make CXXFLAGS='-w' and I get a different set of errors --- ranker.cc: In member function 'std::string Xapian::Ranker::get_cwd()': ranker.cc:80:36: error:

Hi, I have several questions regarding the letor module,I looked at the framework of learning to rank in xapian http://rishabhmehrotra.com/gsoc/17.png, I am a little confused. Why using deep learning to find unsupervised features in test data? Since in my understanding, learning to rank model usually learn features from the training data then apply the model to the test data? Why test set and