search for: qrel

Thanks for your reply! For the third question: In https://inex.mmci.uni-saarland.de/data/documentcollection.jsp, I can find inex2010-article.qrels in 2010 assessment, but can?t find query files. Could you send me the link? I have registered on INEX website. And I also need to download ``INEX 2009 collection without annotation tags: (unofficial)`` on http://www.mpi-inf.mpg.de/departments/d5/software/inex/, right? Thank you! Jiarong Wei On...

...- -nts. > and you'll need a way to use letor from omega, or you'll have trained a > model for no good reason :) Sorry, I may have misunderstood you here but why would we need a way to use letor from omega? For training Letor module, wouldn't we just need two files i.e. Query and Qrel as mentioned in the xapian-letor docs? Letor API can then generate the final training file using those two files. And to mine the relevance judgements for Qrel file from logs, we'll need to train one of the click models such as DBM etc.. Is there a better way to mine the relevance judgements...

...p://trac.xapian.org/wiki/GSoC2011/LTR/Notes#QueryLevelNorm, it mentioned "the first column is the relevance judgement?. I think the value of the relevance judgement is just 0 or 1. But the code saves it as a ?double?. Is it just for convenience? Or I misunderstand the whole thing? 3. I?ve got qrels file of INEX 2010, but I can find query file. How can I get it? I can?t find it on INEX website. Thank you! Jiarong Wei -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20140308/4d845805/attachment-000...

Hi Parth, I implemented DFR algorithms in Xapian as a part of GSOC last year under the mentorship of Olly. This year, I want to work on analyzing and optimizing the performance of the DFR algorithms and comparing them with BM25.I also want to work on profiling the query expansion schemes and test the relevance(precision and recall) / speed(time taken) of the

...ject. I think 2009 INEX Wikipedia Collection <http://www.mpi-inf.mpg.de/departments/databases-and-information-systems/software/inex/> should work fine. It's a collection of 2,666,190 XML articles, 115 topics <http://inex.mmci.uni-saarland.de/protected/adhoc/2009-topics.zip>, 50,275 qrel <http://inex.mmci.uni-saarland.de/protected/adhoc/2009-inex_eval.zip> labels and has an uncompressed size of 50.75 gb (5.52 GB compressed). Another similar alternative is 2013 INEX Wikipedia LOD Collection <http://inex-lod.mpi-inf.mpg.de/2013/>. It's a collection of 12,216,109 XML...

...rence pair models which I mentioned earlier. We have some options here as descibed in book "Click Models for Web Search" such as DBN, DCN, CCN etc. which will be trained on a relevance dataset to provide us with relevance scores of results links in our logs using which we'll generate Qrel file as used by xapian-letor. To train a click model, we'd need a relevance prediction dataset that should contain human generated binary relevance labels for query-document pairs. I'm curious to know from where we can obtain such a dataset. One that I know of is Yandex web seach challenge...

Hi Parth, I have a few questions- 1. Could you please provide me with the link for query-file, qrel-file for the dataset available at http://www.mpi-inf.mpg.de/departments/d5/software/inex/ . 2. I wanted to know how automated testing would be implemented. Will there be test cases like a test query must match this particular N results and this particular ranking. Or will it be in terms of evaluat...

...out the letor? > > And I'd like to test the function of letor. But I can't find code > > example. Can u give me some suggestions? > > Hopefully Parth can help here. > In order to get started with the Letor code you need to train the model and test it with the topic and qrel file. The necessary details about training and data is available at: http://trac.xapian.org/wiki/GSoC2011/LTR Cheers, Parth. > > Cheers, > Olly > > _______________________________________________ > Xapian-devel mailing list > Xapian-devel at lists.xapian.org > http://l...

...ion of questletor.cc has a lot of unusable and broken functions and is custom made for training with INEX 2010 dataset. The intention is to make it usable for a user provided database. Currently I am using xapian-docsprint/data/100-objects-v1.csv as my database and some manually written queries and qrels to make things work. 2. Going through v-hasu's GSoC 2014 code to understand extra functionalities added by him and planning how to introduce code from his branch. In summary, the approach I will follow is going to be: 1. Creating a code example that lets the user use 100-objects-v1.csv as th...

...resent Xapian::RankList create_rank_list(const Xapian::MSet & mset, std::string & qid, bool train) defined in FeatureManager does the job of preparing the FeatureVector for a query. At the time of preparing the training file, FeatureVector calculation can be done while parsing the query and qrel file, independently of RankList. Therefore, eliminating the need of maintaing a global qrel storage (map<string, map<string, int> > qrel; in FeatureManager::Internal) and thus eliminating the need of load_relevance(*) and getlabel(*) functions. The score in FeatureVector is simply the...

Hi, I am Jiarong Wei (irc: VcamX). I?m a 3rd year computer science student at Zhejiang University, China. I?m very willing to contribute to Xapian as part of GSoC 2014. Now I?m at Simon Fraser University, Canada, as an exchange student. I?ll go back to China on the end of April. I think it doesn?t matter I?ll change the time zone :) From the list of project?s ideas, Learning to Rank interests me

Hi Olly. Thanks for your reply to the previous email. To have an appropriate subject I've started this new thread for further discussions. > There's a $log{} command available in Omega templates. We can't log from > the result page template, as the clicks happen after that is used, but we > could make result links redirect via a second Omega template which does > the