similar to: DFR framework as a GSOC project

Hello guys.I am working on implementing the PL2 weighting scheme of the DFR framework by Gianni Amati. It uses the Poisson approximation of the Binomial as the probabilistic model (P), the Laplace law of succession to calculate the after effect of sampling or the risk gain (L) and within document frequency normalization H2(2) (as proposed by Amati in his PHD thesis). The formula for w(t,d) in

Hey guys, hi :) I have some ideas about implementing a general DFR framework that will be able to generate any DFR model. I'm thinking of implementing a dummy DFR framework that will be capable of generating a fixed named scheme right now but can be extended later to include all the schemes.Is it okay if I send a pull request for such a dummy framework to get it reviewed by the community so

Hello guys. I just ran the updated tests for PL2 and they are not giving the mset order I expect.Now,the thing is, dfr's behavior is a bit hard to predict and so even if I expect a particular order ,it may give another order and still be correct.So,the only way to write correct tests for PL2 is to manually calculate the weight of the documents to decide the expected order.For that,I need to

Hi, guys I am Wenjin from Graduate School of Chinese Academy of Science, pursing a master degree and my current research interests including using Data mining and Information retrieve technology to analysis software engineering (SE) data and support SE. I have great interested in "Weight Schemes" project. and in the last few days I have learnt some detail about DFR model family by

Hello guys.I just realized that Ive not set the weighting scheme to PL2 in the tests for PL2 and so a default weighting scheme of BM25 is used. I am extremely sorry for this and am updating the tests by setting the weighting scheme to PL2. -Regards -Aarsh -------------- next part -------------- An HTML attachment was scrubbed... URL:

Hi olly, I rewrote the test for PL2 after adding code to deal with negative weights. It passes on all backends other than inmemory . I see a different value of weight for inmemory backend. The code to calculate the lower bound is implemented in init().Please can I get some help with this ? -Regards -Aarsh -------------- next part -------------- An HTML attachment was scrubbed... URL:

Hello guys.I have added various tests to the PL2 pull request.They are working fine. Have also added PL2 to the registry and to the java and csharp makefiles.Please do let me know what you'll think.Other than the collection frequency problem discussed on IRC, it is ready.Am now beginning work on adding code and tests for DPH to the same branch. -Regards -Aarsh -------------- next part

Hi Parth, I implemented DFR algorithms in Xapian as a part of GSOC last year under the mentorship of Olly. This year, I want to work on analyzing and optimizing the performance of the DFR algorithms and comparing them with BM25.I also want to work on profiling the query expansion schemes and test the relevance(precision and recall) / speed(time taken) of the

Hi everyone, I am thinking of working on the following ideas for my GSOC proposal based on my discussions with Olly and my own understanding. Rather than focusing on an entire perftest module, I have decided to focus on implementing performance tests for weighting schemes based on a wikipedia dump and in addition to that, build a framework to measure the

Hello guys, :) Please do take a look at the pull request for the TfIdf patch Ive sent because I want to start working on writing DFR schemes for us and want to incorporate the feedback into making a good hack for the DFR schemes.The patch incorporates all normalizations possible with our current statistics and passed all the tests I wrote for it.Have also attached the tests with the pull request.

Hello there, I'm trying to do lots of plots in one for-loop. But somehow ggplot does not evaluate arguments as expected. Here is an example: library(lattice) library(ggplot2) pl <- list() pl2 <- list() cDat <- as.data.frame(cbind(x1=0:100,x2=0:10,x3=1:20)) for(obs in c("x1", "x2")) { pl[[obs]] <- xyplot(cDat[,obs] ~ cDat[,"x3"], main=obs)

Hello Guys. I have updated the code,tests,documentation,makefile entries and the registry entry of the* *TfIdf patch as per the feedback.Please do let me know if any additional changes are required before the patch can be merged, -Regards -Aarsh On Sun, Mar 3, 2013 at 2:50 PM, aarsh shah <aarshkshah1992 at gmail.com> wrote: > Hello guys.I have sent a pull request for the code and

Hi everyone, I'm Akshay, an Information Science undergrad from Bangalore. I'm interested in Information Retrieval and I'd like to contribute to Xapian as a part of GSoC and later to feed my interests. I liked the idea of adding more weighting schemes (Project #2). I did a project last semester on Document Retrieval on Hadoop using TF-IDF and Cosine Similarity (the query had to be a

Hello, I'm trying to generate a log-log plot with ggplot. Within this plot, I would like to draw a straight line with the help of ggabline which does not work. The code is pl1 <- qplot(correlFunc, correlFunc.ref, data=all, log=c("xy")) pl1 <- ggabline(pl1, slope=1, intercept=0) print(pl1) ## no line in plot :( I can produce plots as I want, if I transform all values before

Hello guys.I have sent a pull request for the code and tests of the Tf-Idf weighting scheme. Please do let me know if any changes are required.Meanwhile,Ill begin working on implementing normalizations which require additional statistics and on the DFR schemes. https://github.com/xapian/xapian/pull/6 On Tue, Feb 26, 2013 at 5:30 PM, <xapian-devel-request at lists.xapian.org>wrote: >

Hi, Here's a problem in openssh, some logs, and a very minor patch that cures this: Issue: (open)ssh client WILL NOT talk to F secure SSH-2.0-2.1.0pl2 client S/W version: openssh-2.3.0p1 client O/S version: SunOS 5.7 Generic_106541-11 sun4u sparc server S/W version: SSH-2.0-2.1.0pl2 server O/S version: SunOS 5.7 Generic_106541-11 sun4u sparc Log/Details: : % telnet <mymachine> 22

Hello guys.I just read up about tf-idf schemes and want to implement it in Xapian (with some frequently used normalizations) as it will also give me a good hang of implementing a weighting scheme before I start working on implementing DFR schemes. I read the following as references and I think Ive understood it well and can write the hack :- 1.)

Dear all, let consider the following function: Fun1 <- function() { library(lattice) plot1 <- 1:10~1:10 pl1 <- xyplot(plot1) return(pl1$call$x) } In R 2.5.0 (or older version) we have > Fun1() plot1 but starting from R 2.5.1 until the latest R 2.8.0 we obtain instead > Fun1() NULL because pl1$call seems to be equal to xyplot() without arguments. Something like

Hi, I encountered a booting problem with memdisk 2.83, USB and IBM Thinkpad T61, apparently the same issue as described here: http://syslinux.zytor.com/archives/2008-April/009850.html The boot process always stops after "Loading boot sector... booting...". With debug tracers enabled, the last few output lines are: Loading boot sector... FR<p>Dbooting...

A loose idea for *post*-0.50 development I've been giving a some (but not all that many) thoughts to whether some of the conceptual difficulties facing newcomers could be avoided by having simplified functions for common operations. We already have parts of this, e.g. in Kurts ctest routines. Specifically, I was thinking about data frames: How about