similar to: Lucene 3.6.2 backend for xapian (#25)

On Mon, Sep 02, 2013 at 09:21:48AM +0800, jiangwen jiang wrote: > TfIdfWeight and BM25(b=0) also need wdf_upper_bound, it is not exists in > Lucene backends. If you don't provide an implementation of wdf_upper_bound(), the default is to use the collection frequency of the term, so provided that information is available in the lucene files, the lack of wdf_upper_bound information

Hi, all: I have wrote a demo patch for Backend for Lucene format indexes, Lucene version is 3.6.2. http://lucene.apache.org/core/3_6_2/fileformats.html Now, this demo patch just support the basic features in Lucene. Compound File(.cfs/.cfe)?term vector(.tvx/.tvd/.tvf) delete document(.del) are not supported, skip list in .fdx is not supported too example/quest.cc is used to test this demo.

On Tue, Aug 20, 2013 at 07:28:42PM +0800, jiangwen jiang wrote: > I think norm(t, d) in Lucene can used to caculate the number which is > similar to doc length(see norm(t,d) in > http://lucene.apache.org/core/3_5_0/api/all/org/apache/lucene/search/Similarity.html#formula_norm). It sounds similar (especially if document and field boosts aren't in use), though some places may rely on

On Mon, Aug 26, 2013 at 09:41:07AM +0800, jiangwen jiang wrote: > > For now, using weighting schemes which don't use document length is > > probably the simplest answer. > > There's tf-idf weighting scheme on svn master, is it suitable for lucene > backend? Yes - TfIdfWeight doesn't ever use the document length (at least with the normalisations currently

*Or do you mean that it's one number per document whereas the other stats are per database, so it's harder to store it?* yes, I mean this. It's a huge data. If a new doclength list(contains all the doclength in a list, like chert) is added by myself, I am concern about: 1. This doclength list may be the bottlenect in this backend, http://trac.xapian.org/ticket/326 2. Change too much

Hi all, Having problems trying to get Ferret to read an index generated by Lucene. Am I right in thinking Ferret should be able to read a Lucene generated index no problem? Using the code snippets detailed in http://www.ruby-forum.com/topic/64099#new Any advice gratefully received. Many Thanks, Steven -- Posted via http://www.ruby-forum.com/.

I haven''t dug into this yet, but wanted to report it. I''ve built an index with Java Lucene (1.9, from svn trunk) and then trying to search it with Ferret and getting this: /usr/lib/ruby/site_ruby/1.8/ferret/index/term_buffer.rb:31:in `read'': undefined method `name'' for nil:NilClass (NoMethodError) from /usr/lib/ruby/site_ruby/1.8/ferret/index/

Hi all, I've been toying with xapian (mostly using the Python bindings) and I think I've hit a bug in the TermIterator::skip_to() method (or maybe in QuartzAllTermsList::skip_to()). I've attached a c++ source file that demonstrates the issue. In short, if you have a WritableDatabase, ask for the all-terms TermIterator with db.allterms_begin(), and then skip_to() a word that is itself

Hi, I've tryed build dovecot with the lucene plugin but I have this is output: /bin/sh ../../../libtool --tag=CXX --mode=compile c++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -I/usr/local/include -I/usr/local/lib -O2 -pipe -fno-strict-aliasing

Hi ,i get an compile error security -Werror=format-security -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c lucene-wrapper.cc -fPIC -DPIC -o .libs/lucene-wrapper.o lucene-wrapper.cc: In function 'lucene_index* lucene_index_init(const char*, mailbox_list*, const fts_lucene_settings*)': lucene-wrapper.cc:128:2: error: 'else' without a previous 'if' make[5]:

We use Nutch and Lucene for our heavy duty text analysis jobs but I?m trying to use ferrret to do some experiments. I understood that Ferret used the same index format as lucene but I cannot look into a lucene index with ferret and cannot read a ferret index with luke (the lucene index browser). Am I doing somehting wrong or have the formats diverged? -------------- next part -------------- An

I found it quite fun to generate index by ferret. However, I had to use lucene now. Can I use the index that I''ve already done by ferret? ps. Lucene 2.2.0 vs Ferret 0.11.4 Thx! -- Posted via http://www.ruby-forum.com/.

Hi, It looks like I've found something strange. It looks like dovecot updates fts-lucene index every time I'm opening virtual folder that contains FTS query: indexer-worker(dion): Warning: fts-lucene: Settings have changed, rebuilding index for mailbox dovecot-virtual is pretty simple: archive/INBOX BODY "test" first of all, I'm trying to perform FTS in archive/INBOX

Hello, I have an "archive" folder in my inbox, where I manually stick old mails into a compressed mbox format. Since upgrading to Dovecot 2.2.1, I've started seeing messages like the following in my log files: imap(...): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable indexer-worker(...): Error: lucene: Failed to sync mailbox INBOX: Mailbox

Hi, I'm interested to implement the idea of using search indexes build by Lucene. I have some idea of how to do that. I'm currently studying the index format of both. I have written an application of search using Lucene and now I'll do the same using Xapian. then I'll check the details of those index format and then find how to convert Lucene index to Xapian one. Is there somebody

Hi, I'm currently looking over the FTS pages to enable it in my dovecot. But I'm unsure what the best settings of the lucene plugin are, if you receive german and english mails. Wiki says: textcat_conf=<path> textcat_dir=<path>: If specified, enable guessing the stemming language for emails and search keywords. This is a little bit problematic in practice, since indexing and

Kevin Burton has posted about poor ranking in Lucene preferring shorter documents over longer ones[1]. A similar search in Xapian returns documents in the expected order: Performing query `Xapian::Query(foo)' 3 results found ID 3 99% [foo foo foo] ID 2 94% [foo foo] ID 1 80% [foo] Anyone know what Lucene is doing here? Their FAQ doesn't mention what weighting scheme they use, and I

I'm getting some log errors with clucene that I am having no luck tracking down on the interwebs. Errors: May 19 05:05:16 indexer-worker(gessel at blackrosetech.com)<62971><aPAEI3zLw17A/QAA0J78UA:EF25M3zLw1779QAA0J78UA>: Error: lucene index /mail/blackrosetech.com/gessel//lucene-indexes: IndexWriter::addDocument() failed (#4): language not available for stemming May 19 05:05:16

I have a script (available on demand) that moves all my mail for a given month to a separate NAMESPACE (#ARCHIVE). When I do that, the lucene-indexes folder in the PRIMARY namespace seems to get cleaned out, and NOTHING gets put in the lucene-indexes folder (although it gets created) in the #ARCHIVE namespace. Is this to spec? the archiving is done via doveadm move commands. Thanks! --

Hello, try to use the new 2.1rc and don't get any data in the searchindex. Tried first lucene, and switched than back so solr. If I do an search in an mailfolder, i get in the dovecot-log 2012-02-06 22:17:11 | dovecot: | indexer-worker(xxxxxx): Indexed 0 messages in INBOX/dovecot Log from solr. Feb 6, 2012 10:17:11 PM org.apache.solr.core.SolrCore execute INFO: []