similar to: "Value in posting list too large" error with 1.1.4 (chert and brass, not flint)

Hi, What's the main difference between chert and flint? What above vs lucene? I am mainly asking about data structure (lexicon, posting list, document data), what's in memory, what's on disk, hash vs b-tree and reasons behind them. Any pointer is appreciated. Thanks! Crystal -------------- next part -------------- An HTML attachment was scrubbed... URL:

Hi, I am new to xapian and read somewhere that xapain was using flint before released version 1.1 now it is using chert. I am looking for differences between both flint vs chert respective to xapian and advantages of chert. thanks Smriti

Report by Eric Lindblad 30-09-2015 http://www.ericlindblad.blogspot.com The xapian-core-1.2.21 'ambiguous overload' error on the files /backends/brass/brass_check.cc and /backends/chert/chert_check.cc appear to be resultant of a bug reported in gcc-3.3.4, which was fixed for 3.4.2 [Sept. 6, 2004] and 3.5. Bug 16854 - streams missing "long long" specializations on Tru64

Report by Eric Lindblad 06-10-2015 http://www.ericlindblad.blogspot.com cf: http://permalink.gmane.org/gmane.comp.search.xapian.general/9885 modifications allowing for brass and chert to compile on SFU Interix 3.5 (modified) [code change not performance critical] /xapian-core-1.2.21/backends/brass/brass_check.cc /xapian-core-1.2.21/backends/chert/chert_check.cc #include <climits> +

As Chert backend is not the default Xapian backend for the 1.2 branch, I am thinking about switching from Flint to Chert. However, I am wondering how do both compare in performance and index size? My indexes are updated quite frequently, so this is my priority. Which is better in this case? Regards, PK

hi, i use xapian to index a txt file, it's size is 268M. i take each line as a document, and each line has two field like 13445511 | 111115151. the recored size is 10000000. the XAPIAN_FLUSH_THRESHOLD set 1000000. it takes 1026544ms to index the file, it is more slower than lucene. The lucene speed is about 40000 records per second. code: try { Xapian::WritableDatabase

Flint failed to deliver indexing performance to Quartz. I am proposing to remove Flint as default database and place Quartz database back as default. The catch is not that Flint database is smaller and faster during searches then Quartz database as developers were concerning when were measuring and neglecting to measure performance when creating the large indexes. The truth is that Flint

On Sat, Mar 25, 2017 at 06:36:25PM -0500, Ryan Cross wrote: > After upgrades my stack is now: > > Python 2.7 > Django 1.8 > Haystack 2.6.0 > Xapian 1.4.3. (latest xapian haystack backend with some modifications) > > Using the same rebuild command as below but with —batch-size=50000 > > The issue has now become one of performance. I am indexing 2.2 million >

I'm playing around with a machine that has 2 GB of memory. Indexing about 5GB of data average of 2MB per document. The documents are plain text. I notice the omindex's memory fott print get's biger an bigger then the machine starts to swap and it all slows down to a crawl. In regards to export XAPIAN_FLUSH_THRESHOLD I know the default is 10000 Am I right in saying that for my setup

Hi guys, I discovered xapian today and I'm trying it because it looks promising compared to other FTS engines. However I'm in a windows environment so it's a bit painful to set up as there is no official binaries. I've heard about http://www.flax.co.uk/xapian_binaries And I'm trying to build their latest release (1.2.8). After a few tries I manage to compile everything with

I'm one of the developers of Notmuch, an email client that makes extensive use of Xapian. For some time, full multi-version concurrency control (MVCC) has been on our wish list. Olly mentioned that brass now uses free lists---a key step toward full MVCC---and I offered to pitch in on putting the other pieces into place. Before I wade too deep into this, I wanted to propose my plan. Full

Report by Eric Lindblad 13-09-2015 http://www.ericlindblad.blogspot.com cf: http://comments.gmane.org/gmane.comp.search.xapian.general/9862 I ported xapian-core-1.2.21 to Interix today having disabled the default chert and development brass backends, using flint, which was the 1.0.x series' default, which ships with the 1.2.x series; the disabling do to an unresolved 'ambiguous

Greetings, I've been wondering about the index format and backwards compatibility. We're using the dev version (for chert) and each svn up means that any indexes created prior to this revision cannot be read. Is this purely a cautious move to prevent errors, and, barring any obvious index format changes, can I safely force the current revision to read existing indexes? eg, by

There is is bug when setting XAPIAN_FLUSH_THRESHOLD=20000000 When trying for force Xapian flush documents to flush after 20 million documents Xapian ignores the size and flush it after only 10,000 documents. Data captured from delve after 60 seconds interval when has been set as follow: XAPIAN_FLUSH_THRESHOLD=20000000 perl -e ' while(1) { system("delve ."); sleep(60); } '

Hi, w.r.t. http://trac.xapian.org/ticket/326 I'd like to try out some of the patches mentioned against my xapian svn trunk to see what kind of improvement can be realised. There are quite a few patches, so can I simply try them all top-to-bottom, or which of them are recommended (ie, I don't want to screw with the index format)? Thanks h

Report by Eric Lindblad 14-09-2015 http://www.ericlindblad.blogspot.com cf: http://permalink.gmane.org/gmane.comp.search.xapian.general/9880 The 'gmake check' compile error on xapian-core-1.2.21/tests/api_backend.cc was resolved by the modification (at the suggestion of a mailing-list member) of the offending string from "q.OP_PHRASE" to "Xapian::Query::OP_PHRASE".

I've released Xapian 1.1.5. We're getting really close to releasing a stable 1.2.0, and this is a possible release candidate. So please try it out and report any issues you find. We're hopefully looking at a 1.2.0 release later this month. However, this is still nominally a development release with the usual provisos about ABI and database formats potentially being subject to

I am suffering for slow searching performance on Xapian. I am using Xapian for indexing about 150,000,000 documents. It was implemented in C++; The performance of searching was not that fast. e.g. Searching a query, which includes about 20 terms, needs 2 secs avg. For searching, I followed such steps: 1. construct a QueryParser for certain string 2. parse the query to get a Xapian::Query

On Sun, 2 Apr 2017, at 20:29, Olly Betts wrote: > On Sat, Mar 25, 2017 at 06:36:25PM -0500, Ryan Cross wrote: > > After upgrades my stack is now: > > > > Python 2.7 > > Django 1.8 > > Haystack 2.6.0 > > Xapian 1.4.3. (latest xapian haystack backend with some modifications) > > > > Using the same rebuild command as below but with

Gentoo Linux 2.6 8 AMD Opteron 64-bit Processors 32GB Memory -------------------------------------------------------------------------------- Environment: ------------------ XAPIAN_FLUSH_THRESHOLD=21000000 XAPIAN_FLUSH_THRESHOLD_LENGTH=16000000 XAPIAN_PREFER_FLINT=True Indexing 20 million documents: --stemmer=none ------------------------------------------- real 79m9.378s user 77m28.696s