search for: chert

...lltermslist.o backends/.libs/dbcheck.o backends/.libs/database.o backends/.libs/databasereplicator.o backends/.libs/dbfactory.o backends/.libs/slowvaluelist.o backends/.libs/valuelist.o backends/.libs/dbfactory_remote.o backends/.libs/contiguousalldocspostlist.o backends/.libs/flint_lock.o backends/chert/.libs/chert_alldocsmodifiedpostlist.o backends/chert/.libs/chert_alldocspostlist.o backends/chert/.libs/chert_alltermslist.o backends/chert/.libs/chert_btreebase.o backends/chert/.libs/chert_check.o backends/chert/.libs/chert_compact.o backends/chert/.libs/chert_cursor.o backends/chert/.libs/chert_...

As Chert backend is not the default Xapian backend for the 1.2 branch, I am thinking about switching from Flint to Chert. However, I am wondering how do both compare in performance and index size? My indexes are updated quite frequently, so this is my priority. Which is better in this case? Regards, PK

Hi, I am new to xapian and read somewhere that xapain was using flint before released version 1.1 now it is using chert. I am looking for differences between both flint vs chert respective to xapian and advantages of chert. thanks Smriti

...lure" To: Xapian Development <xapian-devel at lists.xapian.org> I have been waiting for reply regarding any further steps to take. Following are the outputs of commands that you asked me to run. All these commands were executed in order. Commands: make remove-cached-databases rm -rf .chert .glass .multichert .multiglass .replicatmp .stub Command:./runtest ./apitest --verbose topercent2 Output: Running test './apitest --verbose topercent2' under eatmydata and valgrind Running tests with backend "none"... Running tests with backend "inmemory"... Running tes...

...ze=50000 > > The issue has now become one of performance. I am indexing 2.2 million > documents. Using delve I can see that performance starts off at about > 100,000 records an hour. This is consistent with the roughly 24 hour > rebuild time I was experiencing with Xapian 1.2.21 (chert). However, > after 75 hours of build time, the index is about 75% complete and > records are processing at a rate of 10,000/hr. The index is 51GB is > size, 30GB is position.glass. One of the big differences between chert and glass is that glass stores positional data in a different o...

Hi, What's the main difference between chert and flint? What above vs lucene? I am mainly asking about data structure (lexicon, posting list, document data), what's in memory, what's on disk, hash vs b-tree and reasons behind them. Any pointer is appreciated. Thanks! Crystal -------------- next part -------------- An HTML attachme...

Greetings, I've been wondering about the index format and backwards compatibility. We're using the dev version (for chert) and each svn up means that any indexes created prior to this revision cannot be read. Is this purely a cautious move to prevent errors, and, barring any obvious index format changes, can I safely force the current revision to read existing indexes? eg, by commenting out the throw in xapian-core/...

I've some problems converting a xapian db, created with core 1.1.3 (using chert), to the new chert format. I'm using xapian-chert-update, compiled from the core-1.2.4. The conversion seems to run without errors: #./xapian-core-1.2.4/bin/xapian-chert-update old new postlist: Reduced by 33.3333% 16K (48K -> 32K) record: Size unchanged (8K) termlist: doesn't exist pos...

...reopen() and retry the operation Down below the output of "dmalloc". This happens only on the production system. On my local maschine, "valgrind" reports "all blocks freed". Thanks a lot Marcus total-size count source 294912 36 Line 64 of "backends/chert/chert_cursor.cc" 122880 15 Line 2070 of "backends/chert/chert_table.cc" 26021 5 Line 262 of "backends/chert/chert_btreebase.cc" 26021 5 Line 263 of "backends/chert/chert_btreebase.cc" 2056 1 Line 358 of "backends...

Report by Eric Lindblad 06-10-2015 http://www.ericlindblad.blogspot.com cf: http://permalink.gmane.org/gmane.comp.search.xapian.general/9885 modifications allowing for brass and chert to compile on SFU Interix 3.5 (modified) [code change not performance critical] /xapian-core-1.2.21/backends/brass/brass_check.cc /xapian-core-1.2.21/backends/chert/chert_check.cc #include <climits> + +#include "str.h" out << "base" << (char)B.base_letter...

Report by Eric Lindblad 30-09-2015 http://www.ericlindblad.blogspot.com The xapian-core-1.2.21 'ambiguous overload' error on the files /backends/brass/brass_check.cc and /backends/chert/chert_check.cc appear to be resultant of a bug reported in gcc-3.3.4, which was fixed for 3.4.2 [Sept. 6, 2004] and 3.5. Bug 16854 - streams missing "long long" specializations on Tru64 https://gcc.gnu.org/bugzilla/show_bug.cgi?format=multiple&id=16854 Running 'g++ foo.cc' (...

So, given what I've read in the documentation I would create a text file named document_database.txt that might have the following: remote 192.168.1.10:30000 chert /var/lib/xapian_database/segment1 remote 192.168.1.10:30000 chert /var/lib/xapian_database/segment2 remote 192.168.1.10:30000 chert /var/lib/xapian_database/segment3 etc. I would then in my PHP program open document_database.txt as the database and then perform normal Xapian calls. The same w...

...apian-check reports no errors found, so at least the database seems consistent on disk. What's interesting is that if I set XAPIAN_FLUSH_THRESHOLD to be a value larger than the document set size (and I am not explicitly flushing either), the problem does not occur. The problem occurs with the chert and brass backends, but not flint. (Nor does it, perhaps obviously, occur with xapian-core 1.0.16.) Just to rule it out, I also tried 1.1.4 built with gcc 4.1 (rather than 4.2 with -fno-strict-aliasing) but the result was the same. I am trying to reproduce the problem as a smaller test case, but...

Please find my inline reply. On 27 January 2015 at 03:02, Olly Betts <olly at survex.com> wrote: > > The script which automates most of the release process runs "make > distcheck", which runs the testsuite, so these tests didn't fail for me > when I made the 1.3.2 release. > > When I run "make distcheck" in the "build" folder, it gives

Hi, w.r.t. http://trac.xapian.org/ticket/326 I'd like to try out some of the patches mentioned against my xapian svn trunk to see what kind of improvement can be realised. There are quite a few patches, so can I simply try them all top-to-bottom, or which of them are recommended (ie, I don't want to screw with the index format)? Thanks h

...; Recoll builds snippets by partially reconstructing documents out of index > > contents. > > > [...] > > > > The specific operation which has become slow is opening many term position > > lists, each quite short. > > The difference will actually be chert vs glass, rather than 1.2 vs 1.4 > as such (glass is the new backend in 1.4 and now the default). I had sort of guessed this far :) > [... physical ordering of the position list table changed ...] > > I'm not sure there's much we can do about this directly - changing the...

...> The issue has now become one of performance. I am indexing 2.2 million > > documents. Using delve I can see that performance starts off at about > > 100,000 records an hour. This is consistent with the roughly 24 hour > > rebuild time I was experiencing with Xapian 1.2.21 (chert). However, > > after 75 hours of build time, the index is about 75% complete and > > records are processing at a rate of 10,000/hr. The index is 51GB is > > size, 30GB is position.glass. > > One of the big differences between chert and glass is that glass stores > po...

Hi, I have had reports that Recoll has become unbearingly slow in some instances. After inquiry, this happens with Xapian 1.4 only, and the part which does not work any more is the snippets extraction. Recoll builds snippets by partially reconstructing documents out of index contents. For this, after determining a set of document term positions to be displayed (around the hopefully interesting

.... For searching, I followed such steps: 1. construct a QueryParser for certain string 2. parse the query to get a Xapian::Query 3. construct an Enquire for searching by calling get_mset method here is the function-time-cost for searching: samples % symbol name 75649 28.0401 ChertPostList::move_forward_in_chunk_to_at_least(unsigned int) 30118 11.1635 Xapian::BM25Weight::get_sumpart(unsigned int, unsigned int) const 21291 7.8917 AndMaybePostList::process_next_or_skip_to(double, Xapian::PostingIterator::Internal*) 17803 6.5989 OrPostList::next(double) 12481 4...

...ng a bunch of patches). Naturally, I'd now do a native 64-bit build (unless somebody tells me a 32-bit build would actually be preferable in terms of memory savings due to pointer sizes?). Doing some light (read-only!) testing, it seems to work fine to access the old 32-bit built Xapian "chert" database with 64-bit notmuch/Xapian. Is that (a) generally safe and expected to work fine, (b) there may be issues, or (c) don't do that, or don't know? (Of course, I'll eventually want to rebuild the database, to take advantage of several new features both on the notmuch and Xa...