search for: postingiterator

olly wrote: > Log message (7 lines): > backends/database.cc: Database::Internal can't call the > PostingIterator(PostingIterator::Internal*) ctor (at least under > g++ 3.3.5) because it isn't a friend (only class Database is). For the record, Mark just reported this to me under windows so it was a problem there too, but it does work under GCC 4.1. No idea which compiler is "correct", but...

Hi, I'm using xapian with c# and mono and i'm having a segfault in get_weight. When i print the index variable, the value is clearly too high. I think something write over it. Do you have any idea on how i could trace the beginning of the segmentation fault ? Thanks, -- Yann

...> in which case we only need to track the docid, so that could be just 4 > bytes per result which is ~65MB. Right, that's great news as we creep towards 50 or 100 million docs > Incidentally you don't mind the export order and only have single term > queries you can just use a PostingIterator to get a stream of document > ids matching a particular term (in the order documents were added), > which should use at most ~80KB (per shard if you're using a sharded > database). Thanks for that tip on PostingIterator, I'll keep it in mind; but I think there's usually >=...

Hi all. I want to iterate over all the documents in my database. my $pi = $db->postlist_begin(""); while ("$pi" =~ qr/END/) { my $oldid = $pi->get_docid; $pi++; #... } That used to work with Search::Xapian in perl version 1.2, but now with xapian-bindings-1.4.4 it does not seem to. How are you supposed to tell when you have reached the

....28.so [.] _dl_lookup_symbol_x 0.06% /mnt/btr/public perl [.] PerlIO_isutf8 0.06% script/public-i perl [.] Perl_gv_fetchpvn_flags 0.06% script/public-i libxapian.so.30.8.0 [.] std::__push_heap<__gnu_cxx::__normal_iterator<Xapian::PostingIterator::Internal**, std::vector<Xapian::PostingIterator::Internal*, std::allocator<Xapian::PostingIterator::Internal*> > >, long, Xapian::PostingIterator::Internal*, __gnu_cxx::__ops::_Iter_comp_val<Xapian::Internal::ComparePostListTermFreqAscending> > 0.06% /mnt/btr/public...

Going back to the "prioritizing aggregated DBs" thread from February 2020, I've got 390 Xapian shards for 130 public inboxes I want to search against(*). There's more on the horizon (we're expecting tens of thousands of public inboxes). After bumping RLIMIT_NOFILE and running ->add_database a bunch, the actual queries seem to be taking ~30s (not good :x). Now I'm

...rror LNK2001: ????????? "public: virtual class RemoteDatabase * __thiscall RemoteDatabase::as_remotedatabase(void)" (?as_remotedatabase at RemoteDatabase@@UAEPAV1 at XZ) libinmemory.lib(inmemory_database.obj) : error LNK2001: ????????? "public: virtual unsigned int __thiscall Xapian::PostingIterator::Internal::get_wdf(void)const " (?get_wdf at Internal@PostingIterator at Xapian@@UBEIXZ) libinmemory.lib(inmemory_database.obj) : error LNK2001: ????????? "public: virtual class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const * __t...

Hello, I noticed a problem with DatabaseCorruptError exceptions with public-inbox and I guess it's user error... The problem is public-inbox was calling replace_document to modify the DB while iterating through a PostingIterator. At first I thought it was a glass problem, but I've hit it with chert on my dataset, too. I have a standalone Perl script to reproduce the problem at https://yhbt.net/skel.bug.perl and 81M gzipped dataset which reproduces the problem at https://yhbt.net/skel.bug.gz (each line is: MID [REFER...

Hello again, /thanks to Peter for previous response. I've been digging around trying to find sample usage of XapianTermIterator/XapianPositionIterator in PHP. The idea is to code up a test case in PHP to perform snippet extraction (with a possible view to coding a pecl extension in C). I found a C++ sample, but that wasn't much help. I must be dense this morning though, since I

...of size 216 free'd ==2265126== at 0x4C2A360: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==2265126== by 0x99C0B75: operator() (queryinternal.cc:65) ==2265126== by 0x99C0B75: for_each<__gnu_cxx::__normal_iterator<Xapian::PostingIterator::Internal**, std::vector<Xapian::PostingIterator::Internal*> >, delete_ptr<Xapian::PostingIterator::Internal> > (stl_algo.h:3755) ==2265126== by 0x99C0B75: Xapian::Internal::Context::reset() (queryinternal.cc:155) ==2265126== by 0x99C481F: Xapian::Int...

...odifiedError' > > Using `eval' from Perl5 doesn't seem effective in catching it. > > I'm using postlist_begin, postlist_end and ++ to iterate a > PositionIterator, and reading XS/*Iterator.xs, I see the XS > wrappers are missing: FWIW, postlist_begin gives you a PostingIterator (positionlist_begin would give you a PositionIterator). > try { } catch { handle_exception(); } > > in several places. Not sure if that's it or if something > else (I know C, but am not remotely fluent in C++). That is indeed what's missing. I'm open to clean patches...

Hi, As part of my whole indexing thing, I want to move the databases around, from within the code. So, I figured that the safest bet would be to close the DB before attempting to move it. But there's no Xapian::Database::close() or even Xapian::WritableDatabase::close(). I have a global variable that holds the database, and I open it in the main(): // globals Xapian::WritableDatabase

...on-time-cost for searching: samples % symbol name 75649 28.0401 ChertPostList::move_forward_in_chunk_to_at_least(unsigned int) 30118 11.1635 Xapian::BM25Weight::get_sumpart(unsigned int, unsigned int) const 21291 7.8917 AndMaybePostList::process_next_or_skip_to(double, Xapian::PostingIterator::Internal*) 17803 6.5989 OrPostList::next(double) 12481 4.6262 AndMaybePostList::get_weight() const 10729 3.9768 OrPostList::get_weight() const 10096 3.7422 AndMaybePostList::next(double) 8743 3.2407 ChertDatabase::get_doclength(unsigned int) const 7527 2.7900 LeafPo...

I'm using Search::Xapian XS from Debian stable and I'm getting what appears to be an unwrapped C++ exception: terminate called after throwing an instance of 'Xapian::DatabaseModifiedError' Using `eval' from Perl5 doesn't seem effective in catching it. I'm using postlist_begin, postlist_end and ++ to iterate a PositionIterator, and reading XS/*Iterator.xs, I see the

...::Database srcdb = Xapian::Database(); while (*sources) { srcdb.add_database(Xapian::Database(*sources++)); } /* create a destination database */ Xapian::WritableDatabase destdb = Xapian::WritableDatabase(dest, Xapian::DB_CREATE); /* copy all matching documents to the new DB */ Xapian::PostingIterator it; for (it = srcdb.postlist_begin(""); it != srcdb.postlist_end(""); it++) { Xapian::docid did = *it; Xapian::Document doc = srcdb.get_document(did); std::string cyrusid = doc.get_value(SLOT_CYRUSID); if (cb(cyrusid.c_str(), rock)) { destdb.add_document(d...

...by 0x5F4E171: check_helper (multiandpostlist.h:97) ==11180== by 0x5F4E171: MultiAndPostList::find_next_match(double) (multiandpostlist.cc:217) ==11180== by 0x5F44C3F: skip_to_handling_prune (branchpostlist.h:98) ==11180== by 0x5F44C3F: AndNotPostList::advance_to_next_match(double, Xapian::PostingIterator::Internal*) (andnotpostlist.cc:50) ==11180== by 0x5F4E317: next_helper (multiandpostlist.h:76) ==11180== by 0x5F4E317: MultiAndPostList::next(double) (multiandpostlist.cc:238) ==11180== by 0x5F4FACC: next_handling_prune (branchpostlist.h:85) ==11180== by 0x5F4FACC: MultiMatch::get_mset(...

Hello, I'm still testing PHP5 bindings and I could'nt find a way to delete all documents from a DB without deleting other informations stored in the DB such as synonyms. Since the process of adding synonyms is time consuming, I would like to use the same DB but restart my test without any document in the DB, is this possibile? I could'nt find a delete_all or a method like that, nor

...anifying blib/man3/Search::Xapian::DatabaseCorruptError.3pm Manifying blib/man3/Search::Xapian::BoolWeight.3pm Manifying blib/man3/Search::Xapian::DocNotFoundError.3pm Manifying blib/man3/Search::Xapian::LogicError.3pm Manifying blib/man3/Search::Xapian::Stem.3pm Manifying blib/man3/Search::Xapian::PostingIterator.3pm Manifying blib/man3/Search::Xapian::PositionIterator.3pm Manifying blib/man3/Search::Xapian::MultiValueSorter.3pm Manifying blib/man3/Search::Xapian::FeatureUnavailableError.3pm Manifying blib/man3/Search::Xapian::DatabaseVersionError.3pm Manifying blib/man3/Search::Xapian::DatabaseOpeningError...

I'm trying to compact (or at least merge) multiple databases, while stripping search records which are no longer required. Backstory: I've inherited the Cyrus IMAPd xapian-based search code from Greg Banks when he left Opera. One of the unfinished parts was removing expunged emails from the search database. We moved from having a single search database to supporting multiple

...ist.o api/.libs/editdistance.o api/.libs/emptypostlist.o api/.libs/error.o api/.libs/errorhandler.o api/.libs/expanddecider.o api/.libs/keymaker.o api/.libs/leafpostlist.o api/.libs/matchspy.o api/.libs/omdatabase.o api/.libs/omdocument.o api/.libs/omenquire.o api/.libs/positioniterator.o api/.libs/postingiterator.o api/.libs/postingsource.o api/.libs/postlist.o api/.libs/query.o api/.libs/queryinternal.o api/.libs/registry.o api/.libs/replication.o api/.libs/smallvector.o api/.libs/sortable-serialise.o api/.libs/termiterator.o api/.libs/termlist.o api/.libs/valueiterator.o api/.libs/valuerangeproc.o api/.li...