search for: multimatch

...ith serialization approach when given value types meet float/double. Such an approach is deployed on sort_key related fields as well, where the xapian requires KeyMaker::operator() must return an serialized float/dobule variable. Then heap sort comes and ranks the vector<MSetItem> items (multimatch.cc MultiMatch::get_mset()) by comparing serialized sort_keys (std::string) straightforwardly according to <IEEE-754 doubles>. Subsequently sort_keys will be unserialized when user needs to read its real float/double values during iterations of result MSet. Obviously, serialization and un...

...pian::PostingIterator::Internal*) (andnotpostlist.cc:50) ==11180== by 0x5F4E317: next_helper (multiandpostlist.h:76) ==11180== by 0x5F4E317: MultiAndPostList::next(double) (multiandpostlist.cc:238) ==11180== by 0x5F4FACC: next_handling_prune (branchpostlist.h:85) ==11180== by 0x5F4FACC: MultiMatch::get_mset(unsigned int, unsigned int, unsigned int, Xapian::MSet&, Xapian::Weight::Internal&, Xapian::MatchDecider const*, Xapian::KeyMaker const*) (multimatch.cc:570) ==11180== by 0x5E485CE: Xapian::Enquire::Internal::get_mset(unsigned int, unsigned int, unsigned int, Xapian::RSet const...

...code for the proxy,internal and iterator classes of Eset and Mset to get a feel of how those sets work.I then traced the working of Enquire::get_eset( ) (understood it well other than how a Termlist tree is built ) and Enquire::get_mset( ) (didn't understand this one completely,got lost during Multimatch::get_mset()) .I also read up the code for Xapian::Weight (both proxy and internal class) and the codes of BM25 and TradWeight classes . The hack now seems fairly straightforward as the only difference between BM25 and TradWeight (as far as ranking terms to build an Eset is concerned) is the replac...

...here are some important things which would be essential) 1) **IMPORTANT** Add get_revision_info() in Database API (optionally guaranty that, if returned, strings could be compared between revisions to know which revision comes first, using string::compare or strcmp) 2) **IMPORTANT** Make a public MultiMatch API (to be able to re-implement the remote protocol ourselves) 3) **IMPORTANT** Send selected database path using MSG_SELECT to the remote protocol if REPLY_UPDATE returns with no database selected (that's what our protocol would do) 4) *OPTIONAL* Add MSG_SELECT to remote protocol (as an exam...

...ist::skip_to(unsigned int, double) 4341 1.6090 ChertPostList::next_in_chunk() 4207 1.5594 ChertPostList::get_docid() const 4065 1.5067 ChertPostList::at_end() const 3988 1.4782 AndMaybePostList::at_end() const 3899 1.4452 OrPostList::get_docid() const 3655 1.3548 MultiMatch::get_mset(unsigned int, unsigned int, unsigned int, Xapian::MSet&, Xapian::Weight::Internal const&, Xapian::MatchDecider const*, Xapian::MatchDecider const*, Xapian::KeyMaker const*) 3172 1.1757 OrPostList::at_end() const 3061 1.1346 ChertPostList::get_wdf() const most of the...

...branchpostlist.o matcher/.libs/collapser.o matcher/.libs/exactphrasepostlist.o matcher/.libs/externalpostlist.o matcher/.libs/localsubmatch.o matcher/.libs/maxpostlist.o matcher/.libs/mergepostlist.o matcher/.libs/msetcmp.o matcher/.libs/msetpostlist.o matcher/.libs/multiandpostlist.o matcher/.libs/multimatch.o matcher/.libs/multixorpostlist.o matcher/.libs/nearpostlist.o matcher/.libs/orpositionlist.o matcher/.libs/orpospostlist.o matcher/.libs/orpostlist.o matcher/.libs/phrasepostlist.o matcher/.libs/selectpostlist.o matcher/.libs/synonympostlist.o matcher/.libs/valuegepostlist.o matcher/.libs/valuera...

...ckend.a/inmemory_document.o .libs/libxapian.lax/libbackend.a/flint_document.o .libs/libxapian.lax/libmatcher.a/expand.o .libs/libxapian.lax/libmatcher.a/tradweight.o .libs/libxapian.lax/libmatcher.a/expandweight.o .libs/libxapian.lax/libmatcher.a/andmaybepostlist.o .libs/libxapian.lax/libmatcher.a/multimatch.o .libs/libxapian.lax/libmatcher.a/filterpostlist.o .libs/libxapian.lax/libmatcher.a/rset.o .libs/libxapian.lax/libmatcher.a/andnotpostlist.o .libs/libxapian.lax/libmatcher.a/networkmatch.o .libs/libxapian.lax/libmatcher.a/ortermlist.o .libs/libxapian.lax/libmatcher.a/localmatch.o .libs/libxapian.l...

...scalarseq 0.04% perl perl [.] Perl_op_linklist.part.16 0.04% script/public-i libc-2.28.so [.] __memset_sse2_unaligned_erms 0.04% script/public-i libxapian.so.30.8.0 [.] GlassPostList::init 0.04% script/public-i libxapian.so.30.8.0 [.] MultiMatch::get_mset 0.04% perl perl [.] Perl_op_sibling_splice 0.04% perl libpthread-2.28.so [.] __pthread_mutex_lock 0.04% perl perl [.] Perl_op_convert_list 0.04% git libc-2.28.so [.] __access...

Going back to the "prioritizing aggregated DBs" thread from February 2020, I've got 390 Xapian shards for 130 public inboxes I want to search against(*). There's more on the horizon (we're expecting tens of thousands of public inboxes). After bumping RLIMIT_NOFILE and running ->add_database a bunch, the actual queries seem to be taking ~30s (not good :x). Now I'm