search for: get_mset

Hello, This is a somewhat idle question about Enquire::get_mset(), possibly based on wrong assumptions on how Xapian works, but I would be grateful if someone satisfied my curiosity. Xapian::MSet Xapian::Enquire::get_mset(Xapian::doccount first, Xapian::doccount maxitems, const Xapian::RSet * omrset = 0,...

I got some performance issue for document.get_data() and enquire.get_mset(). It costs 35 seconds for matches = enquire.get_mset(0,200), and 3 seconds for iterating all doc in matches to get_data. Is't normal? My index contains 30millions documents. I use python binding to operate xapian. Bellow it's my index structure # value: 0:date, 1:site # data: json message...

On Thu, Aug 17, 2023 at 09:28:26PM +0000, Eric Wong wrote: > In other words, is it possible to avoid duplicates if new > documents are inserted into the DB by another process in-between > ->get_mset calls when reusing Xapian::Enquire objects? The Database object itself effectively does (it works in a snapshot of the state of the database when you open it, or last called reopen() which updates that snapshot to what's currently committed). However we don't currently have any locking of...

Olly Betts <olly at survex.com> wrote: > On Wed, Aug 23, 2023 at 01:53:27PM +0000, Eric Wong wrote: > > I'm already retrying the ->get_mset operations; but now I'm > > wondering where I'd hit DatabaseModifiedErrors while inside a > > Xapian::MSetIterator loop. > > > > I assume ->get_document is a place where it gets thrown; > > but once a document is retrieved, can iterating through > > t...

In other words, is it possible to avoid duplicates if new documents are inserted into the DB by another process in-between ->get_mset calls when reusing Xapian::Enquire objects? I do some expensive processing on each mset window, so I always limit the results to limit heap usage even if I'm planning on going through a big chunk of the DB: $mset = $enq->get_mset(0, 1000); do_something_slow_with_mset($mset); $mset = $en...

On Wed, Aug 23, 2023 at 01:53:27PM +0000, Eric Wong wrote: > I'm already retrying the ->get_mset operations; but now I'm > wondering where I'd hit DatabaseModifiedErrors while inside a > Xapian::MSetIterator loop. > > I assume ->get_document is a place where it gets thrown; > but once a document is retrieved, can iterating through > terms in one document (using Te...

Hi, I try to use XapianMatchDecider to filtering results, but the code : $Decider = new MatchDecider_Search($XapianDatabase->_cPtr); $mset = $XapEnquire->get_mset(0, 3000, null, $Decider); Always return : PHP Fatal error: No matching function for overloaded 'Enquire_get_mset' in xapian.php on line 1082 I saw that SWIG was not supporting Directors for PHP, but now, it seems to be patched. Why is it still not working ? Do you have a work arround ?...

...t afterwards +#include <iostream> +#include <fstream> + #include <config.h> #include "xapian/enquire.h" @@ -650,7 +654,16 @@ const MatchDecider *mdecider, const MatchDecider *matchspy_legacy) const { - LOGCALL(MATCH, MSet, "Enquire::Internal::get_mset", first | maxitems | check_at_least | rset | mdecider | matchspy_legacy); + /* + cout << "In Enquire::Internal::get_mset" << endl; + // For now usind DB's UUID to record logs + string logname = "../logs/" + db.get_uuid(); + cout << "Log file: &quo...

My code is not on Github. I am using the tarball as of now. The following it the error that occurred: http://pastebin.com/cVJrjUZX On Sun, Apr 13, 2014 at 8:16 PM, James Aylett <james-xapian at tartarus.org>wrote: > On 13 Apr 2014, at 15:37, Pallavi Gudipati <pallavigudipati at gmail.com> > wrote: > > > A linker error is encountered even after following the above

Greets, Is it possible to obtain the actual match count if you're using set_collapse_key()? ie, the total count *before* the collapsing occurs (without using get_mset()). Alternatively, will MSet::get_matches_estimated() return the true - pre-collapse - count, or will it also be affected by collapsing? Thanks Henry

Hello I defined a weighting scheme to simulate a king of "euclidean" distance. To test it, i used a database with 1000 documents. If I run : enquire.set_weighting_scheme(MyWeight()); Xapian::MSet matches = enquire.get_mset(0, 1000); I have a correct list of results. But if I run Xapian::MSet matches = enquire.get_mset(0, 10); I don't have the top-10 results. If I run Xapian::MSet matches = enquire.get_mset(0, 20); I d'ont have the top-20 results and it's even not the same order than the previous top-10 (...

...formance of searching was not that fast. e.g. Searching a query, which includes about 20 terms, needs 2 secs avg. For searching, I followed such steps: 1. construct a QueryParser for certain string 2. parse the query to get a Xapian::Query 3. construct an Enquire for searching by calling get_mset method here is the function-time-cost for searching: samples % symbol name 75649 28.0401 ChertPostList::move_forward_in_chunk_to_at_least(unsigned int) 30118 11.1635 Xapian::BM25Weight::get_sumpart(unsigned int, unsigned int) const 21291 7.8917 AndMaybePostList::process_next_...

...gIterator::Internal*) (andnotpostlist.cc:50) ==11180== by 0x5F4E317: next_helper (multiandpostlist.h:76) ==11180== by 0x5F4E317: MultiAndPostList::next(double) (multiandpostlist.cc:238) ==11180== by 0x5F4FACC: next_handling_prune (branchpostlist.h:85) ==11180== by 0x5F4FACC: MultiMatch::get_mset(unsigned int, unsigned int, unsigned int, Xapian::MSet&, Xapian::Weight::Internal&, Xapian::MatchDecider const*, Xapian::KeyMaker const*) (multimatch.cc:570) ==11180== by 0x5E485CE: Xapian::Enquire::Internal::get_mset(unsigned int, unsigned int, unsigned int, Xapian::RSet const*, Xapian:...

...s". For the same query, the number of results will be estimated to 20000+ when results are first sorted by date and time with set_sort_by_value_then_relevance(4) and to only 100 if I use set_sort_by_relevance(). The first figure is the correct one. Note that the MSet is obtained with Enquire::get_mset(0, 100, 101), so that probably explains where the 100 comes from. The estimate will also be correct with set_sort_by_relevance_then_value(4). If I am not mistaken, a similar problem was reported, and apparently fixed, back in September : http://comments.gmane.org/gmane.comp.search.xapian.general/...

I'm already retrying the ->get_mset operations; but now I'm wondering where I'd hit DatabaseModifiedErrors while inside a Xapian::MSetIterator loop. I assume ->get_document is a place where it gets thrown; but once a document is retrieved, can iterating through terms in one document (using TermIterator) also throw DB modi...

...d why these happen, but is it possible to ignore this error and execute the search against the stale revision? It's likely only a few seconds out of date, and if I follow the advice and reopen() the database, it's quite possible that it will have been modified again before I manage to call get_mset. Andrew

Hello, I've installed xapian-core 1.1.3 and xapian-bindings 1.1.4 from the tarballs announced by Olly the other day. With these versions, Enquire.get_mset() seems to consistently be raising xapian.DocNotFoundError. I've attached a small test case which reproduces this. The same test case works fine with 1.0.16 (not the latest 1.0.x, but it's what I had installed). Program output with 1.1.3 (core) / 1.1.4 (bindings): tack a...

Hello, As I know, get_matches_estimated() return an estimate for the number of documents with matches the query. But now, I found it get a disparity between the return value and real mathced number. For an example: the real matched number is 58, but the return value is 458; so when the users click the hinder page, get a blank page ... so they often complain to me. I found that the main reason is

...ser to use a self coded scheme) for ranking the terms . I read up the code for the proxy,internal and iterator classes of Eset and Mset to get a feel of how those sets work.I then traced the working of Enquire::get_eset( ) (understood it well other than how a Termlist tree is built ) and Enquire::get_mset( ) (didn't understand this one completely,got lost during Multimatch::get_mset()) .I also read up the code for Xapian::Weight (both proxy and internal class) and the codes of BM25 and TradWeight classes . The hack now seems fairly straightforward as the only difference between BM25 and TradWei...

Hi, I have previously had a problem with getting this error on a get_mset call, and solved it by subclassing XapianEnquire with a backoff-and-retry algorithm (as suggested by this list, many thanks!). However, I now get it intermittently when calling get_data on a XapianDocument. The same solution doesn't seem to be quite as easy in this case, because: 1. The docu...