similar to: does Xapian::Enquire hold an MVCC revision?

On Thu, Aug 17, 2023 at 09:28:26PM +0000, Eric Wong wrote: > In other words, is it possible to avoid duplicates if new > documents are inserted into the DB by another process in-between > ->get_mset calls when reusing Xapian::Enquire objects? The Database object itself effectively does (it works in a snapshot of the state of the database when you open it, or last called reopen() which

Olly Betts <olly at survex.com> wrote: > On Wed, Aug 23, 2023 at 01:53:27PM +0000, Eric Wong wrote: > > I'm already retrying the ->get_mset operations; but now I'm > > wondering where I'd hit DatabaseModifiedErrors while inside a > > Xapian::MSetIterator loop. > > > > I assume ->get_document is a place where it gets thrown; > > but

Hello, This is a somewhat idle question about Enquire::get_mset(), possibly based on wrong assumptions on how Xapian works, but I would be grateful if someone satisfied my curiosity. Xapian::MSet Xapian::Enquire::get_mset(Xapian::doccount first, Xapian::doccount maxitems, const Xapian::RSet * omrset = 0, const

Olly Betts <olly at survex.com> wrote: > On Fri, Aug 18, 2023 at 10:41:52AM +0000, Eric Wong wrote: > > Olly Betts <olly at survex.com> wrote: > > > While the match is running, get_mset(2000, 1000) needs to track > > > 3000 entries so this won't reduce your heap usage (at least not > > > peak usage). > > > > > > Is the heap

my $mset = $enq->get_mset($nstart,$nrecords); for(my $mit=$mset->begin(); $mit != $mset->end();$mit++) { my $doc = $mit->get_document(); my $dat = $doc->get_data(); my $id = $doc->get_docid(); } [Fri Jan 20 10:35:06 2012] newmail.cgi: Can't locate auto/Search/Xapian/Document/get_docid.al in @INC (@INC contains: /etc/perl

On Wed, Aug 23, 2023 at 01:53:27PM +0000, Eric Wong wrote: > I'm already retrying the ->get_mset operations; but now I'm > wondering where I'd hit DatabaseModifiedErrors while inside a > Xapian::MSetIterator loop. > > I assume ->get_document is a place where it gets thrown; > but once a document is retrieved, can iterating through > terms in one document

I am suffering for slow searching performance on Xapian. I am using Xapian for indexing about 150,000,000 documents. It was implemented in C++; The performance of searching was not that fast. e.g. Searching a query, which includes about 20 terms, needs 2 secs avg. For searching, I followed such steps: 1. construct a QueryParser for certain string 2. parse the query to get a Xapian::Query

Hi all, I found that the figure returned by MSet::get_matches_estimated() varies depending on how results are to be sorted. For instance, in my index, value 4 contains date and time in the format "yyyymmddhhmmss". For the same query, the number of results will be estimated to 20000+ when results are first sorted by date and time with set_sort_by_value_then_relevance(4) and to only 100

Hello, is there a way to optimize sorting by certain values for queries which return a huge amount of results? For example, I just want a simple query that gives me the 200 most recent emails out of millions. The elapsed time for get_mset increases as the number of documents ($n * 2000) increases. I suppose I could store a pre-sorted set using SQLite or similar. Thanks in advance for any

At least it isn't working the way it used to. Code: $db = Search::Xapian::Database->new( $dx ); my $qp = Search::Xapian::QueryParser->new(); my $dbSize=$db->get_doccount(); # $qp->set_stemmer(new Search::Xapian::Stem("english")); # $qp->set_stemming_strategy(STEM_SOME); # $qp->set_default_op($defaultop); my $par =

On 08/25/2013 05:02 PM, Olly Betts wrote: > So the simple fix is > probably just to install the perl-Search-Xapian RPM instead. Thanks, the Centos 6 repos don't have that rpm and the http://xapian.org/download page seems to only cover the XS bindings, if I am reading this correctly: But I was able to remove the rpm packages and compile and install the core and swig from source.

i use the perl interface of Search::Xapian to index documents, now i got metadata i store with the index like title, date, author, .. and i wonder how to retrieve them from the index again without pulling them from the database. i am pretty sure this is a stupid question and that the answer is obvious i dont seem to be able to find it. regards m

I got some performance issue for document.get_data() and enquire.get_mset(). It costs 35 seconds for matches = enquire.get_mset(0,200), and 3 seconds for iterating all doc in matches to get_data. Is't normal? My index contains 30millions documents. I use python binding to operate xapian. Bellow it's my index structure # value: 0:date, 1:site # data: json message which contains: author,

Hello I defined a weighting scheme to simulate a king of "euclidean" distance. To test it, i used a database with 1000 documents. If I run : enquire.set_weighting_scheme(MyWeight()); Xapian::MSet matches = enquire.get_mset(0, 1000); I have a correct list of results. But if I run Xapian::MSet matches = enquire.get_mset(0, 10); I don't have the top-10 results. If I run Xapian::MSet

My code is not on Github. I am using the tarball as of now. The following it the error that occurred: http://pastebin.com/cVJrjUZX On Sun, Apr 13, 2014 at 8:16 PM, James Aylett <james-xapian at tartarus.org>wrote: > On 13 Apr 2014, at 15:37, Pallavi Gudipati <pallavigudipati at gmail.com> > wrote: > > > A linker error is encountered even after following the above

I'm trying to compact (or at least merge) multiple databases, while stripping search records which are no longer required. Backstory: I've inherited the Cyrus IMAPd xapian-based search code from Greg Banks when he left Opera. One of the unfinished parts was removing expunged emails from the search database. We moved from having a single search database to supporting multiple

We are using the --enable-maintainer-mode and will move to git soon. The diff file is attached. *Siddhant Mutha* Undergraduate Student Department of Computer Science and Engineering IIT Madras Chennai http://www.siddhantmutha.com/ <http:/www.siddhantmutha.com/> On Sun, Apr 13, 2014 at 8:26 PM, James Aylett <james-xapian at tartarus.org>wrote: > On 13 Apr 2014, at 15:48, Pallavi

Hello list, looks like one can open a Xapian database in read-only mode and do the following: $db = Search::Xapian::Database->new("/foo/bar/"); $enq = $db->enquire("XIDblub"); the same doesn't seem to be possible with a database opened in read-write mode: $db = Search::Xapian::WritableDatabase->new("/foo/bar/",

Have been working recently on a site that classifies posts using tags in taxonomies, so a post about the Oil spill in the Gulf of Mexico might be tagged 'Subscribers only' (access level), 'Barack Obama' (person), 'Tony Hayward' (person), 'BP' (company), 'Transocean' (company), 'Gulf of Mexico' (location). With some advice from Richard Boulton I

Hi there, I have a xapian index whose results are being sorted by a value, with (PHP bindings): $enquire->set_sort_by_value($sort_data_value); This is because I want the results returned in chronological order of publication date. However, I now have a need to have certain results be 'sticky' at the top of the resultset, regardless of their publication date. Obviously there are