search for: mset

Displaying 20 results from an estimated 129 matches for "mset".

Did you mean: meet
2023 Aug 28
1
DatabaseModifiedError while iterating on mset
Olly Betts <olly at survex.com> wrote: > On Wed, Aug 23, 2023 at 01:53:27PM +0000, Eric Wong wrote: > > I'm already retrying the ->get_mset operations; but now I'm > > wondering where I'd hit DatabaseModifiedErrors while inside a > > Xapian::MSetIterator loop. > > > > I assume ->get_document is a place where it gets thrown; > > but once a document is retrieved, can iterating through > > t...
2016 Jul 30
2
Letor: returning MSet after re-ranking
...ic API that don't get > used by end users. However because LTR is outside the Xapian build > tree, we can't easily give it privileged access to Xapian internals. > Sorry for a delayed response. The way I was thinking of performing reranking with updated weights was to add a class MSetRanker (basically containing a copy of MSet to be re-ranked) to Xapian public API, which would be a friend of MSet::Internal class so that it could access/update the weights and sort MSetItems accordingly, returning an updated MSet. As you pointed out, two obvious issues with this are: 1) Unnecessa...
2023 Aug 27
1
DatabaseModifiedError while iterating on mset
On Wed, Aug 23, 2023 at 01:53:27PM +0000, Eric Wong wrote: > I'm already retrying the ->get_mset operations; but now I'm > wondering where I'd hit DatabaseModifiedErrors while inside a > Xapian::MSetIterator loop. > > I assume ->get_document is a place where it gets thrown; > but once a document is retrieved, can iterating through > terms in one document (using Te...
2023 Aug 18
1
does Xapian::Enquire hold an MVCC revision?
On Thu, Aug 17, 2023 at 09:28:26PM +0000, Eric Wong wrote: > In other words, is it possible to avoid duplicates if new > documents are inserted into the DB by another process in-between > ->get_mset calls when reusing Xapian::Enquire objects? The Database object itself effectively does (it works in a snapshot of the state of the database when you open it, or last called reopen() which updates that snapshot to what's currently committed). However we don't currently have any locking of...
2023 Aug 17
1
does Xapian::Enquire hold an MVCC revision?
In other words, is it possible to avoid duplicates if new documents are inserted into the DB by another process in-between ->get_mset calls when reusing Xapian::Enquire objects? I do some expensive processing on each mset window, so I always limit the results to limit heap usage even if I'm planning on going through a big chunk of the DB: $mset = $enq->get_mset(0, 1000); do_something_slow_with_mset($mset); $mset = $en...
2016 Jul 31
2
Letor: returning MSet after re-ranking
On Sun, Jul 31, 2016 at 12:44:16AM +0100, Olly Betts wrote: > Would a method which swapped two elements of an MSet provide what you > need? That would provide a more generic way to adjust the ranking of > an MSet which for example could be used to implement a diversification > feature or something like SQL "GROUP BY". Isn't the most common use going to be that the client (letor or whate...
2016 Jul 29
3
Pull requests: CJK words and Snippet generator
...:) I've just opened a pull request for the CJK tokenizer: https://github.com/xapian/xapian/pull/114 > I wonder if we can arrange suitable defaults to use your > implementation with the older API, and come up with a newer API that > allows a SnippetGenerator class to be used from the MSet. The FastMail snippet generator has been written when MSet didn't create snippets. I'll first compare both implementations to see if there is a good reason for them to coexist, or might just as well merge any additional features into MSet. > A good start would certainly be rebasing aga...
2011 Mar 08
1
MSet order
Hello I defined a weighting scheme to simulate a king of "euclidean" distance. To test it, i used a database with 1000 documents. If I run : enquire.set_weighting_scheme(MyWeight()); Xapian::MSet matches = enquire.get_mset(0, 1000); I have a correct list of results. But if I run Xapian::MSet matches = enquire.get_mset(0, 10); I don't have the top-10 results. If I run Xapian::MSet matches = enquire.get_mset(0, 20); I d'ont have the top-20 results and it's even not the same order...
2018 Jul 02
0
Incomplete HTML escaping by Xapian::MSet::snippet() (CVE-2018-0499)
Hi folks, I spotted an HTML escaping bug in Xapian::MSet::snippet() while working on the code. This issue has been assigned CVE-2018-0499 (though currently there's no useful information on cve.mitre.org for it). I've added a wiki page for it here: https://trac.xapian.org/wiki/SecurityFixes/2018-07-02 The intended behaviour is that the selecte...
2018 Apr 03
0
sorting large msets
...glass. > > > > The reverse order (ENQ_ASCENDING) is really fast - about 0.0001 seconds. > > This is because in that case we can just stop once we've found 200 > > matches. > > So that sounds like it's O(1) and independent of how many > documents are in the mset? It won't be independent of the number of documents in the MSet - something needs to be done for each document added to the MSet, so clearly it'll be at least O(n) in that, but if n is fixed at 200 then the O() complexity isn't relevant. It'll be fairly independent of the total nu...
2010 Jan 09
1
Using HASH() and REALTIME_HASH()
...ASH(call_info,exten,${dest})}); the syntax doesn't seem to be happy: -- Executing [s at appel_deb:8] Set("SIP/maverick-00000000", "HASH(info)=,101,maverick,0,0,max,0,0,123456,123654") in new stack [Jan 9 22:07:25] WARNING[27801]: pbx.c:9107 pbx_builtin_setvar_multiple: MSet: ignoring entry '101' with no '=' (in s at appel_deb:8 [Jan 9 22:07:25] WARNING[27801]: pbx.c:9107 pbx_builtin_setvar_multiple: MSet: ignoring entry 'maverick' with no '=' (in s at appel_deb:8 [Jan 9 22:07:25] WARNING[27801]: pbx.c:9107 pbx_builtin_setvar_multiple:...
2014 May 21
2
Some questions about Letor project
...o parts of functions: the training part and the ranking part. I?ll use SVMRanker as an example. The training part basically uses the libsvm library and training data to train a model, then save the model file. The ranking part will calculate score for each document in searching results (MSet) by using the trained model file. My question is for each of our three rankers: 1) SVMRanker 2) ListMLE 3) ListNet, do we need three different types of training part? (The ranking part for each of those have the same form I think) I?m not sure the parameters for these three different ra...
2018 Mar 31
2
sorting large msets
...1 seconds for git > master with glass. > > The reverse order (ENQ_ASCENDING) is really fast - about 0.0001 seconds. > This is because in that case we can just stop once we've found 200 > matches. So that sounds like it's O(1) and independent of how many documents are in the mset? Would it be possible to teach Xapian to optimize its storage for certain queries so it can stop once it's found 200 matches? >From what I recall, SQL implementations are pretty good at that.
2009 Sep 16
3
Music on Hold
...ars silence. Here is the full debug output (asterisk -rvvvvv): == Using SIP RTP CoS mark 5 -- Executing [xxxxxxx at phones:1] Goto("SIP/ATA-xxxxxxxxxx-L1-024b6d88", "1xxxxxxxxxx,1") in new stack -- Goto (phones,1xxxxxxxxxx,1) -- Executing [1xxxxxxxxxx at phones:1] MSet("SIP/ATA-xxxxxxxxxx-L1-024b6d88", "oldcidnum=0") in new stack -- Executing [1xxxxxxxxxx at phones:2] MSet("SIP/ATA-xxxxxxxxxx-L1-024b6d88", "CALLERID(name)=""") in new stack -- Executing [1xxxxxxxxxx at phones:3] MSet("SIP/ATA-xxxxxxxxx...
2018 Nov 11
2
Fwd: GoXapian
On Mon, Nov 05, 2018 at 09:04:23PM +0000, Olly Betts wrote: > Dave Courtois wrote: > > Hi there just a little word to tell you that Xapian 1.4.6 work in > > Golang 1.9. There is some issue with the MSet ESet, the namespace part > > of the name is drop by swig. There is also some error with define > > macro in .i files, but with minor change all work perfectly. I had a look and I think that at least the macro changes are only needed because the golang branch hasn't been updated for...
2018 Mar 30
2
sorting large msets
Hello, is there a way to optimize sorting by certain values for queries which return a huge amount of results? For example, I just want a simple query that gives me the 200 most recent emails out of millions. The elapsed time for get_mset increases as the number of documents ($n * 2000) increases. I suppose I could store a pre-sorted set using SQLite or similar. Thanks in advance for any advice/help you can provide. -----------8<-------- #!/usr/bin/perl -w use strict; use warnings; use Search::Xapian::Document; use Search::Xap...
2023 Aug 23
1
DatabaseModifiedError while iterating on mset
I'm already retrying the ->get_mset operations; but now I'm wondering where I'd hit DatabaseModifiedErrors while inside a Xapian::MSetIterator loop. I assume ->get_document is a place where it gets thrown; but once a document is retrieved, can iterating through terms in one document (using TermIterator) also throw DB modi...
2018 Mar 30
0
sorting large msets
On Fri, Mar 30, 2018 at 05:21:43PM +0000, Eric Wong wrote: > Hello, is there a way to optimize sorting by certain values > for queries which return a huge amount of results? [...] > $enquire->set_sort_by_value_then_relevance(0, 1); If you're just wanting the 200 newest, it'll be faster not to calculate weights, so: $enquire->set_sort_by_value(0, 1);
2018 Apr 06
1
sorting large msets
> > Olly Betts <olly at survex.com> wrote: > > > > > > The reverse order (ENQ_ASCENDING) is really fast - about 0.0001 seconds. > > > This is because in that case we can just stop once we've found 200 > > > matches. With a few million documents, that ENQ_ASCENDING sounds promising :) So, it looks like if I had ideal ordering, I could do
2009 Feb 12
1
problem when using xapian's static libs in windows
...mp;)" (??4Document at Xapian@@QAEXABV01@@Z)??????? "public: void __thiscall Xapian::ValueIterator::operator=(class Xapian::ValueIterator const &)" (??4ValueIterator at Xapian@@QAEXABV01@@Z) ???? libnet.lib(serialise.obj) : error LNK2019: ????????? "public: __thiscall Xapian::MSet::~MSet(void)" (??1MSet at Xapian@@QAE at XZ)??????? "public: __thiscall Xapian::MSetIterator::~MSetIterator(void)" (??1MSetIterator at Xapian@@QAE at XZ) ???? libnet.lib(serialise.obj) : error LNK2019: ????????? "public: char const * __thiscall Xapian::Error::get_error_string(vo...