Displaying 4 results from an estimated 4 matches for "enq_descend".
2018 Mar 30
2
sorting large msets
Hello, is there a way to optimize sorting by certain values
for queries which return a huge amount of results?
For example, I just want a simple query that gives me the 200
most recent emails out of millions. The elapsed time for
get_mset increases as the number of documents ($n * 2000)
increases.
I suppose I could store a pre-sorted set using SQLite or
similar. Thanks in advance for any
2018 Mar 31
2
sorting large msets
...t; If you do want faster, the simplest solution is to arrange that the
> document id order matches the document age order, and then you can
> specify to just sort by that:
>
> $enquire->set_weighting_scheme(new Xapian::BoolWeight());
> $enquire->set_docid_order(Search::Xapian::ENQ_DESCENDING);
That would be tricky with emails being delivered out-of-order;
not to mention old archives being imported + indexed.
> That's more like 0.053 seconds for 1.4.5 and 0.021 seconds for git
> master with glass.
>
> The reverse order (ENQ_ASCENDING) is really fast - about 0.0001...
2018 Mar 30
0
sorting large msets
...the latest 1.4.x release.
If you do want faster, the simplest solution is to arrange that the
document id order matches the document age order, and then you can
specify to just sort by that:
$enquire->set_weighting_scheme(new Xapian::BoolWeight());
$enquire->set_docid_order(Search::Xapian::ENQ_DESCENDING);
That's more like 0.053 seconds for 1.4.5 and 0.021 seconds for git
master with glass.
The reverse order (ENQ_ASCENDING) is really fast - about 0.0001 seconds.
This is because in that case we can just stop once we've found 200
matches.
Cheers,
Olly
2018 Apr 03
0
sorting large msets
...the simplest solution is to arrange that the
> > document id order matches the document age order, and then you can
> > specify to just sort by that:
> >
> > $enquire->set_weighting_scheme(new Xapian::BoolWeight());
> > $enquire->set_docid_order(Search::Xapian::ENQ_DESCENDING);
>
> That would be tricky with emails being delivered out-of-order;
> not to mention old archives being imported + indexed.
This was the trick we used with search.gmane.org. We mostly ignored
the archive import issue there, which wasn't ideal but a periodic
reindex cleaned it up...