Displaying 2 results from an estimated 2 matches for "pack_uint_preserving_sort".
2018 Jan 03
2
Storing the documents text: data record or value ?
Hi,
Following the Recoll snippets generation performance problem caused by the
new positions list storage scheme in Xapian 1.4, I am experimenting with
generating snippets from the complete document text stored in the index.
This increases the index size much less than I would have expected (around
10-15% apparently with my home directory data), which is good news
obviously.
I have tried
2018 Jan 04
0
Storing the documents text: data record or value ?
...uld mean more pointless overhead for your case.
If you want to store the document text separately, I'd put it in the
user metadata (build a key from the docid, ideally one which sorts in
the same order as the integer docids do so that append works very
efficiently - you could copy Xapian's pack_uint_preserving_sort() for
that).
You'll want to compress the document text yourself (currently at least,
though I wonder if we should support transparent compression of user
metadata entries - mostly they aren't compressed because they're stored
in the postlist table which doesn't have transparent com...