Displaying 1 result from an estimated 1 matches for "omdocumentterm".
2014 Mar 28
2
Reducing Xapian memory usage
Hey guys
I noticed xapian using a lot of memory while indexing [1] so I decided to look
at the bottle necks and where this can be improved.
Here are some large spots that I noticed (Chert) -
1. Every document has map<string, OmDocumentTerm> and OmDocumentTerm contains
the same string again. This results in every term being stored in memory
twice. Additionally multiple documents may have the same terms, and each of
them would have their own copies to the string, even if the term is the same.
2. Spelling db - It too allocates st...