search for: collapse_count

...match is running, get_mset(2000, 1000) needs to track 3000 entries so this won't reduce your heap usage (at least not peak usage). Is the heap usage problematic? Looking at the code, for git master each entry is currently: double weight; Xapian::docid did; Xapian::doccount collapse_count; std::string collapse_key; std::string sort_key; We're always going to need the docid, but the other fields aren't always needed and this could be slimmed down depending on what options are in use if the size is causing problems. It is as it is just for simplicity really...

In other words, is it possible to avoid duplicates if new documents are inserted into the DB by another process in-between ->get_mset calls when reusing Xapian::Enquire objects? I do some expensive processing on each mset window, so I always limit the results to limit heap usage even if I'm planning on going through a big chunk of the DB: $mset = $enq->get_mset(0, 1000);