search for: mod_plists

...ex, the Omindex continues to consume the memory and never to release till it stops which cause too much memory is occupied. check the source code: follwing codes are found. // FIXME: this should be done by checking memory usage, not the number of // changes. // We could also look at: // * mod_plists.size() // * doclens.size() // * freq_deltas.size() // // cout << "+++ mod_plists.size() " << mod_plists.size() << // ", doclens.size() " << doclens.size() << // ", freq_deltas.size() " << freq_deltas.size()...

Hey guys I noticed xapian using a lot of memory while indexing [1] so I decided to look at the bottle necks and where this can be improved. Here are some large spots that I noticed (Chert) - 1. Every document has map<string, OmDocumentTerm> and OmDocumentTerm contains the same string again. This results in every term being stored in memory twice. Additionally multiple documents may

...uartzBufferedTable class which adds memory buffering of changes to this: http://www.xapian.org/docs/sourcedoc/html/classQuartzBufferedTable.html However, as of 0.8.0 we now buffer changes to the posting lists in QuartzWritableDatabase (in the "Private attributes" totlen_added through to mod_plists): http://www.xapian.org/docs/sourcedoc/html/classQuartzWritableDatabase.html This probably removes the main advantage of having QuartzBufferedTable. If we're adding new documents to a database, it probably doesn't help us at all. It probably still helps efficiency a little for scattered...

There is is bug when setting XAPIAN_FLUSH_THRESHOLD=20000000 When trying for force Xapian flush documents to flush after 20 million documents Xapian ignores the size and flush it after only 10,000 documents. Data captured from delve after 60 seconds interval when has been set as follow: XAPIAN_FLUSH_THRESHOLD=20000000 perl -e ' while(1) { system("delve ."); sleep(60); } '