search for: mod_plist

...ex, the Omindex continues to consume the memory and never to release till it stops which cause too much memory is occupied. check the source code: follwing codes are found. // FIXME: this should be done by checking memory usage, not the number of // changes. // We could also look at: // * mod_plists.size() // * doclens.size() // * freq_deltas.size() // // cout << "+++ mod_plists.size() " << mod_plists.size() << // ", doclens.size() " << doclens.size() << // ", freq_deltas.size() " << freq_deltas.size()...

...ns the same string again. This results in every term being stored in memory twice. Additionally multiple documents may have the same terms, and each of them would have their own copies to the string, even if the term is the same. 2. Spelling db - It too allocates std::strings again 3. database mod_plist - ditto 4. When fetching the terms for a document the entire term list is loaded in one go. This causes a huge block on memory to be loaded. Depending on the number of terms in a document, it can get quite bad. We might want to do this in smaller chunks. -- The main way that I think this can...

...uartzBufferedTable class which adds memory buffering of changes to this: http://www.xapian.org/docs/sourcedoc/html/classQuartzBufferedTable.html However, as of 0.8.0 we now buffer changes to the posting lists in QuartzWritableDatabase (in the "Private attributes" totlen_added through to mod_plists): http://www.xapian.org/docs/sourcedoc/html/classQuartzWritableDatabase.html This probably removes the main advantage of having QuartzBufferedTable. If we're adding new documents to a database, it probably doesn't help us at all. It probably still helps efficiency a little for scattered...

There is is bug when setting XAPIAN_FLUSH_THRESHOLD=20000000 When trying for force Xapian flush documents to flush after 20 million documents Xapian ignores the size and flush it after only 10,000 documents. Data captured from delve after 60 seconds interval when has been set as follow: XAPIAN_FLUSH_THRESHOLD=20000000 perl -e ' while(1) { system("delve ."); sleep(60); } '