Xapian discuss - Jan 2011 - Memory leak

Hello,

There is a memory leak in Xapian 1.2.4.
We use a persistant connection in FastCGI processes. As soon
as we catch this exception, "dmalloc" recognizes memory leaks:
     The revision being read has been discarded - you should
     call Xapian::Database::reopen() and retry the operation
Down below the output of "dmalloc".

This happens only on the production system. On my local maschine,
"valgrind" reports "all blocks freed".

Thanks a lot
Marcus


total-size  count  source
     294912     36  Line 64 of "backends/chert/chert_cursor.cc"
     122880     15  Line 2070 of "backends/chert/chert_table.cc"
      26021      5  Line 262 of "backends/chert/chert_btreebase.cc"
      26021      5  Line 263 of "backends/chert/chert_btreebase.cc"
       2056      1  Line 358 of "backends/dbfactory.cc"
       1836     17  Line 383 of "./include/xapian/weight.h"
       1700     17  Line 871 of "backends/chert/chert_database.cc"
        864     18  Line 62 of "backends/chert/chert_cursor.cc"
        780     13  Line 408 of "matcher/queryoptimiser.cc"
        576     18  Line 1331 of "backends/chert/chert_table.cc"
        112      2  Line 1663 of "backends/chert/chert_table.cc"
        100      1  Line 66 of "backends/chert/chert_postlist.cc"
         48      1  Line 66 of "matcher/orpostlist.cc"
         40      1  Line 189 of "matcher/queryoptimiser.cc"
         24      1  Line 33 of "matcher/multiandpostlist.cc"
         15      1  Line 31 of "matcher/multiandpostlist.cc"

On Mon, Jan 24, 2011 at 09:50:34PM +0100, double wrote:
> There is a memory leak in Xapian 1.2.4.
> We use a persistant connection in FastCGI processes. As soon
> as we catch this exception, "dmalloc" recognizes memory leaks:
>     The revision being read has been discarded - you should
>     call Xapian::Database::reopen() and retry the operation
> Down below the output of "dmalloc".
Hmm, I think there's just one leak which drags all the others with it,
but it's hard to tell from that output.

Does this patch fix the problem for you?

http://oligarchy.co.uk/xapian/patches/matcher-pl-leak-fix.patch
> This happens only on the production system. On my local maschine,
> "valgrind" reports "all blocks freed".
It's quite sensitive to the precise situation - we need to successfully
open the postlists, but fail to read a subsequent chunk of one because
it is overwritten.  I've failed to create a testcase for it so far...

Cheers,
    Olly

Hi Olly,

Sorry for the late answer - this issue is definitly
gone. Thanks a lot for fixing.

Greetings
Marcus


Am 2011-02-03 05:09, schrieb Olly Betts:
> If the patch actually fixes a leak you were seeing, then updated
> dmalloc output would also be useful as it should reduce the number
> of allocation sites (valgrind is better though).
>
> Cheers,
>      Olly
>