Xapian discuss - Aug 2013 - Potential memory leak when assigning MSetItem values

Olly, the process size does stay constant with the results from one query
set, but running other queries will cause it to grow (once). Is it possible
that this is a bug with the SWIG python bindings?


On Fri, Jul 5, 2013 at 6:18 AM, Olly Betts <olly at survex.com> wrote:
> On Wed, Jul 03, 2013 at 03:59:21PM -0400, Jeff Rand wrote:
> > Run the following code and monitor the PID's memory usage in top
or a
> > similar program. I've observed the resident memory for this
example go
> from
> > 18m to 52m after deleting objects and running garbage collection.
>
> If I set it to repeat the call to run_query 10 times, the memory usage
> doesn't keep growing, so it looks to me like the heap of the process
> has just grown, and doesn't get returned to the OS again.
>
> Certainly the number of objects Python knows about is constant (add a
> call to print len(gc.get_objects()) after gc.collect() to see that).
>
> I was using Python 2.6.6 and Xapian trunk, as I have those to hand.
> This could be version dependent of course - can you try repeating
> run_query() to see if the process size keeps growing for you?
>
> Cheers,
>     Olly
>

We also met MemoryError with xapian sometimes and we use python too.

This might be a problem with SWIG python binding.


2013/7/10 Jeff Rand <jeffreyrand at gmail.com>
> Olly, the process size does stay constant with the results from one query
> set, but running other queries will cause it to grow (once). Is it possible
> that this is a bug with the SWIG python bindings?
>
>
> On Fri, Jul 5, 2013 at 6:18 AM, Olly Betts <olly at survex.com>
wrote:
>
> > On Wed, Jul 03, 2013 at 03:59:21PM -0400, Jeff Rand wrote:
> > > Run the following code and monitor the PID's memory usage in
top or a
> > > similar program. I've observed the resident memory for this
example go
> > from
> > > 18m to 52m after deleting objects and running garbage collection.
> >
> > If I set it to repeat the call to run_query 10 times, the memory usage
> > doesn't keep growing, so it looks to me like the heap of the
process
> > has just grown, and doesn't get returned to the OS again.
> >
> > Certainly the number of objects Python knows about is constant (add a
> > call to print len(gc.get_objects()) after gc.collect() to see that).
> >
> > I was using Python 2.6.6 and Xapian trunk, as I have those to hand.
> > This could be version dependent of course - can you try repeating
> > run_query() to see if the process size keeps growing for you?
> >
> > Cheers,
> >     Olly
> >
> _______________________________________________
> Xapian-discuss mailing list
> Xapian-discuss at lists.xapian.org
> http://lists.xapian.org/mailman/listinfo/xapian-discuss
>


-- 
???
http://weibo.com/panjunyong

?? ???
http://everydo.com
????OA

I added a ticket for this:

http://trac.xapian.org/ticket/637


On Thu, Aug 8, 2013 at 9:35 PM, ??? <panjunyong at gmail.com> wrote:
> We also met MemoryError with xapian sometimes and we use python too.
>
> This might be a problem with SWIG python binding.
>
>
> 2013/7/10 Jeff Rand <jeffreyrand at gmail.com>
>
>> Olly, the process size does stay constant with the results from one
query
>> set, but running other queries will cause it to grow (once). Is it
>> possible
>> that this is a bug with the SWIG python bindings?
>>
>>
>> On Fri, Jul 5, 2013 at 6:18 AM, Olly Betts <olly at survex.com>
wrote:
>>
>> > On Wed, Jul 03, 2013 at 03:59:21PM -0400, Jeff Rand wrote:
>> > > Run the following code and monitor the PID's memory usage
in top or a
>> > > similar program. I've observed the resident memory for
this example go
>> > from
>> > > 18m to 52m after deleting objects and running garbage
collection.
>> >
>> > If I set it to repeat the call to run_query 10 times, the memory
usage
>> > doesn't keep growing, so it looks to me like the heap of the
process
>> > has just grown, and doesn't get returned to the OS again.
>> >
>> > Certainly the number of objects Python knows about is constant
(add a
>> > call to print len(gc.get_objects()) after gc.collect() to see
that).
>> >
>> > I was using Python 2.6.6 and Xapian trunk, as I have those to
hand.
>> > This could be version dependent of course - can you try repeating
>> > run_query() to see if the process size keeps growing for you?
>> >
>> > Cheers,
>> >     Olly
>> >
>> _______________________________________________
>> Xapian-discuss mailing list
>> Xapian-discuss at lists.xapian.org
>> http://lists.xapian.org/mailman/listinfo/xapian-discuss
>>
>
>
>
> --
> ???
> http://weibo.com/panjunyong
>
> ?? ???
> http://everydo.com
> ????OA
>
>