Pkg xen devel - Nov 2014 - Bug#767295: Bug#767295: Bug#767295: xl: apparent memory leak

On Fri, 2014-11-07 at 21:32 -0500, Gedalya wrote:
> On 11/06/2014 09:12 AM, Ian Campbell wrote:
> > I've posted a fix for this upstream:
> > http://lists.xen.org/archives/html/xen-devel/2014-11/msg00542.html
> >
> > I've requested that it go into the next 4.4.x release. I also plan
to
> > backport it to the package regardless once it is accepted into the dev
> > branch.
> >
> > Ian.
> 
> So I've tried building xen 4.4.1-3 with this patch (attached), and I 
> replaced the binary packages libxen-4.4 and xen-utils-4.4 with the new
ones.
> The situation is the same: the xl process for jessie amd64 guests starts 
> off at a resident size of 15mb as opposed to ~500kb for wheezy i386 
> guests. At guest reboot it jumps to 17mb and then, after a few seconds, 
> to ~34mb.
> I also rebooted the host to help make sure the test is valid.
Please can you try running xl under valgrind, something similar to what
I described earlier should work.

Could you also post your guest cfg file please.

Thanks,
Ian.

On 11/08/2014 05:41 AM, Ian Campbell wrote:
> Please can you try running xl under valgrind, something similar to what
> I described earlier should work.
I guess it didn't find much..

# valgrind --leak-check=full xl cr -F /etc/xen/auto/asterisk_deb80.cfg
==6736== Memcheck, a memory error detector
==6736== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==6736== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
==6736== Command: /usr/sbin/xl cr -F /etc/xen/auto/asterisk_deb80.cfg
==6736===6740===6740== HEAP SUMMARY:
==6740==     in use at exit: 11,663 bytes in 40 blocks
==6740==   total heap usage: 74 allocs, 34 frees, 44,570 bytes allocated
==6740===6740== LEAK SUMMARY:
==6740==    definitely lost: 0 bytes in 0 blocks
==6740==    indirectly lost: 0 bytes in 0 blocks
==6740==      possibly lost: 0 bytes in 0 blocks
==6740==    still reachable: 11,663 bytes in 40 blocks
==6740==         suppressed: 0 bytes in 0 blocks
==6740== Reachable blocks (those to which a pointer was found) are not 
shown.
==6740== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==6740===6740== For counts of detected and suppressed errors, rerun with: -v
==6740== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==6739===6739== HEAP SUMMARY:
==6739==     in use at exit: 10,711 bytes in 35 blocks
==6739==   total heap usage: 63 allocs, 28 frees, 35,257 bytes allocated
==6739===6739== LEAK SUMMARY:
==6739==    definitely lost: 0 bytes in 0 blocks
==6739==    indirectly lost: 0 bytes in 0 blocks
==6739==      possibly lost: 0 bytes in 0 blocks
==6739==    still reachable: 10,711 bytes in 35 blocks
==6739==         suppressed: 0 bytes in 0 blocks
==6739== Reachable blocks (those to which a pointer was found) are not 
shown.
==6739== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==6739===6739== For counts of detected and suppressed errors, rerun with: -v
==6739== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==6744===6744== HEAP SUMMARY:
==6744==     in use at exit: 4,827 bytes in 42 blocks
==6744==   total heap usage: 91 allocs, 49 frees, 38,343 bytes allocated
==6744===6744== LEAK SUMMARY:
==6744==    definitely lost: 0 bytes in 0 blocks
==6744==    indirectly lost: 0 bytes in 0 blocks
==6744==      possibly lost: 0 bytes in 0 blocks
==6744==    still reachable: 4,827 bytes in 42 blocks
==6744==         suppressed: 0 bytes in 0 blocks
==6744== Reachable blocks (those to which a pointer was found) are not 
shown.
==6744== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==6744===6744== For counts of detected and suppressed errors, rerun with: -v
==6744== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==6745===6745== HEAP SUMMARY:
==6745==     in use at exit: 13,034 bytes in 44 blocks
==6745==   total heap usage: 97 allocs, 53 frees, 48,672 bytes allocated
==6745===6745== LEAK SUMMARY:
==6745==    definitely lost: 0 bytes in 0 blocks
==6745==    indirectly lost: 0 bytes in 0 blocks
==6745==      possibly lost: 0 bytes in 0 blocks
==6745==    still reachable: 13,034 bytes in 44 blocks
==6745==         suppressed: 0 bytes in 0 blocks
==6745== Reachable blocks (those to which a pointer was found) are not 
shown.
==6745== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==6745===6745== For counts of detected and suppressed errors, rerun with: -v
==6745== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==6738===6738== HEAP SUMMARY:
==6738==     in use at exit: 11,017 bytes in 41 blocks
==6738==   total heap usage: 91 allocs, 50 frees, 48,555 bytes allocated
==6738===6738== LEAK SUMMARY:
==6738==    definitely lost: 0 bytes in 0 blocks
==6738==    indirectly lost: 0 bytes in 0 blocks
==6738==      possibly lost: 0 bytes in 0 blocks
==6738==    still reachable: 11,017 bytes in 41 blocks
==6738==         suppressed: 0 bytes in 0 blocks
==6738== Reachable blocks (those to which a pointer was found) are not 
shown.
==6738== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==6738===6738== For counts of detected and suppressed errors, rerun with: -v
==6738== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Parsing config from /etc/xen/auto/asterisk_deb80.cfg
Waiting for domain asterisk_deb08 (domid 8) to die [pid 6736]
Domain 8 has shut down, reason code 1 0x1
Action for shutdown reason code 1 is restart
Domain 8 needs to be cleaned up: destroying the domain
Done. Rebooting now
Waiting for domain asterisk_deb08 (domid 9) to die [pid 6736]
Domain 9 has shut down, reason code 0 0x0
Action for shutdown reason code 0 is destroy
Domain 9 needs to be cleaned up: destroying the domain
Done. Exiting now

> Could you also post your guest cfg file please.
bootloader="pygrub"
memory = 1024
name = "asterisk_deb08"
vcpus = 2
pvh = 1
vif = ['mac=00:16:3e:00:00:12, bridge=breth1',
'mac=00:16:3e:00:00:13,
bridge=breth0']
disk = ['phy:/dev/rvg0/asterisk_deb80_rootfs,xvda,w',
     'phy:/dev/rvg0/asterisk_deb80_var,xvdb,w',
     'phy:/dev/rvg0/asterisk_deb80_rec,xvdc,w',
     'phy:/dev/rvg0/asterisk_deb80_swap,xvdd,w']

(The pvh part doesn't seem to make a difference)

On Sat, 2014-11-08 at 08:32 -0500, Gedalya wrote:
> On 11/08/2014 05:41 AM, Ian Campbell wrote:
> > Please can you try running xl under valgrind, something similar to
what
> > I described earlier should work.
> I guess it didn't find much..
Indeed not :-/

I managed to get a test box up and running, so I backported a couple of
additional upstream patches to help valgrind analyse Xen toolstacks and
even with those I'm not seeing leaks.

However with pmap I am seeing an additional

+00007f37f516a000   23852   14464   14464 rw---   [ anon ]

with each reboot. The size seems  to be the same each time. I can't
quite figure out what this new mapping is. I'll keep digging.

[...]
> > Could you also post your guest cfg file please.
> 
Thanks, nothing unusual here but worth confirming.

Ian.

[0] b1dfc531d8ad and 171c6d7ac17e, FWIW