Pkg xen devel - Aug 2013 - double free or corruption in xl 4.2

Hi,

I backported the 4.2 packages from sid to wheezy, it works, but sometime
I obtain that kind of error, when trying to start a PV VM :

# xl create /etc/xen/ronflex.cfg
Parsing config from /etc/xen/ronflex.cfg
*** glibc detected *** /usr/lib/xen-4.2/bin/xl: double free or corruption
(!prev): 0x0000000001e6a960 ***
======= Backtrace:
========/lib/x86_64-linux-gnu/libc.so.6(+0x76d76)[0x7f23da771d76]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x6c)[0x7f23da776aac]
/lib/x86_64-linux-gnu/libc.so.6(fclose+0x14d)[0x7f23da762ccd]
/usr/lib/xen-4.2/bin/../lib/libxenlight.so(libxl_userdata_store+0x113)[0x7f23db0f7883]
/usr/lib/xen-4.2/bin/xl[0x40c564]
/usr/lib/xen-4.2/bin/xl[0x41028e]
/usr/lib/xen-4.2/bin/xl[0x406d24]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7f23da719ead]
/usr/lib/xen-4.2/bin/xl[0x406f3d]
======= Memory map: =======00400000-00421000 r-xp 00000000 fe:00 24790          
/usr/lib/xen-4.2/bin/xl
00620000-00623000 rw-p 00020000 fe:00 24790                             
/usr/lib/xen-4.2/bin/xl
01e69000-01ead000 rw-p 00000000 00:00 0                                  [heap]
7f23d4000000-7f23d4021000 rw-p 00000000 00:00 0 
7f23d4021000-7f23d8000000 ---p 00000000 00:00 0 
7f23d9465000-7f23d947a000 r-xp 00000000 fe:00 268                       
/lib/x86_64-linux-gnu/libgcc_s.so.1
7f23d947a000-7f23d967a000 ---p 00015000 fe:00 268                       
/lib/x86_64-linux-gnu/libgcc_s.so.1
7f23d967a000-7f23d967b000 rw-p 00015000 fe:00 268                       
/lib/x86_64-linux-gnu/libgcc_s.so.1
7f23d967b000-7f23d9691000 r-xp 00000000 fe:00 145                       
/lib/x86_64-linux-gnu/libz.so.1.2.7
7f23d9691000-7f23d9890000 ---p 00016000 fe:00 145                       
/lib/x86_64-linux-gnu/libz.so.1.2.7
7f23d9890000-7f23d9891000 r--p 00015000 fe:00 145                       
/lib/x86_64-linux-gnu/libz.so.1.2.7
7f23d9891000-7f23d9892000 rw-p 00016000 fe:00 145                       
/lib/x86_64-linux-gnu/libz.so.1.2.7
7f23d9892000-7f23d98b4000 r-xp 00000000 fe:00 222                       
/lib/x86_64-linux-gnu/liblzma.so.5.0.0
7f23d98b4000-7f23d9ab3000 ---p 00022000 fe:00 222                       
/lib/x86_64-linux-gnu/liblzma.so.5.0.0
7f23d9ab3000-7f23d9ab4000 r--p 00021000 fe:00 222                       
/lib/x86_64-linux-gnu/liblzma.so.5.0.0
7f23d9ab4000-7f23d9ab5000 rw-p 00022000 fe:00 222                       
/lib/x86_64-linux-gnu/liblzma.so.5.0.0
7f23d9ab5000-7f23d9ab7000 r-xp 00000000 fe:00 274                       
/lib/x86_64-linux-gnu/libdl-2.13.so
7f23d9ab7000-7f23d9cb7000 ---p 00002000 fe:00 274                       
/lib/x86_64-linux-gnu/libdl-2.13.so
7f23d9cb7000-7f23d9cb8000 r--p 00002000 fe:00 274                       
/lib/x86_64-linux-gnu/libdl-2.13.so
7f23d9cb8000-7f23d9cb9000 rw-p 00003000 fe:00 274                       
/lib/x86_64-linux-gnu/libdl-2.13.so
7f23d9cb9000-7f23d9cbd000 r-xp 00000000 fe:00 205                       
/lib/x86_64-linux-gnu/libuuid.so.1.3.0
7f23d9cbd000-7f23d9ebc000 ---p 00004000 fe:00 205                       
/lib/x86_64-linux-gnu/libuuid.so.1.3.0
7f23d9ebc000-7f23d9ebd000 r--p 00003000 fe:00 205                       
/lib/x86_64-linux-gnu/libuuid.so.1.3.0
7f23d9ebd000-7f23d9ebe000 rw-p 00004000 fe:00 205                       
/lib/x86_64-linux-gnu/libuuid.so.1.3.0
7f23d9ebe000-7f23d9ec0000 r-xp 00000000 fe:00 174                       
/lib/x86_64-linux-gnu/libutil-2.13.so
7f23d9ec0000-7f23da0bf000 ---p 00002000 fe:00 174                       
/lib/x86_64-linux-gnu/libutil-2.13.so
7f23da0bf000-7f23da0c0000 r--p 00001000 fe:00 174                       
/lib/x86_64-linux-gnu/libutil-2.13.so
7f23da0c0000-7f23da0c1000 rw-p 00002000 fe:00 174                       
/lib/x86_64-linux-gnu/libutil-2.13.so
7f23da0c1000-7f23da0c8000 r-xp 00000000 fe:00 24756                     
/usr/lib/xen-4.2/lib/libblktapctl.so
7f23da0c8000-7f23da2c7000 ---p 00007000 fe:00 24756                     
/usr/lib/xen-4.2/lib/libblktapctl.so
7f23da2c7000-7f23da2c8000 rw-p 00006000 fe:00 24756                     
/usr/lib/xen-4.2/lib/libblktapctl.so
7f23da2c8000-7f23da2ce000 r-xp 00000000 fe:00 24379                     
/usr/lib/libxenstore.so.3.0.2
7f23da2ce000-7f23da4cd000 ---p 00006000 fe:00 24379                     
/usr/lib/libxenstore.so.3.0.2
7f23da4cd000-7f23da4ce000 rw-p 00005000 fe:00 24379                     
/usr/lib/libxenstore.so.3.0.2
7f23da4ce000-7f23da4d1000 rw-p 00000000 00:00 0 
7f23da4d1000-7f23da4fa000 r-xp 00000000 fe:00 24365                     
/usr/lib/libxenguest-4.2.so
7f23da4fa000-7f23da6f9000 ---p 00029000 fe:00 24365                     
/usr/lib/libxenguest-4.2.so
7f23da6f9000-7f23da6fb000 rw-p 00028000 fe:00 24365                     
/usr/lib/libxenguest-4.2.so
7f23da6fb000-7f23da87b000 r-xp 00000000 fe:00 279                       
/lib/x86_64-linux-gnu/libc-2.13.so
7f23da87b000-7f23daa7b000 ---p 00180000 fe:00 279                       
/lib/x86_64-linux-gnu/libc-2.13.so
7f23daa7b000-7f23daa7f000 r--p 00180000 fe:00 279                       
/lib/x86_64-linux-gnu/libc-2.13.so
7f23daa7f000-7f23daa80000 rw-p 00184000 fe:00 279                       
/lib/x86_64-linux-gnu/libc-2.13.so
7f23daa80000-7f23daa85000 rw-p 00000000 00:00 0 
7f23daa85000-7f23daa9c000 r-xp 00000000 fe:00 212                       
/lib/x86_64-linux-gnu/libpthread-2.13.so
7f23daa9c000-7f23dac9b000 ---p 00017000 fe:00 212                       
/lib/x86_64-linux-gnu/libpthread-2.13.so
7f23dac9b000-7f23dac9c000 r--p 00016000 fe:00 212                       
/lib/x86_64-linux-gnu/libpthread-2.13.so
7f23dac9c000-7f23dac9d000 rw-p 00017000 fe:00 212                       
/lib/x86_64-linux-gnu/libpthread-2.13.so
7f23dac9d000-7f23daca1000 rw-p 00000000 00:00 0 
7f23daca1000-7f23dacaa000 r-xp 00000000 fe:00 24348                     
/usr/lib/x86_64-linux-gnu/libyajl.so.2.0.4
7f23dacaa000-7f23daea9000 ---p 00009000 fe:00 24348                     
/usr/lib/x86_64-linux-gnu/libyajl.so.2.0.4
7f23daea9000-7f23daeaa000 r--p 00008000 fe:00 24348                     
/usr/lib/x86_64-linux-gnu/libyajl.so.2.0.4
7f23daeaa000-7f23daeab000 rw-p 00009000 fe:00 24348                     
/usr/lib/x86_64-linux-gnu/libyajl.so.2.0.4
7f23daeab000-7f23daece000 r-xp 00000000 fe:00 24364                     
/usr/lib/libxenctrl-4.2.so
7f23daece000-7f23db0cd000 ---p 00023000 fe:00 24364                     
/usr/lib/libxenctrl-4.2.so
7f23db0cd000-7f23db0ce000 rw-p 00022000 fe:00 24364                     
/usr/lib/libxenctrl-4.2.so
7f23db0ce000-7f23db126000 r-xp 00000000 fe:00 24759                     
/usr/lib/xen-4.2/lib/libxenlight.so
7f23db126000-7f23db326000 ---p 00058000 fe:00 24759                     
/usr/lib/xen-4.2/lib/libxenlight.so
7f23db326000-7f23db329000 rw-p 00058000 fe:00 24759                     
/usr/lib/xen-4.2/lib/libxenlight.so
7f23db329000-7f23db337000 r-xp 00000000 fe:00 24755                     
/usr/lib/xen-4.2/lib/libxlutil.so
7f23db337000-7f23db537000 ---p 0000e000 fe:00 24755                     
/usr/lib/xen-4.2/lib/libxlutil.so
7f23db537000-7f23db538000 rw-p 0000e000 fe:00 24755                     
/usr/lib/xen-4.2/lib/libxlutil.so
7f23db538000-7f23db558000 r-xp 00000000 fe:00 172                       
/lib/x86_64-linux-gnu/ld-2.13.so
7f23db744000-7f23db745000 ---p 00000000 00:00 0 
7f23db745000-7f23db748000 rw-p 00000000 00:00 0                         
[stack:10394]
7f23db748000-7f23db749000 rw-p 00000000 00:00 0 
7f23db749000-7f23db751000 rw-p 00000000 00:00 0 
7f23db751000-7f23db752000 rw-p 00000000 00:00 0 
7f23db752000-7f23db753000 rw-p 00000000 00:00 0 
7f23db753000-7f23db754000 rw-p 00000000 00:00 0 
7f23db754000-7f23db757000 rw-p 00000000 00:00 0 
7f23db757000-7f23db758000 r--p 0001f000 fe:00 172                       
/lib/x86_64-linux-gnu/ld-2.13.so
7f23db758000-7f23db759000 rw-p 00020000 fe:00 172                       
/lib/x86_64-linux-gnu/ld-2.13.so
7f23db759000-7f23db75a000 rw-p 00000000 00:00 0 
7fff7131b000-7fff7133c000 rw-p 00000000 00:00 0                          [stack]
7fff713fe000-7fff71400000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                 
[vsyscall]
Aborted


Is it an xl bug, that I should report upstream ?

Thanks,
Olivier

On Sat, 2013-08-17 at 09:35 +0200, Olivier Bonvalet
wrote:
> 
> Is it an xl bug, that I should report upstream ? 
Well it's certainly a bug but it is hard to say if it is upstream or if
it is caused by (perhaps partially incomplete) backports in the
packaging. Before reporting upstream it would ideally be reproduced with
either the latest 4.2.x or the development branch.

Do you have symbols for the xl binary? That might make the backtrace
more useful.

"xl -vvv create" may also give some information about what was going
on.

You could also try running under valgrind and see if that can pinpoint
the issue. If it is crashing early enough then maybe regular valrind
will work, but if it is happening later (specifically after any
hypercalls are made) then you would need the new upstream valgrind
support for knowing about Xen -- this is only in svn though.

Ian.

Le samedi 17 ao?t 2013 ? 11:37 +0100, Ian Campbell a ?crit
:
> On Sat, 2013-08-17 at 09:35 +0200, Olivier Bonvalet wrote:
> > 
> > Is it an xl bug, that I should report upstream ? 
> 
> Well it's certainly a bug but it is hard to say if it is upstream or if
> it is caused by (perhaps partially incomplete) backports in the
> packaging. Before reporting upstream it would ideally be reproduced with
> either the latest 4.2.x or the development branch.
> 
> Do you have symbols for the xl binary? That might make the backtrace
> more useful.
> 
> "xl -vvv create" may also give some information about what was
going on.
> 
> You could also try running under valgrind and see if that can pinpoint
> the issue. If it is crashing early enough then maybe regular valrind
> will work, but if it is happening later (specifically after any
> hypercalls are made) then you would need the new upstream valgrind
> support for knowing about Xen -- this is only in svn though.
> 
> Ian.
> 
> 

Hi,

thanks for your answer.

After more tests, I saw that it's just a "disk full" problem. When
the
disk of the dom0 is full xl crash instead of returning a more explicit
error message.

So, no big problem here, if the disk is not null, all works fine.

Thanks for your time,
Olivier