Any one has a Idea where to start looking for the solution?
I found this in the xend.log, but is not really helping.
Xend.log
[2008-09-04 14:16:49 4765] WARNING (XendDomainInfo:1283) Domain has
crashed: name=windows2003x64-db2 id=15.
[2008-09-04 14:16:49 4765] DEBUG (XendDomainInfo:1990)
XendDomainInfo.destroy: domid=15
[2008-09-04 14:16:49 4765] DEBUG (XendDomainInfo:1032) Storing domain
details: {''console/port'': ''6'',
''cpu/3/availability'': ''online'',
''name'':
''windows2003x64-db2'', ''console/limit'':
''1048576'', ''store/port'':
''5'',
''cpu/2/availability'': ''online'',
''vm'':
''/vm/55a80eac-5bf6-11ee-bc51-9fe0c174b267'',
''domid'': ''15'',
''image/suspend-cancel'': ''1'',
''cpu/0/availability'': ''online'',
''memory/target'': ''4608000'',
''control/platform-feature-multiprocessor-suspend'':
''1'',
''store/ring-ref'': ''983038'',
''cpu/1/availability'': ''online'',
''console/type'': ''ioemu''}
[2008-09-04 14:16:49 4765] DEBUG (XendDomainInfo:2007)
XendDomainInfo.destroyDomain(15)
[2008-09-04 14:16:50 4765] DEBUG (XendDomainInfo:1580) Destroying device
model
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1587) Releasing devices
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing vif/0
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590)
XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing vbd/768
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590)
XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/768
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing vbd/5632
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590)
XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/5632
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing vbd/832
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590)
XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/832
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing vfb/0
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590)
XendDomainInfo.destroyDevice: deviceClass = vfb, device = vfb/0
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing
console/0
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590)
XendDomainInfo.destroyDevice: deviceClass = console, device = console/0
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1585) No device model
[2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1587) Releasing devices
[2
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
If this crash happens on a regular base, i would assume some cronjob in dom0 leaving too less ressources for qemu-dm. Besides this, are you able to get some log information from within your crashing domU ? cheers, Stephan Richard Drent schrieb:> Any one has a Idea where to start looking for the solution? > > I found this in the xend.log, but is not really helping. > > Xend.log > > [2008-09-04 14:16:49 4765] WARNING (XendDomainInfo:1283) Domain has > crashed: name=windows2003x64-db2 id=15. > [2008-09-04 14:16:49 4765] DEBUG (XendDomainInfo:1990) > XendDomainInfo.destroy: domid=15 > [2008-09-04 14:16:49 4765] DEBUG (XendDomainInfo:1032) Storing domain > details: {''console/port'': ''6'', ''cpu/3/availability'': ''online'', ''name'': > ''windows2003x64-db2'', ''console/limit'': ''1048576'', ''store/port'': ''5'', > ''cpu/2/availability'': ''online'', ''vm'': > ''/vm/55a80eac-5bf6-11ee-bc51-9fe0c174b267'', ''domid'': ''15'', > ''image/suspend-cancel'': ''1'', ''cpu/0/availability'': ''online'', > ''memory/target'': ''4608000'', > ''control/platform-feature-multiprocessor-suspend'': ''1'', > ''store/ring-ref'': ''983038'', ''cpu/1/availability'': ''online'', > ''console/type'': ''ioemu''} > [2008-09-04 14:16:49 4765] DEBUG (XendDomainInfo:2007) > XendDomainInfo.destroyDomain(15) > [2008-09-04 14:16:50 4765] DEBUG (XendDomainInfo:1580) Destroying device > model > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1587) Releasing devices > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing vif/0 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590) > XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing vbd/768 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590) > XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/768 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing vbd/5632 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590) > XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/5632 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing vbd/832 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590) > XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/832 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing vfb/0 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590) > XendDomainInfo.destroyDevice: deviceClass = vfb, device = vfb/0 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing > console/0 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590) > XendDomainInfo.destroyDevice: deviceClass = console, device = console/0 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1585) No device model > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1587) Releasing devices > [2 > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users-- Stephan Seitz Senior System Administrator *netz-haut* e.K. multimediale kommunikation zweierweg 22 97074 würzburg fon: +49 931 2876247 fax: +49 931 2876248 web: www.netz-haut.de <http://www.netz-haut.de/> registriergericht: amtsgericht würzburg, hra 5054 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Stephan Seitz schreef:> If this crash happens on a regular base, i would assume some cronjob in > dom0 leaving too less ressources for qemu-dm.Did you ever look at the memory footprint of xend after two days of running? That thing is one big memory leak. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAki/6UcACgkQYH1+F2Rqwn3figCfbzXlebtHBeZrCvUz5GjyUhJd XnMAn3tH1Ca4hBQBel+Ow/m6Cv0ayRs1 =4mmm -----END PGP SIGNATURE----- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Nope, no data in de windows server availibale,.. so at ones its exit for the server.. And it is to inregular for a cron job... Thnx Richard -----Oorspronkelijk bericht----- Van: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] Namens Stephan Seitz Verzonden: donderdag 4 september 2008 15:52 Aan: Xen users mailing list Onderwerp: Re: [Xen-users] Domain Crashes everyday If this crash happens on a regular base, i would assume some cronjob in dom0 leaving too less ressources for qemu-dm. Besides this, are you able to get some log information from within your crashing domU ? cheers, Stephan Richard Drent schrieb:> Any one has a Idea where to start looking for the solution? > > I found this in the xend.log, but is not really helping. > > Xend.log > > [2008-09-04 14:16:49 4765] WARNING (XendDomainInfo:1283) Domain has > crashed: name=windows2003x64-db2 id=15. > [2008-09-04 14:16:49 4765] DEBUG (XendDomainInfo:1990) > XendDomainInfo.destroy: domid=15 > [2008-09-04 14:16:49 4765] DEBUG (XendDomainInfo:1032) Storing domain > details: {''console/port'': ''6'', ''cpu/3/availability'': ''online'', ''name'': > ''windows2003x64-db2'', ''console/limit'': ''1048576'', ''store/port'': ''5'', > ''cpu/2/availability'': ''online'', ''vm'': > ''/vm/55a80eac-5bf6-11ee-bc51-9fe0c174b267'', ''domid'': ''15'', > ''image/suspend-cancel'': ''1'', ''cpu/0/availability'': ''online'', > ''memory/target'': ''4608000'', > ''control/platform-feature-multiprocessor-suspend'': ''1'', > ''store/ring-ref'': ''983038'', ''cpu/1/availability'': ''online'', > ''console/type'': ''ioemu''} > [2008-09-04 14:16:49 4765] DEBUG (XendDomainInfo:2007) > XendDomainInfo.destroyDomain(15) > [2008-09-04 14:16:50 4765] DEBUG (XendDomainInfo:1580) Destroying > device model > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1587) Releasing > devices > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing vif/0 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590) > XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing > vbd/768 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590) > XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/768 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing > vbd/5632 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590) > XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/5632 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing > vbd/832 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590) > XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/832 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing vfb/0 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590) > XendDomainInfo.destroyDevice: deviceClass = vfb, device = vfb/0 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1593) Removing > console/0 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:590) > XendDomainInfo.destroyDevice: deviceClass = console, device = > console/0 > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1585) No device model > [2008-09-04 14:16:51 4765] DEBUG (XendDomainInfo:1587) Releasing > devices > [2 > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users-- Stephan Seitz Senior System Administrator *netz-haut* e.K. multimediale kommunikation zweierweg 22 97074 würzburg fon: +49 931 2876247 fax: +49 931 2876248 web: www.netz-haut.de <http://www.netz-haut.de/> registriergericht: amtsgericht würzburg, hra 5054 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Where do I look and for what for the memory footprint? Thnx Richard -----Oorspronkelijk bericht----- Van: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] Namens Stefan de Konink Verzonden: donderdag 4 september 2008 15:57 Aan: Stephan Seitz CC: Xen users mailing list Onderwerp: Re: [Xen-users] Domain Crashes everyday -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Stephan Seitz schreef:> If this crash happens on a regular base, i would assume some cronjobin> dom0 leaving too less ressources for qemu-dm.Did you ever look at the memory footprint of xend after two days of running? That thing is one big memory leak. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAki/6UcACgkQYH1+F2Rqwn3figCfbzXlebtHBeZrCvUz5GjyUhJd XnMAn3tH1Ca4hBQBel+Ow/m6Cv0ayRs1 =4mmm -----END PGP SIGNATURE----- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Thu, Sep 4, 2008 at 8:57 AM, Stefan de Konink <skinkie@xs4all.nl> wrote:> Did you ever look at the memory footprint of xend after two days of > running? That thing is one big memory leak.Such a statement should at least include a link to the bugzilla entry regarding the problem, or even better, a patch to fix it. ;-) jerry -- There''s plenty of youth in America - it''s time we find the "fountain of smart". _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Jerry Amundson schrieb:> On Thu, Sep 4, 2008 at 8:57 AM, Stefan de Konink <skinkie@xs4all.nl> wrote: >> Did you ever look at the memory footprint of xend after two days of >> running? That thing is one big memory leak. > > Such a statement should at least include a link to the bugzilla entry > regarding the problem, or even better, a patch to fix it. ;-)Actually, I''m unable to confirm that issue in general, maybe it''s some problem with particular python versions and/or xen tools versions. A recent look on a dom0 hosting 26 averaged loaded domUs showed xend consuming about 350M out 2G available at dom0 after an uptime of 60 days. System is 3.2.0 x86_64 2.6.18.8-xen w/ ubuntu gutsy userland. The only time i faced similar problem that the OP wrote, was when running backported distro kernels at domU. 2.6.18.8-xen never showed such issues on 32 as well as on 64bit. cheers, Stephan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Jerry Amundson schreef:> On Thu, Sep 4, 2008 at 8:57 AM, Stefan de Konink <skinkie@xs4all.nl> wrote: >> Did you ever look at the memory footprint of xend after two days of >> running? That thing is one big memory leak. > > Such a statement should at least include a link to the bugzilla entry > regarding the problem, or even better, a patch to fix it. ;-)Please if you worry about my ''bugzilla'' entries go look for my name there and the total unresponsiveness of far bigger issues! Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkjFQ4UACgkQYH1+F2Rqwn3/gACeIrB0pGnWdi62XrMTqoknIUoh GsEAnjJiYqr87/uQT0mkbNfQhZzV9Sg7 =Jn0p -----END PGP SIGNATURE----- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Stephan Seitz schreef:> Jerry Amundson schrieb: >> On Thu, Sep 4, 2008 at 8:57 AM, Stefan de Konink <skinkie@xs4all.nl> wrote: >>> Did you ever look at the memory footprint of xend after two days of >>> running? That thing is one big memory leak. >> Such a statement should at least include a link to the bugzilla entry >> regarding the problem, or even better, a patch to fix it. ;-) > > Actually, I''m unable to confirm that issue in general, maybe it''s some > problem with particular python versions and/or xen tools versions. > A recent look on a dom0 hosting 26 averaged loaded domUs showed xend > consuming about 350M out 2G available at dom0 after an uptime of 60 > days. System is 3.2.0 x86_64 2.6.18.8-xen w/ ubuntu gutsy userland. > > The only time i faced similar problem that the OP wrote, was when > running backported distro kernels at domU. 2.6.18.8-xen never showed > such issues on 32 as well as on 64bit. >skinkie@xenapi ~ $ ps aux | grep xend root 4633 0.0 1.9 102984 10240 ? S Sep02 0:00 /usr/bin/python2.5 /usr/sbin/xend start root 4634 0.0 28.1 409960 147420 ? Sl Sep02 0:32 /usr/bin/python2.5 /usr/sbin/xend start Show me your output :) This is 3.3 (with the vmmetrics patch!) and Python 2.5.2 Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkjFRBgACgkQYH1+F2Rqwn0p3ACfV9H6GYXHBkR4sEb6ZRkxhLQc X9oAn1Sg+gYxUs7Al16mb3qCtcQUYD/y =Wtku -----END PGP SIGNATURE----- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Stefan de Konink schrieb:> Stephan Seitz schreef: >> Jerry Amundson schrieb: >>> On Thu, Sep 4, 2008 at 8:57 AM, Stefan de Konink <skinkie@xs4all.nl> > wrote: >>>> Did you ever look at the memory footprint of xend after two days of >>>> running? That thing is one big memory leak. >>> Such a statement should at least include a link to the bugzilla entry >>> regarding the problem, or even better, a patch to fix it. ;-) > >> Actually, I''m unable to confirm that issue in general, maybe it''s some >> problem with particular python versions and/or xen tools versions. >> A recent look on a dom0 hosting 26 averaged loaded domUs showed xend >> consuming about 350M out 2G available at dom0 after an uptime of 60 >> days. System is 3.2.0 x86_64 2.6.18.8-xen w/ ubuntu gutsy userland. > >> The only time i faced similar problem that the OP wrote, was when >> running backported distro kernels at domU. 2.6.18.8-xen never showed >> such issues on 32 as well as on 64bit. > > > skinkie@xenapi ~ $ ps aux | grep xend > root 4633 0.0 1.9 102984 10240 ? S Sep02 0:00 > /usr/bin/python2.5 /usr/sbin/xend start > root 4634 0.0 28.1 409960 147420 ? Sl Sep02 0:32 > /usr/bin/python2.5 /usr/sbin/xend start > > > Show me your output :) This is 3.3 (with the vmmetrics patch!) and > Python 2.5.2root@osiris:~# ps xau | grep xend root 4854 0.0 0.4 103744 9812 ? S Jul12 0:00 python /usr/sbin/xend start root 4858 0.0 1.7 276092 36456 ? Sl Jul12 16:11 python /usr/sbin/xend start ubuntu gutsy x86_64 / xen 3.2.1-rc1 (sorry, I thought this was 3.2.0) python is 2.5.1 In fact, python/xend is consuming the biggest chunk of memory used in dom0 at all, but this isn''t really hard for this particular dom0 since there''s nothing running except the usual suspects. cheers, Stephan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users