Maciej Liżewski
2010-Jul-06 17:06 UTC
[Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24
Hi, Recently I have installed Debian Lenny on two different machines (different ram size, disks, Xeon dual and quad core, filesystems both xfs and ext3, etc). Packages versions: Dom0: ii libc6-xen 2.7-18lenny4 GNU C Library: Shared libraries [Xen version] ii libxenstore3.0 3.2.1-2 Xenstore communications library for Xen ii linux-image-2.6.26-2-xen-686 2.6.26-24 Linux 2.6.26 image on i686, oldstyle Xen support ii linux-modules-2.6.26-2-xen-686 2.6.26-24 Linux 2.6.26 modules on i686 ii xen-hypervisor-3.2-1-i386 3.2.1-2 The Xen Hypervisor on i386 ii xen-linux-system-2.6.26-2-xen-686 2.6.26-24 XEN system with Linux 2.6.26 image on i686 ii xen-shell 1.8-3 Console based Xen administration utility ii xen-tools 3.9-4 Tools to manage Debian XEN virtual servers ii xen-utils-3.2-1 3.2.1-2 XEN administrative tools ii xen-utils-common 3.2.0-2 XEN administrative tools - common files ii xenstore-utils 3.2.1-2 Xenstore utilities for Xen domU''s: ii libc6-xen 2.7-18lenny4 GNU C Library: Shared libraries [Xen version ii linux-modules-2.6.26-2-xen-686 2.6.26-24 Linux 2.6.26 modules on i686 those are common for both installations. Anly standard packages were installed using "apt-get". No compiling, no unstable releases.. There are 2-3 DomU''s on each machine. There have very little usage (load average under 1.0). Both installations was made about month ago. During this time domU''s are randomly crashing. Interval between those crashes vary between single day and week. There are no logs about reason of those crashes, just "Domain has crashed" in xend.log and after that log about xen trying to respawn DomU but previous instance is still running. Only a single DomU on machine is crashing at a time - others run stable at the same time. Which DomU will crash next time is undeterminable and random. Funny, that times of most those crashes are similar to MRTG executed from CRON (and log about this execution is often last in syslog just before crash), however there are nothing specific in MRTG config (getting cpu usage, load average, disk space) but using piped commands instead of snmp (they are simple: cat /proc/meminfo | grep | awk. and so on) Dom0 during those crashes is stable and never crashed or been restarted. Any suggestion how to solve this situation? Downgrade / upgrade to some "unstable" packages that are known to be much more stable than those from default Lenny installation? _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Peter Viskup
2010-Jul-06 19:20 UTC
Re: [Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24
On 07/06/2010 07:06 PM, Maciej Liżewski wrote:> > Hi, > > Recently I have installed Debian Lenny on two different machines > (different ram size, disks, Xeon dual and quad core, filesystems both > xfs and ext3, etc). Packages versions: > > Dom0: > > ii libc6-xen 2.7-18lenny4 GNU C > Library: Shared libraries [Xen version] > > ii libxenstore3.0 3.2.1-2 > Xenstore communications library for Xen > > ii linux-image-2.6.26-2-xen-686 2.6.26-24 Linux > 2.6.26 image on i686, oldstyle Xen support > > ii linux-modules-2.6.26-2-xen-686 2.6.26-24 Linux > 2.6.26 modules on i686 > > ii xen-hypervisor-3.2-1-i386 3.2.1-2 The > Xen Hypervisor on i386 > > ii xen-linux-system-2.6.26-2-xen-686 2.6.26-24 XEN > system with Linux 2.6.26 image on i686 > > ii xen-shell 1.8-3 > Console based Xen administration utility > > ii xen-tools 3.9-4 Tools > to manage Debian XEN virtual servers > > ii xen-utils-3.2-1 3.2.1-2 XEN > administrative tools > > ii xen-utils-common 3.2.0-2 XEN > administrative tools - common files > > ii xenstore-utils 3.2.1-2 > Xenstore utilities for Xen > > domU''s: > > ii libc6-xen 2.7-18lenny4 GNU C > Library: Shared libraries [Xen version > > ii linux-modules-2.6.26-2-xen-686 2.6.26-24 Linux > 2.6.26 modules on i686 > > those are common for both installations. Anly standard packages were > installed using "apt-get". No compiling, no unstable releases.... > There are 2-3 DomU''s on each machine. There have very little usage > (load average under 1.0). Both installations was made about month ago. > During this time domU''s are randomly crashing. Interval between those > crashes vary between single day and week. There are no logs about > reason of those crashes, just "Domain has crashed" in xend.log and > after that log about xen trying to respawn DomU but previous instance > is still running. Only a single DomU on machine is crashing at a time > -- others run stable at the same time. Which DomU will crash next time > is undeterminable and random... > > Funny, that times of most those crashes are similar to MRTG executed > from CRON (and log about this execution is often last in syslog just > before crash), however there are nothing specific in MRTG config > (getting cpu usage, load average, disk space) but using piped commands > instead of snmp (they are simple: cat /proc/meminfo | grep | awk... > and so on) > > Dom0 during those crashes is stable and never crashed or been restarted... > > Any suggestion how to solve this situation? Downgrade / upgrade to > some "unstable" packages that are known to be much more stable than > those from default Lenny installation? > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-usersHi, first I would recommend you to switch to amd64 kernel at least for dom0. There were some issues with this ''xenified'' Debian kernel version 2.6.26 some of us were fighting against them [1]. But these seems to be solved in current kernel. The configuration option "(dom0-cpus 1)" in /etc/xen/xend-config.sxp on dom0 could probably solve the issue you are experiencing. Other things you can do: - enable DEBUG loglevel in xend-config.sxp with "(loglevel DEBUG)" - configure your domU''s with option "on_crash = ''destroy''" - dom0 will not try to restart it in case domU will crash again Could you describe your dom0/domU''s configuration in more details? [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=571603 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=542250 -- Peter _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Maciej Liżewski
2010-Jul-09 13:58 UTC
RE: [Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24
Hi, Thanks for answer. I''ll try your suggestions about changes in configuration. Don''t know if amd64 kernel will run stable, as this is dual Intel Xeon machine. My configuration is very simple, example of one machine: dom0 is just Xen host with network configured as custom script using two bridges (eth0, eth1). Root partition has 10GB, rest is managed by LVM for DomU''s. It has ~700MB ram. That''s all, there are no other services on Dom0 DomU''s are created to provide services: DomU 1: MySQL 5.1 (from lenny-backports), ext3 (former I have tried XFS here with same effect), 3GB ram DomU 2: Web (Apache2, php5 as fastcgi, vsftpd, dkim-filter for emails sent from scripts), XFS, 2GB ram DomU 3: Reverse proxy (Nginx, ftp_proxy, awstats as fastcgi), XFS, 1.2GB ram Every domain (dom0, domU''s) have postfix as MTA Everything installed via apt-get, no compilation, just default apt repository + lenny-backports in some cases. 3rd DomU (reverse proxy) is most stable. It haven''t crash at all. Others crash at least one of them in week. If you need any other information - just let me know. -- Maciek From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Peter Viskup Sent: Tuesday, July 06, 2010 9:21 PM To: xen-users@lists.xensource.com Subject: Re: [Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24 On 07/06/2010 07:06 PM, Maciej Liżewski wrote: Hi, Recently I have installed Debian Lenny on two different machines (different ram size, disks, Xeon dual and quad core, filesystems both xfs and ext3, etc). Packages versions: Dom0: ii libc6-xen 2.7-18lenny4 GNU C Library: Shared libraries [Xen version] ii libxenstore3.0 3.2.1-2 Xenstore communications library for Xen ii linux-image-2.6.26-2-xen-686 2.6.26-24 Linux 2.6.26 image on i686, oldstyle Xen support ii linux-modules-2.6.26-2-xen-686 2.6.26-24 Linux 2.6.26 modules on i686 ii xen-hypervisor-3.2-1-i386 3.2.1-2 The Xen Hypervisor on i386 ii xen-linux-system-2.6.26-2-xen-686 2.6.26-24 XEN system with Linux 2.6.26 image on i686 ii xen-shell 1.8-3 Console based Xen administration utility ii xen-tools 3.9-4 Tools to manage Debian XEN virtual servers ii xen-utils-3.2-1 3.2.1-2 XEN administrative tools ii xen-utils-common 3.2.0-2 XEN administrative tools - common files ii xenstore-utils 3.2.1-2 Xenstore utilities for Xen domU''s: ii libc6-xen 2.7-18lenny4 GNU C Library: Shared libraries [Xen version ii linux-modules-2.6.26-2-xen-686 2.6.26-24 Linux 2.6.26 modules on i686 those are common for both installations. Anly standard packages were installed using "apt-get". No compiling, no unstable releases.. There are 2-3 DomU''s on each machine. There have very little usage (load average under 1.0). Both installations was made about month ago. During this time domU''s are randomly crashing. Interval between those crashes vary between single day and week. There are no logs about reason of those crashes, just "Domain has crashed" in xend.log and after that log about xen trying to respawn DomU but previous instance is still running. Only a single DomU on machine is crashing at a time - others run stable at the same time. Which DomU will crash next time is undeterminable and random. Funny, that times of most those crashes are similar to MRTG executed from CRON (and log about this execution is often last in syslog just before crash), however there are nothing specific in MRTG config (getting cpu usage, load average, disk space) but using piped commands instead of snmp (they are simple: cat /proc/meminfo | grep | awk. and so on) Dom0 during those crashes is stable and never crashed or been restarted. Any suggestion how to solve this situation? Downgrade / upgrade to some "unstable" packages that are known to be much more stable than those from default Lenny installation? _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users Hi, first I would recommend you to switch to amd64 kernel at least for dom0. There were some issues with this ''xenified'' Debian kernel version 2.6.26 some of us were fighting against them [1]. But these seems to be solved in current kernel. The configuration option "(dom0-cpus 1)" in /etc/xen/xend-config.sxp on dom0 could probably solve the issue you are experiencing. Other things you can do: - enable DEBUG loglevel in xend-config.sxp with "(loglevel DEBUG)" - configure your domU''s with option "on_crash = ''destroy''" - dom0 will not try to restart it in case domU will crash again Could you describe your dom0/domU''s configuration in more details? [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=571603 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=542250 -- Peter _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Matej Zary
2010-Jul-09 14:23 UTC
RE: [Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24
Hello, amd64 in Debian case means it''s for 64bit x86 CPUs (x86-64) - either AMD or Intel. :) Regards Matej -----Original Message----- From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Maciej Liżewski Sent: Friday, July 09, 2010 3:58 PM To: xen-users@lists.xensource.com Subject: RE: [Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24 Hi, Thanks for answer. I''ll try your suggestions about changes in configuration. Don''t know if amd64 kernel will run stable, as this is dual Intel Xeon machine. My configuration is very simple, example of one machine: dom0 is just Xen host with network configured as custom script using two bridges (eth0, eth1). Root partition has 10GB, rest is managed by LVM for DomU''s. It has ~700MB ram. That''s all, there are no other services on Dom0 DomU''s are created to provide services: DomU 1: MySQL 5.1 (from lenny-backports), ext3 (former I have tried XFS here with same effect), 3GB ram DomU 2: Web (Apache2, php5 as fastcgi, vsftpd, dkim-filter for emails sent from scripts), XFS, 2GB ram DomU 3: Reverse proxy (Nginx, ftp_proxy, awstats as fastcgi), XFS, 1.2GB ram Every domain (dom0, domU''s) have postfix as MTA Everything installed via apt-get, no compilation, just default apt repository + lenny-backports in some cases. 3rd DomU (reverse proxy) is most stable. It haven''t crash at all. Others crash at least one of them in week. If you need any other information - just let me know. -- Maciek From _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Maciej Liżewski
2010-Jul-12 13:17 UTC
RE: [Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24
Hi, Ok. I have no other solution. How to switch to 64bit kernel? Assume I have configuration like descibed earlier. Do I have to reinstal everything from scratch? Or just install 64bit packages? What packages? How about domU''s? do I have to reinstall something there also? Does anyone have working configuration like this? 64bit dom0 and 32bit domU''s? is it stable witch Lenny packages? We have other servers running Debian, so changing distro to CentOS is rather last thing I would think of, but maybe this would be better solution than messing with Lenny? One thing I am afraid of with CentOS I will have problems with installing required new versions of Apache, Mysql, php as fastcgi... as everything will have to compile from sources. Please help me with selecting proper solution. In fact - everything will be better that current unstable system... TIA -- Maciej -----Original Message----- From: Matej Zary [mailto:matej.zary@cvtisr.sk] Sent: Friday, July 09, 2010 4:23 PM To: Maciej Liżewski; xen-users@lists.xensource.com Subject: RE: [Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24 Hello, amd64 in Debian case means it''s for 64bit x86 CPUs (x86-64) - either AMD or Intel. :) Regards Matej -----Original Message----- From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Maciej Liżewski Sent: Friday, July 09, 2010 3:58 PM To: xen-users@lists.xensource.com Subject: RE: [Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24 Hi, Thanks for answer. I''ll try your suggestions about changes in configuration. Don''t know if amd64 kernel will run stable, as this is dual Intel Xeon machine. My configuration is very simple, example of one machine: dom0 is just Xen host with network configured as custom script using two bridges (eth0, eth1). Root partition has 10GB, rest is managed by LVM for DomU''s. It has ~700MB ram. That''s all, there are no other services on Dom0 DomU''s are created to provide services: DomU 1: MySQL 5.1 (from lenny-backports), ext3 (former I have tried XFS here with same effect), 3GB ram DomU 2: Web (Apache2, php5 as fastcgi, vsftpd, dkim-filter for emails sent from scripts), XFS, 2GB ram DomU 3: Reverse proxy (Nginx, ftp_proxy, awstats as fastcgi), XFS, 1.2GB ram Every domain (dom0, domU''s) have postfix as MTA Everything installed via apt-get, no compilation, just default apt repository + lenny-backports in some cases. 3rd DomU (reverse proxy) is most stable. It haven''t crash at all. Others crash at least one of them in week. If you need any other information - just let me know. -- Maciek From _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Peter Viskup
2010-Jul-12 21:59 UTC
Re: [Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24
Hi Maciej, you have to reinstall dom0 when you would like to switch to amd64. You can leave your domU''s as they are or you can reinstall them too...as you wish. It is up to you. 64bit dom0 can run 32bit domU without any issue. Did you try to set DEBUG log-level - any errors in the log? Any error message on dom0 or domU will help. We were experiencing issue with older Debian''s 2.6.26 xen kernel when domU''s were freezing from-time-to-time. Now we are running Debian''s xen kernel 2.6.18 in domU and this solved the issue with freezing domU''s. You can probably try this - install 2.6.18 on dom0 and/or domU. In our case the domU''s were freezing only when running with more than one vcpu - you can probably try to configure domU''s with only one vcpu. For any other investigation/proposal are needed some error messages... -- Peter On 07/12/2010 03:17 PM, Maciej Liżewski wrote:> Hi, > > Ok. I have no other solution. How to switch to 64bit kernel? > Assume I have configuration like descibed earlier. Do I have to reinstal > everything from scratch? Or just install 64bit packages? What packages? How > about domU''s? do I have to reinstall something there also? > > Does anyone have working configuration like this? 64bit dom0 and 32bit > domU''s? is it stable witch Lenny packages? > > We have other servers running Debian, so changing distro to CentOS is rather > last thing I would think of, but maybe this would be better solution than > messing with Lenny? > One thing I am afraid of with CentOS I will have problems with installing > required new versions of Apache, Mysql, php as fastcgi... as everything will > have to compile from sources. > > > Please help me with selecting proper solution. In fact - everything will be > better that current unstable system... > > TIA > > -- > Maciej > > > -----Original Message----- > From: Matej Zary [mailto:matej.zary@cvtisr.sk] > Sent: Friday, July 09, 2010 4:23 PM > To: Maciej Liżewski; xen-users@lists.xensource.com > Subject: RE: [Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24 > > Hello, > > amd64 in Debian case means it''s for 64bit x86 CPUs (x86-64) - either AMD or > Intel. :) > > > Regards > > Matej > > > -----Original Message----- > From: xen-users-bounces@lists.xensource.com > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Maciej Liżewski > Sent: Friday, July 09, 2010 3:58 PM > To: xen-users@lists.xensource.com > Subject: RE: [Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24 > > Hi, > > > > Thanks for answer. I''ll try your suggestions about changes in configuration. > > > > Don''t know if amd64 kernel will run stable, as this is dual Intel Xeon > machine. > > > > My configuration is very simple, example of one machine: dom0 is just Xen > host with network configured as custom script using two bridges (eth0, > eth1). Root partition has 10GB, rest is managed by LVM for DomU''s. It has > ~700MB ram. That''s all, there are no other services on Dom0 > > > > DomU''s are created to provide services: > > DomU 1: MySQL 5.1 (from lenny-backports), ext3 (former I have tried XFS here > with same effect), 3GB ram > > DomU 2: Web (Apache2, php5 as fastcgi, vsftpd, dkim-filter for emails sent > from scripts), XFS, 2GB ram > > DomU 3: Reverse proxy (Nginx, ftp_proxy, awstats as fastcgi), XFS, 1.2GB ram > > > > Every domain (dom0, domU''s) have postfix as MTA > > > > Everything installed via apt-get, no compilation, just default apt > repository + lenny-backports in some cases. 3rd DomU (reverse proxy) is most > stable. It haven''t crash at all. Others crash at least one of them in week. > > If you need any other information - just let me know. > > > > -- > > Maciek > > > > From > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Maciej Liżewski
2010-Jul-13 08:35 UTC
RE: [Xen-users] Xen 3.2.1-2 on Debian Lenny 2.6.26 2.6.26-24
Hi, Unfortunately there are no errors either In logs or console output. In xen.log there is: [2010-07-11 04:00:02 4078] WARNING (XendDomainInfo:1258) Domain has crashed: name=web id=5. [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:1914) XendDomainInfo.destroyDomain(5) [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:1529) Destroying device model [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:1536) Releasing devices [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:1542) Removing vif/0 [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:590) XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0 [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:1542) Removing vbd/51713 [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:590) XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/51713 [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:1542) Removing vbd/51714 [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:590) XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/51714 [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:1542) Removing console/0 [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:590) XendDomainInfo.destroyDevice: deviceClass = console, device = console/0 [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:1534) No device model [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:1536) Releasing devices [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:106) XendDomainInfo.create_from_dict({''vcpus_params'': {''cap'': 0, ''weight'': 256}, ''PV_args'': ''root=/dev/xvda2 ro clocksource =jiffies'', ''features'': '''', ''cpus'': [], ''paused'': 0, ''actions_after_reboot'': ''restart'', ''shutdown'': 0, ''VCPUs_live'': 1, ''PV_bootloader'': '''', ''actions_after_crash'': ''restart'' , ''vbd_refs'': [''4af090ff-7ded-2e40-e641-0f6a812cb1b7'', ''0bd0bd32-432e-62f0-291f-e52caadef552''], ''PV_ramdisk'': ''/boot/initrd.img-2.6.26-2-xen-686'', ''is_control_domain'': Fals e, ''name_label'': ''web'', ''VCPUs_at_startup'': 1, ''HVM_boot_params'': {}, ''platform'': {}, ''PV_kernel'': ''/boot/vmlinuz-2.6.26-2-xen-686'', ''console_refs'': [''0de68767-8e44 -424e-bce3-4e75700a90fd''], ''online_vcpus'': 4, ''blocked'': 0, ''on_xend_stop'': ''ignore'', ''memory_static_min'': 0, ''HVM_boot_policy'': '''', ''shutdown_reason'': 3, ''VCPUs_max'': 4, '' start_time'': 1278275055.0122731, ''memory_static_max'': 2147483648L, ''actions_after_shutdown'': ''destroy'', ''on_xend_start'': ''ignore'', ''crashed'': 0, ''memory_dynamic_max'': 21474 83648L, ''actions_after_suspend'': '''', ''is_a_template'': False, ''PV_bootloader_args'': '''', ''memory_dynamic_min'': 2147483648L, ''uuid'': ''103c0d3b-7612-fd0e-9adf-4345c84e5602'', ''c pu_time'': 45506.519344966, ''shadow_memory'': 0, ''dying'': 0, ''vcpu_avail'': 15, ''notes'': {''HV_START_LOW'': 4118806528L, ''FEATURES'': ''writable_page_tables|writable_descriptor_ta bles|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel'', ''VIRT_BASE'': 3221225472L, ''GUEST_VERSION'': ''2.6'', ''PADDR_OFFSET'': 0, ''GUEST_OS'': ''linux'', ''HYPERCA LL_PAGE'': 3222278144L, ''LOADER'': ''generic'', ''SUSPEND_CANCEL'': 1, ''PAE_MODE'': ''yes'', ''ENTRY'': 3222274048L, ''XEN_VERSION'': ''xen-3.0''}, ''other_config'': {}, ''running'': 0, ''domi d'': 5, ''vif_refs'': [''583adee6-2d81-2ee4-e5cf-06bce4c1dc4a''], ''vtpm_refs'': [], ''devices'': {''0bd0bd32-432e-62f0-291f-e52caadef552'': (''vbd'', {''uuid'': ''0bd0bd32-432e-62f0-291f- e52caadef552'', ''bootable'': 0, ''devid'': 51714, ''driver'': ''paravirtualised'', ''dev'': ''xvda2'', ''uname'': ''phy:/dev/lvg/web-disk'', ''mode'': ''w''}), ''4af090ff-7ded-2e40-e641 -0f6a812cb1b7'': (''vbd'', {''uuid'': ''4af090ff-7ded-2e40-e641-0f6a812cb1b7'', ''bootable'': 1, ''devid'': 51713, ''driver'': ''paravirtualised'', ''dev'': ''xvda1'', ''uname'': ''phy:/dev/lvg/ web-swap'', ''mode'': ''w''}), ''0de68767-8e44-424e-bce3-4e75700a90fd'': (''console'', {''location'': ''2'', ''devid'': 0, ''protocol'': ''vt100'', ''uuid'': ''0de68767-8e44-424e-bce3-4e 75700a90fd'', ''other_config'': {}}), ''583adee6-2d81-2ee4-e5cf-06bce4c1dc4a'': (''vif'', {''ip'': ''10.20.22.131'', ''mac'': ''00:16:3E:16:9A:43'', ''devid'': 0, ''uuid'': ''583adee6-2d81-2ee 4-e5cf-06bce4c1dc4a'', ''bridge'': ''eth1''})}}) [2010-07-11 04:00:02 4078] ERROR (XendDomainInfo:111) Domain construction failed Traceback (most recent call last): File "/usr/lib/xen-3.2-1/lib/python/xen/xend/XendDomainInfo.py", line 109, in create_from_dict vm.start() File "/usr/lib/xen-3.2-1/lib/python/xen/xend/XendDomainInfo.py", line 444, in start raise XendError(''VM already running'') XendError: VM already running [2010-07-11 04:00:02 4078] DEBUG (XendDomainInfo:1897) XendDomainInfo.destroy: domid=5 [2010-07-11 04:00:02 4078] ERROR (XendDomainInfo:1425) Failed to restart domain 5. Same time on domU (restart was initiated by watchdog on dom0 - script on cron checking if every domU is up and running): Jul 11 03:59:01 web /USR/SBIN/CRON[32055]: (xxxx) CMD (sh /home/xxxx/generator.sh 1>/dev/null) Jul 11 03:59:01 web /USR/SBIN/CRON[32056]: (xyyy) CMD (sh /home/yyyy/generator.sh 1>/dev/null) Jul 11 04:02:12 web kernel: imklog 3.18.6, log source = /proc/kmsg started. Jul 11 04:02:12 web kernel: [ 0.000000] Initializing cgroup subsys cpuset Jul 11 04:02:12 web kernel: [ 0.000000] Initializing cgroup subsys cpu Jul 11 04:02:12 web kernel: [ 0.000000] Linux version 2.6.26-2-xen-686 (Debian 2.6.26-24) (dannf@debian.org) (gcc version 4.1.3 20080704 (prerelease) (Debian 4.1.2-25)) #1 SMP Mon Jun 21 10:37:05 UTC 2010 Jul 11 04:02:12 web kernel: [ 0.000000] Reserving virtual address space above 0xf5800000 Jul 11 04:02:12 web kernel: [ 0.000000] BIOS-provided physical RAM map: Jul 11 04:02:12 web kernel: [ 0.000000] Xen: 0000000000000000 - 0000000080800000 (usable) Jul 11 04:02:12 web kernel: [ 0.000000] 1328MB HIGHMEM available. Jul 11 04:02:12 web kernel: [ 0.000000] 728MB LOWMEM available. Jul 11 04:02:12 web kernel: [ 0.000000] NX (Execute Disable) protection: active Jul 11 04:02:12 web kernel: [ 0.000000] Entering add_active_range(0, 0, 526336) 0 entries of 256 used Jul 11 04:02:12 web kernel: [ 0.000000] Zone PFN ranges: Jul 11 04:02:12 web kernel: [ 0.000000] DMA 0 -> 4096 Jul 11 04:02:12 web kernel: [ 0.000000] Normal 4096 -> 186368 Jul 11 04:02:12 web kernel: [ 0.000000] HighMem 186368 -> 526336 Jul 11 04:02:12 web kernel: [ 0.000000] Movable zone start PFN for each node Jul 11 04:02:12 web kernel: [ 0.000000] early_node_map[1] active PFN ranges Jul 11 04:02:12 web kernel: [ 0.000000] 0: 0 -> 526336 Jul 11 04:02:12 web kernel: [ 0.000000] On node 0 totalpages: 526336 Cron scripts have schedule to run every minute, so those calls aren''t unusual and cannot be primary reason for crash. There are some segfault from php5-cgi, but they were not present at the crash time (guess they have something to do with php5-suhosin module). Jul 8 22:12:51 web kernel: [345103.172278] php5-cgi[9975]: segfault at 120 ip 082dd95f sp bfff76d0 error 4 in php5-cgi[8048000+4cf000] Jul 8 22:20:24 web kernel: [345556.528923] php5-cgi[10293]: segfault at 120 ip 082dd95f sp bfff7fb0 error 4 in php5-cgi[8048000+4cf000] Jul 8 22:24:44 web kernel: [345816.800299] php5-cgi[10524]: segfault at 120 ip 082dd95f sp bfff8ba0 error 4 in php5-cgi[8048000+4cf000] Jul 8 22:36:39 web kernel: [346531.763830] php5-cgi[10640]: segfault at 120 ip 082dd95f sp bfff7510 error 4 in php5-cgi[8048000+4cf000] Jul 8 22:48:49 web kernel: [347261.661357] php5-cgi[10949]: segfault at 120 ip 082dd95f sp bfff7b10 error 4 in php5-cgi[8048000+4cf000] Jul 8 22:49:10 web kernel: [347282.927768] php5-cgi[11241]: segfault at 120 ip 082dd95f sp bfff7690 error 4 in php5-cgi[8048000+4cf000] Jul 8 22:51:10 web kernel: [347402.846540] php5-cgi[11253]: segfault at 120 ip 082dd95f sp bfff7fe0 error 4 in php5-cgi[8048000+4cf000] Jul 9 06:25:01 web kernel: Kernel logging (proc) stopped. Jul 9 06:25:02 web kernel: imklog 3.18.6, log source = /proc/kmsg started. Jul 9 06:25:02 web rsyslogd: [origin software="rsyslogd" swVersion="3.18.6" x-pid="2120" x-info="http://www.rsyslog.com"] restart Jul 10 06:25:01 web kernel: Kernel logging (proc) stopped. Jul 10 06:25:01 web kernel: imklog 3.18.6, log source = /proc/kmsg started. Jul 10 06:25:01 web rsyslogd: [origin software="rsyslogd" swVersion="3.18.6" x-pid="2120" x-info="http://www.rsyslog.com"] restart Jul 11 04:02:12 web kernel: imklog 3.18.6, log source = /proc/kmsg started. Jul 11 04:02:12 web kernel: [ 0.000000] Initializing cgroup subsys cpuset Jul 11 04:02:12 web kernel: [ 0.000000] Initializing cgroup subsys cpu Jul 11 04:02:12 web kernel: [ 0.000000] Linux version 2.6.26-2-xen-686 (Debian 2.6.26-24) (dannf@debian.org) (gcc version 4.1.3 20080704 (prerelease) (Debian 4.1.2-25)) #1 SMP Mon Jun 21 10:37:05 UTC 2010 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users