Heya. :) Just wanted to second dom0 - crashing behavior. Due to the fact that I have had never the same problems with 2.0.7 (and at least the same CPU/network utilization -> HDD-utilization should be compareable) I can second that with 3.0.x the behavior of dom0 got bad. Normal uptime with 2.0.7 was up to 2 months; with 3.0.x its pending from 24 hours up to a week. 2 interesting things to mention: 1st: When the whole system had crashed (domains also had to be started new) the Xen-System stays up round about a week. When I just reboot the server (and thus Dom0 with frozen domU''s while reboot), server dies more often. 2.nd: I tried to create a simple 2 GB file using dd if=/dev/zero of... inside(!) a vm, and it froze the system right at the beginning of file creation. All VMs are running using files as virtual harddisks; so basically it are no processes that do anything "special". All running VMs are paravirtualized ones - no HVM guests. However, its weird that I can''t use swap inside a dom0 - you can bet your soul on the fact that as soon as you reduce the mem (192MB maybe) and you have processes that want to swap out, your Dom0 WILL crash - not right away, but soon enough. So basically if you want to increase Dom0 livetime, you gotta increase memory dedicated to dom0. Never had THESE issues with vanilla kernels (or as mentioned the 2.0.7 ones) Funny thing: with a Dom0-machine just staying ready for action and running a samba fileserver (well, its a backup server that could run a VM if needed where I didn''t want to put the sambaserver in a VM) I get the same behaviour - all 2 weeks the dom0 just gets unresponsible when you want to access hdd. Please, don''t start something like "well, its not supposed to run this or that inside a dom0" - I can''t export it to a VM as long as I can''t make sure I can access the data in the VM whenever I need them. I don''t want to do a Backup-VM-Recovery just to recover the other vm''s data... So for a small company you have to rely on some backup mechanisms or that sorta stuff. Paying 2-5000 dollar just to get a dom0 working that shouldn''t have these child illnesses (sorry to call them that way... I think its just a bug that needs to be fixed) So thats no insult on the Xen-Devs... just a plea to fix it. When I can help, lemme know. Regards, Bigfoot29 On So, 2007-03-04 at 11:19 +0100, xen-users-request@lists.xensource.com wrote:> dom0 freezes under high IO load_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 4 Mar 2007 at 15:12, bigfoot29 wrote:> Heya. :) > > Just wanted to second dom0 - crashing behavior. > > Due to the fact that I have had never the same problems with 2.0.7 (and > at least the same CPU/network utilization -> HDD-utilization should be > compareable) I can second that with 3.0.x the behavior of dom0 got bad. > > Normal uptime with 2.0.7 was up to 2 months; with 3.0.x its pending from > 24 hours up to a week.SLES10-x86_64: # uptime 8:39am up 97 days 17:02, 1 user, load average: 1.87, 1.83, 1.76> > 2 interesting things to mention: > 1st: When the whole system had crashed (domains also had to be started > new) the Xen-System stays up round about a week. When I just reboot the > server (and thus Dom0 with frozen domU''s while reboot), server dies more > often. > 2.nd: I tried to create a simple 2 GB file using dd if=/dev/zero of... > inside(!) a vm, and it froze the system right at the beginning of file > creation. > > All VMs are running using files as virtual harddisks; so basically it > are no processes that do anything "special".I''m using LVM LVs for virtual harddisks.> > All running VMs are paravirtualized ones - no HVM guests.Same here.> > However, its weird that I can''t use swap inside a dom0 - you can bet > your soul on the fact that as soon as you reduce the mem (192MB maybe) > and you have processes that want to swap out, your Dom0 WILL crash - not > right away, but soon enough.I can use swap in Dom0: # free -m total used free shared buffers cached Mem: 384 380 3 0 59 154 -/+ buffers/cache: 165 218 Swap: 20481 5 20476> > So basically if you want to increase Dom0 livetime, you gotta increase > memory dedicated to dom0.Probably there exists a critical minumum amount of RAM where Dom0 can run stable.> > Never had THESE issues with vanilla kernels (or as mentioned the 2.0.7 > ones) > > Funny thing: with a Dom0-machine just staying ready for action and > running a samba fileserver (well, its a backup server that could run a > VM if needed where I didn''t want to put the sambaserver in a VM) I get > the same behaviour - all 2 weeks the dom0 just gets unresponsible when > you want to access hdd.Why would you run a samba server in Dom0, and did you install the recent security updates for samba? Ulrich> > Please, don''t start something like "well, its not supposed to run this > or that inside a dom0" - I can''t export it to a VM as long as I can''t > make sure I can access the data in the VM whenever I need them. I don''t > want to do a Backup-VM-Recovery just to recover the other vm''s data... > So for a small company you have to rely on some backup mechanisms or > that sorta stuff. Paying 2-5000 dollar just to get a dom0 working that > shouldn''t have these child illnesses (sorry to call them that way... I > think its just a bug that needs to be fixed) > > So thats no insult on the Xen-Devs... just a plea to fix it. When I can > help, lemme know. > > > Regards, Bigfoot29 > > > On So, 2007-03-04 at 11:19 +0100, xen-users-request@lists.xensource.com > wrote: > > dom0 freezes under high IO load > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, 2007-03-05 at 08:44 +0100, Ulrich Windl wrote:> On 4 Mar 2007 at 15:12, bigfoot29 wrote: > > > Heya. :) > > > > Just wanted to second dom0 - crashing behavior. > > > > Due to the fact that I have had never the same problems with 2.0.7 (and > > at least the same CPU/network utilization -> HDD-utilization should be > > compareable) I can second that with 3.0.x the behavior of dom0 got bad. > > > > Normal uptime with 2.0.7 was up to 2 months; with 3.0.x its pending from > > 24 hours up to a week. > > SLES10-x86_64: > # uptime > 8:39am up 97 days 17:02, 1 user, load average: 1.87, 1.83, 1.76 >Same here except Ubuntu LTS root@horizon2:/proc/xen# uptime 03:00:01 up 111 days, 5:33, 2 users, load average: 0.00, 0.00, 0.00 root@horizon2:/proc/xen# pvscan PV /dev/sda VG raid1 lvm2 [1.82 TB / 733.39 GB free] Total: 1 [1.82 TB] / in use: 1 [1.82 TB] / in no VG: 0 [0 ] root@horizon2:/proc/xen# Everything is LVM backed on a 2TB raid5 array. Supporting 40+ bull-in-china-shop guests for 111 days without issue, all PV.> Why would you run a samba server in Dom0, and did you install the recent security > updates for samba? >Especially dom-0 with no swap :) Running any network FS without swap is suicide. This is [ I suspect one of many ] of the cause of the "child illnesses", not Xen.> Ulrich >Best, --Tim _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
bigfoot29@www.bios.kicks-ass.org
2007-Mar-05 11:08 UTC
[Xen-users] Re: dom0 freezes under high IO load
Hmmm... my settings aren''t so much different...
free -m
total used free shared buffers cached
Mem: 342 325 16 0 63 16
-/+ buffers/cache: 245 96
Swap: 486 0 486
uptime
11:45:34 up 1 day, 14:04, 1 user, load average: 0.00, 0.00, 0.00
uname -a
Linux domain.local 2.6.16.33-xen0-8 #4 SMP Sat Jan 20 22:16:05 CET 2007
x86_64 GNU/Linux
Samba is just running at the backup system, not at the mastersystem.
Load is empty cuz I am doing problem research at mastersystem (4 GB Ram).
Swap is not deactivated, but just boosts the crash risk in case the system
uses swap. The problem also persists over various CPUs (all AMD A64 AM2),
RAM and mainboards. - It just locks up sometimes without warning. (eg when
creating a big file inside a VM)
Are you using pre-built or individual kernels?
Memtest worked without any problems and heat isn''t the problem as well
(datacenter and temperature measurement didn''t bring temperatures that
should alert)
I would love to see the stability back I have had in the past, but I
don''t
know any more where to search for bugs. (hardware or software wise)
Regards, Bigfoot29
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users