Hello, Currently am using 2.6.19-1.2911.6.5.fc6xen version of the kernel because the newest version was giving us problems. However with this one we are running these domains: Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 2307 2 r----- 4520.9 mars 10 1511 1 -b---- 174.0 monitoring 14 229 1 -b---- 12.1 The problem we are experiencing is after about an hour the servers do not respond. We have to do an xm destroy and then create to get them to work again. Is this a known issue and what could be possible fixes for the problem? Thank you, Travis This e-mail and any files transmitted with it are for the sole use of Blue Slate Solutions and the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on this e-mail is strictly prohibited and may be unlawful. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Here is what I am seeing in the log files. When the kernel hung and the following error message was displayed on the domain''s console every 5 seconds. ======================BUG: soft lockup detected on CPU#0! [<c1044e8b>] softlockup_tick+0xaa/0xc1 [<c10087af>] timer_interrupt+0x552/0x59f [<c1029612>] lock_timer_base+0x15/0x2f [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c102971e>] __mod_timer+0x94/0x9e [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c1045101>] handle_IRQ_event+0x1e/0x47 [<c1046464>] handle_level_irq+0x93/0xdf [<c10463d1>] handle_level_irq+0x0/0xdf [<c10068d4>] do_IRQ+0xb5/0xdd [<c1215c95>] do_general_protection+0x11c/0x16f [<c114a90d>] evtchn_do_upcall+0x5f/0x97 [<c1005006>] hypervisor_callback+0x46/0x50 ======================BUG: soft lockup detected on CPU#0! [<c1044e8b>] softlockup_tick+0xaa/0xc1 [<c10087af>] timer_interrupt+0x552/0x59f [<c1029612>] lock_timer_base+0x15/0x2f [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c102971e>] __mod_timer+0x94/0x9e [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c1045101>] handle_IRQ_event+0x1e/0x47 [<c1046464>] handle_level_irq+0x93/0xdf [<c10463d1>] handle_level_irq+0x0/0xdf [<c10068d4>] do_IRQ+0xb5/0xdd [<c114a90d>] evtchn_do_upcall+0x5f/0x97 [<c100626f>] do_iret_error+0x0/0xb1 [<c1005006>] hypervisor_callback+0x46/0x50 [<c100626f>] do_iret_error+0x0/0xb1 [<c100628d>] do_iret_error+0x1e/0xb1 [<c1004ea5>] restore_nocheck_notrace+0xe/0xf [<c1006778>] do_softirq+0x1f/0xc6 [<c1006778>] do_softirq+0x1f/0xc6 [<c1006778>] do_softirq+0x1f/0xc6 [<c1004ea5>] restore_nocheck_notrace+0xe/0xf [<c103080b>] search_exception_tables+0x14/0x25 [<c1017f9f>] fixup_exception+0xb/0x20 [<c1215c95>] do_general_protection+0x11c/0x16f [<c114a90d>] evtchn_do_upcall+0x5f/0x97 [<c1005006>] hypervisor_callback+0x46/0x50 [<c1215b79>] do_general_protection+0x0/0x16f [<c100626f>] do_iret_error+0x0/0xb1 [<c121577d>] error_code+0x35/0x3c ====================== -----Original Message----- From: Formoso, Travis Sent: Monday, April 16, 2007 10:55 AM To: xen-users Subject: [Xen-users] Xen servers not responsive Hello, Currently am using 2.6.19-1.2911.6.5.fc6xen version of the kernel because the newest version was giving us problems. However with this one we are running these domains: Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 2307 2 r----- 4520.9 mars 10 1511 1 -b---- 174.0 monitoring 14 229 1 -b---- 12.1 The problem we are experiencing is after about an hour the servers do not respond. We have to do an xm destroy and then create to get them to work again. Is this a known issue and what could be possible fixes for the problem? Thank you, Travis This e-mail and any files transmitted with it are for the sole use of Blue Slate Solutions and the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on this e-mail is strictly prohibited and may be unlawful. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users This e-mail and any files transmitted with it are for the sole use of Blue Slate Solutions and the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on this e-mail is strictly prohibited and may be unlawful. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I had the same problem on a previous server. I ultimately replaced the CPU and I think that solved the problem. Jon Formoso, Travis wrote:> Here is what I am seeing in the log files. When the kernel hung and the following error message was displayed on the domain''s console every 5 seconds. > > ======================> BUG: soft lockup detected on CPU#0! > [<c1044e8b>] softlockup_tick+0xaa/0xc1 > [<c10087af>] timer_interrupt+0x552/0x59f > [<c1029612>] lock_timer_base+0x15/0x2f > [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 > [<c102971e>] __mod_timer+0x94/0x9e > [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 > [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 > [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 > [<c1045101>] handle_IRQ_event+0x1e/0x47 > [<c1046464>] handle_level_irq+0x93/0xdf > [<c10463d1>] handle_level_irq+0x0/0xdf > [<c10068d4>] do_IRQ+0xb5/0xdd > [<c1215c95>] do_general_protection+0x11c/0x16f > [<c114a90d>] evtchn_do_upcall+0x5f/0x97 > [<c1005006>] hypervisor_callback+0x46/0x50 > ======================> BUG: soft lockup detected on CPU#0! > [<c1044e8b>] softlockup_tick+0xaa/0xc1 > [<c10087af>] timer_interrupt+0x552/0x59f > [<c1029612>] lock_timer_base+0x15/0x2f > [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 > [<c102971e>] __mod_timer+0x94/0x9e > [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 > [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 > [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 > [<c1045101>] handle_IRQ_event+0x1e/0x47 > [<c1046464>] handle_level_irq+0x93/0xdf > [<c10463d1>] handle_level_irq+0x0/0xdf > [<c10068d4>] do_IRQ+0xb5/0xdd > [<c114a90d>] evtchn_do_upcall+0x5f/0x97 > [<c100626f>] do_iret_error+0x0/0xb1 > [<c1005006>] hypervisor_callback+0x46/0x50 > [<c100626f>] do_iret_error+0x0/0xb1 > [<c100628d>] do_iret_error+0x1e/0xb1 > [<c1004ea5>] restore_nocheck_notrace+0xe/0xf > [<c1006778>] do_softirq+0x1f/0xc6 > [<c1006778>] do_softirq+0x1f/0xc6 > [<c1006778>] do_softirq+0x1f/0xc6 > [<c1004ea5>] restore_nocheck_notrace+0xe/0xf > [<c103080b>] search_exception_tables+0x14/0x25 > [<c1017f9f>] fixup_exception+0xb/0x20 > [<c1215c95>] do_general_protection+0x11c/0x16f > [<c114a90d>] evtchn_do_upcall+0x5f/0x97 > [<c1005006>] hypervisor_callback+0x46/0x50 > [<c1215b79>] do_general_protection+0x0/0x16f > [<c100626f>] do_iret_error+0x0/0xb1 > [<c121577d>] error_code+0x35/0x3c > ======================> > -----Original Message----- > From: Formoso, Travis > Sent: Monday, April 16, 2007 10:55 AM > To: xen-users > Subject: [Xen-users] Xen servers not responsive > > Hello, > > Currently am using 2.6.19-1.2911.6.5.fc6xen version of the kernel because the newest version was giving us problems. However with this one we are running these domains: > > Name ID Mem(MiB) VCPUs State Time(s) > Domain-0 0 2307 2 r----- 4520.9 > mars 10 1511 1 -b---- 174.0 > monitoring 14 229 1 -b---- 12.1 > > The problem we are experiencing is after about an hour the servers do not respond. We have to do an xm destroy and then create to get them to work again. Is this a known issue and what could be possible fixes for the problem? > > Thank you, > > Travis > > > > > This e-mail and any files transmitted with it are for the sole use of > Blue Slate Solutions and the intended recipient(s) and may contain > confidential and privileged information. If you are not the intended > recipient, please contact the sender by reply e-mail and destroy all > copies of the original message. Any unauthorized review, use, > disclosure, dissemination, forwarding, printing or copying of this email > or any action taken in reliance on this e-mail is strictly prohibited > and may be unlawful. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > > > > > This e-mail and any files transmitted with it are for the sole use of > Blue Slate Solutions and the intended recipient(s) and may contain > confidential and privileged information. If you are not the intended > recipient, please contact the sender by reply e-mail and destroy all > copies of the original message. Any unauthorized review, use, > disclosure, dissemination, forwarding, printing or copying of this email > or any action taken in reliance on this e-mail is strictly prohibited > and may be unlawful. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> I had the same problem on a previous server. I ultimately replaced the CPU and I think that solved the problem.> JonI hope I do not have to do this as this is a brand new server! I did see a bug related http://lists.xensource.com/archives/html/xen-bugs/2007-02/msg00074.html Says to go to a different kernel version - might give that a shot - however if anyone else has had this problem and has a fix let me know. I will also check out the CPU and see if it is a hardware related problem. Thanks, Travis Formoso, Travis wrote: Here is what I am seeing in the log files. When the kernel hung and the following error message was displayed on the domain''s console every 5 seconds. ======================BUG: soft lockup detected on CPU#0! [<c1044e8b>] softlockup_tick+0xaa/0xc1 [<c10087af>] timer_interrupt+0x552/0x59f [<c1029612>] lock_timer_base+0x15/0x2f [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c102971e>] __mod_timer+0x94/0x9e [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c1045101>] handle_IRQ_event+0x1e/0x47 [<c1046464>] handle_level_irq+0x93/0xdf [<c10463d1>] handle_level_irq+0x0/0xdf [<c10068d4>] do_IRQ+0xb5/0xdd [<c1215c95>] do_general_protection+0x11c/0x16f [<c114a90d>] evtchn_do_upcall+0x5f/0x97 [<c1005006>] hypervisor_callback+0x46/0x50 ======================BUG: soft lockup detected on CPU#0! [<c1044e8b>] softlockup_tick+0xaa/0xc1 [<c10087af>] timer_interrupt+0x552/0x59f [<c1029612>] lock_timer_base+0x15/0x2f [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c102971e>] __mod_timer+0x94/0x9e [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c121564f>] _spin_unlock_irqrestore+0x8/0x16 [<c1045101>] handle_IRQ_event+0x1e/0x47 [<c1046464>] handle_level_irq+0x93/0xdf [<c10463d1>] handle_level_irq+0x0/0xdf [<c10068d4>] do_IRQ+0xb5/0xdd [<c114a90d>] evtchn_do_upcall+0x5f/0x97 [<c100626f>] do_iret_error+0x0/0xb1 [<c1005006>] hypervisor_callback+0x46/0x50 [<c100626f>] do_iret_error+0x0/0xb1 [<c100628d>] do_iret_error+0x1e/0xb1 [<c1004ea5>] restore_nocheck_notrace+0xe/0xf [<c1006778>] do_softirq+0x1f/0xc6 [<c1006778>] do_softirq+0x1f/0xc6 [<c1006778>] do_softirq+0x1f/0xc6 [<c1004ea5>] restore_nocheck_notrace+0xe/0xf [<c103080b>] search_exception_tables+0x14/0x25 [<c1017f9f>] fixup_exception+0xb/0x20 [<c1215c95>] do_general_protection+0x11c/0x16f [<c114a90d>] evtchn_do_upcall+0x5f/0x97 [<c1005006>] hypervisor_callback+0x46/0x50 [<c1215b79>] do_general_protection+0x0/0x16f [<c100626f>] do_iret_error+0x0/0xb1 [<c121577d>] error_code+0x35/0x3c ====================== -----Original Message----- From: Formoso, Travis Sent: Monday, April 16, 2007 10:55 AM To: xen-users Subject: [Xen-users] Xen servers not responsive Hello, Currently am using 2.6.19-1.2911.6.5.fc6xen version of the kernel because the newest version was giving us problems. However with this one we are running these domains: Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 2307 2 r----- 4520.9 mars 10 1511 1 -b---- 174.0 monitoring 14 229 1 -b---- 12.1 The problem we are experiencing is after about an hour the servers do not respond. We have to do an xm destroy and then create to get them to work again. Is this a known issue and what could be possible fixes for the problem? Thank you, Travis This e-mail and any files transmitted with it are for the sole use of Blue Slate Solutions and the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on this e-mail is strictly prohibited and may be unlawful. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users This e-mail and any files transmitted with it are for the sole use of Blue Slate Solutions and the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on this e-mail is strictly prohibited and may be unlawful. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users This e-mail and any files transmitted with it are for the sole use of Blue Slate Solutions and the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on this e-mail is strictly prohibited and may be unlawful. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users