Daniel De Graaf
2011-Jan-10 15:06 UTC
[Xen-devel] 2.6.37 dom0 under Xen 4.1 clocksource not working
When booting 2.6.37 (from kernel.org) under Xen 4.1, there are apparently issues with timer interrupts being delivered to the dom0. Repeatedly causing an interrupt (for example, the ACPI power button) will allow the system to boot, as does adding "clocksource=jiffies" to the kernel command line. This is only present under Xen, not when booting the same kernel on real hardware. When boot with clocksource=jiffies, the system will sometimes freeze for about 30 seconds and then continue; however, this could be related to kernel debugging that I have enabled. The only clock-related message in dmesg is: [ 153.483813] Clocksource tsc unstable (delta = -77312555973 ns) /sys/devices/system/clocksource/clocksource0/available_clocksource: xen jiffies tsc I can attach kernel config or other information as required. I have observed this on at least two platforms; the 2.6.32.x xen kernels do not exhibit this behavior. -- Daniel De Graaf National Security Agency _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2011-Jan-11 15:26 UTC
Re: [Xen-devel] 2.6.37 dom0 under Xen 4.1 clocksource not working
On Mon, Jan 10, 2011 at 10:06:19AM -0500, Daniel De Graaf wrote:> When booting 2.6.37 (from kernel.org) under Xen 4.1, there are > apparently issues with timer interrupts being delivered to the dom0. > Repeatedly causing an interrupt (for example, the ACPI power button) will > allow the system to boot, as does adding "clocksource=jiffies" to the > kernel command line. This is only present under Xen, not when booting the > same kernel on real hardware. > > When boot with clocksource=jiffies, the system will sometimes freeze > for about 30 seconds and then continue; however, this could be related > to kernel debugging that I have enabled. > > The only clock-related message in dmesg is: > [ 153.483813] Clocksource tsc unstable (delta = -77312555973 ns)I see it too.> > /sys/devices/system/clocksource/clocksource0/available_clocksource: > xen jiffies tsc > > I can attach kernel config or other information as required. I have > observed this on at least two platforms; the 2.6.32.x xen kernels do not > exhibit this behavior.I think we are just missing some of the Xen time code that is in 2.6.32 but has not been ported over to 2.6.37. I don''t remember the name of the patches thought.. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2011-Jan-26 15:40 UTC
Re: [Xen-devel] 2.6.37 dom0 under Xen 4.1 clocksource not working
On 01/10/2011 07:06 AM, Daniel De Graaf wrote:> When booting 2.6.37 (from kernel.org) under Xen 4.1, there are > apparently issues with timer interrupts being delivered to the dom0. > Repeatedly causing an interrupt (for example, the ACPI power button) will > allow the system to boot, as does adding "clocksource=jiffies" to the > kernel command line. This is only present under Xen, not when booting the > same kernel on real hardware.Ah, yes. This is from the cpuidle stuff, I think. Does booting with "idle=halt" work?> When boot with clocksource=jiffies, the system will sometimes freeze > for about 30 seconds and then continue; however, this could be related > to kernel debugging that I have enabled. > > The only clock-related message in dmesg is: > [ 153.483813] Clocksource tsc unstable (delta = -77312555973 ns) > > /sys/devices/system/clocksource/clocksource0/available_clocksource: > xen jiffies tsc > > I can attach kernel config or other information as required. I have > observed this on at least two platforms; the 2.6.32.x xen kernels do not > exhibit this behavior. >J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Daniel De Graaf
2011-Jan-26 16:11 UTC
Re: [Xen-devel] 2.6.37 dom0 under Xen 4.1 clocksource not working
On 01/26/2011 10:40 AM, Jeremy Fitzhardinge wrote:> On 01/10/2011 07:06 AM, Daniel De Graaf wrote: >> When booting 2.6.37 (from kernel.org) under Xen 4.1, there are >> apparently issues with timer interrupts being delivered to the dom0. >> Repeatedly causing an interrupt (for example, the ACPI power button) will >> allow the system to boot, as does adding "clocksource=jiffies" to the >> kernel command line. This is only present under Xen, not when booting the >> same kernel on real hardware. > > Ah, yes. This is from the cpuidle stuff, I think. Does booting with > "idle=halt" work? >Yes, clocksource=xen idle=halt works.>> When boot with clocksource=jiffies, the system will sometimes freeze >> for about 30 seconds and then continue; however, this could be related >> to kernel debugging that I have enabled. >> >> The only clock-related message in dmesg is: >> [ 153.483813] Clocksource tsc unstable (delta = -77312555973 ns) >> >> /sys/devices/system/clocksource/clocksource0/available_clocksource: >> xen jiffies tsc >> >> I can attach kernel config or other information as required. I have >> observed this on at least two platforms; the 2.6.32.x xen kernels do not >> exhibit this behavior. >>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2011-Jan-26 17:07 UTC
Re: [Xen-devel] 2.6.37 dom0 under Xen 4.1 clocksource not working
On Wed, 26 Jan 2011, Daniel De Graaf wrote:> On 01/26/2011 10:40 AM, Jeremy Fitzhardinge wrote: > > On 01/10/2011 07:06 AM, Daniel De Graaf wrote: > >> When booting 2.6.37 (from kernel.org) under Xen 4.1, there are > >> apparently issues with timer interrupts being delivered to the dom0. > >> Repeatedly causing an interrupt (for example, the ACPI power button) will > >> allow the system to boot, as does adding "clocksource=jiffies" to the > >> kernel command line. This is only present under Xen, not when booting the > >> same kernel on real hardware. > > > > Ah, yes. This is from the cpuidle stuff, I think. Does booting with > > "idle=halt" work? > > > > Yes, clocksource=xen idle=halt works.I have a testbox that can reproduce this problem. It seems to be caused by the cpuidle driver that overrides pm_idle unless boot_option_idle_override is set to something other than the default. This patch fixes the problem for me: --- diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c index a3d28a1..7e85979 100644 --- a/arch/x86/xen/setup.c +++ b/arch/x86/xen/setup.c @@ -353,6 +353,7 @@ void __init xen_arch_setup(void) boot_cpu_data.hlt_works_ok = 1; #endif pm_idle = default_idle; + boot_option_idle_override = IDLE_HALT; fiddle_vdso(); } _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2011-Jan-26 20:12 UTC
Re: [Xen-devel] 2.6.37 dom0 under Xen 4.1 clocksource not working
On Wed, Jan 26, 2011 at 05:07:27PM +0000, Stefano Stabellini wrote:> On Wed, 26 Jan 2011, Daniel De Graaf wrote: > > On 01/26/2011 10:40 AM, Jeremy Fitzhardinge wrote: > > > On 01/10/2011 07:06 AM, Daniel De Graaf wrote: > > >> When booting 2.6.37 (from kernel.org) under Xen 4.1, there are > > >> apparently issues with timer interrupts being delivered to the dom0. > > >> Repeatedly causing an interrupt (for example, the ACPI power button) will > > >> allow the system to boot, as does adding "clocksource=jiffies" to the > > >> kernel command line. This is only present under Xen, not when booting the > > >> same kernel on real hardware. > > > > > > Ah, yes. This is from the cpuidle stuff, I think. Does booting with > > > "idle=halt" work? > > > > > > > Yes, clocksource=xen idle=halt works. > > I have a testbox that can reproduce this problem. > It seems to be caused by the cpuidle driver that overrides pm_idle > unless boot_option_idle_override is set to something other than the > default. > This patch fixes the problem for me: > > --- > > diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c > index a3d28a1..7e85979 100644 > --- a/arch/x86/xen/setup.c > +++ b/arch/x86/xen/setup.c > @@ -353,6 +353,7 @@ void __init xen_arch_setup(void) > boot_cpu_data.hlt_works_ok = 1; > #endif > pm_idle = default_idle; > + boot_option_idle_override = IDLE_HALT;Is this how 2.6.32 did it? Or was there no cpuidle driver in 2.6.32 so we never had tried this? With this being overwridden to HALT does that mean the hypervisor cycles through the Cx states? What about using MWAIT instead? How does this impact AMD platforms?> > fiddle_vdso(); > } > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2011-Jan-27 01:18 UTC
Re: [Xen-devel] 2.6.37 dom0 under Xen 4.1 clocksource not working
On 01/26/2011 12:12 PM, Konrad Rzeszutek Wilk wrote:> Is this how 2.6.32 did it? Or was there no cpuidle driver in 2.6.32 so > we never had tried this?CPUIDLE is new.> With this being overwridden to HALT does that > mean the hypervisor cycles through the Cx states? What about using > MWAIT instead? How does this impact AMD platforms?"halt" in this case is the safe_halt pvop, so it ends up going into the appropriate hypercall. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel