Andrew Cooper
2011-Aug-18 13:31 UTC
[Xen-devel] [PATCH] KEXEC: disable hpet legacy broadcasts earlier
On x2apic machines which booted in xapic mode, hpet_disable_legacy_broadcast() sends an event check IPI to all online processors. This leads to a protection fault as the genapic blindly pokes x2apic MSRs while the local apic is in xapic mode. One option is to change genapic when we shut down the local apic, but there are still problems with trying to IPI processors in the online processor map which are actually sitting in NMI loops Another option is to have each CPU take itself out of the online CPU map during the NMI shootdown. Realistically however, disabling hpet legacy broadcasts earlier in the kexec path is the easiest fix to the problem. Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> diff -r fc2be6cb89ad -r dcb3ed7b7dbd xen/arch/x86/crash.c --- a/xen/arch/x86/crash.c Tue Aug 16 15:05:55 2011 +0100 +++ b/xen/arch/x86/crash.c Thu Aug 18 14:30:32 2011 +0100 @@ -29,6 +29,7 @@ #include <asm/io_apic.h> #include <xen/iommu.h> #include <xen/pci.h> +#include <asm/hpet.h> static atomic_t waiting_for_crash_ipi; static unsigned int crashing_cpu; @@ -61,6 +62,9 @@ static void nmi_shootdown_cpus(void) local_irq_disable(); + if ( hpet_broadcast_is_available() ) + hpet_disable_legacy_broadcast(); + crashing_cpu = smp_processor_id(); local_irq_count(crashing_cpu) = 0; diff -r fc2be6cb89ad -r dcb3ed7b7dbd xen/arch/x86/machine_kexec.c --- a/xen/arch/x86/machine_kexec.c Tue Aug 16 15:05:55 2011 +0100 +++ b/xen/arch/x86/machine_kexec.c Thu Aug 18 14:30:32 2011 +0100 @@ -85,9 +85,6 @@ void machine_kexec(xen_kexec_image_t *im .limit = LAST_RESERVED_GDT_BYTE }; - if ( hpet_broadcast_is_available() ) - hpet_disable_legacy_broadcast(); - /* We are about to permenantly jump out of the Xen context into the kexec * purgatory code. We really dont want to be still servicing interupts. */ _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-Aug-18 13:47 UTC
Re: [Xen-devel] [PATCH] KEXEC: disable hpet legacy broadcasts earlier
On 18/08/2011 14:31, "Andrew Cooper" <andrew.cooper3@citrix.com> wrote:> On x2apic machines which booted in xapic mode, > hpet_disable_legacy_broadcast() sends an event check IPI to all online > processors. This leads to a protection fault as the genapic blindly > pokes x2apic MSRs while the local apic is in xapic mode. > > One option is to change genapic when we shut down the local apic, but > there are still problems with trying to IPI processors in the online > processor map which are actually sitting in NMI loops > > Another option is to have each CPU take itself out of the online CPU > map during the NMI shootdown. > > Realistically however, disabling hpet legacy broadcasts earlier in the > kexec path is the easiest fix to the problem.This for 4.1.2 as well I guess?> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> > > diff -r fc2be6cb89ad -r dcb3ed7b7dbd xen/arch/x86/crash.c > --- a/xen/arch/x86/crash.c Tue Aug 16 15:05:55 2011 +0100 > +++ b/xen/arch/x86/crash.c Thu Aug 18 14:30:32 2011 +0100 > @@ -29,6 +29,7 @@ > #include <asm/io_apic.h> > #include <xen/iommu.h> > #include <xen/pci.h> > +#include <asm/hpet.h> > > static atomic_t waiting_for_crash_ipi; > static unsigned int crashing_cpu; > @@ -61,6 +62,9 @@ static void nmi_shootdown_cpus(void) > > local_irq_disable(); > > + if ( hpet_broadcast_is_available() ) > + hpet_disable_legacy_broadcast(); > + > crashing_cpu = smp_processor_id(); > local_irq_count(crashing_cpu) = 0; > > diff -r fc2be6cb89ad -r dcb3ed7b7dbd xen/arch/x86/machine_kexec.c > --- a/xen/arch/x86/machine_kexec.c Tue Aug 16 15:05:55 2011 +0100 > +++ b/xen/arch/x86/machine_kexec.c Thu Aug 18 14:30:32 2011 +0100 > @@ -85,9 +85,6 @@ void machine_kexec(xen_kexec_image_t *im > .limit = LAST_RESERVED_GDT_BYTE > }; > > - if ( hpet_broadcast_is_available() ) > - hpet_disable_legacy_broadcast(); > - > /* We are about to permenantly jump out of the Xen context into the kexec > * purgatory code. We really dont want to be still servicing interupts. > */ > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Andrew Cooper
2011-Aug-18 13:48 UTC
Re: [Xen-devel] [PATCH] KEXEC: disable hpet legacy broadcasts earlier
On 18/08/11 14:47, Keir Fraser wrote:> On 18/08/2011 14:31, "Andrew Cooper" <andrew.cooper3@citrix.com> wrote: > >> On x2apic machines which booted in xapic mode, >> hpet_disable_legacy_broadcast() sends an event check IPI to all online >> processors. This leads to a protection fault as the genapic blindly >> pokes x2apic MSRs while the local apic is in xapic mode. >> >> One option is to change genapic when we shut down the local apic, but >> there are still problems with trying to IPI processors in the online >> processor map which are actually sitting in NMI loops >> >> Another option is to have each CPU take itself out of the online CPU >> map during the NMI shootdown. >> >> Realistically however, disabling hpet legacy broadcasts earlier in the >> kexec path is the easiest fix to the problem. > This for 4.1.2 as well I guess?Yes please ~Andrew>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> >> >> diff -r fc2be6cb89ad -r dcb3ed7b7dbd xen/arch/x86/crash.c >> --- a/xen/arch/x86/crash.c Tue Aug 16 15:05:55 2011 +0100 >> +++ b/xen/arch/x86/crash.c Thu Aug 18 14:30:32 2011 +0100 >> @@ -29,6 +29,7 @@ >> #include <asm/io_apic.h> >> #include <xen/iommu.h> >> #include <xen/pci.h> >> +#include <asm/hpet.h> >> >> static atomic_t waiting_for_crash_ipi; >> static unsigned int crashing_cpu; >> @@ -61,6 +62,9 @@ static void nmi_shootdown_cpus(void) >> >> local_irq_disable(); >> >> + if ( hpet_broadcast_is_available() ) >> + hpet_disable_legacy_broadcast(); >> + >> crashing_cpu = smp_processor_id(); >> local_irq_count(crashing_cpu) = 0; >> >> diff -r fc2be6cb89ad -r dcb3ed7b7dbd xen/arch/x86/machine_kexec.c >> --- a/xen/arch/x86/machine_kexec.c Tue Aug 16 15:05:55 2011 +0100 >> +++ b/xen/arch/x86/machine_kexec.c Thu Aug 18 14:30:32 2011 +0100 >> @@ -85,9 +85,6 @@ void machine_kexec(xen_kexec_image_t *im >> .limit = LAST_RESERVED_GDT_BYTE >> }; >> >> - if ( hpet_broadcast_is_available() ) >> - hpet_disable_legacy_broadcast(); >> - >> /* We are about to permenantly jump out of the Xen context into the kexec >> * purgatory code. We really dont want to be still servicing interupts. >> */ >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >-- Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer T: +44 (0)1223 225 900, http://www.citrix.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel