On Tue, Sep 13, 2016 at 06:52:19PM +0300, Andriy Gapon wrote:> On 13/09/2016 18:22, Konstantin Belousov wrote: > > Any access > > to the LAPIC registers page in x2APIC mode faults. > > Is this a fact? > I read the following in the specification: > > In x2APIC mode, the memory mapped interface is not available and any > access to the MMIO interface will behave similar to that of a legacy xAPIC > in globally disabled state. > > But I couldn't find what actually happens for the legacy xAPIC in globally > disabled state. For AMD processors it is documented that if xAPIC is disabled > then accessing the APIC memory range works the same as accessing the regular > memory. That can be different for Intel processors, of course.Look at the native_lapic_init(), very beginning of the function. If x2apic_mode is non-zero, lapic_map is cleared after DMAP page cache attributes are changed. Right now, if BIOS pass control to OS in x2APIC mode, thinks just work because no LAPIC accesses are done until native_lapic_enable_x2apic() is called. This just happens, I thought about more organized receipt of the current LAPIC mode. Issue is that decision for LAPIC mode is performed by madt enumerator which pref. should not read LAPIC config (too early). And it is not clear at all, what to do if there is reason to use xAPIC, as checked by madt_setup_local(), but we are in x2APIC mode already. Anyway, returning to the original issue, I do not believe that the hand-off on that machine occured in the x2APIC mode when X2APIC_OPT_OUT is specified. Kernel would fault in that case.
On Wed, Sep 14, 2016 at 02:36:34PM +0300, Konstantin Belousov wrote:> On Tue, Sep 13, 2016 at 06:52:19PM +0300, Andriy Gapon wrote: > > On 13/09/2016 18:22, Konstantin Belousov wrote: > > > Any access > > > to the LAPIC registers page in x2APIC mode faults. > > > > Is this a fact? > > I read the following in the specification: > > > > In x2APIC mode, the memory mapped interface is not available and any > > access to the MMIO interface will behave similar to that of a legacy xAPIC > > in globally disabled state. > > > > But I couldn't find what actually happens for the legacy xAPIC in globally > > disabled state. For AMD processors it is documented that if xAPIC is disabled > > then accessing the APIC memory range works the same as accessing the regular > > memory. That can be different for Intel processors, of course. > > Look at the native_lapic_init(), very beginning of the function. If > x2apic_mode is non-zero, lapic_map is cleared after DMAP page cache > attributes are changed. > > Right now, if BIOS pass control to OS in x2APIC mode, thinks just work > because no LAPIC accesses are done until native_lapic_enable_x2apic() is > called. This just happens, I thought about more organized receipt of the > current LAPIC mode. Issue is that decision for LAPIC mode is performed > by madt enumerator which pref. should not read LAPIC config (too early). > And it is not clear at all, what to do if there is reason to use xAPIC, > as checked by madt_setup_local(), but we are in x2APIC mode already. > > Anyway, returning to the original issue, I do not believe that the > hand-off on that machine occured in the x2APIC mode when X2APIC_OPT_OUT > is specified. Kernel would fault in that case.As I assume, other OS don't fault in this case.
On 14/09/2016 14:36, Konstantin Belousov wrote:> On Tue, Sep 13, 2016 at 06:52:19PM +0300, Andriy Gapon wrote: >> On 13/09/2016 18:22, Konstantin Belousov wrote: >>> Any access >>> to the LAPIC registers page in x2APIC mode faults. >> >> Is this a fact? >> I read the following in the specification: >> >> In x2APIC mode, the memory mapped interface is not available and any >> access to the MMIO interface will behave similar to that of a legacy xAPIC >> in globally disabled state. >> >> But I couldn't find what actually happens for the legacy xAPIC in globally >> disabled state. For AMD processors it is documented that if xAPIC is disabled >> then accessing the APIC memory range works the same as accessing the regular >> memory. That can be different for Intel processors, of course. > > Look at the native_lapic_init(), very beginning of the function. If > x2apic_mode is non-zero, lapic_map is cleared after DMAP page cache > attributes are changed.Right, but we are talking about the case where x2apic_mode *is zero* while the hardware in x2APIC mode.> Right now, if BIOS pass control to OS in x2APIC mode, thinks just work > because no LAPIC accesses are done until native_lapic_enable_x2apic() is > called. This just happens, I thought about more organized receipt of the > current LAPIC mode. Issue is that decision for LAPIC mode is performed > by madt enumerator which pref. should not read LAPIC config (too early). > And it is not clear at all, what to do if there is reason to use xAPIC, > as checked by madt_setup_local(), but we are in x2APIC mode already.Yes. As I mentioned earlier we should at least panic with an informative message. Maybe we could add some code to so something smarter later.> Anyway, returning to the original issue, I do not believe that the > hand-off on that machine occured in the x2APIC mode when X2APIC_OPT_OUT > is specified. Kernel would fault in that case.I think that it should be easy to check that if Slawa is still willing to try another diagnostic patch. diff --git a/sys/x86/x86/local_apic.c b/sys/x86/x86/local_apic.c index 203e9d00e8acc..37ac03fb9d811 100644 --- a/sys/x86/x86/local_apic.c +++ b/sys/x86/x86/local_apic.c @@ -429,6 +429,12 @@ native_lapic_init(vm_paddr_t addr) if (x2apic_mode) { native_lapic_enable_x2apic(); lapic_map = NULL; + } else if ((cpu_feature2 & CPUID2_X2APIC) != 0) { + r = rdmsr(MSR_APICBASE); + printf("MSR_APICBASE = 0x%16jx\n", (uintmax_t)r); + if ((r & (APICBASE_X2APIC | APICBASE_ENABLED)) =+ (APICBASE_X2APIC | APICBASE_ENABLED)) + printf("x2APIC is prohibited but turned on by BIOS\n"); } /* Setup the spurious interrupt handler. */ -- Andriy Gapon