Zhang, Fengzhe
2011-Jan-21 12:21 UTC
[Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
x86:x2apic: Disable x2apic on x86-32 permanently x2apic initialization on x86_32 uses vcpu pointer before it is initialized. As x2apic is unlikely to be used on x86_32, this patch disables x2apic permanently on x86_32. It also asserts the sanity of vcpu pointer before dereference to prevent further misuse. Signed-off-by: Fengzhe Zhang <fengzhe.zhang@intel.com> diff -r 02c0af2bf280 xen/arch/x86/apic.c --- a/xen/arch/x86/apic.c Mon Jan 17 18:05:52 2011 +0000 +++ b/xen/arch/x86/apic.c Wed Jan 19 03:24:16 2011 -0500 @@ -961,6 +961,22 @@ if ( !cpu_has_x2apic ) return; +#ifdef __i386__ + clear_bit(X86_FEATURE_X2APIC, boot_cpu_data.x86_capability); + if (x2apic_enabled) { + uint64_t msr_content; + + rdmsrl(MSR_IA32_APICBASE, msr_content); + msr_content &= ~(MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD); + wrmsrl(MSR_IA32_APICBASE, msr_content); + msr_content |= MSR_IA32_APICBASE_ENABLE; + wrmsrl(MSR_IA32_APICBASE, msr_content); + x2apic_enabled = 0; + } + printk("x2APIC disabled permanently on x86_32.\n"); + return; +#endif + if ( !opt_x2apic ) { if ( !x2apic_enabled ) diff -r 02c0af2bf280 xen/arch/x86/x86_32/domain_page.c --- a/xen/arch/x86/x86_32/domain_page.c Mon Jan 17 18:05:52 2011 +0000 +++ b/xen/arch/x86/x86_32/domain_page.c Wed Jan 19 03:24:16 2011 -0500 @@ -53,6 +53,8 @@ perfc_incr(map_domain_page_count); v = mapcache_current_vcpu(); + /* Prevent vcpu pointer being used before initialize. */ + ASSERT((unsigned long)v != 0xfffff000); dcache = &v->domain->arch.mapcache; vcache = &v->arch.mapcache; _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-Jan-21 13:09 UTC
Re: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
Could you give more info on the bug, such as a backtrace? I wonder whether this might be better fixed in some more elegant way. Also I can''t eyeball where map_domain_page() is being called during x2apic setup. Thanks, Keir On 21/01/2011 12:21, "Zhang, Fengzhe" <fengzhe.zhang@intel.com> wrote:> x86:x2apic: Disable x2apic on x86-32 permanently > > x2apic initialization on x86_32 uses vcpu pointer before it is initialized. As > x2apic is unlikely to be used on x86_32, this patch disables x2apic > permanently on x86_32. It also asserts the sanity of vcpu pointer before > dereference to prevent further misuse. > > Signed-off-by: Fengzhe Zhang <fengzhe.zhang@intel.com> > > diff -r 02c0af2bf280 xen/arch/x86/apic.c > --- a/xen/arch/x86/apic.c Mon Jan 17 18:05:52 2011 +0000 > +++ b/xen/arch/x86/apic.c Wed Jan 19 03:24:16 2011 -0500 > @@ -961,6 +961,22 @@ > if ( !cpu_has_x2apic ) > return; > > +#ifdef __i386__ > + clear_bit(X86_FEATURE_X2APIC, boot_cpu_data.x86_capability); > + if (x2apic_enabled) { > + uint64_t msr_content; > + > + rdmsrl(MSR_IA32_APICBASE, msr_content); > + msr_content &= ~(MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD); > + wrmsrl(MSR_IA32_APICBASE, msr_content); > + msr_content |= MSR_IA32_APICBASE_ENABLE; > + wrmsrl(MSR_IA32_APICBASE, msr_content); > + x2apic_enabled = 0; > + } > + printk("x2APIC disabled permanently on x86_32.\n"); > + return; > +#endif > + > if ( !opt_x2apic ) > { > if ( !x2apic_enabled ) > diff -r 02c0af2bf280 xen/arch/x86/x86_32/domain_page.c > --- a/xen/arch/x86/x86_32/domain_page.c Mon Jan 17 18:05:52 2011 +0000 > +++ b/xen/arch/x86/x86_32/domain_page.c Wed Jan 19 03:24:16 2011 -0500 > @@ -53,6 +53,8 @@ > perfc_incr(map_domain_page_count); > > v = mapcache_current_vcpu(); > + /* Prevent vcpu pointer being used before initialize. */ > + ASSERT((unsigned long)v != 0xfffff000); > > dcache = &v->domain->arch.mapcache; > vcache = &v->arch.mapcache; > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Li, Xin
2011-Jan-21 15:19 UTC
RE: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
Below is original log. We found Xen hypervisor needs to map VT-d/x2apci/MSI/MSI-x resources using map_domain_page. And on 32bit, the map page pool is trending to be used up. Maybe we can fix it, but for large machines with many devices and CPUs, we probably will still meet the 32bit limitation. So for new features which need map page in Xen hypervisor, I tend to enable on 64 bit only. I''m not sure how important 32bit Xen hypervisor is, but I think 64 bit is becoming more and more popular. Simply disabling is not technically elegant, but for longer term, I think it''s the trend because it makes things simpler :). Comments? Thanks! -Xin root (hd0,2) Filesystem type is ext2fs, partition type 0x83 kernel (hd0,2)/boot/xen.gz dom0_mem=512M iommu=1 loglvl=all guest_loglvl=all un restricted_guest=1 console=com1 sync_console iommu=1 msi=1 conring_size=4M [Multiboot-elf, <0x100000:0x14433c:0x34cc4>, shtab=0x279078, entry=0x100000] module (hd0,2)/boot/vmlinuz-2.6-xen ro root=/dev/sda3 console=hvc0 earlyprintkxen ignore_loglevel pci=assign-busses [Multiboot-module @ 0x27a000, 0x3ede40 bytes] module (hd0,2)/boot/initrd-2.6-xen.img [Multiboot-module @ 0x668000, 0x369800 bytes] __ __ _ _ _ _ _ _ \ \/ /___ _ __ | || | / | _ _ _ __ ___| |_ __ _| |__ | | ___ \ // _ \ ''_ \ | || |_ | |__| | | | ''_ \/ __| __/ _` | ''_ \| |/ _ \ / \ __/ | | | |__ _|| |__| |_| | | | \__ \ || (_| | |_) | | __/ /_/\_\___|_| |_| |_|(_)_| \__,_|_| |_|___/\__\__,_|_.__/|_|\___| (XEN) Xen version 4.1-unstable (build@sh.intel.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) Sat Sep 11 06:03:28 CST 2010 (XEN) Latest ChangeSet: Fri Sep 10 19:06:33 2010 +0100 22132:3985fea87987 (XEN) Console output is synchronous. (XEN) Bootloader: GNU GRUB 0.97 (XEN) Command line: dom0_mem=512M iommu=1 loglvl=all guest_loglvl=all unrestricted_guest=1 console=com1 sync_console iommu=1 msi=1 conring_size=4M (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds (XEN) EDID info not retrieved because no DDC retrieval method detected (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009bc00 (usable) (XEN) 000000000009bc00 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 0000000020050000 (usable) (XEN) 0000000020050000 - 0000000020250000 (reserved) (XEN) 0000000020250000 - 0000000040000000 (usable) (XEN) 0000000040000000 - 0000000040200000 (reserved) (XEN) 0000000040200000 - 000000007f553000 (usable) (XEN) 000000007f553000 - 000000007f5e7000 (reserved) (XEN) 000000007f5e7000 - 000000007f7e7000 (ACPI NVS) (XEN) 000000007f7e7000 - 000000007f7ff000 (ACPI data) (XEN) 000000007f7ff000 - 000000007f800000 (usable) (XEN) 000000007f800000 - 0000000080000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec01000 (reserved) (XEN) 00000000fed10000 - 00000000fed14000 (reserved) (XEN) 00000000fed18000 - 00000000fed1a000 (reserved) (XEN) 00000000fed1c000 - 00000000fed20000 (reserved) (XEN) 00000000fee00000 - 00000000fee01000 (reserved) (XEN) 00000000ff980000 - 00000000ffc00000 (reserved) (XEN) 00000000ffd80000 - 0000000100000000 (reserved) (XEN) System RAM: 2032MB (2081724kB) (XEN) ACPI: RSDP 000F0410, 0024 (r2 INTEL) (XEN) ACPI: XSDT 7F7FDE18, 006C (r1 INTEL SNB-CPT 6222004 MSFT 10013) (XEN) ACPI: FACP 7F752D98, 00F4 (r4 INTEL SNB-CPT 6222004 MSFT 10013) (XEN) ACPI: DSDT 7F725018, EACA (r2 INTEL SNB-CPT 0 INTL 20100331) (XEN) ACPI: FACS 7F7E6D40, 0040 (XEN) ACPI: APIC 7F7FCF18, 0092 (r2 INTEL SNB-CPT 6222004 MSFT 10013) (XEN) ACPI: TCPA 7F79FD18, 0032 (r2 0 0) (XEN) ACPI: MCFG 7F79FC98, 003C (r1 INTEL SNDYBRDG 6222004 MSFT 97) (XEN) ACPI: HPET 7F79FC18, 0038 (r1 A M I PCHHPET 6222004 AMI. 3) (XEN) ACPI: SSDT 7F753018, 1068 (r1 TrmRef PtidDevc 1000 INTL 20100331) (XEN) ACPI: BOOT 7F79FB98, 0028 (r1 INTEL SNB-CPT 6222004 AMI 10013) (XEN) ACPI: SSDT 7F751018, 0996 (r1 PmRef CpuPm 3000 INTL 20100331) (XEN) ACPI: DMAR 7F752F18, 00B0 (r1 INTEL SNB 1 INTL 1) (XEN) No NUMA configuration found (XEN) Faking a node at 0000000000000000-000000007f800000 (XEN) Xen heap: 9MB (9716kB) (XEN) Domain heap initialised (XEN) found SMP MP-table at 000fcaa0 (XEN) DMI 2.6 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x408 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0] (XEN) ACPI: 32/64X FACS address mismatch in FADT - 7f79df40/000000007f7e6d40, using 32 (XEN) ACPI: wakeup_vec[7f79df4c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) (XEN) Processor #0 6:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) (XEN) Processor #2 6:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled) (XEN) Processor #4 6:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled) (XEN) Processor #6 6:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled) (XEN) Processor #1 6:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled) (XEN) Processor #3 6:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled) (XEN) Processor #5 6:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled) (XEN) Processor #7 6:10 APIC version 21 (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000 (XEN) Table is not found! (XEN) Using ACPI (MADT) for SMP configuration information (XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X (XEN) Early fatal page fault at e008:ff1da1c6 (cr2=ffffffb8, ec=0000) (XEN) Stack dump: ff279610 00000080 ff24bfb0 fffff000 00000000 ff24bdd4 ffffc000 00000000 ff1da1c6 0000e008 00010082 00000000 00000000 ff24bdd4 ff130e89 ff21828a 00000006 ffbfec64 ff279610 00bde0a0 00000000 ff24be04 ff1384f7 0007e95c 00000002 00000000 ff11d4ef 00000096 00000096 f73de0a0 ff279610 ff279638 ff2795b8 ff24be34 ff13c740 ff279568 00000004 c9008020 00000000 ff21ac1c ff2795e4 00000096 ff279568 ff21a7bc 00000008 ff24be54 ff13e038 ff2795b8 00010970 02fa0000 ff279910 ff279910 00000007 ff24be74 ff144b0a ff279910 ff24be74 ff232570 00000007 00000000 00100800 ff24bfa4 ff234bf7 ff21dc0c 00000080 00000008 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 0007bef0 00000000 00000000 ff220260 00000000 00000000 1f000000 00000000 00000000 ff07bc50 ff07bfb0 ff07bef0 00757640 00848000 00000015 ffffffff 00000000 00000000 00000000 00000000 00000000 00000000 7f7ff000 00000000 40000000 00000000 00000000 ff07bef0 00000000 00000000 00000000 00000000 00000007 00000007 00000008 000001a4 000001a4 ff07bee0 01000000 00000000 ffd80000 00000000 ffd80000 00000000 00000015 ffffffff ff07bf24 0000097d 00000000 00000000 00000000 00000008 0000006e 00000001 00000003 000002f8 00000000 00000000 00220d70 ff2737c0 00067ebc ff10006c 0007bfb0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 fffff000 00000000> -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Keir Fraser > Sent: Friday, January 21, 2011 9:09 PM > To: Zhang, Fengzhe; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently > > Could you give more info on the bug, such as a backtrace? I wonder whether > this might be better fixed in some more elegant way. Also I can''t eyeball > where map_domain_page() is being called during x2apic setup. > > Thanks, > Keir > > On 21/01/2011 12:21, "Zhang, Fengzhe" <fengzhe.zhang@intel.com> wrote: > > > x86:x2apic: Disable x2apic on x86-32 permanently > > > > x2apic initialization on x86_32 uses vcpu pointer before it is initialized. As > > x2apic is unlikely to be used on x86_32, this patch disables x2apic > > permanently on x86_32. It also asserts the sanity of vcpu pointer before > > dereference to prevent further misuse. > > > > Signed-off-by: Fengzhe Zhang <fengzhe.zhang@intel.com> > > > > diff -r 02c0af2bf280 xen/arch/x86/apic.c > > --- a/xen/arch/x86/apic.c Mon Jan 17 18:05:52 2011 +0000 > > +++ b/xen/arch/x86/apic.c Wed Jan 19 03:24:16 2011 -0500 > > @@ -961,6 +961,22 @@ > > if ( !cpu_has_x2apic ) > > return; > > > > +#ifdef __i386__ > > + clear_bit(X86_FEATURE_X2APIC, boot_cpu_data.x86_capability); > > + if (x2apic_enabled) { > > + uint64_t msr_content; > > + > > + rdmsrl(MSR_IA32_APICBASE, msr_content); > > + msr_content &= ~(MSR_IA32_APICBASE_ENABLE | > MSR_IA32_APICBASE_EXTD); > > + wrmsrl(MSR_IA32_APICBASE, msr_content); > > + msr_content |= MSR_IA32_APICBASE_ENABLE; > > + wrmsrl(MSR_IA32_APICBASE, msr_content); > > + x2apic_enabled = 0; > > + } > > + printk("x2APIC disabled permanently on x86_32.\n"); > > + return; > > +#endif > > + > > if ( !opt_x2apic ) > > { > > if ( !x2apic_enabled ) > > diff -r 02c0af2bf280 xen/arch/x86/x86_32/domain_page.c > > --- a/xen/arch/x86/x86_32/domain_page.c Mon Jan 17 18:05:52 2011 +0000 > > +++ b/xen/arch/x86/x86_32/domain_page.c Wed Jan 19 03:24:16 2011 -0500 > > @@ -53,6 +53,8 @@ > > perfc_incr(map_domain_page_count); > > > > v = mapcache_current_vcpu(); > > + /* Prevent vcpu pointer being used before initialize. */ > > + ASSERT((unsigned long)v != 0xfffff000); > > > > dcache = &v->domain->arch.mapcache; > > vcache = &v->arch.mapcache; > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-Jan-21 15:28 UTC
Re: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
Okay. I think after 4.1 is branched we should simply remove 32-bit host support entirely. There''s really no good reason for it to persist imo. -- Keir On 21/01/2011 15:19, "Li, Xin" <xin.li@intel.com> wrote:> Below is original log. We found Xen hypervisor needs to map > VT-d/x2apci/MSI/MSI-x resources using map_domain_page. And on 32bit, the map > page pool is trending to be used up. Maybe we can fix it, but for large > machines with many devices and CPUs, we probably will still meet the 32bit > limitation. So for new features which need map page in Xen hypervisor, I tend > to enable on 64 bit only. I''m not sure how important 32bit Xen hypervisor is, > but I think 64 bit is becoming more and more popular. Simply disabling is not > technically elegant, but for longer term, I think it''s the trend because it > makes things simpler :). Comments? > Thanks! > -Xin > > root (hd0,2) > Filesystem type is ext2fs, partition type 0x83 > kernel (hd0,2)/boot/xen.gz dom0_mem=512M iommu=1 loglvl=all guest_loglvl=all > un > restricted_guest=1 console=com1 sync_console iommu=1 msi=1 conring_size=4M > [Multiboot-elf, <0x100000:0x14433c:0x34cc4>, shtab=0x279078, > entry=0x100000] > module (hd0,2)/boot/vmlinuz-2.6-xen ro root=/dev/sda3 console=hvc0 > earlyprintk> xen ignore_loglevel pci=assign-busses > [Multiboot-module @ 0x27a000, 0x3ede40 bytes] > module (hd0,2)/boot/initrd-2.6-xen.img > [Multiboot-module @ 0x668000, 0x369800 bytes] > > __ __ _ _ _ _ _ _ > \ \/ /___ _ __ | || | / | _ _ _ __ ___| |_ __ _| |__ | | ___ > \ // _ \ ''_ \ | || |_ | |__| | | | ''_ \/ __| __/ _` | ''_ \| |/ _ \ > / \ __/ | | | |__ _|| |__| |_| | | | \__ \ || (_| | |_) | | __/ > /_/\_\___|_| |_| |_|(_)_| \__,_|_| |_|___/\__\__,_|_.__/|_|\___| > > (XEN) Xen version 4.1-unstable (build@sh.intel.com) (gcc version 4.1.2 > 20080704 (Red Hat 4.1.2-44)) Sat Sep 11 06:03:28 CST 2010 > (XEN) Latest ChangeSet: Fri Sep 10 19:06:33 2010 +0100 22132:3985fea87987 > (XEN) Console output is synchronous. > (XEN) Bootloader: GNU GRUB 0.97 > (XEN) Command line: dom0_mem=512M iommu=1 loglvl=all guest_loglvl=all > unrestricted_guest=1 console=com1 sync_console iommu=1 msi=1 conring_size=4M > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds > (XEN) EDID info not retrieved because no DDC retrieval method detected > (XEN) Disc information: > (XEN) Found 1 MBR signatures > (XEN) Found 1 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009bc00 (usable) > (XEN) 000000000009bc00 - 00000000000a0000 (reserved) > (XEN) 00000000000e0000 - 0000000000100000 (reserved) > (XEN) 0000000000100000 - 0000000020050000 (usable) > (XEN) 0000000020050000 - 0000000020250000 (reserved) > (XEN) 0000000020250000 - 0000000040000000 (usable) > (XEN) 0000000040000000 - 0000000040200000 (reserved) > (XEN) 0000000040200000 - 000000007f553000 (usable) > (XEN) 000000007f553000 - 000000007f5e7000 (reserved) > (XEN) 000000007f5e7000 - 000000007f7e7000 (ACPI NVS) > (XEN) 000000007f7e7000 - 000000007f7ff000 (ACPI data) > (XEN) 000000007f7ff000 - 000000007f800000 (usable) > (XEN) 000000007f800000 - 0000000080000000 (reserved) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec01000 (reserved) > (XEN) 00000000fed10000 - 00000000fed14000 (reserved) > (XEN) 00000000fed18000 - 00000000fed1a000 (reserved) > (XEN) 00000000fed1c000 - 00000000fed20000 (reserved) > (XEN) 00000000fee00000 - 00000000fee01000 (reserved) > (XEN) 00000000ff980000 - 00000000ffc00000 (reserved) > (XEN) 00000000ffd80000 - 0000000100000000 (reserved) > (XEN) System RAM: 2032MB (2081724kB) > (XEN) ACPI: RSDP 000F0410, 0024 (r2 INTEL) > (XEN) ACPI: XSDT 7F7FDE18, 006C (r1 INTEL SNB-CPT 6222004 MSFT 10013) > (XEN) ACPI: FACP 7F752D98, 00F4 (r4 INTEL SNB-CPT 6222004 MSFT 10013) > (XEN) ACPI: DSDT 7F725018, EACA (r2 INTEL SNB-CPT 0 INTL 20100331) > (XEN) ACPI: FACS 7F7E6D40, 0040 > (XEN) ACPI: APIC 7F7FCF18, 0092 (r2 INTEL SNB-CPT 6222004 MSFT 10013) > (XEN) ACPI: TCPA 7F79FD18, 0032 (r2 0 0) > (XEN) ACPI: MCFG 7F79FC98, 003C (r1 INTEL SNDYBRDG 6222004 MSFT 97) > (XEN) ACPI: HPET 7F79FC18, 0038 (r1 A M I PCHHPET 6222004 AMI. 3) > (XEN) ACPI: SSDT 7F753018, 1068 (r1 TrmRef PtidDevc 1000 INTL 20100331) > (XEN) ACPI: BOOT 7F79FB98, 0028 (r1 INTEL SNB-CPT 6222004 AMI 10013) > (XEN) ACPI: SSDT 7F751018, 0996 (r1 PmRef CpuPm 3000 INTL 20100331) > (XEN) ACPI: DMAR 7F752F18, 00B0 (r1 INTEL SNB 1 INTL 1) > (XEN) No NUMA configuration found > (XEN) Faking a node at 0000000000000000-000000007f800000 > (XEN) Xen heap: 9MB (9716kB) > (XEN) Domain heap initialised > (XEN) found SMP MP-table at 000fcaa0 > (XEN) DMI 2.6 present. > (XEN) Using APIC driver default > (XEN) ACPI: PM-Timer IO Port: 0x408 > (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0] > (XEN) ACPI: 32/64X FACS address mismatch in FADT - 7f79df40/000000007f7e6d40, > using 32 > (XEN) ACPI: wakeup_vec[7f79df4c], vec_size[20] > (XEN) ACPI: Local APIC address 0xfee00000 > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) > (XEN) Processor #0 6:10 APIC version 21 > (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) > (XEN) Processor #2 6:10 APIC version 21 > (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled) > (XEN) Processor #4 6:10 APIC version 21 > (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled) > (XEN) Processor #6 6:10 APIC version 21 > (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled) > (XEN) Processor #1 6:10 APIC version 21 > (XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled) > (XEN) Processor #3 6:10 APIC version 21 > (XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled) > (XEN) Processor #5 6:10 APIC version 21 > (XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled) > (XEN) Processor #7 6:10 APIC version 21 > (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1]) > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) > (XEN) ACPI: IRQ0 used by override. > (XEN) ACPI: IRQ2 used by override. > (XEN) ACPI: IRQ9 used by override. > (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs > (XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000 > (XEN) Table is not found! > (XEN) Using ACPI (MADT) for SMP configuration information > (XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X > (XEN) Early fatal page fault at e008:ff1da1c6 (cr2=ffffffb8, ec=0000) > (XEN) Stack dump: ff279610 00000080 ff24bfb0 fffff000 00000000 ff24bdd4 > ffffc000 00000000 > ff1da1c6 0000e008 00010082 00000000 00000000 ff24bdd4 ff130e89 ff21828a > 00000006 ffbfec64 ff279610 00bde0a0 00000000 ff24be04 ff1384f7 0007e95c > 00000002 00000000 ff11d4ef 00000096 00000096 f73de0a0 ff279610 ff279638 > ff2795b8 ff24be34 ff13c740 ff279568 00000004 c9008020 00000000 ff21ac1c > ff2795e4 00000096 ff279568 ff21a7bc 00000008 ff24be54 ff13e038 ff2795b8 > 00010970 02fa0000 ff279910 ff279910 00000007 ff24be74 ff144b0a ff279910 > ff24be74 ff232570 00000007 00000000 00100800 ff24bfa4 ff234bf7 ff21dc0c > 00000080 00000008 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 00000000 0007bef0 00000000 00000000 > ff220260 00000000 00000000 1f000000 00000000 00000000 ff07bc50 ff07bfb0 > ff07bef0 00757640 00848000 00000015 ffffffff 00000000 00000000 00000000 > 00000000 00000000 00000000 7f7ff000 00000000 40000000 00000000 00000000 > ff07bef0 00000000 00000000 00000000 00000000 00000007 00000007 00000008 > 000001a4 000001a4 ff07bee0 01000000 00000000 ffd80000 00000000 ffd80000 > 00000000 00000015 ffffffff ff07bf24 0000097d 00000000 00000000 00000000 > 00000008 0000006e 00000001 00000003 000002f8 00000000 00000000 00220d70 > ff2737c0 00067ebc ff10006c 0007bfb0 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 00000000 00000000 fffff000 00000000 > >> -----Original Message----- >> From: xen-devel-bounces@lists.xensource.com >> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Keir Fraser >> Sent: Friday, January 21, 2011 9:09 PM >> To: Zhang, Fengzhe; xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 >> permanently >> >> Could you give more info on the bug, such as a backtrace? I wonder whether >> this might be better fixed in some more elegant way. Also I can''t eyeball >> where map_domain_page() is being called during x2apic setup. >> >> Thanks, >> Keir >> >> On 21/01/2011 12:21, "Zhang, Fengzhe" <fengzhe.zhang@intel.com> wrote: >> >>> x86:x2apic: Disable x2apic on x86-32 permanently >>> >>> x2apic initialization on x86_32 uses vcpu pointer before it is initialized. >>> As >>> x2apic is unlikely to be used on x86_32, this patch disables x2apic >>> permanently on x86_32. It also asserts the sanity of vcpu pointer before >>> dereference to prevent further misuse. >>> >>> Signed-off-by: Fengzhe Zhang <fengzhe.zhang@intel.com> >>> >>> diff -r 02c0af2bf280 xen/arch/x86/apic.c >>> --- a/xen/arch/x86/apic.c Mon Jan 17 18:05:52 2011 +0000 >>> +++ b/xen/arch/x86/apic.c Wed Jan 19 03:24:16 2011 -0500 >>> @@ -961,6 +961,22 @@ >>> if ( !cpu_has_x2apic ) >>> return; >>> >>> +#ifdef __i386__ >>> + clear_bit(X86_FEATURE_X2APIC, boot_cpu_data.x86_capability); >>> + if (x2apic_enabled) { >>> + uint64_t msr_content; >>> + >>> + rdmsrl(MSR_IA32_APICBASE, msr_content); >>> + msr_content &= ~(MSR_IA32_APICBASE_ENABLE | >> MSR_IA32_APICBASE_EXTD); >>> + wrmsrl(MSR_IA32_APICBASE, msr_content); >>> + msr_content |= MSR_IA32_APICBASE_ENABLE; >>> + wrmsrl(MSR_IA32_APICBASE, msr_content); >>> + x2apic_enabled = 0; >>> + } >>> + printk("x2APIC disabled permanently on x86_32.\n"); >>> + return; >>> +#endif >>> + >>> if ( !opt_x2apic ) >>> { >>> if ( !x2apic_enabled ) >>> diff -r 02c0af2bf280 xen/arch/x86/x86_32/domain_page.c >>> --- a/xen/arch/x86/x86_32/domain_page.c Mon Jan 17 18:05:52 2011 +0000 >>> +++ b/xen/arch/x86/x86_32/domain_page.c Wed Jan 19 03:24:16 2011 -0500 >>> @@ -53,6 +53,8 @@ >>> perfc_incr(map_domain_page_count); >>> >>> v = mapcache_current_vcpu(); >>> + /* Prevent vcpu pointer being used before initialize. */ >>> + ASSERT((unsigned long)v != 0xfffff000); >>> >>> dcache = &v->domain->arch.mapcache; >>> vcache = &v->arch.mapcache; >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel >> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Li, Xin
2011-Jan-21 16:07 UTC
RE: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
> Okay. I think after 4.1 is branched we should simply remove 32-bit host > support entirely. There''s really no good reason for it to persist imo.You''re much more aggressive than me, but I totally agree :) Thanks! -Xin> On 21/01/2011 15:19, "Li, Xin" <xin.li@intel.com> wrote: > > > Below is original log. We found Xen hypervisor needs to map > > VT-d/x2apci/MSI/MSI-x resources using map_domain_page. And on 32bit, the > map > > page pool is trending to be used up. Maybe we can fix it, but for large > > machines with many devices and CPUs, we probably will still meet the 32bit > > limitation. So for new features which need map page in Xen hypervisor, I tend > > to enable on 64 bit only. I''m not sure how important 32bit Xen hypervisor is, > > but I think 64 bit is becoming more and more popular. Simply disabling is not > > technically elegant, but for longer term, I think it''s the trend because it > > makes things simpler :). Comments? > > Thanks! > > -Xin > > > > root (hd0,2) > > Filesystem type is ext2fs, partition type 0x83 > > kernel (hd0,2)/boot/xen.gz dom0_mem=512M iommu=1 loglvl=all guest_loglvl=all > > un > > restricted_guest=1 console=com1 sync_console iommu=1 msi=1 > conring_size=4M > > [Multiboot-elf, <0x100000:0x14433c:0x34cc4>, shtab=0x279078, > > entry=0x100000] > > module (hd0,2)/boot/vmlinuz-2.6-xen ro root=/dev/sda3 console=hvc0 > > earlyprintk> > xen ignore_loglevel pci=assign-busses > > [Multiboot-module @ 0x27a000, 0x3ede40 bytes] > > module (hd0,2)/boot/initrd-2.6-xen.img > > [Multiboot-module @ 0x668000, 0x369800 bytes] > > > > __ __ _ _ _ _ _ _ > > \ \/ /___ _ __ | || | / | _ _ _ __ ___| |_ __ _| |__ | | ___ > > \ // _ \ ''_ \ | || |_ | |__| | | | ''_ \/ __| __/ _` | ''_ \| |/ _ \ > > / \ __/ | | | |__ _|| |__| |_| | | | \__ \ || (_| | |_) | | __/ > > /_/\_\___|_| |_| |_|(_)_| \__,_|_| |_|___/\__\__,_|_.__/|_|\___| > > > > (XEN) Xen version 4.1-unstable (build@sh.intel.com) (gcc version 4.1.2 > > 20080704 (Red Hat 4.1.2-44)) Sat Sep 11 06:03:28 CST 2010 > > (XEN) Latest ChangeSet: Fri Sep 10 19:06:33 2010 +0100 22132:3985fea87987 > > (XEN) Console output is synchronous. > > (XEN) Bootloader: GNU GRUB 0.97 > > (XEN) Command line: dom0_mem=512M iommu=1 loglvl=all guest_loglvl=all > > unrestricted_guest=1 console=com1 sync_console iommu=1 msi=1 > conring_size=4M > > (XEN) Video information: > > (XEN) VGA is text mode 80x25, font 8x16 > > (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds > > (XEN) EDID info not retrieved because no DDC retrieval method detected > > (XEN) Disc information: > > (XEN) Found 1 MBR signatures > > (XEN) Found 1 EDD information structures > > (XEN) Xen-e820 RAM map: > > (XEN) 0000000000000000 - 000000000009bc00 (usable) > > (XEN) 000000000009bc00 - 00000000000a0000 (reserved) > > (XEN) 00000000000e0000 - 0000000000100000 (reserved) > > (XEN) 0000000000100000 - 0000000020050000 (usable) > > (XEN) 0000000020050000 - 0000000020250000 (reserved) > > (XEN) 0000000020250000 - 0000000040000000 (usable) > > (XEN) 0000000040000000 - 0000000040200000 (reserved) > > (XEN) 0000000040200000 - 000000007f553000 (usable) > > (XEN) 000000007f553000 - 000000007f5e7000 (reserved) > > (XEN) 000000007f5e7000 - 000000007f7e7000 (ACPI NVS) > > (XEN) 000000007f7e7000 - 000000007f7ff000 (ACPI data) > > (XEN) 000000007f7ff000 - 000000007f800000 (usable) > > (XEN) 000000007f800000 - 0000000080000000 (reserved) > > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > > (XEN) 00000000fec00000 - 00000000fec01000 (reserved) > > (XEN) 00000000fed10000 - 00000000fed14000 (reserved) > > (XEN) 00000000fed18000 - 00000000fed1a000 (reserved) > > (XEN) 00000000fed1c000 - 00000000fed20000 (reserved) > > (XEN) 00000000fee00000 - 00000000fee01000 (reserved) > > (XEN) 00000000ff980000 - 00000000ffc00000 (reserved) > > (XEN) 00000000ffd80000 - 0000000100000000 (reserved) > > (XEN) System RAM: 2032MB (2081724kB) > > (XEN) ACPI: RSDP 000F0410, 0024 (r2 INTEL) > > (XEN) ACPI: XSDT 7F7FDE18, 006C (r1 INTEL SNB-CPT 6222004 MSFT > 10013) > > (XEN) ACPI: FACP 7F752D98, 00F4 (r4 INTEL SNB-CPT 6222004 MSFT > 10013) > > (XEN) ACPI: DSDT 7F725018, EACA (r2 INTEL SNB-CPT 0 INTL > 20100331) > > (XEN) ACPI: FACS 7F7E6D40, 0040 > > (XEN) ACPI: APIC 7F7FCF18, 0092 (r2 INTEL SNB-CPT 6222004 MSFT > 10013) > > (XEN) ACPI: TCPA 7F79FD18, 0032 (r2 0 > 0) > > (XEN) ACPI: MCFG 7F79FC98, 003C (r1 INTEL SNDYBRDG 6222004 MSFT > 97) > > (XEN) ACPI: HPET 7F79FC18, 0038 (r1 A M I PCHHPET 6222004 AMI. > 3) > > (XEN) ACPI: SSDT 7F753018, 1068 (r1 TrmRef PtidDevc 1000 INTL 20100331) > > (XEN) ACPI: BOOT 7F79FB98, 0028 (r1 INTEL SNB-CPT 6222004 AMI > 10013) > > (XEN) ACPI: SSDT 7F751018, 0996 (r1 PmRef CpuPm 3000 INTL > 20100331) > > (XEN) ACPI: DMAR 7F752F18, 00B0 (r1 INTEL SNB 1 INTL > 1) > > (XEN) No NUMA configuration found > > (XEN) Faking a node at 0000000000000000-000000007f800000 > > (XEN) Xen heap: 9MB (9716kB) > > (XEN) Domain heap initialised > > (XEN) found SMP MP-table at 000fcaa0 > > (XEN) DMI 2.6 present. > > (XEN) Using APIC driver default > > (XEN) ACPI: PM-Timer IO Port: 0x408 > > (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0] > > (XEN) ACPI: 32/64X FACS address mismatch in FADT - > 7f79df40/000000007f7e6d40, > > using 32 > > (XEN) ACPI: wakeup_vec[7f79df4c], vec_size[20] > > (XEN) ACPI: Local APIC address 0xfee00000 > > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) > > (XEN) Processor #0 6:10 APIC version 21 > > (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) > > (XEN) Processor #2 6:10 APIC version 21 > > (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled) > > (XEN) Processor #4 6:10 APIC version 21 > > (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled) > > (XEN) Processor #6 6:10 APIC version 21 > > (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled) > > (XEN) Processor #1 6:10 APIC version 21 > > (XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled) > > (XEN) Processor #3 6:10 APIC version 21 > > (XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled) > > (XEN) Processor #5 6:10 APIC version 21 > > (XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled) > > (XEN) Processor #7 6:10 APIC version 21 > > (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1]) > > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 > > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) > > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) > > (XEN) ACPI: IRQ0 used by override. > > (XEN) ACPI: IRQ2 used by override. > > (XEN) ACPI: IRQ9 used by override. > > (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs > > (XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000 > > (XEN) Table is not found! > > (XEN) Using ACPI (MADT) for SMP configuration information > > (XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X > > (XEN) Early fatal page fault at e008:ff1da1c6 (cr2=ffffffb8, ec=0000) > > (XEN) Stack dump: ff279610 00000080 ff24bfb0 fffff000 00000000 ff24bdd4 > > ffffc000 00000000 > > ff1da1c6 0000e008 00010082 00000000 00000000 ff24bdd4 ff130e89 ff21828a > > 00000006 ffbfec64 ff279610 00bde0a0 00000000 ff24be04 ff1384f7 0007e95c > > 00000002 00000000 ff11d4ef 00000096 00000096 f73de0a0 ff279610 ff279638 > > ff2795b8 ff24be34 ff13c740 ff279568 00000004 c9008020 00000000 ff21ac1c > > ff2795e4 00000096 ff279568 ff21a7bc 00000008 ff24be54 ff13e038 ff2795b8 > > 00010970 02fa0000 ff279910 ff279910 00000007 ff24be74 ff144b0a ff279910 > > ff24be74 ff232570 00000007 00000000 00100800 ff24bfa4 ff234bf7 ff21dc0c > > 00000080 00000008 00000000 00000000 00000000 00000000 00000000 > 00000000 > > 00000000 00000000 00000000 00000000 00000000 0007bef0 00000000 > 00000000 > > ff220260 00000000 00000000 1f000000 00000000 00000000 ff07bc50 ff07bfb0 > > ff07bef0 00757640 00848000 00000015 ffffffff 00000000 00000000 00000000 > > 00000000 00000000 00000000 7f7ff000 00000000 40000000 00000000 00000000 > > ff07bef0 00000000 00000000 00000000 00000000 00000007 00000007 00000008 > > 000001a4 000001a4 ff07bee0 01000000 00000000 ffd80000 00000000 ffd80000 > > 00000000 00000015 ffffffff ff07bf24 0000097d 00000000 00000000 00000000 > > 00000008 0000006e 00000001 00000003 000002f8 00000000 00000000 > 00220d70 > > ff2737c0 00067ebc ff10006c 0007bfb0 00000000 00000000 00000000 00000000 > > 00000000 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 > > 00000000 00000000 00000000 00000000 00000000 00000000 fffff000 00000000 > > > >> -----Original Message----- > >> From: xen-devel-bounces@lists.xensource.com > >> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Keir Fraser > >> Sent: Friday, January 21, 2011 9:09 PM > >> To: Zhang, Fengzhe; xen-devel@lists.xensource.com > >> Subject: Re: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 > >> permanently > >> > >> Could you give more info on the bug, such as a backtrace? I wonder whether > >> this might be better fixed in some more elegant way. Also I can''t eyeball > >> where map_domain_page() is being called during x2apic setup. > >> > >> Thanks, > >> Keir > >> > >> On 21/01/2011 12:21, "Zhang, Fengzhe" <fengzhe.zhang@intel.com> wrote: > >> > >>> x86:x2apic: Disable x2apic on x86-32 permanently > >>> > >>> x2apic initialization on x86_32 uses vcpu pointer before it is initialized. > >>> As > >>> x2apic is unlikely to be used on x86_32, this patch disables x2apic > >>> permanently on x86_32. It also asserts the sanity of vcpu pointer before > >>> dereference to prevent further misuse. > >>> > >>> Signed-off-by: Fengzhe Zhang <fengzhe.zhang@intel.com> > >>> > >>> diff -r 02c0af2bf280 xen/arch/x86/apic.c > >>> --- a/xen/arch/x86/apic.c Mon Jan 17 18:05:52 2011 +0000 > >>> +++ b/xen/arch/x86/apic.c Wed Jan 19 03:24:16 2011 -0500 > >>> @@ -961,6 +961,22 @@ > >>> if ( !cpu_has_x2apic ) > >>> return; > >>> > >>> +#ifdef __i386__ > >>> + clear_bit(X86_FEATURE_X2APIC, boot_cpu_data.x86_capability); > >>> + if (x2apic_enabled) { > >>> + uint64_t msr_content; > >>> + > >>> + rdmsrl(MSR_IA32_APICBASE, msr_content); > >>> + msr_content &= ~(MSR_IA32_APICBASE_ENABLE | > >> MSR_IA32_APICBASE_EXTD); > >>> + wrmsrl(MSR_IA32_APICBASE, msr_content); > >>> + msr_content |= MSR_IA32_APICBASE_ENABLE; > >>> + wrmsrl(MSR_IA32_APICBASE, msr_content); > >>> + x2apic_enabled = 0; > >>> + } > >>> + printk("x2APIC disabled permanently on x86_32.\n"); > >>> + return; > >>> +#endif > >>> + > >>> if ( !opt_x2apic ) > >>> { > >>> if ( !x2apic_enabled ) > >>> diff -r 02c0af2bf280 xen/arch/x86/x86_32/domain_page.c > >>> --- a/xen/arch/x86/x86_32/domain_page.c Mon Jan 17 18:05:52 2011 +0000 > >>> +++ b/xen/arch/x86/x86_32/domain_page.c Wed Jan 19 03:24:16 2011 -0500 > >>> @@ -53,6 +53,8 @@ > >>> perfc_incr(map_domain_page_count); > >>> > >>> v = mapcache_current_vcpu(); > >>> + /* Prevent vcpu pointer being used before initialize. */ > >>> + ASSERT((unsigned long)v != 0xfffff000); > >>> > >>> dcache = &v->domain->arch.mapcache; > >>> vcache = &v->arch.mapcache; > >>> _______________________________________________ > >>> Xen-devel mailing list > >>> Xen-devel@lists.xensource.com > >>> http://lists.xensource.com/xen-devel > >> > >> > >> > >> _______________________________________________ > >> Xen-devel mailing list > >> Xen-devel@lists.xensource.com > >> http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Wei, Gang
2011-Jan-22 14:36 UTC
RE: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
Keir Fraser wrote on 2011-01-21:> Could you give more info on the bug, such as a backtrace? I wonder > whether this might be better fixed in some more elegant way. Also I > can''t eyeball where > map_domain_page() is being called during x2apic setup.Just FYI. x2apic_bsp_setup -> iommu_enable_IR -> enable_qinval-> alloc_pgtable_maddr-> __map_domain_page. Jimmy _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2011-Jan-31 09:58 UTC
Re: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
>>> On 21.01.11 at 13:21, "Zhang, Fengzhe" <fengzhe.zhang@intel.com> wrote: > --- a/xen/arch/x86/apic.c Mon Jan 17 18:05:52 2011 +0000 > +++ b/xen/arch/x86/apic.c Wed Jan 19 03:24:16 2011 -0500 > @@ -961,6 +961,22 @@ > if ( !cpu_has_x2apic ) > return; > > +#ifdef __i386__ > + clear_bit(X86_FEATURE_X2APIC, boot_cpu_data.x86_capability); > + if (x2apic_enabled) { > + uint64_t msr_content; > + > + rdmsrl(MSR_IA32_APICBASE, msr_content); > + msr_content &= ~(MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD); > + wrmsrl(MSR_IA32_APICBASE, msr_content); > + msr_content |= MSR_IA32_APICBASE_ENABLE; > + wrmsrl(MSR_IA32_APICBASE, msr_content); > + x2apic_enabled = 0; > + } > + printk("x2APIC disabled permanently on x86_32.\n"); > + return; > +#endif > + > if ( !opt_x2apic ) > { > if ( !x2apic_enabled )How does this (namely the x2apic_enabled part) play together with the selection of the APIC driver, which in this case would have happened quite a bit earlier (from generic_apic_probe())? I would therefore think that this change really belongs into check_x2apic_preenabled(). Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Wei, Gang
2011-Feb-09 06:57 UTC
RE: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
Jan Beulich wrote on 2011-01-31:> How does this (namely the x2apic_enabled part) play together with the > selection of the APIC driver, which in this case would have happened > quite a bit earlier (from generic_apic_probe())? > > I would therefore think that this change really belongs into > check_x2apic_preenabled().It is really a problem. But I do think we shall simply remove below line from check_x2apic_preenabled() fn: genapic = apic_x2apic_probe(); The same line exists in x2apic_bsp_setup() fn. Jimmy _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2011-Feb-09 08:02 UTC
RE: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
>>> On 09.02.11 at 07:57, "Wei, Gang" <gang.wei@intel.com> wrote: > Jan Beulich wrote on 2011-01-31: >> How does this (namely the x2apic_enabled part) play together with the >> selection of the APIC driver, which in this case would have happened >> quite a bit earlier (from generic_apic_probe())? >> >> I would therefore think that this change really belongs into >> check_x2apic_preenabled(). > > It is really a problem. But I do think we shall simply remove below line > from check_x2apic_preenabled() fn: > > genapic = apic_x2apic_probe(); > > The same line exists in x2apic_bsp_setup() fn.No, that would be exactly the wrong way round - we need genapic to be set early in case x2apic was pre-enabled (see -unstable c/s 22707). Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Wei, Gang
2011-Feb-09 08:43 UTC
RE: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
Jan Beulich wrote on 2011-02-09:>>>> On 09.02.11 at 07:57, "Wei, Gang" <gang.wei@intel.com> wrote: >> Jan Beulich wrote on 2011-01-31: >>> How does this (namely the x2apic_enabled part) play together with >>> the selection of the APIC driver, which in this case would have >>> happened quite a bit earlier (from generic_apic_probe())? >>> >>> I would therefore think that this change really belongs into >>> check_x2apic_preenabled(). >> >> It is really a problem. But I do think we shall simply remove below >> line from check_x2apic_preenabled() fn: >> >> genapic = apic_x2apic_probe(); >> The same line exists in x2apic_bsp_setup() fn. > > No, that would be exactly the wrong way round - we need genapic to be > set early in case x2apic was pre-enabled (see -unstable c/s 22707).I could not get the reason to set genapic as x2apic in case x2apic was pre-enabled while referring to c/s 22707. Genapic->xxx was never called before calling x2apic_bsp_setup(). Are you trying to keep consistence (x2apic pre-enabled, so genapic have to be x2apic as early as possible)? Jimmy _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2011-Feb-09 08:58 UTC
RE: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
>>> On 09.02.11 at 09:43, "Wei, Gang" <gang.wei@intel.com> wrote: > Jan Beulich wrote on 2011-02-09: >>>>> On 09.02.11 at 07:57, "Wei, Gang" <gang.wei@intel.com> wrote: >>> Jan Beulich wrote on 2011-01-31: >>>> How does this (namely the x2apic_enabled part) play together with >>>> the selection of the APIC driver, which in this case would have >>>> happened quite a bit earlier (from generic_apic_probe())? >>>> >>>> I would therefore think that this change really belongs into >>>> check_x2apic_preenabled(). >>> >>> It is really a problem. But I do think we shall simply remove below >>> line from check_x2apic_preenabled() fn: >>> >>> genapic = apic_x2apic_probe(); >>> The same line exists in x2apic_bsp_setup() fn. >> >> No, that would be exactly the wrong way round - we need genapic to be >> set early in case x2apic was pre-enabled (see -unstable c/s 22707). > > I could not get the reason to set genapic as x2apic in case x2apic was > pre-enabled while referring to c/s 22707. Genapic->xxx was never called before > calling x2apic_bsp_setup(). Are you trying to keep consistence (x2apic > pre-enabled, so genapic have to be x2apic as early as possible)?Without setting genapic here, generic_apic_probe() would set it to &apic_default, which is obviously wrong (even if it later gets overridden) - just look at the various APIC driver related messages that can result on such a system (if you''re trying to analyze a problem, seeing the APIC driver change perhaps multiple time in the log is certainly not helpful). Apart from that - what problem do you see with just moving the change you did into check_x2apic_preenabled()? We''re using below patch on 4.0.x as replacement/extension to your -unstable c/s 22789 (i.e. a patch against current -unstable would need to revert some of what your change did). Jan **************************************************** --- a/xen/arch/x86/apic.c +++ b/xen/arch/x86/apic.c @@ -958,6 +958,10 @@ void x2apic_setup(void) if ( !cpu_has_x2apic ) return; +#ifdef __i386__ + BUG(); +#else + if ( !opt_x2apic ) { if ( !x2apic_enabled ) @@ -1019,6 +1023,7 @@ restore_out: unmask_8259A(); out: +#endif /* !__i386__ */ if ( ioapic_entries ) free_ioapic_entries(ioapic_entries); } --- a/xen/arch/x86/genapic/x2apic.c +++ b/xen/arch/x86/genapic/x2apic.c @@ -25,6 +25,8 @@ #include <xen/smp.h> #include <asm/mach-default/mach_mpparse.h> +#ifndef __i386__ + static int x2apic_phys; /* By default we use logical cluster mode. */ boolean_param("x2apic_phys", x2apic_phys); @@ -137,6 +139,8 @@ const struct genapic *apic_x2apic_probe( return x2apic_phys ? &apic_x2apic_phys : &apic_x2apic_cluster; } +#endif /* !__i386__ */ + void __init check_x2apic_preenabled(void) { u32 lo, hi; @@ -149,7 +153,19 @@ void __init check_x2apic_preenabled(void if ( lo & MSR_IA32_APICBASE_EXTD ) { printk("x2APIC mode is already enabled by BIOS.\n"); +#ifndef __i386__ x2apic_enabled = 1; genapic = apic_x2apic_probe(); +#else + lo &= ~(MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD); + wrmsr(MSR_IA32_APICBASE, lo, hi); + lo |= MSR_IA32_APICBASE_ENABLE; + wrmsr(MSR_IA32_APICBASE, lo, hi); + printk("x2APIC disabled permanently on x86_32.\n"); +#endif } + +#ifdef __i386__ + clear_bit(X86_FEATURE_X2APIC, boot_cpu_data.x86_capability); +#endif } _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Wei, Gang
2011-Feb-09 09:14 UTC
RE: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
Jan Beulich wrote on 2011-02-09:>>>> On 09.02.11 at 09:43, "Wei, Gang" <gang.wei@intel.com> wrote: >> Jan Beulich wrote on 2011-02-09: >>>>>> On 09.02.11 at 07:57, "Wei, Gang" <gang.wei@intel.com> wrote: >>>> Jan Beulich wrote on 2011-01-31: >>>>> How does this (namely the x2apic_enabled part) play together with >>>>> the selection of the APIC driver, which in this case would have >>>>> happened quite a bit earlier (from generic_apic_probe())? >>>>> >>>>> I would therefore think that this change really belongs into >>>>> check_x2apic_preenabled(). >>>> >>>> It is really a problem. But I do think we shall simply remove >>>> below line from check_x2apic_preenabled() fn: >>>> >>>> genapic = apic_x2apic_probe(); The same line exists in >>>> x2apic_bsp_setup() fn. >>> >>> No, that would be exactly the wrong way round - we need genapic to >>> be set early in case x2apic was pre-enabled (see -unstable c/s 22707). >> >> I could not get the reason to set genapic as x2apic in case x2apic >> was pre-enabled while referring to c/s 22707. Genapic->xxx was never >> called before calling x2apic_bsp_setup(). Are you trying to keep >> consistence (x2apic pre-enabled, so genapic have to be x2apic as >> early as > possible)? > > Without setting genapic here, generic_apic_probe() would set it to > &apic_default, which is obviously wrong (even if it later gets > overridden) - just look at the various APIC driver related messages > that can result on such a system (if you''re trying to analyze a > problem, seeing the APIC driver change perhaps multiple time in the log is certainly not helpful). > > Apart from that - what problem do you see with just moving the change > you did into check_x2apic_preenabled()? We''re using below patch on > 4.0.x as replacement/extension to your -unstable c/s 22789 (i.e. a > patch against current -unstable would need to revert some of what your change did).I just want to make it simple. Both your reason and patch are nice. So could you please send a formal patch for 4.1? Jimmy _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel