Hello, I am currently cleaning up the APIC code for the sake of shutdown/reboot/crashdump and have a query about the (modified for brevity) snippet of code: uint64_t msr_content; rdmsrl(MSR_IA32_APICBASE, msr_content); msr_content |= MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD; msr_content = (uint32_t)msr_content; wrmsrl(MSR_IA32_APICBASE, msr_content); which is added into apic.c in changeset b622e411eef8, and has propagated elsewhere in the codebase during subsequent cleanups etc. The MP spec and x2apic spec states that bits [35:12] of MSR_IA32_APICBASE is the base APIC MMIO address. Is there reason why the code (almost always) clears the top 4 bits, or is it just an overlooked mistake? -- Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer T: +44 (0)1223 225 900, http://www.citrix.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>>> On 17.05.11 at 15:25, Andrew Cooper <andrew.cooper3@citrix.com> wrote: > Hello, > > I am currently cleaning up the APIC code for the sake of > shutdown/reboot/crashdump and have a query about the (modified for > brevity) snippet of code: > > uint64_t msr_content; > rdmsrl(MSR_IA32_APICBASE, msr_content); > msr_content |= MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD; > msr_content = (uint32_t)msr_content; > wrmsrl(MSR_IA32_APICBASE, msr_content); > > which is added into apic.c in changeset b622e411eef8, and has propagated > elsewhere in the codebase during subsequent cleanups etc. > > The MP spec and x2apic spec states that bits [35:12] of > MSR_IA32_APICBASE is the base APIC MMIO address. Is there reason why > the code (almost always) clears the top 4 bits, or is it just an > overlooked mistake?I think this is a benign mistake. Benign because I don''t think there is a meaningful (to Xen at least) number of systems that would not have their LAPIC at the default address (which fits in 32 bits). Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 17/05/11 14:43, Jan Beulich wrote:>>>> On 17.05.11 at 15:25, Andrew Cooper<andrew.cooper3@citrix.com> wrote: >> Hello, >> >> I am currently cleaning up the APIC code for the sake of >> shutdown/reboot/crashdump and have a query about the (modified for >> brevity) snippet of code: >> >> uint64_t msr_content; >> rdmsrl(MSR_IA32_APICBASE, msr_content); >> msr_content |= MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD; >> msr_content = (uint32_t)msr_content; >> wrmsrl(MSR_IA32_APICBASE, msr_content); >> >> which is added into apic.c in changeset b622e411eef8, and has propagated >> elsewhere in the codebase during subsequent cleanups etc. >> >> The MP spec and x2apic spec states that bits [35:12] of >> MSR_IA32_APICBASE is the base APIC MMIO address. Is there reason why >> the code (almost always) clears the top 4 bits, or is it just an >> overlooked mistake? > I think this is a benign mistake. Benign because I don''t think there is > a meaningful (to Xen at least) number of systems that would not > have their LAPIC at the default address (which fits in 32 bits). > > Jan >Ok - I will fix this up in my cleanup. -- Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer T: +44 (0)1223 225 900, http://www.citrix.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 2011-05-17 at 14:43 +0100, Jan Beulich wrote:> >>> On 17.05.11 at 15:25, Andrew Cooper <andrew.cooper3@citrix.com> wrote: > > Hello, > > > > I am currently cleaning up the APIC code for the sake of > > shutdown/reboot/crashdump and have a query about the (modified for > > brevity) snippet of code: > > > > uint64_t msr_content; > > rdmsrl(MSR_IA32_APICBASE, msr_content); > > msr_content |= MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD; > > msr_content = (uint32_t)msr_content; > > wrmsrl(MSR_IA32_APICBASE, msr_content); > > > > which is added into apic.c in changeset b622e411eef8, and has propagated > > elsewhere in the codebase during subsequent cleanups etc. > > > > The MP spec and x2apic spec states that bits [35:12] of > > MSR_IA32_APICBASE is the base APIC MMIO address. Is there reason why > > the code (almost always) clears the top 4 bits, or is it just an > > overlooked mistake? > > I think this is a benign mistake. Benign because I don''t think there is > a meaningful (to Xen at least) number of systems that would not > have their LAPIC at the default address (which fits in 32 bits).That "msr_content = (uint32_t)msr_content;" seems to be pretty deliberate, what else would it be trying to do? FWIW enable_x2apic in Linux seems to have a similar construct which throws away the top half of the MSR: void enable_x2apic(void) { int msr, msr2; rdmsr(MSR_IA32_APICBASE, msr, msr2); if (!(msr & X2APIC_ENABLE)) { printk("Enabling x2apic\n"); wrmsr(MSR_IA32_APICBASE, msr | X2APIC_ENABLE, 0); } } (FWIW the original Xen code in 17545:9fd00ff95068 looked a lot like this too, b622e411eef8 just switched to wrmsrl and preserved the clearing behaviour). Perhaps there is some errata? Google didn''t find one, but ... Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 17/05/11 14:59, Ian Campbell wrote:> On Tue, 2011-05-17 at 14:43 +0100, Jan Beulich wrote: >>>>> On 17.05.11 at 15:25, Andrew Cooper<andrew.cooper3@citrix.com> wrote: >>> Hello, >>> >>> I am currently cleaning up the APIC code for the sake of >>> shutdown/reboot/crashdump and have a query about the (modified for >>> brevity) snippet of code: >>> >>> uint64_t msr_content; >>> rdmsrl(MSR_IA32_APICBASE, msr_content); >>> msr_content |= MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD; >>> msr_content = (uint32_t)msr_content; >>> wrmsrl(MSR_IA32_APICBASE, msr_content); >>> >>> which is added into apic.c in changeset b622e411eef8, and has propagated >>> elsewhere in the codebase during subsequent cleanups etc. >>> >>> The MP spec and x2apic spec states that bits [35:12] of >>> MSR_IA32_APICBASE is the base APIC MMIO address. Is there reason why >>> the code (almost always) clears the top 4 bits, or is it just an >>> overlooked mistake? >> I think this is a benign mistake. Benign because I don''t think there is >> a meaningful (to Xen at least) number of systems that would not >> have their LAPIC at the default address (which fits in 32 bits). > That "msr_content = (uint32_t)msr_content;" seems to be pretty > deliberate, what else would it be trying to do? > > FWIW enable_x2apic in Linux seems to have a similar construct which > throws away the top half of the MSR: > > void enable_x2apic(void) > { > int msr, msr2; > > rdmsr(MSR_IA32_APICBASE, msr, msr2); > if (!(msr& X2APIC_ENABLE)) { > printk("Enabling x2apic\n"); > wrmsr(MSR_IA32_APICBASE, msr | X2APIC_ENABLE, 0); > } > } > > (FWIW the original Xen code in 17545:9fd00ff95068 looked a lot like this > too, b622e411eef8 just switched to wrmsrl and preserved the clearing > behaviour). > > Perhaps there is some errata? Google didn''t find one, but ... > > Ian. >I couldn''t find any errata which is why I asked here. Bits [63:36] are reserved so should be WriteAsZero - it is possible that whoever put it into Linux just wanted to zero the top bits and either missed the top 4 bits or decided that they would never be set. -- Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer T: +44 (0)1223 225 900, http://www.citrix.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>>> On 17.05.11 at 15:59, Ian Campbell <Ian.Campbell@citrix.com> wrote: > On Tue, 2011-05-17 at 14:43 +0100, Jan Beulich wrote: >> >>> On 17.05.11 at 15:25, Andrew Cooper <andrew.cooper3@citrix.com> wrote: >> > Hello, >> > >> > I am currently cleaning up the APIC code for the sake of >> > shutdown/reboot/crashdump and have a query about the (modified for >> > brevity) snippet of code: >> > >> > uint64_t msr_content; >> > rdmsrl(MSR_IA32_APICBASE, msr_content); >> > msr_content |= MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD; >> > msr_content = (uint32_t)msr_content; >> > wrmsrl(MSR_IA32_APICBASE, msr_content); >> > >> > which is added into apic.c in changeset b622e411eef8, and has propagated >> > elsewhere in the codebase during subsequent cleanups etc. >> > >> > The MP spec and x2apic spec states that bits [35:12] of >> > MSR_IA32_APICBASE is the base APIC MMIO address. Is there reason why >> > the code (almost always) clears the top 4 bits, or is it just an >> > overlooked mistake? >> >> I think this is a benign mistake. Benign because I don''t think there is >> a meaningful (to Xen at least) number of systems that would not >> have their LAPIC at the default address (which fits in 32 bits). > > That "msr_content = (uint32_t)msr_content;" seems to be pretty > deliberate, what else would it be trying to do? > > FWIW enable_x2apic in Linux seems to have a similar construct which > throws away the top half of the MSR:Surely the Xen code got cloned from the Linux one.> void enable_x2apic(void) > { > int msr, msr2; > > rdmsr(MSR_IA32_APICBASE, msr, msr2); > if (!(msr & X2APIC_ENABLE)) { > printk("Enabling x2apic\n"); > wrmsr(MSR_IA32_APICBASE, msr | X2APIC_ENABLE, 0); > } > } > > (FWIW the original Xen code in 17545:9fd00ff95068 looked a lot like this > too, b622e411eef8 just switched to wrmsrl and preserved the clearing > behaviour).This is what I assumed, and I''m sure it''s just for simplicity that 0 gets passed here. I don''t think it''s correct, the more that the actual base address doesn''t matter while in x2apic mode (but would matter when switching back to legacy mode e.g. for kexec). Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 17/05/2011 15:08, "Jan Beulich" <JBeulich@novell.com> wrote:>>> I think this is a benign mistake. Benign because I don''t think there is >>> a meaningful (to Xen at least) number of systems that would not >>> have their LAPIC at the default address (which fits in 32 bits). >> >> That "msr_content = (uint32_t)msr_content;" seems to be pretty >> deliberate, what else would it be trying to do? >> >> FWIW enable_x2apic in Linux seems to have a similar construct which >> throws away the top half of the MSR: > > Surely the Xen code got cloned from the Linux one.If you actually look at the change in 21669:b622e411, you can see that the above line is deliberate. The old code did wrmsr(msr, lo, 0), and clearing upper 32 bits before calling wrmsrl() is the equivalent of that. So this should not be ''cleaned up'' without understanding that old code (which almost certainly itself comes from Linux). And I should also warn that I will simply ignore gratuitous cleanups in the APIC code. The code is mostly Linux sourced, I''d rather we synced again than diverge further. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel