Hi,
Been chasing down this message from guest boot:
(XEN) mm.c:1841:d1 Error pfn 7f36a: rd=ffff8300cea28080,
od=0000000000000000, caf=00000000, taf=0000000000000000
(XEN) mm.c:730:d1 Error getting mfn 7f36a (pfn 5555555555555555) from L1
entry 000000007f36a025 for dom1
(XEN) mm.c:3700:d1 ptwr_emulate: fixing up invalid PAE PTE
000000007f36a025
Firstly, on a >64GB system, looks like a 32bit guest can get mfn above 64G.
The above msg comes when the PV guest tries to do WP check. To that end,
it does set_pte for mapping a (some swapper) temp page in test_wp_bit():
__set_fixmap(FIX_WP_TEST, __pa_symbol(&swapper_pg_dir),
PAGE_READONLY);
boot_cpu_data.wp_works_ok = do_test_wp_bit();
clear_fixmap(FIX_WP_TEST);
...
/* use writable pagetables */
static inline void set_pte(pte_t *ptep, pte_t pte)
{
ptep->pte_high = pte.pte_high;
smp_wmb();
ptep->pte_low = pte.pte_low;
}
During the clear fixmap, the pte high write results in clearing upper
32bits portion of pte/mfn, as a result the pte low write results in
hypervisor getting wrong mfn, 7f36a instead of 1f7f36a.
I understand writeable page tables allow guest to do this, but I assume
they are for mapping user and not kernel pages, in which case we should
be doing a hypercall here? Or, would switching the order, first set low pte
then high pte work?
Thanks,
Mukesh
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On 14/04/2009 04:39, "Mukesh Rathor" <mukesh.rathor@oracle.com> wrote:> During the clear fixmap, the pte high write results in clearing upper > 32bits portion of pte/mfn, as a result the pte low write results in > hypervisor getting wrong mfn, 7f36a instead of 1f7f36a. > > I understand writeable page tables allow guest to do this, but I assume > they are for mapping user and not kernel pages, in which case we should > be doing a hypercall here? Or, would switching the order, first set low pte > then high pte work?Implementing clear_fixmap() with set_pte() is not correct, even on native. Since it clears high then low, it temporarily leaves you with a possibly invalid present PTE -- even on native this can cause problems if e.g., the invalid PTE maps uncacheable I/O memory. In our kernel we simply solved this by implementing __set_fixmap() with a hypercall that could update all 64 bits at once. An alternative is indeed to clear low then high. Basically, clearing a pte has to be done the opposite way round to setting a pte. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2009-Apr-14 15:59 UTC
Re: [Xen-devel] 32bit PAE PV guest on 64bit hypervisor
Mukesh Rathor wrote:> Hi, > > Been chasing down this message from guest boot: > > (XEN) mm.c:1841:d1 Error pfn 7f36a: rd=ffff8300cea28080, > od=0000000000000000, caf=00000000, taf=0000000000000000 > (XEN) mm.c:730:d1 Error getting mfn 7f36a (pfn 5555555555555555) from L1 > entry 000000007f36a025 for dom1 > (XEN) mm.c:3700:d1 ptwr_emulate: fixing up invalid PAE PTE > 000000007f36a025 > > Firstly, on a >64GB system, looks like a 32bit guest can get mfn above > 64G. > > The above msg comes when the PV guest tries to do WP check. To that end, > it does set_pte for mapping a (some swapper) temp page in test_wp_bit(): > > __set_fixmap(FIX_WP_TEST, __pa_symbol(&swapper_pg_dir), > PAGE_READONLY); > boot_cpu_data.wp_works_ok = do_test_wp_bit(); > clear_fixmap(FIX_WP_TEST); > > ... > /* use writable pagetables */ > static inline void set_pte(pte_t *ptep, pte_t pte) > { > ptep->pte_high = pte.pte_high; > smp_wmb(); > ptep->pte_low = pte.pte_low; > }What kernel version is this? J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Mukesh Rathor
2009-Apr-14 18:06 UTC
Re: [Xen-devel] 32bit PAE PV guest on 64bit hypervisor
Jeremy Fitzhardinge wrote:> Mukesh Rathor wrote: >> Hi, >> >> Been chasing down this message from guest boot: >> >> (XEN) mm.c:1841:d1 Error pfn 7f36a: rd=ffff8300cea28080, >> od=0000000000000000, caf=00000000, taf=0000000000000000 >> (XEN) mm.c:730:d1 Error getting mfn 7f36a (pfn 5555555555555555) from L1 >> entry 000000007f36a025 for dom1 >> (XEN) mm.c:3700:d1 ptwr_emulate: fixing up invalid PAE PTE >> 000000007f36a025 >> >> Firstly, on a >64GB system, looks like a 32bit guest can get mfn above >> 64G. >> >> The above msg comes when the PV guest tries to do WP check. To that end, >> it does set_pte for mapping a (some swapper) temp page in test_wp_bit(): >> >> __set_fixmap(FIX_WP_TEST, __pa_symbol(&swapper_pg_dir), >> PAGE_READONLY); >> boot_cpu_data.wp_works_ok = do_test_wp_bit(); >> clear_fixmap(FIX_WP_TEST); >> >> ... >> /* use writable pagetables */ >> static inline void set_pte(pte_t *ptep, pte_t pte) >> { >> ptep->pte_high = pte.pte_high; >> smp_wmb(); >> ptep->pte_low = pte.pte_low; >> } > > What kernel version is this? > > JWhile I was debugging 2.6.18-92, I see code is same on 2.6.18-128. The xen version, just in case, 3.3.1. Thanks, Mukesh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Mukesh Rathor
2009-Apr-23 01:57 UTC
Re: [Xen-devel] 32bit PAE PV guest on 64bit hypervisor
Keir Fraser wrote:> On 14/04/2009 04:39, "Mukesh Rathor" <mukesh.rathor@oracle.com> wrote: >..> Implementing clear_fixmap() with set_pte() is not correct, even on native. > Since it clears high then low, it temporarily leaves you with a possibly > invalid present PTE -- even on native this can cause problems if e.g., the > invalid PTE maps uncacheable I/O memory. > > In our kernel we simply solved this by implementing __set_fixmap() with a > hypercall that could update all 64 bits at once. An alternative is indeed to > clear low then high. Basically, clearing a pte has to be done the opposite > way round to setting a pte. > > -- KeirJust a quick update, I changed to hypercall and it worked. BTW, I also had to increase the __PHYSICAL_MASK_SHIFT in guest (to 40) as I''m on system with 128GB. With both changes in the 32bit PAE guest, it''s doing OK now. Thanks for the help. Mukesh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel