Xen devel - Mar 2008 - where is writable page table implemented?

Hi,

I want to read the code about the implementation of writable page table. But
I can''t find it in the source codes.

I assume that when a guest OS allocate a page table (e.g. when create a
process), it will tell hypervisor the address of the page table, then xen
will set permissions on that page to trap the writes from the guest OS. But
I can''t find such code.

Thanks,
Weiming


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 12:18 -0400 on 24 Mar (1206361139), weiming wrote:
> Hi,
> 
> I want to read the code about the implementation of writable page table.
But
> I can''t find it in the source codes.
xen/arch/x86/mm.c, the functions after the comment "Writable
Pagetables"
(ptwr_*)

The guest is responsible for revoking write access to its own
pagetables, but that is checked by the recursive pagetable walk started
by the MMUEXT_PIN_* hypercall.

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Citrix Systems (R&D) Ltd.
[Company #02300071, SL9 0DZ, UK.]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Tim,

Thanks a lot!

Now, I know that fixup_page_fault() -> ptwr_do_page_fault handles the write
attempts to a pagetable page.

But
1) how does the guest OS tell xen which page is a pagetable page?  What if
the guest doesn''t do so? I assume xen should check if a pagetable had
already been registered or not. But in do_mmuext_op(), there is no such
check before setting the cr3.

2) in ptwr_do_page_fault(), it should disconnect the page from page
directory and then mark the page as writable.  Where are they done? In
x86_emulate()?

Thanks in advance!
Weiming

 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c#L3661>



On Mon, Mar 24, 2008 at 12:25 PM, Tim Deegan <Tim.Deegan@citrix.com>
wrote:
> At 12:18 -0400 on 24 Mar (1206361139), weiming wrote:
> > Hi,
> >
> > I want to read the code about the implementation of writable page
table.
> But
> > I can''t find it in the source codes.
>
> xen/arch/x86/mm.c, the functions after the comment "Writable
Pagetables"
> (ptwr_*)
>
> The guest is responsible for revoking write access to its own
> pagetables, but that is checked by the recursive pagetable walk started
> by the MMUEXT_PIN_* hypercall.
>
> Cheers,
>
> Tim.
>
> --
> Tim Deegan <Tim.Deegan@citrix.com>
> Principal Software Engineer, Citrix Systems (R&D) Ltd.
> [Company #02300071, SL9 0DZ, UK.]
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 14:26 -0400 on 24 Mar (1206368762), weiming wrote:
> 1) how does the guest OS tell xen which page is a pagetable page? 
With an MMUEXT_OP hypercall, if I recall correctly.
> What if the guest doesn''t do so? 
Then it''s not allowed to use it as a pagetable.
> I assume xen should check if a pagetable had
> already been registered or not. But in do_mmuext_op(), there is no such
> check before setting the cr3.
In new_guest_cr3(), the call to get_page_and_type_from_pagenr() will try
to take a typed reference count on the page, which marks it as a
pagetable. 

The check for paging_mode_refcounts(d) is to say that if the guest is
running under full shadow pagetables/HAP, then we skip the usual
protection mechanisms and leave it to the paging-assistance code. 
> 2) in ptwr_do_page_fault(), it should disconnect the page from page
> directory and then mark the page as writable.  Where are they done? In
> x86_emulate()?
That''s not done any more.  Instead, x86_emulate() emulates the
instruction and verifies the result is OK. 

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Citrix Systems (R&D) Ltd.
[Company #02300071, SL9 0DZ, UK.]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> 2) in ptwr_do_page_fault(), it should disconnect the page from page
> directory and then mark the page as writable.  Where are they done? In
> x86_emulate()?
The disconnection was originally implemented so that guests could make further 
writes to that page without incurring any more traps into Xen.  The goal was 
to improve the performance of batch pagetable changes (e.g. when a fork() is 
being performed).

This approach was abandoned in favour of trapping every direct pagetable 
write, and having guests explicitly hypercall into Xen when they really do 
want to modify a batch of pages.  This reduced the overheads in the usual 
case when the guest is only modifying a small number of pagetable entries.

The new approach gives an overall performance benefit but is ABI compatible 
with guests from before this change (they just won''t get the batching).

Cheers,
Mark

-- 
Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Tim,

Very appreciate for your explanation.

Another question that puzzles me is:

with writable page table, why guest os still calls update_va_mapping to
update the pte? Is it because writable page table only traps updates to L4
level page table? So in order to update pud, pmd, we have to use
update_va_mapping? and what''s purpose of the hypercall Pin/unpin page
table
page?

Very appreciate!
Weiming





On Tue, Mar 25, 2008 at 6:43 AM, Tim Deegan <Tim.Deegan@citrix.com> wrote:
> At 14:26 -0400 on 24 Mar (1206368762), weiming wrote:
> > 1) how does the guest OS tell xen which page is a pagetable page?
>
> With an MMUEXT_OP hypercall, if I recall correctly.
>
> > What if the guest doesn''t do so?
>
> Then it''s not allowed to use it as a pagetable.
>
> > I assume xen should check if a pagetable had
> > already been registered or not. But in do_mmuext_op(), there is no
such
> > check before setting the cr3.
>
> In new_guest_cr3(), the call to get_page_and_type_from_pagenr() will try
> to take a typed reference count on the page, which marks it as a
> pagetable.
>
> The check for paging_mode_refcounts(d) is to say that if the guest is
> running under full shadow pagetables/HAP, then we skip the usual
> protection mechanisms and leave it to the paging-assistance code.
>
> > 2) in ptwr_do_page_fault(), it should disconnect the page from page
> > directory and then mark the page as writable.  Where are they done? In
> > x86_emulate()?
>
> That''s not done any more.  Instead, x86_emulate() emulates the
> instruction and verifies the result is OK.
>
> Tim.
>
> --
> Tim Deegan <Tim.Deegan@citrix.com>
> Principal Software Engineer, Citrix Systems (R&D) Ltd.
> [Company #02300071, SL9 0DZ, UK.]
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 19:43 -0400 on 30 Mar (1206906207), weiming wrote:
> with writable page table, why guest os still calls update_va_mapping to
> update the pte? 
It''s faster to call into Xen with an explicit modification, or a list
of
them, than to have xen trap and do a full x86 instruction emaulation to
figure out what you werere doing.

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Citrix Systems (R&D) Ltd.
[Company #02300071, SL9 0DZ, UK.]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Tim,

Do you know under which situation we use writable page table instead of
calling update_va_mapping?

And do you know the purpose of the counter defined in
page_info.u.in_use.type_info?
I understand that xen needs to keep track of the type of each page
(L1_page_table_page, L2_page_table_page, pinned, unpinned, validated, etc),
but why do we need to use the lower 26 bits of "type_info" as a
counter? (in
"get_page_type()", the counter is increased. )  Similarly,  get_page()
/
put_page() also increases/decreases the page.count_info. Could you please
give me some hint about the usage of such counters?

Thanks a lot!
Weiming


On Tue, Apr 1, 2008 at 4:38 AM, Tim Deegan <Tim.Deegan@citrix.com> wrote:
> At 19:43 -0400 on 30 Mar (1206906207), weiming wrote:
> > with writable page table, why guest os still calls update_va_mapping
to
> > update the pte?
>
> It''s faster to call into Xen with an explicit modification, or a
list of
> them, than to have xen trap and do a full x86 instruction emaulation to
> figure out what you werere doing.
>
> Cheers,
>
> Tim.
>
> --
> Tim Deegan <Tim.Deegan@citrix.com>
> Principal Software Engineer, Citrix Systems (R&D) Ltd.
> [Company #02300071, SL9 0DZ, UK.]
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 11:37 -0400 on 01 Apr (1207049874), weiming wrote:
> Do you know under which situation we use writable page table instead of
> calling update_va_mapping?
Not in any detail.
> And do you know the purpose of the counter defined in
> page_info.u.in_use.type_info?
> I understand that xen needs to keep track of the type of each page
> (L1_page_table_page, L2_page_table_page, pinned, unpinned, validated, etc),
> but why do we need to use the lower 26 bits of "type_info" as a
counter? (in
> "get_page_type()", the counter is increased. )  Similarly, 
get_page() /
> put_page() also increases/decreases the page.count_info. Could you please
> give me some hint about the usage of such counters?
You should read the comments at the top of xen/arch/x86/mm.c.

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Citrix Systems (R&D) Ltd.
[Company #02300071, SL9 0DZ, UK.]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel