Xen devel - Feb 2009 - a quick question about _PAGE_GLOBAL flag in para virt mode

Hi,

I noticed that in adjust_guest_l1e()  in arch/x86/mm.c, _PAGE_GLOBAL flag is
set for guest''s non-kernel pages. Meanwhile, _PAGE_GUEST_KERNEL page
cannot
have the Global bit set.

What''s the purpose of setting Global bit for guest''s user
pages? AFAIK, when
the Global bit is set, it prevents those entries be flushed from TLB. So it
makes more sense to set it for guest''s kernel pages.

Below are the codes:

876 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L876>
#define adjust_guest_l1e
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=adjust_guest_l1e>(pl1e,
d)                                            \
877 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L877>
    do {
      \
878 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L878>
        if ( likely
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=likely>(l1e_get_flags
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=l1e_get_flags>((pl1e))
& _PAGE_PRESENT
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=_PAGE_PRESENT>) &&
          \
879 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L879>
             likely
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=likely>(!is_pv_32on64_domain
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=is_pv_32on64_domain>(d))
)                               \
880 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L880>
        {
      \
881 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L881>
            */* _PAGE_GUEST_KERNEL page cannot have the Global bit
set. */*    \
882 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L882>
            if ( (l1e_get_flags
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=l1e_get_flags>((pl1e))
& (_PAGE_GUEST_KERNEL
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=_PAGE_GUEST_KERNEL>|_PAGE_GLOBAL
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=_PAGE_GLOBAL>)) \
883 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L883>
                 == (_PAGE_GUEST_KERNEL
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=_PAGE_GUEST_KERNEL>|_PAGE_GLOBAL
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=_PAGE_GLOBAL>) )
              \
884 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L884>
                MEM_LOG
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=MEM_LOG>(*"Global bit
is set to kernel page %lx"*,              \
885 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L885>
                        l1e_get_pfn
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=l1e_get_pfn>((pl1e)));
                              \
886 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L886>
            if ( !(l1e_get_flags
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=l1e_get_flags>((pl1e))
& _PAGE_USER
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=_PAGE_USER>)
)                     \
887 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L887>
                l1e_add_flags
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=l1e_add_flags>((pl1e),
(_PAGE_GUEST_KERNEL
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=_PAGE_GUEST_KERNEL>|_PAGE_USER
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=_PAGE_USER>));      \
888 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L888>
            if ( !(l1e_get_flags
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=l1e_get_flags>((pl1e))
& _PAGE_GUEST_KERNEL
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=_PAGE_GUEST_KERNEL>) )
           \
889 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L889>
                l1e_add_flags
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=l1e_add_flags>((pl1e),
(_PAGE_GLOBAL
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=_PAGE_GLOBAL>|_PAGE_USER
<http://lxr.xensource.com/lxr/ident?a=x86_64;i=_PAGE_USER>));
  \
890 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L890>
        }
      \
891 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L891>
    } while ( 0 )
 <http://lxr.xensource.com/lxr/source/xen/arch/x86/mm.c?a=x86_64#L892>



Thanks
weiming


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

weiming wrote:
> Hi,
>
> I noticed that in adjust_guest_l1e()  in arch/x86/mm.c, _PAGE_GLOBAL 
> flag is set for guest''s non-kernel pages. Meanwhile, 
> _PAGE_GUEST_KERNEL page cannot have the Global bit set.
>
> What''s the purpose of setting Global bit for guest''s user
pages?
> AFAIK, when the Global bit is set, it prevents those entries be 
> flushed from TLB. So it makes more sense to set it for guest''s
kernel
> pages. 
In 64-bit, syscalls must go via Xen, which entails a cr3 reload.  This 
would normally flush the tlb, but since the usermode pages are global, 
they remain in the tlb during the user->xen->kernel->xen->user
transition.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Thanks, Jeremy

When a normal context switch happens (like process switch), those tlb
entries will still be flushed no matter if global bit is set or not, is it
correct?

why not set global bit for user kernel pages? I guess this is because when
domain switchs, all tlb entries will still be flushed, so it''s not
necessary
for user kernel pages. Is my understanding right?

Thanks
Weiming




On Sun, Feb 15, 2009 at 4:37 AM, Jeremy Fitzhardinge
<jeremy@goop.org>wrote:
> weiming wrote:
>
>> Hi,
>>
>> I noticed that in adjust_guest_l1e()  in arch/x86/mm.c, _PAGE_GLOBAL
flag
>> is set for guest''s non-kernel pages. Meanwhile,
_PAGE_GUEST_KERNEL page
>> cannot have the Global bit set.
>>
>> What''s the purpose of setting Global bit for guest''s
user pages? AFAIK,
>> when the Global bit is set, it prevents those entries be flushed from
TLB.
>> So it makes more sense to set it for guest''s kernel pages.
>>
>
> In 64-bit, syscalls must go via Xen, which entails a cr3 reload.  This
> would normally flush the tlb, but since the usermode pages are global, they
> remain in the tlb during the user->xen->kernel->xen->user
transition.
>
>   J
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

weiming wrote:
> When a normal context switch happens (like process switch), those tlb 
> entries will still be flushed no matter if global bit is set or not, 
> is it correct?
Things would go very wrong if it didn''t...  I haven''t checked
in detail,
but presumably when the kernel does a user cr3 reload, Xen will do a 
global tlb flush (or at least make sure one happens when needed).
> why not set global bit for user kernel pages? I guess this is because 
> when domain switchs, all tlb entries will still be flushed, so
it''s
> not necessary for user kernel pages. Is my understanding right?
64-bit Xen uses paging to protect the guest kernel from guest usermode, 
and protect Xen from the guest kernel, and as a result has to do a cr3 
reload for any user->kernel switch (which is why it has to go via Xen).  
We don''t want kernel mappings to be visible to usermode, so they have
to
only be in tlb while we''re actually running in guest kernel mode; 
setting global on the kernel mappings wouldn''t be useful because we 
always need to flush them anyway when returning to usermode.

Or to put it another way, it makes sense to set global on user mappings 
because they''re always mapped, regardless of what context
we''re in; the
only time they need to be flushed is either doing a user process context 
switch, or a vcpu context switch.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Sun, Feb 15, 2009 at 02:26:13PM -0800, Jeremy Fitzhardinge wrote:
> >why not set global bit for user kernel pages? I guess this is because 
> >when domain switchs, all tlb entries will still be flushed, so
it''s
> >not necessary for user kernel pages. Is my understanding right?
> 
> 64-bit Xen uses paging to protect the guest kernel from guest usermode, 
> and protect Xen from the guest kernel, and as a result has to do a cr3 
> reload for any user->kernel switch (which is why it has to go via Xen).
> We don''t want kernel mappings to be visible to usermode, so they
have to
> only be in tlb while we''re actually running in guest kernel mode; 
And here''s a real example of what happens when you don''t get
this right:

http://blogs.sun.com/levon/entry/xen_compatibility_with_solaris

regards
john

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Oh, right!  In 64-bit mode, both guest kernel and user processes are both in
ring 3, so we can''t pin the mapping of user kernel in TLB, otherwise,
user
code would have access to kernels. Thanks!



On Sun, Feb 15, 2009 at 5:26 PM, Jeremy Fitzhardinge
<jeremy@goop.org>wrote:
> weiming wrote:
>
>> When a normal context switch happens (like process switch), those tlb
>> entries will still be flushed no matter if global bit is set or not, is
it
>> correct?
>>
>
> Things would go very wrong if it didn''t...  I haven''t
checked in detail,
> but presumably when the kernel does a user cr3 reload, Xen will do a global
> tlb flush (or at least make sure one happens when needed).
>
>  why not set global bit for user kernel pages? I guess this is because when
>> domain switchs, all tlb entries will still be flushed, so it''s
not necessary
>> for user kernel pages. Is my understanding right?
>>
>
> 64-bit Xen uses paging to protect the guest kernel from guest usermode, and
> protect Xen from the guest kernel, and as a result has to do a cr3 reload
> for any user->kernel switch (which is why it has to go via Xen).  We
don''t
> want kernel mappings to be visible to usermode, so they have to only be in
> tlb while we''re actually running in guest kernel mode; setting
global on the
> kernel mappings wouldn''t be useful because we always need to flush
them
> anyway when returning to usermode.
>
> Or to put it another way, it makes sense to set global on user mappings
> because they''re always mapped, regardless of what context
we''re in; the only
> time they need to be flushed is either doing a user process context switch,
> or a vcpu context switch.
>
>   J
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi,

I develop windows pv driver make a WHQL test on it. Now i hit a problem 
when i pass WHQL NDISTest 6.5(manual). During this test, the user will 
be required to disconnect the network card, it is OK to unplug the 
network cable in physical environment. But in the virtual situation, it 
is difficult to simulate this situation.

I have an initial idea about how to realize this feature as following. 
Please point out if problems exist in my idea.

In the backend, adding a new value in xenstore, and  front end can watch 
this value to detect network connection status. When users want to 
disconnect the network card, they can change the value in the xenstore. 
Then front end get this notification by watcher of this value, and do 
more process to realize function of disconnection.

Waiting more suggestions to realize this feature.......

Thanks
Annie.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel