Xen devel - Oct 2004 - domain_page tlb flushing problem?

I think I''ve noticed a tlb flushing bug in xen''s domain_page.c
stuff.
Most of this code was based on the Linux PKMAP code (as noted in the
comment at the top of domain_page.c), but that code does not have
the following tlb flushing bug.

The bug is basically that flush_all_ready_maps() doesn''t cause all
processors to flush their tlb''s.  Or at least, I can''t see
anything
that would cause them to.

Here''s the problem:
flush_all_ready_maps() can cause a map cache entry to get zero''ed
(i.e. permissions to that virtual address are reduced/removed),
but it only ever notifies the local processor''s TLB.  However,
all the processors share the map cache, and so there''s no guarantee
that a reduction in privilege in the page tables will be observed
by any other processor (since their TLBs are explicitly flushed).

map_domain_mem() tries to protect itself against this somewhat
by trying to detect if the shadow_map_idx[processor_id()] is greater
than the current map_idx (i.e. some other processor caused map_idx
to wrap around), but that''s not sufficient.  Another processor may
have caused over 1024 map_domain_mem() calls quite easily (validating
a bunch of new PTEs, for example).

Theoretically, a processor shouldn''t have touched any memory in the
map cache area "above" its current shadow_map_idx, and so ideally,
this test in map_domain_mem() might be thought to be sufficient, but
it isn''t.  Processors (starting with the P6, and certainly true with
P4) can create arbitrary speculative accesses to any cachable memory,
and cause TLB loads for such speculative addresses.  So arguing about
whatever
memory is actually touched by the code path executed by a processor
isn''t sufficient.  It''s all about what might have been
speculatively
touched.

I am quite new to the Xen source base, and if I''m missed something
fundamental, I''d be most appreciative if someone could help point me
in the right direction.  But so far, this just looks busted to me.

Comments?

Michael Fetterman



-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Yes, at the time I wrote this code I thought that it was safe because
no index > shadow_map_idx[cpu] could be in cpu''s TLB. I have since
learned better and now assume that, as soon as a PTE is accessible by
walking from CR3, that PTE''s current value can enter the TLB at any
time and stay there until a TLB flush or suitable INVLPG. For example,
the PTE can end up in the TLB even if the CPU never accesses that
virtual address range, and even after the page table is unmapped from
the page directory. :-(

I think that the guest pagetable update code is now safe under this
assumption, but I had not thought to check domain_page.c. I''ve now
checked in what I think is a suitable fix.

 Thanks, 
 Keir
> Here''s the problem:
> flush_all_ready_maps() can cause a map cache entry to get zero''ed
> (i.e. permissions to that virtual address are reduced/removed),
> but it only ever notifies the local processor''s TLB.  However,
> all the processors share the map cache, and so there''s no
guarantee
> that a reduction in privilege in the page tables will be observed
> by any other processor (since their TLBs are explicitly flushed).
> 
> map_domain_mem() tries to protect itself against this somewhat
> by trying to detect if the shadow_map_idx[processor_id()] is greater
> than the current map_idx (i.e. some other processor caused map_idx
> to wrap around), but that''s not sufficient.  Another processor may
> have caused over 1024 map_domain_mem() calls quite easily (validating
> a bunch of new PTEs, for example).
> 
> Theoretically, a processor shouldn''t have touched any memory in
the
> map cache area "above" its current shadow_map_idx, and so
ideally,
> this test in map_domain_mem() might be thought to be sufficient, but
> it isn''t.  Processors (starting with the P6, and certainly true
with
> P4) can create arbitrary speculative accesses to any cachable memory,
> and cause TLB loads for such speculative addresses.  So arguing about
> whatever
> memory is actually touched by the code path executed by a processor
> isn''t sufficient.  It''s all about what might have been
speculatively
> touched.
> 
> I am quite new to the Xen source base, and if I''m missed something
> fundamental, I''d be most appreciative if someone could help point
me
> in the right direction.  But so far, this just looks busted to me.

-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel