Xen devel - May 2010 - Does xc_map_foreign_pages() update IOMMU tables?

Does the xc_map_foreign_pages() update also the IOMMU/VT-d tables so
that the mapped pages belong also to Dom0 (from the IOMMU point of view)?

It seems like it does indeed. We use this function in Qubes GUI
virtualization to map composition buffers from DomUs into Dom0. Then we
use XShmPutImage() in Dom0 to display them. We''ve been told [1] that
XShmPutImage() often uses DMA to copy pages to the video memory,
especially in case of accelerated drivers. And we verified that our GUI
works fine on VT-d systems e.g. with Intel drivers with h/w acceleration
enabled and with iommu=pv enabled. This suggests that
xc_map_foreign_pages() indeed must be adding mapped pages to the IOMMU
tables for Dom0, as otherwise XShmPutImage() would not be able to
DMA-read the contents of those pages, that originally belonged to some
other DomUs.

Am I missing something? If not, can somebody point me to the actual code
in Xen which does this update?

Thanks a lot!
joanna.

[1] http://groups.google.com/group/qubes-devel/msg/9d23cd9c26127ed7



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 13/05/2010 12:48, "Joanna Rutkowska"
<joanna@invisiblethingslab.com>
wrote:
> Does the xc_map_foreign_pages() update also the IOMMU/VT-d tables so
> that the mapped pages belong also to Dom0 (from the IOMMU point of view)?
Seeing as how xc_map_foreign_pages() maps pages into a particular
guest-process virtual address space, whereas IOMMU tables are a mapping from
guest-physical address space, it''s clear that x_m_f_p() cannot sensibly
be
having any effect on the IOMMU.

 -- Keir
> It seems like it does indeed. We use this function in Qubes GUI
> virtualization to map composition buffers from DomUs into Dom0. Then we
> use XShmPutImage() in Dom0 to display them. We''ve been told [1]
that
> XShmPutImage() often uses DMA to copy pages to the video memory,
> especially in case of accelerated drivers. And we verified that our GUI
> works fine on VT-d systems e.g. with Intel drivers with h/w acceleration
> enabled and with iommu=pv enabled. This suggests that
> xc_map_foreign_pages() indeed must be adding mapped pages to the IOMMU
> tables for Dom0, as otherwise XShmPutImage() would not be able to
> DMA-read the contents of those pages, that originally belonged to some
> other DomUs.
> 
> Am I missing something? If not, can somebody point me to the actual code
> in Xen which does this update?
> 
> Thanks a lot!
> joanna.
> 
> [1] http://groups.google.com/group/qubes-devel/msg/9d23cd9c26127ed7
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 05/13/2010 02:00 PM, Keir Fraser wrote:
> On 13/05/2010 12:48, "Joanna Rutkowska"
<joanna@invisiblethingslab.com>
> wrote:
> 
>> Does the xc_map_foreign_pages() update also the IOMMU/VT-d tables so
>> that the mapped pages belong also to Dom0 (from the IOMMU point of
view)?
> 
> Seeing as how xc_map_foreign_pages() maps pages into a particular
> guest-process virtual address space, whereas IOMMU tables are a mapping
from
> guest-physical address space, it''s clear that x_m_f_p() cannot
sensibly be
> having any effect on the IOMMU.
> 
Interesting. That suggests the graphics drivers we tested do not use DMA
for XShmPutImage()...

Anyway, what would be the recommend way of updating IOMMU mappings in
this case? Would you consider patching of x_m_f_p() an acceptable
solution? (perhaps with some additional flag: "update_iommu")?

joanna.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 13/05/2010 13:09, "Joanna Rutkowska"
<joanna@invisiblethingslab.com>
wrote:
>> Seeing as how xc_map_foreign_pages() maps pages into a particular
>> guest-process virtual address space, whereas IOMMU tables are a mapping
from
>> guest-physical address space, it''s clear that x_m_f_p() cannot
sensibly be
>> having any effect on the IOMMU.
>> 
> Interesting. That suggests the graphics drivers we tested do not use DMA
> for XShmPutImage()...
Ah, actually dom0 is special in that it gets a 1:1 mapping of all of RAM
(except for some private Xen areas). See
drivers/passthrough/vtd/x86/vtd.c:iommu_set_dom0_mapping(). I forgot about
that when I wrote my previous reply!
> Anyway, what would be the recommend way of updating IOMMU mappings in
> this case? Would you consider patching of x_m_f_p() an acceptable
> solution? (perhaps with some additional flag: "update_iommu")?
Like I said, the concept does not make sense: IOMMU tables and host page
tables map from different address spaces. But of course this does not matter
anyway given dom0 gets a pretty comprehensive mapping set up by default. So
I think you are good to go.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 05/13/2010 02:27 PM, Keir Fraser wrote:
> On 13/05/2010 13:09, "Joanna Rutkowska"
<joanna@invisiblethingslab.com>
> wrote:
> 
>>> Seeing as how xc_map_foreign_pages() maps pages into a particular
>>> guest-process virtual address space, whereas IOMMU tables are a
mapping from
>>> guest-physical address space, it''s clear that x_m_f_p()
cannot sensibly be
>>> having any effect on the IOMMU.
>>>
>> Interesting. That suggests the graphics drivers we tested do not use
DMA
>> for XShmPutImage()...
> 
> Ah, actually dom0 is special in that it gets a 1:1 mapping of all of RAM
> (except for some private Xen areas). See
> drivers/passthrough/vtd/x86/vtd.c:iommu_set_dom0_mapping(). I forgot about
> that when I wrote my previous reply!
> 
Ah, indeed! In fact Rafal and I even had this very fragment of code in
our slides 2 years ago and we both forgot about it :) [Back then it was
in drivers/passthrough/vtd/iommu.c:intel_iommu_domain_init().

Thanks for clarifying!

joanna.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel