Jean Guyader
2011-Nov-04 10:38 UTC
[Xen-devel] [PATCH 0/6] IOMMU, vtd and iotlb flush rework
In one of my previous email I detailed a bug I was seeing when passing
through a Intel GPU on a guest that has more that 4G or RAM.
Allen suggested that I go for the Plan B but after a discussion with Tim
we agreed that Plan B was way to disruptive in term of code change.
This patch series implements Plan A.
http://xen.1045712.n5.nabble.com/VTD-Intel-iommu-IOTLB-flush-really-slow-td4952866.html
Jean Guyader (6):
vtd: Refactor iotlb flush code
iommu: Introduce iommu_flush and iommu_flush_all.
add_to_physmap: Move the code for XENMEM_add_to_physmap.
mm: Add new map space for add_to_physmap, XENMAPSPACE_gmfn_range.
hvmloader: Change memory relocation loop when overlap with PCI hole.
Introduce domain flag (dont_flush_iotlb) to avoid unnecessary iotlb flush.
tools/firmware/hvmloader/pci.c | 20 +++-
xen/arch/x86/mm.c | 197 ++++++++++++++++++++---------------
xen/drivers/passthrough/iommu.c | 26 +++++
xen/drivers/passthrough/vtd/iommu.c | 100 ++++++++++--------
xen/include/public/memory.h | 4 +
xen/include/xen/iommu.h | 5 +
xen/include/xen/sched.h | 1 +
7 files changed, 222 insertions(+), 131 deletions(-)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Factorize the iotlb flush code from map_page and unmap_page into it''s own function. Signed-off-by: Jean Guyader <jean.guyader@eu.citrix.com> --- xen/drivers/passthrough/vtd/iommu.c | 86 +++++++++++++++++----------------- 1 files changed, 43 insertions(+), 43 deletions(-) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jean Guyader
2011-Nov-04 10:38 UTC
[Xen-devel] [PATCH 2/6] iommu: Introduce iommu_flush and iommu_flush_all.
Signed-off-by: Jean Guyader <jean.guyader@eu.citrix.com> --- xen/drivers/passthrough/iommu.c | 20 ++++++++++++++++++++ xen/drivers/passthrough/vtd/iommu.c | 12 ++++++++++++ xen/include/xen/iommu.h | 5 +++++ 3 files changed, 37 insertions(+), 0 deletions(-) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jean Guyader
2011-Nov-04 10:38 UTC
[Xen-devel] [PATCH 3/6] add_to_physmap: Move the code for XENMEM_add_to_physmap.
Move the code for the XENMEM_add_to_physmap case into it''s own function (xenmem_add_to_physmap). Signed-off-by: Jean Guyader <jean.guyader@eu.citrix.com> --- xen/arch/x86/mm.c | 188 ++++++++++++++++++++++++++++------------------------- 1 files changed, 99 insertions(+), 89 deletions(-) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jean Guyader
2011-Nov-04 10:38 UTC
[Xen-devel] [PATCH 4/6] mm: Add new map space for add_to_physmap, XENMAPSPACE_gmfn_range.
XENMAPSPACE_gmfn_range is like XENMAPSPACE_gmfn but with a size which is the number of pages on which xen will iterate. Signed-off-by: Jean Guyader <jean.guyader@eu.citrix.com> --- xen/arch/x86/mm.c | 15 ++++++++++++++- xen/include/public/memory.h | 4 ++++ 2 files changed, 18 insertions(+), 1 deletions(-) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jean Guyader
2011-Nov-04 10:38 UTC
[Xen-devel] [PATCH 5/6] hvmloader: Change memory relocation loop when overlap with PCI hole.
Change the way we relocate the memory page if they overlap with pci hole. Use new map space (XENMAPSPACE_gmfn_range) to move the loop into xen. This code usually get triggered when a device is pass through to a guest and the PCI hole has to be extended to have enough room to map the device BARs. The PCI hole will starts lower and it might overlap with some RAM that has been alocated for the guest. That usually happen if the guest has more than 4G of RAM. We have to relocate those pages in high mem otherwise they won''t be accessible. Signed-off-by: Jean Guyader <jean.guyader@eu.citrix.com> --- tools/firmware/hvmloader/pci.c | 20 ++++++++++++++------ 1 files changed, 14 insertions(+), 6 deletions(-) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jean Guyader
2011-Nov-04 10:38 UTC
[Xen-devel] [PATCH 6/6] Introduce domain flag (dont_flush_iotlb) to avoid unnecessary iotlb flush.
Add a domain level flag that will be check by the iommu low level code to skip iotlb flush. iommu_iotlb_flush has to be called explicitly. Signed-off-by: Jean Guyader <jean.guyader@eu.citrix.com> --- xen/arch/x86/mm.c | 12 +++++++++++- xen/drivers/passthrough/iommu.c | 6 ++++++ xen/drivers/passthrough/vtd/iommu.c | 6 ++++-- xen/include/xen/sched.h | 1 + 4 files changed, 22 insertions(+), 3 deletions(-) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2011-Nov-04 12:32 UTC
Re: [Xen-devel] [PATCH 4/6] mm: Add new map space for add_to_physmap, XENMAPSPACE_gmfn_range.
>>> On 04.11.11 at 11:38, Jean Guyader <jean.guyader@eu.citrix.com> wrote: > XENMAPSPACE_gmfn_range is like XENMAPSPACE_gmfn but with a size which > is the number of pages on which xen will iterate. > > Signed-off-by: Jean Guyader <jean.guyader@eu.citrix.com> > --- > xen/arch/x86/mm.c | 15 ++++++++++++++- > xen/include/public/memory.h | 4 ++++ > 2 files changed, 18 insertions(+), 1 deletions(-)You can''t add a member to a structure that''s part of the public interface. You''ll need to create a new structure. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2011-Nov-04 12:41 UTC
Re: [Xen-devel] [PATCH 6/6] Introduce domain flag (dont_flush_iotlb) to avoid unnecessary iotlb flush.
>>> On 04.11.11 at 11:38, Jean Guyader <jean.guyader@eu.citrix.com> wrote:> Add a domain level flag that will be check by the iommu low level code > to skip iotlb flush. iommu_iotlb_flush has to be called explicitly. > > Signed-off-by: Jean Guyader <jean.guyader@eu.citrix.com> > --- > xen/arch/x86/mm.c | 12 +++++++++++- > xen/drivers/passthrough/iommu.c | 6 ++++++ > xen/drivers/passthrough/vtd/iommu.c | 6 ++++-- > xen/include/xen/sched.h | 1 + > 4 files changed, 22 insertions(+), 3 deletions(-)Neither the patch nor its description make clear what lock is used to protect this flag from having an effect on other but the current vCPU of the guest in question - I doubt that all other relevant code paths indeed use domain_lock(). Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-Nov-04 14:47 UTC
Re: [Xen-devel] [PATCH 4/6] mm: Add new map space for add_to_physmap, XENMAPSPACE_gmfn_range.
On 04/11/2011 10:38, "Jean Guyader" <jean.guyader@eu.citrix.com> wrote:> > XENMAPSPACE_gmfn_range is like XENMAPSPACE_gmfn but with a size which > is the number of pages on which xen will iterate.You can''t really extend the size of an existing ABI structure, and always copy that extended size. Older guests won''t know to guarantee that the extended space is accessible. I suggest you make your new field a uint16_t, placed directly after the domid field. Then you are making use of existing pad space. 64k pages = 256MB at a time should be plenty of amortisation. And, even with the reduced field width, I could imagine 64k remappings taking a good while. The remapping loop should regularly (even every iteration) check hypercall_preempt_check(), then hypercall_create_continuation() and exit if the hypercall is being requested to voluntarily yield. -- Keir> Signed-off-by: Jean Guyader <jean.guyader@eu.citrix.com> > --- > xen/arch/x86/mm.c | 15 ++++++++++++++- > xen/include/public/memory.h | 4 ++++ > 2 files changed, 18 insertions(+), 1 deletions(-) > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-Nov-04 14:52 UTC
Re: [Xen-devel] [PATCH 5/6] hvmloader: Change memory relocation loop when overlap with PCI hole.
On 04/11/2011 10:38, "Jean Guyader" <jean.guyader@eu.citrix.com> wrote:> > Change the way we relocate the memory page if they overlap with pci hole. > Use new map space (XENMAPSPACE_gmfn_range) to move the loop into xen. > > This code usually get triggered when a device is pass through to a guest > and the PCI hole has to be extended to have enough room to map the device > BARs. > The PCI hole will starts lower and it might overlap with some RAM that has > been > alocated for the guest. That usually happen if the guest has more than 4G of > RAM. > We have to relocate those pages in high mem otherwise they won''t be > accessible.If the size field in the add_to_physmap structure is reduced to 16 bits, the new if() stmt will probably have to become a while() again. The new printf''s are just excess verbiage. We dump the memory map as we exit hvmloader anyway. -- Keir> Signed-off-by: Jean Guyader <jean.guyader@eu.citrix.com> > --- > tools/firmware/hvmloader/pci.c | 20 ++++++++++++++------ > 1 files changed, 14 insertions(+), 6 deletions(-) > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel