thr3ads.net - Xen devel - [Xen-devel] [PATCH 0 of 6] amd iommu: add ats device support [Oct 2011]

If this information is useful, please help other people find it:
Share via:

Wei Wang

2011-Oct-21 12:55 UTC

[Xen-devel] [PATCH 0 of 6] amd iommu: add ats device support

Hi, this patch set enables ats devices on amd systems with following changes:
1) Move vendor independent ATS functions into public directories.
2) Add new ATS helper functions.
3) Add amd specific enablement.

Thanks,
Wei


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Wei Wang

2011-Oct-21 12:55 UTC

head link

[Xen-devel] [PATCH 1 of 6] Move some ats functions into vendor neutral directories

# HG changeset patch
# User Wei Wang <wei.wang2@amd.com>
# Date 1319201416 -7200
# Node ID a559e27ffb2c2a3a90dc25f09205b66668dcdbbb
# Parent  121af976b2988de389db139231103ceedd11bb8a
Move some ats functions into vendor neutral directories.

Signed-off-by: Wei Wang <wei.wang2@amd.com>

diff -r 121af976b298 -r a559e27ffb2c xen/drivers/passthrough/pci.c
--- a/xen/drivers/passthrough/pci.c	Fri Oct 14 10:17:22 2011 +0200
+++ b/xen/drivers/passthrough/pci.c	Fri Oct 21 14:50:16 2011 +0200
@@ -748,6 +748,95 @@ static int __init setup_dump_pcidevs(voi
 __initcall(setup_dump_pcidevs);
 #endif
 
+int enable_ats_device(int seg, int bus, int devfn)
+{
+    struct pci_ats_dev *pdev = NULL;
+    u32 value;
+    int pos;
+
+    pos = pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS);
+    BUG_ON(!pos);
+
+    if ( iommu_verbose )
+        dprintk(XENLOG_INFO VTDPREFIX,
+                "%04x:%02x:%02x.%u: ATS capability found\n",
+                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
+
+    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
+                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
+    if ( value & ATS_ENABLE )
+    {
+        list_for_each_entry ( pdev, &ats_devices, list )
+        {
+            if ( pdev->seg == seg && pdev->bus == bus &&
pdev->devfn == devfn )
+            {
+                pos = 0;
+                break;
+            }
+        }
+    }
+    if ( pos )
+        pdev = xmalloc(struct pci_ats_dev);
+    if ( !pdev )
+        return -ENOMEM;
+
+    if ( !(value & ATS_ENABLE) )
+    {
+        value |= ATS_ENABLE;
+        pci_conf_write16(seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
+                         pos + ATS_REG_CTL, value);
+    }
+
+    if ( pos )
+    {
+        pdev->seg = seg;
+        pdev->bus = bus;
+        pdev->devfn = devfn;
+        value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
+                                PCI_FUNC(devfn), pos + ATS_REG_CAP);
+        pdev->ats_queue_depth = value & ATS_QUEUE_DEPTH_MASK;
+        list_add(&pdev->list, &ats_devices);
+    }
+
+    if ( iommu_verbose )
+        dprintk(XENLOG_INFO VTDPREFIX,
+                "%04x:%02x:%02x.%u: ATS %s enabled\n",
+                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
+                pos ? "is" : "was");
+
+    return pos;
+}
+
+void disable_ats_device(int seg, int bus, int devfn)
+{
+    struct pci_ats_dev *pdev;
+    u32 value;
+    int pos;
+
+    pos = pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS);
+    BUG_ON(!pos);
+
+    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
+                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
+    value &= ~ATS_ENABLE;
+    pci_conf_write16(seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
+                     pos + ATS_REG_CTL, value);
+
+    list_for_each_entry ( pdev, &ats_devices, list )
+    {
+        if ( pdev->seg == seg && pdev->bus == bus &&
pdev->devfn == devfn )
+        {
+            list_del(&pdev->list);
+            xfree(pdev);
+            break;
+        }
+    }
+
+    if ( iommu_verbose )
+        dprintk(XENLOG_INFO VTDPREFIX,
+                "%04x:%02x:%02x.%u: ATS is disabled\n",
+                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
+}
 
 /*
  * Local variables:
diff -r 121af976b298 -r a559e27ffb2c xen/drivers/passthrough/vtd/extern.h
--- a/xen/drivers/passthrough/vtd/extern.h	Fri Oct 14 10:17:22 2011 +0200
+++ b/xen/drivers/passthrough/vtd/extern.h	Fri Oct 21 14:50:16 2011 +0200
@@ -62,8 +62,6 @@ extern bool_t ats_enabled;
 struct acpi_drhd_unit * find_ats_dev_drhd(struct iommu *iommu);
 
 int ats_device(int seg, int bus, int devfn);
-int enable_ats_device(int seg, int bus, int devfn);
-void disable_ats_device(int seg, int bus, int devfn);
 int invalidate_ats_tcs(struct iommu *iommu);
 
 int dev_invalidate_iotlb(struct iommu *iommu, u16 did,
diff -r 121af976b298 -r a559e27ffb2c xen/drivers/passthrough/vtd/x86/ats.c
--- a/xen/drivers/passthrough/vtd/x86/ats.c	Fri Oct 14 10:17:22 2011 +0200
+++ b/xen/drivers/passthrough/vtd/x86/ats.c	Fri Oct 21 14:50:16 2011 +0200
@@ -30,20 +30,6 @@
 
 static LIST_HEAD(ats_dev_drhd_units);
 
-#define ATS_REG_CAP    4
-#define ATS_REG_CTL    6
-#define ATS_QUEUE_DEPTH_MASK     0xF
-#define ATS_ENABLE               (1<<15)
-
-struct pci_ats_dev {
-    struct list_head list;
-    u16 seg;
-    u8 bus;
-    u8 devfn;
-    u16 ats_queue_depth;    /* ATS device invalidation queue depth */
-};
-static LIST_HEAD(ats_devices);
-
 static void parse_ats_param(char *s);
 custom_param("ats", parse_ats_param);
 
@@ -121,97 +107,6 @@ int ats_device(int seg, int bus, int dev
     return pos;
 }
 
-int enable_ats_device(int seg, int bus, int devfn)
-{
-    struct pci_ats_dev *pdev = NULL;
-    u32 value;
-    int pos;
-
-    pos = pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS);
-    BUG_ON(!pos);
-
-    if ( iommu_verbose )
-        dprintk(XENLOG_INFO VTDPREFIX,
-                "%04x:%02x:%02x.%u: ATS capability found\n",
-                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
-
-    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
-                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
-    if ( value & ATS_ENABLE )
-    {
-        list_for_each_entry ( pdev, &ats_devices, list )
-        {
-            if ( pdev->seg == seg && pdev->bus == bus &&
pdev->devfn == devfn )
-            {
-                pos = 0;
-                break;
-            }
-        }
-    }
-    if ( pos )
-        pdev = xmalloc(struct pci_ats_dev);
-    if ( !pdev )
-        return -ENOMEM;
-
-    if ( !(value & ATS_ENABLE) )
-    {
-        value |= ATS_ENABLE;
-        pci_conf_write16(seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
-                         pos + ATS_REG_CTL, value);
-    }
-
-    if ( pos )
-    {
-        pdev->seg = seg;
-        pdev->bus = bus;
-        pdev->devfn = devfn;
-        value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
-                                PCI_FUNC(devfn), pos + ATS_REG_CAP);
-        pdev->ats_queue_depth = value & ATS_QUEUE_DEPTH_MASK;
-        list_add(&pdev->list, &ats_devices);
-    }
-
-    if ( iommu_verbose )
-        dprintk(XENLOG_INFO VTDPREFIX,
-                "%04x:%02x:%02x.%u: ATS %s enabled\n",
-                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
-                pos ? "is" : "was");
-
-    return pos;
-}
-
-void disable_ats_device(int seg, int bus, int devfn)
-{
-    struct pci_ats_dev *pdev;
-    u32 value;
-    int pos;
-
-    pos = pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS);
-    BUG_ON(!pos);
-
-    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
-                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
-    value &= ~ATS_ENABLE;
-    pci_conf_write16(seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
-                     pos + ATS_REG_CTL, value);
-
-    list_for_each_entry ( pdev, &ats_devices, list )
-    {
-        if ( pdev->seg == seg && pdev->bus == bus &&
pdev->devfn == devfn )
-        {
-            list_del(&pdev->list);
-            xfree(pdev);
-            break;
-        }
-    }
-
-    if ( iommu_verbose )
-        dprintk(XENLOG_INFO VTDPREFIX,
-                "%04x:%02x:%02x.%u: ATS is disabled\n",
-                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
-}
-
-
 static int device_in_domain(struct iommu *iommu, struct pci_ats_dev *pdev, u16
did)
 {
     struct root_entry *root_entry = NULL;
diff -r 121af976b298 -r a559e27ffb2c xen/include/xen/pci.h
--- a/xen/include/xen/pci.h	Fri Oct 14 10:17:22 2011 +0200
+++ b/xen/include/xen/pci.h	Fri Oct 21 14:50:16 2011 +0200
@@ -63,6 +63,20 @@ struct pci_dev {
     u64 vf_rlen[6];
 };
 
+#define ATS_REG_CAP    4
+#define ATS_REG_CTL    6
+#define ATS_QUEUE_DEPTH_MASK     0xF
+#define ATS_ENABLE               (1<<15)
+
+struct pci_ats_dev {
+    struct list_head list;
+    u16 seg;
+    u8 bus;
+    u8 devfn;
+    u16 ats_queue_depth;    /* ATS device invalidation queue depth */
+};
+static LIST_HEAD(ats_devices);
+
 #define for_each_pdev(domain, pdev) \
     list_for_each_entry(pdev, &(domain->arch.pdev_list), domain_list)
 
@@ -136,4 +150,7 @@ void msixtbl_pt_unregister(struct domain
 void msixtbl_pt_cleanup(struct domain *d);
 void pci_enable_acs(struct pci_dev *pdev);
 
+int enable_ats_device(int seg, int bus, int devfn);
+void disable_ats_device(int seg, int bus, int devfn);
+
 #endif /* __XEN_PCI_H__ */


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Wei Wang

2011-Oct-21 12:55 UTC

head link

[Xen-devel] [PATCH 2 of 6] Remove VTD prefix from debug output

# HG changeset patch
# User Wei Wang <wei.wang2@amd.com>
# Date 1319201420 -7200
# Node ID 1b618fbedd2371b2c4a00118f5fde8b063131d27
# Parent  a559e27ffb2c2a3a90dc25f09205b66668dcdbbb
Remove VTD prefix from debug output.

Signed-off-by: Wei Wang <wei.wang2@amd.com>

diff -r a559e27ffb2c -r 1b618fbedd23 xen/drivers/passthrough/pci.c
--- a/xen/drivers/passthrough/pci.c	Fri Oct 21 14:50:16 2011 +0200
+++ b/xen/drivers/passthrough/pci.c	Fri Oct 21 14:50:20 2011 +0200
@@ -758,7 +758,7 @@ int enable_ats_device(int seg, int bus, 
     BUG_ON(!pos);
 
     if ( iommu_verbose )
-        dprintk(XENLOG_INFO VTDPREFIX,
+        dprintk(XENLOG_INFO,
                 "%04x:%02x:%02x.%u: ATS capability found\n",
                 seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
 
@@ -799,7 +799,7 @@ int enable_ats_device(int seg, int bus, 
     }
 
     if ( iommu_verbose )
-        dprintk(XENLOG_INFO VTDPREFIX,
+        dprintk(XENLOG_INFO,
                 "%04x:%02x:%02x.%u: ATS %s enabled\n",
                 seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
                 pos ? "is" : "was");
@@ -833,7 +833,7 @@ void disable_ats_device(int seg, int bus
     }
 
     if ( iommu_verbose )
-        dprintk(XENLOG_INFO VTDPREFIX,
+        dprintk(XENLOG_INFO,
                 "%04x:%02x:%02x.%u: ATS is disabled\n",
                 seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
 }


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Wei Wang

2011-Oct-21 12:55 UTC

head link

[Xen-devel] [PATCH 3 of 6] Fix iommu page size encoding when page order > 0

# HG changeset patch
# User Wei Wang <wei.wang2@amd.com>
# Date 1319201421 -7200
# Node ID bb1330a1f8c7149fb39578382d9f5dfeef13ffa9
# Parent  1b618fbedd2371b2c4a00118f5fde8b063131d27
Fix iommu page size encoding when page order > 0

Signed-off-by: Wei Wang <wei.wang2@amd.com>

diff -r 1b618fbedd23 -r bb1330a1f8c7 xen/drivers/passthrough/amd/iommu_map.c
--- a/xen/drivers/passthrough/amd/iommu_map.c	Fri Oct 21 14:50:20 2011 +0200
+++ b/xen/drivers/passthrough/amd/iommu_map.c	Fri Oct 21 14:50:21 2011 +0200
@@ -77,23 +77,24 @@ static void invalidate_iommu_pages(struc
 {
     u64 addr_lo, addr_hi;
     u32 cmd[4], entry;
-    u64 mask = 0;
     int sflag = 0, pde = 0;
 
+    ASSERT ( order == 0 || order == 9 || order == 18 );
+
+    /* All pages associated with the domainID are invalidated */
+    if ( order || (io_addr == INV_IOMMU_ALL_PAGES_ADDRESS ) )
+    {
+        sflag = 1;
+        pde = 1;
+    }
+
     /* If sflag == 1, the size of the invalidate command is determined
      by the first zero bit in the address starting from Address[12] */
-    if ( order == 9 || order == 18 )
+    if ( order )
     {
-        mask = ((1ULL << (order - 1)) - 1) << PAGE_SHIFT;
-        io_addr |= mask;
-        sflag = 1;
-    }
-
-    /* All pages associated with the domainID are invalidated */
-    else if ( io_addr == 0x7FFFFFFFFFFFF000ULL )
-    {
-        sflag = 1;
-        pde = 1;
+        u64 mask = 1ULL << (order - 1 + PAGE_SHIFT);
+        io_addr &= ~mask;
+        io_addr |= mask - 1;
     }
 
     addr_lo = io_addr & DMA_32BIT_MASK;
@@ -917,7 +918,7 @@ static void _amd_iommu_flush_pages(struc
 
 void amd_iommu_flush_all_pages(struct domain *d)
 {
-    _amd_iommu_flush_pages(d, 0x7FFFFFFFFFFFFULL, 0);
+    _amd_iommu_flush_pages(d, INV_IOMMU_ALL_PAGES_ADDRESS, 0);
 }
 
 void amd_iommu_flush_pages(struct domain *d,
diff -r 1b618fbedd23 -r bb1330a1f8c7
xen/include/asm-x86/hvm/svm/amd-iommu-defs.h
--- a/xen/include/asm-x86/hvm/svm/amd-iommu-defs.h	Fri Oct 21 14:50:20 2011
+0200
+++ b/xen/include/asm-x86/hvm/svm/amd-iommu-defs.h	Fri Oct 21 14:50:21 2011
+0200
@@ -407,4 +407,6 @@
 #define INT_REMAP_ENTRY_VECTOR_MASK     0x00FF0000
 #define INT_REMAP_ENTRY_VECTOR_SHIFT    16
 
+#define INV_IOMMU_ALL_PAGES_ADDRESS     0x7FFFFFFFFFFFFULL
+
 #endif /* _ASM_X86_64_AMD_IOMMU_DEFS_H */


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Wei Wang

2011-Oct-21 12:55 UTC

head link

[Xen-devel] [PATCH 4 of 6] Add new ATS helper functions

# HG changeset patch
# User Wei Wang <wei.wang2@amd.com>
# Date 1319201422 -7200
# Node ID f2fbc041f4a710b66f98e76ad8905dcac7920c18
# Parent  bb1330a1f8c7149fb39578382d9f5dfeef13ffa9
Add new ATS helper functions

Signed-off-by Wei Wang <wei.wang2@amd.com>

diff -r bb1330a1f8c7 -r f2fbc041f4a7 xen/drivers/passthrough/pci.c
--- a/xen/drivers/passthrough/pci.c	Fri Oct 21 14:50:21 2011 +0200
+++ b/xen/drivers/passthrough/pci.c	Fri Oct 21 14:50:22 2011 +0200
@@ -838,6 +838,74 @@ void disable_ats_device(int seg, int bus
                 seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
 }
 
+int pci_ats_enabled(int seg, int bus, int devfn)
+{
+    u32 value;
+    int pos;
+
+    pos = pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS);
+    BUG_ON(!pos);
+
+    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
+                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
+    return value & ATS_ENABLE;
+}
+
+static void parse_ats_param(char *s);
+custom_param("ats", parse_ats_param);
+
+bool_t __read_mostly ats_enabled = 1;
+
+static void __init parse_ats_param(char *s)
+{
+    char *ss;
+
+    do {
+        ss = strchr(s, '','');
+        if ( ss )
+            *ss = ''\0'';
+
+        switch ( parse_bool(s) )
+        {
+        case 0:
+            ats_enabled = 0;
+            break;
+        case 1:
+            ats_enabled = 1;
+            break;
+        }
+
+        s = ss + 1;
+    } while ( ss );
+}
+
+int pci_ats_device(int seg, int bus, int devfn)
+{
+    if ( !ats_enabled )
+        return 0;
+
+    if ( !pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS) )
+        return 0;
+
+    return 1;
+}
+
+struct pci_ats_dev* get_ats_device(int seg, int bus, int devfn)
+{
+    struct pci_ats_dev *pdev;
+
+    if ( !pci_ats_device(seg, bus, devfn) )
+        return NULL;
+
+    list_for_each_entry ( pdev, &ats_devices, list )
+    {
+        if ( pdev->seg == seg && pdev->bus == bus &&
pdev->devfn == devfn )
+            return pdev;
+    }
+
+    return NULL;
+}
+
 /*
  * Local variables:
  * mode: C
diff -r bb1330a1f8c7 -r f2fbc041f4a7 xen/drivers/passthrough/vtd/x86/ats.c
--- a/xen/drivers/passthrough/vtd/x86/ats.c	Fri Oct 21 14:50:21 2011 +0200
+++ b/xen/drivers/passthrough/vtd/x86/ats.c	Fri Oct 21 14:50:22 2011 +0200
@@ -29,35 +29,6 @@
 #include "../extern.h"
 
 static LIST_HEAD(ats_dev_drhd_units);
-
-static void parse_ats_param(char *s);
-custom_param("ats", parse_ats_param);
-
-bool_t __read_mostly ats_enabled = 1;
-
-static void __init parse_ats_param(char *s)
-{
-    char *ss;
-
-    do {
-        ss = strchr(s, '','');
-        if ( ss )
-            *ss = ''\0'';
-
-        switch ( parse_bool(s) )
-        {
-        case 0:
-            ats_enabled = 0;
-            break;
-        case 1:
-            ats_enabled = 1;
-            break;
-        }
-
-        s = ss + 1;
-    } while ( ss );
-}
-
 struct acpi_drhd_unit * find_ats_dev_drhd(struct iommu *iommu)
 {
     struct acpi_drhd_unit *drhd;
diff -r bb1330a1f8c7 -r f2fbc041f4a7 xen/include/xen/pci.h
--- a/xen/include/xen/pci.h	Fri Oct 21 14:50:21 2011 +0200
+++ b/xen/include/xen/pci.h	Fri Oct 21 14:50:22 2011 +0200
@@ -152,5 +152,8 @@ void pci_enable_acs(struct pci_dev *pdev
 
 int enable_ats_device(int seg, int bus, int devfn);
 void disable_ats_device(int seg, int bus, int devfn);
+int pci_ats_enabled(int seg, int bus, int devfn);
+int pci_ats_device(int seg, int bus, int devfn);
+struct pci_ats_dev* get_ats_device(int seg, int bus, int devfn);
 
 #endif /* __XEN_PCI_H__ */


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Wei Wang

2011-Oct-21 12:55 UTC

head link

[Xen-devel] [PATCH 5 of 6] Add iotlb invalidation command for amd iommu

# HG changeset patch
# User Wei Wang <wei.wang2@amd.com>
# Date 1319201423 -7200
# Node ID 6009089b0511c843f54435d02dcd72394d7e164f
# Parent  f2fbc041f4a710b66f98e76ad8905dcac7920c18
Add iotlb invalidation command for amd iommu

Signed-off-by: Wei Wang <wei.wang2@amd.com>

diff -r f2fbc041f4a7 -r 6009089b0511 xen/drivers/passthrough/amd/iommu_map.c
--- a/xen/drivers/passthrough/amd/iommu_map.c	Fri Oct 21 14:50:22 2011 +0200
+++ b/xen/drivers/passthrough/amd/iommu_map.c	Fri Oct 21 14:50:23 2011 +0200
@@ -128,6 +128,75 @@ static void invalidate_iommu_pages(struc
     send_iommu_command(iommu, cmd);
 }
 
+static void invalidate_iotlb_pages(struct amd_iommu *iommu,
+                                   u16 maxpend, u32 pasid, u16 queueid,
+                                   u64 io_addr, u16 dev_id, u16 order)
+{
+    u64 addr_lo, addr_hi;
+    u32 cmd[4], entry;
+    int sflag = 0;
+
+    ASSERT ( order == 0 || order == 9 || order == 18 );
+
+    if ( order || (io_addr == INV_IOMMU_ALL_PAGES_ADDRESS ) )
+        sflag = 1;
+
+    /* If sflag == 1, the size of the invalidate command is determined
+     by the first zero bit in the address starting from Address[12] */
+    if ( order )
+    {
+        u64 mask = 1ULL << (order - 1 + PAGE_SHIFT);
+        io_addr &= ~mask;
+        io_addr |= mask - 1;
+    }
+
+    addr_lo = io_addr & DMA_32BIT_MASK;
+    addr_hi = io_addr >> 32;
+
+    set_field_in_reg_u32(dev_id, 0,
+                         IOMMU_INV_IOTLB_PAGES_DEVICE_ID_MASK,
+                         IOMMU_INV_IOTLB_PAGES_DEVICE_ID_SHIFT, &entry);
+
+    set_field_in_reg_u32(maxpend, entry,
+                         IOMMU_INV_IOTLB_PAGES_MAXPEND_MASK,
+                         IOMMU_INV_IOTLB_PAGES_MAXPEND_SHIFT, &entry);
+
+    set_field_in_reg_u32(pasid & 0xff, entry,
+                         IOMMU_INV_IOTLB_PAGES_PASID1_MASK,
+                         IOMMU_INV_IOTLB_PAGES_PASID1_SHIFT, &entry);
+    cmd[0] = entry;
+
+    set_field_in_reg_u32(IOMMU_CMD_INVALIDATE_IOTLB_PAGES, 0,
+                         IOMMU_CMD_OPCODE_MASK, IOMMU_CMD_OPCODE_SHIFT,
+                         &entry);
+
+    set_field_in_reg_u32(pasid >> 8, entry,
+                         IOMMU_INV_IOTLB_PAGES_PASID2_MASK, 
+                         IOMMU_INV_IOTLB_PAGES_PASID2_SHIFT,
+                         &entry);
+
+    set_field_in_reg_u32(queueid, entry,
+                         IOMMU_INV_IOTLB_PAGES_QUEUEID_MASK, 
+                         IOMMU_INV_IOTLB_PAGES_QUEUEID_SHIFT,
+                         &entry);
+    cmd[1] = entry;
+
+    set_field_in_reg_u32(sflag, 0,
+                         IOMMU_INV_IOTLB_PAGES_S_FLAG_MASK,
+                         IOMMU_INV_IOTLB_PAGES_S_FLAG_MASK, &entry);
+
+    set_field_in_reg_u32((u32)addr_lo >> PAGE_SHIFT, entry,
+                         IOMMU_INV_IOTLB_PAGES_ADDR_LOW_MASK,
+                         IOMMU_INV_IOTLB_PAGES_ADDR_LOW_SHIFT, &entry);
+    cmd[2] = entry;
+
+    set_field_in_reg_u32((u32)addr_hi, 0,
+                         IOMMU_INV_IOTLB_PAGES_ADDR_HIGH_MASK,
+                         IOMMU_INV_IOTLB_PAGES_ADDR_HIGH_SHIFT, &entry);
+    cmd[3] = entry;
+
+    send_iommu_command(iommu, cmd);
+}
 void flush_command_buffer(struct amd_iommu *iommu)
 {
     u32 cmd[4], status;
@@ -896,6 +965,62 @@ int amd_iommu_reserve_domain_unity_map(s
     return 0;
 }
 
+void amd_iommu_flush_iotlb(struct pci_ats_dev *pdev,
+                           uint64_t gaddr, unsigned int order)
+{
+    unsigned long flags;
+    struct amd_iommu *iommu;
+    unsigned int bdf, req_id, queueid, maxpend;
+
+    if ( !ats_enabled )
+        return;
+
+    bdf = (pdev->bus << 8) | pdev->devfn;
+    iommu = find_iommu_for_device(pdev->seg, bdf);
+
+    if ( !iommu )
+    {
+        AMD_IOMMU_DEBUG("%s: Fail to find iommu for device
%04x:%02x:%02x.%u\n",
+                        __func__, pdev->seg, pdev->bus,
PCI_SLOT(pdev->devfn),
+                        PCI_FUNC(pdev->devfn));
+        return;
+    }
+
+    if ( !iommu->iotlb_support )
+        return;
+
+    req_id = get_dma_requestor_id(iommu->seg, bdf);
+    queueid = req_id;
+    maxpend = (pdev->ats_queue_depth + 32) & 0xff;
+
+    /* send INVALIDATE_IOTLB_PAGES command */
+    spin_lock_irqsave(&iommu->lock, flags);
+    invalidate_iotlb_pages(iommu, maxpend, 0, queueid,
+                           gaddr, req_id, order);
+    flush_command_buffer(iommu);
+    spin_unlock_irqrestore(&iommu->lock, flags);
+}
+
+void amd_iommu_flush_all_iotlbs(struct domain *d, 
+                                uint64_t gaddr, unsigned int order)
+{
+    struct pci_ats_dev *ats_pdev;
+    struct pci_dev *pdev;
+
+    if ( !ats_enabled )
+        return;
+
+    for_each_pdev( d, pdev )
+    {
+        ats_pdev = get_ats_device(pdev->seg, pdev->bus, pdev->devfn);
+        if ( ats_pdev == NULL )
+            continue;
+        if ( !pci_ats_enabled(ats_pdev->seg, ats_pdev->bus,
ats_pdev->devfn) )
+            continue;
+
+        amd_iommu_flush_iotlb(ats_pdev, gaddr, order);
+    }
+}
 
 /* Flush iommu cache after p2m changes. */
 static void _amd_iommu_flush_pages(struct domain *d,
@@ -914,6 +1039,9 @@ static void _amd_iommu_flush_pages(struc
         flush_command_buffer(iommu);
         spin_unlock_irqrestore(&iommu->lock, flags);
     }
+
+    if ( ats_enabled )
+        amd_iommu_flush_all_iotlbs(d, gaddr, order);
 }
 
 void amd_iommu_flush_all_pages(struct domain *d)
diff -r f2fbc041f4a7 -r 6009089b0511
xen/include/asm-x86/hvm/svm/amd-iommu-defs.h
--- a/xen/include/asm-x86/hvm/svm/amd-iommu-defs.h	Fri Oct 21 14:50:22 2011
+0200
+++ b/xen/include/asm-x86/hvm/svm/amd-iommu-defs.h	Fri Oct 21 14:50:23 2011
+0200
@@ -233,6 +233,24 @@
 #define IOMMU_INV_INT_TABLE_DEVICE_ID_MASK   0x0000FFFF
 #define IOMMU_INV_INT_TABLE_DEVICE_ID_SHIFT  0
 
+/* INVALIDATE_IOTLB_PAGES command */
+#define IOMMU_INV_IOTLB_PAGES_MAXPEND_MASK          0xff000000
+#define IOMMU_INV_IOTLB_PAGES_MAXPEND_SHIFT         24
+#define IOMMU_INV_IOTLB_PAGES_PASID1_MASK           0x00ff0000
+#define IOMMU_INV_IOTLB_PAGES_PASID1_SHIFT          16
+#define IOMMU_INV_IOTLB_PAGES_PASID2_MASK           0x0fff0000
+#define IOMMU_INV_IOTLB_PAGES_PASID2_SHIFT          16
+#define IOMMU_INV_IOTLB_PAGES_QUEUEID_MASK          0x0000ffff
+#define IOMMU_INV_IOTLB_PAGES_QUEUEID_SHIFT         0
+#define IOMMU_INV_IOTLB_PAGES_DEVICE_ID_MASK        0x0000FFFF
+#define IOMMU_INV_IOTLB_PAGES_DEVICE_ID_SHIFT       0
+#define IOMMU_INV_IOTLB_PAGES_ADDR_LOW_MASK         0xFFFFF000
+#define IOMMU_INV_IOTLB_PAGES_ADDR_LOW_SHIFT        12
+#define IOMMU_INV_IOTLB_PAGES_ADDR_HIGH_MASK        0xFFFFFFFF
+#define IOMMU_INV_IOTLB_PAGES_ADDR_HIGH_SHIFT       0
+#define IOMMU_INV_IOTLB_PAGES_S_FLAG_MASK           0x00000001
+#define IOMMU_INV_IOTLB_PAGES_S_FLAG_SHIFT          0
+
 /* Event Log */
 #define IOMMU_EVENT_LOG_BASE_LOW_OFFSET		0x10
 #define IOMMU_EVENT_LOG_BASE_HIGH_OFFSET	0x14
diff -r f2fbc041f4a7 -r 6009089b0511
xen/include/asm-x86/hvm/svm/amd-iommu-proto.h
--- a/xen/include/asm-x86/hvm/svm/amd-iommu-proto.h	Fri Oct 21 14:50:22 2011
+0200
+++ b/xen/include/asm-x86/hvm/svm/amd-iommu-proto.h	Fri Oct 21 14:50:23 2011
+0200
@@ -55,6 +55,8 @@ int amd_iommu_unmap_page(struct domain *
 void amd_iommu_flush_pages(struct domain *d, unsigned long gfn,
                            unsigned int order);
 void amd_iommu_flush_all_pages(struct domain *d);
+void amd_iommu_flush_iotlb(struct pci_ats_dev *pdev,
+                           uint64_t gaddr, unsigned int order);
 
 u64 amd_iommu_get_next_table_from_pte(u32 *entry);
 int amd_iommu_reserve_domain_unity_map(struct domain *domain,
@@ -165,4 +167,6 @@ static inline void __free_amd_iommu_tabl
     free_xenheap_pages(table, order);
 }
 
+extern bool_t ats_enabled;
+
 #endif /* _ASM_X86_64_AMD_IOMMU_PROTO_H */


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Wei Wang

2011-Oct-21 12:55 UTC

head link

[Xen-devel] [PATCH 6 of 6] Enable ats devices on amd iommu systems

# HG changeset patch
# User Wei Wang <wei.wang2@amd.com>
# Date 1319201424 -7200
# Node ID 1ccad89116b57c95c0ead34c9f8cc4b67fec9316
# Parent  6009089b0511c843f54435d02dcd72394d7e164f
Enable ats devices on amd iommu systems

Signed-off-by: Wei Wang <wei.wang2@amd.com>

diff -r 6009089b0511 -r 1ccad89116b5 xen/drivers/passthrough/amd/iommu_map.c
--- a/xen/drivers/passthrough/amd/iommu_map.c	Fri Oct 21 14:50:23 2011 +0200
+++ b/xen/drivers/passthrough/amd/iommu_map.c	Fri Oct 21 14:50:24 2011 +0200
@@ -368,6 +368,17 @@ void amd_iommu_set_root_page_table(
     dte[0] = entry;
 }
 
+void iommu_dte_set_iotlb(u32 *dte, u8 i)
+{
+    u32 entry;
+
+    entry = dte[3];
+    set_field_in_reg_u32(!!i, entry,
+                         IOMMU_DEV_TABLE_IOTLB_SUPPORT_MASK,
+                         IOMMU_DEV_TABLE_IOTLB_SUPPORT_SHIFT, &entry);
+    dte[3] = entry;
+}
+
 void __init amd_iommu_set_intremap_table(
     u32 *dte, u64 intremap_ptr, u8 int_valid)
 {
diff -r 6009089b0511 -r 1ccad89116b5 xen/drivers/passthrough/amd/pci_amd_iommu.c
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c	Fri Oct 21 14:50:23 2011 +0200
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c	Fri Oct 21 14:50:24 2011 +0200
@@ -81,11 +81,12 @@ static void disable_translation(u32 *dte
 }
 
 static void amd_iommu_setup_domain_device(
-    struct domain *domain, struct amd_iommu *iommu, int bdf)
+    struct domain *domain, struct amd_iommu *iommu, u8 bus, u8 devfn)
 {
     void *dte;
     unsigned long flags;
     int req_id, valid = 1;
+    int dte_i = 0;
 
     struct hvm_iommu *hd = domain_hvm_iommu(domain);
 
@@ -94,8 +95,11 @@ static void amd_iommu_setup_domain_devic
     if ( iommu_passthrough && (domain->domain_id == 0) )
         valid = 0;
 
+    if ( ats_enabled )
+        dte_i = 1;
+
     /* get device-table entry */
-    req_id = get_dma_requestor_id(iommu->seg, bdf);
+    req_id = get_dma_requestor_id(iommu->seg, (bus << 8) | devfn);
     dte = iommu->dev_table.buffer + (req_id * IOMMU_DEV_TABLE_ENTRY_SIZE);
 
     spin_lock_irqsave(&iommu->lock, flags);
@@ -107,6 +111,9 @@ static void amd_iommu_setup_domain_devic
             (u32 *)dte, page_to_maddr(hd->root_table), hd->domain_id,
             hd->paging_mode, valid);
 
+        if ( pci_ats_device(iommu->seg, bus, devfn) &&
iommu->iotlb_support )
+            iommu_dte_set_iotlb((u32 *)dte, dte_i);
+
         invalidate_dev_table_entry(iommu, req_id);
         flush_command_buffer(iommu);
 
@@ -118,6 +125,19 @@ static void amd_iommu_setup_domain_devic
     }
 
     spin_unlock_irqrestore(&iommu->lock, flags);
+
+    ASSERT(spin_is_locked(&pcidevs_lock));
+
+    if ( pci_ats_device(iommu->seg, bus, devfn) && ats_enabled
&&
+         !pci_ats_enabled(iommu->seg, bus, devfn) )
+    {
+        struct pci_ats_dev* pdev;
+        enable_ats_device(iommu->seg, bus, devfn);
+        pdev = get_ats_device(iommu->seg, bus, devfn);
+
+        if ( pdev )
+            amd_iommu_flush_iotlb(pdev, INV_IOMMU_ALL_PAGES_ADDRESS, 0);
+    }
 }
 
 static void __init amd_iommu_setup_dom0_device(struct pci_dev *pdev)
@@ -126,7 +146,8 @@ static void __init amd_iommu_setup_dom0_
     struct amd_iommu *iommu = find_iommu_for_device(pdev->seg, bdf);
 
     if ( likely(iommu != NULL) )
-        amd_iommu_setup_domain_device(pdev->domain, iommu, bdf);
+        amd_iommu_setup_domain_device(pdev->domain, iommu, pdev->bus, 
+                                      pdev->devfn);
     else
         AMD_IOMMU_DEBUG("No iommu for device %04x:%02x:%02x.%u\n",
                         pdev->seg, pdev->bus,
@@ -261,21 +282,26 @@ static void __init amd_iommu_dom0_init(s
     setup_dom0_pci_devices(d, amd_iommu_setup_dom0_device);
 }
 
-static void amd_iommu_disable_domain_device(
-    struct domain *domain, struct amd_iommu *iommu, int bdf)
+void amd_iommu_disable_domain_device(struct domain *domain, 
+                                     struct amd_iommu *iommu, u8 bus, u8 devfn)
 {
     void *dte;
     unsigned long flags;
     int req_id;
 
     BUG_ON ( iommu->dev_table.buffer == NULL );
-    req_id = get_dma_requestor_id(iommu->seg, bdf);
+
+    req_id = get_dma_requestor_id(iommu->seg, (bus << 8) | devfn);
     dte = iommu->dev_table.buffer + (req_id * IOMMU_DEV_TABLE_ENTRY_SIZE);
 
     spin_lock_irqsave(&iommu->lock, flags);
     if ( is_translation_valid((u32 *)dte) )
     {
         disable_translation((u32 *)dte);
+
+        if ( pci_ats_device(iommu->seg, bus, devfn) &&
iommu->iotlb_support )
+            iommu_dte_set_iotlb((u32 *)dte, 0);
+
         invalidate_dev_table_entry(iommu, req_id);
         flush_command_buffer(iommu);
         AMD_IOMMU_DEBUG("Disable: device id = 0x%04x, "
@@ -284,6 +310,12 @@ static void amd_iommu_disable_domain_dev
                         domain_hvm_iommu(domain)->paging_mode);
     }
     spin_unlock_irqrestore(&iommu->lock, flags);
+
+    ASSERT(spin_is_locked(&pcidevs_lock));
+
+    if ( pci_ats_device(iommu->seg, bus, devfn) && ats_enabled
&&
+         pci_ats_enabled(iommu->seg, bus, devfn) )
+       disable_ats_device(iommu->seg, bus, devfn); 
 }
 
 static int reassign_device( struct domain *source, struct domain *target,
@@ -310,7 +342,7 @@ static int reassign_device( struct domai
         return -ENODEV;
     }
 
-    amd_iommu_disable_domain_device(source, iommu, bdf);
+    amd_iommu_disable_domain_device(source, iommu, bus, devfn);
 
     list_move(&pdev->domain_list, &target->arch.pdev_list);
     pdev->domain = target;
@@ -320,7 +352,7 @@ static int reassign_device( struct domai
     if ( t->root_table == NULL )
         allocate_domain_resources(t);
 
-    amd_iommu_setup_domain_device(target, iommu, bdf);
+    amd_iommu_setup_domain_device(target, iommu, bus, devfn);
     AMD_IOMMU_DEBUG("Re-assign %04x:%02x:%02x.%u from dom%d to
dom%d\n",
                     seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
                     source->domain_id, target->domain_id);
@@ -432,7 +464,8 @@ static int amd_iommu_add_device(struct p
         return -ENODEV;
     }
 
-    amd_iommu_setup_domain_device(pdev->domain, iommu, bdf);
+    amd_iommu_setup_domain_device(pdev->domain, iommu, 
+                                  pdev->bus, pdev->devfn);
     return 0;
 }
 
@@ -454,7 +487,8 @@ static int amd_iommu_remove_device(struc
         return -ENODEV;
     }
 
-    amd_iommu_disable_domain_device(pdev->domain, iommu, bdf);
+    amd_iommu_disable_domain_device(pdev->domain, iommu, 
+                                    pdev->bus, pdev->devfn);
     return 0;
 }
 
diff -r 6009089b0511 -r 1ccad89116b5
xen/include/asm-x86/hvm/svm/amd-iommu-proto.h
--- a/xen/include/asm-x86/hvm/svm/amd-iommu-proto.h	Fri Oct 21 14:50:23 2011
+0200
+++ b/xen/include/asm-x86/hvm/svm/amd-iommu-proto.h	Fri Oct 21 14:50:24 2011
+0200
@@ -75,6 +75,7 @@ void amd_iommu_set_intremap_table(
     u32 *dte, u64 intremap_ptr, u8 int_valid);
 void amd_iommu_set_root_page_table(
     u32 *dte, u64 root_ptr, u16 domain_id, u8 paging_mode, u8 valid);
+void iommu_dte_set_iotlb(u32 *dte, u8 i);
 void invalidate_dev_table_entry(struct amd_iommu *iommu, u16 devic_id);
 
 /* send cmd to iommu */


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jan Beulich

2011-Oct-21 13:08 UTC

head link

Re: [Xen-devel] [PATCH 1 of 6] Move some ats functions into vendor neutral directories

>>> On 21.10.11 at 14:55, Wei Wang <wei.wang2@amd.com> wrote:
> # HG changeset patch
> # User Wei Wang <wei.wang2@amd.com>
> # Date 1319201416 -7200
> # Node ID a559e27ffb2c2a3a90dc25f09205b66668dcdbbb
> # Parent  121af976b2988de389db139231103ceedd11bb8a
> Move some ats functions into vendor neutral directories.
You move them from an x86-specific place to common code. I suppose
Intel had reasons to expect them to be implemented differently in ia64.
Hence they should go into e.g. xen/drivers/passthrough/x86/ instead.

Jan
> Signed-off-by: Wei Wang <wei.wang2@amd.com>
> 
> diff -r 121af976b298 -r a559e27ffb2c xen/drivers/passthrough/pci.c
> --- a/xen/drivers/passthrough/pci.c	Fri Oct 14 10:17:22 2011 +0200
> +++ b/xen/drivers/passthrough/pci.c	Fri Oct 21 14:50:16 2011 +0200
> @@ -748,6 +748,95 @@ static int __init setup_dump_pcidevs(voi
>  __initcall(setup_dump_pcidevs);
>  #endif
>  
> +int enable_ats_device(int seg, int bus, int devfn)
> +{
> +    struct pci_ats_dev *pdev = NULL;
> +    u32 value;
> +    int pos;
> +
> +    pos = pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS);
> +    BUG_ON(!pos);
> +
> +    if ( iommu_verbose )
> +        dprintk(XENLOG_INFO VTDPREFIX,
> +                "%04x:%02x:%02x.%u: ATS capability found\n",
> +                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
> +
> +    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> +                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
> +    if ( value & ATS_ENABLE )
> +    {
> +        list_for_each_entry ( pdev, &ats_devices, list )
> +        {
> +            if ( pdev->seg == seg && pdev->bus == bus
&& pdev->devfn == devfn )
> +            {
> +                pos = 0;
> +                break;
> +            }
> +        }
> +    }
> +    if ( pos )
> +        pdev = xmalloc(struct pci_ats_dev);
> +    if ( !pdev )
> +        return -ENOMEM;
> +
> +    if ( !(value & ATS_ENABLE) )
> +    {
> +        value |= ATS_ENABLE;
> +        pci_conf_write16(seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> +                         pos + ATS_REG_CTL, value);
> +    }
> +
> +    if ( pos )
> +    {
> +        pdev->seg = seg;
> +        pdev->bus = bus;
> +        pdev->devfn = devfn;
> +        value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> +                                PCI_FUNC(devfn), pos + ATS_REG_CAP);
> +        pdev->ats_queue_depth = value & ATS_QUEUE_DEPTH_MASK;
> +        list_add(&pdev->list, &ats_devices);
> +    }
> +
> +    if ( iommu_verbose )
> +        dprintk(XENLOG_INFO VTDPREFIX,
> +                "%04x:%02x:%02x.%u: ATS %s enabled\n",
> +                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> +                pos ? "is" : "was");
> +
> +    return pos;
> +}
> +
> +void disable_ats_device(int seg, int bus, int devfn)
> +{
> +    struct pci_ats_dev *pdev;
> +    u32 value;
> +    int pos;
> +
> +    pos = pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS);
> +    BUG_ON(!pos);
> +
> +    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> +                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
> +    value &= ~ATS_ENABLE;
> +    pci_conf_write16(seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> +                     pos + ATS_REG_CTL, value);
> +
> +    list_for_each_entry ( pdev, &ats_devices, list )
> +    {
> +        if ( pdev->seg == seg && pdev->bus == bus &&
pdev->devfn == devfn )
> +        {
> +            list_del(&pdev->list);
> +            xfree(pdev);
> +            break;
> +        }
> +    }
> +
> +    if ( iommu_verbose )
> +        dprintk(XENLOG_INFO VTDPREFIX,
> +                "%04x:%02x:%02x.%u: ATS is disabled\n",
> +                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
> +}
>  
>  /*
>   * Local variables:
> diff -r 121af976b298 -r a559e27ffb2c xen/drivers/passthrough/vtd/extern.h
> --- a/xen/drivers/passthrough/vtd/extern.h	Fri Oct 14 10:17:22 2011 +0200
> +++ b/xen/drivers/passthrough/vtd/extern.h	Fri Oct 21 14:50:16 2011 +0200
> @@ -62,8 +62,6 @@ extern bool_t ats_enabled;
>  struct acpi_drhd_unit * find_ats_dev_drhd(struct iommu *iommu);
>  
>  int ats_device(int seg, int bus, int devfn);
> -int enable_ats_device(int seg, int bus, int devfn);
> -void disable_ats_device(int seg, int bus, int devfn);
>  int invalidate_ats_tcs(struct iommu *iommu);
>  
>  int dev_invalidate_iotlb(struct iommu *iommu, u16 did,
> diff -r 121af976b298 -r a559e27ffb2c xen/drivers/passthrough/vtd/x86/ats.c
> --- a/xen/drivers/passthrough/vtd/x86/ats.c	Fri Oct 14 10:17:22 2011 +0200
> +++ b/xen/drivers/passthrough/vtd/x86/ats.c	Fri Oct 21 14:50:16 2011 +0200
> @@ -30,20 +30,6 @@
>  
>  static LIST_HEAD(ats_dev_drhd_units);
>  
> -#define ATS_REG_CAP    4
> -#define ATS_REG_CTL    6
> -#define ATS_QUEUE_DEPTH_MASK     0xF
> -#define ATS_ENABLE               (1<<15)
> -
> -struct pci_ats_dev {
> -    struct list_head list;
> -    u16 seg;
> -    u8 bus;
> -    u8 devfn;
> -    u16 ats_queue_depth;    /* ATS device invalidation queue depth */
> -};
> -static LIST_HEAD(ats_devices);
> -
>  static void parse_ats_param(char *s);
>  custom_param("ats", parse_ats_param);
>  
> @@ -121,97 +107,6 @@ int ats_device(int seg, int bus, int dev
>      return pos;
>  }
>  
> -int enable_ats_device(int seg, int bus, int devfn)
> -{
> -    struct pci_ats_dev *pdev = NULL;
> -    u32 value;
> -    int pos;
> -
> -    pos = pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS);
> -    BUG_ON(!pos);
> -
> -    if ( iommu_verbose )
> -        dprintk(XENLOG_INFO VTDPREFIX,
> -                "%04x:%02x:%02x.%u: ATS capability found\n",
> -                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
> -
> -    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> -                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
> -    if ( value & ATS_ENABLE )
> -    {
> -        list_for_each_entry ( pdev, &ats_devices, list )
> -        {
> -            if ( pdev->seg == seg && pdev->bus == bus
&& pdev->devfn == devfn )
> -            {
> -                pos = 0;
> -                break;
> -            }
> -        }
> -    }
> -    if ( pos )
> -        pdev = xmalloc(struct pci_ats_dev);
> -    if ( !pdev )
> -        return -ENOMEM;
> -
> -    if ( !(value & ATS_ENABLE) )
> -    {
> -        value |= ATS_ENABLE;
> -        pci_conf_write16(seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> -                         pos + ATS_REG_CTL, value);
> -    }
> -
> -    if ( pos )
> -    {
> -        pdev->seg = seg;
> -        pdev->bus = bus;
> -        pdev->devfn = devfn;
> -        value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> -                                PCI_FUNC(devfn), pos + ATS_REG_CAP);
> -        pdev->ats_queue_depth = value & ATS_QUEUE_DEPTH_MASK;
> -        list_add(&pdev->list, &ats_devices);
> -    }
> -
> -    if ( iommu_verbose )
> -        dprintk(XENLOG_INFO VTDPREFIX,
> -                "%04x:%02x:%02x.%u: ATS %s enabled\n",
> -                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> -                pos ? "is" : "was");
> -
> -    return pos;
> -}
> -
> -void disable_ats_device(int seg, int bus, int devfn)
> -{
> -    struct pci_ats_dev *pdev;
> -    u32 value;
> -    int pos;
> -
> -    pos = pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS);
> -    BUG_ON(!pos);
> -
> -    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> -                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
> -    value &= ~ATS_ENABLE;
> -    pci_conf_write16(seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> -                     pos + ATS_REG_CTL, value);
> -
> -    list_for_each_entry ( pdev, &ats_devices, list )
> -    {
> -        if ( pdev->seg == seg && pdev->bus == bus &&
pdev->devfn == devfn )
> -        {
> -            list_del(&pdev->list);
> -            xfree(pdev);
> -            break;
> -        }
> -    }
> -
> -    if ( iommu_verbose )
> -        dprintk(XENLOG_INFO VTDPREFIX,
> -                "%04x:%02x:%02x.%u: ATS is disabled\n",
> -                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
> -}
> -
> -
>  static int device_in_domain(struct iommu *iommu, struct pci_ats_dev *pdev,
> u16 did)
>  {
>      struct root_entry *root_entry = NULL;
> diff -r 121af976b298 -r a559e27ffb2c xen/include/xen/pci.h
> --- a/xen/include/xen/pci.h	Fri Oct 14 10:17:22 2011 +0200
> +++ b/xen/include/xen/pci.h	Fri Oct 21 14:50:16 2011 +0200
> @@ -63,6 +63,20 @@ struct pci_dev {
>      u64 vf_rlen[6];
>  };
>  
> +#define ATS_REG_CAP    4
> +#define ATS_REG_CTL    6
> +#define ATS_QUEUE_DEPTH_MASK     0xF
> +#define ATS_ENABLE               (1<<15)
> +
> +struct pci_ats_dev {
> +    struct list_head list;
> +    u16 seg;
> +    u8 bus;
> +    u8 devfn;
> +    u16 ats_queue_depth;    /* ATS device invalidation queue depth */
> +};
> +static LIST_HEAD(ats_devices);
> +
>  #define for_each_pdev(domain, pdev) \
>      list_for_each_entry(pdev, &(domain->arch.pdev_list),
domain_list)
>  
> @@ -136,4 +150,7 @@ void msixtbl_pt_unregister(struct domain
>  void msixtbl_pt_cleanup(struct domain *d);
>  void pci_enable_acs(struct pci_dev *pdev);
>  
> +int enable_ats_device(int seg, int bus, int devfn);
> +void disable_ats_device(int seg, int bus, int devfn);
> +
>  #endif /* __XEN_PCI_H__ */
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com 
> http://lists.xensource.com/xen-devel 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jan Beulich

2011-Oct-21 13:11 UTC

head link

Re: [Xen-devel] [PATCH 4 of 6] Add new ATS helper functions

>>> On 21.10.11 at 14:55, Wei Wang <wei.wang2@amd.com> wrote:
> # HG changeset patch
> # User Wei Wang <wei.wang2@amd.com>
> # Date 1319201422 -7200
> # Node ID f2fbc041f4a710b66f98e76ad8905dcac7920c18
> # Parent  bb1330a1f8c7149fb39578382d9f5dfeef13ffa9
> Add new ATS helper functions
> 
> Signed-off-by Wei Wang <wei.wang2@amd.com>
> 
> diff -r bb1330a1f8c7 -r f2fbc041f4a7 xen/drivers/passthrough/pci.c
> --- a/xen/drivers/passthrough/pci.c	Fri Oct 21 14:50:21 2011 +0200
> +++ b/xen/drivers/passthrough/pci.c	Fri Oct 21 14:50:22 2011 +0200
> @@ -838,6 +838,74 @@ void disable_ats_device(int seg, int bus
>                  seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
>  }
>  
> +int pci_ats_enabled(int seg, int bus, int devfn)
> +{
> +    u32 value;
> +    int pos;
> +
> +    pos = pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS);
> +    BUG_ON(!pos);
> +
> +    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> +                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
> +    return value & ATS_ENABLE;
> +}
> +
> +static void parse_ats_param(char *s);
> +custom_param("ats", parse_ats_param);
> +
> +bool_t __read_mostly ats_enabled = 1;
> +
> +static void __init parse_ats_param(char *s)
> +{
> +    char *ss;
> +
> +    do {
> +        ss = strchr(s, '','');
> +        if ( ss )
> +            *ss = ''\0'';
> +
> +        switch ( parse_bool(s) )
> +        {
> +        case 0:
> +            ats_enabled = 0;
> +            break;
> +        case 1:
> +            ats_enabled = 1;
> +            break;
> +        }
> +
> +        s = ss + 1;
> +    } while ( ss );
> +}
> +
> +int pci_ats_device(int seg, int bus, int devfn)
> +{
> +    if ( !ats_enabled )
> +        return 0;
> +
> +    if ( !pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS) )
> +        return 0;
> +
> +    return 1;
> +}
> +
> +struct pci_ats_dev* get_ats_device(int seg, int bus, int devfn)
> +{
> +    struct pci_ats_dev *pdev;
> +
> +    if ( !pci_ats_device(seg, bus, devfn) )
> +        return NULL;
> +
> +    list_for_each_entry ( pdev, &ats_devices, list )
> +    {
> +        if ( pdev->seg == seg && pdev->bus == bus &&
pdev->devfn == devfn )
> +            return pdev;
> +    }
> +
> +    return NULL;
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff -r bb1330a1f8c7 -r f2fbc041f4a7 xen/drivers/passthrough/vtd/x86/ats.c
> --- a/xen/drivers/passthrough/vtd/x86/ats.c	Fri Oct 21 14:50:21 2011 +0200
> +++ b/xen/drivers/passthrough/vtd/x86/ats.c	Fri Oct 21 14:50:22 2011 +0200
> @@ -29,35 +29,6 @@
>  #include "../extern.h"
>  
>  static LIST_HEAD(ats_dev_drhd_units);
> -
> -static void parse_ats_param(char *s);
> -custom_param("ats", parse_ats_param);
> -
> -bool_t __read_mostly ats_enabled = 1;
> -
> -static void __init parse_ats_param(char *s)
> -{
> -    char *ss;
> -
> -    do {
> -        ss = strchr(s, '','');
> -        if ( ss )
> -            *ss = ''\0'';
> -
> -        switch ( parse_bool(s) )
> -        {
> -        case 0:
> -            ats_enabled = 0;
> -            break;
> -        case 1:
> -            ats_enabled = 1;
> -            break;
> -        }
> -
> -        s = ss + 1;
> -    } while ( ss );
> -}
> -
Why don''t you move this chunk together with the other movement you
do?

Jan
>  struct acpi_drhd_unit * find_ats_dev_drhd(struct iommu *iommu)
>  {
>      struct acpi_drhd_unit *drhd;
> diff -r bb1330a1f8c7 -r f2fbc041f4a7 xen/include/xen/pci.h
> --- a/xen/include/xen/pci.h	Fri Oct 21 14:50:21 2011 +0200
> +++ b/xen/include/xen/pci.h	Fri Oct 21 14:50:22 2011 +0200
> @@ -152,5 +152,8 @@ void pci_enable_acs(struct pci_dev *pdev
>  
>  int enable_ats_device(int seg, int bus, int devfn);
>  void disable_ats_device(int seg, int bus, int devfn);
> +int pci_ats_enabled(int seg, int bus, int devfn);
> +int pci_ats_device(int seg, int bus, int devfn);
> +struct pci_ats_dev* get_ats_device(int seg, int bus, int devfn);
>  
>  #endif /* __XEN_PCI_H__ */
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com 
> http://lists.xensource.com/xen-devel 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Wei Wang2

2011-Oct-21 13:57 UTC

head link

Re: [Xen-devel] [PATCH 1 of 6] Move some ats functions into vendor neutral directories

On Friday 21 October 2011 15:08:50 Jan Beulich wrote:> >>> On 21.10.11 at 14:55, Wei Wang <wei.wang2@amd.com>
wrote:
> >
> > # HG changeset patch
> > # User Wei Wang <wei.wang2@amd.com>
> > # Date 1319201416 -7200
> > # Node ID a559e27ffb2c2a3a90dc25f09205b66668dcdbbb
> > # Parent  121af976b2988de389db139231103ceedd11bb8a
> > Move some ats functions into vendor neutral directories.
>
> You move them from an x86-specific place to common code. I suppose
> Intel had reasons to expect them to be implemented differently in ia64.
> Hence they should go into e.g. xen/drivers/passthrough/x86/ instead.
>
> Jan
Well, we don''t have that directory yet. Could we change folder
structures  in
passthrough to be architecture first, then vendors, for example:
passthough/x86/vtd
passthough/x86/amd                             
passthough/ia64/vtd

Thanks,
Wei

> > Signed-off-by: Wei Wang <wei.wang2@amd.com>
> >
> > diff -r 121af976b298 -r a559e27ffb2c xen/drivers/passthrough/pci.c
> > --- a/xen/drivers/passthrough/pci.c	Fri Oct 14 10:17:22 2011 +0200
> > +++ b/xen/drivers/passthrough/pci.c	Fri Oct 21 14:50:16 2011 +0200
> > @@ -748,6 +748,95 @@ static int __init setup_dump_pcidevs(voi
> >  __initcall(setup_dump_pcidevs);
> >  #endif
> >
> > +int enable_ats_device(int seg, int bus, int devfn)
> > +{
> > +    struct pci_ats_dev *pdev = NULL;
> > +    u32 value;
> > +    int pos;
> > +
> > +    pos = pci_find_ext_capability(seg, bus, devfn,
PCI_EXT_CAP_ID_ATS);
> > +    BUG_ON(!pos);
> > +
> > +    if ( iommu_verbose )
> > +        dprintk(XENLOG_INFO VTDPREFIX,
> > +                "%04x:%02x:%02x.%u: ATS capability
found\n",
> > +                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
> > +
> > +    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> > +                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
> > +    if ( value & ATS_ENABLE )
> > +    {
> > +        list_for_each_entry ( pdev, &ats_devices, list )
> > +        {
> > +            if ( pdev->seg == seg && pdev->bus == bus
&& pdev->devfn => > devfn ) +            {
> > +                pos = 0;
> > +                break;
> > +            }
> > +        }
> > +    }
> > +    if ( pos )
> > +        pdev = xmalloc(struct pci_ats_dev);
> > +    if ( !pdev )
> > +        return -ENOMEM;
> > +
> > +    if ( !(value & ATS_ENABLE) )
> > +    {
> > +        value |= ATS_ENABLE;
> > +        pci_conf_write16(seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> > +                         pos + ATS_REG_CTL, value);
> > +    }
> > +
> > +    if ( pos )
> > +    {
> > +        pdev->seg = seg;
> > +        pdev->bus = bus;
> > +        pdev->devfn = devfn;
> > +        value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> > +                                PCI_FUNC(devfn), pos + ATS_REG_CAP);
> > +        pdev->ats_queue_depth = value & ATS_QUEUE_DEPTH_MASK;
> > +        list_add(&pdev->list, &ats_devices);
> > +    }
> > +
> > +    if ( iommu_verbose )
> > +        dprintk(XENLOG_INFO VTDPREFIX,
> > +                "%04x:%02x:%02x.%u: ATS %s enabled\n",
> > +                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> > +                pos ? "is" : "was");
> > +
> > +    return pos;
> > +}
> > +
> > +void disable_ats_device(int seg, int bus, int devfn)
> > +{
> > +    struct pci_ats_dev *pdev;
> > +    u32 value;
> > +    int pos;
> > +
> > +    pos = pci_find_ext_capability(seg, bus, devfn,
PCI_EXT_CAP_ID_ATS);
> > +    BUG_ON(!pos);
> > +
> > +    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> > +                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
> > +    value &= ~ATS_ENABLE;
> > +    pci_conf_write16(seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> > +                     pos + ATS_REG_CTL, value);
> > +
> > +    list_for_each_entry ( pdev, &ats_devices, list )
> > +    {
> > +        if ( pdev->seg == seg && pdev->bus == bus
&& pdev->devfn => > devfn ) +        {
> > +            list_del(&pdev->list);
> > +            xfree(pdev);
> > +            break;
> > +        }
> > +    }
> > +
> > +    if ( iommu_verbose )
> > +        dprintk(XENLOG_INFO VTDPREFIX,
> > +                "%04x:%02x:%02x.%u: ATS is disabled\n",
> > +                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
> > +}
> >
> >  /*
> >   * Local variables:
> > diff -r 121af976b298 -r a559e27ffb2c
xen/drivers/passthrough/vtd/extern.h
> > --- a/xen/drivers/passthrough/vtd/extern.h	Fri Oct 14 10:17:22 2011
+0200
> > +++ b/xen/drivers/passthrough/vtd/extern.h	Fri Oct 21 14:50:16 2011
+0200
> > @@ -62,8 +62,6 @@ extern bool_t ats_enabled;
> >  struct acpi_drhd_unit * find_ats_dev_drhd(struct iommu *iommu);
> >
> >  int ats_device(int seg, int bus, int devfn);
> > -int enable_ats_device(int seg, int bus, int devfn);
> > -void disable_ats_device(int seg, int bus, int devfn);
> >  int invalidate_ats_tcs(struct iommu *iommu);
> >
> >  int dev_invalidate_iotlb(struct iommu *iommu, u16 did,
> > diff -r 121af976b298 -r a559e27ffb2c
> > xen/drivers/passthrough/vtd/x86/ats.c ---
> > a/xen/drivers/passthrough/vtd/x86/ats.c	Fri Oct 14 10:17:22 2011 +0200
> > +++ b/xen/drivers/passthrough/vtd/x86/ats.c	Fri Oct 21 14:50:16 2011
> > +0200 @@ -30,20 +30,6 @@
> >
> >  static LIST_HEAD(ats_dev_drhd_units);
> >
> > -#define ATS_REG_CAP    4
> > -#define ATS_REG_CTL    6
> > -#define ATS_QUEUE_DEPTH_MASK     0xF
> > -#define ATS_ENABLE               (1<<15)
> > -
> > -struct pci_ats_dev {
> > -    struct list_head list;
> > -    u16 seg;
> > -    u8 bus;
> > -    u8 devfn;
> > -    u16 ats_queue_depth;    /* ATS device invalidation queue depth */
> > -};
> > -static LIST_HEAD(ats_devices);
> > -
> >  static void parse_ats_param(char *s);
> >  custom_param("ats", parse_ats_param);
> >
> > @@ -121,97 +107,6 @@ int ats_device(int seg, int bus, int dev
> >      return pos;
> >  }
> >
> > -int enable_ats_device(int seg, int bus, int devfn)
> > -{
> > -    struct pci_ats_dev *pdev = NULL;
> > -    u32 value;
> > -    int pos;
> > -
> > -    pos = pci_find_ext_capability(seg, bus, devfn,
PCI_EXT_CAP_ID_ATS);
> > -    BUG_ON(!pos);
> > -
> > -    if ( iommu_verbose )
> > -        dprintk(XENLOG_INFO VTDPREFIX,
> > -                "%04x:%02x:%02x.%u: ATS capability
found\n",
> > -                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
> > -
> > -    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> > -                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
> > -    if ( value & ATS_ENABLE )
> > -    {
> > -        list_for_each_entry ( pdev, &ats_devices, list )
> > -        {
> > -            if ( pdev->seg == seg && pdev->bus == bus
&& pdev->devfn => > devfn ) -            {
> > -                pos = 0;
> > -                break;
> > -            }
> > -        }
> > -    }
> > -    if ( pos )
> > -        pdev = xmalloc(struct pci_ats_dev);
> > -    if ( !pdev )
> > -        return -ENOMEM;
> > -
> > -    if ( !(value & ATS_ENABLE) )
> > -    {
> > -        value |= ATS_ENABLE;
> > -        pci_conf_write16(seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> > -                         pos + ATS_REG_CTL, value);
> > -    }
> > -
> > -    if ( pos )
> > -    {
> > -        pdev->seg = seg;
> > -        pdev->bus = bus;
> > -        pdev->devfn = devfn;
> > -        value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> > -                                PCI_FUNC(devfn), pos + ATS_REG_CAP);
> > -        pdev->ats_queue_depth = value & ATS_QUEUE_DEPTH_MASK;
> > -        list_add(&pdev->list, &ats_devices);
> > -    }
> > -
> > -    if ( iommu_verbose )
> > -        dprintk(XENLOG_INFO VTDPREFIX,
> > -                "%04x:%02x:%02x.%u: ATS %s enabled\n",
> > -                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> > -                pos ? "is" : "was");
> > -
> > -    return pos;
> > -}
> > -
> > -void disable_ats_device(int seg, int bus, int devfn)
> > -{
> > -    struct pci_ats_dev *pdev;
> > -    u32 value;
> > -    int pos;
> > -
> > -    pos = pci_find_ext_capability(seg, bus, devfn,
PCI_EXT_CAP_ID_ATS);
> > -    BUG_ON(!pos);
> > -
> > -    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> > -                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
> > -    value &= ~ATS_ENABLE;
> > -    pci_conf_write16(seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> > -                     pos + ATS_REG_CTL, value);
> > -
> > -    list_for_each_entry ( pdev, &ats_devices, list )
> > -    {
> > -        if ( pdev->seg == seg && pdev->bus == bus
&& pdev->devfn => > devfn ) -        {
> > -            list_del(&pdev->list);
> > -            xfree(pdev);
> > -            break;
> > -        }
> > -    }
> > -
> > -    if ( iommu_verbose )
> > -        dprintk(XENLOG_INFO VTDPREFIX,
> > -                "%04x:%02x:%02x.%u: ATS is disabled\n",
> > -                seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
> > -}
> > -
> > -
> >  static int device_in_domain(struct iommu *iommu, struct pci_ats_dev
> > *pdev, u16 did)
> >  {
> >      struct root_entry *root_entry = NULL;
> > diff -r 121af976b298 -r a559e27ffb2c xen/include/xen/pci.h
> > --- a/xen/include/xen/pci.h	Fri Oct 14 10:17:22 2011 +0200
> > +++ b/xen/include/xen/pci.h	Fri Oct 21 14:50:16 2011 +0200
> > @@ -63,6 +63,20 @@ struct pci_dev {
> >      u64 vf_rlen[6];
> >  };
> >
> > +#define ATS_REG_CAP    4
> > +#define ATS_REG_CTL    6
> > +#define ATS_QUEUE_DEPTH_MASK     0xF
> > +#define ATS_ENABLE               (1<<15)
> > +
> > +struct pci_ats_dev {
> > +    struct list_head list;
> > +    u16 seg;
> > +    u8 bus;
> > +    u8 devfn;
> > +    u16 ats_queue_depth;    /* ATS device invalidation queue depth */
> > +};
> > +static LIST_HEAD(ats_devices);
> > +
> >  #define for_each_pdev(domain, pdev) \
> >      list_for_each_entry(pdev, &(domain->arch.pdev_list),
domain_list)
> >
> > @@ -136,4 +150,7 @@ void msixtbl_pt_unregister(struct domain
> >  void msixtbl_pt_cleanup(struct domain *d);
> >  void pci_enable_acs(struct pci_dev *pdev);
> >
> > +int enable_ats_device(int seg, int bus, int devfn);
> > +void disable_ats_device(int seg, int bus, int devfn);
> > +
> >  #endif /* __XEN_PCI_H__ */
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Wei Wang2

2011-Oct-21 13:58 UTC

head link

Re: [Xen-devel] [PATCH 4 of 6] Add new ATS helper functions

On Friday 21 October 2011 15:11:38 Jan Beulich wrote:> >>> On 21.10.11 at 14:55, Wei Wang <wei.wang2@amd.com>
wrote:
> >
> > # HG changeset patch
> > # User Wei Wang <wei.wang2@amd.com>
> > # Date 1319201422 -7200
> > # Node ID f2fbc041f4a710b66f98e76ad8905dcac7920c18
> > # Parent  bb1330a1f8c7149fb39578382d9f5dfeef13ffa9
> > Add new ATS helper functions
> >
> > Signed-off-by Wei Wang <wei.wang2@amd.com>
> >
> > diff -r bb1330a1f8c7 -r f2fbc041f4a7 xen/drivers/passthrough/pci.c
> > --- a/xen/drivers/passthrough/pci.c	Fri Oct 21 14:50:21 2011 +0200
> > +++ b/xen/drivers/passthrough/pci.c	Fri Oct 21 14:50:22 2011 +0200
> > @@ -838,6 +838,74 @@ void disable_ats_device(int seg, int bus
> >                  seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
> >  }
> >
> > +int pci_ats_enabled(int seg, int bus, int devfn)
> > +{
> > +    u32 value;
> > +    int pos;
> > +
> > +    pos = pci_find_ext_capability(seg, bus, devfn,
PCI_EXT_CAP_ID_ATS);
> > +    BUG_ON(!pos);
> > +
> > +    value = pci_conf_read16(seg, bus, PCI_SLOT(devfn),
> > +                            PCI_FUNC(devfn), pos + ATS_REG_CTL);
> > +    return value & ATS_ENABLE;
> > +}
> > +
> > +static void parse_ats_param(char *s);
> > +custom_param("ats", parse_ats_param);
> > +
> > +bool_t __read_mostly ats_enabled = 1;
> > +
> > +static void __init parse_ats_param(char *s)
> > +{
> > +    char *ss;
> > +
> > +    do {
> > +        ss = strchr(s, '','');
> > +        if ( ss )
> > +            *ss = ''\0'';
> > +
> > +        switch ( parse_bool(s) )
> > +        {
> > +        case 0:
> > +            ats_enabled = 0;
> > +            break;
> > +        case 1:
> > +            ats_enabled = 1;
> > +            break;
> > +        }
> > +
> > +        s = ss + 1;
> > +    } while ( ss );
> > +}
> > +
> > +int pci_ats_device(int seg, int bus, int devfn)
> > +{
> > +    if ( !ats_enabled )
> > +        return 0;
> > +
> > +    if ( !pci_find_ext_capability(seg, bus, devfn,
PCI_EXT_CAP_ID_ATS) )
> > +        return 0;
> > +
> > +    return 1;
> > +}
> > +
> > +struct pci_ats_dev* get_ats_device(int seg, int bus, int devfn)
> > +{
> > +    struct pci_ats_dev *pdev;
> > +
> > +    if ( !pci_ats_device(seg, bus, devfn) )
> > +        return NULL;
> > +
> > +    list_for_each_entry ( pdev, &ats_devices, list )
> > +    {
> > +        if ( pdev->seg == seg && pdev->bus == bus
&& pdev->devfn => > devfn ) +            return pdev;
> > +    }
> > +
> > +    return NULL;
> > +}
> > +
> >  /*
> >   * Local variables:
> >   * mode: C
> > diff -r bb1330a1f8c7 -r f2fbc041f4a7
> > xen/drivers/passthrough/vtd/x86/ats.c ---
> > a/xen/drivers/passthrough/vtd/x86/ats.c	Fri Oct 21 14:50:21 2011 +0200
> > +++ b/xen/drivers/passthrough/vtd/x86/ats.c	Fri Oct 21 14:50:22 2011
> > +0200 @@ -29,35 +29,6 @@
> >  #include "../extern.h"
> >
> >  static LIST_HEAD(ats_dev_drhd_units);
> > -
> > -static void parse_ats_param(char *s);
> > -custom_param("ats", parse_ats_param);
> > -
> > -bool_t __read_mostly ats_enabled = 1;
> > -
> > -static void __init parse_ats_param(char *s)
> > -{
> > -    char *ss;
> > -
> > -    do {
> > -        ss = strchr(s, '','');
> > -        if ( ss )
> > -            *ss = ''\0'';
> > -
> > -        switch ( parse_bool(s) )
> > -        {
> > -        case 0:
> > -            ats_enabled = 0;
> > -            break;
> > -        case 1:
> > -            ats_enabled = 1;
> > -            break;
> > -        }
> > -
> > -        s = ss + 1;
> > -    } while ( ss );
> > -}
> > -
>
> Why don''t you move this chunk together with the other movement you
> do?Sure, this should be combined with patch 1. I will fix it in the next try.
Thanks,
Wei
> Jan
>
> >  struct acpi_drhd_unit * find_ats_dev_drhd(struct iommu *iommu)
> >  {
> >      struct acpi_drhd_unit *drhd;
> > diff -r bb1330a1f8c7 -r f2fbc041f4a7 xen/include/xen/pci.h
> > --- a/xen/include/xen/pci.h	Fri Oct 21 14:50:21 2011 +0200
> > +++ b/xen/include/xen/pci.h	Fri Oct 21 14:50:22 2011 +0200
> > @@ -152,5 +152,8 @@ void pci_enable_acs(struct pci_dev *pdev
> >
> >  int enable_ats_device(int seg, int bus, int devfn);
> >  void disable_ats_device(int seg, int bus, int devfn);
> > +int pci_ats_enabled(int seg, int bus, int devfn);
> > +int pci_ats_device(int seg, int bus, int devfn);
> > +struct pci_ats_dev* get_ats_device(int seg, int bus, int devfn);
> >
> >  #endif /* __XEN_PCI_H__ */
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jan Beulich

2011-Oct-21 14:07 UTC

head link

Re: [Xen-devel] [PATCH 1 of 6] Move some ats functions into vendor neutraldirectories

>>> On 21.10.11 at 15:57, Wei Wang2 <wei.wang2@amd.com> wrote:
> On Friday 21 October 2011 15:08:50 Jan Beulich wrote:
>> >>> On 21.10.11 at 14:55, Wei Wang <wei.wang2@amd.com>
wrote:
>> >
>> > # HG changeset patch
>> > # User Wei Wang <wei.wang2@amd.com>
>> > # Date 1319201416 -7200
>> > # Node ID a559e27ffb2c2a3a90dc25f09205b66668dcdbbb
>> > # Parent  121af976b2988de389db139231103ceedd11bb8a
>> > Move some ats functions into vendor neutral directories.
>>
>> You move them from an x86-specific place to common code. I suppose
>> Intel had reasons to expect them to be implemented differently in ia64.
>> Hence they should go into e.g. xen/drivers/passthrough/x86/ instead.
> 
> Well, we don''t have that directory yet. Could we change folder
structures
> in 
> passthrough to be architecture first, then vendors, for example:
> passthough/x86/vtd
> passthough/x86/amd                             
> passthough/ia64/vtd
That doesn''t make sense - VT-d is for both ia64 and x86. Code that is
common between vendors but architecture specific would reasonably
belong into drivers/passthrough/<arch>/, which is what I
recommended.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Oct 2011 - [PATCH 0 of 6] amd iommu: add ats device support

[Xen-devel] [PATCH 0 of 6] amd iommu: add ats device support

[Xen-devel] [PATCH 1 of 6] Move some ats functions into vendor neutral directories

[Xen-devel] [PATCH 2 of 6] Remove VTD prefix from debug output

[Xen-devel] [PATCH 3 of 6] Fix iommu page size encoding when page order > 0

[Xen-devel] [PATCH 4 of 6] Add new ATS helper functions

[Xen-devel] [PATCH 5 of 6] Add iotlb invalidation command for amd iommu

[Xen-devel] [PATCH 6 of 6] Enable ats devices on amd iommu systems

Re: [Xen-devel] [PATCH 1 of 6] Move some ats functions into vendor neutral directories

Re: [Xen-devel] [PATCH 4 of 6] Add new ATS helper functions

Re: [Xen-devel] [PATCH 1 of 6] Move some ats functions into vendor neutral directories

Re: [Xen-devel] [PATCH 4 of 6] Add new ATS helper functions

Re: [Xen-devel] [PATCH 1 of 6] Move some ats functions into vendor neutraldirectories