Xen devel - Dec 2009 - [PATCH PCIFRONT]

Hey Jeremy,

Attached are eight patches to make Xen PCI front work as a module.

The first two:
 [PATCH 1/8] commit eaa959df299157e2640fcb3321537501b6afd9e6
 [PATCH 2/8] [PCI]: Export pci_walk_bus function.

Touch the pci subsystem. The first is backport from 2.6.31.something, while the
second has be OK-ed by Jesse (PCI subsystem maintainer).

The rest of the patches implement a registration mechanism for
Xen PCI front to hook up when it gets loaded. The mechanism is fairly
simple as there are no consumers during module loading and unloading so
we swap the pointers.

 [PATCH 3/8] [xen/events] Export xen_clear_irq_pending and xen_poll_irq_timeout
as the PCI front module utilizes them.
 [PATCH 4/8] [xen/pci] Provide a registration mechanism for the PCI frontend
module functions.
 [PATCH 5/8] [xen/pci] Switch over from pci_frontend_* to xen_pci_frontend*
functions.
 [PATCH 6/8] [xen/pcifront] Register pci_frontend_[enable|disable]_[msi|msix]
functions.
 [PATCH 7/8] [xen/pcifront] Enable Xen PCI front to be compiled as module and
also on x86.
 [PATCH 8/8] [xen/pcifront] Check pci_claim_resource return value.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Author: Jesse Barnes <jbarnes@virtuousgeek.org>
 Date:   Tue Jun 30 21:45:44 2009 -0700

    PCI: export pci_claim_resource for driver use

    yenta needs this for example.

    Acked-by: Matthew Wilcox <willy@linux.intel.com>
    Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
    Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/pci/setup-res.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 88cdd1a..706f82d 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -119,6 +119,7 @@ int pci_claim_resource(struct pci_dev *dev, int resource)
 
 	return err;
 }
+EXPORT_SYMBOL(pci_claim_resource);
 
 #ifdef CONFIG_PCI_QUIRKS
 void pci_disable_bridge_window(struct pci_dev *dev)
-- 
1.6.2.5


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

In preperation of modularizing Xen-pcifront the pci_walk_bus
needs to be exported so that the xen-pcifront module can walk
call the pci subsystem to walk the PCI devices and claim them.

Acked-by: Jesse Barnes <jbarnes@virtuousgeek.org>
[http://marc.info/?l=linux-pci&m=126149958010298&w=2]
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/pci/bus.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index cef28a7..1940183 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -249,6 +249,7 @@ void pci_walk_bus(struct pci_bus *top, int (*cb)(struct
pci_dev *, void *),
 	up_read(&pci_bus_sem);
 }
 
+EXPORT_SYMBOL_GPL(pci_walk_bus);
 EXPORT_SYMBOL(pci_bus_alloc_resource);
 EXPORT_SYMBOL_GPL(pci_bus_add_device);
 EXPORT_SYMBOL(pci_bus_add_devices);
-- 
1.6.2.5


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

These two functions are used in the IRQ handler of the PCI front
driver to check its state. They need to be accessible if
PCI front is compiled as a module.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/xen/events.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index d977ab2..9836768 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -1253,7 +1253,7 @@ void xen_clear_irq_pending(int irq)
 	if (VALID_EVTCHN(evtchn))
 		clear_evtchn(evtchn);
 }
-
+EXPORT_SYMBOL(xen_clear_irq_pending);
 void xen_set_irq_pending(int irq)
 {
 	int evtchn = evtchn_from_irq(irq);
@@ -1290,6 +1290,7 @@ void xen_poll_irq_timeout(int irq, u64 timeout)
 			BUG();
 	}
 }
+EXPORT_SYMBOL(xen_poll_irq_timeout);
 /* Poll waiting for an irq to become pending.  In the usual case, the
    irq will be disabled so it won''t deliver an interrupt. */
 void xen_poll_irq(int irq)
-- 
1.6.2.5


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

This registration mechanism allows us to "hook-up" the PCI frontend
enable/disable MSI/MSI-X calls when the module (or if it is compiled in)
is loaded. If it is not loaded, the caller (next patch) will just
call an empty stub.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/include/asm/xen/pci.h |   45 ++++++++++++++++++++++++++++-----------
 1 files changed, 32 insertions(+), 13 deletions(-)

diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
index 6c022c8..9f0a14b 100644
--- a/arch/x86/include/asm/xen/pci.h
+++ b/arch/x86/include/asm/xen/pci.h
@@ -35,24 +35,43 @@ static inline int xen_setup_msi_irqs(struct pci_dev *dev,
int nvec, int type)
 	return -1;
 }
 #endif
-#if defined(CONFIG_PCI_MSI) && defined(CONFIG_XEN_PCIDEV_FRONTEND)
-/* Defined in drivers/pci/xen-pcifront.c */
-int pci_frontend_enable_msi(struct pci_dev *dev, int **vectors);
-void pci_frontend_disable_msi(struct pci_dev *dev);
-int pci_frontend_enable_msix(struct pci_dev *dev,
-			     int **vectors, int nvec);
-void pci_frontend_disable_msix(struct pci_dev *dev);
-#else
-static inline int pci_frontend_enable_msi(struct pci_dev *dev, int **vectors)
+#if defined(CONFIG_PCI_MSI)
+
+/* The drivers/pci/xen-pcifront.c sets this structure to
+ * its own functions.
+ */
+struct xen_pci_frontend_ops {
+	int (*enable_msi)(struct pci_dev *dev, int **vectors);
+	void (*disable_msi)(struct pci_dev *dev);
+	int (*enable_msix)(struct pci_dev *dev, int **vectors, int nvec);
+	void (*disable_msix)(struct pci_dev *dev);
+};
+
+extern struct xen_pci_frontend_ops *xen_pci_frontend;
+	
+static inline int xen_pci_frontend_enable_msi(struct pci_dev *dev,
+					      int **vectors)
 {
+	if (xen_pci_frontend && xen_pci_frontend->enable_msi)
+		return xen_pci_frontend->enable_msi(dev, vectors);
 	return -1;
 }
-static inline void pci_frontend_disable_msi(struct pci_dev *dev) { }
-static inline int pci_frontend_enable_msix(struct pci_dev *dev,
-					   int **vectors, int nvec)
+static inline void xen_pci_frontend_disable_msi(struct pci_dev *dev)
 {
+	if (xen_pci_frontend && xen_pci_frontend->disable_msi)
+			xen_pci_frontend->disable_msi(dev);
+}
+static inline int xen_pci_frontend_enable_msix(struct pci_dev *dev,
+					       int **vectors, int nvec)
+{
+	if (xen_pci_frontend && xen_pci_frontend->enable_msix)
+		return xen_pci_frontend->enable_msix(dev, vectors, nvec);
 	return -1;
 }
-static inline void pci_frontend_disable_msix(struct pci_dev *dev) { }
+static inline void xen_pci_frontend_disable_msix(struct pci_dev *dev)
+{
+	if (xen_pci_frontend && xen_pci_frontend->disable_msix)
+			xen_pci_frontend->disable_msix(dev);
+}
 #endif
 #endif	/* _ASM_X86_XEN_PCI_H */
-- 
1.6.2.5


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Utilize the new mechanism which allows us to call those functions
even if the PCI frontend module is not loaded.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/xen/pci.c |   12 ++++++++----
 1 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/arch/x86/xen/pci.c b/arch/x86/xen/pci.c
index b572f26..b1d0fb0 100644
--- a/arch/x86/xen/pci.c
+++ b/arch/x86/xen/pci.c
@@ -88,6 +88,10 @@ void __init xen_setup_pirqs(void)
 }
 
 #ifdef CONFIG_PCI_MSI
+
+struct xen_pci_frontend_ops *xen_pci_frontend;
+EXPORT_SYMBOL(xen_pci_frontend);
+
 int xen_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
 {
 	int irq, ret, i;
@@ -100,9 +104,9 @@ int xen_setup_msi_irqs(struct pci_dev *dev, int nvec, int
type)
 
 	if (!xen_initial_domain()) {
 		if (type == PCI_CAP_ID_MSIX)
-			ret = pci_frontend_enable_msix(dev, &v, nvec);
+			ret = xen_pci_frontend_enable_msix(dev, &v, nvec);
 		else
-			ret = pci_frontend_enable_msi(dev, &v);
+			ret = xen_pci_frontend_enable_msi(dev, &v);
 		if (ret)
 			goto error;
 	}
@@ -134,9 +138,9 @@ void xen_teardown_msi_dev(struct pci_dev *dev)
 
 		msidesc = list_entry(dev->msi_list.next, struct msi_desc, list);
 		if (msidesc->msi_attrib.is_msix)
-			pci_frontend_disable_msix(dev);
+			xen_pci_frontend_disable_msix(dev);
 		else
-			pci_frontend_disable_msi(dev);
+			xen_pci_frontend_disable_msi(dev);
 	}
 
 }
-- 
1.6.2.5


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

[xen/pcifront] Register pci_frontend_[enable|disable]_[msi|msix] functions.
    
    Take advantage of the new registration mechanism.
    Remove the EXPORT_SYMBOL and alter the four functions to be static.

diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index cc3b51b..f262b6b 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -256,8 +256,8 @@ struct pci_ops pcifront_bus_ops = {
 };
 
 #ifdef CONFIG_PCI_MSI
-int pci_frontend_enable_msix(struct pci_dev *dev,
-		int **vector, int nvec)
+static int pci_frontend_enable_msix(struct pci_dev *dev,
+				    int **vector, int nvec)
 {
 	int err;
 	int i;
@@ -304,9 +304,8 @@ int pci_frontend_enable_msix(struct pci_dev *dev,
 		return err;
 	}
 }
-EXPORT_SYMBOL_GPL(pci_frontend_enable_msix);
 
-void pci_frontend_disable_msix(struct pci_dev *dev)
+static void pci_frontend_disable_msix(struct pci_dev *dev)
 {
 	int err;
 	struct xen_pci_op op = {
@@ -324,9 +323,8 @@ void pci_frontend_disable_msix(struct pci_dev *dev)
 	if (err)
 		dev_err(&dev->dev, "pci_disable_msix get err %x\n", err);
 }
-EXPORT_SYMBOL_GPL(pci_frontend_disable_msix);
 
-int pci_frontend_enable_msi(struct pci_dev *dev, int **vector)
+static int pci_frontend_enable_msi(struct pci_dev *dev, int **vector)
 {
 	int err;
 	struct xen_pci_op op = {
@@ -348,9 +346,8 @@ int pci_frontend_enable_msi(struct pci_dev *dev, int
**vector)
 	}
 	return err;
 }
-EXPORT_SYMBOL(pci_frontend_enable_msi);
 
-void pci_frontend_disable_msi(struct pci_dev *dev)
+static void pci_frontend_disable_msi(struct pci_dev *dev)
 {
 	int err;
 	struct xen_pci_op op = {
@@ -372,7 +369,24 @@ void pci_frontend_disable_msi(struct pci_dev *dev)
 		/* how can pciback notify us fail? */
 		printk(KERN_DEBUG "get fake response frombackend \n");
 }
-EXPORT_SYMBOL_GPL(pci_frontend_disable_msi);
+
+static void pci_frontend_registrar(int enable)
+{
+
+	struct xen_pci_frontend_ops pci_frontend_ops = {
+		.enable_msi = pci_frontend_enable_msi,
+		.disable_msi = pci_frontend_disable_msi,
+		.enable_msix = pci_frontend_enable_msix,
+		.disable_msix = pci_frontend_disable_msix,
+	};
+
+	if (enable)
+		xen_pci_frontend = &pci_frontend_ops;
+	else
+		xen_pci_frontend = NULL;
+};
+#else
+static inline void pci_frontend_registrar(int enable) { };
 #endif /* CONFIG_PCI_MSI */
 
 /* Claim resources for the PCI frontend as-is, backend won''t allow
changes */
@@ -1119,12 +1133,15 @@ static int __init pcifront_init(void)
 	if (!xen_domain())
 		return -ENODEV;
 
+	pci_frontend_registrar(1 /* enable */);
+
 	return xenbus_register_frontend(&xenbus_pcifront_driver);
 }
 
 static void __exit pcifront_cleanup(void)
 {
 	xenbus_unregister_driver(&xenbus_pcifront_driver);
+	pci_frontend_registrar(0 /* disable */);
 }
 module_init(pcifront_init);
 module_exit(pcifront_cleanup);

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Allow the xen-pcifront driver to be compiled as module. Found out
that it was turned off on x86 - testing showed no problems so this
patch turns it on 32-bit as well.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/pci/Kconfig |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index 4a658b1..a1a08f2 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -52,8 +52,8 @@ config PCI_STUB
 	  When in doubt, say N.
 
 config XEN_PCIDEV_FRONTEND
-        bool "Xen PCI Frontend"
-        depends on XEN && PCI && X86_64
+        tristate "Xen PCI Frontend"
+        depends on XEN && PCI
         select HOTPLUG
 	select XEN_XENBUS_FRONTEND
         help
-- 
1.6.2.5


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Currently if we allocate more than 4GB to the unprivileged domain
we cannot use PCI passthrough. Make the user aware that the device
is no good.

Will remove this once the 4GB issue is fixed.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/pci/xen-pcifront.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index 10b2227..9ad0ea5 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -402,7 +402,12 @@ static int pcifront_claim_resource(struct pci_dev *dev,
void *data)
 		if (!r->parent && r->start && r->flags) {
 			dev_dbg(&pdev->xdev->dev, "claiming resource %s/%d\n",
 				pci_name(dev), i);
-			pci_claim_resource(dev, i);
+			if (pci_claim_resource(dev, i)) {
+				dev_err(&pdev->xdev->dev, "Could not claim "
+					"resource %s/%d! Device offline. Try "
+					"giving less than 4GB to domain.\n",
+					pci_name(dev), i);
+			}
 		}
 	}
 
-- 
1.6.2.5


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel