horms@vergenet.net
2009-Mar-25 03:32 UTC
[Xen-devel] [patch 0/4 v2] ioemu: specify the slot for pass-through devices
Currently a slot may be specified for a hot-plug device,
but not for a pass-through device that is inserted at boot time.
This patch adds support for the latter.
The syntax is:
BUS:DEV.FUNC[@VSLOT]
e.g: 0000:00:1d:0@7
This may be important as recent changes that allow any free PCI slot to be
used for pass-through (and hotplug) may case pass-through devices to be
assigned in different locations to before. Amongst other things, specifying
the slot will allow users to move them back, if there is a need.
These changes need the patches that I posted today as the series
"[patch 0/2] ioemu: Allow any unused PCI device to be used for
pass-through"
There is also a xend portion of this series, which will be posted separately.
History
-------
This series was originally posted as:
[rfc 0/4] ioemu: specify the slot for pass-through devices
├>[rfc 1/4] ioemu: Register dpci_infos.php_devs.pt_dev in a common location
├>[rfc 2/4] ioemu: Do slot parsing inside of next_bdf
├>[rfc 3/4] ioemu: make __insert_to_pci_slot indempotent
└>[rfc 4/4] ioemu: allow xend to specify the slot for pass-through devices
The 3rd patch has been applied to qemu-xen-unstable.
The first three patches of this series comprises of patches
1, 2 and 4 of the original series, which have been updated
as per their individual changelog entries.
There is a new 4th patch which actually reads vslot information
from xend using xenstore. There is a corresponding patch to
xend for this new 4th patch. All other xend patches relating to this change
have already been merged into xen-unstable.
--
Simon Horman
VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
horms@vergenet.net
2009-Mar-25 03:32 UTC
[Xen-devel] [patch 1/4 v2] ioemu: Register dpci_infos.php_devs.pt_dev in a common location
Currently power_on_php_slot() sets dpci_infos.php_devs[i].pt_dev itself
after the call to register_real_device(). While pt_init relies on
power_on_php_slot() to set this pointer.
It seems sensible to use the same behaviour for both callers.
There are no other callers of register_real_device()
Signed-off-by: Simon Horman <horms@verge.net.au>
---
* Sat, 21 Mar 2009 09:24:36 +1100
Initial Public Release
* Wed, 25 Mar 2009 14:18:56 +1100
Removed bogus second argument to PCI_SLOT()
Index: ioemu-remote/hw/pass-through.c
==================================================================---
ioemu-remote.orig/hw/pass-through.c 2009-03-25 13:02:58.000000000 +1100
+++ ioemu-remote/hw/pass-through.c 2009-03-25 14:22:44.000000000 +1100
@@ -3644,7 +3644,7 @@ struct pt_dev * register_real_device(PCI
struct pci_dev *pci_dev;
uint8_t e_device, e_intx;
struct pci_config_cf8 machine_bdf;
- int free_slot = -1;
+ int free_slot;
char *key, *val;
int msi_translate, power_mgmt;
@@ -3732,8 +3732,7 @@ struct pt_dev * register_real_device(PCI
return NULL;
}
- if ( free_slot > 0 )
- dpci_infos.php_devs[free_slot].pt_dev = assigned_device;
+ dpci_infos.php_devs[PCI_SLOT(e_devfn)].pt_dev = assigned_device;
assigned_device->pci_dev = pci_dev;
assigned_device->msi_trans_cap = msi_translate;
@@ -3900,8 +3899,6 @@ int power_on_php_slot(int slot)
php_dev->opt = NULL;
- php_dev->pt_dev = pt_dev;
-
return 0;
}
--
--
Simon Horman
VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
horms@vergenet.net
2009-Mar-25 03:32 UTC
[Xen-devel] [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf
Currently only hotplug provides vslot information from xend.
A subsequent patch will have xend provide this information
for boot-time inserted pass-through devices too.
With this in mind, this patch makes some infrastructure
to parse bdf + slot information.
Signed-off-by: Simon Horman <horms@verge.net.au>
---
* Sat, 21 Mar 2009 09:24:37 +1100
Initial Public Release
* Mon, 23 Mar 2009 20:38:40 +1100
Remove a bogus second assignment of *vslot* inside next_bdf().
Unfortunately this crept into the version I posted while
my testing was against a versino without it.
* Wed, 25 Mar 2009 14:20:29 +1100
Remove register_real_device fragment, it has been folded into
"ioemu: Register dpci_infos.php_devs.pt_dev in a common location"
Index: ioemu-remote/hw/pass-through.c
==================================================================---
ioemu-remote.orig/hw/pass-through.c 2009-03-25 14:13:18.000000000 +1100
+++ ioemu-remote/hw/pass-through.c 2009-03-25 14:21:50.000000000 +1100
@@ -776,9 +776,10 @@ static int token_value(char *token)
return strtol(token, NULL, 16);
}
-static int next_bdf(char **str, int *seg, int *bus, int *dev, int *func, char
**opt)
+static int next_bdf(char **str, int *seg, int *bus, int *dev, int *func,
+ char **opt, int *vslot)
{
- char *token;
+ char *token, *endptr;
const char *delim = ":.-";
if ( !(*str) ||
@@ -795,7 +796,20 @@ static int next_bdf(char **str, int *seg
*dev = token_value(token);
token = strsep(str, delim);
- *opt = strchr(token, '','');
+
+ *opt = strchr(token, ''@'');
+ if (*opt)
+ {
+ *(*opt)++ = ''\0'';
+ *vslot = token_value(*opt);
+ }
+ else
+ {
+ *vslot = AUTO_PHP_SLOT;
+ *opt = token;
+ }
+
+ *opt = strchr(*opt, '','');
if (*opt)
*(*opt)++ = ''\0'';
@@ -881,14 +895,9 @@ found:
int insert_to_pci_slot(char *bdf_slt)
{
int seg, bus, dev, func, slot;
- char *bdf_str, *slt_str, *opt, *endptr;
- const char *delim="@";
-
- bdf_str = strsep(&bdf_slt, delim);
- slt_str = bdf_slt;
- slot = token_value(slt_str);
+ char *opt;
- if ( !next_bdf(&bdf_str, &seg, &bus, &dev, &func,
&opt))
+ if ( !next_bdf(&bdf_slt, &seg, &bus, &dev, &func,
&opt, &slot) )
{
return -1;
}
@@ -916,10 +925,10 @@ int test_pci_slot(int slot)
/* find the pci slot for pass-through dev with specified BDF */
int bdf_to_slot(char *bdf_str)
{
- int seg, bus, dev, func, i;
+ int seg, bus, dev, func, slot, i;
char *opt;
- if ( !next_bdf(&bdf_str, &seg, &bus, &dev, &func,
&opt))
+ if ( !next_bdf(&bdf_str, &seg, &bus, &dev, &func,
&opt, &slot))
{
return -1;
}
@@ -3910,7 +3919,7 @@ int power_off_php_slot(int php_slot)
int pt_init(PCIBus *e_bus, const char *direct_pci)
{
- int seg, b, d, f, status = -1;
+ int seg, b, d, f, s, status = -1;
struct pt_dev *pt_dev;
struct pci_access *pci_access;
char *vslots;
@@ -3946,7 +3955,7 @@ int pt_init(PCIBus *e_bus, const char *d
vslots = qemu_mallocz ( strlen(direct_pci) / 3 );
/* Assign given devices to guest */
- while ( next_bdf(&direct_pci_p, &seg, &b, &d, &f,
&opt) )
+ while ( next_bdf(&direct_pci_p, &seg, &b, &d, &f,
&opt, &s) )
{
/* Register real device with the emulated bus */
pt_dev = register_real_device(e_bus, "DIRECT PCI",
PT_VIRT_DEVFN_AUTO,
--
--
Simon Horman
VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
horms@vergenet.net
2009-Mar-25 03:32 UTC
[Xen-devel] [patch 3/4 v2] ioemu: allow xend to specify the slot for pass-through devices
Currently a slot may be specified for a hot-plug device,
but not for a pass-through device that is inserted at boot time.
This patch adds support for the latter.
The syntax is:
BUS:DEV.FUNC[@VSLOT]
e.g: 0000:00:1d:0@7
This may be important as recent changes that allow any free PCI
slot to be used for pass-through (and hotplug) may case pass-through
devices to be assigned in different locations to before. Amongst
other things, specifying the slot will allow users to move them
back, if there is a need.
There is also a xend portion of this patch, which will be posted separately.
Signed-off-by: Simon Horman <horms@vereg.net.au>
---
Some implementation details
* Pass the slot parsed from xend to register_real_device().
If none was passed, next_bdf() will supply AUTO_PHP_SLOT.
* Make use of the fact that __insert_to_pci_slot() is
now indempotent and always call it in register_real_device()
* Change the error message int he case where __insert_to_pci_slot()
fails to not refer to hot-plut. e_devfn will always
be non-auto in the hotplug case as register_real_device() is
called by power_on_php_slot() which is called by acpi_php_add()
which is called by acpi_php_add(). acpi_php_add() handles
calling __insert_to_pci_slot() and acpi_php_add() handles
the case where the call failed, before calling power_on_php_slot().
* Remove PT_VIRT_DEVFN_AUTO, it is no longer used.
Index: ioemu-remote/hw/pass-through.c
==================================================================---
ioemu-remote.orig/hw/pass-through.c 2009-03-20 21:34:06.000000000 +1100
+++ ioemu-remote/hw/pass-through.c 2009-03-20 21:35:25.000000000 +1100
@@ -3560,7 +3560,7 @@ static int pt_pmcsr_reg_restore(struct p
}
struct pt_dev * register_real_device(PCIBus *e_bus,
- const char *e_dev_name, int e_devfn, uint8_t r_bus, uint8_t r_dev,
+ const char *e_dev_name, int e_slot, uint8_t r_bus, uint8_t r_dev,
uint8_t r_func, uint32_t machine_irq, struct pci_access *pci_access,
char *opt)
{
@@ -3569,7 +3569,6 @@ struct pt_dev * register_real_device(PCI
struct pci_dev *pci_dev;
uint8_t e_device, e_intx;
struct pci_config_cf8 machine_bdf;
- int free_slot;
char *key, *val;
int msi_translate;
@@ -3592,15 +3591,10 @@ struct pt_dev * register_real_device(PCI
pci_fill_info(pci_dev, PCI_FILL_IRQ | PCI_FILL_BASES | PCI_FILL_ROM_BASE |
PCI_FILL_SIZES);
pt_libpci_fixup(pci_dev);
- if ( e_devfn == PT_VIRT_DEVFN_AUTO ) {
- /*indicate a static assignment(not hotplug), so find a free PCI hot
plug slot */
- free_slot = __insert_to_pci_slot(r_bus, r_dev, r_func,
- AUTO_PHP_SLOT, NULL);
- if ( free_slot < 0 ) {
- PT_LOG("Error: no free virtual PCI hot plug slot, thus no live
migration.\n");
- return NULL;
- }
- e_devfn = free_slot << 3;
+ e_slot = __insert_to_pci_slot(r_bus, r_dev, r_func, e_slot, NULL);
+ if ( e_slot < 0 ) {
+ PT_LOG("Error: no free virtual PCI slot\n");
+ return NULL;
}
msi_translate = direct_pci_msitranslate;
@@ -3633,7 +3627,7 @@ struct pt_dev * register_real_device(PCI
/* Register device */
assigned_device = (struct pt_dev *) pci_register_device(e_bus, e_dev_name,
- sizeof(struct pt_dev), e_devfn,
+ sizeof(struct pt_dev), PCI_DEVFN(e_slot, 0),
pt_pci_read_config, pt_pci_write_config);
if ( assigned_device == NULL )
{
@@ -3641,7 +3635,7 @@ struct pt_dev * register_real_device(PCI
return NULL;
}
- dpci_infos.php_devs[PCI_SLOT(e_devfn)].pt_dev = assigned_device;
+ dpci_infos.php_devs[e_slot].pt_dev = assigned_device;
assigned_device->pci_dev = pci_dev;
assigned_device->msi_trans_cap = msi_translate;
@@ -3802,7 +3796,7 @@ int power_on_php_slot(int slot)
pt_dev register_real_device(dpci_infos.e_bus,
"DIRECT PCI",
- slot << 3,
+ slot,
php_dev->r_bus,
php_dev->r_dev,
php_dev->r_func,
@@ -3862,7 +3856,7 @@ int pt_init(PCIBus *e_bus, const char *d
while ( next_bdf(&direct_pci_p, &seg, &b, &d, &f,
&opt, &s) )
{
/* Register real device with the emulated bus */
- pt_dev = register_real_device(e_bus, "DIRECT PCI",
PT_VIRT_DEVFN_AUTO,
+ pt_dev = register_real_device(e_bus, "DIRECT PCI", s,
b, d, f, PT_MACHINE_IRQ_AUTO, pci_access, opt);
if ( pt_dev == NULL )
{
Index: ioemu-remote/hw/pass-through.h
==================================================================---
ioemu-remote.orig/hw/pass-through.h 2009-03-20 21:24:13.000000000 +1100
+++ ioemu-remote/hw/pass-through.h 2009-03-20 21:35:25.000000000 +1100
@@ -39,7 +39,6 @@
// #define PT_DEBUG_PCI_CONFIG_ACCESS
#define PT_MACHINE_IRQ_AUTO (0xFFFFFFFF)
-#define PT_VIRT_DEVFN_AUTO (-1)
/* Misc PCI constants that should be moved to a separate library :) */
#define PCI_CONFIG_SIZE (256)
--
--
Simon Horman
VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
horms@vergenet.net
2009-Mar-25 03:32 UTC
[Xen-devel] [patch 4/4 v2] ioemu: Read pass-through vslot from xend
This reads the vslot information supplied by xend, and should be
the final piece for this feature on the ioemu side.
There is also a xend portion of this patch which I will post separately.
Signed-off-by: Simon Horman <horms@verge.net.au>
Index: ioemu-remote/xenstore.c
==================================================================---
ioemu-remote.orig/xenstore.c 2009-03-24 09:25:36.000000000 +1100
+++ ioemu-remote/xenstore.c 2009-03-24 22:58:28.000000000 +1100
@@ -582,6 +582,24 @@ void xenstore_parse_domain_config(int hv
strcat(direct_pci_str, dev);
+ if (pasprintf(&buf,
"/local/domain/0/backend/pci/%u/%u/vslot-%d",
+ hvm_domid, pci_devid, i) != -1) {
+ free(dev);
+ dev = xs_read(xsh, XBT_NULL, buf, &len);
+ }
+ if ( dev ) {
+ if (strlen(dev) + strlen(direct_pci_str) >
+ DIRECT_PCI_STR_LEN - 2) {
+ fprintf(stderr, "qemu: too many pci pass-through
"
+ "devices\n");
+ memset(direct_pci_str, 0, DIRECT_PCI_STR_LEN);
+ goto out;
+ }
+ strcat(direct_pci_str, "@");
+ strcat(direct_pci_str, dev);
+ }
+
+
if (pasprintf(&buf,
"/local/domain/0/backend/pci/%u/%u/opts-%d",
hvm_domid, pci_devid, i) != -1) {
free(dev);
--
--
Simon Horman
VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Cui, Dexuan
2009-Mar-25 09:54 UTC
RE: [Xen-devel] [patch 4/4 v2] ioemu: Read pass-through vslot from xend
horms@vergenet.net wrote:> This reads the vslot information supplied by xend, and should be > the final piece for this feature on the ioemu side. > > There is also a xend portion of this patch which I will post > separately. > > Signed-off-by: Simon Horman <horms@verge.net.au> > > > Index: ioemu-remote/xenstore.c > ==================================================================> --- ioemu-remote.orig/xenstore.c 2009-03-24 09:25:36.000000000 +1100 > +++ ioemu-remote/xenstore.c 2009-03-24 22:58:28.000000000 +1100 > @@ -582,6 +582,24 @@ void xenstore_parse_domain_config(int hv > > strcat(direct_pci_str, dev); > > + if (pasprintf(&buf, > "/local/domain/0/backend/pci/%u/%u/vslot-%d", + > hvm_domid, pci_devid, i) != -1) { + free(dev); > + dev = xs_read(xsh, XBT_NULL, buf, &len); > + } > + if ( dev ) { > + if (strlen(dev) + strlen(direct_pci_str) > > + DIRECT_PCI_STR_LEN - 2) { > + fprintf(stderr, "qemu: too many pci pass-through > " + "devices\n"); > + memset(direct_pci_str, 0, DIRECT_PCI_STR_LEN); > + goto out; > + } > + strcat(direct_pci_str, "@"); > + strcat(direct_pci_str, dev); > + } > + > + > if (pasprintf(&buf, > > "/local/domain/0/backend/pci/%u/%u/opts-%d", > hvm_domid, pci_devid, i) != -1) { free(dev); > > --Oh, I see the patches... :-) Thanks, -- Dexuan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2009-Mar-25 11:37 UTC
[Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf
horms@vergenet.net writes ("[patch 2/4 v2] ioemu: Do slot parsing inside of
next_bdf"):> Currently only hotplug provides vslot information from xend.
> A subsequent patch will have xend provide this information
> for boot-time inserted pass-through devices too.
If I apply 2/4, 3/4 and 4/4 of your series, will it still work with
old xends ?
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Simon Horman
2009-Mar-25 23:23 UTC
Re: [Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf
On Wed, Mar 25, 2009 at 11:37:36AM +0000, Ian Jackson wrote:> horms@vergenet.net writes ("[patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf"): > > Currently only hotplug provides vslot information from xend. > > A subsequent patch will have xend provide this information > > for boot-time inserted pass-through devices too. > > If I apply 2/4, 3/4 and 4/4 of your series, will it still work with > old xends ?Hi Ian, It depends what you mean by an old xend (do you have a particular version in mind) and I will need to re-check, but I don''t think that those changes break compatibility. -- Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Simon Horman
2009-Mar-26 00:18 UTC
Re: [Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf
On Thu, Mar 26, 2009 at 10:23:52AM +1100, Simon Horman wrote:> On Wed, Mar 25, 2009 at 11:37:36AM +0000, Ian Jackson wrote: > > horms@vergenet.net writes ("[patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf"): > > > Currently only hotplug provides vslot information from xend. > > > A subsequent patch will have xend provide this information > > > for boot-time inserted pass-through devices too. > > > > If I apply 2/4, 3/4 and 4/4 of your series, will it still work with > > old xends ? > > Hi Ian, > > It depends what you mean by an old xend (do you have a particular > version in mind) and I will need to re-check, but I don''t think > that those changes break compatibility.Hi, I have checked against the following trees, and there do not seem to be any compatibiliy issues introduced by any of these patches. qemu-xen-unstable 545882583159e9cc53cde5965277967092459f37 cirrus: Do not clear vram area to 0xff when not yet allocated. xen-unstable 19426:0b13d9787622 libxc: fix link error of xc_save on ia64 During testing I noted that 2/4 needs to be applied before 3/4, but there don''t seem to be any other ordering constraints. More speficically, I tested the following with static and hot-plug pass-through and found them to work. Patches applied - none - 2/4 - 4/4 - 2/4 3/4 - 2/4 4/4 - 2/4 3/4 4/4 -- Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2009-Mar-31 10:43 UTC
Re: [Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf
Simon Horman writes ("Re: [Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot
parsing inside of next_bdf"):> It depends what you mean by an old xend (do you have a particular
> version in mind) and I will need to re-check, but I don''t think
> that those changes break compatibility.
I just meant that since you were submitting corresponding patches to
xen-unstable and qemu-xen-unstable, I was worried that you might
introduce a cross-version incompatibility. Reviewing the parsing code
by eye didn''t reassure me so I thought I''d ask.
I''m trying to be better about keeping version-skewed versions of the
trees compatible, as not doing so makes many things much more
difficult. Anyway, thanks for the reassurance in this and your next
message.
Regards,
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Simon Horman
2009-Mar-31 10:51 UTC
Re: [Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf
On Tue, Mar 31, 2009 at 11:43:46AM +0100, Ian Jackson wrote:> Simon Horman writes ("Re: [Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf"): > > It depends what you mean by an old xend (do you have a particular > > version in mind) and I will need to re-check, but I don''t think > > that those changes break compatibility. > > I just meant that since you were submitting corresponding patches to > xen-unstable and qemu-xen-unstable, I was worried that you might > introduce a cross-version incompatibility. Reviewing the parsing code > by eye didn''t reassure me so I thought I''d ask. > > I''m trying to be better about keeping version-skewed versions of the > trees compatible, as not doing so makes many things much more > difficult. Anyway, thanks for the reassurance in this and your next > message.Thanks for the clarification. I don''t believe that any of the patches that I have posted but haven''t been applied break compatibility between xen-unstable and qemu-xen-unstable. In future I will try and be better about noting when any incompatibilities are introduced. -- Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel