horms@vergenet.net
2009-Mar-25 03:32 UTC
[Xen-devel] [patch 0/4 v2] ioemu: specify the slot for pass-through devices
Currently a slot may be specified for a hot-plug device, but not for a pass-through device that is inserted at boot time. This patch adds support for the latter. The syntax is: BUS:DEV.FUNC[@VSLOT] e.g: 0000:00:1d:0@7 This may be important as recent changes that allow any free PCI slot to be used for pass-through (and hotplug) may case pass-through devices to be assigned in different locations to before. Amongst other things, specifying the slot will allow users to move them back, if there is a need. These changes need the patches that I posted today as the series "[patch 0/2] ioemu: Allow any unused PCI device to be used for pass-through" There is also a xend portion of this series, which will be posted separately. History ------- This series was originally posted as: [rfc 0/4] ioemu: specify the slot for pass-through devices ├>[rfc 1/4] ioemu: Register dpci_infos.php_devs.pt_dev in a common location ├>[rfc 2/4] ioemu: Do slot parsing inside of next_bdf ├>[rfc 3/4] ioemu: make __insert_to_pci_slot indempotent └>[rfc 4/4] ioemu: allow xend to specify the slot for pass-through devices The 3rd patch has been applied to qemu-xen-unstable. The first three patches of this series comprises of patches 1, 2 and 4 of the original series, which have been updated as per their individual changelog entries. There is a new 4th patch which actually reads vslot information from xend using xenstore. There is a corresponding patch to xend for this new 4th patch. All other xend patches relating to this change have already been merged into xen-unstable. -- Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
horms@vergenet.net
2009-Mar-25 03:32 UTC
[Xen-devel] [patch 1/4 v2] ioemu: Register dpci_infos.php_devs.pt_dev in a common location
Currently power_on_php_slot() sets dpci_infos.php_devs[i].pt_dev itself after the call to register_real_device(). While pt_init relies on power_on_php_slot() to set this pointer. It seems sensible to use the same behaviour for both callers. There are no other callers of register_real_device() Signed-off-by: Simon Horman <horms@verge.net.au> --- * Sat, 21 Mar 2009 09:24:36 +1100 Initial Public Release * Wed, 25 Mar 2009 14:18:56 +1100 Removed bogus second argument to PCI_SLOT() Index: ioemu-remote/hw/pass-through.c ==================================================================--- ioemu-remote.orig/hw/pass-through.c 2009-03-25 13:02:58.000000000 +1100 +++ ioemu-remote/hw/pass-through.c 2009-03-25 14:22:44.000000000 +1100 @@ -3644,7 +3644,7 @@ struct pt_dev * register_real_device(PCI struct pci_dev *pci_dev; uint8_t e_device, e_intx; struct pci_config_cf8 machine_bdf; - int free_slot = -1; + int free_slot; char *key, *val; int msi_translate, power_mgmt; @@ -3732,8 +3732,7 @@ struct pt_dev * register_real_device(PCI return NULL; } - if ( free_slot > 0 ) - dpci_infos.php_devs[free_slot].pt_dev = assigned_device; + dpci_infos.php_devs[PCI_SLOT(e_devfn)].pt_dev = assigned_device; assigned_device->pci_dev = pci_dev; assigned_device->msi_trans_cap = msi_translate; @@ -3900,8 +3899,6 @@ int power_on_php_slot(int slot) php_dev->opt = NULL; - php_dev->pt_dev = pt_dev; - return 0; } -- -- Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
horms@vergenet.net
2009-Mar-25 03:32 UTC
[Xen-devel] [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf
Currently only hotplug provides vslot information from xend. A subsequent patch will have xend provide this information for boot-time inserted pass-through devices too. With this in mind, this patch makes some infrastructure to parse bdf + slot information. Signed-off-by: Simon Horman <horms@verge.net.au> --- * Sat, 21 Mar 2009 09:24:37 +1100 Initial Public Release * Mon, 23 Mar 2009 20:38:40 +1100 Remove a bogus second assignment of *vslot* inside next_bdf(). Unfortunately this crept into the version I posted while my testing was against a versino without it. * Wed, 25 Mar 2009 14:20:29 +1100 Remove register_real_device fragment, it has been folded into "ioemu: Register dpci_infos.php_devs.pt_dev in a common location" Index: ioemu-remote/hw/pass-through.c ==================================================================--- ioemu-remote.orig/hw/pass-through.c 2009-03-25 14:13:18.000000000 +1100 +++ ioemu-remote/hw/pass-through.c 2009-03-25 14:21:50.000000000 +1100 @@ -776,9 +776,10 @@ static int token_value(char *token) return strtol(token, NULL, 16); } -static int next_bdf(char **str, int *seg, int *bus, int *dev, int *func, char **opt) +static int next_bdf(char **str, int *seg, int *bus, int *dev, int *func, + char **opt, int *vslot) { - char *token; + char *token, *endptr; const char *delim = ":.-"; if ( !(*str) || @@ -795,7 +796,20 @@ static int next_bdf(char **str, int *seg *dev = token_value(token); token = strsep(str, delim); - *opt = strchr(token, '',''); + + *opt = strchr(token, ''@''); + if (*opt) + { + *(*opt)++ = ''\0''; + *vslot = token_value(*opt); + } + else + { + *vslot = AUTO_PHP_SLOT; + *opt = token; + } + + *opt = strchr(*opt, '',''); if (*opt) *(*opt)++ = ''\0''; @@ -881,14 +895,9 @@ found: int insert_to_pci_slot(char *bdf_slt) { int seg, bus, dev, func, slot; - char *bdf_str, *slt_str, *opt, *endptr; - const char *delim="@"; - - bdf_str = strsep(&bdf_slt, delim); - slt_str = bdf_slt; - slot = token_value(slt_str); + char *opt; - if ( !next_bdf(&bdf_str, &seg, &bus, &dev, &func, &opt)) + if ( !next_bdf(&bdf_slt, &seg, &bus, &dev, &func, &opt, &slot) ) { return -1; } @@ -916,10 +925,10 @@ int test_pci_slot(int slot) /* find the pci slot for pass-through dev with specified BDF */ int bdf_to_slot(char *bdf_str) { - int seg, bus, dev, func, i; + int seg, bus, dev, func, slot, i; char *opt; - if ( !next_bdf(&bdf_str, &seg, &bus, &dev, &func, &opt)) + if ( !next_bdf(&bdf_str, &seg, &bus, &dev, &func, &opt, &slot)) { return -1; } @@ -3910,7 +3919,7 @@ int power_off_php_slot(int php_slot) int pt_init(PCIBus *e_bus, const char *direct_pci) { - int seg, b, d, f, status = -1; + int seg, b, d, f, s, status = -1; struct pt_dev *pt_dev; struct pci_access *pci_access; char *vslots; @@ -3946,7 +3955,7 @@ int pt_init(PCIBus *e_bus, const char *d vslots = qemu_mallocz ( strlen(direct_pci) / 3 ); /* Assign given devices to guest */ - while ( next_bdf(&direct_pci_p, &seg, &b, &d, &f, &opt) ) + while ( next_bdf(&direct_pci_p, &seg, &b, &d, &f, &opt, &s) ) { /* Register real device with the emulated bus */ pt_dev = register_real_device(e_bus, "DIRECT PCI", PT_VIRT_DEVFN_AUTO, -- -- Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
horms@vergenet.net
2009-Mar-25 03:32 UTC
[Xen-devel] [patch 3/4 v2] ioemu: allow xend to specify the slot for pass-through devices
Currently a slot may be specified for a hot-plug device, but not for a pass-through device that is inserted at boot time. This patch adds support for the latter. The syntax is: BUS:DEV.FUNC[@VSLOT] e.g: 0000:00:1d:0@7 This may be important as recent changes that allow any free PCI slot to be used for pass-through (and hotplug) may case pass-through devices to be assigned in different locations to before. Amongst other things, specifying the slot will allow users to move them back, if there is a need. There is also a xend portion of this patch, which will be posted separately. Signed-off-by: Simon Horman <horms@vereg.net.au> --- Some implementation details * Pass the slot parsed from xend to register_real_device(). If none was passed, next_bdf() will supply AUTO_PHP_SLOT. * Make use of the fact that __insert_to_pci_slot() is now indempotent and always call it in register_real_device() * Change the error message int he case where __insert_to_pci_slot() fails to not refer to hot-plut. e_devfn will always be non-auto in the hotplug case as register_real_device() is called by power_on_php_slot() which is called by acpi_php_add() which is called by acpi_php_add(). acpi_php_add() handles calling __insert_to_pci_slot() and acpi_php_add() handles the case where the call failed, before calling power_on_php_slot(). * Remove PT_VIRT_DEVFN_AUTO, it is no longer used. Index: ioemu-remote/hw/pass-through.c ==================================================================--- ioemu-remote.orig/hw/pass-through.c 2009-03-20 21:34:06.000000000 +1100 +++ ioemu-remote/hw/pass-through.c 2009-03-20 21:35:25.000000000 +1100 @@ -3560,7 +3560,7 @@ static int pt_pmcsr_reg_restore(struct p } struct pt_dev * register_real_device(PCIBus *e_bus, - const char *e_dev_name, int e_devfn, uint8_t r_bus, uint8_t r_dev, + const char *e_dev_name, int e_slot, uint8_t r_bus, uint8_t r_dev, uint8_t r_func, uint32_t machine_irq, struct pci_access *pci_access, char *opt) { @@ -3569,7 +3569,6 @@ struct pt_dev * register_real_device(PCI struct pci_dev *pci_dev; uint8_t e_device, e_intx; struct pci_config_cf8 machine_bdf; - int free_slot; char *key, *val; int msi_translate; @@ -3592,15 +3591,10 @@ struct pt_dev * register_real_device(PCI pci_fill_info(pci_dev, PCI_FILL_IRQ | PCI_FILL_BASES | PCI_FILL_ROM_BASE | PCI_FILL_SIZES); pt_libpci_fixup(pci_dev); - if ( e_devfn == PT_VIRT_DEVFN_AUTO ) { - /*indicate a static assignment(not hotplug), so find a free PCI hot plug slot */ - free_slot = __insert_to_pci_slot(r_bus, r_dev, r_func, - AUTO_PHP_SLOT, NULL); - if ( free_slot < 0 ) { - PT_LOG("Error: no free virtual PCI hot plug slot, thus no live migration.\n"); - return NULL; - } - e_devfn = free_slot << 3; + e_slot = __insert_to_pci_slot(r_bus, r_dev, r_func, e_slot, NULL); + if ( e_slot < 0 ) { + PT_LOG("Error: no free virtual PCI slot\n"); + return NULL; } msi_translate = direct_pci_msitranslate; @@ -3633,7 +3627,7 @@ struct pt_dev * register_real_device(PCI /* Register device */ assigned_device = (struct pt_dev *) pci_register_device(e_bus, e_dev_name, - sizeof(struct pt_dev), e_devfn, + sizeof(struct pt_dev), PCI_DEVFN(e_slot, 0), pt_pci_read_config, pt_pci_write_config); if ( assigned_device == NULL ) { @@ -3641,7 +3635,7 @@ struct pt_dev * register_real_device(PCI return NULL; } - dpci_infos.php_devs[PCI_SLOT(e_devfn)].pt_dev = assigned_device; + dpci_infos.php_devs[e_slot].pt_dev = assigned_device; assigned_device->pci_dev = pci_dev; assigned_device->msi_trans_cap = msi_translate; @@ -3802,7 +3796,7 @@ int power_on_php_slot(int slot) pt_dev register_real_device(dpci_infos.e_bus, "DIRECT PCI", - slot << 3, + slot, php_dev->r_bus, php_dev->r_dev, php_dev->r_func, @@ -3862,7 +3856,7 @@ int pt_init(PCIBus *e_bus, const char *d while ( next_bdf(&direct_pci_p, &seg, &b, &d, &f, &opt, &s) ) { /* Register real device with the emulated bus */ - pt_dev = register_real_device(e_bus, "DIRECT PCI", PT_VIRT_DEVFN_AUTO, + pt_dev = register_real_device(e_bus, "DIRECT PCI", s, b, d, f, PT_MACHINE_IRQ_AUTO, pci_access, opt); if ( pt_dev == NULL ) { Index: ioemu-remote/hw/pass-through.h ==================================================================--- ioemu-remote.orig/hw/pass-through.h 2009-03-20 21:24:13.000000000 +1100 +++ ioemu-remote/hw/pass-through.h 2009-03-20 21:35:25.000000000 +1100 @@ -39,7 +39,6 @@ // #define PT_DEBUG_PCI_CONFIG_ACCESS #define PT_MACHINE_IRQ_AUTO (0xFFFFFFFF) -#define PT_VIRT_DEVFN_AUTO (-1) /* Misc PCI constants that should be moved to a separate library :) */ #define PCI_CONFIG_SIZE (256) -- -- Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
horms@vergenet.net
2009-Mar-25 03:32 UTC
[Xen-devel] [patch 4/4 v2] ioemu: Read pass-through vslot from xend
This reads the vslot information supplied by xend, and should be the final piece for this feature on the ioemu side. There is also a xend portion of this patch which I will post separately. Signed-off-by: Simon Horman <horms@verge.net.au> Index: ioemu-remote/xenstore.c ==================================================================--- ioemu-remote.orig/xenstore.c 2009-03-24 09:25:36.000000000 +1100 +++ ioemu-remote/xenstore.c 2009-03-24 22:58:28.000000000 +1100 @@ -582,6 +582,24 @@ void xenstore_parse_domain_config(int hv strcat(direct_pci_str, dev); + if (pasprintf(&buf, "/local/domain/0/backend/pci/%u/%u/vslot-%d", + hvm_domid, pci_devid, i) != -1) { + free(dev); + dev = xs_read(xsh, XBT_NULL, buf, &len); + } + if ( dev ) { + if (strlen(dev) + strlen(direct_pci_str) > + DIRECT_PCI_STR_LEN - 2) { + fprintf(stderr, "qemu: too many pci pass-through " + "devices\n"); + memset(direct_pci_str, 0, DIRECT_PCI_STR_LEN); + goto out; + } + strcat(direct_pci_str, "@"); + strcat(direct_pci_str, dev); + } + + if (pasprintf(&buf, "/local/domain/0/backend/pci/%u/%u/opts-%d", hvm_domid, pci_devid, i) != -1) { free(dev); -- -- Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Cui, Dexuan
2009-Mar-25 09:54 UTC
RE: [Xen-devel] [patch 4/4 v2] ioemu: Read pass-through vslot from xend
horms@vergenet.net wrote:> This reads the vslot information supplied by xend, and should be > the final piece for this feature on the ioemu side. > > There is also a xend portion of this patch which I will post > separately. > > Signed-off-by: Simon Horman <horms@verge.net.au> > > > Index: ioemu-remote/xenstore.c > ==================================================================> --- ioemu-remote.orig/xenstore.c 2009-03-24 09:25:36.000000000 +1100 > +++ ioemu-remote/xenstore.c 2009-03-24 22:58:28.000000000 +1100 > @@ -582,6 +582,24 @@ void xenstore_parse_domain_config(int hv > > strcat(direct_pci_str, dev); > > + if (pasprintf(&buf, > "/local/domain/0/backend/pci/%u/%u/vslot-%d", + > hvm_domid, pci_devid, i) != -1) { + free(dev); > + dev = xs_read(xsh, XBT_NULL, buf, &len); > + } > + if ( dev ) { > + if (strlen(dev) + strlen(direct_pci_str) > > + DIRECT_PCI_STR_LEN - 2) { > + fprintf(stderr, "qemu: too many pci pass-through > " + "devices\n"); > + memset(direct_pci_str, 0, DIRECT_PCI_STR_LEN); > + goto out; > + } > + strcat(direct_pci_str, "@"); > + strcat(direct_pci_str, dev); > + } > + > + > if (pasprintf(&buf, > > "/local/domain/0/backend/pci/%u/%u/opts-%d", > hvm_domid, pci_devid, i) != -1) { free(dev); > > --Oh, I see the patches... :-) Thanks, -- Dexuan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2009-Mar-25 11:37 UTC
[Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf
horms@vergenet.net writes ("[patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf"):> Currently only hotplug provides vslot information from xend. > A subsequent patch will have xend provide this information > for boot-time inserted pass-through devices too.If I apply 2/4, 3/4 and 4/4 of your series, will it still work with old xends ? Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Simon Horman
2009-Mar-25 23:23 UTC
Re: [Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf
On Wed, Mar 25, 2009 at 11:37:36AM +0000, Ian Jackson wrote:> horms@vergenet.net writes ("[patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf"): > > Currently only hotplug provides vslot information from xend. > > A subsequent patch will have xend provide this information > > for boot-time inserted pass-through devices too. > > If I apply 2/4, 3/4 and 4/4 of your series, will it still work with > old xends ?Hi Ian, It depends what you mean by an old xend (do you have a particular version in mind) and I will need to re-check, but I don''t think that those changes break compatibility. -- Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Simon Horman
2009-Mar-26 00:18 UTC
Re: [Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf
On Thu, Mar 26, 2009 at 10:23:52AM +1100, Simon Horman wrote:> On Wed, Mar 25, 2009 at 11:37:36AM +0000, Ian Jackson wrote: > > horms@vergenet.net writes ("[patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf"): > > > Currently only hotplug provides vslot information from xend. > > > A subsequent patch will have xend provide this information > > > for boot-time inserted pass-through devices too. > > > > If I apply 2/4, 3/4 and 4/4 of your series, will it still work with > > old xends ? > > Hi Ian, > > It depends what you mean by an old xend (do you have a particular > version in mind) and I will need to re-check, but I don''t think > that those changes break compatibility.Hi, I have checked against the following trees, and there do not seem to be any compatibiliy issues introduced by any of these patches. qemu-xen-unstable 545882583159e9cc53cde5965277967092459f37 cirrus: Do not clear vram area to 0xff when not yet allocated. xen-unstable 19426:0b13d9787622 libxc: fix link error of xc_save on ia64 During testing I noted that 2/4 needs to be applied before 3/4, but there don''t seem to be any other ordering constraints. More speficically, I tested the following with static and hot-plug pass-through and found them to work. Patches applied - none - 2/4 - 4/4 - 2/4 3/4 - 2/4 4/4 - 2/4 3/4 4/4 -- Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2009-Mar-31 10:43 UTC
Re: [Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf
Simon Horman writes ("Re: [Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf"):> It depends what you mean by an old xend (do you have a particular > version in mind) and I will need to re-check, but I don''t think > that those changes break compatibility.I just meant that since you were submitting corresponding patches to xen-unstable and qemu-xen-unstable, I was worried that you might introduce a cross-version incompatibility. Reviewing the parsing code by eye didn''t reassure me so I thought I''d ask. I''m trying to be better about keeping version-skewed versions of the trees compatible, as not doing so makes many things much more difficult. Anyway, thanks for the reassurance in this and your next message. Regards, Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Simon Horman
2009-Mar-31 10:51 UTC
Re: [Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf
On Tue, Mar 31, 2009 at 11:43:46AM +0100, Ian Jackson wrote:> Simon Horman writes ("Re: [Xen-devel] Re: [patch 2/4 v2] ioemu: Do slot parsing inside of next_bdf"): > > It depends what you mean by an old xend (do you have a particular > > version in mind) and I will need to re-check, but I don''t think > > that those changes break compatibility. > > I just meant that since you were submitting corresponding patches to > xen-unstable and qemu-xen-unstable, I was worried that you might > introduce a cross-version incompatibility. Reviewing the parsing code > by eye didn''t reassure me so I thought I''d ask. > > I''m trying to be better about keeping version-skewed versions of the > trees compatible, as not doing so makes many things much more > difficult. Anyway, thanks for the reassurance in this and your next > message.Thanks for the clarification. I don''t believe that any of the patches that I have posted but haven''t been applied break compatibility between xen-unstable and qemu-xen-unstable. In future I will try and be better about noting when any incompatibilities are introduced. -- Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel