Hey, I was wondering if the ACPI PCI hotplug (so inserting a new PCIe card in a server that supports said functionality) is something that Intel has been testing or using?
Are you playing with Xen ? so far, Xen doesn''t support PCIe device hot-plug feature yet. But if you want to hot-plug a PCIe device to a Xen guest, it should supported well. Xiantao> -----Original Message----- > From: Konrad Rzeszutek [mailto:ketuzsezr@gmail.com] On Behalf Of Konrad > Rzeszutek Wilk > Sent: Wednesday, December 19, 2012 6:46 AM > To: Zhang, Xiantao; xen-devel@lists.xensource.com > Subject: Xen 4.2 and PCI hotplug. > > Hey, > > I was wondering if the ACPI PCI hotplug (so inserting a new PCIe card in a > server that supports said functionality) is something that Intel has been > testing or using?
>>> On 19.12.12 at 09:13, "Zhang, Xiantao" <xiantao.zhang@intel.com> wrote: > Are you playing with Xen ? so far, Xen doesn''t support PCIe device hot-plug > feature yet.When saying Xen, I assume you mean the pv-ops kernel instead? So far I was under the impression that this worked even with the very old 2.6.18 tree (as much or as little as hotplug there worked in the native case). And given that there are no special requirements on the hypervisor to make this work, it''s not even obvious to me what would be missing in the pv-ops kernel to make it work. All that is of course with me not having any practical experience with hotplug, due to the lack of capable hardware... Jan
> -----Original Message----- > From: Jan Beulich [mailto:JBeulich@suse.com] > Sent: Wednesday, December 19, 2012 4:48 PM > To: Zhang, Xiantao > Cc: Konrad Rzeszutek Wilk; xen-devel > Subject: Re: [Xen-devel] Xen 4.2 and PCI hotplug. > > >>> On 19.12.12 at 09:13, "Zhang, Xiantao" <xiantao.zhang@intel.com> > wrote: > > Are you playing with Xen ? so far, Xen doesn''t support PCIe device > > hot-plug feature yet. > > When saying Xen, I assume you mean the pv-ops kernel instead? So far I was > under the impression that this worked even with the very old 2.6.18 tree (as > much or as little as hotplug there worked in the native case). And given that > there are no special requirements on the hypervisor to make this work, it''s > not even obvious to me what would be missing in the pv-ops kernel to make > it work.Oh, my fault! Perhaps we don''t need to do anything for pv-ops kernel to support device hot-plug if native system has it supported. Actually, we didn''t do such testings before, since it is a native feature, not a Xen-specific one. Xiantao
On 19/12/2012 09:14, Zhang, Xiantao wrote:>> -----Original Message----- >> From: Jan Beulich [mailto:JBeulich@suse.com] >> Sent: Wednesday, December 19, 2012 4:48 PM >> To: Zhang, Xiantao >> Cc: Konrad Rzeszutek Wilk; xen-devel >> Subject: Re: [Xen-devel] Xen 4.2 and PCI hotplug. >> >>>>> On 19.12.12 at 09:13, "Zhang, Xiantao" <xiantao.zhang@intel.com> >> wrote: >>> Are you playing with Xen ? so far, Xen doesn''t support PCIe device >>> hot-plug feature yet. >> When saying Xen, I assume you mean the pv-ops kernel instead? So far I was >> under the impression that this worked even with the very old 2.6.18 tree (as >> much or as little as hotplug there worked in the native case). And given that >> there are no special requirements on the hypervisor to make this work, it''s >> not even obvious to me what would be missing in the pv-ops kernel to make >> it work. > Oh, my fault! Perhaps we don''t need to do anything for pv-ops kernel to support device hot-plug if native system has it supported. Actually, we didn''t do such testings before, since it is a native feature, not a Xen-specific one. > XiantaoMy current understanding is that on boot, Xen scans the PCI bus, then dom0 rescans it later. If a hotplug event gets serviced by dom0, does there not need to be some hypercall informing Xen that a new device has appeared? I expect PCIPassthrough would not work correctly on a hotplugged device which Xen is unaware of. (But if I have got the wrong end of the stick, or this mechanism already exist, please ignore me. It just strikes me as a little xen-specific, even if the bulk of it is native) ~Andrew> > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
>>> On 19.12.12 at 15:36, Andrew Cooper <andrew.cooper3@citrix.com> wrote: > On 19/12/2012 09:14, Zhang, Xiantao wrote: >>> -----Original Message----- >>> From: Jan Beulich [mailto:JBeulich@suse.com] >>> Sent: Wednesday, December 19, 2012 4:48 PM >>> To: Zhang, Xiantao >>> Cc: Konrad Rzeszutek Wilk; xen-devel >>> Subject: Re: [Xen-devel] Xen 4.2 and PCI hotplug. >>> >>>>>> On 19.12.12 at 09:13, "Zhang, Xiantao" <xiantao.zhang@intel.com> >>> wrote: >>>> Are you playing with Xen ? so far, Xen doesn''t support PCIe device >>>> hot-plug feature yet. >>> When saying Xen, I assume you mean the pv-ops kernel instead? So far I was >>> under the impression that this worked even with the very old 2.6.18 tree (as >>> much or as little as hotplug there worked in the native case). And given > that >>> there are no special requirements on the hypervisor to make this work, it''s >>> not even obvious to me what would be missing in the pv-ops kernel to make >>> it work. >> Oh, my fault! Perhaps we don''t need to do anything for pv-ops kernel to > support device hot-plug if native system has it supported. Actually, we > didn''t do such testings before, since it is a native feature, not a > Xen-specific one. >> Xiantao > > My current understanding is that on boot, Xen scans the PCI bus, then > dom0 rescans it later. If a hotplug event gets serviced by dom0, does > there not need to be some hypercall informing Xen that a new device has > appeared? I expect PCIPassthrough would not work correctly on a > hotplugged device which Xen is unaware of.Sure - such a hypercall exists and is - from all I can tell - being made not only during the boot time bus scan, but also during hotplug processing. See drivers/xen/pci.c. Jan
On Wed, Dec 19, 2012 at 08:47:58AM +0000, Jan Beulich wrote:> >>> On 19.12.12 at 09:13, "Zhang, Xiantao" <xiantao.zhang@intel.com> wrote: > > Are you playing with Xen ? so far, Xen doesn''t support PCIe device hot-plug > > feature yet. > > When saying Xen, I assume you mean the pv-ops kernel instead? SoI am looking at the PCI AER which can do native hotplug. And from the Linux side it ought to work right -- but there is some scanning of said new PCI device "before" the notifications are sent (which we latch on and do the hypercall on). At least that is from a cursory read of the code. I presume that "before" the hypercall is sent Xen would just ignore the 0xcf8 reads. But after the hypercall it would populate appropiately whatever it needs. But it would never get to that point b/c Linux couldn''t read the device. I might be of course completly off-base - hence asking if Intel has done any sort of testing on this?> far I was under the impression that this worked even with the very > old 2.6.18 tree (as much or as little as hotplug there worked in the > native case). And given that there are no special requirements on > the hypervisor to make this work, it''s not even obvious to me what > would be missing in the pv-ops kernel to make it work. > > All that is of course with me not having any practical experience > with hotplug, due to the lack of capable hardware... > > Jan > >
On Wed, Dec 19, 2012 at 03:08:31PM +0000, Jan Beulich wrote:> >>> On 19.12.12 at 15:36, Andrew Cooper <andrew.cooper3@citrix.com> wrote: > > On 19/12/2012 09:14, Zhang, Xiantao wrote: > >>> -----Original Message----- > >>> From: Jan Beulich [mailto:JBeulich@suse.com] > >>> Sent: Wednesday, December 19, 2012 4:48 PM > >>> To: Zhang, Xiantao > >>> Cc: Konrad Rzeszutek Wilk; xen-devel > >>> Subject: Re: [Xen-devel] Xen 4.2 and PCI hotplug. > >>> > >>>>>> On 19.12.12 at 09:13, "Zhang, Xiantao" <xiantao.zhang@intel.com> > >>> wrote: > >>>> Are you playing with Xen ? so far, Xen doesn''t support PCIe device > >>>> hot-plug feature yet. > >>> When saying Xen, I assume you mean the pv-ops kernel instead? So far I was > >>> under the impression that this worked even with the very old 2.6.18 tree (as > >>> much or as little as hotplug there worked in the native case). And given > > that > >>> there are no special requirements on the hypervisor to make this work, it''s > >>> not even obvious to me what would be missing in the pv-ops kernel to make > >>> it work. > >> Oh, my fault! Perhaps we don''t need to do anything for pv-ops kernel to > > support device hot-plug if native system has it supported. Actually, we > > didn''t do such testings before, since it is a native feature, not a > > Xen-specific one. > >> Xiantao > > > > My current understanding is that on boot, Xen scans the PCI bus, then > > dom0 rescans it later. If a hotplug event gets serviced by dom0, does > > there not need to be some hypercall informing Xen that a new device has > > appeared? I expect PCIPassthrough would not work correctly on a > > hotplugged device which Xen is unaware of. > > Sure - such a hypercall exists and is - from all I can tell - being made > not only during the boot time bus scan, but also during hotplug > processing. See drivers/xen/pci.c.Right. What I am not sure about - and this is why I am asking Intel - whether that notifier gets called _after_ dom0 has interogated the new PCI device. And if so, would Xen (which gets trapped on the 0xcf8) would not let dom0 actually interogate the new BDF b/c it hasn''t been told about this BDF existence. This is all speculation - and I might be very well completly wrong about all of this - hence this email-thread. Testing would of course tell for sure, but I am not sure if I even have such hardware.> > Jan > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >