Xen devel - Jan 2008 - [VTD][PATCH] Do FLR of assigned devices with VT-d

Currently there is a pdev_flr() function to do FLR before device
assignment in qemu, but most of devices don''t have FLR capability.
What''s more, should do FLR before assignment and deassignment for
keeping correct device status. If the device doesn''t have FLR
capablility, this patch implemented to enter D3hot and return to D0 to
do FLR. And exposed pdev_flr() in VT-d utils, then it can be invoked by
assignment and deassignment functions. 

Signed-off-by: Weidong Han <weidong.han@intel.com>
Signed-off-by: Anthony Xu <anthony.xu@intel.com>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

That''s a good addition, why moving this functionality to the
hypervisor?
BTW, can you give real world example where this was needed?

Thanks,
Guy. 
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com 
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of 
> Han, Weidong
> Sent: Thursday, January 24, 2008 4:39 AM
> To: xen-devel@lists.xensource.com
> Cc: Xu, Anthony; Kay,Allen M
> Subject: [Xen-devel] [VTD][PATCH] Do FLR of assigned devices with VT-d
> 
> Currently there is a pdev_flr() function to do FLR before 
> device assignment in qemu, but most of devices don''t have FLR 
> capability.
> What''s more, should do FLR before assignment and deassignment 
> for keeping correct device status. If the device doesn''t have 
> FLR capablility, this patch implemented to enter D3hot and 
> return to D0 to do FLR. And exposed pdev_flr() in VT-d utils, 
> then it can be invoked by assignment and deassignment functions. 
> 
> Signed-off-by: Weidong Han <weidong.han@intel.com>
> Signed-off-by: Anthony Xu <anthony.xu@intel.com>
> 
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>That''s a good addition, why moving this functionality to the 
>hypervisor?
Qemu code is not guaranteed to execute, like a Qemu crash.
>BTW, can you give real world example where this was needed?
On the above case, we may leave the device in an uncertain status, and
it may still generate interrupts.
-Xin
>
>> -----Original Message-----
>> From: xen-devel-bounces@lists.xensource.com 
>> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of 
>> Han, Weidong
>> Sent: Thursday, January 24, 2008 4:39 AM
>> To: xen-devel@lists.xensource.com
>> Cc: Xu, Anthony; Kay,Allen M
>> Subject: [Xen-devel] [VTD][PATCH] Do FLR of assigned devices 
>with VT-d
>> 
>> Currently there is a pdev_flr() function to do FLR before 
>> device assignment in qemu, but most of devices don''t have FLR 
>> capability.
>> What''s more, should do FLR before assignment and deassignment 
>> for keeping correct device status. If the device doesn''t have 
>> FLR capablility, this patch implemented to enter D3hot and 
>> return to D0 to do FLR. And exposed pdev_flr() in VT-d utils, 
>> then it can be invoked by assignment and deassignment functions. 
>> 
>> Signed-off-by: Weidong Han <weidong.han@intel.com>
>> Signed-off-by: Anthony Xu <anthony.xu@intel.com>
>> 
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Guy,

If you use "xm destroy" or click "x" on qemu window to
destroy guest,
the assigned device may be stayed in a incorrect status. We met that the
irq of the assigned device may be disabled by dom0 after destroy guest,
then all the devices using the same irq can''t work any more. We also
met
guest complained the device was not in a correct status during its
booting period.

We think there are three places where need invoking FLR function. The
first place is before device assignment, the second one is after
exit/destroy guest, the last one is dynamic reassignment after support
vtd hotplug. In the latter two cases, it is hard to catch the point to
do FLR in qemu.

Randy (Weidong)

Guy Zana wrote:
> That''s a good addition, why moving this functionality to the
> hypervisor? BTW, can you give real world example where this was
> needed? 
> 
> Thanks,
> Guy.
> 
>> -----Original Message-----
>> From: xen-devel-bounces@lists.xensource.com
>> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of
>> Han, Weidong
>> Sent: Thursday, January 24, 2008 4:39 AM
>> To: xen-devel@lists.xensource.com
>> Cc: Xu, Anthony; Kay,Allen M
>> Subject: [Xen-devel] [VTD][PATCH] Do FLR of assigned devices with
>> VT-d 
>> 
>> Currently there is a pdev_flr() function to do FLR before
>> device assignment in qemu, but most of devices don''t have FLR
>> capability. What''s more, should do FLR before assignment and
>> deassignment 
>> for keeping correct device status. If the device doesn''t have
>> FLR capablility, this patch implemented to enter D3hot and
>> return to D0 to do FLR. And exposed pdev_flr() in VT-d utils,
>> then it can be invoked by assignment and deassignment functions.
>> 
>> Signed-off-by: Weidong Han <weidong.han@intel.com>
>> Signed-off-by: Anthony Xu <anthony.xu@intel.com
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> -----Original Message-----
> From: Li, Xin B [mailto:xin.b.li@intel.com] 
> Sent: Saturday, January 26, 2008 4:00 AM
> To: Guy Zana; Han, Weidong; xen-devel@lists.xensource.com
> Cc: Xu, Anthony; Kay, Allen M
> Subject: RE: [Xen-devel] [VTD][PATCH] Do FLR of assigned 
> devices with VT-d
> 
> >That''s a good addition, why moving this functionality to the 
> >hypervisor?
> 
> Qemu code is not guaranteed to execute, like a Qemu crash.
> 
It is guaranteed to execute, if QEMU will crash the domain will die, and in that
case there is no effect in doing FLR reset.
Can you leave the FLR reset in qemu please? so it won''t be VT-d
specific.

Thanks,
Guy.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi,

Oh, now I see why you did that. I''m eager to test the PCI hotplug stuff
:-)

The only thing that is concerning me is that it becomes VT-d specific.
And I still think that the hypervisor is a bad place to put that code...

Thanks,
Guy.
> -----Original Message-----
> From: Han, Weidong [mailto:weidong.han@intel.com] 
> Sent: Sunday, January 27, 2008 2:38 PM
> To: Guy Zana; xen-devel@lists.xensource.com
> Cc: Xu, Anthony; Kay, Allen M
> Subject: RE: [Xen-devel] [VTD][PATCH] Do FLR of assigned 
> devices with VT-d
> 
> Hi Guy,
> 
> If you use "xm destroy" or click "x" on qemu window to 
> destroy guest, the assigned device may be stayed in a 
> incorrect status. We met that the irq of the assigned device 
> may be disabled by dom0 after destroy guest, then all the 
> devices using the same irq can''t work any more. We also met 
> guest complained the device was not in a correct status 
> during its booting period.
> 
> We think there are three places where need invoking FLR 
> function. The first place is before device assignment, the 
> second one is after exit/destroy guest, the last one is 
> dynamic reassignment after support vtd hotplug. In the latter 
> two cases, it is hard to catch the point to do FLR in qemu.
> 
> Randy (Weidong)
> 
> Guy Zana wrote:
> > That''s a good addition, why moving this functionality to the 
> > hypervisor? BTW, can you give real world example where this was 
> > needed?
> > 
> > Thanks,
> > Guy.
> > 
> >> -----Original Message-----
> >> From: xen-devel-bounces@lists.xensource.com
> >> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Han, 
> >> Weidong
> >> Sent: Thursday, January 24, 2008 4:39 AM
> >> To: xen-devel@lists.xensource.com
> >> Cc: Xu, Anthony; Kay,Allen M
> >> Subject: [Xen-devel] [VTD][PATCH] Do FLR of assigned devices with 
> >> VT-d
> >> 
> >> Currently there is a pdev_flr() function to do FLR before device 
> >> assignment in qemu, but most of devices don''t have FLR
capability.
> >> What''s more, should do FLR before assignment and
deassignment for
> >> keeping correct device status. If the device doesn''t have
FLR
> >> capablility, this patch implemented to enter D3hot and 
> return to D0 
> >> to do FLR. And exposed pdev_flr() in VT-d utils, then it can be 
> >> invoked by assignment and deassignment functions.
> >> 
> >> Signed-off-by: Weidong Han <weidong.han@intel.com>
> >> Signed-off-by: Anthony Xu <anthony.xu@intel.com
> 
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 29/1/08 18:39, "Guy Zana" <guy@neocleus.com> wrote:
> Oh, now I see why you did that. I''m eager to test the PCI hotplug
stuff :-)
> 
> The only thing that is concerning me is that it becomes VT-d specific.
> And I still think that the hypervisor is a bad place to put that code...
I don''t see any particular reason not to have it in the hypervisor. The
only
other possibly sensible place would be dom0 kernel, probably in pciback. It
would actually be fairly sensible to have pciback as the place for PCI setup
and control for passed-thru devices, for both PV and HVM guests.

Just because it''s in Xen doesn''t mean it has to be vt-d
specific either.
There needs to be code sharing between vt-d, pacifica and the 1:1 nativedom
approaches.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel