Xen devel - Nov 2008 - one question to PHYSDEVOP_manage_pci_remove

Hi, Espen, when I''m working on the MSI lock issue, I have one question
to PHYSDEVOP_manage_pci_remove. From the kernel, seems it will happen for PCI
device remove. Is it target only for pci hotplug?
I asked this because I noticed that in current code, it will always clear the
VT-d entry, no matter if the device is owned by other domain, I''m not
sure if it reqiures the device is not owned by a guest (maybe except dom0).

Thanks
Yunhong Jiang
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

The idea was to use PHYSDEVOP_manage_pci_{add,remove} for
hot-pluggable devices and SR-IOV or ARI capable devices.  In general,
when a new PCI device function is added or removed during runtime,
dom0 will invoke the hypercall to register or unrigister the
corresponding struct pci_dev within Xen.

Xen clears the VT-d entry because the device is about to be removed
from the system --- either physically removed or just completely
disabled by dom0.  The device is forcibly removed from the guest if it
is assigned to one.  It is up to dom0 to do the actual device
reassignment.  Although the device can not access any host memory I
suppose it would not hurt to also do deassign_device() when the device
is removed.  This would tear down the IOMMU page tables for the domain
if they are not needed anymore.

	eSk


[Yunhong Jiang]
> Hi, Espen, when I''m working on the MSI lock issue, I have one
> question to PHYSDEVOP_manage_pci_remove. From the kernel, seems it
> will happen for PCI device remove. Is it target only for pci
> hotplug?
> I asked this because I noticed that in current code, it will always
> clear the VT-d entry, no matter if the device is owned by other
> domain, I''m not sure if it reqiures the device is not owned by a
> guest (maybe except dom0).

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Espen Skoglund <mailto:espen.skoglund@netronome.com>
wrote:
> The idea was to use PHYSDEVOP_manage_pci_{add,remove} for
> hot-pluggable devices and SR-IOV or ARI capable devices.  In general,
> when a new PCI device function is added or removed during runtime,
> dom0 will invoke the hypercall to register or unrigister the
> corresponding struct pci_dev within Xen.
> 
> Xen clears the VT-d entry because the device is about to be removed
> from the system --- either physically removed or just completely
> disabled by dom0.  The device is forcibly removed from the guest if it
> is assigned to one.  It is up to dom0 to do the actual device
> reassignment.  Although the device can not access any host memory I
> suppose it would not hurt to also do deassign_device() when the device
> is removed.  This would tear down the IOMMU page tables for the domain
> if they are not needed anymore.
> 
> 	eSk
Thanks for your information, so I think at least a log is needed if the device
is still assigned to a guest.
Also, have you checked if dom0 have chance to "do the actual device
reassignment"? I suppose user space tools should be involeved in such
process.

Thanks
Yunhong Jiang
> 
> 
> [Yunhong Jiang]
>> Hi, Espen, when I''m working on the MSI lock issue, I have one
>> question to PHYSDEVOP_manage_pci_remove. From the kernel, seems it
>> will happen for PCI device remove. Is it target only for pci
>> hotplug?
> 
>> I asked this because I noticed that in current code, it will always
>> clear the VT-d entry, no matter if the device is owned by other
>> domain, I''m not sure if it reqiures the device is not owned by
a
>> guest (maybe except dom0).
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

[Yunhong Jiang]
> Espen Skoglund <mailto:espen.skoglund@netronome.com> wrote:
>> The idea was to use PHYSDEVOP_manage_pci_{add,remove} for
>> hot-pluggable devices and SR-IOV or ARI capable devices.  In general,
>> when a new PCI device function is added or removed during runtime,
>> dom0 will invoke the hypercall to register or unrigister the
>> corresponding struct pci_dev within Xen.
>> 
>> Xen clears the VT-d entry because the device is about to be removed
>> from the system --- either physically removed or just completely
>> disabled by dom0.  The device is forcibly removed from the guest if it
>> is assigned to one.  It is up to dom0 to do the actual device
>> reassignment.  Although the device can not access any host memory I
>> suppose it would not hurt to also do deassign_device() when the device
>> is removed.  This would tear down the IOMMU page tables for the domain
>> if they are not needed anymore.
>> 
>> 	eSk
> Thanks for your information, so I think at least a log is needed if
> the device is still assigned to a guest.  Also, have you checked if
> dom0 have chance to "do the actual device reassignment"? I
suppose
> user space tools should be involeved in such process.
Yes, user-level tools would typically be involved in the process, so
there''s not much more the hypervisor itself can do.  If dom0 (or some
other controlling entity) can not reassign the device then you''re
pretty much screwed anyway, and the best you can do is to disable the
device from accessing host memory (which is what currently happens).

	eSk



> Thanks
> Yunhong Jiang
>> 
>> 
>> [Yunhong Jiang]
>>> Hi, Espen, when I''m working on the MSI lock issue, I have
one
>>> question to PHYSDEVOP_manage_pci_remove. From the kernel, seems it
>>> will happen for PCI device remove. Is it target only for pci
>>> hotplug?
>> 
>>> I asked this because I noticed that in current code, it will always
>>> clear the VT-d entry, no matter if the device is owned by other
>>> domain, I''m not sure if it reqiures the device is not
owned by a
>>> guest (maybe except dom0).
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel