Xen devel - Jun 2012 - [PATCH] amd iommu: re-enable iommu msi if dom0 disabled it

Hi Jan,
I found that recent dom0 (e.g 3.4 pv_ops) disables iommu msi capability 
for some reasons and iommu cannot generate any interrupts in this case.
Attached patch re-enables it in device assignment.

Thanks,
Wei

--
Advanced Micro Devices GmbH
Sitz: Dornach, Gemeinde Aschheim,
Landkreis München Registergericht München,
HRB Nr. 43632 WEEE Registrierungsnummer 129 19551
Geschäftsführer:
Alberto Bozzo


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On 08/06/12 14:28, Wei Wang wrote:
> Hi Jan,
> I found that recent dom0 (e.g 3.4 pv_ops) disables iommu msi capability 
> for some reasons and iommu cannot generate any interrupts in this case.
> Attached patch re-enables it in device assignment.
Under which circumstances should dom0 able to play with the IOMMUs ? 
Surely the fact that dom0 can play with the IOMMUs is a bug in itself.
>
> Thanks,
> Wei
>
> --
> Advanced Micro Devices GmbH
> Sitz: Dornach, Gemeinde Aschheim,
> Landkreis München Registergericht München,
> HRB Nr. 43632 WEEE Registrierungsnummer 129 19551
> Geschäftsführer:
> Alberto Bozzo
-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com

On 06/08/2012 03:31 PM, Andrew Cooper wrote:
> On 08/06/12 14:28, Wei Wang wrote:
>> Hi Jan,
>> I found that recent dom0 (e.g 3.4 pv_ops) disables iommu msi capability
>> for some reasons and iommu cannot generate any interrupts in this case.
>> Attached patch re-enables it in device assignment.
>
> Under which circumstances should dom0 able to play with the IOMMUs ?
> Surely the fact that dom0 can play with the IOMMUs is a bug in itself.
It looks like some generic msi/pci codes disable it, not the Linux iommu 
driver itself, which is only loaded on bare metal. AMD IOMMU expose 
interrupt capability as a normal msi block. So the general pci/msi layer 
of dom0 might touch it...

Thanks,
Wei

>>
>> Thanks,
>> Wei
>>
>> --
>> Advanced Micro Devices GmbH
>> Sitz: Dornach, Gemeinde Aschheim,
>> Landkreis München Registergericht München,
>> HRB Nr. 43632 WEEE Registrierungsnummer 129 19551
>> Geschäftsführer:
>> Alberto Bozzo
>

>>> On 08.06.12 at 15:28, Wei Wang <wei.wang2@amd.com> wrote:
> I found that recent dom0 (e.g 3.4 pv_ops) disables iommu msi capability 
> for some reasons and iommu cannot generate any interrupts in this case.
That shouldn''t happen in the first place, and hence should be fixed
in the kernel.
> Attached patch re-enables it in device assignment.
I''m not really keen on taking this - it''s silently working
around a
problem, and doesn''t appear to cover the case where Dom0
would have its I/O got through the IOMMU as well.

If anything, I''d be much more in favor of hiding the entire config
space of the IOMMU device from Dom0 (or at least making it
read-only), which admittedly might be a little tricky. So first of
all let''s understand whether this can''t reasonably be fixed in
Linux.

Jan

>>> On 08.06.12 at 15:41, Wei Wang <wei.wang2@amd.com> wrote:
> On 06/08/2012 03:31 PM, Andrew Cooper wrote:
>> On 08/06/12 14:28, Wei Wang wrote:
>>> I found that recent dom0 (e.g 3.4 pv_ops) disables iommu msi
capability
>>> for some reasons and iommu cannot generate any interrupts in this
case.
>>> Attached patch re-enables it in device assignment.
>>
>> Under which circumstances should dom0 able to play with the IOMMUs ?
>> Surely the fact that dom0 can play with the IOMMUs is a bug in itself.
> 
> It looks like some generic msi/pci codes disable it, not the Linux iommu 
> driver itself, which is only loaded on bare metal. AMD IOMMU expose 
> interrupt capability as a normal msi block. So the general pci/msi layer 
> of dom0 might touch it...
In that case it wouldn''t be on pv-ops alone (which your patch
says).

Also, your patch using IOMMU_CONTROL_ENABLED instead of
PCI_MSI_FLAGS_ENABLE is quite misleading (as it hides the fact
the what you play with is not IOMMU-specific).

Jan

On 06/08/2012 04:08 PM, Jan Beulich wrote:
>>>> On 08.06.12 at 15:41, Wei Wang<wei.wang2@amd.com>  wrote:
>> On 06/08/2012 03:31 PM, Andrew Cooper wrote:
>>> On 08/06/12 14:28, Wei Wang wrote:
>>>> I found that recent dom0 (e.g 3.4 pv_ops) disables iommu msi
capability
>>>> for some reasons and iommu cannot generate any interrupts in
this case.
>>>> Attached patch re-enables it in device assignment.
>>>
>>> Under which circumstances should dom0 able to play with the IOMMUs
?
>>> Surely the fact that dom0 can play with the IOMMUs is a bug in
itself.
>>
>> It looks like some generic msi/pci codes disable it, not the Linux
iommu
>> driver itself, which is only loaded on bare metal. AMD IOMMU expose
>> interrupt capability as a normal msi block. So the general pci/msi
layer
>> of dom0 might touch it...
>
> In that case it wouldn''t be on pv-ops alone (which your patch
> says).
Yes, probably upstream Linux also has this issue. I will check that. But 
I did not see this issue on 3.2 pv_ops & 2.6 pv_ops.
> Also, your patch using IOMMU_CONTROL_ENABLED instead of
> PCI_MSI_FLAGS_ENABLE is quite misleading (as it hides the fact
> the what you play with is not IOMMU-specific).
Thanks, I will use PCI_MSI_FLAGS_ENABLE in next post.

Wei
> Jan
>
>