Sander Eikelenboom
2010-Aug-10 12:26 UTC
[Xen-devel] AMD iommu , pci passthrough problems assigning devices to HVM
Hi, I''m running a machine with: - Hypervisor: xen-unstable - Dom0: xen/next-2.6.32.17 commit c3f5aba942c7516a5c76c47345ff0a473daf2542 Hardware: -MSI 890FXA-GD70 with AMD IOMMU -Phenom X6 I''m trying to do PCI passthrough to a HVM, this fails with: Error: Failed to assign device to IOMMU (0000:04:00.0@100,msitranslate=1,power_mgmt=0) Error: pci: to avoid potential security issue, 0000:08:00.0 is not allowed to be assigned to guest since it is behind PCIe switch that does not support or enable ACS. I have found this thread https://bugzilla.redhat.com/show_bug.cgi?id=526713 regarding the ACS problem, which says the pci bridges need these capabilities: ScrValid+, ReqRedir+, CmpltRedir+, and UpstreamFwd+ When i do a lspci -vvv on my system i see these capabilities, but i do see some disturbing debug message on boot when amd_iommu_debug is on. Attached: - xm info - xm dmesg (booted with amd_iommu_debug on) - dmesg - lspci -vvv -- Sander _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Rose, Gregory V
2010-Aug-10 20:51 UTC
RE: [Xen-devel] AMD iommu , pci passthrough problems assigning devices to HVM
In your device 0b.0 I see this: ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+ ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans- So it looks to me as if the device is ACS capable but that the capability is not turned on? I think every device in the path has to be ACS capable and the capability enabled. - Greg Rose LAD Intel Corp.>-----Original Message----- >From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel- >bounces@lists.xensource.com] On Behalf Of Sander Eikelenboom >Sent: Tuesday, August 10, 2010 5:26 AM >To: Huang2, Wei >Cc: Wang2, Wei; Przywara, Andre; Joerg Roedel; xen- >devel@lists.xensource.com >Subject: [Xen-devel] AMD iommu , pci passthrough problems assigning >devices to HVM > >Hi, > >I''m running a machine with: >- Hypervisor: xen-unstable >- Dom0: xen/next-2.6.32.17 commit >c3f5aba942c7516a5c76c47345ff0a473daf2542 > >Hardware: >-MSI 890FXA-GD70 with AMD IOMMU >-Phenom X6 > >I''m trying to do PCI passthrough to a HVM, this fails with: > >Error: Failed to assign device to IOMMU >(0000:04:00.0@100,msitranslate=1,power_mgmt=0) >Error: pci: to avoid potential security issue, 0000:08:00.0 is not >allowed to be assigned to guest since it is behind PCIe switch that does >not support or enable ACS. > > >I have found this thread >https://bugzilla.redhat.com/show_bug.cgi?id=526713 regarding the ACS >problem, which says the pci bridges need these capabilities: > ScrValid+, ReqRedir+, CmpltRedir+, and UpstreamFwd+ > >When i do a lspci -vvv on my system i see these capabilities, but i do >see some disturbing debug message on boot when amd_iommu_debug is on. > >Attached: >- xm info >- xm dmesg (booted with amd_iommu_debug on) >- dmesg >- lspci -vvv > >-- >Sander_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Wei Wang2
2010-Aug-11 10:11 UTC
[Xen-devel] Re: AMD iommu , pci passthrough problems assigning devices to HVM
Hi, In your debug output I saw: (XEN) AMD-Vi: Fail to find iommu for device04:00.0 This means no iommu attached to 04:00.0, so it cannot be assigned to any hvm guest. What is this device in your system? An add-on device or on board device? It looks like BIOS failed to create an IVRS entry for this device. Wei On Tuesday 10 August 2010 14:26:14 Sander Eikelenboom wrote:> Hi, > > I''m running a machine with: > - Hypervisor: xen-unstable > - Dom0: xen/next-2.6.32.17 commit c3f5aba942c7516a5c76c47345ff0a473daf2542 > > Hardware: > -MSI 890FXA-GD70 with AMD IOMMU > -Phenom X6 > > I''m trying to do PCI passthrough to a HVM, this fails with: > > Error: Failed to assign device to IOMMU > (0000:04:00.0@100,msitranslate=1,power_mgmt=0) Error: pci: to avoid > potential security issue, 0000:08:00.0 is not allowed to be assigned to > guest since it is behind PCIe switch that does not support or enable ACS. > > > I have found this thread https://bugzilla.redhat.com/show_bug.cgi?id=526713 > regarding the ACS problem, which says the pci bridges need these > capabilities: ScrValid+, ReqRedir+, CmpltRedir+, and UpstreamFwd+ > > When i do a lspci -vvv on my system i see these capabilities, but i do see > some disturbing debug message on boot when amd_iommu_debug is on. > > Attached: > - xm info > - xm dmesg (booted with amd_iommu_debug on) > - dmesg > - lspci -vvv > > -- > Sander_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Sander Eikelenboom
2010-Aug-11 11:26 UTC
[Xen-devel] Re: AMD iommu , pci passthrough problems assigning devices to HVM
Hello Wei, It''s an pci express usb3/xhci add-on card, there also seemed to be an alias problem with another device, see this thread on LKML, it also contains the result of amd_iommu_dump on baremetal 2.6.35 kernel. At first the kernel didn''t boot at all, though xen-unstable did load, so i guess there are a few differences in the amd iommu implementation and parsing between the linux kernel and xen. http://lkml.indiana.edu/hypermail/linux/kernel/1008.1/00782.html Would be nice to have some good explanation of the problem, so i could contact MSI to get the bios fixed. -- Sander Wednesday, August 11, 2010, 12:11:58 PM, you wrote:> Hi, > In your debug output I saw: > (XEN) AMD-Vi: Fail to find iommu for device04:00.0 > This means no iommu attached to 04:00.0, so it cannot be assigned to any hvm > guest. What is this device in your system? An add-on device or on board > device? It looks like BIOS failed to create an IVRS entry for this device.> Wei> On Tuesday 10 August 2010 14:26:14 Sander Eikelenboom wrote: >> Hi, >> >> I''m running a machine with: >> - Hypervisor: xen-unstable >> - Dom0: xen/next-2.6.32.17 commit c3f5aba942c7516a5c76c47345ff0a473daf2542 >> >> Hardware: >> -MSI 890FXA-GD70 with AMD IOMMU >> -Phenom X6 >> >> I''m trying to do PCI passthrough to a HVM, this fails with: >> >> Error: Failed to assign device to IOMMU >> (0000:04:00.0@100,msitranslate=1,power_mgmt=0) Error: pci: to avoid >> potential security issue, 0000:08:00.0 is not allowed to be assigned to >> guest since it is behind PCIe switch that does not support or enable ACS. >> >> >> I have found this thread https://bugzilla.redhat.com/show_bug.cgi?id=526713 >> regarding the ACS problem, which says the pci bridges need these >> capabilities: ScrValid+, ReqRedir+, CmpltRedir+, and UpstreamFwd+ >> >> When i do a lspci -vvv on my system i see these capabilities, but i do see >> some disturbing debug message on boot when amd_iommu_debug is on. >> >> Attached: >> - xm info >> - xm dmesg (booted with amd_iommu_debug on) >> - dmesg >> - lspci -vvv >> >> -- >> Sander-- Best regards, Sander mailto:linux@eikelenboom.it _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Possibly Parallel Threads
- RESEND [Xen-unstable][Qemu-xen] HVM Guest reading of Expansion ROM from passthroughed PCI device returns data from emulated VGA rom
- [PATCH 1/1 V3] x86/AMD-Vi: Add additional check for invalid special->handle
- [PATCH 0 of 4] amd iommu: IOMMUv2 support
- [PATCHv2 0 of 2] Deal with IOMMU faults in softirq context.
- [PATCH V2] x86/AMD-Vi: Add additional check for invalid special->handle