Michaael Mair-Keimberger
2013-Dec-22 14:08 UTC
vga passthrogh on mainboards with broken ivrs tables
Hi Lists,
I'm having troubles getting vga passthrough working again with latest
xen-4.3.1.
My system is a bulldozer amd 8core cpu with a gigabyte GA-990FXA-UD5
motherboard.
The host system uses a amd hd6850 while the guest (windows 7) uses a
amd hd 7970.
So far everything worked great until xsa-36 [1]. I've haven't updated
xen since this "fix", but yesterday i'll gave it another try.
I've
updated to xen-4.3.1 and tried to fix those broken ivrs tables with
the newly introduced ivrx_ioapic kernel parameter. I'm not sure if this
should help me, but at least it sounds promising.
However i didn't get it to work and i hope someone can give my some hints
how to fix that, otherwise i have to downgrade xen again.
grub2 config:
menuentry 'Gentoo Linux XEN' {
root=hd0,1
multiboot /boot/xen.gz dom0_mem=8192M,max:8192M iommu=1 dom0_max_vcpus=4
dom0_vcpus_pin ivrs_ioapic[8]=00:14.0
module /boot/gentoo-3.12.6 root=/dev/md3 radeon.pcie_gen2=1 radeon.dpm=1
softlevel=xen xen-pciback.permissive
xen-pciback.hide=(01:00.0)(01:00.1)(00:12.0)(00:12.2)
}
lspci:
00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI
bridge (external gfx0 port B) (rev 02)
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory
Management Unit (IOMMU)
00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI
bridge (PCI express gpp port B)
00:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI
bridge (PCI express gpp port H)
00:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI
bridge (external gfx1 port A)
00:0d.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI
bridge (external gfx1 port B)
00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI]
SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0
USB OHCI0 Controller
00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0
USB EHCI Controller
00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0
USB OHCI0 Controller
00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0
USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller
(rev 42)
00:14.1 IDE interface: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0
IDE Controller (rev 40)
00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel
HDA) (rev 40)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC
host controller (rev 40)
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI
Bridge (rev 40)
00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0
USB OHCI2 Controller
00:15.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI
to PCI bridge (PCIE port 0)
00:15.1 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI
to PCI bridge (PCIE port 1)
00:15.2 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI to PCI
bridge (PCIE port 2)
00:15.3 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI to PCI
bridge (PCIE port 3)
00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0
USB OHCI0 Controller
00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0
USB EHCI Controller
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor
Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor
Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor
Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor
Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor
Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor
Function 5
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti
XT [Radeon HD 7970]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT HDMI
Audio [Radeon HD 7970 Series]
02:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller
(rev 01)
03:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s
Controller (rev 11)
04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Barts
PRO [Radeon HD 6850]
04:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Barts HDMI Audio
[Radeon HD 6800 Series]
05:0e.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)]
IEEE 1394 OHCI Controller (rev c0)
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411
PCI Express Gigabit Ethernet Controller (rev 06)
07:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller
(rev 01)
08:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s
Controller (rev 11)
lspci -t:
-[0000:00]-+-00.0
+-00.2
+-02.0-[01]--+-00.0
| \-00.1
+-09.0-[02]----00.0
+-0a.0-[03]----00.0
+-0d.0-[04]--+-00.0
| \-00.1
+-11.0
+-12.0
+-12.2
+-13.0
+-13.2
+-14.0
+-14.1
+-14.2
+-14.3
+-14.4-[05]----0e.0
+-14.5
+-15.0-[06]----00.0
+-15.1-[07]----00.0
+-15.2-[08]----00.0
+-15.3-[09]--
+-16.0
+-16.2
+-18.0
+-18.1
+-18.2
+-18.3
+-18.4
\-18.5
xl dmesg (shorten):
(XEN) Xen version 4.3.1 (@xonet.at) (x86_64-pc-linux-gnu-gcc (Gentoo 4.8.2 p1.0,
pie-0.5.8) 4.8.2) debug=n Sat Dec 21 11:55:39 CET 2013
(XEN) Latest ChangeSet:
(XEN) Bootloader: GRUB 2.02~beta1
(XEN) Command line: dom0_mem=8192M,max:8192M iommu=1 dom0_max_vcpus=4
dom0_vcpus_pin ivrs_ioapic[8]=00:14.0
...
(XEN) IOAPIC[0]: apic_id 8, version 33, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 4026.984 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
(XEN) IVHD Error: Conflicting IO-APIC 0x8 entries
(XEN) AMD-Vi: Error initialization
(XEN) I/O virtualisation disabled
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) HVM: ASIDs enabled.
(XEN) SVM: Supported advanced features:
(XEN) - Nested Page Tables (NPT)
(XEN) - Last Branch Record (LBR) Virtualisation
(XEN) - Next-RIP Saved on #VMEXIT
(XEN) - VMCB Clean Bits
(XEN) - DecodeAssists
(XEN) - Pause-Intercept Filter
(XEN) - TSC Rate MSR
(XEN) HVM: SVM enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) spurious 8259A interrupt: IRQ7.
I've tried a few other configuration's (like adding the ivrs_ioapic
parameter
to the module line in grub or also adding
vfio_iommu_type1.allow_unsafe_interrupts=1
to the cmdline), but so far i couldn't get it to work.
Since i'm not completely sure what i actually have to insert at the kernel
parameter i'm glad if someone could give me some insight. However, since
the output of xl dmesg is always the same, even though with or without
the parameter, i might missed some additional parameters.
If you need any more information, I'll be glad to provide them.
Thanks in advance.
Michael
[1] http://lists.xen.org/archives/html/xen-announce/2013-02/msg00002.html