I've made alot of progress on getting a Windows VM to have control over a 290x using Xen, but it's not yet working properly. I have a ton of details I can share on what I've done, but for now I have some specific questions for those who have gone through these steps successfuly on similar hardware. Let me know if any other details would be helpful. None of my PCI devices (including the 290x) shows FLR (function level reset) enabled. Is this a requirement for all GPUs that will be passed to a VM or is it just for swapping the card between dom0 and domU? Would I still need FLR enabled if I ensured that dom0 only sees loads xen-pciback module for my 290x? 'sudo lspci -vv' reports FLReset- for all devices (not just my 290x). First I need to ensure FLR is critical for a successful PCI (or GPU) passthrouth. If so, then this makes me think I need to either 1) enable something in the BIOS or 2) change a kernel or module parameter. I've heard that some motherboards (for instance the Intel DH55HC) have an option to enable FLR in the chipset section of the BIOS. My motherboard (ASRock Z97 Extreme6) has VT-d support, but doesn't have any FLR option in the BIOS. Does anyone with experience regarding GPU passthrough have FLReset+ next to their GPUs? Here is the output of of 'xl dmesg': Xen 4.4.0 (XEN) Xen version 4.4.0 (joshua@home.network) (gcc (GCC) 4.9.0 20140521 (prerelease)) debug=n Sat May 31 15:52:14 PDT 2014 (XEN) Latest ChangeSet: (XEN) Bootloader: GRUB 2.02~beta2 (XEN) Command line: /xen-4.4.0.gz iommu=1 iommu_inclusive_mapping=1 dom0_mem=16G xsave=1 (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 0 MBR signatures (XEN) Found 0 EDD information structures (XEN) Multiboot-e820 RAM map: (XEN) 0000000000000000 - 0000000000058000 (usable) (XEN) 0000000000058000 - 0000000000059000 (reserved) (XEN) 0000000000059000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 0000000069e4c000 (usable) (XEN) 0000000069e4c000 - 0000000069e53000 (ACPI NVS) (XEN) 0000000069e53000 - 000000006ab54000 (usable) (XEN) 000000006ab54000 - 000000006b085000 (reserved) (XEN) 000000006b085000 - 000000007d185000 (usable) (XEN) 000000007d185000 - 000000007d216000 (reserved) (XEN) 000000007d216000 - 000000007d25d000 (usable) (XEN) 000000007d25d000 - 000000007d398000 (ACPI NVS) (XEN) 000000007d398000 - 000000007df7d000 (reserved) (XEN) 000000007df7d000 - 000000007dfff000 type 20 (XEN) 000000007dfff000 - 000000007e000000 (usable) (XEN) 000000007f000000 - 00000000bf200000 (reserved) (XEN) 00000000f0000000 - 00000000f8000000 (reserved) (XEN) 00000000fec00000 - 00000000fec01000 (reserved) (XEN) 00000000fed00000 - 00000000fed04000 (reserved) (XEN) 00000000fed1c000 - 00000000fed20000 (reserved) (XEN) 00000000fee00000 - 00000000fee01000 (reserved) (XEN) 00000000ff000000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 000000083fe00000 (usable) (XEN) ACPI: RSDP 000F0010, 0024 (r2 ALASKA) (XEN) ACPI: XSDT 7D365088, 0094 (r1 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: FACP 7D376090, 010C (r5 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: DSDT 7D3651B8, 10ED6 (r2 ALASKA A M I 35 INTL 20120711) (XEN) ACPI: FACS 7D397F80, 0040 (XEN) ACPI: APIC 7D3761A0, 0092 (r3 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: FPDT 7D376238, 0044 (r1 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: SSDT 7D376280, 0539 (r1 PmRef Cpu0Ist 3000 INTL 20051117) (XEN) ACPI: SSDT 7D3767C0, 0B74 (r1 CpuRef CpuSsdt 3000 INTL 20051117) (XEN) ACPI: SSDT 7D377338, 01C7 (r1 PmRef LakeTiny 3000 INTL 20051117) (XEN) ACPI: MCFG 7D377500, 003C (r1 ALASKA A M I 1072009 MSFT 97) (XEN) ACPI: HPET 7D377540, 0038 (r1 ALASKA A M I 1072009 AMI. 5) (XEN) ACPI: SSDT 7D377578, 036D (r1 SataRe SataTabl 1000 INTL 20120711) (XEN) ACPI: SSDT 7D3778E8, 5B5E (r1 SaSsdt SaSsdt 3000 INTL 20120711) (XEN) ACPI: AAFT 7D37D448, 049F (r1 ALASKA OEMAAFT 1072009 MSFT 97) (XEN) ACPI: BGRT 7D37D8E8, 0038 (r0 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: DMAR 7D37D920, 00B8 (r1 INTEL BDW 1 INTL 1) (XEN) ACPI: SSDT 7D37D9D8, 0579 (r1 Intel_ IsctTabl 1000 INTL 20120711) (XEN) System RAM: 31690MB (32450764kB) (XEN) Domain heap initialised (XEN) ACPI: 32/64X FACS address mismatch in FADT - 7d397f80/0000000000000000, using 32 (XEN) Processor #0 7:12 APIC version 21 (XEN) Processor #2 7:12 APIC version 21 (XEN) Processor #4 7:12 APIC version 21 (XEN) Processor #6 7:12 APIC version 21 (XEN) Processor #1 7:12 APIC version 21 (XEN) Processor #3 7:12 APIC version 21 (XEN) Processor #5 7:12 APIC version 21 (XEN) Processor #7 7:12 APIC version 21 (XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23 (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) Failed to enable Interrupt Remapping: Will not enable x2APIC. (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 3598.990 MHz processor. (XEN) Initing memory sharing. (XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7 (XEN) spurious 8259A interrupt: IRQ7. (XEN) Intel VT-d iommu 0 supported page sizes: 4kB. (XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB. (XEN) Intel VT-d Snoop Control not enabled. (XEN) Intel VT-d Dom0 DMA Passthrough not enabled. (XEN) Intel VT-d Queued Invalidation enabled. (XEN) Intel VT-d Interrupt Remapping enabled. (XEN) Intel VT-d Shared EPT tables not enabled. (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) Interrupt remapping enabled (XEN) Enabled directed EOI with ioapic_ack_old on! (XEN) ENABLING IO-APIC IRQs (XEN) -> Using old ACK method (XEN) Platform timer is 14.318MHz HPET (XEN) Allocated console ring of 16 KiB. (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Extended Page Tables (EPT) (XEN) - Virtual-Processor Identifiers (VPID) (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) - Unrestricted Guest (XEN) - VMCS shadowing (XEN) HVM: ASIDs enabled. (XEN) HVM: VMX enabled (XEN) HVM: Hardware Assisted Paging (HAP) detected (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB (XEN) Brought up 8 CPUs (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1f5b000 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000820000000->0000000828000000 (4158762 pages to be allocated) (XEN) Init. ramdisk: 000000083f32a000->000000083fdff200 (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff81000000->ffffffff81f5b000 (XEN) Init. ramdisk: ffffffff81f5b000->ffffffff82a30200 (XEN) Phys-Mach map: ffffffff82a31000->ffffffff84a31000 (XEN) Start info: ffffffff84a31000->ffffffff84a314b4 (XEN) Page tables: ffffffff84a32000->ffffffff84a5b000 (XEN) Boot stack: ffffffff84a5b000->ffffffff84a5c000 (XEN) TOTAL: ffffffff80000000->ffffffff84c00000 (XEN) ENTRY ADDRESS: ffffffff818ee1f0 (XEN) Dom0 has maximum 8 VCPUs (XEN) Scrubbing Free RAM: .....................................................................................................................................................done. (XEN) Initial low memory virq threshold set at 0x4000 pages. (XEN) Std. Loglevel: Errors and warnings (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen) (XEN) Freed 260kB init memory. Here is my /etc/xen/vwin.hvm (vm config): name = 'vwin' builder = 'hvm' memory = 8192 vcpus = 2 disk = [ '/home/joshua/vms/vwin.img,raw,xvda,w', 'file:/home/joshua/vms/win81.iso,hdc:cdrom,r' ] vif = [ 'bridge=xenbr0,ip=192.168.1.115' ] vnclisten = '0.0.0.0' vnc = 1 pci = [ '01:00.0', '01:00.1' ] pci_permissive = 1 When I successfully start the VM, here is the output: Parsing config from /etc/xen/vwin.hvm libxl: error: libxl_pci.c:990:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:01:00.0 libxl: error: libxl_pci.c:990:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:01:00.1 The above is what first made me think there is an issue (which led me to find out more regarding FLR). When Windows boots I see the 290x, and when I run the AMD test to see what hardware I have it finds it as being a 290x. I can install the drivers for the card and no errors are reported. I reboot and look at Device Manager and I see the following message associated with the problem 290x device: This device cannot find enough free resources that it can use. (Code 12) I've looked around for other devices which may be conflicting with the 290x, and there is one other device which has no drivers currently. Other than that there is another Microsoft display device in the same section of the list next to the 290x. Disabling one or both of these devices and rebooting has no effect. Here's the relevant portion of my grub config: multiboot /xen-4.4.0.gz /xen-4.4.0.gz iommu=1 iommu_inclusive_mapping=1 dom0_mem=16G xsave=1 module /vmlinuz-linux /vmlinuz-linux root=UUID=917153ed-f386-4987-8fd0-9e293f8d969a rw modprobe.blacklist=radeon i915 quiet rootflags=subvol=rootvol console=tty0 xen-pciback.passthrough=1 xen-pciback.hide=(01:00.0)(01:00.1) I think 'iommu=1' is related to previous versions of Xen, and I'm not sure what 'iommu_inclusive_mapping=1' even does. I haven't verified that I get the same current functionality without these lines yet. Please request any other details if you think it would be helpful. This is an up-to-date archlinux, Xen 4.4.0-4 system with the following specs: motherboard: ASRock Z97 Extreme6 (BIOS v1.30) cpu: Intel i7-4790 memory: G.SKILL Ripjaws Z Series F3-2133C11Q-32GZL domU OS: archlinux domU a/v: Asus R9290X DirectCU II OC dom0 OS: Windows8.1 64-bit dom0 a/v: Intel HD Graphics 4600 ssd: 2x Samsung EVO 840 250GB optical: LG Blu-ray UH12NS30 psu: Enermax Maxrevo EMR1500EWT