This issue has been discussed many times before, but I haven''t found an answer in my specific problem. Motherboard: Asus Sabertooth 990fx R2.0 Linux 3.2.0-35 through 3.8.x (tested 5 kernels to be ridiculously thorough) Affected: Xen-hypervisor >4.1.2 Not Affected: Xen-hypervisor <4.1.2 Xen 4.1.2 and earlier works fine regardless of the BIOS version of the Mobo. After upgrading Xen hypervisor beyond 4.1.2, we get the following: (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs (XEN) Table is not found! (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 3110.540 MHz processor. (XEN) Initing memory sharing. (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007 (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff (XEN) IVHD Error: Invalid IO-APIC 0xff (XEN) AMD-Vi: Error initialization (XEN) I/O virtualisation disabled From what I understand, Xen used to just care less about the state of your ACPI/IVRS but that this was potentially a security concern and checks were implemented to disable IOMMU whenever it would be unsafe (potentially) to have it enabled. Now, I understand the logic here, but I disagree with the implementation. There are a fairly large number of people who are affected and we know Asus isn''t going to fix the IVRS table. I recognize the importance of implementing these checks and hell, even automatically disabling IOMMU without notice (though on a consumer board, this is a bit paranoid). I understand there are kernel command line options you can pass to forcibly bypass the checks and leave IOMMU enabled in most cases (ie: iommu=no-amd-iommu-perdev-intremap) but thus far none of them seem to be enough to force IOMMU back on. I read somewhere in one of the many discussions on the subject, that there are cases where you simply aren''t allowed to disable the checks (I believe this was when the IVRS table indicies don''t line up or something, which seems to be my issue). As I said, I know other people have precisely this problem with a whole line of mid range Asus boards, but haven''t seen a lot of discussion on the list. Anyone reading this run into the same problem? I bought this board specifically because of it''s IOMMU implementation, which worked fine... at the time... Ultimately I know for a fact that the boards ACPI/IVRS can''t be TOO wrong as it''s been working for 6 months without an issue. I''m just sorta in a situation where it''s time to upgrade my base OS and hacking Xen 4.1.2 in seems like a silly amount of work when all I really need is an option to disable the checks. xm dmesg from a working vs. borked system Working: (XEN) Xen version 4.1.2 (Ubuntu 4.1.2-2ubuntu2) (stefan.bader@canonical.com) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu2) ) Thu Mar 8 14:31:11 UTC 2012 (XEN) Bootloader: GRUB 1.99-21ubuntu3.9 (XEN) Command line: pci_msitranslate=0 xen-pciback=passthrough xen-pciback.hide=(06:00.0)(06.00.1) (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 6 MBR signatures (XEN) Found 6 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009e800 (usable) (XEN) 000000000009e800 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000bc348000 (usable) (XEN) 00000000bc348000 - 00000000bc78c000 (reserved) (XEN) 00000000bc78c000 - 00000000bc797000 (ACPI data) (XEN) 00000000bc797000 - 00000000bd4f8000 (ACPI NVS) (XEN) 00000000bd4f8000 - 00000000bd8d5000 (reserved) (XEN) 00000000bd8d5000 - 00000000bd8d6000 (usable) (XEN) 00000000bd8d6000 - 00000000bdad9000 (ACPI NVS) (XEN) 00000000bdad9000 - 00000000bdf00000 (usable) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec01000 (reserved) (XEN) 00000000fec10000 - 00000000fec11000 (reserved) (XEN) 00000000fec20000 - 00000000fec21000 (reserved) (XEN) 00000000fed00000 - 00000000fed01000 (reserved) (XEN) 00000000fed61000 - 00000000fed71000 (reserved) (XEN) 00000000fed80000 - 00000000fed90000 (reserved) (XEN) 00000000fef00000 - 0000000100000000 (reserved) (XEN) 0000000100001000 - 000000043f000000 (usable) (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA) (XEN) ACPI: XSDT BC78E078, 0064 (r1 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: FACP BC795BA8, 00F4 (r4 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has zero address or length: 0000000000000000/1 [20070126] (XEN) ACPI: DSDT BC78E170, 7A38 (r2 ALASKA A M I 0 INTL 20051117) (XEN) ACPI: FACS BD4F2F80, 0040 (XEN) ACPI: APIC BC795CA0, 009E (r3 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: FPDT BC795D40, 0044 (r1 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: MCFG BC795D88, 003C (r1 ALASKA A M I 1072009 MSFT 10013) (XEN) ACPI: HPET BC795DC8, 0038 (r1 ALASKA A M I 1072009 AMI 5) (XEN) ACPI: BGRT BC796260, 0038 (r0 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: IVRS BC795E58, 0100 (r1 AMD RD890S 202031 AMD 0) (XEN) ACPI: SSDT BC795F58, 0304 (r1 AMD POWERNOW 1 AMD 1) (XEN) System RAM: 16311MB (16702516kB) (XEN) Domain heap initialised (XEN) ACPI: 32/64X FACS address mismatch in FADT - bd4f2f80/0000000000000000, using 32 (XEN) Processor #16 5:1 APIC version 16 (XEN) Processor #17 5:1 APIC version 16 (XEN) Processor #18 5:1 APIC version 16 (XEN) Processor #19 5:1 APIC version 16 (XEN) Processor #20 5:1 APIC version 16 (XEN) Processor #21 5:1 APIC version 16 (XEN) Processor #22 5:1 APIC version 16 (XEN) Processor #23 5:1 APIC version 16 (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs (XEN) Table is not found! (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 3110.553 MHz processor. (XEN) Initing memory sharing. (XEN) AMD-Vi: IOMMU 0 Enabled. (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) Platform timer is 14.318MHz HPET (XEN) Allocated console ring of 16 KiB. (XEN) HVM: ASIDs enabled. (XEN) SVM: Supported advanced features: (XEN) - Nested Page Tables (NPT) (XEN) - Last Branch Record (LBR) Virtualisation (XEN) - Next-RIP Saved on #VMEXIT (XEN) - VMCB Clean Bits (XEN) - Pause-Intercept Filter (XEN) HVM: SVM enabled (XEN) HVM: Hardware Assisted Paging detected. (XEN) Brought up 8 CPUs (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x205f000 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4044532 pages to be allocated) (XEN) Init. ramdisk: 000000043c7dd000->000000043efffa00 (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff81000000->ffffffff8205f000 (XEN) Init. ramdisk: ffffffff8205f000->ffffffff84881a00 (XEN) Phys-Mach map: ffffffff84882000->ffffffff867b18b8 (XEN) Start info: ffffffff867b2000->ffffffff867b24b4 (XEN) Page tables: ffffffff867b3000->ffffffff867ec000 (XEN) Boot stack: ffffffff867ec000->ffffffff867ed000 (XEN) TOTAL: ffffffff80000000->ffffffff86c00000 (XEN) ENTRY ADDRESS: ffffffff81cfb200 (XEN) Dom0 has maximum 8 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: Errors and warnings (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to switch input to Xen) (XEN) Freed 220kB init memory. (XEN) traps.c:2432:d0 Domain attempted WRMSR 00000000c0010201 from 0x0000000000000000 to 0x000000000000abcd. (XEN) physdev.c:155: dom0: wrong map_pirq type 3 BORKED: (XEN) Xen version 4.2.1 (Ubuntu 4.2.1-0ubuntu3.1) (stefan.bader@canonical.com) (gcc (Ubuntu/Linaro 4.7.3-1ubuntu1) 4.7.3) Mon Apr 29 19:35:31 UTC 2013 (XEN) Bootloader: GRUB 2.00-13ubuntu3 (XEN) Command line: iommu=no-amd-iommu-perdev-intremap x2apic=off xen-pciback.hide=(06:00.0)(06.00.1) (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 7 MBR signatures (XEN) Found 6 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009e800 (usable) (XEN) 000000000009e800 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000ba7ac000 (usable) (XEN) 00000000ba7ac000 - 00000000babe0000 (reserved) (XEN) 00000000babe0000 - 00000000babf0000 (ACPI data) (XEN) 00000000babf0000 - 00000000bb958000 (ACPI NVS) (XEN) 00000000bb958000 - 00000000bca35000 (reserved) (XEN) 00000000bca35000 - 00000000bca36000 (usable) (XEN) 00000000bca36000 - 00000000bcc3c000 (ACPI NVS) (XEN) 00000000bcc3c000 - 00000000bd083000 (usable) (XEN) 00000000bd083000 - 00000000bd7f4000 (reserved) (XEN) 00000000bd7f4000 - 00000000bd800000 (usable) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec01000 (reserved) (XEN) 00000000fec10000 - 00000000fec11000 (reserved) (XEN) 00000000fec20000 - 00000000fec21000 (reserved) (XEN) 00000000fed00000 - 00000000fed01000 (reserved) (XEN) 00000000fed61000 - 00000000fed71000 (reserved) (XEN) 00000000fed80000 - 00000000fed90000 (reserved) (XEN) 00000000fef00000 - 0000000100000000 (reserved) (XEN) 0000000100001000 - 000000043f000000 (usable) (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA) (XEN) ACPI: XSDT BABE7078, 0064 (r1 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: FACP BABEE118, 010C (r5 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI Warning (tbfadt-0232): FADT (revision 5) is longer than ACPI 2.0 version, truncating length 0x10C to 0xF4 [20070126] (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has zero address or length: 0000000000000000/1 [20070126] (XEN) ACPI: DSDT BABE7170, 6FA8 (r2 ALASKA A M I 0 INTL 20051117) (XEN) ACPI: FACS BB952F80, 0040 (XEN) ACPI: APIC BABEE228, 009E (r3 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: FPDT BABEE2C8, 0044 (r1 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: MCFG BABEE310, 003C (r1 ALASKA A M I 1072009 MSFT 10013) (XEN) ACPI: HPET BABEE350, 0038 (r1 ALASKA A M I 1072009 AMI 5) (XEN) ACPI: BGRT BABEFBF8, 0038 (r0 ALASKA A M I 1072009 AMI 10013) (XEN) ACPI: IVRS BABEE3E0, 0100 (r1 AMD RD890S 202031 AMD 0) (XEN) ACPI: SSDT BABEE4E0, 1714 (r1 AMD POWERNOW 1 AMD 1) (XEN) System RAM: 16283MB (16674420kB) (XEN) Domain heap initialised (XEN) ACPI: 32/64X FACS address mismatch in FADT - bb952f80/0000000000000000, using 32 (XEN) Processor #16 5:1 APIC version 16 (XEN) Processor #17 5:1 APIC version 16 (XEN) Processor #18 5:1 APIC version 16 (XEN) Processor #19 5:1 APIC version 16 (XEN) Processor #20 5:1 APIC version 16 (XEN) Processor #21 5:1 APIC version 16 (XEN) Processor #22 5:1 APIC version 16 (XEN) Processor #23 5:1 APIC version 16 (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs (XEN) Table is not found! (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 3110.540 MHz processor. (XEN) Initing memory sharing. (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007 (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff (XEN) IVHD Error: Invalid IO-APIC 0xff (XEN) AMD-Vi: Error initialization (XEN) I/O virtualisation disabled (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) Platform timer is 14.318MHz HPET (XEN) Allocated console ring of 16 KiB. (XEN) HVM: ASIDs enabled. (XEN) SVM: Supported advanced features: (XEN) - Nested Page Tables (NPT) (XEN) - Last Branch Record (LBR) Virtualisation (XEN) - Next-RIP Saved on #VMEXIT (XEN) - VMCB Clean Bits (XEN) - DecodeAssists (XEN) - Pause-Intercept Filter (XEN) - TSC Rate MSR (XEN) HVM: SVM enabled (XEN) HVM: Hardware Assisted Paging (HAP) detected (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB (XEN) Brought up 8 CPUs (XEN) mtrr: your CPUs had inconsistent variable MTRR settings (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2346000 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4035583 pages to be allocated) (XEN) Init. ramdisk: 0000000439dc6000->000000043efff800 (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff81000000->ffffffff82346000 (XEN) Init. ramdisk: ffffffff82346000->ffffffff8757f800 (XEN) Phys-Mach map: ffffffff87580000->ffffffff894b31c8 (XEN) Start info: ffffffff894b4000->ffffffff894b44b4 (XEN) Page tables: ffffffff894b5000->ffffffff89504000 (XEN) Boot stack: ffffffff89504000->ffffffff89505000 (XEN) TOTAL: ffffffff80000000->ffffffff89800000 (XEN) ENTRY ADDRESS: ffffffff81d06210 (XEN) Dom0 has maximum 8 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Initial low memory virq threshold set at 0x4000 pages. (XEN) Std. Loglevel: Errors and warnings (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to switch input to Xen) (XEN) Freed 244kB init memory.
David Sutton
2013-May-28 07:31 UTC
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah
Feral, I have the R1.0 version of that motherboard. The problem is with the BIOS; its returning bad information in the IVRS table for the IO-APICs, which the new parser is catching and causing AMD-Vi initialization to fail. I''ve already put in a support ticket via the online form, included all the information such as links to the document which shows what the valid details should be. The ticket was basically closed with a note saying to look out for BIOS updates. I''m currently using 4.2.1 (which was before the table checking was added) and am able to use AMD-Vi. If you want to fix this, you''re either going to have to get ASUS to issue a fixed BIOS (which doesn''t feel likely at this point) or you''d have to either disable interrupt remapping or edit the source code to disable that check (and deal with any potential issue that might cause later) Regards, David On Mon, May 27, 2013 at 11:42 PM, feral <blistovmhz@gmail.com> wrote:> This issue has been discussed many times before, but I haven''t found > an answer in my specific problem. > > Motherboard: Asus Sabertooth 990fx R2.0 > Linux 3.2.0-35 through 3.8.x (tested 5 kernels to be ridiculously thorough) > > Affected: Xen-hypervisor >4.1.2 > Not Affected: Xen-hypervisor <4.1.2 > > > Xen 4.1.2 and earlier works fine regardless of the BIOS version of the > Mobo. After upgrading Xen hypervisor beyond 4.1.2, we get the > following: > (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 > (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 > (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs > (XEN) Table is not found! > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Detected 3110.540 MHz processor. > (XEN) Initing memory sharing. > (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007 > (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff > (XEN) IVHD Error: Invalid IO-APIC 0xff > (XEN) AMD-Vi: Error initialization > (XEN) I/O virtualisation disabled > > > From what I understand, Xen used to just care less about the state of > your ACPI/IVRS but that this was potentially a security concern and > checks were implemented to disable IOMMU whenever it would be unsafe > (potentially) to have it enabled. > Now, I understand the logic here, but I disagree with the > implementation. There are a fairly large number of people who are > affected and we know Asus isn''t going to fix the IVRS table. I > recognize the importance of implementing these checks and hell, even > automatically disabling IOMMU without notice (though on a consumer > board, this is a bit paranoid). > I understand there are kernel command line options you can pass to > forcibly bypass the checks and leave IOMMU enabled in most cases (ie: > iommu=no-amd-iommu-perdev-intremap) but thus far none of them seem to > be enough to force IOMMU back on. > I read somewhere in one of the many discussions on the subject, that > there are cases where you simply aren''t allowed to disable the checks > (I believe this was when the IVRS table indicies don''t line up or > something, which seems to be my issue). > > As I said, I know other people have precisely this problem with a > whole line of mid range Asus boards, but haven''t seen a lot of > discussion on the list. Anyone reading this run into the same > problem? I bought this board specifically because of it''s IOMMU > implementation, which worked fine... at the time... Ultimately I know > for a fact that the boards ACPI/IVRS can''t be TOO wrong as it''s been > working for 6 months without an issue. I''m just sorta in a situation > where it''s time to upgrade my base OS and hacking Xen 4.1.2 in seems > like a silly amount of work when all I really need is an option to > disable the checks. > > > xm dmesg from a working vs. borked system > Working: > (XEN) Xen version 4.1.2 (Ubuntu 4.1.2-2ubuntu2) > (stefan.bader@canonical.com) (gcc version 4.6.3 (Ubuntu/Linaro > 4.6.3-1ubuntu2) ) Thu Mar 8 14:31:11 UTC 2012 > (XEN) Bootloader: GRUB 1.99-21ubuntu3.9 > (XEN) Command line: pci_msitranslate=0 xen-pciback=passthrough > xen-pciback.hide=(06:00.0)(06.00.1) > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 6 MBR signatures > (XEN) Found 6 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009e800 (usable) > (XEN) 000000000009e800 - 00000000000a0000 (reserved) > (XEN) 00000000000e0000 - 0000000000100000 (reserved) > (XEN) 0000000000100000 - 00000000bc348000 (usable) > (XEN) 00000000bc348000 - 00000000bc78c000 (reserved) > (XEN) 00000000bc78c000 - 00000000bc797000 (ACPI data) > (XEN) 00000000bc797000 - 00000000bd4f8000 (ACPI NVS) > (XEN) 00000000bd4f8000 - 00000000bd8d5000 (reserved) > (XEN) 00000000bd8d5000 - 00000000bd8d6000 (usable) > (XEN) 00000000bd8d6000 - 00000000bdad9000 (ACPI NVS) > (XEN) 00000000bdad9000 - 00000000bdf00000 (usable) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec01000 (reserved) > (XEN) 00000000fec10000 - 00000000fec11000 (reserved) > (XEN) 00000000fec20000 - 00000000fec21000 (reserved) > (XEN) 00000000fed00000 - 00000000fed01000 (reserved) > (XEN) 00000000fed61000 - 00000000fed71000 (reserved) > (XEN) 00000000fed80000 - 00000000fed90000 (reserved) > (XEN) 00000000fef00000 - 0000000100000000 (reserved) > (XEN) 0000000100001000 - 000000043f000000 (usable) > (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA) > (XEN) ACPI: XSDT BC78E078, 0064 (r1 ALASKA A M I 1072009 AMI 10013) > (XEN) ACPI: FACP BC795BA8, 00F4 (r4 ALASKA A M I 1072009 AMI 10013) > (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has > zero address or length: 0000000000000000/1 [20070126] > (XEN) ACPI: DSDT BC78E170, 7A38 (r2 ALASKA A M I 0 INTL 20051117) > (XEN) ACPI: FACS BD4F2F80, 0040 > (XEN) ACPI: APIC BC795CA0, 009E (r3 ALASKA A M I 1072009 AMI 10013) > (XEN) ACPI: FPDT BC795D40, 0044 (r1 ALASKA A M I 1072009 AMI 10013) > (XEN) ACPI: MCFG BC795D88, 003C (r1 ALASKA A M I 1072009 MSFT 10013) > (XEN) ACPI: HPET BC795DC8, 0038 (r1 ALASKA A M I 1072009 AMI 5) > (XEN) ACPI: BGRT BC796260, 0038 (r0 ALASKA A M I 1072009 AMI 10013) > (XEN) ACPI: IVRS BC795E58, 0100 (r1 AMD RD890S 202031 AMD 0) > (XEN) ACPI: SSDT BC795F58, 0304 (r1 AMD POWERNOW 1 AMD 1) > (XEN) System RAM: 16311MB (16702516kB) > (XEN) Domain heap initialised > (XEN) ACPI: 32/64X FACS address mismatch in FADT - > bd4f2f80/0000000000000000, using 32 > (XEN) Processor #16 5:1 APIC version 16 > (XEN) Processor #17 5:1 APIC version 16 > (XEN) Processor #18 5:1 APIC version 16 > (XEN) Processor #19 5:1 APIC version 16 > (XEN) Processor #20 5:1 APIC version 16 > (XEN) Processor #21 5:1 APIC version 16 > (XEN) Processor #22 5:1 APIC version 16 > (XEN) Processor #23 5:1 APIC version 16 > (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 > (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 > (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs > (XEN) Table is not found! > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Detected 3110.553 MHz processor. > (XEN) Initing memory sharing. > (XEN) AMD-Vi: IOMMU 0 Enabled. > (XEN) I/O virtualisation enabled > (XEN) - Dom0 mode: Relaxed > (XEN) ENABLING IO-APIC IRQs > (XEN) -> Using new ACK method > (XEN) Platform timer is 14.318MHz HPET > (XEN) Allocated console ring of 16 KiB. > (XEN) HVM: ASIDs enabled. > (XEN) SVM: Supported advanced features: > (XEN) - Nested Page Tables (NPT) > (XEN) - Last Branch Record (LBR) Virtualisation > (XEN) - Next-RIP Saved on #VMEXIT > (XEN) - VMCB Clean Bits > (XEN) - Pause-Intercept Filter > (XEN) HVM: SVM enabled > (XEN) HVM: Hardware Assisted Paging detected. > (XEN) Brought up 8 CPUs > (XEN) *** LOADING DOMAIN 0 *** > (XEN) Xen kernel: 64-bit, lsb, compat32 > (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x205f000 > (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4044532 > pages to be allocated) > (XEN) Init. ramdisk: 000000043c7dd000->000000043efffa00 > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: ffffffff81000000->ffffffff8205f000 > (XEN) Init. ramdisk: ffffffff8205f000->ffffffff84881a00 > (XEN) Phys-Mach map: ffffffff84882000->ffffffff867b18b8 > (XEN) Start info: ffffffff867b2000->ffffffff867b24b4 > (XEN) Page tables: ffffffff867b3000->ffffffff867ec000 > (XEN) Boot stack: ffffffff867ec000->ffffffff867ed000 > (XEN) TOTAL: ffffffff80000000->ffffffff86c00000 > (XEN) ENTRY ADDRESS: ffffffff81cfb200 > (XEN) Dom0 has maximum 8 VCPUs > (XEN) Scrubbing Free RAM: .done. > (XEN) Xen trace buffers: disabled > (XEN) Std. Loglevel: Errors and warnings > (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) > (XEN) Xen is relinquishing VGA console. > (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to > switch input to Xen) > (XEN) Freed 220kB init memory. > (XEN) traps.c:2432:d0 Domain attempted WRMSR 00000000c0010201 from > 0x0000000000000000 to 0x000000000000abcd. > (XEN) physdev.c:155: dom0: wrong map_pirq type 3 > > > BORKED: > (XEN) Xen version 4.2.1 (Ubuntu 4.2.1-0ubuntu3.1) > (stefan.bader@canonical.com) (gcc (Ubuntu/Linaro 4.7.3-1ubuntu1) > 4.7.3) Mon Apr 29 19:35:31 UTC 2013 > (XEN) Bootloader: GRUB 2.00-13ubuntu3 > (XEN) Command line: iommu=no-amd-iommu-perdev-intremap x2apic=off > xen-pciback.hide=(06:00.0)(06.00.1) > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 7 MBR signatures > (XEN) Found 6 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009e800 (usable) > (XEN) 000000000009e800 - 00000000000a0000 (reserved) > (XEN) 00000000000e0000 - 0000000000100000 (reserved) > (XEN) 0000000000100000 - 00000000ba7ac000 (usable) > (XEN) 00000000ba7ac000 - 00000000babe0000 (reserved) > (XEN) 00000000babe0000 - 00000000babf0000 (ACPI data) > (XEN) 00000000babf0000 - 00000000bb958000 (ACPI NVS) > (XEN) 00000000bb958000 - 00000000bca35000 (reserved) > (XEN) 00000000bca35000 - 00000000bca36000 (usable) > (XEN) 00000000bca36000 - 00000000bcc3c000 (ACPI NVS) > (XEN) 00000000bcc3c000 - 00000000bd083000 (usable) > (XEN) 00000000bd083000 - 00000000bd7f4000 (reserved) > (XEN) 00000000bd7f4000 - 00000000bd800000 (usable) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec01000 (reserved) > (XEN) 00000000fec10000 - 00000000fec11000 (reserved) > (XEN) 00000000fec20000 - 00000000fec21000 (reserved) > (XEN) 00000000fed00000 - 00000000fed01000 (reserved) > (XEN) 00000000fed61000 - 00000000fed71000 (reserved) > (XEN) 00000000fed80000 - 00000000fed90000 (reserved) > (XEN) 00000000fef00000 - 0000000100000000 (reserved) > (XEN) 0000000100001000 - 000000043f000000 (usable) > (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA) > (XEN) ACPI: XSDT BABE7078, 0064 (r1 ALASKA A M I 1072009 AMI 10013) > (XEN) ACPI: FACP BABEE118, 010C (r5 ALASKA A M I 1072009 AMI 10013) > (XEN) ACPI Warning (tbfadt-0232): FADT (revision 5) is longer than > ACPI 2.0 version, truncating length 0x10C to 0xF4 [20070126] > (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has > zero address or length: 0000000000000000/1 [20070126] > (XEN) ACPI: DSDT BABE7170, 6FA8 (r2 ALASKA A M I 0 INTL 20051117) > (XEN) ACPI: FACS BB952F80, 0040 > (XEN) ACPI: APIC BABEE228, 009E (r3 ALASKA A M I 1072009 AMI 10013) > (XEN) ACPI: FPDT BABEE2C8, 0044 (r1 ALASKA A M I 1072009 AMI 10013) > (XEN) ACPI: MCFG BABEE310, 003C (r1 ALASKA A M I 1072009 MSFT 10013) > (XEN) ACPI: HPET BABEE350, 0038 (r1 ALASKA A M I 1072009 AMI 5) > (XEN) ACPI: BGRT BABEFBF8, 0038 (r0 ALASKA A M I 1072009 AMI 10013) > (XEN) ACPI: IVRS BABEE3E0, 0100 (r1 AMD RD890S 202031 AMD 0) > (XEN) ACPI: SSDT BABEE4E0, 1714 (r1 AMD POWERNOW 1 AMD 1) > (XEN) System RAM: 16283MB (16674420kB) > (XEN) Domain heap initialised > (XEN) ACPI: 32/64X FACS address mismatch in FADT - > bb952f80/0000000000000000, using 32 > (XEN) Processor #16 5:1 APIC version 16 > (XEN) Processor #17 5:1 APIC version 16 > (XEN) Processor #18 5:1 APIC version 16 > (XEN) Processor #19 5:1 APIC version 16 > (XEN) Processor #20 5:1 APIC version 16 > (XEN) Processor #21 5:1 APIC version 16 > (XEN) Processor #22 5:1 APIC version 16 > (XEN) Processor #23 5:1 APIC version 16 > (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 > (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 > (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs > (XEN) Table is not found! > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Detected 3110.540 MHz processor. > (XEN) Initing memory sharing. > (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007 > (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff > (XEN) IVHD Error: Invalid IO-APIC 0xff > (XEN) AMD-Vi: Error initialization > (XEN) I/O virtualisation disabled > (XEN) ENABLING IO-APIC IRQs > (XEN) -> Using new ACK method > (XEN) Platform timer is 14.318MHz HPET > (XEN) Allocated console ring of 16 KiB. > (XEN) HVM: ASIDs enabled. > (XEN) SVM: Supported advanced features: > (XEN) - Nested Page Tables (NPT) > (XEN) - Last Branch Record (LBR) Virtualisation > (XEN) - Next-RIP Saved on #VMEXIT > (XEN) - VMCB Clean Bits > (XEN) - DecodeAssists > (XEN) - Pause-Intercept Filter > (XEN) - TSC Rate MSR > (XEN) HVM: SVM enabled > (XEN) HVM: Hardware Assisted Paging (HAP) detected > (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB > (XEN) Brought up 8 CPUs > (XEN) mtrr: your CPUs had inconsistent variable MTRR settings > (XEN) *** LOADING DOMAIN 0 *** > (XEN) Xen kernel: 64-bit, lsb, compat32 > (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2346000 > (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4035583 > pages to be allocated) > (XEN) Init. ramdisk: 0000000439dc6000->000000043efff800 > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: ffffffff81000000->ffffffff82346000 > (XEN) Init. ramdisk: ffffffff82346000->ffffffff8757f800 > (XEN) Phys-Mach map: ffffffff87580000->ffffffff894b31c8 > (XEN) Start info: ffffffff894b4000->ffffffff894b44b4 > (XEN) Page tables: ffffffff894b5000->ffffffff89504000 > (XEN) Boot stack: ffffffff89504000->ffffffff89505000 > (XEN) TOTAL: ffffffff80000000->ffffffff89800000 > (XEN) ENTRY ADDRESS: ffffffff81d06210 > (XEN) Dom0 has maximum 8 VCPUs > (XEN) Scrubbing Free RAM: .done. > (XEN) Initial low memory virq threshold set at 0x4000 pages. > (XEN) Std. Loglevel: Errors and warnings > (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) > (XEN) Xen is relinquishing VGA console. > (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to > switch input to Xen) > (XEN) Freed 244kB init memory. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Gizmo Chicken
2013-May-28 10:11 UTC
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah
David and feral, I have an Asus M5A99FX Pro 2.0 motherboard and, at least as far as I can tell, IOMMU works well with that motherboard and Xen 4.1. (I got VGA passthrough working with a Windows guest.) Although I did some preliminary testing with Xen 4.2 on that motherboard, because of some other issues, I never got around to creating a VM that required IOMMU support, and so can''t say for certain whether IOMMU is working with that motherboard and Xen 4.2. Apart from creating a VM that requires IOMMU support (which would take more time than I''d prefer to spend right now) to see if it works, what would good way to determine whether my Asus M5A99FX Pro 2.0 motherboard suffers, or if free from, the IVRS table issue that you describe? Thanks much for any help with this! Best regards, GizmoChicken On Tue, May 28, 2013 at 3:31 AM, David Sutton <kantras@gmail.com> wrote:> Feral, > > I have the R1.0 version of that motherboard. The problem is with the > BIOS; its returning bad information in the IVRS table for the IO-APICs, > which the new parser is catching and causing AMD-Vi initialization to fail. > I''ve already put in a support ticket via the online form, included all the > information such as links to the document which shows what the valid > details should be. The ticket was basically closed with a note saying to > look out for BIOS updates. I''m currently using 4.2.1 (which was before the > table checking was added) and am able to use AMD-Vi. If you want to fix > this, you''re either going to have to get ASUS to issue a fixed BIOS (which > doesn''t feel likely at this point) or you''d have to either disable > interrupt remapping or edit the source code to disable that check (and deal > with any potential issue that might cause later) > > Regards, > > David > > > On Mon, May 27, 2013 at 11:42 PM, feral <blistovmhz@gmail.com> wrote: > >> This issue has been discussed many times before, but I haven''t found >> an answer in my specific problem. >> >> Motherboard: Asus Sabertooth 990fx R2.0 >> Linux 3.2.0-35 through 3.8.x (tested 5 kernels to be ridiculously >> thorough) >> >> Affected: Xen-hypervisor >4.1.2 >> Not Affected: Xen-hypervisor <4.1.2 >> >> >> Xen 4.1.2 and earlier works fine regardless of the BIOS version of the >> Mobo. After upgrading Xen hypervisor beyond 4.1.2, we get the >> following: >> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 >> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 >> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs >> (XEN) Table is not found! >> (XEN) Using scheduler: SMP Credit Scheduler (credit) >> (XEN) Detected 3110.540 MHz processor. >> (XEN) Initing memory sharing. >> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007 >> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff >> (XEN) IVHD Error: Invalid IO-APIC 0xff >> (XEN) AMD-Vi: Error initialization >> (XEN) I/O virtualisation disabled >> >> >> From what I understand, Xen used to just care less about the state of >> your ACPI/IVRS but that this was potentially a security concern and >> checks were implemented to disable IOMMU whenever it would be unsafe >> (potentially) to have it enabled. >> Now, I understand the logic here, but I disagree with the >> implementation. There are a fairly large number of people who are >> affected and we know Asus isn''t going to fix the IVRS table. I >> recognize the importance of implementing these checks and hell, even >> automatically disabling IOMMU without notice (though on a consumer >> board, this is a bit paranoid). >> I understand there are kernel command line options you can pass to >> forcibly bypass the checks and leave IOMMU enabled in most cases (ie: >> iommu=no-amd-iommu-perdev-intremap) but thus far none of them seem to >> be enough to force IOMMU back on. >> I read somewhere in one of the many discussions on the subject, that >> there are cases where you simply aren''t allowed to disable the checks >> (I believe this was when the IVRS table indicies don''t line up or >> something, which seems to be my issue). >> >> As I said, I know other people have precisely this problem with a >> whole line of mid range Asus boards, but haven''t seen a lot of >> discussion on the list. Anyone reading this run into the same >> problem? I bought this board specifically because of it''s IOMMU >> implementation, which worked fine... at the time... Ultimately I know >> for a fact that the boards ACPI/IVRS can''t be TOO wrong as it''s been >> working for 6 months without an issue. I''m just sorta in a situation >> where it''s time to upgrade my base OS and hacking Xen 4.1.2 in seems >> like a silly amount of work when all I really need is an option to >> disable the checks. >> >> >> xm dmesg from a working vs. borked system >> Working: >> (XEN) Xen version 4.1.2 (Ubuntu 4.1.2-2ubuntu2) >> (stefan.bader@canonical.com) (gcc version 4.6.3 (Ubuntu/Linaro >> 4.6.3-1ubuntu2) ) Thu Mar 8 14:31:11 UTC 2012 >> (XEN) Bootloader: GRUB 1.99-21ubuntu3.9 >> (XEN) Command line: pci_msitranslate=0 xen-pciback=passthrough >> xen-pciback.hide=(06:00.0)(06.00.1) >> (XEN) Video information: >> (XEN) VGA is text mode 80x25, font 8x16 >> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >> (XEN) Disc information: >> (XEN) Found 6 MBR signatures >> (XEN) Found 6 EDD information structures >> (XEN) Xen-e820 RAM map: >> (XEN) 0000000000000000 - 000000000009e800 (usable) >> (XEN) 000000000009e800 - 00000000000a0000 (reserved) >> (XEN) 00000000000e0000 - 0000000000100000 (reserved) >> (XEN) 0000000000100000 - 00000000bc348000 (usable) >> (XEN) 00000000bc348000 - 00000000bc78c000 (reserved) >> (XEN) 00000000bc78c000 - 00000000bc797000 (ACPI data) >> (XEN) 00000000bc797000 - 00000000bd4f8000 (ACPI NVS) >> (XEN) 00000000bd4f8000 - 00000000bd8d5000 (reserved) >> (XEN) 00000000bd8d5000 - 00000000bd8d6000 (usable) >> (XEN) 00000000bd8d6000 - 00000000bdad9000 (ACPI NVS) >> (XEN) 00000000bdad9000 - 00000000bdf00000 (usable) >> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >> (XEN) 00000000fec00000 - 00000000fec01000 (reserved) >> (XEN) 00000000fec10000 - 00000000fec11000 (reserved) >> (XEN) 00000000fec20000 - 00000000fec21000 (reserved) >> (XEN) 00000000fed00000 - 00000000fed01000 (reserved) >> (XEN) 00000000fed61000 - 00000000fed71000 (reserved) >> (XEN) 00000000fed80000 - 00000000fed90000 (reserved) >> (XEN) 00000000fef00000 - 0000000100000000 (reserved) >> (XEN) 0000000100001000 - 000000043f000000 (usable) >> (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA) >> (XEN) ACPI: XSDT BC78E078, 0064 (r1 ALASKA A M I 1072009 AMI >> 10013) >> (XEN) ACPI: FACP BC795BA8, 00F4 (r4 ALASKA A M I 1072009 AMI >> 10013) >> (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has >> zero address or length: 0000000000000000/1 [20070126] >> (XEN) ACPI: DSDT BC78E170, 7A38 (r2 ALASKA A M I 0 INTL >> 20051117) >> (XEN) ACPI: FACS BD4F2F80, 0040 >> (XEN) ACPI: APIC BC795CA0, 009E (r3 ALASKA A M I 1072009 AMI >> 10013) >> (XEN) ACPI: FPDT BC795D40, 0044 (r1 ALASKA A M I 1072009 AMI >> 10013) >> (XEN) ACPI: MCFG BC795D88, 003C (r1 ALASKA A M I 1072009 MSFT >> 10013) >> (XEN) ACPI: HPET BC795DC8, 0038 (r1 ALASKA A M I 1072009 AMI >> 5) >> (XEN) ACPI: BGRT BC796260, 0038 (r0 ALASKA A M I 1072009 AMI >> 10013) >> (XEN) ACPI: IVRS BC795E58, 0100 (r1 AMD RD890S 202031 AMD >> 0) >> (XEN) ACPI: SSDT BC795F58, 0304 (r1 AMD POWERNOW 1 AMD >> 1) >> (XEN) System RAM: 16311MB (16702516kB) >> (XEN) Domain heap initialised >> (XEN) ACPI: 32/64X FACS address mismatch in FADT - >> bd4f2f80/0000000000000000, using 32 >> (XEN) Processor #16 5:1 APIC version 16 >> (XEN) Processor #17 5:1 APIC version 16 >> (XEN) Processor #18 5:1 APIC version 16 >> (XEN) Processor #19 5:1 APIC version 16 >> (XEN) Processor #20 5:1 APIC version 16 >> (XEN) Processor #21 5:1 APIC version 16 >> (XEN) Processor #22 5:1 APIC version 16 >> (XEN) Processor #23 5:1 APIC version 16 >> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 >> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 >> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs >> (XEN) Table is not found! >> (XEN) Using scheduler: SMP Credit Scheduler (credit) >> (XEN) Detected 3110.553 MHz processor. >> (XEN) Initing memory sharing. >> (XEN) AMD-Vi: IOMMU 0 Enabled. >> (XEN) I/O virtualisation enabled >> (XEN) - Dom0 mode: Relaxed >> (XEN) ENABLING IO-APIC IRQs >> (XEN) -> Using new ACK method >> (XEN) Platform timer is 14.318MHz HPET >> (XEN) Allocated console ring of 16 KiB. >> (XEN) HVM: ASIDs enabled. >> (XEN) SVM: Supported advanced features: >> (XEN) - Nested Page Tables (NPT) >> (XEN) - Last Branch Record (LBR) Virtualisation >> (XEN) - Next-RIP Saved on #VMEXIT >> (XEN) - VMCB Clean Bits >> (XEN) - Pause-Intercept Filter >> (XEN) HVM: SVM enabled >> (XEN) HVM: Hardware Assisted Paging detected. >> (XEN) Brought up 8 CPUs >> (XEN) *** LOADING DOMAIN 0 *** >> (XEN) Xen kernel: 64-bit, lsb, compat32 >> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x205f000 >> (XEN) PHYSICAL MEMORY ARRANGEMENT: >> (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4044532 >> pages to be allocated) >> (XEN) Init. ramdisk: 000000043c7dd000->000000043efffa00 >> (XEN) VIRTUAL MEMORY ARRANGEMENT: >> (XEN) Loaded kernel: ffffffff81000000->ffffffff8205f000 >> (XEN) Init. ramdisk: ffffffff8205f000->ffffffff84881a00 >> (XEN) Phys-Mach map: ffffffff84882000->ffffffff867b18b8 >> (XEN) Start info: ffffffff867b2000->ffffffff867b24b4 >> (XEN) Page tables: ffffffff867b3000->ffffffff867ec000 >> (XEN) Boot stack: ffffffff867ec000->ffffffff867ed000 >> (XEN) TOTAL: ffffffff80000000->ffffffff86c00000 >> (XEN) ENTRY ADDRESS: ffffffff81cfb200 >> (XEN) Dom0 has maximum 8 VCPUs >> (XEN) Scrubbing Free RAM: .done. >> (XEN) Xen trace buffers: disabled >> (XEN) Std. Loglevel: Errors and warnings >> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) >> (XEN) Xen is relinquishing VGA console. >> (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to >> switch input to Xen) >> (XEN) Freed 220kB init memory. >> (XEN) traps.c:2432:d0 Domain attempted WRMSR 00000000c0010201 from >> 0x0000000000000000 to 0x000000000000abcd. >> (XEN) physdev.c:155: dom0: wrong map_pirq type 3 >> >> >> BORKED: >> (XEN) Xen version 4.2.1 (Ubuntu 4.2.1-0ubuntu3.1) >> (stefan.bader@canonical.com) (gcc (Ubuntu/Linaro 4.7.3-1ubuntu1) >> 4.7.3) Mon Apr 29 19:35:31 UTC 2013 >> (XEN) Bootloader: GRUB 2.00-13ubuntu3 >> (XEN) Command line: iommu=no-amd-iommu-perdev-intremap x2apic=off >> xen-pciback.hide=(06:00.0)(06.00.1) >> (XEN) Video information: >> (XEN) VGA is text mode 80x25, font 8x16 >> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >> (XEN) Disc information: >> (XEN) Found 7 MBR signatures >> (XEN) Found 6 EDD information structures >> (XEN) Xen-e820 RAM map: >> (XEN) 0000000000000000 - 000000000009e800 (usable) >> (XEN) 000000000009e800 - 00000000000a0000 (reserved) >> (XEN) 00000000000e0000 - 0000000000100000 (reserved) >> (XEN) 0000000000100000 - 00000000ba7ac000 (usable) >> (XEN) 00000000ba7ac000 - 00000000babe0000 (reserved) >> (XEN) 00000000babe0000 - 00000000babf0000 (ACPI data) >> (XEN) 00000000babf0000 - 00000000bb958000 (ACPI NVS) >> (XEN) 00000000bb958000 - 00000000bca35000 (reserved) >> (XEN) 00000000bca35000 - 00000000bca36000 (usable) >> (XEN) 00000000bca36000 - 00000000bcc3c000 (ACPI NVS) >> (XEN) 00000000bcc3c000 - 00000000bd083000 (usable) >> (XEN) 00000000bd083000 - 00000000bd7f4000 (reserved) >> (XEN) 00000000bd7f4000 - 00000000bd800000 (usable) >> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >> (XEN) 00000000fec00000 - 00000000fec01000 (reserved) >> (XEN) 00000000fec10000 - 00000000fec11000 (reserved) >> (XEN) 00000000fec20000 - 00000000fec21000 (reserved) >> (XEN) 00000000fed00000 - 00000000fed01000 (reserved) >> (XEN) 00000000fed61000 - 00000000fed71000 (reserved) >> (XEN) 00000000fed80000 - 00000000fed90000 (reserved) >> (XEN) 00000000fef00000 - 0000000100000000 (reserved) >> (XEN) 0000000100001000 - 000000043f000000 (usable) >> (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA) >> (XEN) ACPI: XSDT BABE7078, 0064 (r1 ALASKA A M I 1072009 AMI >> 10013) >> (XEN) ACPI: FACP BABEE118, 010C (r5 ALASKA A M I 1072009 AMI >> 10013) >> (XEN) ACPI Warning (tbfadt-0232): FADT (revision 5) is longer than >> ACPI 2.0 version, truncating length 0x10C to 0xF4 [20070126] >> (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has >> zero address or length: 0000000000000000/1 [20070126] >> (XEN) ACPI: DSDT BABE7170, 6FA8 (r2 ALASKA A M I 0 INTL >> 20051117) >> (XEN) ACPI: FACS BB952F80, 0040 >> (XEN) ACPI: APIC BABEE228, 009E (r3 ALASKA A M I 1072009 AMI >> 10013) >> (XEN) ACPI: FPDT BABEE2C8, 0044 (r1 ALASKA A M I 1072009 AMI >> 10013) >> (XEN) ACPI: MCFG BABEE310, 003C (r1 ALASKA A M I 1072009 MSFT >> 10013) >> (XEN) ACPI: HPET BABEE350, 0038 (r1 ALASKA A M I 1072009 AMI >> 5) >> (XEN) ACPI: BGRT BABEFBF8, 0038 (r0 ALASKA A M I 1072009 AMI >> 10013) >> (XEN) ACPI: IVRS BABEE3E0, 0100 (r1 AMD RD890S 202031 AMD >> 0) >> (XEN) ACPI: SSDT BABEE4E0, 1714 (r1 AMD POWERNOW 1 AMD >> 1) >> (XEN) System RAM: 16283MB (16674420kB) >> (XEN) Domain heap initialised >> (XEN) ACPI: 32/64X FACS address mismatch in FADT - >> bb952f80/0000000000000000, using 32 >> (XEN) Processor #16 5:1 APIC version 16 >> (XEN) Processor #17 5:1 APIC version 16 >> (XEN) Processor #18 5:1 APIC version 16 >> (XEN) Processor #19 5:1 APIC version 16 >> (XEN) Processor #20 5:1 APIC version 16 >> (XEN) Processor #21 5:1 APIC version 16 >> (XEN) Processor #22 5:1 APIC version 16 >> (XEN) Processor #23 5:1 APIC version 16 >> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 >> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 >> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs >> (XEN) Table is not found! >> (XEN) Using scheduler: SMP Credit Scheduler (credit) >> (XEN) Detected 3110.540 MHz processor. >> (XEN) Initing memory sharing. >> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007 >> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff >> (XEN) IVHD Error: Invalid IO-APIC 0xff >> (XEN) AMD-Vi: Error initialization >> (XEN) I/O virtualisation disabled >> (XEN) ENABLING IO-APIC IRQs >> (XEN) -> Using new ACK method >> (XEN) Platform timer is 14.318MHz HPET >> (XEN) Allocated console ring of 16 KiB. >> (XEN) HVM: ASIDs enabled. >> (XEN) SVM: Supported advanced features: >> (XEN) - Nested Page Tables (NPT) >> (XEN) - Last Branch Record (LBR) Virtualisation >> (XEN) - Next-RIP Saved on #VMEXIT >> (XEN) - VMCB Clean Bits >> (XEN) - DecodeAssists >> (XEN) - Pause-Intercept Filter >> (XEN) - TSC Rate MSR >> (XEN) HVM: SVM enabled >> (XEN) HVM: Hardware Assisted Paging (HAP) detected >> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB >> (XEN) Brought up 8 CPUs >> (XEN) mtrr: your CPUs had inconsistent variable MTRR settings >> (XEN) *** LOADING DOMAIN 0 *** >> (XEN) Xen kernel: 64-bit, lsb, compat32 >> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2346000 >> (XEN) PHYSICAL MEMORY ARRANGEMENT: >> (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4035583 >> pages to be allocated) >> (XEN) Init. ramdisk: 0000000439dc6000->000000043efff800 >> (XEN) VIRTUAL MEMORY ARRANGEMENT: >> (XEN) Loaded kernel: ffffffff81000000->ffffffff82346000 >> (XEN) Init. ramdisk: ffffffff82346000->ffffffff8757f800 >> (XEN) Phys-Mach map: ffffffff87580000->ffffffff894b31c8 >> (XEN) Start info: ffffffff894b4000->ffffffff894b44b4 >> (XEN) Page tables: ffffffff894b5000->ffffffff89504000 >> (XEN) Boot stack: ffffffff89504000->ffffffff89505000 >> (XEN) TOTAL: ffffffff80000000->ffffffff89800000 >> (XEN) ENTRY ADDRESS: ffffffff81d06210 >> (XEN) Dom0 has maximum 8 VCPUs >> (XEN) Scrubbing Free RAM: .done. >> (XEN) Initial low memory virq threshold set at 0x4000 pages. >> (XEN) Std. Loglevel: Errors and warnings >> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) >> (XEN) Xen is relinquishing VGA console. >> (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to >> switch input to Xen) >> (XEN) Freed 244kB init memory. >> >> _______________________________________________ >> Xen-users mailing list >> Xen-users@lists.xen.org >> http://lists.xen.org/xen-users >> > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Hi guys, i''m also experiencing this problem with M5A97 EVO R2.0. It has been "solved" in Xen 4.2.2 (discused here http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html). But i agree on that that it should be repaired on ASUS side. And I''m also facing with prolem about Adaptec 3805 Raid card with enabled IOMMU on this motherbord. Without IOMMU storage adapter works perfectly, but with enabled IOMMU in BIOS i see the device under linux as block device but it''s unusable - no data can read or written. Does anyone know what to do here, please? Could it be related with bad IOMMU implementation? Thanks in advance Jan Dne 28.5.2013 12:11, Gizmo Chicken napsal(a):> David and feral, > > I have an Asus M5A99FX Pro 2.0 motherboard and, at least as far as I > can tell, IOMMU works well with that motherboard and Xen 4.1. (I got > VGA passthrough working with a Windows guest.) Although I did some > preliminary testing with Xen 4.2 on that motherboard, because of some > other issues, I never got around to creating a VM that required IOMMU > support, and so can''t say for certain whether IOMMU is working with > that motherboard and Xen 4.2. > > Apart from creating a VM that requires IOMMU support (which would take > more time than I''d prefer to spend right now) to see if it works, what > would good way to determine whether my Asus M5A99FX Pro 2.0 > motherboard suffers, or if free from, the IVRS table issue that you > describe? > > Thanks much for any help with this! > > Best regards, > GizmoChicken > > > On Tue, May 28, 2013 at 3:31 AM, David Sutton <kantras@gmail.com > <mailto:kantras@gmail.com>> wrote: > > Feral, > > I have the R1.0 version of that motherboard. The problem is with > the BIOS; its returning bad information in the IVRS table for the > IO-APICs, which the new parser is catching and causing AMD-Vi > initialization to fail. I''ve already put in a support ticket via > the online form, included all the information such as links to the > document which shows what the valid details should be. The ticket > was basically closed with a note saying to look out for BIOS > updates. I''m currently using 4.2.1 (which was before the table > checking was added) and am able to use AMD-Vi. If you want to fix > this, you''re either going to have to get ASUS to issue a fixed > BIOS (which doesn''t feel likely at this point) or you''d have to > either disable interrupt remapping or edit the source code to > disable that check (and deal with any potential issue that might > cause later) > > Regards, > > David > > > On Mon, May 27, 2013 at 11:42 PM, feral <blistovmhz@gmail.com > <mailto:blistovmhz@gmail.com>> wrote: > > This issue has been discussed many times before, but I haven''t > found > an answer in my specific problem. > > Motherboard: Asus Sabertooth 990fx R2.0 > Linux 3.2.0-35 through 3.8.x (tested 5 kernels to be > ridiculously thorough) > > Affected: Xen-hypervisor >4.1.2 > Not Affected: Xen-hypervisor <4.1.2 > > > Xen 4.1.2 and earlier works fine regardless of the BIOS > version of the > Mobo. After upgrading Xen hypervisor beyond 4.1.2, we get the > following: > (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, > GSI 0-23 > (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, > GSI 24-55 > (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs > (XEN) Table is not found! > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Detected 3110.540 MHz processor. > (XEN) Initing memory sharing. > (XEN) xstate_init: using cntxt_size: 0x3c0 and states: > 0x4000000000000007 > (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff > (XEN) IVHD Error: Invalid IO-APIC 0xff > (XEN) AMD-Vi: Error initialization > (XEN) I/O virtualisation disabled > > > From what I understand, Xen used to just care less about the > state of > your ACPI/IVRS but that this was potentially a security > concern and > checks were implemented to disable IOMMU whenever it would be > unsafe > (potentially) to have it enabled. > Now, I understand the logic here, but I disagree with the > implementation. There are a fairly large number of people who are > affected and we know Asus isn''t going to fix the IVRS table. I > recognize the importance of implementing these checks and > hell, even > automatically disabling IOMMU without notice (though on a consumer > board, this is a bit paranoid). > I understand there are kernel command line options you can pass to > forcibly bypass the checks and leave IOMMU enabled in most > cases (ie: > iommu=no-amd-iommu-perdev-intremap) but thus far none of them > seem to > be enough to force IOMMU back on. > I read somewhere in one of the many discussions on the > subject, that > there are cases where you simply aren''t allowed to disable the > checks > (I believe this was when the IVRS table indicies don''t line up or > something, which seems to be my issue). > > As I said, I know other people have precisely this problem with a > whole line of mid range Asus boards, but haven''t seen a lot of > discussion on the list. Anyone reading this run into the same > problem? I bought this board specifically because of it''s IOMMU > implementation, which worked fine... at the time... > Ultimately I know > for a fact that the boards ACPI/IVRS can''t be TOO wrong as > it''s been > working for 6 months without an issue. I''m just sorta in a > situation > where it''s time to upgrade my base OS and hacking Xen 4.1.2 in > seems > like a silly amount of work when all I really need is an option to > disable the checks. > > > xm dmesg from a working vs. borked system > Working: > (XEN) Xen version 4.1.2 (Ubuntu 4.1.2-2ubuntu2) > (stefan.bader@canonical.com > <mailto:stefan.bader@canonical.com>) (gcc version 4.6.3 > (Ubuntu/Linaro > 4.6.3-1ubuntu2) ) Thu Mar 8 14:31:11 UTC 2012 > (XEN) Bootloader: GRUB 1.99-21ubuntu3.9 > (XEN) Command line: pci_msitranslate=0 xen-pciback=passthrough > xen-pciback.hide=(06:00.0)(06.00.1) > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 6 MBR signatures > (XEN) Found 6 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009e800 (usable) > (XEN) 000000000009e800 - 00000000000a0000 (reserved) > (XEN) 00000000000e0000 - 0000000000100000 (reserved) > (XEN) 0000000000100000 - 00000000bc348000 (usable) > (XEN) 00000000bc348000 - 00000000bc78c000 (reserved) > (XEN) 00000000bc78c000 - 00000000bc797000 (ACPI data) > (XEN) 00000000bc797000 - 00000000bd4f8000 (ACPI NVS) > (XEN) 00000000bd4f8000 - 00000000bd8d5000 (reserved) > (XEN) 00000000bd8d5000 - 00000000bd8d6000 (usable) > (XEN) 00000000bd8d6000 - 00000000bdad9000 (ACPI NVS) > (XEN) 00000000bdad9000 - 00000000bdf00000 (usable) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec01000 (reserved) > (XEN) 00000000fec10000 - 00000000fec11000 (reserved) > (XEN) 00000000fec20000 - 00000000fec21000 (reserved) > (XEN) 00000000fed00000 - 00000000fed01000 (reserved) > (XEN) 00000000fed61000 - 00000000fed71000 (reserved) > (XEN) 00000000fed80000 - 00000000fed90000 (reserved) > (XEN) 00000000fef00000 - 0000000100000000 (reserved) > (XEN) 0000000100001000 - 000000043f000000 (usable) > (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA) > (XEN) ACPI: XSDT BC78E078, 0064 (r1 ALASKA A M I 1072009 > AMI 10013) > (XEN) ACPI: FACP BC795BA8, 00F4 (r4 ALASKA A M I 1072009 > AMI 10013) > (XEN) ACPI Warning (tbfadt-0444): Optional field > "Pm2ControlBlock" has > zero address or length: 0000000000000000/1 [20070126] > (XEN) ACPI: DSDT BC78E170, 7A38 (r2 ALASKA A M I 0 > INTL 20051117) > (XEN) ACPI: FACS BD4F2F80, 0040 > (XEN) ACPI: APIC BC795CA0, 009E (r3 ALASKA A M I 1072009 > AMI 10013) > (XEN) ACPI: FPDT BC795D40, 0044 (r1 ALASKA A M I 1072009 > AMI 10013) > (XEN) ACPI: MCFG BC795D88, 003C (r1 ALASKA A M I 1072009 > MSFT 10013) > (XEN) ACPI: HPET BC795DC8, 0038 (r1 ALASKA A M I 1072009 > AMI 5) > (XEN) ACPI: BGRT BC796260, 0038 (r0 ALASKA A M I 1072009 > AMI 10013) > (XEN) ACPI: IVRS BC795E58, 0100 (r1 AMD RD890S 202031 AMD > 0) > (XEN) ACPI: SSDT BC795F58, 0304 (r1 AMD POWERNOW 1 AMD > 1) > (XEN) System RAM: 16311MB (16702516kB) > (XEN) Domain heap initialised > (XEN) ACPI: 32/64X FACS address mismatch in FADT - > bd4f2f80/0000000000000000, using 32 > (XEN) Processor #16 5:1 APIC version 16 > (XEN) Processor #17 5:1 APIC version 16 > (XEN) Processor #18 5:1 APIC version 16 > (XEN) Processor #19 5:1 APIC version 16 > (XEN) Processor #20 5:1 APIC version 16 > (XEN) Processor #21 5:1 APIC version 16 > (XEN) Processor #22 5:1 APIC version 16 > (XEN) Processor #23 5:1 APIC version 16 > (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, > GSI 0-23 > (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, > GSI 24-55 > (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs > (XEN) Table is not found! > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Detected 3110.553 MHz processor. > (XEN) Initing memory sharing. > (XEN) AMD-Vi: IOMMU 0 Enabled. > (XEN) I/O virtualisation enabled > (XEN) - Dom0 mode: Relaxed > (XEN) ENABLING IO-APIC IRQs > (XEN) -> Using new ACK method > (XEN) Platform timer is 14.318MHz HPET > (XEN) Allocated console ring of 16 KiB. > (XEN) HVM: ASIDs enabled. > (XEN) SVM: Supported advanced features: > (XEN) - Nested Page Tables (NPT) > (XEN) - Last Branch Record (LBR) Virtualisation > (XEN) - Next-RIP Saved on #VMEXIT > (XEN) - VMCB Clean Bits > (XEN) - Pause-Intercept Filter > (XEN) HVM: SVM enabled > (XEN) HVM: Hardware Assisted Paging detected. > (XEN) Brought up 8 CPUs > (XEN) *** LOADING DOMAIN 0 *** > (XEN) Xen kernel: 64-bit, lsb, compat32 > (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x205f000 > (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4044532 > pages to be allocated) > (XEN) Init. ramdisk: 000000043c7dd000->000000043efffa00 > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: ffffffff81000000->ffffffff8205f000 > (XEN) Init. ramdisk: ffffffff8205f000->ffffffff84881a00 > (XEN) Phys-Mach map: ffffffff84882000->ffffffff867b18b8 > (XEN) Start info: ffffffff867b2000->ffffffff867b24b4 > (XEN) Page tables: ffffffff867b3000->ffffffff867ec000 > (XEN) Boot stack: ffffffff867ec000->ffffffff867ed000 > (XEN) TOTAL: ffffffff80000000->ffffffff86c00000 > (XEN) ENTRY ADDRESS: ffffffff81cfb200 > (XEN) Dom0 has maximum 8 VCPUs > (XEN) Scrubbing Free RAM: .done. > (XEN) Xen trace buffers: disabled > (XEN) Std. Loglevel: Errors and warnings > (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) > (XEN) Xen is relinquishing VGA console. > (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to > switch input to Xen) > (XEN) Freed 220kB init memory. > (XEN) traps.c:2432:d0 Domain attempted WRMSR 00000000c0010201 from > 0x0000000000000000 to 0x000000000000abcd. > (XEN) physdev.c:155: dom0: wrong map_pirq type 3 > > > BORKED: > (XEN) Xen version 4.2.1 (Ubuntu 4.2.1-0ubuntu3.1) > (stefan.bader@canonical.com > <mailto:stefan.bader@canonical.com>) (gcc (Ubuntu/Linaro > 4.7.3-1ubuntu1) > 4.7.3) Mon Apr 29 19:35:31 UTC 2013 > (XEN) Bootloader: GRUB 2.00-13ubuntu3 > (XEN) Command line: iommu=no-amd-iommu-perdev-intremap x2apic=off > xen-pciback.hide=(06:00.0)(06.00.1) > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 7 MBR signatures > (XEN) Found 6 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009e800 (usable) > (XEN) 000000000009e800 - 00000000000a0000 (reserved) > (XEN) 00000000000e0000 - 0000000000100000 (reserved) > (XEN) 0000000000100000 - 00000000ba7ac000 (usable) > (XEN) 00000000ba7ac000 - 00000000babe0000 (reserved) > (XEN) 00000000babe0000 - 00000000babf0000 (ACPI data) > (XEN) 00000000babf0000 - 00000000bb958000 (ACPI NVS) > (XEN) 00000000bb958000 - 00000000bca35000 (reserved) > (XEN) 00000000bca35000 - 00000000bca36000 (usable) > (XEN) 00000000bca36000 - 00000000bcc3c000 (ACPI NVS) > (XEN) 00000000bcc3c000 - 00000000bd083000 (usable) > (XEN) 00000000bd083000 - 00000000bd7f4000 (reserved) > (XEN) 00000000bd7f4000 - 00000000bd800000 (usable) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec01000 (reserved) > (XEN) 00000000fec10000 - 00000000fec11000 (reserved) > (XEN) 00000000fec20000 - 00000000fec21000 (reserved) > (XEN) 00000000fed00000 - 00000000fed01000 (reserved) > (XEN) 00000000fed61000 - 00000000fed71000 (reserved) > (XEN) 00000000fed80000 - 00000000fed90000 (reserved) > (XEN) 00000000fef00000 - 0000000100000000 (reserved) > (XEN) 0000000100001000 - 000000043f000000 (usable) > (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA) > (XEN) ACPI: XSDT BABE7078, 0064 (r1 ALASKA A M I 1072009 > AMI 10013) > (XEN) ACPI: FACP BABEE118, 010C (r5 ALASKA A M I 1072009 > AMI 10013) > (XEN) ACPI Warning (tbfadt-0232): FADT (revision 5) is longer than > ACPI 2.0 version, truncating length 0x10C to 0xF4 [20070126] > (XEN) ACPI Warning (tbfadt-0444): Optional field > "Pm2ControlBlock" has > zero address or length: 0000000000000000/1 [20070126] > (XEN) ACPI: DSDT BABE7170, 6FA8 (r2 ALASKA A M I 0 > INTL 20051117) > (XEN) ACPI: FACS BB952F80, 0040 > (XEN) ACPI: APIC BABEE228, 009E (r3 ALASKA A M I 1072009 > AMI 10013) > (XEN) ACPI: FPDT BABEE2C8, 0044 (r1 ALASKA A M I 1072009 > AMI 10013) > (XEN) ACPI: MCFG BABEE310, 003C (r1 ALASKA A M I 1072009 > MSFT 10013) > (XEN) ACPI: HPET BABEE350, 0038 (r1 ALASKA A M I 1072009 > AMI 5) > (XEN) ACPI: BGRT BABEFBF8, 0038 (r0 ALASKA A M I 1072009 > AMI 10013) > (XEN) ACPI: IVRS BABEE3E0, 0100 (r1 AMD RD890S 202031 AMD > 0) > (XEN) ACPI: SSDT BABEE4E0, 1714 (r1 AMD POWERNOW 1 AMD > 1) > (XEN) System RAM: 16283MB (16674420kB) > (XEN) Domain heap initialised > (XEN) ACPI: 32/64X FACS address mismatch in FADT - > bb952f80/0000000000000000, using 32 > (XEN) Processor #16 5:1 APIC version 16 > (XEN) Processor #17 5:1 APIC version 16 > (XEN) Processor #18 5:1 APIC version 16 > (XEN) Processor #19 5:1 APIC version 16 > (XEN) Processor #20 5:1 APIC version 16 > (XEN) Processor #21 5:1 APIC version 16 > (XEN) Processor #22 5:1 APIC version 16 > (XEN) Processor #23 5:1 APIC version 16 > (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, > GSI 0-23 > (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, > GSI 24-55 > (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs > (XEN) Table is not found! > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Detected 3110.540 MHz processor. > (XEN) Initing memory sharing. > (XEN) xstate_init: using cntxt_size: 0x3c0 and states: > 0x4000000000000007 > (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff > (XEN) IVHD Error: Invalid IO-APIC 0xff > (XEN) AMD-Vi: Error initialization > (XEN) I/O virtualisation disabled > (XEN) ENABLING IO-APIC IRQs > (XEN) -> Using new ACK method > (XEN) Platform timer is 14.318MHz HPET > (XEN) Allocated console ring of 16 KiB. > (XEN) HVM: ASIDs enabled. > (XEN) SVM: Supported advanced features: > (XEN) - Nested Page Tables (NPT) > (XEN) - Last Branch Record (LBR) Virtualisation > (XEN) - Next-RIP Saved on #VMEXIT > (XEN) - VMCB Clean Bits > (XEN) - DecodeAssists > (XEN) - Pause-Intercept Filter > (XEN) - TSC Rate MSR > (XEN) HVM: SVM enabled > (XEN) HVM: Hardware Assisted Paging (HAP) detected > (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB > (XEN) Brought up 8 CPUs > (XEN) mtrr: your CPUs had inconsistent variable MTRR settings > (XEN) *** LOADING DOMAIN 0 *** > (XEN) Xen kernel: 64-bit, lsb, compat32 > (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2346000 > (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4035583 > pages to be allocated) > (XEN) Init. ramdisk: 0000000439dc6000->000000043efff800 > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: ffffffff81000000->ffffffff82346000 > (XEN) Init. ramdisk: ffffffff82346000->ffffffff8757f800 > (XEN) Phys-Mach map: ffffffff87580000->ffffffff894b31c8 > (XEN) Start info: ffffffff894b4000->ffffffff894b44b4 > (XEN) Page tables: ffffffff894b5000->ffffffff89504000 > (XEN) Boot stack: ffffffff89504000->ffffffff89505000 > (XEN) TOTAL: ffffffff80000000->ffffffff89800000 > (XEN) ENTRY ADDRESS: ffffffff81d06210 > (XEN) Dom0 has maximum 8 VCPUs > (XEN) Scrubbing Free RAM: .done. > (XEN) Initial low memory virq threshold set at 0x4000 pages. > (XEN) Std. Loglevel: Errors and warnings > (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) > (XEN) Xen is relinquishing VGA console. > (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to > switch input to Xen) > (XEN) Freed 244kB init memory. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org <mailto:Xen-users@lists.xen.org> > http://lists.xen.org/xen-users > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org <mailto:Xen-users@lists.xen.org> > http://lists.xen.org/xen-users > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
David Sutton
2013-May-28 14:07 UTC
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah
GizmoChicken, To debug the issue when I first ran across it, I made sure I added "iommu=debug,verbose" onto the boot options in my grub.conf file. I then used "xl dmesg" to see the log when booting up - You''d be looking for something like: -- (XEN) AMD-Vi: Found MSI capability block at 0x54 (XEN) AMD-Vi: ACPI Table: (XEN) AMD-Vi: Signature IVRS (XEN) AMD-Vi: Length 0xf0 (XEN) AMD-Vi: Revision 0x1 (XEN) AMD-Vi: CheckSum 0x6e (XEN) AMD-Vi: OEM_Id AMD (XEN) AMD-Vi: OEM_Table_Id RD890S (XEN) AMD-Vi: OEM_Revision 0x202031 (XEN) AMD-Vi: Creator_Id AMD (XEN) AMD-Vi: Creator_Revision 0x0 (XEN) AMD-Vi: IVRS Block: type 0x10 flags 0x3e len 0xc0 id 0x2 (XEN) AMD-Vi: IVHD Device Entry: type 0x3 id 0x0 flags 0x0 (XEN) AMD-Vi: Dev_Id Range: 0x0 -> 0x2 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x10 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x3 id 0x100 flags 0x0 (XEN) AMD-Vi: Dev_Id Range: 0x100 -> 0x101 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x20 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x200 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x28 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x300 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x30 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x400 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x48 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x500 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x58 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x3 id 0x600 flags 0x0 (XEN) AMD-Vi: Dev_Id Range: 0x600 -> 0x601 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x88 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x3 id 0x90 flags 0x0 (XEN) AMD-Vi: Dev_Id Range: 0x90 -> 0x92 (XEN) AMD-Vi: IVHD Device Entry: type 0x3 id 0x98 flags 0x0 (XEN) AMD-Vi: Dev_Id Range: 0x98 -> 0x9a (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa0 flags 0xd7 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa2 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa3 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa4 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x0 id 0x0 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x43 id 0x700 flags 0x0 (XEN) AMD-Vi: Dev_Id Range: 0x700 -> 0x7ff alias 0xa4 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa5 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa8 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0xa9 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x2 id 0x900 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x3 id 0xb0 flags 0x0 (XEN) AMD-Vi: Dev_Id Range: 0xb0 -> 0xb2 (XEN) AMD-Vi: IVHD Device Entry: type 0x0 id 0x0 flags 0x0 (XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0x0 flags 0xd7 (XEN) AMD-Vi: IVHD Special: 0000:00:14.0 variety 0x1 handle 0x0 (XEN) IVHD Error: Invalid IO-APIC 0x0 (XEN) AMD-Vi: Error initialization -- In my case, the handle id is wrong ( you can see what it should be on the support document http://support.amd.com/us/Processor_TechDocs/48882.pdf - per tables 77/79 it should be the ID of the IO-APIC). Unlike the conflict error, which can be worked around by passing another command line option, invalid IO-APIC entries in the IVRS table will always cause AMD-Vi to fail to be enabled if you have the XSA-36 patch installed. From the notes in the patch for 4.2: -- In addition, because some BIOSes may incorrectly program IVRS entries for IOAPIC try to check for entry''s consistency. Specifically, if conflicting entries are found disable IOMMU if per-device remapping table is used. If entries refer to bogus IOAPIC IDs disable IOMMU unconditionally -- Regards, David On Tue, May 28, 2013 at 5:11 AM, Gizmo Chicken <gizmochicken@gmail.com>wrote:> David and feral, > > I have an Asus M5A99FX Pro 2.0 motherboard and, at least as far as I can > tell, IOMMU works well with that motherboard and Xen 4.1. (I got VGA > passthrough working with a Windows guest.) Although I did some > preliminary testing with Xen 4.2 on that motherboard, because of some other > issues, I never got around to creating a VM that required IOMMU support, > and so can''t say for certain whether IOMMU is working with that motherboard > and Xen 4.2. > > Apart from creating a VM that requires IOMMU support (which would take > more time than I''d prefer to spend right now) to see if it works, what > would good way to determine whether my Asus M5A99FX Pro 2.0 motherboard > suffers, or if free from, the IVRS table issue that you describe? > > Thanks much for any help with this! > > Best regards, > GizmoChicken > > > On Tue, May 28, 2013 at 3:31 AM, David Sutton <kantras@gmail.com> wrote: > >> Feral, >> >> I have the R1.0 version of that motherboard. The problem is with the >> BIOS; its returning bad information in the IVRS table for the IO-APICs, >> which the new parser is catching and causing AMD-Vi initialization to fail. >> I''ve already put in a support ticket via the online form, included all the >> information such as links to the document which shows what the valid >> details should be. The ticket was basically closed with a note saying to >> look out for BIOS updates. I''m currently using 4.2.1 (which was before the >> table checking was added) and am able to use AMD-Vi. If you want to fix >> this, you''re either going to have to get ASUS to issue a fixed BIOS (which >> doesn''t feel likely at this point) or you''d have to either disable >> interrupt remapping or edit the source code to disable that check (and deal >> with any potential issue that might cause later) >> >> Regards, >> >> David >> >> >> On Mon, May 27, 2013 at 11:42 PM, feral <blistovmhz@gmail.com> wrote: >> >>> This issue has been discussed many times before, but I haven''t found >>> an answer in my specific problem. >>> >>> Motherboard: Asus Sabertooth 990fx R2.0 >>> Linux 3.2.0-35 through 3.8.x (tested 5 kernels to be ridiculously >>> thorough) >>> >>> Affected: Xen-hypervisor >4.1.2 >>> Not Affected: Xen-hypervisor <4.1.2 >>> >>> >>> Xen 4.1.2 and earlier works fine regardless of the BIOS version of the >>> Mobo. After upgrading Xen hypervisor beyond 4.1.2, we get the >>> following: >>> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 >>> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 >>> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs >>> (XEN) Table is not found! >>> (XEN) Using scheduler: SMP Credit Scheduler (credit) >>> (XEN) Detected 3110.540 MHz processor. >>> (XEN) Initing memory sharing. >>> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007 >>> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff >>> (XEN) IVHD Error: Invalid IO-APIC 0xff >>> (XEN) AMD-Vi: Error initialization >>> (XEN) I/O virtualisation disabled >>> >>> >>> From what I understand, Xen used to just care less about the state of >>> your ACPI/IVRS but that this was potentially a security concern and >>> checks were implemented to disable IOMMU whenever it would be unsafe >>> (potentially) to have it enabled. >>> Now, I understand the logic here, but I disagree with the >>> implementation. There are a fairly large number of people who are >>> affected and we know Asus isn''t going to fix the IVRS table. I >>> recognize the importance of implementing these checks and hell, even >>> automatically disabling IOMMU without notice (though on a consumer >>> board, this is a bit paranoid). >>> I understand there are kernel command line options you can pass to >>> forcibly bypass the checks and leave IOMMU enabled in most cases (ie: >>> iommu=no-amd-iommu-perdev-intremap) but thus far none of them seem to >>> be enough to force IOMMU back on. >>> I read somewhere in one of the many discussions on the subject, that >>> there are cases where you simply aren''t allowed to disable the checks >>> (I believe this was when the IVRS table indicies don''t line up or >>> something, which seems to be my issue). >>> >>> As I said, I know other people have precisely this problem with a >>> whole line of mid range Asus boards, but haven''t seen a lot of >>> discussion on the list. Anyone reading this run into the same >>> problem? I bought this board specifically because of it''s IOMMU >>> implementation, which worked fine... at the time... Ultimately I know >>> for a fact that the boards ACPI/IVRS can''t be TOO wrong as it''s been >>> working for 6 months without an issue. I''m just sorta in a situation >>> where it''s time to upgrade my base OS and hacking Xen 4.1.2 in seems >>> like a silly amount of work when all I really need is an option to >>> disable the checks. >>> >>> >>> xm dmesg from a working vs. borked system >>> Working: >>> (XEN) Xen version 4.1.2 (Ubuntu 4.1.2-2ubuntu2) >>> (stefan.bader@canonical.com) (gcc version 4.6.3 (Ubuntu/Linaro >>> 4.6.3-1ubuntu2) ) Thu Mar 8 14:31:11 UTC 2012 >>> (XEN) Bootloader: GRUB 1.99-21ubuntu3.9 >>> (XEN) Command line: pci_msitranslate=0 xen-pciback=passthrough >>> xen-pciback.hide=(06:00.0)(06.00.1) >>> (XEN) Video information: >>> (XEN) VGA is text mode 80x25, font 8x16 >>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >>> (XEN) Disc information: >>> (XEN) Found 6 MBR signatures >>> (XEN) Found 6 EDD information structures >>> (XEN) Xen-e820 RAM map: >>> (XEN) 0000000000000000 - 000000000009e800 (usable) >>> (XEN) 000000000009e800 - 00000000000a0000 (reserved) >>> (XEN) 00000000000e0000 - 0000000000100000 (reserved) >>> (XEN) 0000000000100000 - 00000000bc348000 (usable) >>> (XEN) 00000000bc348000 - 00000000bc78c000 (reserved) >>> (XEN) 00000000bc78c000 - 00000000bc797000 (ACPI data) >>> (XEN) 00000000bc797000 - 00000000bd4f8000 (ACPI NVS) >>> (XEN) 00000000bd4f8000 - 00000000bd8d5000 (reserved) >>> (XEN) 00000000bd8d5000 - 00000000bd8d6000 (usable) >>> (XEN) 00000000bd8d6000 - 00000000bdad9000 (ACPI NVS) >>> (XEN) 00000000bdad9000 - 00000000bdf00000 (usable) >>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >>> (XEN) 00000000fec00000 - 00000000fec01000 (reserved) >>> (XEN) 00000000fec10000 - 00000000fec11000 (reserved) >>> (XEN) 00000000fec20000 - 00000000fec21000 (reserved) >>> (XEN) 00000000fed00000 - 00000000fed01000 (reserved) >>> (XEN) 00000000fed61000 - 00000000fed71000 (reserved) >>> (XEN) 00000000fed80000 - 00000000fed90000 (reserved) >>> (XEN) 00000000fef00000 - 0000000100000000 (reserved) >>> (XEN) 0000000100001000 - 000000043f000000 (usable) >>> (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA) >>> (XEN) ACPI: XSDT BC78E078, 0064 (r1 ALASKA A M I 1072009 AMI >>> 10013) >>> (XEN) ACPI: FACP BC795BA8, 00F4 (r4 ALASKA A M I 1072009 AMI >>> 10013) >>> (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has >>> zero address or length: 0000000000000000/1 [20070126] >>> (XEN) ACPI: DSDT BC78E170, 7A38 (r2 ALASKA A M I 0 INTL >>> 20051117) >>> (XEN) ACPI: FACS BD4F2F80, 0040 >>> (XEN) ACPI: APIC BC795CA0, 009E (r3 ALASKA A M I 1072009 AMI >>> 10013) >>> (XEN) ACPI: FPDT BC795D40, 0044 (r1 ALASKA A M I 1072009 AMI >>> 10013) >>> (XEN) ACPI: MCFG BC795D88, 003C (r1 ALASKA A M I 1072009 MSFT >>> 10013) >>> (XEN) ACPI: HPET BC795DC8, 0038 (r1 ALASKA A M I 1072009 AMI >>> 5) >>> (XEN) ACPI: BGRT BC796260, 0038 (r0 ALASKA A M I 1072009 AMI >>> 10013) >>> (XEN) ACPI: IVRS BC795E58, 0100 (r1 AMD RD890S 202031 AMD >>> 0) >>> (XEN) ACPI: SSDT BC795F58, 0304 (r1 AMD POWERNOW 1 AMD >>> 1) >>> (XEN) System RAM: 16311MB (16702516kB) >>> (XEN) Domain heap initialised >>> (XEN) ACPI: 32/64X FACS address mismatch in FADT - >>> bd4f2f80/0000000000000000, using 32 >>> (XEN) Processor #16 5:1 APIC version 16 >>> (XEN) Processor #17 5:1 APIC version 16 >>> (XEN) Processor #18 5:1 APIC version 16 >>> (XEN) Processor #19 5:1 APIC version 16 >>> (XEN) Processor #20 5:1 APIC version 16 >>> (XEN) Processor #21 5:1 APIC version 16 >>> (XEN) Processor #22 5:1 APIC version 16 >>> (XEN) Processor #23 5:1 APIC version 16 >>> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 >>> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 >>> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs >>> (XEN) Table is not found! >>> (XEN) Using scheduler: SMP Credit Scheduler (credit) >>> (XEN) Detected 3110.553 MHz processor. >>> (XEN) Initing memory sharing. >>> (XEN) AMD-Vi: IOMMU 0 Enabled. >>> (XEN) I/O virtualisation enabled >>> (XEN) - Dom0 mode: Relaxed >>> (XEN) ENABLING IO-APIC IRQs >>> (XEN) -> Using new ACK method >>> (XEN) Platform timer is 14.318MHz HPET >>> (XEN) Allocated console ring of 16 KiB. >>> (XEN) HVM: ASIDs enabled. >>> (XEN) SVM: Supported advanced features: >>> (XEN) - Nested Page Tables (NPT) >>> (XEN) - Last Branch Record (LBR) Virtualisation >>> (XEN) - Next-RIP Saved on #VMEXIT >>> (XEN) - VMCB Clean Bits >>> (XEN) - Pause-Intercept Filter >>> (XEN) HVM: SVM enabled >>> (XEN) HVM: Hardware Assisted Paging detected. >>> (XEN) Brought up 8 CPUs >>> (XEN) *** LOADING DOMAIN 0 *** >>> (XEN) Xen kernel: 64-bit, lsb, compat32 >>> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x205f000 >>> (XEN) PHYSICAL MEMORY ARRANGEMENT: >>> (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4044532 >>> pages to be allocated) >>> (XEN) Init. ramdisk: 000000043c7dd000->000000043efffa00 >>> (XEN) VIRTUAL MEMORY ARRANGEMENT: >>> (XEN) Loaded kernel: ffffffff81000000->ffffffff8205f000 >>> (XEN) Init. ramdisk: ffffffff8205f000->ffffffff84881a00 >>> (XEN) Phys-Mach map: ffffffff84882000->ffffffff867b18b8 >>> (XEN) Start info: ffffffff867b2000->ffffffff867b24b4 >>> (XEN) Page tables: ffffffff867b3000->ffffffff867ec000 >>> (XEN) Boot stack: ffffffff867ec000->ffffffff867ed000 >>> (XEN) TOTAL: ffffffff80000000->ffffffff86c00000 >>> (XEN) ENTRY ADDRESS: ffffffff81cfb200 >>> (XEN) Dom0 has maximum 8 VCPUs >>> (XEN) Scrubbing Free RAM: .done. >>> (XEN) Xen trace buffers: disabled >>> (XEN) Std. Loglevel: Errors and warnings >>> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) >>> (XEN) Xen is relinquishing VGA console. >>> (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to >>> switch input to Xen) >>> (XEN) Freed 220kB init memory. >>> (XEN) traps.c:2432:d0 Domain attempted WRMSR 00000000c0010201 from >>> 0x0000000000000000 to 0x000000000000abcd. >>> (XEN) physdev.c:155: dom0: wrong map_pirq type 3 >>> >>> >>> BORKED: >>> (XEN) Xen version 4.2.1 (Ubuntu 4.2.1-0ubuntu3.1) >>> (stefan.bader@canonical.com) (gcc (Ubuntu/Linaro 4.7.3-1ubuntu1) >>> 4.7.3) Mon Apr 29 19:35:31 UTC 2013 >>> (XEN) Bootloader: GRUB 2.00-13ubuntu3 >>> (XEN) Command line: iommu=no-amd-iommu-perdev-intremap x2apic=off >>> xen-pciback.hide=(06:00.0)(06.00.1) >>> (XEN) Video information: >>> (XEN) VGA is text mode 80x25, font 8x16 >>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >>> (XEN) Disc information: >>> (XEN) Found 7 MBR signatures >>> (XEN) Found 6 EDD information structures >>> (XEN) Xen-e820 RAM map: >>> (XEN) 0000000000000000 - 000000000009e800 (usable) >>> (XEN) 000000000009e800 - 00000000000a0000 (reserved) >>> (XEN) 00000000000e0000 - 0000000000100000 (reserved) >>> (XEN) 0000000000100000 - 00000000ba7ac000 (usable) >>> (XEN) 00000000ba7ac000 - 00000000babe0000 (reserved) >>> (XEN) 00000000babe0000 - 00000000babf0000 (ACPI data) >>> (XEN) 00000000babf0000 - 00000000bb958000 (ACPI NVS) >>> (XEN) 00000000bb958000 - 00000000bca35000 (reserved) >>> (XEN) 00000000bca35000 - 00000000bca36000 (usable) >>> (XEN) 00000000bca36000 - 00000000bcc3c000 (ACPI NVS) >>> (XEN) 00000000bcc3c000 - 00000000bd083000 (usable) >>> (XEN) 00000000bd083000 - 00000000bd7f4000 (reserved) >>> (XEN) 00000000bd7f4000 - 00000000bd800000 (usable) >>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >>> (XEN) 00000000fec00000 - 00000000fec01000 (reserved) >>> (XEN) 00000000fec10000 - 00000000fec11000 (reserved) >>> (XEN) 00000000fec20000 - 00000000fec21000 (reserved) >>> (XEN) 00000000fed00000 - 00000000fed01000 (reserved) >>> (XEN) 00000000fed61000 - 00000000fed71000 (reserved) >>> (XEN) 00000000fed80000 - 00000000fed90000 (reserved) >>> (XEN) 00000000fef00000 - 0000000100000000 (reserved) >>> (XEN) 0000000100001000 - 000000043f000000 (usable) >>> (XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA) >>> (XEN) ACPI: XSDT BABE7078, 0064 (r1 ALASKA A M I 1072009 AMI >>> 10013) >>> (XEN) ACPI: FACP BABEE118, 010C (r5 ALASKA A M I 1072009 AMI >>> 10013) >>> (XEN) ACPI Warning (tbfadt-0232): FADT (revision 5) is longer than >>> ACPI 2.0 version, truncating length 0x10C to 0xF4 [20070126] >>> (XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has >>> zero address or length: 0000000000000000/1 [20070126] >>> (XEN) ACPI: DSDT BABE7170, 6FA8 (r2 ALASKA A M I 0 INTL >>> 20051117) >>> (XEN) ACPI: FACS BB952F80, 0040 >>> (XEN) ACPI: APIC BABEE228, 009E (r3 ALASKA A M I 1072009 AMI >>> 10013) >>> (XEN) ACPI: FPDT BABEE2C8, 0044 (r1 ALASKA A M I 1072009 AMI >>> 10013) >>> (XEN) ACPI: MCFG BABEE310, 003C (r1 ALASKA A M I 1072009 MSFT >>> 10013) >>> (XEN) ACPI: HPET BABEE350, 0038 (r1 ALASKA A M I 1072009 AMI >>> 5) >>> (XEN) ACPI: BGRT BABEFBF8, 0038 (r0 ALASKA A M I 1072009 AMI >>> 10013) >>> (XEN) ACPI: IVRS BABEE3E0, 0100 (r1 AMD RD890S 202031 AMD >>> 0) >>> (XEN) ACPI: SSDT BABEE4E0, 1714 (r1 AMD POWERNOW 1 AMD >>> 1) >>> (XEN) System RAM: 16283MB (16674420kB) >>> (XEN) Domain heap initialised >>> (XEN) ACPI: 32/64X FACS address mismatch in FADT - >>> bb952f80/0000000000000000, using 32 >>> (XEN) Processor #16 5:1 APIC version 16 >>> (XEN) Processor #17 5:1 APIC version 16 >>> (XEN) Processor #18 5:1 APIC version 16 >>> (XEN) Processor #19 5:1 APIC version 16 >>> (XEN) Processor #20 5:1 APIC version 16 >>> (XEN) Processor #21 5:1 APIC version 16 >>> (XEN) Processor #22 5:1 APIC version 16 >>> (XEN) Processor #23 5:1 APIC version 16 >>> (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 >>> (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 >>> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs >>> (XEN) Table is not found! >>> (XEN) Using scheduler: SMP Credit Scheduler (credit) >>> (XEN) Detected 3110.540 MHz processor. >>> (XEN) Initing memory sharing. >>> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007 >>> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff >>> (XEN) IVHD Error: Invalid IO-APIC 0xff >>> (XEN) AMD-Vi: Error initialization >>> (XEN) I/O virtualisation disabled >>> (XEN) ENABLING IO-APIC IRQs >>> (XEN) -> Using new ACK method >>> (XEN) Platform timer is 14.318MHz HPET >>> (XEN) Allocated console ring of 16 KiB. >>> (XEN) HVM: ASIDs enabled. >>> (XEN) SVM: Supported advanced features: >>> (XEN) - Nested Page Tables (NPT) >>> (XEN) - Last Branch Record (LBR) Virtualisation >>> (XEN) - Next-RIP Saved on #VMEXIT >>> (XEN) - VMCB Clean Bits >>> (XEN) - DecodeAssists >>> (XEN) - Pause-Intercept Filter >>> (XEN) - TSC Rate MSR >>> (XEN) HVM: SVM enabled >>> (XEN) HVM: Hardware Assisted Paging (HAP) detected >>> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB >>> (XEN) Brought up 8 CPUs >>> (XEN) mtrr: your CPUs had inconsistent variable MTRR settings >>> (XEN) *** LOADING DOMAIN 0 *** >>> (XEN) Xen kernel: 64-bit, lsb, compat32 >>> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2346000 >>> (XEN) PHYSICAL MEMORY ARRANGEMENT: >>> (XEN) Dom0 alloc.: 0000000420000000->0000000428000000 (4035583 >>> pages to be allocated) >>> (XEN) Init. ramdisk: 0000000439dc6000->000000043efff800 >>> (XEN) VIRTUAL MEMORY ARRANGEMENT: >>> (XEN) Loaded kernel: ffffffff81000000->ffffffff82346000 >>> (XEN) Init. ramdisk: ffffffff82346000->ffffffff8757f800 >>> (XEN) Phys-Mach map: ffffffff87580000->ffffffff894b31c8 >>> (XEN) Start info: ffffffff894b4000->ffffffff894b44b4 >>> (XEN) Page tables: ffffffff894b5000->ffffffff89504000 >>> (XEN) Boot stack: ffffffff89504000->ffffffff89505000 >>> (XEN) TOTAL: ffffffff80000000->ffffffff89800000 >>> (XEN) ENTRY ADDRESS: ffffffff81d06210 >>> (XEN) Dom0 has maximum 8 VCPUs >>> (XEN) Scrubbing Free RAM: .done. >>> (XEN) Initial low memory virq threshold set at 0x4000 pages. >>> (XEN) Std. Loglevel: Errors and warnings >>> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) >>> (XEN) Xen is relinquishing VGA console. >>> (XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to >>> switch input to Xen) >>> (XEN) Freed 244kB init memory. >>> >>> _______________________________________________ >>> Xen-users mailing list >>> Xen-users@lists.xen.org >>> http://lists.xen.org/xen-users >>> >> >> >> _______________________________________________ >> Xen-users mailing list >> Xen-users@lists.xen.org >> http://lists.xen.org/xen-users >> > >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
> In my case, the handle id is wrong ( you can see what it should be on the > support document http://support.amd.com/us/Processor_TechDocs/48882.pdf - > per tables 77/79 it should be the ID of the IO-APIC). Unlike the conflict > error, which can be worked around by passing another command line option, > invalid IO-APIC entries in the IVRS table will always cause AMD-Vi to fail > to be enabled if you have the XSA-36 patch installed.This is the part I have an angry nerd rage issue with. I''m pretty sure I fall into the latter category where there is no option to forcibly enable IOMMU. I''d understand if the patch wasn''t actually removing functionality but in our case, it seems the patch does just this even though the "bug" being addressed doesn''t concern me (and probably quite a few others) whatsoever. The machine in question is my gaming rig primarily, and gets a lot of use in QA, but nothing mission critical. I''d rather hobble along with a known bad IVRS table if it has no affect on my work (or games) than be locked out entirely :p. I''m still a little confused though as I''ve heard a few people say that this patch wasn''t introduced until Xen 4.2 but I saw the issue first pop up in Xen 4.1.2 and later. Am I even looking at the right bug? And what options are available to forcibly disable the checks other than the "iommu=no-amd-iommu-perdev-intremap" ?
On Tue, May 28, 2013 at 4:24 AM, Jan Hejl <jh@excello.cz> wrote:> Hi guys, > > i''m also experiencing this problem with M5A97 EVO R2.0. It has been "solved" > in Xen 4.2.2 (discused here > http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html). But > i agree on that that it should be repaired on ASUS side. > > And I''m also facing with prolem about Adaptec 3805 Raid card with enabled > IOMMU on this motherbord. Without IOMMU storage adapter works perfectly, but > with enabled IOMMU in BIOS i see the device under linux as block device but > it''s unusable - no data can read or written. Does anyone know what to do > here, please? Could it be related with bad IOMMU implementation? > > Thanks in advance > JanWhat am I missing :p ? What does "solved" in Xen 4.2.2 mean? The linked message discusses the original issue of the broken IVRS tables and the "fix" is to implement checks to disable IOMMU on buggy bios''s after version 4.1.3. In this case, the "fix" is the problem, unless I missed something?
Missing? May be some irony and sarcasm about that "solved". Sure this could be done better in Xen implementation, but primarly ASUS should be the first to take a care. These boards are not cheapest so i suppose that they should work properly. May be i''m wrong. 28. 5. 2013 v 18:56, feral <blistovmhz@gmail.com>:> On Tue, May 28, 2013 at 4:24 AM, Jan Hejl <jh@excello.cz> wrote: >> Hi guys, >> >> i''m also experiencing this problem with M5A97 EVO R2.0. It has been "solved" >> in Xen 4.2.2 (discused here >> http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html). But >> i agree on that that it should be repaired on ASUS side. >> >> And I''m also facing with prolem about Adaptec 3805 Raid card with enabled >> IOMMU on this motherbord. Without IOMMU storage adapter works perfectly, but >> with enabled IOMMU in BIOS i see the device under linux as block device but >> it''s unusable - no data can read or written. Does anyone know what to do >> here, please? Could it be related with bad IOMMU implementation? >> >> Thanks in advance >> Jan > > What am I missing :p ? What does "solved" in Xen 4.2.2 mean? The > linked message discusses the original issue of the broken IVRS tables > and the "fix" is to implement checks to disable IOMMU on buggy bios''s > after version 4.1.3. In this case, the "fix" is the problem, unless I > missed something?_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
I agree entirely about who should be taking responsibility, but we all know of course, that Asus will never fix this. There are probably less than 5000 people in the world affected by this, and if we assume most of them will still buy Asus''s next board that has working IVRS tables, Asus doesn''t care. I''m more interested in knowing why the "iommu=no-amd-iommu-perdev-intremap" option doesn''t cover my issue though. It seems odd to me that this option was implemented precisely to work around the regression issue, but somehow it didn''t catch them all. Goddamn I can''t wait for the Xen Documentation to take form :) On Tue, May 28, 2013 at 10:35 AM, Jan Hejl <jh@excello.cz> wrote:> Missing? May be some irony and sarcasm about that "solved". Sure this could be done better in Xen implementation, but primarly ASUS should be the first to take a care. These boards are not cheapest so i suppose that they should work properly. May be i''m wrong. > > 28. 5. 2013 v 18:56, feral <blistovmhz@gmail.com>: > >> On Tue, May 28, 2013 at 4:24 AM, Jan Hejl <jh@excello.cz> wrote: >>> Hi guys, >>> >>> i''m also experiencing this problem with M5A97 EVO R2.0. It has been "solved" >>> in Xen 4.2.2 (discused here >>> http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html). But >>> i agree on that that it should be repaired on ASUS side. >>> >>> And I''m also facing with prolem about Adaptec 3805 Raid card with enabled >>> IOMMU on this motherbord. Without IOMMU storage adapter works perfectly, but >>> with enabled IOMMU in BIOS i see the device under linux as block device but >>> it''s unusable - no data can read or written. Does anyone know what to do >>> here, please? Could it be related with bad IOMMU implementation? >>> >>> Thanks in advance >>> Jan >> >> What am I missing :p ? What does "solved" in Xen 4.2.2 mean? The >> linked message discusses the original issue of the broken IVRS tables >> and the "fix" is to implement checks to disable IOMMU on buggy bios''s >> after version 4.1.3. In this case, the "fix" is the problem, unless I >> missed something?-- _____ Fact: 1. Ninjas are mammals. 2. Ninjas fight ALL the time. 3. The purpose of the ninja is to flip out and kill people.
Also, I''m pretty much n00b when it comes to IVRS so I''m not entirely sure what I''m talking about. Meanwhile I''ve got my Asus rep trying to fix the issue. Does anyone have a technical explanation of what specifically, Asus has to do to fix this issue? On Tue, May 28, 2013 at 10:40 AM, feral <blistovmhz@gmail.com> wrote:> I agree entirely about who should be taking responsibility, but we all > know of course, that Asus will never fix this. There are probably > less than 5000 people in the world affected by this, and if we assume > most of them will still buy Asus''s next board that has working IVRS > tables, Asus doesn''t care. > > I''m more interested in knowing why the > "iommu=no-amd-iommu-perdev-intremap" option doesn''t cover my issue > though. It seems odd to me that this option was implemented precisely > to work around the regression issue, but somehow it didn''t catch them > all. Goddamn I can''t wait for the Xen Documentation to take form :) > > On Tue, May 28, 2013 at 10:35 AM, Jan Hejl <jh@excello.cz> wrote: >> Missing? May be some irony and sarcasm about that "solved". Sure this could be done better in Xen implementation, but primarly ASUS should be the first to take a care. These boards are not cheapest so i suppose that they should work properly. May be i''m wrong. >> >> 28. 5. 2013 v 18:56, feral <blistovmhz@gmail.com>: >> >>> On Tue, May 28, 2013 at 4:24 AM, Jan Hejl <jh@excello.cz> wrote: >>>> Hi guys, >>>> >>>> i''m also experiencing this problem with M5A97 EVO R2.0. It has been "solved" >>>> in Xen 4.2.2 (discused here >>>> http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html). But >>>> i agree on that that it should be repaired on ASUS side. >>>> >>>> And I''m also facing with prolem about Adaptec 3805 Raid card with enabled >>>> IOMMU on this motherbord. Without IOMMU storage adapter works perfectly, but >>>> with enabled IOMMU in BIOS i see the device under linux as block device but >>>> it''s unusable - no data can read or written. Does anyone know what to do >>>> here, please? Could it be related with bad IOMMU implementation? >>>> >>>> Thanks in advance >>>> Jan >>> >>> What am I missing :p ? What does "solved" in Xen 4.2.2 mean? The >>> linked message discusses the original issue of the broken IVRS tables >>> and the "fix" is to implement checks to disable IOMMU on buggy bios''s >>> after version 4.1.3. In this case, the "fix" is the problem, unless I >>> missed something? > > > > -- > _____ > Fact: > 1. Ninjas are mammals. > 2. Ninjas fight ALL the time. > 3. The purpose of the ninja is to flip out and kill people.-- _____ Fact: 1. Ninjas are mammals. 2. Ninjas fight ALL the time. 3. The purpose of the ninja is to flip out and kill people.
David Sutton
2013-May-28 18:20 UTC
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah
Feral, On Tue, May 28, 2013 at 11:49 AM, feral <blistovmhz@gmail.com> wrote:> > In my case, the handle id is wrong ( you can see what it should be on the > > support document http://support.amd.com/us/Processor_TechDocs/48882.pdf- > > per tables 77/79 it should be the ID of the IO-APIC). Unlike the conflict > > error, which can be worked around by passing another command line option, > > invalid IO-APIC entries in the IVRS table will always cause AMD-Vi to > fail > > to be enabled if you have the XSA-36 patch installed. > > This is the part I have an angry nerd rage issue with. I''m pretty > sure I fall into the latter category where there is no option to > forcibly enable IOMMU. I''d understand if the patch wasn''t actually > removing functionality but in our case, it seems the patch does just > this even though the "bug" being addressed doesn''t concern me (and > probably quite a few others) whatsoever. The machine in question is > my gaming rig primarily, and gets a lot of use in QA, but nothing > mission critical. I''d rather hobble along with a known bad IVRS table > if it has no affect on my work (or games) than be locked out entirely > :p. > > I''m still a little confused though as I''ve heard a few people say that > this patch wasn''t introduced until Xen 4.2 but I saw the issue first > pop up in Xen 4.1.2 and later. Am I even looking at the right bug? > And what options are available to forcibly disable the checks other > than the "iommu=no-amd-iommu-perdev-intremap" ? >The issue will occur with both 4.1 and 4.2 - there was a security advisory ( XSA-36 ), the patch for which added in parsing of the IVRS table to validate certain information being returned (such as the IO-APIC details). Patches were released for both 4.1 and 4.2. Regards, David _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
David Sutton
2013-May-28 18:25 UTC
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah
Feral, On Tue, May 28, 2013 at 12:40 PM, feral <blistovmhz@gmail.com> wrote:> I agree entirely about who should be taking responsibility, but we all > know of course, that Asus will never fix this. There are probably > less than 5000 people in the world affected by this, and if we assume > most of them will still buy Asus''s next board that has working IVRS > tables, Asus doesn''t care. > > I''m more interested in knowing why the > "iommu=no-amd-iommu-perdev-intremap" option doesn''t cover my issue > though. It seems odd to me that this option was implemented precisely > to work around the regression issue, but somehow it didn''t catch them > all. Goddamn I can''t wait for the Xen Documentation to take form :) > >The problem is that there were two issues which were identified, and that flag only deals with one of them; the first issue is where the IVRS has two entries for the IO-APIC (for NB and SB) but has the same id listed for both (causing a conflict). This means you can''t split across the different IO-APICs because its not sure which is which - the option allows it to use a global pool, decreasing security but allowing things to work. The second issue is where the IVRS table doesn''t have a valid id listed for the IO-APIC at all. This second case, because there isn''t an id listed, is the one where it just disables AMD-Vi support. Regards, David _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
So next question then. How did IOMMU work before these checks if I''m affected by the latter bug, where the ID simply isn''t listed? Did we fall back on global pool anyway? On Tue, May 28, 2013 at 11:25 AM, David Sutton <kantras@gmail.com> wrote:> Feral, > > > On Tue, May 28, 2013 at 12:40 PM, feral <blistovmhz@gmail.com> wrote: >> >> I agree entirely about who should be taking responsibility, but we all >> know of course, that Asus will never fix this. There are probably >> less than 5000 people in the world affected by this, and if we assume >> most of them will still buy Asus''s next board that has working IVRS >> tables, Asus doesn''t care. >> >> I''m more interested in knowing why the >> "iommu=no-amd-iommu-perdev-intremap" option doesn''t cover my issue >> though. It seems odd to me that this option was implemented precisely >> to work around the regression issue, but somehow it didn''t catch them >> all. Goddamn I can''t wait for the Xen Documentation to take form :) >> > > The problem is that there were two issues which were identified, and that > flag only deals with one of them; the first issue is where the IVRS has two > entries for the IO-APIC (for NB and SB) but has the same id listed for both > (causing a conflict). This means you can''t split across the different > IO-APICs because its not sure which is which - the option allows it to use a > global pool, decreasing security but allowing things to work. The second > issue is where the IVRS table doesn''t have a valid id listed for the IO-APIC > at all. This second case, because there isn''t an id listed, is the one where > it just disables AMD-Vi support. > > Regards, > > David-- _____ Fact: 1. Ninjas are mammals. 2. Ninjas fight ALL the time. 3. The purpose of the ninja is to flip out and kill people.
I own a crosshair v formula with this issue, the problem seems to be the reported io-apics handles are wrong so here it is, this hack is for 4.1.5 though it can be adapted for 4.2.2 --- a/xen/drivers/passthrough/amd/iommu_acpi.c 2013-04-23 16:44:20.000000000 +0000 +++ b/xen/drivers/passthrough/amd/iommu_acpi.c 2013-05-18 22:19:18.838434000 +0000 @@ -674,18 +674,18 @@ static u16 __init parse_ivhd_device_spec */ for ( apic = 0; apic < nr_ioapics; apic++ ) { - if ( IO_APIC_ID(apic) != ivhd_device->special.handle ) + if ( ioapic_bdf[IO_APIC_ID(apic)].bdf !ioapic_bdf[ivhd_device->special.handle].bdf ) continue; - if ( ioapic_bdf[ivhd_device->special.handle].pin_setup ) + if ( ioapic_bdf[IO_APIC_ID(apic)].pin_setup ) { - if ( ioapic_bdf[ivhd_device->special.handle].bdf == bdf ) + if ( ioapic_bdf[IO_APIC_ID(apic)].bdf == bdf ) AMD_IOMMU_DEBUG("IVHD Warning: Duplicate IO-APIC %#x entries\n", - ivhd_device->special.handle); + IO_APIC_ID(apic)); else { printk(XENLOG_ERR "IVHD Error: Conflicting IO-APIC %#x entries\n", - ivhd_device->special.handle); + IO_APIC_ID(apic)); if ( amd_iommu_perdev_intremap ) return 0; } @@ -693,9 +693,9 @@ static u16 __init parse_ivhd_device_spec else { /* set device id of ioapic */ - ioapic_bdf[ivhd_device->special.handle].bdf = bdf; + ioapic_bdf[IO_APIC_ID(apic)].bdf = bdf; - ioapic_bdf[ivhd_device->special.handle].pin_setup xzalloc_array( + ioapic_bdf[IO_APIC_ID(apic)].pin_setup = xzalloc_array( unsigned long, BITS_TO_LONGS(nr_ioapic_registers[apic])); if ( nr_ioapic_registers[apic] && !ioapic_bdf[IO_APIC_ID(apic)].pin_setup ) -- View this message in context: http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-tp5716461p5716579.html Sent from the Xen - User mailing list archive at Nabble.com.
Forgot to add, after this hack: (XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23 (XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55 (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 4414.882 MHz processor. (XEN) Initing memory sharing. (XEN) AMD-Vi: IOMMU 0 Enabled. (XEN) AMD-Vi: Enabling per-device vector maps (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) Interrupt remapping enabled -- View this message in context: http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-tp5716461p5716582.html Sent from the Xen - User mailing list archive at Nabble.com.
for 4.2.2: --- a/xen/drivers/passthrough/amd/iommu_acpi.c 2013-04-23 16:42:55.000000000 +0000 +++ b/xen/drivers/passthrough/amd/iommu_acpi.c 2013-05-31 16:21:14.733159937 +0000 @@ -674,19 +674,19 @@ static u16 __init parse_ivhd_device_spec */ for ( apic = 0; apic < nr_ioapics; apic++ ) { - if ( IO_APIC_ID(apic) != special->handle ) + if ( ioapic_sbdf[IO_APIC_ID(apic)].bdf !ioapic_sbdf[special->handle].bdf ) continue; - if ( ioapic_sbdf[special->handle].pin_setup ) + if ( ioapic_sbdf[IO_APIC_ID(apic)].pin_setup ) { - if ( ioapic_sbdf[special->handle].bdf == bdf && - ioapic_sbdf[special->handle].seg == seg ) + if ( ioapic_sbdf[IO_APIC_ID(apic)].bdf == bdf && + ioapic_sbdf[IO_APIC_ID(apic)].seg == seg ) AMD_IOMMU_DEBUG("IVHD Warning: Duplicate IO-APIC %#x entries\n", - special->handle); + IO_APIC_ID(apic)); else { printk(XENLOG_ERR "IVHD Error: Conflicting IO-APIC %#x entries\n", - special->handle); + IO_APIC_ID(apic)); if ( amd_iommu_perdev_intremap ) return 0; } @@ -694,10 +694,10 @@ static u16 __init parse_ivhd_device_spec else { /* set device id of ioapic */ - ioapic_sbdf[special->handle].bdf = bdf; - ioapic_sbdf[special->handle].seg = seg; + ioapic_sbdf[IO_APIC_ID(apic)].bdf = bdf; + ioapic_sbdf[IO_APIC_ID(apic)].seg = seg; - ioapic_sbdf[special->handle].pin_setup = xzalloc_array( + ioapic_sbdf[IO_APIC_ID(apic)].pin_setup = xzalloc_array( unsigned long, BITS_TO_LONGS(nr_ioapic_entries[apic])); if ( nr_ioapic_entries[apic] && !ioapic_sbdf[IO_APIC_ID(apic)].pin_setup ) -- View this message in context: http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-tp5716461p5716583.html Sent from the Xen - User mailing list archive at Nabble.com.
Perhaps my brain is stupid today, but I just noticed this reply and am trying to apply the patch, but both hunks are failing against 4.2.2. bens@octillion ~/Downloads/xen-4.2.2 $ patch -p1 < ivrs.patch patching file xen/drivers/passthrough/amd/iommu_acpi.c Hunk #1 FAILED at 674. Hunk #2 FAILED at 694. 2 out of 2 hunks FAILED -- saving rejects to file xen/drivers/passthrough/amd/iommu_acpi.c.rej On Fri, May 31, 2013 at 12:21 PM, nbhs <santellads@gmail.com> wrote:> for 4.2.2: > > --- a/xen/drivers/passthrough/amd/iommu_acpi.c 2013-04-23 > 16:42:55.000000000 > +0000 > +++ b/xen/drivers/passthrough/amd/iommu_acpi.c 2013-05-31 > 16:21:14.733159937 > +0000 > @@ -674,19 +674,19 @@ static u16 __init parse_ivhd_device_spec > */ > for ( apic = 0; apic < nr_ioapics; apic++ ) > { > - if ( IO_APIC_ID(apic) != special->handle ) > + if ( ioapic_sbdf[IO_APIC_ID(apic)].bdf !> ioapic_sbdf[special->handle].bdf ) > continue; > > - if ( ioapic_sbdf[special->handle].pin_setup ) > + if ( ioapic_sbdf[IO_APIC_ID(apic)].pin_setup ) > { > - if ( ioapic_sbdf[special->handle].bdf == bdf && > - ioapic_sbdf[special->handle].seg == seg ) > + if ( ioapic_sbdf[IO_APIC_ID(apic)].bdf == bdf && > + ioapic_sbdf[IO_APIC_ID(apic)].seg == seg ) > AMD_IOMMU_DEBUG("IVHD Warning: Duplicate IO-APIC %#x > entries\n", > - special->handle); > + IO_APIC_ID(apic)); > else > { > printk(XENLOG_ERR "IVHD Error: Conflicting IO-APIC %#x > entries\n", > - special->handle); > + IO_APIC_ID(apic)); > if ( amd_iommu_perdev_intremap ) > return 0; > } > @@ -694,10 +694,10 @@ static u16 __init parse_ivhd_device_spec > else > { > /* set device id of ioapic */ > - ioapic_sbdf[special->handle].bdf = bdf; > - ioapic_sbdf[special->handle].seg = seg; > + ioapic_sbdf[IO_APIC_ID(apic)].bdf = bdf; > + ioapic_sbdf[IO_APIC_ID(apic)].seg = seg; > > - ioapic_sbdf[special->handle].pin_setup = xzalloc_array( > + ioapic_sbdf[IO_APIC_ID(apic)].pin_setup = xzalloc_array( > unsigned long, BITS_TO_LONGS(nr_ioapic_entries[apic])); > if ( nr_ioapic_entries[apic] && > !ioapic_sbdf[IO_APIC_ID(apic)].pin_setup ) > > > > > -- > View this message in context: > http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-tp5716461p5716583.html > Sent from the Xen - User mailing list archive at Nabble.com. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >-- _____ Fact: 1. Ninjas are mammals. 2. Ninjas fight ALL the time. 3. The purpose of the ninja is to flip out and kill people. _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Nick Khamis
2013-Jun-19 01:27 UTC
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah
Given that this is a thread from our fellow AMD users, and all this talk about IOMMU, I just could not resist. Are there any server based chipsets, cpus, bios... that stably support IOMMU, and can play nice with XEN (i.e., pci-passthrough, iommu...)? We were looking at the SR5690 chipset and the Opteron. Not sure if this combo is dated now? Please forgive me Intel vt-d guy here, trying to make the transition. Kind Regards, Nick.
I uploaded it here http://www.filesend.net/download.php?f=6c7d9422ca9fd1d9be4042fbc6f7fbed -- View this message in context: http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-tp5716461p5717048.html Sent from the Xen - User mailing list archive at Nabble.com.
gizmochicken
2013-Jul-15 23:14 UTC
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah
I haven''t yet tried it with vanilla Xen, but at least with XenServer 6.2 (which relies on Xen 4.1.5), IOMMU can be forced to initiate on my Asus M5A99FX Pro R2.0 motherboard (which suffers from an erroneous IVRS table) using the following: iommu=no-intremap More information about this option, which seems to have been released for Intel setups, can be found here: http://support.citrix.com/article/CTX136517 On Tue, May 28, 2013 at 12:49 PM, feral <blistovmhz@gmail.com> wrote:> > In my case, the handle id is wrong ( you can see what it should be on the > > support document http://support.amd.com/us/Processor_TechDocs/48882.pdf- > > per tables 77/79 it should be the ID of the IO-APIC). Unlike the conflict > > error, which can be worked around by passing another command line option, > > invalid IO-APIC entries in the IVRS table will always cause AMD-Vi to > fail > > to be enabled if you have the XSA-36 patch installed. > > This is the part I have an angry nerd rage issue with. I''m pretty > sure I fall into the latter category where there is no option to > forcibly enable IOMMU. I''d understand if the patch wasn''t actually > removing functionality but in our case, it seems the patch does just > this even though the "bug" being addressed doesn''t concern me (and > probably quite a few others) whatsoever. The machine in question is > my gaming rig primarily, and gets a lot of use in QA, but nothing > mission critical. I''d rather hobble along with a known bad IVRS table > if it has no affect on my work (or games) than be locked out entirely > :p. > > I''m still a little confused though as I''ve heard a few people say that > this patch wasn''t introduced until Xen 4.2 but I saw the issue first > pop up in Xen 4.1.2 and later. Am I even looking at the right bug? > And what options are available to forcibly disable the checks other > than the "iommu=no-amd-iommu-perdev-intremap" ? >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
David Sutton
2013-Sep-05 22:21 UTC
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah
feral, On Thu, Sep 5, 2013 at 2:05 PM, feral <blistovmhz@gmail.com> wrote:> FYI, I finally got a response from Asus Engineering. Unfortunately > they''re all speaking something I am not, so we had to do 3 layers of > translation both ways to get it done, but I believe the issue may now be > resolved. > I sent them the ACPI dump and gave them the correct addresses for IOMMU 1 > and 2 (or 0 and 1?). > I''m in the middle of moving into my new place, so haven''t had a heap of > time to test yet, but passthrough did work correctly (HDMI audio even works > now) and no errors in dmesg. I''m currently testing with KVM. > > So that said, attached is the test BIOS we got hacked together. Flash at > your own risk. I''m running it without any noticeable issues, but as I > said, I haven''t had time to do any rigorous testing or deep eval. on the > actual IOMMU addresses (ie: haven''t dumped ACPI/IVRS yet to confirm > anything). > > Thanks for getting this - unfortunately I have the R1.0 version of thisboard, so most likely won''t be able to make use of it. Having said that, I''m watching progress of a patch which could help with this on the xen-devel list. Regards, David _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
gizmochicken
2013-Sep-05 22:56 UTC
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah
[This email is either empty or too large to be displayed at this time]
powerhouse64
2013-Sep-14 10:22 UTC
Re: Xen IOMMU disabled due to IVRS table... Blah blah blah
I''m very interested to see whether or not Asus has fixed this BIOS bug. How is the test BIOS working? Does anyone have an idea which boards are affected, or does it affect all Asus AMD boards? How would I get this BIOS fix? To whom would I need to turn to at Asus? For reference, I''ve written a little Xen VGA passthrough how-to on the Linux Mint forum and at least one user has encountered this exact issue - see </a> <http://forums.linuxmint.com/viewtopic.php?f=42&t=112013> and go to the last post. -- View this message in context: http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-tp5716461p5718631.html Sent from the Xen - User mailing list archive at Nabble.com.