Hello, I''ve been trying to get VT-D working on my Xen installation for the last few days. I''ve got an Asus P5E-VM DO motherboard, which is explicitly listed on the VTdHowTo page of the wiki as supporting VT-d, and a Core 2 Quad CPU. I''m using Debian as my dom0 OS, and I''ve built Xen 3.4.1 from source. The error I''m getting now is: (XEN) [VT-D]dmar.c:388: RMRR error: base_addr d0000000 end_address cfffffff (XEN) Failed to parse ACPI DMAR. Disabling VT-d. A little googling suggests this is a BIOS issue. I recently upgraded my BIOS to the latest version in order to see if that would resolve a different problem (it didn''t), so now I''m wondering if VT-D would be working on an older BIOS revision. Anyone with an Asus P5E-VM DO and VT-D working care to tell me which revision their BIOS is? Cheers, Mike _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I''ve tried a few different BIOS revisions, still with no luck. I have some further info though: root@monolith:~# xm dmesg | grep -C1 VT-D (XEN) ACPI: HPET id: 0xffffffff base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed90000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed92000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed93000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:388: RMRR error: base_addr d0000000 end_address cfffffff (XEN) Failed to parse ACPI DMAR. Disabling VT-d. root@monolith:~# acpidump -t DMAR Wrong checksum for OEMB! Wrong checksum for ! root@monolith:~# acpidump | grep -B3 -A20 DMAR Wrong checksum for OEMB! Wrong checksum for ! @ 0xcff601b0 0000: 00 4d 41 52 48 01 00 00 01 f5 41 4d 49 00 00 00 .MARH.....AMI... 0010: 4f 45 4d 44 4d 41 52 00 01 00 00 00 4d 53 46 54 OEMDMAR.....MSFT 0020: 97 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00 ....#........... 0030: 00 00 18 00 00 00 00 00 00 00 d9 fe 00 00 00 00 ................ 0040: 01 08 00 00 00 00 1b 00 00 00 28 00 00 00 00 00 ..........(..... 0050: 00 20 d9 fe 00 00 00 00 01 08 00 00 00 00 03 00 . .............. 0060: 01 08 00 00 00 00 03 02 01 08 00 00 00 00 03 03 ................ 0070: 00 00 10 00 01 00 00 00 00 30 d9 fe 00 00 00 00 .........0...... 0080: 01 00 58 00 00 00 00 00 00 d0 0e 00 00 00 00 00 ..X............. 0090: ff ff 0e 00 00 00 00 00 01 08 00 00 00 00 1d 00 ................ 00a0: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02 ................ 00b0: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00 ................ 00c0: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02 ................ 00d0: 01 08 00 00 00 00 1a 07 01 00 18 00 00 00 00 00 ................ 00e0: 00 00 00 d0 00 00 00 00 ff ff ff cf 00 00 00 00 ................ 00f0: 01 00 58 00 00 00 00 00 00 00 fe cf 00 00 00 00 ..X............. 0100: ff ff fe cf 00 00 00 00 01 08 00 00 00 00 1d 00 ................ 0110: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02 ................ 0120: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00 ................ 0130: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02 ................ 0140: 01 08 00 00 00 00 1a 07 ........>From what I can tell AMI has screwed my BIOS so bad acpidump can''t evendetermine which table IS the DMAR table, and that''s what is preventing me from making VT-D work. On Tue, Sep 22, 2009 at 7:01 PM, Michael MacLeod <mikemacleod@gmail.com>wrote:> Hello, > > I''ve been trying to get VT-D working on my Xen installation for the last > few days. I''ve got an Asus P5E-VM DO motherboard, which is explicitly listed > on the VTdHowTo page of the wiki as supporting VT-d, and a Core 2 Quad CPU. > I''m using Debian as my dom0 OS, and I''ve built Xen 3.4.1 from source. > > The error I''m getting now is: > (XEN) [VT-D]dmar.c:388: RMRR error: base_addr d0000000 end_address cfffffff > (XEN) Failed to parse ACPI DMAR. Disabling VT-d. > > A little googling suggests this is a BIOS issue. I recently upgraded my > BIOS to the latest version in order to see if that would resolve a different > problem (it didn''t), so now I''m wondering if VT-D would be working on an > older BIOS revision. > > Anyone with an Asus P5E-VM DO and VT-D working care to tell me which > revision their BIOS is? > > Cheers, > Mike >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Michael MacLeod wrote:> A little googling suggests this is a BIOS issue. I recently upgraded my > BIOS to the latest version in order to see if that would resolve a > different problem (it didn''t), so now I''m wondering if VT-D would be > working on an older BIOS revision. > > Anyone with an Asus P5E-VM DO and VT-D working care to tell me which > revision their BIOS is?I do have a P5E-VM DO running with VT-d using BIOS 0702. Good to know it breaks when upgrading, I was tempted to give it a try recently... I do have error messages like this: (XEN) [VT-D]iommu.c:722: iommu_page_fault: iommu->reg = ffff828bfff56000 (XEN) [VT-D]iommu.c:691: iommu_fault_status: Fault Overflow (XEN) [VT-D]iommu.c:694: iommu_fault_status: Primary Pending Fault (XEN) [VT-D]iommu.c:676: iommu_fault:DMA Write: 0:2.0 addr 400000000 REASON 5 iommu->reg = ffff828bfff56000 (XEN) print_vtd_entries: iommu = ffff83022bde62d0 bdf = 0:2:0 gmfn = 400000 (XEN) root_entry = ffff83022bdcf000 (XEN) root_entry[0] = 224cc8001 (XEN) context = ffff830224cc8000 (XEN) context[10] = 101_22bdb3001 (XEN) l3 = ffff83022bdb3000 (XEN) l3_index = 10 (XEN) l3[10] = 0 (XEN) l3[10] not present But VT-d gets enabled nevertheless and actual pass-through was also working the last time I tried... Best regards, Christian _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Wed, Sep 23, 2009 at 5:27 AM, Christian Tramnitz <chris.ace@gmx.net>wrote:> Michael MacLeod wrote: > >> A little googling suggests this is a BIOS issue. I recently upgraded my >> BIOS to the latest version in order to see if that would resolve a different >> problem (it didn''t), so now I''m wondering if VT-D would be working on an >> older BIOS revision. >> >> Anyone with an Asus P5E-VM DO and VT-D working care to tell me which >> revision their BIOS is? >> > > I do have a P5E-VM DO running with VT-d using BIOS 0702. > Good to know it breaks when upgrading, I was tempted to give it a try > recently... > > I do have error messages like this: > (XEN) [VT-D]iommu.c:722: iommu_page_fault: iommu->reg = ffff828bfff56000 > (XEN) [VT-D]iommu.c:691: iommu_fault_status: Fault Overflow > (XEN) [VT-D]iommu.c:694: iommu_fault_status: Primary Pending Fault > (XEN) [VT-D]iommu.c:676: iommu_fault:DMA Write: 0:2.0 addr 400000000 REASON > 5 iommu->reg = ffff828bfff56000 > (XEN) print_vtd_entries: iommu = ffff83022bde62d0 bdf = 0:2:0 gmfn = 400000 > (XEN) root_entry = ffff83022bdcf000 > (XEN) root_entry[0] = 224cc8001 > (XEN) context = ffff830224cc8000 > (XEN) context[10] = 101_22bdb3001 > (XEN) l3 = ffff83022bdb3000 > (XEN) l3_index = 10 > (XEN) l3[10] = 0 > (XEN) l3[10] not present > > > But VT-d gets enabled nevertheless and actual pass-through was also working > the last time I tried... > > > > Best regards, > Christian >Christian, I''ve tried several different BIOS revisions (currently my BIOS is flashed with 0702). Did you build Xen from source and what version are you running? And would you mind posting your relevant grub menu.lst entry? I want to see what options you''re passing to Xen at boot. Still, it''s good to know that it should be possible to work around this problem and make it work. Mike _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Michael MacLeod wrote:> I''ve tried several different BIOS revisions (currently my BIOS is > flashed with 0702). Did you build Xen from source and what version are > you running? And would you mind posting your relevant grub menu.lst > entry? I want to see what options you''re passing to Xen at boot. > > Still, it''s good to know that it should be possible to work around this > problem and make it work.I was running an old 3.4.0-pre but just gave it a try with latest 3.5-unstable, with the same result. Both were built from source. grub.conf: kernel /boot/xen.gz iommu=1 iommu_inclusive_mapping=1 module /boot/vmlinuz-2.6.29-xen-r1 root=/dev/md3 ro pciback.hide=(01:01.0) (01:01.0 is a PCI card that I use for passthrough-testing) Best regards, Christian _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Wed, Sep 23, 2009 at 8:10 AM, Christian Tramnitz <chris.ace@gmx.net>wrote:> Michael MacLeod wrote: > >> I''ve tried several different BIOS revisions (currently my BIOS is flashed >> with 0702). Did you build Xen from source and what version are you running? >> And would you mind posting your relevant grub menu.lst entry? I want to see >> what options you''re passing to Xen at boot. >> >> Still, it''s good to know that it should be possible to work around this >> problem and make it work. >> > > I was running an old 3.4.0-pre but just gave it a try with latest > 3.5-unstable, with the same result. Both were built from source. > > grub.conf: > kernel /boot/xen.gz iommu=1 iommu_inclusive_mapping=1 > module /boot/vmlinuz-2.6.29-xen-r1 root=/dev/md3 ro pciback.hide=(01:01.0) >That''s essentially the same menu.lst entry I use, but to no avail. Did you pass any particular options to Xen when you built it? I just followed the basic instructions in the README; make world, make install, reboot, basically. Would you mind telling me what output you get from these two commands: acpidump | grep -B3 -A20 DMAR acpidump -t DMAR The DMAR table seems to be what VT-D uses for mapping memory, and on my BIOS appears to be corrupt. However, if it''s working on your system then that suggests that your table shouldn''t be corrupt. However, if we''re on the same BIOS revision, then that''s very odd. Unless the ACPI tables aren''t re-written with a BIOS reflashing; I''m not knowledgable enough about BIOSs to know if that''s the case or not off the top of my head. Cheers Mike _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Wed, Sep 23, 2009 at 10:37 AM, Christian Tramnitz <christian@tramnitz.com> wrote:> While corrupted tables or checksum errors are not really the greatest thing > on earth they are not neccessarily an indicator for broken functionality. > My X58 board also reports checksum errors (just like the Q35, see below) on > various tables but everything is working as expected. > > Do you have any USB devices connected? I had problems mit my X58 once, > where I could only get DMAR correctly parsed if USB was completely disabled > in the BIOS. I''ve got a corrected BIOS from Asus that fixed that, but > haven''t seen the same problem on the Q35 (P5E), but that might be just > because I never used USB at all on that board... >I tried disabling USB in the BIOS completely but that didn''t have any effect, I still get the same error message and VT-D is still disabled. Is it odd that the ACPI tables you have are different from the one I have? Are they generated at boot based on the options enabled, or are they flashed in when the BIOS is installed? Any other ideas on what I could try? _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Wed, Sep 23, 2009 at 10:37 AM, Christian Tramnitz <christian@tramnitz.com> wrote:> While corrupted tables or checksum errors are not really the greatest thing > on earth they are not neccessarily an indicator for broken functionality. > My X58 board also reports checksum errors (just like the Q35, see below) on > various tables but everything is working as expected. > > Do you have any USB devices connected? I had problems mit my X58 once, > where I could only get DMAR correctly parsed if USB was completely disabled > in the BIOS. I''ve got a corrected BIOS from Asus that fixed that, but > haven''t seen the same problem on the Q35 (P5E), but that might be just > because I never used USB at all on that board... >Since disabling USB didn''t work, maybe there''s another option in the BIOS that''s the problem. I know this is asking a lot, but can you send me the details of your BIOS settings? _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users