Tom Rotenberg
2009-Jun-28 14:55 UTC
[Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
Hi, I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. I have also tried to boot with and without the ''iommu_inclusive_mapping'' boot option, but it didn''t seem to help at all. Does anyone knows how to solve this issue? These are the last messages in the UART: (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fda0: (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978 (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40 (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Cui, Dexuan
2009-Jun-29 06:30 UTC
RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100
+++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800
@@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm
{
u64 val;
+ printk("clear_fault_bits: CAP_REG=0x%llx\n",
+ (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG));
val = dmar_readq(
iommu->reg,
cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8);
Hi Tom,
Can you please apply the above patch and send us the entire log?
(I don''t have the same host, so I can''t try it myself.)
Thanks,
-- Dexuan
________________________________
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg
Sent: 2009?6?28? 22:56
To: xen-devel@lists.xensource.com
Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
Hi,
I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to boot xen
with a ''iommu=1'' parameter in the boot line, Xen fails to
boot. I have also tried to boot with and without the
''iommu_inclusive_mapping'' boot option, but it didn''t
seem to help at all.
Does anyone knows how to solve this issue?
These are the last messages in the UART:
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush
(XEN) Intel VT-d Snoop Control supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation supported.
(XEN) Intel VT-d Interrupt Remapping supported.
(XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor
(XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020
(XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90
(XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110
(XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005
(XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc
(XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen stack trace from rsp=ffff828c8026fda0:
(XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490
(XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000
(XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f
(XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60
(XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978
(XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40
(XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000
(XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003
(XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000
(XEN) Xen call trace:
(XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
(XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0
(XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610
(XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110
(XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0
(XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3
(XEN)
(XEN) Pagetable walk from ffff828bfff5aff8:
(XEN) L4[0x105] = 000000007be7a027 5555555555555555
(XEN) L3[0x02f] = 000000007c04b063 5555555555555555
(XEN) L2[0x1ff] = 000000007c04a063 5555555555555555
(XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0000]
(XEN) Faulting linear address: ffff828bfff5aff8
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Cui, Dexuan
2009-Jun-29 06:34 UTC
RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100
+++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800
@@ -1693,7 +1693,7 @@ static int init_vtd_hw(void)
}
dma_msi_data_init(iommu, iommu->vector);
dma_msi_addr_init(iommu, cpu_physical_id(first_cpu(cpu_online_map)));
- clear_fault_bits(iommu);
+ //clear_fault_bits(iommu);
dmar_writel(iommu->reg, DMAR_FECTL_REG, 0);
/* initialize flush functions */
And, can you comment the invocation of clear_fault_bits() in init_vtd_hw() like
above and see if the host can boot fine?
Thanks,
-- Dexuan
-----Original Message-----
From: Cui, Dexuan
Sent: 2009?6?29? 14:30
To: ''Tom Rotenberg''; xen-devel@lists.xensource.com
Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
iommu=1
diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100
+++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800
@@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm
{
u64 val;
+ printk("clear_fault_bits: CAP_REG=0x%llx\n",
+ (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG));
val = dmar_readq(
iommu->reg,
cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8);
Hi Tom,
Can you please apply the above patch and send us the entire log?
(I don''t have the same host, so I can''t try it myself.)
Thanks,
-- Dexuan
________________________________
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg
Sent: 2009?6?28? 22:56
To: xen-devel@lists.xensource.com
Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
Hi,
I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to boot xen
with a ''iommu=1'' parameter in the boot line, Xen fails to
boot. I have also tried to boot with and without the
''iommu_inclusive_mapping'' boot option, but it didn''t
seem to help at all.
Does anyone knows how to solve this issue?
These are the last messages in the UART:
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush
(XEN) Intel VT-d Snoop Control supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation supported.
(XEN) Intel VT-d Interrupt Remapping supported.
(XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor
(XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020
(XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90
(XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110
(XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005
(XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc
(XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen stack trace from rsp=ffff828c8026fda0:
(XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490
(XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000
(XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f
(XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60
(XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978
(XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40
(XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000
(XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003
(XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000
(XEN) Xen call trace:
(XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
(XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0
(XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610
(XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110
(XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0
(XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3
(XEN)
(XEN) Pagetable walk from ffff828bfff5aff8:
(XEN) L4[0x105] = 000000007be7a027 5555555555555555
(XEN) L3[0x02f] = 000000007c04b063 5555555555555555
(XEN) L2[0x1ff] = 000000007c04a063 5555555555555555
(XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0000]
(XEN) Faulting linear address: ffff828bfff5aff8
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Jun-29 09:01 UTC
Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
Hi, Well, just tried what you suggested, and here are the results: * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with iommu=1 The strange thing, is that after it booted one time ok (with the second patch), then it continued to boot ok, even if i reverted the second patch. The relevant lines in the log are: (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 Anyway, here is the full Xen log, with the first patch (print cap''s) applied: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.068 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg ffff828bfff57000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg ffff828bfff56000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg ffff828bfff55000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 (XEN) TOTAL: ffffffff80000000->ffffffff82000000 (XEN) ENTRY ADDRESS: ffffffff80200000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 132kB init memory. Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c > --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 > +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800 > @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) > } > dma_msi_data_init(iommu, iommu->vector); > dma_msi_addr_init(iommu, > cpu_physical_id(first_cpu(cpu_online_map))); > - clear_fault_bits(iommu); > + //clear_fault_bits(iommu); > dmar_writel(iommu->reg, DMAR_FECTL_REG, 0); > > /* initialize flush functions */ > > And, can you comment the invocation of clear_fault_bits() in init_vtd_hw() > like above and see if the host can boot fine? > > Thanks, > -- Dexuan > > > > -----Original Message----- > From: Cui, Dexuan > Sent: 2009?6?29? 14:30 > To: ''Tom Rotenberg''; xen-devel@lists.xensource.com > Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with > iommu=1 > > diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c > --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 > +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800 > @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm > { > u64 val; > > + printk("clear_fault_bits: CAP_REG=0x%llx\n", > + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); > val = dmar_readq( > iommu->reg, > cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8); > > Hi Tom, > Can you please apply the above patch and send us the entire log? > (I don''t have the same host, so I can''t try it myself.) > > Thanks, > -- Dexuan > > ________________________________ > > From: xen-devel-bounces@lists.xensource.com [mailto: > xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg > Sent: 2009?6?28? 22:56 > To: xen-devel@lists.xensource.com > Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with > iommu=1 > > > Hi, > > I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to boot > xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. I have > also tried to boot with and without the ''iommu_inclusive_mapping'' boot > option, but it didn''t seem to help at all. > > Does anyone knows how to solve this issue? > > These are the last messages in the UART: > > (XEN) Brought up 2 CPUs > (XEN) microcode.c:73:d32767 microcode: CPU1 resumed > (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush > (XEN) Intel VT-d Snoop Control supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported. > (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- > (XEN) CPU: 0 > (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 > (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor > (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 > (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 > (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 > (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 > (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc > (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 > (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 > (XEN) Xen stack trace from rsp=ffff828c8026fda0: > (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f > ffff828c80204490 > (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 > 0000000000000000 > (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 > 000000000000001f > (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 > ffff828c80205f60 > (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 > ffff828c80227978 > (XEN) ffff828c8021a589 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 > 000000000008bf40 > (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 > 0000000000000000 > (XEN) ffffffff00000000 0000000800000000 000000010000006e > 0000000000000003 > (XEN) 00000000000002f8 0000000000000000 0000000000000000 > 0000000000067e2c > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) ffff828c801000b5 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > ffff830079f18000 > (XEN) Xen call trace: > (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 > (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 > (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 > (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 > (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 > (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 > (XEN) > (XEN) Pagetable walk from ffff828bfff5aff8: > (XEN) L4[0x105] = 000000007be7a027 5555555555555555 > (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 > (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 > (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff > (XEN) > (XEN) **************************************** > (XEN) Panic on CPU 0: > (XEN) FATAL PAGE FAULT > (XEN) [error_code=0000] > (XEN) Faulting linear address: ffff828bfff5aff8 > (XEN) **************************************** > (XEN) > (XEN) Reboot in five seconds... > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Cui, Dexuan
2009-Jun-29 11:32 UTC
RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
So, if I understand correctly, it''s really strange you can''t
reproduce the panic now.
With only the first printk patch applied (and don''t comment the
clear_fault_bits()), can you try a cold rebooting the host (plug the power cord
off from the host; wait several minutes; plug the cord in and tunn on the host)?
And the log?
BTW, in your original mail, the log shows:
(XEN) Intel VT-d Queued Invalidation supported.
(XEN) Intel VT-d Interrupt Remapping supported.
But in your this new log, it shows:
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
Are you sure you have been operating on the same host? :-)
-- Dexuan
________________________________
From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com]
Sent: 2009?6?29? 17:01
To: Cui, Dexuan
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
iommu=1
Hi,
Well, just tried what you suggested, and here are the results:
* If i comment the ''clear_fault_bits()'' call - then Xen boots
ok, with iommu=1
The strange thing, is that after it booted one time ok (with the second patch),
then it continued to boot ok, even if i reverted the second patch.
The relevant lines in the log are:
(XEN) clear_fault_bits: CAP_REG=0xc0000020630270
(XEN) clear_fault_bits: CAP_REG=0xc0000020630260
(XEN) clear_fault_bits: CAP_REG=0xc9008020e30260
(XEN) clear_fault_bits: CAP_REG=0xc9008020630260
Anyway, here is the full Xen log, with the first patch (print cap''s)
applied:
__ __ _____ _ _ ___
\ \/ /___ _ __ |___ /| || | / _ \
\ // _ \ ''_ \ |_ \| || |_| | | |
/ \ __/ | | | ___) |__ _| |_| |
/_/\_\___|_| |_| |____(_) |_|(_)___/
(XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) )
Mon Jun 29 07:53:38 EDT 2009
(XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd
(XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping
loglvl=all guest_loglvl=all
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f000 (usable)
(XEN) 000000000009f000 - 00000000000a0000 (reserved)
(XEN) 0000000000100000 - 000000007c04d400 (usable)
(XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS)
(XEN) 000000007c04f400 - 000000007f000000 (reserved)
(XEN) 00000000f8000000 - 00000000fc000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fed18000 - 00000000fed1c000 (reserved)
(XEN) 00000000fed20000 - 00000000fed90000 (reserved)
(XEN) 00000000feda0000 - 00000000feda6000 (reserved)
(XEN) 00000000fee00000 - 00000000fee10000 (reserved)
(XEN) 00000000ffe60000 - 0000000100000000 (reserved)
(XEN) System RAM: 1983MB (2031536kB)
(XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL )
(XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61)
(XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61)
(XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624)
(XEN) ACPI: FACS 7C060C00, 0040
(XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61)
(XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61)
(XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47)
(XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61)
(XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61)
(XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0)
(XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61)
(XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624)
(XEN) NUMA turned off
(XEN) Faking a node at 0000000000000000-000000007c04d000
(XEN) Domain heap initialised
(XEN) DMI 2.4 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0]
(XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) Processor #1 7:7 APIC version 20
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
(XEN) [VT-D]dmar.c:485: Host address width 36
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed10000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed11000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed12000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed13000
(XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
(XEN) Intel VT-d DMAR tables have been parsed.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2527.068 MHz processor.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) HVM: VMX enabled
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) mce_init: init bank1
(XEN) mce_init: init bank2
(XEN) mce_init: init bank3
(XEN) mce_init: init bank4
(XEN) mce_init: init bank5
(XEN) CPU0: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU0 has no CMCI support
(XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06
(XEN) Booting processor 1/1 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) mce_init: init bank1
(XEN) mce_init: init bank2
(XEN) mce_init: init bank4
(XEN) mce_init: init bank5
(XEN) CPU1: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
(XEN) clear_fault_bits: CAP_REG=0xc0000020630270
(XEN) clear_fault_bits: CAP_REG=0xc0000020630260
(XEN) clear_fault_bits: CAP_REG=0xc9008020e30260
(XEN) clear_fault_bits: CAP_REG=0xc9008020630260
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) HPET: 4 timers in total, 0 timers will be used for broadcast
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3
(XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
ffff828bfff57000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
ffff828bfff56000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
ffff828bfff55000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
ffff828bfff54000
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 ->
0xffffffff8070ab48
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be
allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48
(XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400
(XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18
(XEN) Start info: ffffffff81f28000->ffffffff81f284b4
(XEN) Page tables: ffffffff81f29000->ffffffff81f3c000
(XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000
(XEN) TOTAL: ffffffff80000000->ffffffff82000000
(XEN) ENTRY ADDRESS: ffffffff80200000
(XEN) Dom0 has maximum 2 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times
to switch input to Xen)
(XEN) Freed 132kB init memory.
Tom
2009/6/29 Cui, Dexuan
<dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>>
diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100
+++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800
@@ -1693,7 +1693,7 @@ static int init_vtd_hw(void)
}
dma_msi_data_init(iommu, iommu->vector);
dma_msi_addr_init(iommu, cpu_physical_id(first_cpu(cpu_online_map)));
- clear_fault_bits(iommu);
+ //clear_fault_bits(iommu);
dmar_writel(iommu->reg, DMAR_FECTL_REG, 0);
/* initialize flush functions */
And, can you comment the invocation of clear_fault_bits() in init_vtd_hw() like
above and see if the host can boot fine?
Thanks,
-- Dexuan
-----Original Message-----
From: Cui, Dexuan
Sent: 2009?6?29? 14:30
To: ''Tom Rotenberg'';
xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>
Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
iommu=1
diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100
+++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800
@@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm
{
u64 val;
+ printk("clear_fault_bits: CAP_REG=0x%llx\n",
+ (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG));
val = dmar_readq(
iommu->reg,
cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8);
Hi Tom,
Can you please apply the above patch and send us the entire log?
(I don''t have the same host, so I can''t try it myself.)
Thanks,
-- Dexuan
________________________________
From:
xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>
[mailto:xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>]
On Behalf Of Tom Rotenberg
Sent: 2009?6?28? 22:56
To: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>
Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
Hi,
I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to boot xen
with a ''iommu=1'' parameter in the boot line, Xen fails to
boot. I have also tried to boot with and without the
''iommu_inclusive_mapping'' boot option, but it didn''t
seem to help at all.
Does anyone knows how to solve this issue?
These are the last messages in the UART:
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush
(XEN) Intel VT-d Snoop Control supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation supported.
(XEN) Intel VT-d Interrupt Remapping supported.
(XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor
(XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020
(XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90
(XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110
(XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005
(XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc
(XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen stack trace from rsp=ffff828c8026fda0:
(XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490
(XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000
(XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f
(XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60
(XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978
(XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40
(XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000
(XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003
(XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000
(XEN) Xen call trace:
(XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
(XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0
(XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610
(XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110
(XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0
(XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3
(XEN)
(XEN) Pagetable walk from ffff828bfff5aff8:
(XEN) L4[0x105] = 000000007be7a027 5555555555555555
(XEN) L3[0x02f] = 000000007c04b063 5555555555555555
(XEN) L2[0x1ff] = 000000007c04a063 5555555555555555
(XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0000]
(XEN) Faulting linear address: ffff828bfff5aff8
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Jun-29 12:14 UTC
Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
Hi, Tried again, this time with cold reboot, it worked again. I also checked the lines you were talking about, and now they are: (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. Could it be that there is some kind of problem with the hardware, which makes it sometimes report invalid values? (with the Lenovo T400 i''m testing, if you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on boot, so you must do a cold reboot). Here is the log: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.058 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg ffff828bfff57000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg ffff828bfff56000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg ffff828bfff55000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 (XEN) TOTAL: ffffffff80000000->ffffffff82000000 (XEN) ENTRY ADDRESS: ffffffff80200000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 132kB init memory. Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com>> So, if I understand correctly, it''s really strange you can''t reproduce > the panic now. > > With only the first printk patch applied (and don''t comment the > clear_fault_bits()), can you try a cold rebooting the host (plug the power > cord off from the host; wait several minutes; plug the cord in and tunn on > the host)? And the log? > > BTW, in your original mail, the log shows: > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported. > > But in your this new log, it shows: > (XEN) Intel VT-d Queued Invalidation* not* supported. > (XEN) Intel VT-d Interrupt Remapping* not* supported. > Are you sure you have been operating on the same host? :-) > > -- Dexuan > > ------------------------------ > *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] > *Sent:* 2009年6月29日 17:01 > *To:* Cui, Dexuan > *Cc:* xen-devel@lists.xensource.com > *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, > with iommu=1 > > Hi, > > Well, just tried what you suggested, and here are the results: > * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with > iommu=1 > The strange thing, is that after it booted one time ok (with the second > patch), then it continued to boot ok, even if i reverted the second patch. > > The relevant lines in the log are: > (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 > (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 > (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 > (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 > > Anyway, here is the full Xen log, with the first patch (print cap''s) > applied: > > __ __ _____ _ _ ___ > \ \/ /___ _ __ |___ /| || | / _ \ > \ // _ \ ''_ \ |_ \| || |_| | | | > / \ __/ | | | ___) |__ _| |_| | > /_/\_\___|_| |_| |____(_) |_|(_)___/ > > (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian > 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 > (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd > (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 > iommu_inclusive_mapping loglvl=all guest_loglvl=all > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 1 MBR signatures > (XEN) Found 1 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009f000 (usable) > (XEN) 000000000009f000 - 00000000000a0000 (reserved) > (XEN) 0000000000100000 - 000000007c04d400 (usable) > (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) > (XEN) 000000007c04f400 - 000000007f000000 (reserved) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec10000 (reserved) > (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) > (XEN) 00000000fed20000 - 00000000fed90000 (reserved) > (XEN) 00000000feda0000 - 00000000feda6000 (reserved) > (XEN) 00000000fee00000 - 00000000fee10000 (reserved) > (XEN) 00000000ffe60000 - 0000000100000000 (reserved) > (XEN) System RAM: 1983MB (2031536kB) > (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) > (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) > (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) > (XEN) ACPI: FACS 7C060C00, 0040 > (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) > (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) > (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL > 61) > (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL > 61) > (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) > (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) > (XEN) NUMA turned off > (XEN) Faking a node at 0000000000000000-000000007c04d000 > (XEN) Domain heap initialised > (XEN) DMI 2.4 present. > (XEN) Using APIC driver default > (XEN) ACPI: PM-Timer IO Port: 0x1008 > (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] > (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] > (XEN) ACPI: Local APIC address 0xfee00000 > (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) > (XEN) Processor #0 7:7 APIC version 20 > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) > (XEN) Processor #1 7:7 APIC version 20 > (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) > (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) > (XEN) ACPI: IRQ0 used by override. > (XEN) ACPI: IRQ2 used by override. > (XEN) ACPI: IRQ9 used by override. > (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs > (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 > (XEN) [VT-D]dmar.c:485: Host address width 36 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 > (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL > (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 > (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 > (XEN) Intel VT-d DMAR tables have been parsed. > (XEN) Using ACPI (MADT) for SMP configuration information > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Initializing CPU#0 > (XEN) Detected 2527.068 MHz processor. > (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K > (XEN) CPU: L2 cache: 6144K > (XEN) CPU: Physical Processor ID: 0 > (XEN) CPU: Processor Core ID: 0 > (XEN) VMX: Supported advanced features: > (XEN) - APIC MMIO access virtualisation > (XEN) - APIC TPR shadow > (XEN) - Virtual NMI > (XEN) - MSR direct-access bitmap > (XEN) HVM: VMX enabled > (XEN) Intel machine check reporting enabled on CPU#0. > (XEN) mce_init: init bank1 > (XEN) mce_init: init bank2 > (XEN) mce_init: init bank3 > (XEN) mce_init: init bank4 > (XEN) mce_init: init bank5 > (XEN) CPU0: Thermal monitoring enabled (TM2) > (XEN) CMCI: CPU0 has no CMCI support > (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 > (XEN) Booting processor 1/1 eip 8c000 > (XEN) Initializing CPU#1 > (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K > (XEN) CPU: L2 cache: 6144K > (XEN) CPU: Physical Processor ID: 0 > (XEN) CPU: Processor Core ID: 1 > (XEN) Intel machine check reporting enabled on CPU#1. > (XEN) mce_init: init bank1 > (XEN) mce_init: init bank2 > (XEN) mce_init: init bank4 > (XEN) mce_init: init bank5 > (XEN) CPU1: Thermal monitoring enabled (TM2) > (XEN) CMCI: CPU1 has no CMCI support > (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 > (XEN) Total of 2 processors activated. > (XEN) ENABLING IO-APIC IRQs > (XEN) -> Using new ACK method > (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 > (XEN) checking TSC synchronization across 2 CPUs: passed. > (XEN) Platform timer is 14.318MHz HPET > (XEN) Brought up 2 CPUs > (XEN) microcode.c:73:d32767 microcode: CPU1 resumed > (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush > (XEN) Intel VT-d Snoop Control not supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation not supported. > (XEN) Intel VT-d Interrupt Remapping not supported. > (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 > (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 > (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 > (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 > (XEN) I/O virtualisation enabled > (XEN) I/O virtualisation for PV guests disabled > (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast > (XEN) ACPI sleep modes: S3 > (XEN) mcheck_poll: Machine check polling timer started. > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 > (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 > (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg > ffff828bfff57000 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg > ffff828bfff56000 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg > ffff828bfff55000 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg > ffff828bfff54000 > (XEN) *** LOADING DOMAIN 0 *** > (XEN) Xen kernel: 64-bit, lsb, compat32 > (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> > 0xffffffff8070ab48 > (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to > be allocated) > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 > (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 > (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 > (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 > (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 > (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 > (XEN) TOTAL: ffffffff80000000->ffffffff82000000 > (XEN) ENTRY ADDRESS: ffffffff80200000 > (XEN) Dom0 has maximum 2 VCPUs > (XEN) Scrubbing Free RAM: .done. > (XEN) Xen trace buffers: disabled > (XEN) Std. Loglevel: All > (XEN) Guest Loglevel: All > (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input > to Xen) > (XEN) Freed 132kB init memory. > > Tom > > 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> > >> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c >> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 >> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800 >> @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) >> } >> dma_msi_data_init(iommu, iommu->vector); >> dma_msi_addr_init(iommu, >> cpu_physical_id(first_cpu(cpu_online_map))); >> - clear_fault_bits(iommu); >> + //clear_fault_bits(iommu); >> dmar_writel(iommu->reg, DMAR_FECTL_REG, 0); >> >> /* initialize flush functions */ >> >> And, can you comment the invocation of clear_fault_bits() in init_vtd_hw() >> like above and see if the host can boot fine? >> >> Thanks, >> -- Dexuan >> >> >> >> -----Original Message----- >> From: Cui, Dexuan >> Sent: 2009?6?29? 14:30 >> To: ''Tom Rotenberg''; xen-devel@lists.xensource.com >> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >> with iommu=1 >> >> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c >> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 >> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800 >> @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm >> { >> u64 val; >> >> + printk("clear_fault_bits: CAP_REG=0x%llx\n", >> + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); >> val = dmar_readq( >> iommu->reg, >> cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8); >> >> Hi Tom, >> Can you please apply the above patch and send us the entire log? >> (I don''t have the same host, so I can''t try it myself.) >> >> Thanks, >> -- Dexuan >> >> ________________________________ >> >> From: xen-devel-bounces@lists.xensource.com [mailto: >> xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >> Sent: 2009?6?28? 22:56 >> To: xen-devel@lists.xensource.com >> Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with >> iommu=1 >> >> >> Hi, >> >> I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to >> boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. I >> have also tried to boot with and without the ''iommu_inclusive_mapping'' boot >> option, but it didn''t seem to help at all. >> >> Does anyone knows how to solve this issue? >> >> These are the last messages in the UART: >> >> (XEN) Brought up 2 CPUs >> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >> (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush >> (XEN) Intel VT-d Snoop Control supported. >> (XEN) Intel VT-d DMA Passthrough not supported. >> (XEN) Intel VT-d Queued Invalidation supported. >> (XEN) Intel VT-d Interrupt Remapping supported. >> (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- >> (XEN) CPU: 0 >> (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >> (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor >> (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: >> 0000000000000020 >> (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: >> ffff830078f82f90 >> (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: >> ffff83007be2c110 >> (XEN) r9: 000000000000001a r10: 0000000000000005 r11: >> 0000000000000005 >> (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: >> ffff830078f82fcc >> (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: >> 00000000000026f0 >> (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 >> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 >> (XEN) Xen stack trace from rsp=ffff828c8026fda0: >> (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f >> ffff828c80204490 >> (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 >> 0000000000000000 >> (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 >> 000000000000001f >> (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 >> ffff828c80205f60 >> (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 >> ffff828c80227978 >> (XEN) ffff828c8021a589 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 >> 000000000008bf40 >> (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 >> 0000000000000000 >> (XEN) ffffffff00000000 0000000800000000 000000010000006e >> 0000000000000003 >> (XEN) 00000000000002f8 0000000000000000 0000000000000000 >> 0000000000067e2c >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) ffff828c801000b5 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> ffff830079f18000 >> (XEN) Xen call trace: >> (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >> (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 >> (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 >> (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 >> (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 >> (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 >> (XEN) >> (XEN) Pagetable walk from ffff828bfff5aff8: >> (XEN) L4[0x105] = 000000007be7a027 5555555555555555 >> (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 >> (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 >> (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff >> (XEN) >> (XEN) **************************************** >> (XEN) Panic on CPU 0: >> (XEN) FATAL PAGE FAULT >> (XEN) [error_code=0000] >> (XEN) Faulting linear address: ffff828bfff5aff8 >> (XEN) **************************************** >> (XEN) >> (XEN) Reboot in five seconds... >> >> >>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Cui, Dexuan
2009-Jun-29 12:26 UTC
RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
It''s really weird... :-)
If I understand correctly, with iommu=1 in Xen:
1) on your "Dell E6400 machine", now you can''t reproduce the
panic any longer;
2) on your "Lenovo T400", you can reproduce the panic sometimes -- if
so, can you post the log (with the printk patch applied) when you meet with the
panic? Can you find out under what condition it''s easy to reproduce the
panic?
Thanks,
-- Dexuan
________________________________
From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com]
Sent: 2009年6月29日 20:15
To: Cui, Dexuan
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
iommu=1
Hi,
Tried again, this time with cold reboot, it worked again. I also checked the
lines you were talking about, and now they are:
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
Could it be that there is some kind of problem with the hardware, which makes it
sometimes report invalid values? (with the Lenovo T400 i''m testing, if
you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on boot,
so you must do a cold reboot).
Here is the log:
__ __ _____ _ _ ___
\ \/ /___ _ __ |___ /| || | / _ \
\ // _ \ ''_ \ |_ \| || |_| | | |
/ \ __/ | | | ___) |__ _| |_| |
/_/\_\___|_| |_| |____(_) |_|(_)___/
(XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) )
Mon Jun 29 11:04:57 EDT 2009
(XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd
(XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping
loglvl=all guest_loglvl=all
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f000 (usable)
(XEN) 000000000009f000 - 00000000000a0000 (reserved)
(XEN) 0000000000100000 - 000000007c04d400 (usable)
(XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS)
(XEN) 000000007c04f400 - 000000007f000000 (reserved)
(XEN) 00000000f8000000 - 00000000fc000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fed18000 - 00000000fed1c000 (reserved)
(XEN) 00000000fed20000 - 00000000fed90000 (reserved)
(XEN) 00000000feda0000 - 00000000feda6000 (reserved)
(XEN) 00000000fee00000 - 00000000fee10000 (reserved)
(XEN) 00000000ffe60000 - 0000000100000000 (reserved)
(XEN) System RAM: 1983MB (2031536kB)
(XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL )
(XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61)
(XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61)
(XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624)
(XEN) ACPI: FACS 7C060C00, 0040
(XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61)
(XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61)
(XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47)
(XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61)
(XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61)
(XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0)
(XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61)
(XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624)
(XEN) NUMA turned off
(XEN) Faking a node at 0000000000000000-000000007c04d000
(XEN) Domain heap initialised
(XEN) DMI 2.4 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0]
(XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) Processor #1 7:7 APIC version 20
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
(XEN) [VT-D]dmar.c:485: Host address width 36
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed10000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed11000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed12000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed13000
(XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
(XEN) Intel VT-d DMAR tables have been parsed.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2527.058 MHz processor.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) HVM: VMX enabled
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) mce_init: init bank1
(XEN) mce_init: init bank2
(XEN) mce_init: init bank3
(XEN) mce_init: init bank4
(XEN) mce_init: init bank5
(XEN) CPU0: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU0 has no CMCI support
(XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06
(XEN) Booting processor 1/1 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) mce_init: init bank1
(XEN) mce_init: init bank2
(XEN) mce_init: init bank4
(XEN) mce_init: init bank5
(XEN) CPU1: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
(XEN) clear_fault_bits: CAP_REG=0xc0000020630270
(XEN) clear_fault_bits: CAP_REG=0xc0000020630260
(XEN) clear_fault_bits: CAP_REG=0xc9008020e30260
(XEN) clear_fault_bits: CAP_REG=0xc9008020630260
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) HPET: 4 timers in total, 0 timers will be used for broadcast
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3
(XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
ffff828bfff57000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
ffff828bfff56000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
ffff828bfff55000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
ffff828bfff54000
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 ->
0xffffffff8070ab48
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be
allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48
(XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400
(XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18
(XEN) Start info: ffffffff81f28000->ffffffff81f284b4
(XEN) Page tables: ffffffff81f29000->ffffffff81f3c000
(XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000
(XEN) TOTAL: ffffffff80000000->ffffffff82000000
(XEN) ENTRY ADDRESS: ffffffff80200000
(XEN) Dom0 has maximum 2 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times
to switch input to Xen)
(XEN) Freed 132kB init memory.
Tom
2009/6/29 Cui, Dexuan
<dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>>
So, if I understand correctly, it''s really strange you can''t
reproduce the panic now.
With only the first printk patch applied (and don''t comment the
clear_fault_bits()), can you try a cold rebooting the host (plug the power cord
off from the host; wait several minutes; plug the cord in and tunn on the host)?
And the log?
BTW, in your original mail, the log shows:
(XEN) Intel VT-d Queued Invalidation supported.
(XEN) Intel VT-d Interrupt Remapping supported.
But in your this new log, it shows:
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
Are you sure you have been operating on the same host? :-)
-- Dexuan
________________________________
From: Tom Rotenberg
[mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>]
Sent: 2009年6月29日 17:01
To: Cui, Dexuan
Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>
Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
iommu=1
Hi,
Well, just tried what you suggested, and here are the results:
* If i comment the ''clear_fault_bits()'' call - then Xen boots
ok, with iommu=1
The strange thing, is that after it booted one time ok (with the second patch),
then it continued to boot ok, even if i reverted the second patch.
The relevant lines in the log are:
(XEN) clear_fault_bits: CAP_REG=0xc0000020630270
(XEN) clear_fault_bits: CAP_REG=0xc0000020630260
(XEN) clear_fault_bits: CAP_REG=0xc9008020e30260
(XEN) clear_fault_bits: CAP_REG=0xc9008020630260
Anyway, here is the full Xen log, with the first patch (print cap''s)
applied:
__ __ _____ _ _ ___
\ \/ /___ _ __ |___ /| || | / _ \
\ // _ \ ''_ \ |_ \| || |_| | | |
/ \ __/ | | | ___) |__ _| |_| |
/_/\_\___|_| |_| |____(_) |_|(_)___/
(XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) )
Mon Jun 29 07:53:38 EDT 2009
(XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd
(XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping
loglvl=all guest_loglvl=all
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f000 (usable)
(XEN) 000000000009f000 - 00000000000a0000 (reserved)
(XEN) 0000000000100000 - 000000007c04d400 (usable)
(XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS)
(XEN) 000000007c04f400 - 000000007f000000 (reserved)
(XEN) 00000000f8000000 - 00000000fc000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fed18000 - 00000000fed1c000 (reserved)
(XEN) 00000000fed20000 - 00000000fed90000 (reserved)
(XEN) 00000000feda0000 - 00000000feda6000 (reserved)
(XEN) 00000000fee00000 - 00000000fee10000 (reserved)
(XEN) 00000000ffe60000 - 0000000100000000 (reserved)
(XEN) System RAM: 1983MB (2031536kB)
(XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL )
(XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61)
(XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61)
(XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624)
(XEN) ACPI: FACS 7C060C00, 0040
(XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61)
(XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61)
(XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47)
(XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61)
(XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61)
(XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0)
(XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61)
(XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624)
(XEN) NUMA turned off
(XEN) Faking a node at 0000000000000000-000000007c04d000
(XEN) Domain heap initialised
(XEN) DMI 2.4 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0]
(XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) Processor #1 7:7 APIC version 20
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
(XEN) [VT-D]dmar.c:485: Host address width 36
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed10000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed11000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed12000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed13000
(XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
(XEN) Intel VT-d DMAR tables have been parsed.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2527.068 MHz processor.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) HVM: VMX enabled
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) mce_init: init bank1
(XEN) mce_init: init bank2
(XEN) mce_init: init bank3
(XEN) mce_init: init bank4
(XEN) mce_init: init bank5
(XEN) CPU0: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU0 has no CMCI support
(XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06
(XEN) Booting processor 1/1 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) mce_init: init bank1
(XEN) mce_init: init bank2
(XEN) mce_init: init bank4
(XEN) mce_init: init bank5
(XEN) CPU1: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
(XEN) clear_fault_bits: CAP_REG=0xc0000020630270
(XEN) clear_fault_bits: CAP_REG=0xc0000020630260
(XEN) clear_fault_bits: CAP_REG=0xc9008020e30260
(XEN) clear_fault_bits: CAP_REG=0xc9008020630260
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) HPET: 4 timers in total, 0 timers will be used for broadcast
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3
(XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
ffff828bfff57000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
ffff828bfff56000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
ffff828bfff55000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
ffff828bfff54000
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 ->
0xffffffff8070ab48
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be
allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48
(XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400
(XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18
(XEN) Start info: ffffffff81f28000->ffffffff81f284b4
(XEN) Page tables: ffffffff81f29000->ffffffff81f3c000
(XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000
(XEN) TOTAL: ffffffff80000000->ffffffff82000000
(XEN) ENTRY ADDRESS: ffffffff80200000
(XEN) Dom0 has maximum 2 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times
to switch input to Xen)
(XEN) Freed 132kB init memory.
Tom
2009/6/29 Cui, Dexuan
<dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>>
diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100
+++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800
@@ -1693,7 +1693,7 @@ static int init_vtd_hw(void)
}
dma_msi_data_init(iommu, iommu->vector);
dma_msi_addr_init(iommu, cpu_physical_id(first_cpu(cpu_online_map)));
- clear_fault_bits(iommu);
+ //clear_fault_bits(iommu);
dmar_writel(iommu->reg, DMAR_FECTL_REG, 0);
/* initialize flush functions */
And, can you comment the invocation of clear_fault_bits() in init_vtd_hw() like
above and see if the host can boot fine?
Thanks,
-- Dexuan
-----Original Message-----
From: Cui, Dexuan
Sent: 2009?6?29? 14:30
To: ''Tom Rotenberg'';
xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>
Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
iommu=1
diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100
+++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800
@@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm
{
u64 val;
+ printk("clear_fault_bits: CAP_REG=0x%llx\n",
+ (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG));
val = dmar_readq(
iommu->reg,
cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8);
Hi Tom,
Can you please apply the above patch and send us the entire log?
(I don''t have the same host, so I can''t try it myself.)
Thanks,
-- Dexuan
________________________________
From:
xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>
[mailto:xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>]
On Behalf Of Tom Rotenberg
Sent: 2009?6?28? 22:56
To: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>
Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
Hi,
I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to boot xen
with a ''iommu=1'' parameter in the boot line, Xen fails to
boot. I have also tried to boot with and without the
''iommu_inclusive_mapping'' boot option, but it didn''t
seem to help at all.
Does anyone knows how to solve this issue?
These are the last messages in the UART:
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush
(XEN) Intel VT-d Snoop Control supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation supported.
(XEN) Intel VT-d Interrupt Remapping supported.
(XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor
(XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020
(XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90
(XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110
(XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005
(XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc
(XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen stack trace from rsp=ffff828c8026fda0:
(XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490
(XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000
(XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f
(XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60
(XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978
(XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40
(XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000
(XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003
(XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000
(XEN) Xen call trace:
(XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
(XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0
(XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610
(XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110
(XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0
(XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3
(XEN)
(XEN) Pagetable walk from ffff828bfff5aff8:
(XEN) L4[0x105] = 000000007be7a027 5555555555555555
(XEN) L3[0x02f] = 000000007c04b063 5555555555555555
(XEN) L2[0x1ff] = 000000007c04a063 5555555555555555
(XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0000]
(XEN) Faulting linear address: ffff828bfff5aff8
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Jun-29 12:37 UTC
Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
1) Yep, i can''t reproduce it anymore. Maybe i''ll try it with another Dell R6400 i have here. 2) On my Lenovo T400, i don''t remember the exact crash, and i don''t have the log of that crash, i just remember something about "DMAR ... failure", which happened, after i turned on the IOMMU, and didn''t perform a cold reboot on that machine. Do u have any idea why these problems happen? Does anyone else working with Dell E6400 had this problem? Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com>> It''s really weird... :-) > > If I understand correctly, with iommu=1 in Xen: > 1) on your "Dell E6400 machine", now you can''t reproduce the panic any > longer; > 2) on your "Lenovo T400", you can reproduce the panic sometimes -- if so, > can you post the log (with the printk patch applied) when you meet with the > panic? Can you find out under what condition it''s easy to reproduce the > panic? > > > Thanks, > -- Dexuan > > > ------------------------------ > *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] > *Sent:* 2009年6月29日 20:15 > > *To:* Cui, Dexuan > *Cc:* xen-devel@lists.xensource.com > *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, > with iommu=1 > > Hi, > > Tried again, this time with cold reboot, it worked again. I also checked > the lines you were talking about, and now they are: > (XEN) Intel VT-d Snoop Control not supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation not supported. > (XEN) Intel VT-d Interrupt Remapping not supported. > > Could it be that there is some kind of problem with the hardware, which > makes it sometimes report invalid values? (with the Lenovo T400 i''m testing, > if you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on > boot, so you must do a cold reboot). > > Here is the log: > __ __ _____ _ _ ___ > \ \/ /___ _ __ |___ /| || | / _ \ > \ // _ \ ''_ \ |_ \| || |_| | | | > / \ __/ | | | ___) |__ _| |_| | > /_/\_\___|_| |_| |____(_) |_|(_)___/ > > (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian > 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009 > (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd > (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 > iommu_inclusive_mapping loglvl=all guest_loglvl=all > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 1 MBR signatures > (XEN) Found 1 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009f000 (usable) > (XEN) 000000000009f000 - 00000000000a0000 (reserved) > (XEN) 0000000000100000 - 000000007c04d400 (usable) > (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) > (XEN) 000000007c04f400 - 000000007f000000 (reserved) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec10000 (reserved) > (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) > (XEN) 00000000fed20000 - 00000000fed90000 (reserved) > (XEN) 00000000feda0000 - 00000000feda6000 (reserved) > (XEN) 00000000fee00000 - 00000000fee10000 (reserved) > (XEN) 00000000ffe60000 - 0000000100000000 (reserved) > (XEN) System RAM: 1983MB (2031536kB) > (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) > (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) > (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) > (XEN) ACPI: FACS 7C060C00, 0040 > (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) > (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) > (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL > 61) > (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL > 61) > (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) > (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) > (XEN) NUMA turned off > (XEN) Faking a node at 0000000000000000-000000007c04d000 > (XEN) Domain heap initialised > (XEN) DMI 2.4 present. > (XEN) Using APIC driver default > (XEN) ACPI: PM-Timer IO Port: 0x1008 > (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] > (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] > (XEN) ACPI: Local APIC address 0xfee00000 > (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) > (XEN) Processor #0 7:7 APIC version 20 > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) > (XEN) Processor #1 7:7 APIC version 20 > (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) > (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) > (XEN) ACPI: IRQ0 used by override. > (XEN) ACPI: IRQ2 used by override. > (XEN) ACPI: IRQ9 used by override. > (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs > (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 > (XEN) [VT-D]dmar.c:485: Host address width 36 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 > (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL > (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 > (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 > (XEN) Intel VT-d DMAR tables have been parsed. > (XEN) Using ACPI (MADT) for SMP configuration information > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Initializing CPU#0 > (XEN) Detected 2527.058 MHz processor. > (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K > (XEN) CPU: L2 cache: 6144K > (XEN) CPU: Physical Processor ID: 0 > (XEN) CPU: Processor Core ID: 0 > (XEN) VMX: Supported advanced features: > (XEN) - APIC MMIO access virtualisation > (XEN) - APIC TPR shadow > (XEN) - Virtual NMI > (XEN) - MSR direct-access bitmap > (XEN) HVM: VMX enabled > (XEN) Intel machine check reporting enabled on CPU#0. > (XEN) mce_init: init bank1 > (XEN) mce_init: init bank2 > (XEN) mce_init: init bank3 > (XEN) mce_init: init bank4 > (XEN) mce_init: init bank5 > (XEN) CPU0: Thermal monitoring enabled (TM2) > (XEN) CMCI: CPU0 has no CMCI support > (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 > (XEN) Booting processor 1/1 eip 8c000 > (XEN) Initializing CPU#1 > (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K > (XEN) CPU: L2 cache: 6144K > (XEN) CPU: Physical Processor ID: 0 > (XEN) CPU: Processor Core ID: 1 > (XEN) Intel machine check reporting enabled on CPU#1. > (XEN) mce_init: init bank1 > (XEN) mce_init: init bank2 > (XEN) mce_init: init bank4 > (XEN) mce_init: init bank5 > (XEN) CPU1: Thermal monitoring enabled (TM2) > (XEN) CMCI: CPU1 has no CMCI support > (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 > (XEN) Total of 2 processors activated. > (XEN) ENABLING IO-APIC IRQs > (XEN) -> Using new ACK method > (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 > (XEN) checking TSC synchronization across 2 CPUs: passed. > (XEN) Platform timer is 14.318MHz HPET > (XEN) Brought up 2 CPUs > (XEN) microcode.c:73:d32767 microcode: CPU1 resumed > (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush > (XEN) Intel VT-d Snoop Control not supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation not supported. > (XEN) Intel VT-d Interrupt Remapping not supported. > (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 > (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 > (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 > (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 > (XEN) I/O virtualisation enabled > (XEN) I/O virtualisation for PV guests disabled > (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast > (XEN) ACPI sleep modes: S3 > (XEN) mcheck_poll: Machine check polling timer started. > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 > (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 > (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg > ffff828bfff57000 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg > ffff828bfff56000 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg > ffff828bfff55000 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg > ffff828bfff54000 > (XEN) *** LOADING DOMAIN 0 *** > (XEN) Xen kernel: 64-bit, lsb, compat32 > (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> > 0xffffffff8070ab48 > (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to > be allocated) > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 > (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 > (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 > (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 > (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 > (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 > (XEN) TOTAL: ffffffff80000000->ffffffff82000000 > (XEN) ENTRY ADDRESS: ffffffff80200000 > (XEN) Dom0 has maximum 2 VCPUs > (XEN) Scrubbing Free RAM: .done. > (XEN) Xen trace buffers: disabled > (XEN) Std. Loglevel: All > (XEN) Guest Loglevel: All > (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input > to Xen) > (XEN) Freed 132kB init memory. > > Tom > > 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> > >> So, if I understand correctly, it''s really strange you can''t reproduce >> the panic now. >> >> With only the first printk patch applied (and don''t comment the >> clear_fault_bits()), can you try a cold rebooting the host (plug the power >> cord off from the host; wait several minutes; plug the cord in and tunn on >> the host)? And the log? >> >> BTW, in your original mail, the log shows: >> (XEN) Intel VT-d Queued Invalidation supported. >> (XEN) Intel VT-d Interrupt Remapping supported. >> >> But in your this new log, it shows: >> (XEN) Intel VT-d Queued Invalidation* not* supported. >> (XEN) Intel VT-d Interrupt Remapping* not* supported. >> Are you sure you have been operating on the same host? :-) >> >> -- Dexuan >> >> ------------------------------ >> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >> *Sent:* 2009年6月29日 17:01 >> *To:* Cui, Dexuan >> *Cc:* xen-devel@lists.xensource.com >> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >> with iommu=1 >> >> Hi, >> >> Well, just tried what you suggested, and here are the results: >> * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with >> iommu=1 >> The strange thing, is that after it booted one time ok (with the second >> patch), then it continued to boot ok, even if i reverted the second patch. >> >> The relevant lines in the log are: >> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >> >> Anyway, here is the full Xen log, with the first patch (print cap''s) >> applied: >> >> __ __ _____ _ _ ___ >> \ \/ /___ _ __ |___ /| || | / _ \ >> \ // _ \ ''_ \ |_ \| || |_| | | | >> / \ __/ | | | ___) |__ _| |_| | >> /_/\_\___|_| |_| |____(_) |_|(_)___/ >> >> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian >> 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 >> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd >> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 >> iommu_inclusive_mapping loglvl=all guest_loglvl=all >> (XEN) Video information: >> (XEN) VGA is text mode 80x25, font 8x16 >> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >> (XEN) Disc information: >> (XEN) Found 1 MBR signatures >> (XEN) Found 1 EDD information structures >> (XEN) Xen-e820 RAM map: >> (XEN) 0000000000000000 - 000000000009f000 (usable) >> (XEN) 000000000009f000 - 00000000000a0000 (reserved) >> (XEN) 0000000000100000 - 000000007c04d400 (usable) >> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) >> (XEN) 000000007c04f400 - 000000007f000000 (reserved) >> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >> (XEN) 00000000fec00000 - 00000000fec10000 (reserved) >> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) >> (XEN) 00000000fed20000 - 00000000fed90000 (reserved) >> (XEN) 00000000feda0000 - 00000000feda6000 (reserved) >> (XEN) 00000000fee00000 - 00000000fee10000 (reserved) >> (XEN) 00000000ffe60000 - 0000000100000000 (reserved) >> (XEN) System RAM: 1983MB (2031536kB) >> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) >> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL >> 20050624) >> (XEN) ACPI: FACS 7C060C00, 0040 >> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL >> 61) >> (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL >> 47) >> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL >> 0) >> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL >> 20050624) >> (XEN) NUMA turned off >> (XEN) Faking a node at 0000000000000000-000000007c04d000 >> (XEN) Domain heap initialised >> (XEN) DMI 2.4 present. >> (XEN) Using APIC driver default >> (XEN) ACPI: PM-Timer IO Port: 0x1008 >> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] >> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] >> (XEN) ACPI: Local APIC address 0xfee00000 >> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) >> (XEN) Processor #0 7:7 APIC version 20 >> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) >> (XEN) Processor #1 7:7 APIC version 20 >> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) >> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) >> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) >> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 >> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) >> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) >> (XEN) ACPI: IRQ0 used by override. >> (XEN) ACPI: IRQ2 used by override. >> (XEN) ACPI: IRQ9 used by override. >> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs >> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 >> (XEN) [VT-D]dmar.c:485: Host address width 36 >> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 >> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >> (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >> (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 >> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 >> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL >> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 >> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >> (XEN) Intel VT-d DMAR tables have been parsed. >> (XEN) Using ACPI (MADT) for SMP configuration information >> (XEN) Using scheduler: SMP Credit Scheduler (credit) >> (XEN) Initializing CPU#0 >> (XEN) Detected 2527.068 MHz processor. >> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >> (XEN) CPU: L2 cache: 6144K >> (XEN) CPU: Physical Processor ID: 0 >> (XEN) CPU: Processor Core ID: 0 >> (XEN) VMX: Supported advanced features: >> (XEN) - APIC MMIO access virtualisation >> (XEN) - APIC TPR shadow >> (XEN) - Virtual NMI >> (XEN) - MSR direct-access bitmap >> (XEN) HVM: VMX enabled >> (XEN) Intel machine check reporting enabled on CPU#0. >> (XEN) mce_init: init bank1 >> (XEN) mce_init: init bank2 >> (XEN) mce_init: init bank3 >> (XEN) mce_init: init bank4 >> (XEN) mce_init: init bank5 >> (XEN) CPU0: Thermal monitoring enabled (TM2) >> (XEN) CMCI: CPU0 has no CMCI support >> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >> (XEN) Booting processor 1/1 eip 8c000 >> (XEN) Initializing CPU#1 >> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >> (XEN) CPU: L2 cache: 6144K >> (XEN) CPU: Physical Processor ID: 0 >> (XEN) CPU: Processor Core ID: 1 >> (XEN) Intel machine check reporting enabled on CPU#1. >> (XEN) mce_init: init bank1 >> (XEN) mce_init: init bank2 >> (XEN) mce_init: init bank4 >> (XEN) mce_init: init bank5 >> (XEN) CPU1: Thermal monitoring enabled (TM2) >> (XEN) CMCI: CPU1 has no CMCI support >> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >> (XEN) Total of 2 processors activated. >> (XEN) ENABLING IO-APIC IRQs >> (XEN) -> Using new ACK method >> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 >> (XEN) checking TSC synchronization across 2 CPUs: passed. >> (XEN) Platform timer is 14.318MHz HPET >> (XEN) Brought up 2 CPUs >> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >> (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush >> (XEN) Intel VT-d Snoop Control not supported. >> (XEN) Intel VT-d DMA Passthrough not supported. >> (XEN) Intel VT-d Queued Invalidation not supported. >> (XEN) Intel VT-d Interrupt Remapping not supported. >> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >> (XEN) I/O virtualisation enabled >> (XEN) I/O virtualisation for PV guests disabled >> (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast >> (XEN) ACPI sleep modes: S3 >> (XEN) mcheck_poll: Machine check polling timer started. >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 >> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 >> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 >> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >> ffff828bfff57000 >> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >> ffff828bfff56000 >> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >> ffff828bfff55000 >> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >> ffff828bfff54000 >> (XEN) *** LOADING DOMAIN 0 *** >> (XEN) Xen kernel: 64-bit, lsb, compat32 >> (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> >> 0xffffffff8070ab48 >> (XEN) PHYSICAL MEMORY ARRANGEMENT: >> (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to >> be allocated) >> (XEN) VIRTUAL MEMORY ARRANGEMENT: >> (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 >> (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 >> (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 >> (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 >> (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 >> (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 >> (XEN) TOTAL: ffffffff80000000->ffffffff82000000 >> (XEN) ENTRY ADDRESS: ffffffff80200000 >> (XEN) Dom0 has maximum 2 VCPUs >> (XEN) Scrubbing Free RAM: .done. >> (XEN) Xen trace buffers: disabled >> (XEN) Std. Loglevel: All >> (XEN) Guest Loglevel: All >> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input >> to Xen) >> (XEN) Freed 132kB init memory. >> >> Tom >> >> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >> >>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c >>> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 >>> +0100 >>> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 >>> +0800 >>> @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) >>> } >>> dma_msi_data_init(iommu, iommu->vector); >>> dma_msi_addr_init(iommu, >>> cpu_physical_id(first_cpu(cpu_online_map))); >>> - clear_fault_bits(iommu); >>> + //clear_fault_bits(iommu); >>> dmar_writel(iommu->reg, DMAR_FECTL_REG, 0); >>> >>> /* initialize flush functions */ >>> >>> And, can you comment the invocation of clear_fault_bits() in >>> init_vtd_hw() like above and see if the host can boot fine? >>> >>> Thanks, >>> -- Dexuan >>> >>> >>> >>> -----Original Message----- >>> From: Cui, Dexuan >>> Sent: 2009?6?29? 14:30 >>> To: ''Tom Rotenberg''; xen-devel@lists.xensource.com >>> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >>> with iommu=1 >>> >>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c >>> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 >>> +0100 >>> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 >>> +0800 >>> @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm >>> { >>> u64 val; >>> >>> + printk("clear_fault_bits: CAP_REG=0x%llx\n", >>> + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); >>> val = dmar_readq( >>> iommu->reg, >>> cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8); >>> >>> Hi Tom, >>> Can you please apply the above patch and send us the entire log? >>> (I don''t have the same host, so I can''t try it myself.) >>> >>> Thanks, >>> -- Dexuan >>> >>> ________________________________ >>> >>> From: xen-devel-bounces@lists.xensource.com [mailto: >>> xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >>> Sent: 2009?6?28? 22:56 >>> To: xen-devel@lists.xensource.com >>> Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with >>> iommu=1 >>> >>> >>> Hi, >>> >>> I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to >>> boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. I >>> have also tried to boot with and without the ''iommu_inclusive_mapping'' boot >>> option, but it didn''t seem to help at all. >>> >>> Does anyone knows how to solve this issue? >>> >>> These are the last messages in the UART: >>> >>> (XEN) Brought up 2 CPUs >>> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >>> (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush >>> (XEN) Intel VT-d Snoop Control supported. >>> (XEN) Intel VT-d DMA Passthrough not supported. >>> (XEN) Intel VT-d Queued Invalidation supported. >>> (XEN) Intel VT-d Interrupt Remapping supported. >>> (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- >>> (XEN) CPU: 0 >>> (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >>> (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor >>> (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: >>> 0000000000000020 >>> (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: >>> ffff830078f82f90 >>> (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: >>> ffff83007be2c110 >>> (XEN) r9: 000000000000001a r10: 0000000000000005 r11: >>> 0000000000000005 >>> (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: >>> ffff830078f82fcc >>> (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: >>> 00000000000026f0 >>> (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 >>> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 >>> (XEN) Xen stack trace from rsp=ffff828c8026fda0: >>> (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f >>> ffff828c80204490 >>> (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 >>> 0000000000000000 >>> (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 >>> 000000000000001f >>> (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 >>> ffff828c80205f60 >>> (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 >>> ffff828c80227978 >>> (XEN) ffff828c8021a589 0000000000000000 0000000000000000 >>> 0000000000000000 >>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 >>> (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 >>> 000000000008bf40 >>> (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 >>> 0000000000000000 >>> (XEN) ffffffff00000000 0000000800000000 000000010000006e >>> 0000000000000003 >>> (XEN) 00000000000002f8 0000000000000000 0000000000000000 >>> 0000000000067e2c >>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 >>> (XEN) ffff828c801000b5 0000000000000000 0000000000000000 >>> 0000000000000000 >>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 >>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 >>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 >>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 >>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 >>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>> ffff830079f18000 >>> (XEN) Xen call trace: >>> (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >>> (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 >>> (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 >>> (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 >>> (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 >>> (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 >>> (XEN) >>> (XEN) Pagetable walk from ffff828bfff5aff8: >>> (XEN) L4[0x105] = 000000007be7a027 5555555555555555 >>> (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 >>> (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 >>> (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff >>> (XEN) >>> (XEN) **************************************** >>> (XEN) Panic on CPU 0: >>> (XEN) FATAL PAGE FAULT >>> (XEN) [error_code=0000] >>> (XEN) Faulting linear address: ffff828bfff5aff8 >>> (XEN) **************************************** >>> (XEN) >>> (XEN) Reboot in five seconds... >>> >>> >>>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Cui, Dexuan
2009-Jun-29 12:56 UTC
RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
I have no idea about it.> (XEN) Intel VT-d Snoop Control supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported.This is from the log of your original mail. We can see SC/QI/IR are supported, but in your later log, we can see they are not supported. I''m curious where the first log comes from. :-) -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com] Sent: 2009年6月29日 20:38 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 1) Yep, i can''t reproduce it anymore. Maybe i''ll try it with another Dell R6400 i have here. 2) On my Lenovo T400, i don''t remember the exact crash, and i don''t have the log of that crash, i just remember something about "DMAR ... failure", which happened, after i turned on the IOMMU, and didn''t perform a cold reboot on that machine. Do u have any idea why these problems happen? Does anyone else working with Dell E6400 had this problem? Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> It''s really weird... :-) If I understand correctly, with iommu=1 in Xen: 1) on your "Dell E6400 machine", now you can''t reproduce the panic any longer; 2) on your "Lenovo T400", you can reproduce the panic sometimes -- if so, can you post the log (with the printk patch applied) when you meet with the panic? Can you find out under what condition it''s easy to reproduce the panic? Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 20:15 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, Tried again, this time with cold reboot, it worked again. I also checked the lines you were talking about, and now they are: (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. Could it be that there is some kind of problem with the hardware, which makes it sometimes report invalid values? (with the Lenovo T400 i''m testing, if you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on boot, so you must do a cold reboot). Here is the log: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.058 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 (XEN) TOTAL: ffffffff80000000->ffffffff82000000 (XEN) ENTRY ADDRESS: ffffffff80200000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 132kB init memory. Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> So, if I understand correctly, it''s really strange you can''t reproduce the panic now. With only the first printk patch applied (and don''t comment the clear_fault_bits()), can you try a cold rebooting the host (plug the power cord off from the host; wait several minutes; plug the cord in and tunn on the host)? And the log? BTW, in your original mail, the log shows: (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. But in your this new log, it shows: (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. Are you sure you have been operating on the same host? :-) -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 17:01 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, Well, just tried what you suggested, and here are the results: * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with iommu=1 The strange thing, is that after it booted one time ok (with the second patch), then it continued to boot ok, even if i reverted the second patch. The relevant lines in the log are: (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 Anyway, here is the full Xen log, with the first patch (print cap''s) applied: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.068 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 (XEN) TOTAL: ffffffff80000000->ffffffff82000000 (XEN) ENTRY ADDRESS: ffffffff80200000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 132kB init memory. Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800 @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) } dma_msi_data_init(iommu, iommu->vector); dma_msi_addr_init(iommu, cpu_physical_id(first_cpu(cpu_online_map))); - clear_fault_bits(iommu); + //clear_fault_bits(iommu); dmar_writel(iommu->reg, DMAR_FECTL_REG, 0); /* initialize flush functions */ And, can you comment the invocation of clear_fault_bits() in init_vtd_hw() like above and see if the host can boot fine? Thanks, -- Dexuan -----Original Message----- From: Cui, Dexuan Sent: 2009?6?29? 14:30 To: ''Tom Rotenberg''; xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800 @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm { u64 val; + printk("clear_fault_bits: CAP_REG=0x%llx\n", + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); val = dmar_readq( iommu->reg, cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8); Hi Tom, Can you please apply the above patch and send us the entire log? (I don''t have the same host, so I can''t try it myself.) Thanks, -- Dexuan ________________________________ From: xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com> [mailto:xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>] On Behalf Of Tom Rotenberg Sent: 2009?6?28? 22:56 To: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. I have also tried to boot with and without the ''iommu_inclusive_mapping'' boot option, but it didn''t seem to help at all. Does anyone knows how to solve this issue? These are the last messages in the UART: (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fda0: (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978 (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40 (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Jun-29 12:59 UTC
Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
I''m almost 100% sure, that this was from the same machine (since i have only 1 such Dell currently on my desk :). However, this log was generated just after turning on the IOMMU (without cold boot). So, let me check if it happened because of this. I''ll check it now... 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com>> I have no idea about it. > > > (XEN) Intel VT-d Snoop Control supported. > > (XEN) Intel VT-d DMA Passthrough not supported. > > (XEN) Intel VT-d Queued Invalidation supported. > > (XEN) Intel VT-d Interrupt Remapping supported. > This is from the log of your original mail. We can see SC/QI/IR are > supported, but in your later log, we can see they are not supported. > I''m curious where the first log comes from. :-) > > -- Dexuan > > ------------------------------ > *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] > *Sent:* 2009年6月29日 20:38 > > *To:* Cui, Dexuan > *Cc:* xen-devel@lists.xensource.com > *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, > with iommu=1 > > > 1) Yep, i can''t reproduce it anymore. Maybe i''ll try it with another Dell > R6400 i have here. > 2) On my Lenovo T400, i don''t remember the exact crash, and i don''t have > the log of that crash, i just remember something about "DMAR ... failure", > which happened, after i turned on the IOMMU, and didn''t perform a cold > reboot on that machine. > > Do u have any idea why these problems happen? > Does anyone else working with Dell E6400 had this problem? > > Tom > > 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> > >> It''s really weird... :-) >> >> If I understand correctly, with iommu=1 in Xen: >> 1) on your "Dell E6400 machine", now you can''t reproduce the panic any >> longer; >> 2) on your "Lenovo T400", you can reproduce the panic sometimes -- if so, >> can you post the log (with the printk patch applied) when you meet with the >> panic? Can you find out under what condition it''s easy to reproduce the >> panic? >> >> >> Thanks, >> -- Dexuan >> >> >> ------------------------------ >> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >> *Sent:* 2009年6月29日 20:15 >> >> *To:* Cui, Dexuan >> *Cc:* xen-devel@lists.xensource.com >> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >> with iommu=1 >> >> Hi, >> >> Tried again, this time with cold reboot, it worked again. I also checked >> the lines you were talking about, and now they are: >> (XEN) Intel VT-d Snoop Control not supported. >> (XEN) Intel VT-d DMA Passthrough not supported. >> (XEN) Intel VT-d Queued Invalidation not supported. >> (XEN) Intel VT-d Interrupt Remapping not supported. >> >> Could it be that there is some kind of problem with the hardware, which >> makes it sometimes report invalid values? (with the Lenovo T400 i''m testing, >> if you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on >> boot, so you must do a cold reboot). >> >> Here is the log: >> __ __ _____ _ _ ___ >> \ \/ /___ _ __ |___ /| || | / _ \ >> \ // _ \ ''_ \ |_ \| || |_| | | | >> / \ __/ | | | ___) |__ _| |_| | >> /_/\_\___|_| |_| |____(_) |_|(_)___/ >> >> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian >> 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009 >> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd >> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 >> iommu_inclusive_mapping loglvl=all guest_loglvl=all >> (XEN) Video information: >> (XEN) VGA is text mode 80x25, font 8x16 >> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >> (XEN) Disc information: >> (XEN) Found 1 MBR signatures >> (XEN) Found 1 EDD information structures >> (XEN) Xen-e820 RAM map: >> (XEN) 0000000000000000 - 000000000009f000 (usable) >> (XEN) 000000000009f000 - 00000000000a0000 (reserved) >> (XEN) 0000000000100000 - 000000007c04d400 (usable) >> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) >> (XEN) 000000007c04f400 - 000000007f000000 (reserved) >> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >> (XEN) 00000000fec00000 - 00000000fec10000 (reserved) >> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) >> (XEN) 00000000fed20000 - 00000000fed90000 (reserved) >> (XEN) 00000000feda0000 - 00000000feda6000 (reserved) >> (XEN) 00000000fee00000 - 00000000fee10000 (reserved) >> (XEN) 00000000ffe60000 - 0000000100000000 (reserved) >> (XEN) System RAM: 1983MB (2031536kB) >> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) >> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL >> 20050624) >> (XEN) ACPI: FACS 7C060C00, 0040 >> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL >> 61) >> (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL >> 47) >> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL >> 0) >> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL >> 20050624) >> (XEN) NUMA turned off >> (XEN) Faking a node at 0000000000000000-000000007c04d000 >> (XEN) Domain heap initialised >> (XEN) DMI 2.4 present. >> (XEN) Using APIC driver default >> (XEN) ACPI: PM-Timer IO Port: 0x1008 >> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] >> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] >> (XEN) ACPI: Local APIC address 0xfee00000 >> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) >> (XEN) Processor #0 7:7 APIC version 20 >> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) >> (XEN) Processor #1 7:7 APIC version 20 >> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) >> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) >> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) >> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 >> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) >> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) >> (XEN) ACPI: IRQ0 used by override. >> (XEN) ACPI: IRQ2 used by override. >> (XEN) ACPI: IRQ9 used by override. >> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs >> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 >> (XEN) [VT-D]dmar.c:485: Host address width 36 >> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 >> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >> (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >> (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 >> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 >> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL >> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 >> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >> (XEN) Intel VT-d DMAR tables have been parsed. >> (XEN) Using ACPI (MADT) for SMP configuration information >> (XEN) Using scheduler: SMP Credit Scheduler (credit) >> (XEN) Initializing CPU#0 >> (XEN) Detected 2527.058 MHz processor. >> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >> (XEN) CPU: L2 cache: 6144K >> (XEN) CPU: Physical Processor ID: 0 >> (XEN) CPU: Processor Core ID: 0 >> (XEN) VMX: Supported advanced features: >> (XEN) - APIC MMIO access virtualisation >> (XEN) - APIC TPR shadow >> (XEN) - Virtual NMI >> (XEN) - MSR direct-access bitmap >> (XEN) HVM: VMX enabled >> (XEN) Intel machine check reporting enabled on CPU#0. >> (XEN) mce_init: init bank1 >> (XEN) mce_init: init bank2 >> (XEN) mce_init: init bank3 >> (XEN) mce_init: init bank4 >> (XEN) mce_init: init bank5 >> (XEN) CPU0: Thermal monitoring enabled (TM2) >> (XEN) CMCI: CPU0 has no CMCI support >> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >> (XEN) Booting processor 1/1 eip 8c000 >> (XEN) Initializing CPU#1 >> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >> (XEN) CPU: L2 cache: 6144K >> (XEN) CPU: Physical Processor ID: 0 >> (XEN) CPU: Processor Core ID: 1 >> (XEN) Intel machine check reporting enabled on CPU#1. >> (XEN) mce_init: init bank1 >> (XEN) mce_init: init bank2 >> (XEN) mce_init: init bank4 >> (XEN) mce_init: init bank5 >> (XEN) CPU1: Thermal monitoring enabled (TM2) >> (XEN) CMCI: CPU1 has no CMCI support >> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >> (XEN) Total of 2 processors activated. >> (XEN) ENABLING IO-APIC IRQs >> (XEN) -> Using new ACK method >> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 >> (XEN) checking TSC synchronization across 2 CPUs: passed. >> (XEN) Platform timer is 14.318MHz HPET >> (XEN) Brought up 2 CPUs >> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >> (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush >> (XEN) Intel VT-d Snoop Control not supported. >> (XEN) Intel VT-d DMA Passthrough not supported. >> (XEN) Intel VT-d Queued Invalidation not supported. >> (XEN) Intel VT-d Interrupt Remapping not supported. >> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >> (XEN) I/O virtualisation enabled >> (XEN) I/O virtualisation for PV guests disabled >> (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast >> (XEN) ACPI sleep modes: S3 >> (XEN) mcheck_poll: Machine check polling timer started. >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 >> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 >> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 >> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 >> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >> ffff828bfff57000 >> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >> ffff828bfff56000 >> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >> ffff828bfff55000 >> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >> ffff828bfff54000 >> (XEN) *** LOADING DOMAIN 0 *** >> (XEN) Xen kernel: 64-bit, lsb, compat32 >> (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> >> 0xffffffff8070ab48 >> (XEN) PHYSICAL MEMORY ARRANGEMENT: >> (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to >> be allocated) >> (XEN) VIRTUAL MEMORY ARRANGEMENT: >> (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 >> (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 >> (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 >> (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 >> (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 >> (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 >> (XEN) TOTAL: ffffffff80000000->ffffffff82000000 >> (XEN) ENTRY ADDRESS: ffffffff80200000 >> (XEN) Dom0 has maximum 2 VCPUs >> (XEN) Scrubbing Free RAM: .done. >> (XEN) Xen trace buffers: disabled >> (XEN) Std. Loglevel: All >> (XEN) Guest Loglevel: All >> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input >> to Xen) >> (XEN) Freed 132kB init memory. >> >> Tom >> >> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >> >>> So, if I understand correctly, it''s really strange you can''t reproduce >>> the panic now. >>> >>> With only the first printk patch applied (and don''t comment the >>> clear_fault_bits()), can you try a cold rebooting the host (plug the power >>> cord off from the host; wait several minutes; plug the cord in and tunn on >>> the host)? And the log? >>> >>> BTW, in your original mail, the log shows: >>> (XEN) Intel VT-d Queued Invalidation supported. >>> (XEN) Intel VT-d Interrupt Remapping supported. >>> >>> But in your this new log, it shows: >>> (XEN) Intel VT-d Queued Invalidation* not* supported. >>> (XEN) Intel VT-d Interrupt Remapping* not* supported. >>> Are you sure you have been operating on the same host? :-) >>> >>> -- Dexuan >>> >>> ------------------------------ >>> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >>> *Sent:* 2009年6月29日 17:01 >>> *To:* Cui, Dexuan >>> *Cc:* xen-devel@lists.xensource.com >>> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >>> with iommu=1 >>> >>> Hi, >>> >>> Well, just tried what you suggested, and here are the results: >>> * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with >>> iommu=1 >>> The strange thing, is that after it booted one time ok (with the second >>> patch), then it continued to boot ok, even if i reverted the second patch. >>> >>> The relevant lines in the log are: >>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >>> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >>> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >>> >>> Anyway, here is the full Xen log, with the first patch (print cap''s) >>> applied: >>> >>> __ __ _____ _ _ ___ >>> \ \/ /___ _ __ |___ /| || | / _ \ >>> \ // _ \ ''_ \ |_ \| || |_| | | | >>> / \ __/ | | | ___) |__ _| |_| | >>> /_/\_\___|_| |_| |____(_) |_|(_)___/ >>> >>> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian >>> 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 >>> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd >>> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 >>> iommu_inclusive_mapping loglvl=all guest_loglvl=all >>> (XEN) Video information: >>> (XEN) VGA is text mode 80x25, font 8x16 >>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >>> (XEN) Disc information: >>> (XEN) Found 1 MBR signatures >>> (XEN) Found 1 EDD information structures >>> (XEN) Xen-e820 RAM map: >>> (XEN) 0000000000000000 - 000000000009f000 (usable) >>> (XEN) 000000000009f000 - 00000000000a0000 (reserved) >>> (XEN) 0000000000100000 - 000000007c04d400 (usable) >>> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) >>> (XEN) 000000007c04f400 - 000000007f000000 (reserved) >>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >>> (XEN) 00000000fec00000 - 00000000fec10000 (reserved) >>> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) >>> (XEN) 00000000fed20000 - 00000000fed90000 (reserved) >>> (XEN) 00000000feda0000 - 00000000feda6000 (reserved) >>> (XEN) 00000000fee00000 - 00000000fee10000 (reserved) >>> (XEN) 00000000ffe60000 - 0000000100000000 (reserved) >>> (XEN) System RAM: 1983MB (2031536kB) >>> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) >>> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL >>> 61) >>> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL >>> 61) >>> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL >>> 20050624) >>> (XEN) ACPI: FACS 7C060C00, 0040 >>> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL >>> 61) >>> (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL >>> 61) >>> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL >>> 47) >>> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL >>> 61) >>> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL >>> 61) >>> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL >>> 0) >>> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL >>> 61) >>> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL >>> 20050624) >>> (XEN) NUMA turned off >>> (XEN) Faking a node at 0000000000000000-000000007c04d000 >>> (XEN) Domain heap initialised >>> (XEN) DMI 2.4 present. >>> (XEN) Using APIC driver default >>> (XEN) ACPI: PM-Timer IO Port: 0x1008 >>> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] >>> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] >>> (XEN) ACPI: Local APIC address 0xfee00000 >>> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) >>> (XEN) Processor #0 7:7 APIC version 20 >>> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) >>> (XEN) Processor #1 7:7 APIC version 20 >>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) >>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) >>> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) >>> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 >>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) >>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) >>> (XEN) ACPI: IRQ0 used by override. >>> (XEN) ACPI: IRQ2 used by override. >>> (XEN) ACPI: IRQ9 used by override. >>> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs >>> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 >>> (XEN) [VT-D]dmar.c:485: Host address width 36 >>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 >>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 >>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 >>> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL >>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 >>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>> (XEN) Intel VT-d DMAR tables have been parsed. >>> (XEN) Using ACPI (MADT) for SMP configuration information >>> (XEN) Using scheduler: SMP Credit Scheduler (credit) >>> (XEN) Initializing CPU#0 >>> (XEN) Detected 2527.068 MHz processor. >>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>> (XEN) CPU: L2 cache: 6144K >>> (XEN) CPU: Physical Processor ID: 0 >>> (XEN) CPU: Processor Core ID: 0 >>> (XEN) VMX: Supported advanced features: >>> (XEN) - APIC MMIO access virtualisation >>> (XEN) - APIC TPR shadow >>> (XEN) - Virtual NMI >>> (XEN) - MSR direct-access bitmap >>> (XEN) HVM: VMX enabled >>> (XEN) Intel machine check reporting enabled on CPU#0. >>> (XEN) mce_init: init bank1 >>> (XEN) mce_init: init bank2 >>> (XEN) mce_init: init bank3 >>> (XEN) mce_init: init bank4 >>> (XEN) mce_init: init bank5 >>> (XEN) CPU0: Thermal monitoring enabled (TM2) >>> (XEN) CMCI: CPU0 has no CMCI support >>> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >>> (XEN) Booting processor 1/1 eip 8c000 >>> (XEN) Initializing CPU#1 >>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>> (XEN) CPU: L2 cache: 6144K >>> (XEN) CPU: Physical Processor ID: 0 >>> (XEN) CPU: Processor Core ID: 1 >>> (XEN) Intel machine check reporting enabled on CPU#1. >>> (XEN) mce_init: init bank1 >>> (XEN) mce_init: init bank2 >>> (XEN) mce_init: init bank4 >>> (XEN) mce_init: init bank5 >>> (XEN) CPU1: Thermal monitoring enabled (TM2) >>> (XEN) CMCI: CPU1 has no CMCI support >>> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >>> (XEN) Total of 2 processors activated. >>> (XEN) ENABLING IO-APIC IRQs >>> (XEN) -> Using new ACK method >>> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 >>> (XEN) checking TSC synchronization across 2 CPUs: passed. >>> (XEN) Platform timer is 14.318MHz HPET >>> (XEN) Brought up 2 CPUs >>> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >>> (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush >>> (XEN) Intel VT-d Snoop Control not supported. >>> (XEN) Intel VT-d DMA Passthrough not supported. >>> (XEN) Intel VT-d Queued Invalidation not supported. >>> (XEN) Intel VT-d Interrupt Remapping not supported. >>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >>> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >>> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >>> (XEN) I/O virtualisation enabled >>> (XEN) I/O virtualisation for PV guests disabled >>> (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast >>> (XEN) ACPI sleep modes: S3 >>> (XEN) mcheck_poll: Machine check polling timer started. >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 >>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 >>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 >>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>> ffff828bfff57000 >>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>> ffff828bfff56000 >>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>> ffff828bfff55000 >>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>> ffff828bfff54000 >>> (XEN) *** LOADING DOMAIN 0 *** >>> (XEN) Xen kernel: 64-bit, lsb, compat32 >>> (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> >>> 0xffffffff8070ab48 >>> (XEN) PHYSICAL MEMORY ARRANGEMENT: >>> (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to >>> be allocated) >>> (XEN) VIRTUAL MEMORY ARRANGEMENT: >>> (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 >>> (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 >>> (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 >>> (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 >>> (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 >>> (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 >>> (XEN) TOTAL: ffffffff80000000->ffffffff82000000 >>> (XEN) ENTRY ADDRESS: ffffffff80200000 >>> (XEN) Dom0 has maximum 2 VCPUs >>> (XEN) Scrubbing Free RAM: .done. >>> (XEN) Xen trace buffers: disabled >>> (XEN) Std. Loglevel: All >>> (XEN) Guest Loglevel: All >>> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input >>> to Xen) >>> (XEN) Freed 132kB init memory. >>> >>> Tom >>> >>> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >>> >>>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c >>>> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 >>>> +0100 >>>> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 >>>> +0800 >>>> @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) >>>> } >>>> dma_msi_data_init(iommu, iommu->vector); >>>> dma_msi_addr_init(iommu, >>>> cpu_physical_id(first_cpu(cpu_online_map))); >>>> - clear_fault_bits(iommu); >>>> + //clear_fault_bits(iommu); >>>> dmar_writel(iommu->reg, DMAR_FECTL_REG, 0); >>>> >>>> /* initialize flush functions */ >>>> >>>> And, can you comment the invocation of clear_fault_bits() in >>>> init_vtd_hw() like above and see if the host can boot fine? >>>> >>>> Thanks, >>>> -- Dexuan >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Cui, Dexuan >>>> Sent: 2009?6?29? 14:30 >>>> To: ''Tom Rotenberg''; xen-devel@lists.xensource.com >>>> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >>>> with iommu=1 >>>> >>>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c >>>> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 >>>> +0100 >>>> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 >>>> +0800 >>>> @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm >>>> { >>>> u64 val; >>>> >>>> + printk("clear_fault_bits: CAP_REG=0x%llx\n", >>>> + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); >>>> val = dmar_readq( >>>> iommu->reg, >>>> cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8); >>>> >>>> Hi Tom, >>>> Can you please apply the above patch and send us the entire log? >>>> (I don''t have the same host, so I can''t try it myself.) >>>> >>>> Thanks, >>>> -- Dexuan >>>> >>>> ________________________________ >>>> >>>> From: xen-devel-bounces@lists.xensource.com [mailto: >>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >>>> Sent: 2009?6?28? 22:56 >>>> To: xen-devel@lists.xensource.com >>>> Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with >>>> iommu=1 >>>> >>>> >>>> Hi, >>>> >>>> I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to >>>> boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. I >>>> have also tried to boot with and without the ''iommu_inclusive_mapping'' boot >>>> option, but it didn''t seem to help at all. >>>> >>>> Does anyone knows how to solve this issue? >>>> >>>> These are the last messages in the UART: >>>> >>>> (XEN) Brought up 2 CPUs >>>> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >>>> (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush >>>> (XEN) Intel VT-d Snoop Control supported. >>>> (XEN) Intel VT-d DMA Passthrough not supported. >>>> (XEN) Intel VT-d Queued Invalidation supported. >>>> (XEN) Intel VT-d Interrupt Remapping supported. >>>> (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- >>>> (XEN) CPU: 0 >>>> (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >>>> (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor >>>> (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: >>>> 0000000000000020 >>>> (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: >>>> ffff830078f82f90 >>>> (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: >>>> ffff83007be2c110 >>>> (XEN) r9: 000000000000001a r10: 0000000000000005 r11: >>>> 0000000000000005 >>>> (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: >>>> ffff830078f82fcc >>>> (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: >>>> 00000000000026f0 >>>> (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 >>>> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 >>>> (XEN) Xen stack trace from rsp=ffff828c8026fda0: >>>> (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f >>>> ffff828c80204490 >>>> (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 >>>> 0000000000000000 >>>> (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 >>>> 000000000000001f >>>> (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 >>>> ffff828c80205f60 >>>> (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 >>>> ffff828c80227978 >>>> (XEN) ffff828c8021a589 0000000000000000 0000000000000000 >>>> 0000000000000000 >>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>> 0000000000000000 >>>> (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 >>>> 000000000008bf40 >>>> (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 >>>> 0000000000000000 >>>> (XEN) ffffffff00000000 0000000800000000 000000010000006e >>>> 0000000000000003 >>>> (XEN) 00000000000002f8 0000000000000000 0000000000000000 >>>> 0000000000067e2c >>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>> 0000000000000000 >>>> (XEN) ffff828c801000b5 0000000000000000 0000000000000000 >>>> 0000000000000000 >>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>> 0000000000000000 >>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>> 0000000000000000 >>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>> 0000000000000000 >>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>> 0000000000000000 >>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>> 0000000000000000 >>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>> ffff830079f18000 >>>> (XEN) Xen call trace: >>>> (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >>>> (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 >>>> (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 >>>> (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 >>>> (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 >>>> (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 >>>> (XEN) >>>> (XEN) Pagetable walk from ffff828bfff5aff8: >>>> (XEN) L4[0x105] = 000000007be7a027 5555555555555555 >>>> (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 >>>> (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 >>>> (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff >>>> (XEN) >>>> (XEN) **************************************** >>>> (XEN) Panic on CPU 0: >>>> (XEN) FATAL PAGE FAULT >>>> (XEN) [error_code=0000] >>>> (XEN) Faulting linear address: ffff828bfff5aff8 >>>> (XEN) **************************************** >>>> (XEN) >>>> (XEN) Reboot in five seconds... >>>> >>>> >>>>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Jun-29 13:08 UTC
Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
Well, it looks like my analysis was right. This is how i reproduced it: 1. Disable the IOMMU 2. Turn off the laptop, and leave it turned off for a few seconds 3. Turn on the laptop 4. Enable the IOMMU (*DON''T* turn off the machine) 5. Continue 6. Xen crashes and, now the log lines are: (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. It looks like there is a problem when enabling IOMMU, without cold boot of the machine. What do u say? Here is the full log: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:40:11 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 00B0 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.037 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fda0: (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 (XEN) ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 ffff828c80227978 (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 00000000002b23c0 ffffff8c80274000 0008bf3000000000 000000000008bf30 (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... 2009/6/29 Tom Rotenberg <tom.rotenberg@gmail.com>> I''m almost 100% sure, that this was from the same machine (since i have > only 1 such Dell currently on my desk :). > However, this log was generated just after turning on the IOMMU (without > cold boot). So, let me check if it happened because of this. > > I''ll check it now... > > > 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> > >> I have no idea about it. >> >> > (XEN) Intel VT-d Snoop Control supported. >> > (XEN) Intel VT-d DMA Passthrough not supported. >> > (XEN) Intel VT-d Queued Invalidation supported. >> > (XEN) Intel VT-d Interrupt Remapping supported. >> This is from the log of your original mail. We can see SC/QI/IR are >> supported, but in your later log, we can see they are not supported. >> I''m curious where the first log comes from. :-) >> >> -- Dexuan >> >> ------------------------------ >> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >> *Sent:* 2009年6月29日 20:38 >> >> *To:* Cui, Dexuan >> *Cc:* xen-devel@lists.xensource.com >> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >> with iommu=1 >> >> >> 1) Yep, i can''t reproduce it anymore. Maybe i''ll try it with another Dell >> R6400 i have here. >> 2) On my Lenovo T400, i don''t remember the exact crash, and i don''t have >> the log of that crash, i just remember something about "DMAR ... failure", >> which happened, after i turned on the IOMMU, and didn''t perform a cold >> reboot on that machine. >> >> Do u have any idea why these problems happen? >> Does anyone else working with Dell E6400 had this problem? >> >> Tom >> >> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >> >>> It''s really weird... :-) >>> >>> If I understand correctly, with iommu=1 in Xen: >>> 1) on your "Dell E6400 machine", now you can''t reproduce the panic any >>> longer; >>> 2) on your "Lenovo T400", you can reproduce the panic sometimes -- if so, >>> can you post the log (with the printk patch applied) when you meet with the >>> panic? Can you find out under what condition it''s easy to reproduce the >>> panic? >>> >>> >>> Thanks, >>> -- Dexuan >>> >>> >>> ------------------------------ >>> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >>> *Sent:* 2009年6月29日 20:15 >>> >>> *To:* Cui, Dexuan >>> *Cc:* xen-devel@lists.xensource.com >>> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >>> with iommu=1 >>> >>> Hi, >>> >>> Tried again, this time with cold reboot, it worked again. I also checked >>> the lines you were talking about, and now they are: >>> (XEN) Intel VT-d Snoop Control not supported. >>> (XEN) Intel VT-d DMA Passthrough not supported. >>> (XEN) Intel VT-d Queued Invalidation not supported. >>> (XEN) Intel VT-d Interrupt Remapping not supported. >>> >>> Could it be that there is some kind of problem with the hardware, which >>> makes it sometimes report invalid values? (with the Lenovo T400 i''m testing, >>> if you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on >>> boot, so you must do a cold reboot). >>> >>> Here is the log: >>> __ __ _____ _ _ ___ >>> \ \/ /___ _ __ |___ /| || | / _ \ >>> \ // _ \ ''_ \ |_ \| || |_| | | | >>> / \ __/ | | | ___) |__ _| |_| | >>> /_/\_\___|_| |_| |____(_) |_|(_)___/ >>> >>> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian >>> 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009 >>> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd >>> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 >>> iommu_inclusive_mapping loglvl=all guest_loglvl=all >>> (XEN) Video information: >>> (XEN) VGA is text mode 80x25, font 8x16 >>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >>> (XEN) Disc information: >>> (XEN) Found 1 MBR signatures >>> (XEN) Found 1 EDD information structures >>> (XEN) Xen-e820 RAM map: >>> (XEN) 0000000000000000 - 000000000009f000 (usable) >>> (XEN) 000000000009f000 - 00000000000a0000 (reserved) >>> (XEN) 0000000000100000 - 000000007c04d400 (usable) >>> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) >>> (XEN) 000000007c04f400 - 000000007f000000 (reserved) >>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >>> (XEN) 00000000fec00000 - 00000000fec10000 (reserved) >>> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) >>> (XEN) 00000000fed20000 - 00000000fed90000 (reserved) >>> (XEN) 00000000feda0000 - 00000000feda6000 (reserved) >>> (XEN) 00000000fee00000 - 00000000fee10000 (reserved) >>> (XEN) 00000000ffe60000 - 0000000100000000 (reserved) >>> (XEN) System RAM: 1983MB (2031536kB) >>> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) >>> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL >>> 61) >>> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL >>> 61) >>> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL >>> 20050624) >>> (XEN) ACPI: FACS 7C060C00, 0040 >>> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL >>> 61) >>> (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL >>> 61) >>> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL >>> 47) >>> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL >>> 61) >>> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL >>> 61) >>> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL >>> 0) >>> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL >>> 61) >>> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL >>> 20050624) >>> (XEN) NUMA turned off >>> (XEN) Faking a node at 0000000000000000-000000007c04d000 >>> (XEN) Domain heap initialised >>> (XEN) DMI 2.4 present. >>> (XEN) Using APIC driver default >>> (XEN) ACPI: PM-Timer IO Port: 0x1008 >>> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] >>> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] >>> (XEN) ACPI: Local APIC address 0xfee00000 >>> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) >>> (XEN) Processor #0 7:7 APIC version 20 >>> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) >>> (XEN) Processor #1 7:7 APIC version 20 >>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) >>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) >>> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) >>> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 >>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) >>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) >>> (XEN) ACPI: IRQ0 used by override. >>> (XEN) ACPI: IRQ2 used by override. >>> (XEN) ACPI: IRQ9 used by override. >>> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs >>> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 >>> (XEN) [VT-D]dmar.c:485: Host address width 36 >>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 >>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 >>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 >>> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL >>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 >>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>> (XEN) Intel VT-d DMAR tables have been parsed. >>> (XEN) Using ACPI (MADT) for SMP configuration information >>> (XEN) Using scheduler: SMP Credit Scheduler (credit) >>> (XEN) Initializing CPU#0 >>> (XEN) Detected 2527.058 MHz processor. >>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>> (XEN) CPU: L2 cache: 6144K >>> (XEN) CPU: Physical Processor ID: 0 >>> (XEN) CPU: Processor Core ID: 0 >>> (XEN) VMX: Supported advanced features: >>> (XEN) - APIC MMIO access virtualisation >>> (XEN) - APIC TPR shadow >>> (XEN) - Virtual NMI >>> (XEN) - MSR direct-access bitmap >>> (XEN) HVM: VMX enabled >>> (XEN) Intel machine check reporting enabled on CPU#0. >>> (XEN) mce_init: init bank1 >>> (XEN) mce_init: init bank2 >>> (XEN) mce_init: init bank3 >>> (XEN) mce_init: init bank4 >>> (XEN) mce_init: init bank5 >>> (XEN) CPU0: Thermal monitoring enabled (TM2) >>> (XEN) CMCI: CPU0 has no CMCI support >>> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >>> (XEN) Booting processor 1/1 eip 8c000 >>> (XEN) Initializing CPU#1 >>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>> (XEN) CPU: L2 cache: 6144K >>> (XEN) CPU: Physical Processor ID: 0 >>> (XEN) CPU: Processor Core ID: 1 >>> (XEN) Intel machine check reporting enabled on CPU#1. >>> (XEN) mce_init: init bank1 >>> (XEN) mce_init: init bank2 >>> (XEN) mce_init: init bank4 >>> (XEN) mce_init: init bank5 >>> (XEN) CPU1: Thermal monitoring enabled (TM2) >>> (XEN) CMCI: CPU1 has no CMCI support >>> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >>> (XEN) Total of 2 processors activated. >>> (XEN) ENABLING IO-APIC IRQs >>> (XEN) -> Using new ACK method >>> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 >>> (XEN) checking TSC synchronization across 2 CPUs: passed. >>> (XEN) Platform timer is 14.318MHz HPET >>> (XEN) Brought up 2 CPUs >>> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >>> (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush >>> (XEN) Intel VT-d Snoop Control not supported. >>> (XEN) Intel VT-d DMA Passthrough not supported. >>> (XEN) Intel VT-d Queued Invalidation not supported. >>> (XEN) Intel VT-d Interrupt Remapping not supported. >>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >>> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >>> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >>> (XEN) I/O virtualisation enabled >>> (XEN) I/O virtualisation for PV guests disabled >>> (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast >>> (XEN) ACPI sleep modes: S3 >>> (XEN) mcheck_poll: Machine check polling timer started. >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 >>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 >>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 >>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 >>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>> ffff828bfff57000 >>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>> ffff828bfff56000 >>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>> ffff828bfff55000 >>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>> ffff828bfff54000 >>> (XEN) *** LOADING DOMAIN 0 *** >>> (XEN) Xen kernel: 64-bit, lsb, compat32 >>> (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> >>> 0xffffffff8070ab48 >>> (XEN) PHYSICAL MEMORY ARRANGEMENT: >>> (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to >>> be allocated) >>> (XEN) VIRTUAL MEMORY ARRANGEMENT: >>> (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 >>> (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 >>> (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 >>> (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 >>> (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 >>> (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 >>> (XEN) TOTAL: ffffffff80000000->ffffffff82000000 >>> (XEN) ENTRY ADDRESS: ffffffff80200000 >>> (XEN) Dom0 has maximum 2 VCPUs >>> (XEN) Scrubbing Free RAM: .done. >>> (XEN) Xen trace buffers: disabled >>> (XEN) Std. Loglevel: All >>> (XEN) Guest Loglevel: All >>> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input >>> to Xen) >>> (XEN) Freed 132kB init memory. >>> >>> Tom >>> >>> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >>> >>>> So, if I understand correctly, it''s really strange you can''t reproduce >>>> the panic now. >>>> >>>> With only the first printk patch applied (and don''t comment the >>>> clear_fault_bits()), can you try a cold rebooting the host (plug the power >>>> cord off from the host; wait several minutes; plug the cord in and tunn on >>>> the host)? And the log? >>>> >>>> BTW, in your original mail, the log shows: >>>> (XEN) Intel VT-d Queued Invalidation supported. >>>> (XEN) Intel VT-d Interrupt Remapping supported. >>>> >>>> But in your this new log, it shows: >>>> (XEN) Intel VT-d Queued Invalidation* not* supported. >>>> (XEN) Intel VT-d Interrupt Remapping* not* supported. >>>> Are you sure you have been operating on the same host? :-) >>>> >>>> -- Dexuan >>>> >>>> ------------------------------ >>>> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >>>> *Sent:* 2009年6月29日 17:01 >>>> *To:* Cui, Dexuan >>>> *Cc:* xen-devel@lists.xensource.com >>>> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell >>>> e6400, with iommu=1 >>>> >>>> Hi, >>>> >>>> Well, just tried what you suggested, and here are the results: >>>> * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with >>>> iommu=1 >>>> The strange thing, is that after it booted one time ok (with the second >>>> patch), then it continued to boot ok, even if i reverted the second patch. >>>> >>>> The relevant lines in the log are: >>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >>>> >>>> Anyway, here is the full Xen log, with the first patch (print cap''s) >>>> applied: >>>> >>>> __ __ _____ _ _ ___ >>>> \ \/ /___ _ __ |___ /| || | / _ \ >>>> \ // _ \ ''_ \ |_ \| || |_| | | | >>>> / \ __/ | | | ___) |__ _| |_| | >>>> /_/\_\___|_| |_| |____(_) |_|(_)___/ >>>> >>>> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian >>>> 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 >>>> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 >>>> 19629:b2128a88e3dd >>>> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 >>>> iommu_inclusive_mapping loglvl=all guest_loglvl=all >>>> (XEN) Video information: >>>> (XEN) VGA is text mode 80x25, font 8x16 >>>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >>>> (XEN) Disc information: >>>> (XEN) Found 1 MBR signatures >>>> (XEN) Found 1 EDD information structures >>>> (XEN) Xen-e820 RAM map: >>>> (XEN) 0000000000000000 - 000000000009f000 (usable) >>>> (XEN) 000000000009f000 - 00000000000a0000 (reserved) >>>> (XEN) 0000000000100000 - 000000007c04d400 (usable) >>>> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) >>>> (XEN) 000000007c04f400 - 000000007f000000 (reserved) >>>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >>>> (XEN) 00000000fec00000 - 00000000fec10000 (reserved) >>>> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) >>>> (XEN) 00000000fed20000 - 00000000fed90000 (reserved) >>>> (XEN) 00000000feda0000 - 00000000feda6000 (reserved) >>>> (XEN) 00000000fee00000 - 00000000fee10000 (reserved) >>>> (XEN) 00000000ffe60000 - 0000000100000000 (reserved) >>>> (XEN) System RAM: 1983MB (2031536kB) >>>> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) >>>> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL >>>> 61) >>>> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL >>>> 61) >>>> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL >>>> 20050624) >>>> (XEN) ACPI: FACS 7C060C00, 0040 >>>> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL >>>> 61) >>>> (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL >>>> 61) >>>> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL >>>> 47) >>>> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL >>>> 61) >>>> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL >>>> 61) >>>> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL >>>> 0) >>>> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL >>>> 61) >>>> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL >>>> 20050624) >>>> (XEN) NUMA turned off >>>> (XEN) Faking a node at 0000000000000000-000000007c04d000 >>>> (XEN) Domain heap initialised >>>> (XEN) DMI 2.4 present. >>>> (XEN) Using APIC driver default >>>> (XEN) ACPI: PM-Timer IO Port: 0x1008 >>>> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] >>>> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] >>>> (XEN) ACPI: Local APIC address 0xfee00000 >>>> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) >>>> (XEN) Processor #0 7:7 APIC version 20 >>>> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) >>>> (XEN) Processor #1 7:7 APIC version 20 >>>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) >>>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) >>>> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) >>>> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 >>>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) >>>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) >>>> (XEN) ACPI: IRQ0 used by override. >>>> (XEN) ACPI: IRQ2 used by override. >>>> (XEN) ACPI: IRQ9 used by override. >>>> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs >>>> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 >>>> (XEN) [VT-D]dmar.c:485: Host address width 36 >>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 >>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 >>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 >>>> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL >>>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 >>>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>>> (XEN) Intel VT-d DMAR tables have been parsed. >>>> (XEN) Using ACPI (MADT) for SMP configuration information >>>> (XEN) Using scheduler: SMP Credit Scheduler (credit) >>>> (XEN) Initializing CPU#0 >>>> (XEN) Detected 2527.068 MHz processor. >>>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>>> (XEN) CPU: L2 cache: 6144K >>>> (XEN) CPU: Physical Processor ID: 0 >>>> (XEN) CPU: Processor Core ID: 0 >>>> (XEN) VMX: Supported advanced features: >>>> (XEN) - APIC MMIO access virtualisation >>>> (XEN) - APIC TPR shadow >>>> (XEN) - Virtual NMI >>>> (XEN) - MSR direct-access bitmap >>>> (XEN) HVM: VMX enabled >>>> (XEN) Intel machine check reporting enabled on CPU#0. >>>> (XEN) mce_init: init bank1 >>>> (XEN) mce_init: init bank2 >>>> (XEN) mce_init: init bank3 >>>> (XEN) mce_init: init bank4 >>>> (XEN) mce_init: init bank5 >>>> (XEN) CPU0: Thermal monitoring enabled (TM2) >>>> (XEN) CMCI: CPU0 has no CMCI support >>>> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >>>> (XEN) Booting processor 1/1 eip 8c000 >>>> (XEN) Initializing CPU#1 >>>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>>> (XEN) CPU: L2 cache: 6144K >>>> (XEN) CPU: Physical Processor ID: 0 >>>> (XEN) CPU: Processor Core ID: 1 >>>> (XEN) Intel machine check reporting enabled on CPU#1. >>>> (XEN) mce_init: init bank1 >>>> (XEN) mce_init: init bank2 >>>> (XEN) mce_init: init bank4 >>>> (XEN) mce_init: init bank5 >>>> (XEN) CPU1: Thermal monitoring enabled (TM2) >>>> (XEN) CMCI: CPU1 has no CMCI support >>>> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >>>> (XEN) Total of 2 processors activated. >>>> (XEN) ENABLING IO-APIC IRQs >>>> (XEN) -> Using new ACK method >>>> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 >>>> (XEN) checking TSC synchronization across 2 CPUs: passed. >>>> (XEN) Platform timer is 14.318MHz HPET >>>> (XEN) Brought up 2 CPUs >>>> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >>>> (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush >>>> (XEN) Intel VT-d Snoop Control not supported. >>>> (XEN) Intel VT-d DMA Passthrough not supported. >>>> (XEN) Intel VT-d Queued Invalidation not supported. >>>> (XEN) Intel VT-d Interrupt Remapping not supported. >>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >>>> (XEN) I/O virtualisation enabled >>>> (XEN) I/O virtualisation for PV guests disabled >>>> (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast >>>> (XEN) ACPI sleep modes: S3 >>>> (XEN) mcheck_poll: Machine check polling timer started. >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 >>>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf >>>> 0:1b.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 >>>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 >>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>> ffff828bfff57000 >>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>> ffff828bfff56000 >>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>> ffff828bfff55000 >>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>> ffff828bfff54000 >>>> (XEN) *** LOADING DOMAIN 0 *** >>>> (XEN) Xen kernel: 64-bit, lsb, compat32 >>>> (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> >>>> 0xffffffff8070ab48 >>>> (XEN) PHYSICAL MEMORY ARRANGEMENT: >>>> (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages >>>> to be allocated) >>>> (XEN) VIRTUAL MEMORY ARRANGEMENT: >>>> (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 >>>> (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 >>>> (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 >>>> (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 >>>> (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 >>>> (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 >>>> (XEN) TOTAL: ffffffff80000000->ffffffff82000000 >>>> (XEN) ENTRY ADDRESS: ffffffff80200000 >>>> (XEN) Dom0 has maximum 2 VCPUs >>>> (XEN) Scrubbing Free RAM: .done. >>>> (XEN) Xen trace buffers: disabled >>>> (XEN) Std. Loglevel: All >>>> (XEN) Guest Loglevel: All >>>> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch >>>> input to Xen) >>>> (XEN) Freed 132kB init memory. >>>> >>>> Tom >>>> >>>> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >>>> >>>>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c >>>>> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 >>>>> +0100 >>>>> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 >>>>> +0800 >>>>> @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) >>>>> } >>>>> dma_msi_data_init(iommu, iommu->vector); >>>>> dma_msi_addr_init(iommu, >>>>> cpu_physical_id(first_cpu(cpu_online_map))); >>>>> - clear_fault_bits(iommu); >>>>> + //clear_fault_bits(iommu); >>>>> dmar_writel(iommu->reg, DMAR_FECTL_REG, 0); >>>>> >>>>> /* initialize flush functions */ >>>>> >>>>> And, can you comment the invocation of clear_fault_bits() in >>>>> init_vtd_hw() like above and see if the host can boot fine? >>>>> >>>>> Thanks, >>>>> -- Dexuan >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Cui, Dexuan >>>>> Sent: 2009?6?29? 14:30 >>>>> To: ''Tom Rotenberg''; xen-devel@lists.xensource.com >>>>> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >>>>> with iommu=1 >>>>> >>>>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c >>>>> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 >>>>> +0100 >>>>> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 >>>>> +0800 >>>>> @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm >>>>> { >>>>> u64 val; >>>>> >>>>> + printk("clear_fault_bits: CAP_REG=0x%llx\n", >>>>> + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); >>>>> val = dmar_readq( >>>>> iommu->reg, >>>>> cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8); >>>>> >>>>> Hi Tom, >>>>> Can you please apply the above patch and send us the entire log? >>>>> (I don''t have the same host, so I can''t try it myself.) >>>>> >>>>> Thanks, >>>>> -- Dexuan >>>>> >>>>> ________________________________ >>>>> >>>>> From: xen-devel-bounces@lists.xensource.com [mailto: >>>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >>>>> Sent: 2009?6?28? 22:56 >>>>> To: xen-devel@lists.xensource.com >>>>> Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with >>>>> iommu=1 >>>>> >>>>> >>>>> Hi, >>>>> >>>>> I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to >>>>> boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. I >>>>> have also tried to boot with and without the ''iommu_inclusive_mapping'' boot >>>>> option, but it didn''t seem to help at all. >>>>> >>>>> Does anyone knows how to solve this issue? >>>>> >>>>> These are the last messages in the UART: >>>>> >>>>> (XEN) Brought up 2 CPUs >>>>> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >>>>> (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush >>>>> (XEN) Intel VT-d Snoop Control supported. >>>>> (XEN) Intel VT-d DMA Passthrough not supported. >>>>> (XEN) Intel VT-d Queued Invalidation supported. >>>>> (XEN) Intel VT-d Interrupt Remapping supported. >>>>> (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- >>>>> (XEN) CPU: 0 >>>>> (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >>>>> (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor >>>>> (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: >>>>> 0000000000000020 >>>>> (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: >>>>> ffff830078f82f90 >>>>> (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: >>>>> ffff83007be2c110 >>>>> (XEN) r9: 000000000000001a r10: 0000000000000005 r11: >>>>> 0000000000000005 >>>>> (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: >>>>> ffff830078f82fcc >>>>> (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: >>>>> 00000000000026f0 >>>>> (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 >>>>> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 >>>>> (XEN) Xen stack trace from rsp=ffff828c8026fda0: >>>>> (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f >>>>> ffff828c80204490 >>>>> (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 >>>>> 0000000000000000 >>>>> (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 >>>>> 000000000000001f >>>>> (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 >>>>> ffff828c80205f60 >>>>> (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 >>>>> ffff828c80227978 >>>>> (XEN) ffff828c8021a589 0000000000000000 0000000000000000 >>>>> 0000000000000000 >>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>> 0000000000000000 >>>>> (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 >>>>> 000000000008bf40 >>>>> (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 >>>>> 0000000000000000 >>>>> (XEN) ffffffff00000000 0000000800000000 000000010000006e >>>>> 0000000000000003 >>>>> (XEN) 00000000000002f8 0000000000000000 0000000000000000 >>>>> 0000000000067e2c >>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>> 0000000000000000 >>>>> (XEN) ffff828c801000b5 0000000000000000 0000000000000000 >>>>> 0000000000000000 >>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>> 0000000000000000 >>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>> 0000000000000000 >>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>> 0000000000000000 >>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>> 0000000000000000 >>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>> 0000000000000000 >>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>> ffff830079f18000 >>>>> (XEN) Xen call trace: >>>>> (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >>>>> (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 >>>>> (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 >>>>> (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 >>>>> (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 >>>>> (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 >>>>> (XEN) >>>>> (XEN) Pagetable walk from ffff828bfff5aff8: >>>>> (XEN) L4[0x105] = 000000007be7a027 5555555555555555 >>>>> (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 >>>>> (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 >>>>> (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff >>>>> (XEN) >>>>> (XEN) **************************************** >>>>> (XEN) Panic on CPU 0: >>>>> (XEN) FATAL PAGE FAULT >>>>> (XEN) [error_code=0000] >>>>> (XEN) Faulting linear address: ffff828bfff5aff8 >>>>> (XEN) **************************************** >>>>> (XEN) >>>>> (XEN) Reboot in five seconds... >>>>> >>>>> >>>>>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Cui, Dexuan
2009-Jun-29 13:43 UTC
RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
When you meet with the panic, can you apply the printk log to print CAP_REG? In your step 1 and 4, you just mean using the xen parameter iommu=0 and iommu=1, correct? During theses steps, you don''t change any BIOS setting and there is not any HW change, correct? I see you''re using "3.4.0 19629:b2128a88e3dd", but 19629''s checksum should be 69293fc2096f (http://xenbits.xensource.com/xen-3.4-testing.hg?rev/69293fc2096f). So I assume you made your own changes to the code? Can you try the latest changeset 19677: b2ddeb734ca7(http://xenbits.xensource.com/xen-3.4-testing.hg)? Sorry, I don''t have the same type of host, so I can''t try it myself. :-( Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com] Sent: 2009年6月29日 21:09 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Well, it looks like my analysis was right. This is how i reproduced it: 1. Disable the IOMMU 2. Turn off the laptop, and leave it turned off for a few seconds 3. Turn on the laptop 4. Enable the IOMMU (*DON''T* turn off the machine) 5. Continue 6. Xen crashes and, now the log lines are: (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. It looks like there is a problem when enabling IOMMU, without cold boot of the machine. What do u say? Here is the full log: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:40:11 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 00B0 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.037 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fda0: (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 (XEN) ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 ffff828c80227978 (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 00000000002b23c0 ffffff8c80274000 0008bf3000000000 000000000008bf30 (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... 2009/6/29 Tom Rotenberg <tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>> I''m almost 100% sure, that this was from the same machine (since i have only 1 such Dell currently on my desk :). However, this log was generated just after turning on the IOMMU (without cold boot). So, let me check if it happened because of this. I''ll check it now... 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> I have no idea about it.> (XEN) Intel VT-d Snoop Control supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported.This is from the log of your original mail. We can see SC/QI/IR are supported, but in your later log, we can see they are not supported. I''m curious where the first log comes from. :-) -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 20:38 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 1) Yep, i can''t reproduce it anymore. Maybe i''ll try it with another Dell R6400 i have here. 2) On my Lenovo T400, i don''t remember the exact crash, and i don''t have the log of that crash, i just remember something about "DMAR ... failure", which happened, after i turned on the IOMMU, and didn''t perform a cold reboot on that machine. Do u have any idea why these problems happen? Does anyone else working with Dell E6400 had this problem? Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> It''s really weird... :-) If I understand correctly, with iommu=1 in Xen: 1) on your "Dell E6400 machine", now you can''t reproduce the panic any longer; 2) on your "Lenovo T400", you can reproduce the panic sometimes -- if so, can you post the log (with the printk patch applied) when you meet with the panic? Can you find out under what condition it''s easy to reproduce the panic? Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 20:15 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, Tried again, this time with cold reboot, it worked again. I also checked the lines you were talking about, and now they are: (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. Could it be that there is some kind of problem with the hardware, which makes it sometimes report invalid values? (with the Lenovo T400 i''m testing, if you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on boot, so you must do a cold reboot). Here is the log: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.058 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 (XEN) TOTAL: ffffffff80000000->ffffffff82000000 (XEN) ENTRY ADDRESS: ffffffff80200000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 132kB init memory. Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> So, if I understand correctly, it''s really strange you can''t reproduce the panic now. With only the first printk patch applied (and don''t comment the clear_fault_bits()), can you try a cold rebooting the host (plug the power cord off from the host; wait several minutes; plug the cord in and tunn on the host)? And the log? BTW, in your original mail, the log shows: (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. But in your this new log, it shows: (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. Are you sure you have been operating on the same host? :-) -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 17:01 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, Well, just tried what you suggested, and here are the results: * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with iommu=1 The strange thing, is that after it booted one time ok (with the second patch), then it continued to boot ok, even if i reverted the second patch. The relevant lines in the log are: (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 Anyway, here is the full Xen log, with the first patch (print cap''s) applied: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.068 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 (XEN) TOTAL: ffffffff80000000->ffffffff82000000 (XEN) ENTRY ADDRESS: ffffffff80200000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 132kB init memory. Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800 @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) } dma_msi_data_init(iommu, iommu->vector); dma_msi_addr_init(iommu, cpu_physical_id(first_cpu(cpu_online_map))); - clear_fault_bits(iommu); + //clear_fault_bits(iommu); dmar_writel(iommu->reg, DMAR_FECTL_REG, 0); /* initialize flush functions */ And, can you comment the invocation of clear_fault_bits() in init_vtd_hw() like above and see if the host can boot fine? Thanks, -- Dexuan -----Original Message----- From: Cui, Dexuan Sent: 2009?6?29? 14:30 To: ''Tom Rotenberg''; xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800 @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm { u64 val; + printk("clear_fault_bits: CAP_REG=0x%llx\n", + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); val = dmar_readq( iommu->reg, cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8); Hi Tom, Can you please apply the above patch and send us the entire log? (I don''t have the same host, so I can''t try it myself.) Thanks, -- Dexuan ________________________________ From: xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com> [mailto:xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>] On Behalf Of Tom Rotenberg Sent: 2009?6?28? 22:56 To: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. I have also tried to boot with and without the ''iommu_inclusive_mapping'' boot option, but it didn''t seem to help at all. Does anyone knows how to solve this issue? These are the last messages in the UART: (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fda0: (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978 (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40 (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Jun-29 14:14 UTC
Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
in steps 1 & 4, i mean disabling and enabling IOMMU in the BIOS (and not in Xen boot parameter). And, this is what probably caused all of this. I am using my own version (above Xen 3.4.0), but i haven''t made any changes in the hyervisor itself (but rather in the hvmloader and rombios), so it''s not related to this. I prefer not to try the latest Xen, unless there is a very good reason for this, as it take long time to compile, and i can''t do it right now. I have reproduced the panic with the print, and here is the relevant part of the log: (XEN) clear_fault_bits: CAP_REG=0xffffffffffffffff (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000000 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff828c801f0564 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fd98 r8: 0000000000004000 (XEN) r9: 0000000000003fff r10: ffff828c80288360 r11: 0000000080287ea0 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fd98: (XEN) ffff830078f82f90 ffff828c8012d35d ffff830078f824b0 ffff828c801e4257 (XEN) ffff828c80204490 ffff828c801e4257 ffff828c802044c0 ffff828c8026fdf8 (XEN) 0000000000000000 ffff828c8012deae ffff830078f82d40 ffff828c80228100 (XEN) 000000000000001f 00000000ffffffed ffff828c80227a78 0000000000000001 (XEN) ffff828c80205f60 ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 (XEN) ffff828c80227978 ffff828c8021a589 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 00000000002b23c0 ffffff8c80274000 0008bf3000000000 (XEN) 000000000008bf30 0000000000000000 ffff83000008bfc0 0000000001ce08a0 (XEN) 0000000000000000 ffffffff00000000 0000000800000000 000000010000006e (XEN) 0000000000000003 00000000000002f8 0000000000000000 0000000000000000 (XEN) 0000000000067e9c 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffff828c801000b5 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0 (XEN) [<ffff828c8012d35d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012deae>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com>> When you meet with the panic, can you apply the printk log to print > CAP_REG? > > In your step 1 and 4, you just mean using the xen parameter iommu=0 and > iommu=1, correct? > During theses steps, you don''t change any BIOS setting and there is not any > HW change, correct? > > I see you''re using "3.4.0 19629:b2128a88e3dd", but 19629''s checksum should > be 69293fc2096f ( > http://xenbits.xensource.com/xen-3.4-testing.hg?rev/69293fc2096f). > So I assume you made your own changes to the code? > Can you try the latest changeset 19677: b2ddeb734ca7( > http://xenbits.xensource.com/xen-3.4-testing.hg)? > > Sorry, I don''t have the same type of host, so I can''t try it myself. :-( > > Thanks, > -- Dexuan > > > ------------------------------ > *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] > *Sent:* 2009年6月29日 21:09 > > *To:* Cui, Dexuan > *Cc:* xen-devel@lists.xensource.com > *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, > with iommu=1 > > Well, it looks like my analysis was right. > This is how i reproduced it: > 1. Disable the IOMMU > 2. Turn off the laptop, and leave it turned off for a few seconds > 3. Turn on the laptop > 4. Enable the IOMMU (*DON''T* turn off the machine) > 5. Continue > 6. Xen crashes > > and, now the log lines are: > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported. > > It looks like there is a problem when enabling IOMMU, without cold boot of > the machine. > > What do u say? > > Here is the full log: > __ __ _____ _ _ ___ > \ \/ /___ _ __ |___ /| || | / _ \ > \ // _ \ ''_ \ |_ \| || |_| | | | > / \ __/ | | | ___) |__ _| |_| | > /_/\_\___|_| |_| |____(_) |_|(_)___/ > > (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian > 4.3.2-1.1) ) Mon Jun 29 07:40:11 EDT 2009 > (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd > (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 > iommu_inclusive_mapping loglvl=all guest_loglvl=all > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 1 MBR signatures > (XEN) Found 1 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009f000 (usable) > (XEN) 000000000009f000 - 00000000000a0000 (reserved) > (XEN) 0000000000100000 - 000000007c04d400 (usable) > (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) > (XEN) 000000007c04f400 - 000000007f000000 (reserved) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec10000 (reserved) > (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) > (XEN) 00000000fed20000 - 00000000fed90000 (reserved) > (XEN) 00000000feda0000 - 00000000feda6000 (reserved) > (XEN) 00000000fee00000 - 00000000fee10000 (reserved) > (XEN) 00000000ffe60000 - 0000000100000000 (reserved) > (XEN) System RAM: 1983MB (2031536kB) > (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) > (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) > (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) > (XEN) ACPI: FACS 7C060C00, 0040 > (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) > (XEN) ACPI: DMAR 7C060400, 00B0 (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) > (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL > 61) > (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL > 61) > (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) > (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) > (XEN) NUMA turned off > (XEN) Faking a node at 0000000000000000-000000007c04d000 > (XEN) Domain heap initialised > (XEN) DMI 2.4 present. > (XEN) Using APIC driver default > (XEN) ACPI: PM-Timer IO Port: 0x1008 > (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] > (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] > (XEN) ACPI: Local APIC address 0xfee00000 > (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) > (XEN) Processor #0 7:7 APIC version 20 > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) > (XEN) Processor #1 7:7 APIC version 20 > (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) > (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) > (XEN) ACPI: IRQ0 used by override. > (XEN) ACPI: IRQ2 used by override. > (XEN) ACPI: IRQ9 used by override. > (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs > (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 > (XEN) [VT-D]dmar.c:485: Host address width 36 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 > (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL > (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 > (XEN) Intel VT-d DMAR tables have been parsed. > (XEN) Using ACPI (MADT) for SMP configuration information > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Initializing CPU#0 > (XEN) Detected 2527.037 MHz processor. > (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K > (XEN) CPU: L2 cache: 6144K > (XEN) CPU: Physical Processor ID: 0 > (XEN) CPU: Processor Core ID: 0 > (XEN) VMX: Supported advanced features: > (XEN) - APIC MMIO access virtualisation > (XEN) - APIC TPR shadow > (XEN) - Virtual NMI > (XEN) - MSR direct-access bitmap > (XEN) HVM: VMX enabled > (XEN) Intel machine check reporting enabled on CPU#0. > (XEN) mce_init: init bank1 > (XEN) mce_init: init bank2 > (XEN) mce_init: init bank3 > (XEN) mce_init: init bank4 > (XEN) mce_init: init bank5 > (XEN) CPU0: Thermal monitoring enabled (TM2) > (XEN) CMCI: CPU0 has no CMCI support > (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 > (XEN) Booting processor 1/1 eip 8c000 > (XEN) Initializing CPU#1 > (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K > (XEN) CPU: L2 cache: 6144K > (XEN) CPU: Physical Processor ID: 0 > (XEN) CPU: Processor Core ID: 1 > (XEN) Intel machine check reporting enabled on CPU#1. > (XEN) mce_init: init bank1 > (XEN) mce_init: init bank2 > (XEN) mce_init: init bank4 > (XEN) mce_init: init bank5 > (XEN) CPU1: Thermal monitoring enabled (TM2) > (XEN) CMCI: CPU1 has no CMCI support > (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 > (XEN) Total of 2 processors activated. > (XEN) ENABLING IO-APIC IRQs > (XEN) -> Using new ACK method > (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 > (XEN) checking TSC synchronization across 2 CPUs: passed. > (XEN) Platform timer is 14.318MHz HPET > (XEN) Brought up 2 CPUs > (XEN) microcode.c:73:d32767 microcode: CPU1 resumed > (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush > (XEN) Intel VT-d Snoop Control supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported. > (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- > (XEN) CPU: 0 > (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 > (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor > (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 > (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 > (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 > (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 > (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc > (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 > (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 > (XEN) Xen stack trace from rsp=ffff828c8026fda0: > (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f > ffff828c80204490 > (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 > 0000000000000000 > (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 > 000000000000001f > (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 > ffff828c80205f60 > (XEN) ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 > ffff828c80227978 > (XEN) ffff828c8021a589 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 00000000002b23c0 ffffff8c80274000 0008bf3000000000 > 000000000008bf30 > (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 > 0000000000000000 > (XEN) ffffffff00000000 0000000800000000 000000010000006e > 0000000000000003 > (XEN) 00000000000002f8 0000000000000000 0000000000000000 > 0000000000067e2c > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) ffff828c801000b5 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > ffff830079f18000 > (XEN) Xen call trace: > (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 > (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 > (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 > (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 > (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 > (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 > (XEN) > (XEN) Pagetable walk from ffff828bfff5aff8: > (XEN) L4[0x105] = 000000007be7a027 5555555555555555 > (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 > (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 > (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff > (XEN) > (XEN) **************************************** > (XEN) Panic on CPU 0: > (XEN) FATAL PAGE FAULT > (XEN) [error_code=0000] > (XEN) Faulting linear address: ffff828bfff5aff8 > (XEN) **************************************** > (XEN) > (XEN) Reboot in five seconds... > > > 2009/6/29 Tom Rotenberg <tom.rotenberg@gmail.com> > >> I''m almost 100% sure, that this was from the same machine (since i have >> only 1 such Dell currently on my desk :). >> However, this log was generated just after turning on the IOMMU (without >> cold boot). So, let me check if it happened because of this. >> >> I''ll check it now... >> >> >> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >> >>> I have no idea about it. >>> >>> > (XEN) Intel VT-d Snoop Control supported. >>> > (XEN) Intel VT-d DMA Passthrough not supported. >>> > (XEN) Intel VT-d Queued Invalidation supported. >>> > (XEN) Intel VT-d Interrupt Remapping supported. >>> This is from the log of your original mail. We can see SC/QI/IR are >>> supported, but in your later log, we can see they are not supported. >>> I''m curious where the first log comes from. :-) >>> >>> -- Dexuan >>> >>> ------------------------------ >>> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >>> *Sent:* 2009年6月29日 20:38 >>> >>> *To:* Cui, Dexuan >>> *Cc:* xen-devel@lists.xensource.com >>> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >>> with iommu=1 >>> >>> >>> 1) Yep, i can''t reproduce it anymore. Maybe i''ll try it with another Dell >>> R6400 i have here. >>> 2) On my Lenovo T400, i don''t remember the exact crash, and i don''t have >>> the log of that crash, i just remember something about "DMAR ... failure", >>> which happened, after i turned on the IOMMU, and didn''t perform a cold >>> reboot on that machine. >>> >>> Do u have any idea why these problems happen? >>> Does anyone else working with Dell E6400 had this problem? >>> >>> Tom >>> >>> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >>> >>>> It''s really weird... :-) >>>> >>>> If I understand correctly, with iommu=1 in Xen: >>>> 1) on your "Dell E6400 machine", now you can''t reproduce the panic any >>>> longer; >>>> 2) on your "Lenovo T400", you can reproduce the panic sometimes -- if >>>> so, can you post the log (with the printk patch applied) when you meet with >>>> the panic? Can you find out under what condition it''s easy to reproduce the >>>> panic? >>>> >>>> >>>> Thanks, >>>> -- Dexuan >>>> >>>> >>>> ------------------------------ >>>> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >>>> *Sent:* 2009年6月29日 20:15 >>>> >>>> *To:* Cui, Dexuan >>>> *Cc:* xen-devel@lists.xensource.com >>>> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell >>>> e6400, with iommu=1 >>>> >>>> Hi, >>>> >>>> Tried again, this time with cold reboot, it worked again. I also checked >>>> the lines you were talking about, and now they are: >>>> (XEN) Intel VT-d Snoop Control not supported. >>>> (XEN) Intel VT-d DMA Passthrough not supported. >>>> (XEN) Intel VT-d Queued Invalidation not supported. >>>> (XEN) Intel VT-d Interrupt Remapping not supported. >>>> >>>> Could it be that there is some kind of problem with the hardware, which >>>> makes it sometimes report invalid values? (with the Lenovo T400 i''m testing, >>>> if you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on >>>> boot, so you must do a cold reboot). >>>> >>>> Here is the log: >>>> __ __ _____ _ _ ___ >>>> \ \/ /___ _ __ |___ /| || | / _ \ >>>> \ // _ \ ''_ \ |_ \| || |_| | | | >>>> / \ __/ | | | ___) |__ _| |_| | >>>> /_/\_\___|_| |_| |____(_) |_|(_)___/ >>>> >>>> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian >>>> 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009 >>>> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 >>>> 19629:b2128a88e3dd >>>> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 >>>> iommu_inclusive_mapping loglvl=all guest_loglvl=all >>>> (XEN) Video information: >>>> (XEN) VGA is text mode 80x25, font 8x16 >>>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >>>> (XEN) Disc information: >>>> (XEN) Found 1 MBR signatures >>>> (XEN) Found 1 EDD information structures >>>> (XEN) Xen-e820 RAM map: >>>> (XEN) 0000000000000000 - 000000000009f000 (usable) >>>> (XEN) 000000000009f000 - 00000000000a0000 (reserved) >>>> (XEN) 0000000000100000 - 000000007c04d400 (usable) >>>> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) >>>> (XEN) 000000007c04f400 - 000000007f000000 (reserved) >>>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >>>> (XEN) 00000000fec00000 - 00000000fec10000 (reserved) >>>> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) >>>> (XEN) 00000000fed20000 - 00000000fed90000 (reserved) >>>> (XEN) 00000000feda0000 - 00000000feda6000 (reserved) >>>> (XEN) 00000000fee00000 - 00000000fee10000 (reserved) >>>> (XEN) 00000000ffe60000 - 0000000100000000 (reserved) >>>> (XEN) System RAM: 1983MB (2031536kB) >>>> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) >>>> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL >>>> 61) >>>> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL >>>> 61) >>>> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL >>>> 20050624) >>>> (XEN) ACPI: FACS 7C060C00, 0040 >>>> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL >>>> 61) >>>> (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL >>>> 61) >>>> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL >>>> 47) >>>> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL >>>> 61) >>>> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL >>>> 61) >>>> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL >>>> 0) >>>> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL >>>> 61) >>>> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL >>>> 20050624) >>>> (XEN) NUMA turned off >>>> (XEN) Faking a node at 0000000000000000-000000007c04d000 >>>> (XEN) Domain heap initialised >>>> (XEN) DMI 2.4 present. >>>> (XEN) Using APIC driver default >>>> (XEN) ACPI: PM-Timer IO Port: 0x1008 >>>> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] >>>> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] >>>> (XEN) ACPI: Local APIC address 0xfee00000 >>>> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) >>>> (XEN) Processor #0 7:7 APIC version 20 >>>> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) >>>> (XEN) Processor #1 7:7 APIC version 20 >>>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) >>>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) >>>> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) >>>> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 >>>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) >>>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) >>>> (XEN) ACPI: IRQ0 used by override. >>>> (XEN) ACPI: IRQ2 used by override. >>>> (XEN) ACPI: IRQ9 used by override. >>>> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs >>>> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 >>>> (XEN) [VT-D]dmar.c:485: Host address width 36 >>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 >>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 >>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 >>>> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL >>>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 >>>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>>> (XEN) Intel VT-d DMAR tables have been parsed. >>>> (XEN) Using ACPI (MADT) for SMP configuration information >>>> (XEN) Using scheduler: SMP Credit Scheduler (credit) >>>> (XEN) Initializing CPU#0 >>>> (XEN) Detected 2527.058 MHz processor. >>>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>>> (XEN) CPU: L2 cache: 6144K >>>> (XEN) CPU: Physical Processor ID: 0 >>>> (XEN) CPU: Processor Core ID: 0 >>>> (XEN) VMX: Supported advanced features: >>>> (XEN) - APIC MMIO access virtualisation >>>> (XEN) - APIC TPR shadow >>>> (XEN) - Virtual NMI >>>> (XEN) - MSR direct-access bitmap >>>> (XEN) HVM: VMX enabled >>>> (XEN) Intel machine check reporting enabled on CPU#0. >>>> (XEN) mce_init: init bank1 >>>> (XEN) mce_init: init bank2 >>>> (XEN) mce_init: init bank3 >>>> (XEN) mce_init: init bank4 >>>> (XEN) mce_init: init bank5 >>>> (XEN) CPU0: Thermal monitoring enabled (TM2) >>>> (XEN) CMCI: CPU0 has no CMCI support >>>> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >>>> (XEN) Booting processor 1/1 eip 8c000 >>>> (XEN) Initializing CPU#1 >>>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>>> (XEN) CPU: L2 cache: 6144K >>>> (XEN) CPU: Physical Processor ID: 0 >>>> (XEN) CPU: Processor Core ID: 1 >>>> (XEN) Intel machine check reporting enabled on CPU#1. >>>> (XEN) mce_init: init bank1 >>>> (XEN) mce_init: init bank2 >>>> (XEN) mce_init: init bank4 >>>> (XEN) mce_init: init bank5 >>>> (XEN) CPU1: Thermal monitoring enabled (TM2) >>>> (XEN) CMCI: CPU1 has no CMCI support >>>> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >>>> (XEN) Total of 2 processors activated. >>>> (XEN) ENABLING IO-APIC IRQs >>>> (XEN) -> Using new ACK method >>>> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 >>>> (XEN) checking TSC synchronization across 2 CPUs: passed. >>>> (XEN) Platform timer is 14.318MHz HPET >>>> (XEN) Brought up 2 CPUs >>>> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >>>> (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush >>>> (XEN) Intel VT-d Snoop Control not supported. >>>> (XEN) Intel VT-d DMA Passthrough not supported. >>>> (XEN) Intel VT-d Queued Invalidation not supported. >>>> (XEN) Intel VT-d Interrupt Remapping not supported. >>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >>>> (XEN) I/O virtualisation enabled >>>> (XEN) I/O virtualisation for PV guests disabled >>>> (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast >>>> (XEN) ACPI sleep modes: S3 >>>> (XEN) mcheck_poll: Machine check polling timer started. >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 >>>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf >>>> 0:1b.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 >>>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 >>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 >>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>> ffff828bfff57000 >>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>> ffff828bfff56000 >>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>> ffff828bfff55000 >>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>> ffff828bfff54000 >>>> (XEN) *** LOADING DOMAIN 0 *** >>>> (XEN) Xen kernel: 64-bit, lsb, compat32 >>>> (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> >>>> 0xffffffff8070ab48 >>>> (XEN) PHYSICAL MEMORY ARRANGEMENT: >>>> (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages >>>> to be allocated) >>>> (XEN) VIRTUAL MEMORY ARRANGEMENT: >>>> (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 >>>> (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 >>>> (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 >>>> (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 >>>> (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 >>>> (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 >>>> (XEN) TOTAL: ffffffff80000000->ffffffff82000000 >>>> (XEN) ENTRY ADDRESS: ffffffff80200000 >>>> (XEN) Dom0 has maximum 2 VCPUs >>>> (XEN) Scrubbing Free RAM: .done. >>>> (XEN) Xen trace buffers: disabled >>>> (XEN) Std. Loglevel: All >>>> (XEN) Guest Loglevel: All >>>> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch >>>> input to Xen) >>>> (XEN) Freed 132kB init memory. >>>> >>>> Tom >>>> >>>> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >>>> >>>>> So, if I understand correctly, it''s really strange you can''t >>>>> reproduce the panic now. >>>>> >>>>> With only the first printk patch applied (and don''t comment the >>>>> clear_fault_bits()), can you try a cold rebooting the host (plug the power >>>>> cord off from the host; wait several minutes; plug the cord in and tunn on >>>>> the host)? And the log? >>>>> >>>>> BTW, in your original mail, the log shows: >>>>> (XEN) Intel VT-d Queued Invalidation supported. >>>>> (XEN) Intel VT-d Interrupt Remapping supported. >>>>> >>>>> But in your this new log, it shows: >>>>> (XEN) Intel VT-d Queued Invalidation* not* supported. >>>>> (XEN) Intel VT-d Interrupt Remapping* not* supported. >>>>> Are you sure you have been operating on the same host? :-) >>>>> >>>>> -- Dexuan >>>>> >>>>> ------------------------------ >>>>> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >>>>> *Sent:* 2009年6月29日 17:01 >>>>> *To:* Cui, Dexuan >>>>> *Cc:* xen-devel@lists.xensource.com >>>>> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell >>>>> e6400, with iommu=1 >>>>> >>>>> Hi, >>>>> >>>>> Well, just tried what you suggested, and here are the results: >>>>> * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with >>>>> iommu=1 >>>>> The strange thing, is that after it booted one time ok (with the second >>>>> patch), then it continued to boot ok, even if i reverted the second patch. >>>>> >>>>> The relevant lines in the log are: >>>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >>>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >>>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >>>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >>>>> >>>>> Anyway, here is the full Xen log, with the first patch (print cap''s) >>>>> applied: >>>>> >>>>> __ __ _____ _ _ ___ >>>>> \ \/ /___ _ __ |___ /| || | / _ \ >>>>> \ // _ \ ''_ \ |_ \| || |_| | | | >>>>> / \ __/ | | | ___) |__ _| |_| | >>>>> /_/\_\___|_| |_| |____(_) |_|(_)___/ >>>>> >>>>> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian >>>>> 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 >>>>> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 >>>>> 19629:b2128a88e3dd >>>>> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 >>>>> iommu_inclusive_mapping loglvl=all guest_loglvl=all >>>>> (XEN) Video information: >>>>> (XEN) VGA is text mode 80x25, font 8x16 >>>>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >>>>> (XEN) Disc information: >>>>> (XEN) Found 1 MBR signatures >>>>> (XEN) Found 1 EDD information structures >>>>> (XEN) Xen-e820 RAM map: >>>>> (XEN) 0000000000000000 - 000000000009f000 (usable) >>>>> (XEN) 000000000009f000 - 00000000000a0000 (reserved) >>>>> (XEN) 0000000000100000 - 000000007c04d400 (usable) >>>>> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) >>>>> (XEN) 000000007c04f400 - 000000007f000000 (reserved) >>>>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >>>>> (XEN) 00000000fec00000 - 00000000fec10000 (reserved) >>>>> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) >>>>> (XEN) 00000000fed20000 - 00000000fed90000 (reserved) >>>>> (XEN) 00000000feda0000 - 00000000feda6000 (reserved) >>>>> (XEN) 00000000fee00000 - 00000000fee10000 (reserved) >>>>> (XEN) 00000000ffe60000 - 0000000100000000 (reserved) >>>>> (XEN) System RAM: 1983MB (2031536kB) >>>>> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) >>>>> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL >>>>> 61) >>>>> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL >>>>> 61) >>>>> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL >>>>> 20050624) >>>>> (XEN) ACPI: FACS 7C060C00, 0040 >>>>> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL >>>>> 61) >>>>> (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL >>>>> 61) >>>>> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL >>>>> 47) >>>>> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D >>>>> ASL 61) >>>>> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D >>>>> ASL 61) >>>>> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 >>>>> ASL 0) >>>>> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL >>>>> 61) >>>>> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL >>>>> 20050624) >>>>> (XEN) NUMA turned off >>>>> (XEN) Faking a node at 0000000000000000-000000007c04d000 >>>>> (XEN) Domain heap initialised >>>>> (XEN) DMI 2.4 present. >>>>> (XEN) Using APIC driver default >>>>> (XEN) ACPI: PM-Timer IO Port: 0x1008 >>>>> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] >>>>> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] >>>>> (XEN) ACPI: Local APIC address 0xfee00000 >>>>> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) >>>>> (XEN) Processor #0 7:7 APIC version 20 >>>>> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) >>>>> (XEN) Processor #1 7:7 APIC version 20 >>>>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) >>>>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) >>>>> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) >>>>> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 >>>>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) >>>>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) >>>>> (XEN) ACPI: IRQ0 used by override. >>>>> (XEN) ACPI: IRQ2 used by override. >>>>> (XEN) ACPI: IRQ9 used by override. >>>>> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs >>>>> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 >>>>> (XEN) [VT-D]dmar.c:485: Host address width 36 >>>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 >>>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 >>>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 >>>>> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL >>>>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 >>>>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>>>> (XEN) Intel VT-d DMAR tables have been parsed. >>>>> (XEN) Using ACPI (MADT) for SMP configuration information >>>>> (XEN) Using scheduler: SMP Credit Scheduler (credit) >>>>> (XEN) Initializing CPU#0 >>>>> (XEN) Detected 2527.068 MHz processor. >>>>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>>>> (XEN) CPU: L2 cache: 6144K >>>>> (XEN) CPU: Physical Processor ID: 0 >>>>> (XEN) CPU: Processor Core ID: 0 >>>>> (XEN) VMX: Supported advanced features: >>>>> (XEN) - APIC MMIO access virtualisation >>>>> (XEN) - APIC TPR shadow >>>>> (XEN) - Virtual NMI >>>>> (XEN) - MSR direct-access bitmap >>>>> (XEN) HVM: VMX enabled >>>>> (XEN) Intel machine check reporting enabled on CPU#0. >>>>> (XEN) mce_init: init bank1 >>>>> (XEN) mce_init: init bank2 >>>>> (XEN) mce_init: init bank3 >>>>> (XEN) mce_init: init bank4 >>>>> (XEN) mce_init: init bank5 >>>>> (XEN) CPU0: Thermal monitoring enabled (TM2) >>>>> (XEN) CMCI: CPU0 has no CMCI support >>>>> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >>>>> (XEN) Booting processor 1/1 eip 8c000 >>>>> (XEN) Initializing CPU#1 >>>>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>>>> (XEN) CPU: L2 cache: 6144K >>>>> (XEN) CPU: Physical Processor ID: 0 >>>>> (XEN) CPU: Processor Core ID: 1 >>>>> (XEN) Intel machine check reporting enabled on CPU#1. >>>>> (XEN) mce_init: init bank1 >>>>> (XEN) mce_init: init bank2 >>>>> (XEN) mce_init: init bank4 >>>>> (XEN) mce_init: init bank5 >>>>> (XEN) CPU1: Thermal monitoring enabled (TM2) >>>>> (XEN) CMCI: CPU1 has no CMCI support >>>>> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >>>>> (XEN) Total of 2 processors activated. >>>>> (XEN) ENABLING IO-APIC IRQs >>>>> (XEN) -> Using new ACK method >>>>> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 >>>>> (XEN) checking TSC synchronization across 2 CPUs: passed. >>>>> (XEN) Platform timer is 14.318MHz HPET >>>>> (XEN) Brought up 2 CPUs >>>>> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >>>>> (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush >>>>> (XEN) Intel VT-d Snoop Control not supported. >>>>> (XEN) Intel VT-d DMA Passthrough not supported. >>>>> (XEN) Intel VT-d Queued Invalidation not supported. >>>>> (XEN) Intel VT-d Interrupt Remapping not supported. >>>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >>>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >>>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >>>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >>>>> (XEN) I/O virtualisation enabled >>>>> (XEN) I/O virtualisation for PV guests disabled >>>>> (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast >>>>> (XEN) ACPI sleep modes: S3 >>>>> (XEN) mcheck_poll: Machine check polling timer started. >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:19.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.1 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.2 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.7 >>>>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf >>>>> 0:1b.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.1 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.2 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.7 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1f.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1f.2 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1f.3 >>>>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf >>>>> c:0.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.1 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.2 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.7 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.1 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.2 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.7 >>>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>>> ffff828bfff57000 >>>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>>> ffff828bfff56000 >>>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>>> ffff828bfff55000 >>>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>>> ffff828bfff54000 >>>>> (XEN) *** LOADING DOMAIN 0 *** >>>>> (XEN) Xen kernel: 64-bit, lsb, compat32 >>>>> (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> >>>>> 0xffffffff8070ab48 >>>>> (XEN) PHYSICAL MEMORY ARRANGEMENT: >>>>> (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages >>>>> to be allocated) >>>>> (XEN) VIRTUAL MEMORY ARRANGEMENT: >>>>> (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 >>>>> (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 >>>>> (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 >>>>> (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 >>>>> (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 >>>>> (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 >>>>> (XEN) TOTAL: ffffffff80000000->ffffffff82000000 >>>>> (XEN) ENTRY ADDRESS: ffffffff80200000 >>>>> (XEN) Dom0 has maximum 2 VCPUs >>>>> (XEN) Scrubbing Free RAM: .done. >>>>> (XEN) Xen trace buffers: disabled >>>>> (XEN) Std. Loglevel: All >>>>> (XEN) Guest Loglevel: All >>>>> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch >>>>> input to Xen) >>>>> (XEN) Freed 132kB init memory. >>>>> >>>>> Tom >>>>> >>>>> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >>>>> >>>>>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c >>>>>> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 >>>>>> +0100 >>>>>> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 >>>>>> +0800 >>>>>> @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) >>>>>> } >>>>>> dma_msi_data_init(iommu, iommu->vector); >>>>>> dma_msi_addr_init(iommu, >>>>>> cpu_physical_id(first_cpu(cpu_online_map))); >>>>>> - clear_fault_bits(iommu); >>>>>> + //clear_fault_bits(iommu); >>>>>> dmar_writel(iommu->reg, DMAR_FECTL_REG, 0); >>>>>> >>>>>> /* initialize flush functions */ >>>>>> >>>>>> And, can you comment the invocation of clear_fault_bits() in >>>>>> init_vtd_hw() like above and see if the host can boot fine? >>>>>> >>>>>> Thanks, >>>>>> -- Dexuan >>>>>> >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>> From: Cui, Dexuan >>>>>> Sent: 2009?6?29? 14:30 >>>>>> To: ''Tom Rotenberg''; xen-devel@lists.xensource.com >>>>>> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >>>>>> with iommu=1 >>>>>> >>>>>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c >>>>>> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 >>>>>> +0100 >>>>>> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 >>>>>> +0800 >>>>>> @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm >>>>>> { >>>>>> u64 val; >>>>>> >>>>>> + printk("clear_fault_bits: CAP_REG=0x%llx\n", >>>>>> + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); >>>>>> val = dmar_readq( >>>>>> iommu->reg, >>>>>> >>>>>> cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8); >>>>>> >>>>>> Hi Tom, >>>>>> Can you please apply the above patch and send us the entire log? >>>>>> (I don''t have the same host, so I can''t try it myself.) >>>>>> >>>>>> Thanks, >>>>>> -- Dexuan >>>>>> >>>>>> ________________________________ >>>>>> >>>>>> From: xen-devel-bounces@lists.xensource.com [mailto: >>>>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >>>>>> Sent: 2009?6?28? 22:56 >>>>>> To: xen-devel@lists.xensource.com >>>>>> Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >>>>>> with iommu=1 >>>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to >>>>>> boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. I >>>>>> have also tried to boot with and without the ''iommu_inclusive_mapping'' boot >>>>>> option, but it didn''t seem to help at all. >>>>>> >>>>>> Does anyone knows how to solve this issue? >>>>>> >>>>>> These are the last messages in the UART: >>>>>> >>>>>> (XEN) Brought up 2 CPUs >>>>>> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >>>>>> (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush >>>>>> (XEN) Intel VT-d Snoop Control supported. >>>>>> (XEN) Intel VT-d DMA Passthrough not supported. >>>>>> (XEN) Intel VT-d Queued Invalidation supported. >>>>>> (XEN) Intel VT-d Interrupt Remapping supported. >>>>>> (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- >>>>>> (XEN) CPU: 0 >>>>>> (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >>>>>> (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor >>>>>> (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: >>>>>> 0000000000000020 >>>>>> (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: >>>>>> ffff830078f82f90 >>>>>> (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: >>>>>> ffff83007be2c110 >>>>>> (XEN) r9: 000000000000001a r10: 0000000000000005 r11: >>>>>> 0000000000000005 >>>>>> (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: >>>>>> ffff830078f82fcc >>>>>> (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: >>>>>> 00000000000026f0 >>>>>> (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 >>>>>> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 >>>>>> (XEN) Xen stack trace from rsp=ffff828c8026fda0: >>>>>> (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f >>>>>> ffff828c80204490 >>>>>> (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 >>>>>> 0000000000000000 >>>>>> (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 >>>>>> 000000000000001f >>>>>> (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 >>>>>> ffff828c80205f60 >>>>>> (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 >>>>>> ffff828c80227978 >>>>>> (XEN) ffff828c8021a589 0000000000000000 0000000000000000 >>>>>> 0000000000000000 >>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>> 0000000000000000 >>>>>> (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 >>>>>> 000000000008bf40 >>>>>> (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 >>>>>> 0000000000000000 >>>>>> (XEN) ffffffff00000000 0000000800000000 000000010000006e >>>>>> 0000000000000003 >>>>>> (XEN) 00000000000002f8 0000000000000000 0000000000000000 >>>>>> 0000000000067e2c >>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>> 0000000000000000 >>>>>> (XEN) ffff828c801000b5 0000000000000000 0000000000000000 >>>>>> 0000000000000000 >>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>> 0000000000000000 >>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>> 0000000000000000 >>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>> 0000000000000000 >>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>> 0000000000000000 >>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>> 0000000000000000 >>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>> ffff830079f18000 >>>>>> (XEN) Xen call trace: >>>>>> (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >>>>>> (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 >>>>>> (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 >>>>>> (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 >>>>>> (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 >>>>>> (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 >>>>>> (XEN) >>>>>> (XEN) Pagetable walk from ffff828bfff5aff8: >>>>>> (XEN) L4[0x105] = 000000007be7a027 5555555555555555 >>>>>> (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 >>>>>> (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 >>>>>> (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff >>>>>> (XEN) >>>>>> (XEN) **************************************** >>>>>> (XEN) Panic on CPU 0: >>>>>> (XEN) FATAL PAGE FAULT >>>>>> (XEN) [error_code=0000] >>>>>> (XEN) Faulting linear address: ffff828bfff5aff8 >>>>>> (XEN) **************************************** >>>>>> (XEN) >>>>>> (XEN) Reboot in five seconds... >>>>>> >>>>>> >>>>>>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Cui, Dexuan
2009-Jun-30 02:54 UTC
RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
> CAP_REG=0xffffffffffffffffHW obviously doesn''t report the value correctly. So, looks the issue is: when we change the iommu option in BIOS (e.g., from disabled to enabled), to make it really take effect, we have to make a "cold shutdown (shutdown the power supply throughly)" to the host ; or else, though Xen can see the ACPI DMAR tables，the HW doesn''t respond to the MMIO accesses to the iommu registers at all -- such read access would return -1 (all-''1''s) and write access would be ignored. I remember you said after you commented the clear_fault_bits(), the host could boot fine with iommu=1 --- I guess in this case actually the DMA remapping is not enabled at all though Xen thinks HW tells it DMA remapping is enabled (Xen writes DMA_GCMD_TE into GCMD (this write is ignored) and polls the GSTS; since reading GSTS returns -1, Xen thinks TE is on, however the underlying HW doesn''t enable DMA remapping!). Please note: this is only my deduction, anyway. :-) You can try to create some hvm guests with devices assigned, I think the host would die weirdly since the DMAs of the assigned devices of the hvm guests are corrupting the host memory! This can be a way to verify my deduction. And I remember you said after you commented the clear_fault_bits() and booted the host fine, you tried to uncomment the line and the host still boots fine! -- this is strange to me, I guess you might have done a cold shutdown before your uncommenting the line? One more thing: I think your HW doesn''t support SnoopControl/QueuedInvalidation/InterruptRemapping, but after you commented the clear_fault_bits(), LOOKS the host could boot fine even after Xen tries to enable them! Maybe the specific HW implementation of your host could account for this... I think changing the iommu option in BIOS should not require a cold shutdown to the host. I''ll try to raise this issue to the BIOS people. For now, you could work around the issue by remember doing a cold shutdown every time you change the iommu BIOS option. Hope this is acceptable to you for now. :-) Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com] Sent: 2009年6月29日 22:15 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 in steps 1 & 4, i mean disabling and enabling IOMMU in the BIOS (and not in Xen boot parameter). And, this is what probably caused all of this. I am using my own version (above Xen 3.4.0), but i haven''t made any changes in the hyervisor itself (but rather in the hvmloader and rombios), so it''s not related to this. I prefer not to try the latest Xen, unless there is a very good reason for this, as it take long time to compile, and i can''t do it right now. I have reproduced the panic with the print, and here is the relevant part of the log: (XEN) clear_fault_bits: CAP_REG=0xffffffffffffffff (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000000 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff828c801f0564 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fd98 r8: 0000000000004000 (XEN) r9: 0000000000003fff r10: ffff828c80288360 r11: 0000000080287ea0 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fd98: (XEN) ffff830078f82f90 ffff828c8012d35d ffff830078f824b0 ffff828c801e4257 (XEN) ffff828c80204490 ffff828c801e4257 ffff828c802044c0 ffff828c8026fdf8 (XEN) 0000000000000000 ffff828c8012deae ffff830078f82d40 ffff828c80228100 (XEN) 000000000000001f 00000000ffffffed ffff828c80227a78 0000000000000001 (XEN) ffff828c80205f60 ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 (XEN) ffff828c80227978 ffff828c8021a589 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 00000000002b23c0 ffffff8c80274000 0008bf3000000000 (XEN) 000000000008bf30 0000000000000000 ffff83000008bfc0 0000000001ce08a0 (XEN) 0000000000000000 ffffffff00000000 0000000800000000 000000010000006e (XEN) 0000000000000003 00000000000002f8 0000000000000000 0000000000000000 (XEN) 0000000000067e9c 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffff828c801000b5 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0 (XEN) [<ffff828c8012d35d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012deae>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> When you meet with the panic, can you apply the printk log to print CAP_REG? In your step 1 and 4, you just mean using the xen parameter iommu=0 and iommu=1, correct? During theses steps, you don''t change any BIOS setting and there is not any HW change, correct? I see you''re using "3.4.0 19629:b2128a88e3dd", but 19629''s checksum should be 69293fc2096f (http://xenbits.xensource.com/xen-3.4-testing.hg?rev/69293fc2096f). So I assume you made your own changes to the code? Can you try the latest changeset 19677: b2ddeb734ca7(http://xenbits.xensource.com/xen-3.4-testing.hg)? Sorry, I don''t have the same type of host, so I can''t try it myself. :-( Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 21:09 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Well, it looks like my analysis was right. This is how i reproduced it: 1. Disable the IOMMU 2. Turn off the laptop, and leave it turned off for a few seconds 3. Turn on the laptop 4. Enable the IOMMU (*DON''T* turn off the machine) 5. Continue 6. Xen crashes and, now the log lines are: (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. It looks like there is a problem when enabling IOMMU, without cold boot of the machine. What do u say? Here is the full log: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:40:11 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 00B0 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.037 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fda0: (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 (XEN) ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 ffff828c80227978 (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 00000000002b23c0 ffffff8c80274000 0008bf3000000000 000000000008bf30 (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... 2009/6/29 Tom Rotenberg <tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>> I''m almost 100% sure, that this was from the same machine (since i have only 1 such Dell currently on my desk :). However, this log was generated just after turning on the IOMMU (without cold boot). So, let me check if it happened because of this. I''ll check it now... 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> I have no idea about it.> (XEN) Intel VT-d Snoop Control supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported.This is from the log of your original mail. We can see SC/QI/IR are supported, but in your later log, we can see they are not supported. I''m curious where the first log comes from. :-) -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 20:38 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 1) Yep, i can''t reproduce it anymore. Maybe i''ll try it with another Dell R6400 i have here. 2) On my Lenovo T400, i don''t remember the exact crash, and i don''t have the log of that crash, i just remember something about "DMAR ... failure", which happened, after i turned on the IOMMU, and didn''t perform a cold reboot on that machine. Do u have any idea why these problems happen? Does anyone else working with Dell E6400 had this problem? Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> It''s really weird... :-) If I understand correctly, with iommu=1 in Xen: 1) on your "Dell E6400 machine", now you can''t reproduce the panic any longer; 2) on your "Lenovo T400", you can reproduce the panic sometimes -- if so, can you post the log (with the printk patch applied) when you meet with the panic? Can you find out under what condition it''s easy to reproduce the panic? Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 20:15 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, Tried again, this time with cold reboot, it worked again. I also checked the lines you were talking about, and now they are: (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. Could it be that there is some kind of problem with the hardware, which makes it sometimes report invalid values? (with the Lenovo T400 i''m testing, if you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on boot, so you must do a cold reboot). Here is the log: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.058 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 (XEN) TOTAL: ffffffff80000000->ffffffff82000000 (XEN) ENTRY ADDRESS: ffffffff80200000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 132kB init memory. Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> So, if I understand correctly, it''s really strange you can''t reproduce the panic now. With only the first printk patch applied (and don''t comment the clear_fault_bits()), can you try a cold rebooting the host (plug the power cord off from the host; wait several minutes; plug the cord in and tunn on the host)? And the log? BTW, in your original mail, the log shows: (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. But in your this new log, it shows: (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. Are you sure you have been operating on the same host? :-) -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 17:01 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, Well, just tried what you suggested, and here are the results: * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with iommu=1 The strange thing, is that after it booted one time ok (with the second patch), then it continued to boot ok, even if i reverted the second patch. The relevant lines in the log are: (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 Anyway, here is the full Xen log, with the first patch (print cap''s) applied: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.068 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 (XEN) TOTAL: ffffffff80000000->ffffffff82000000 (XEN) ENTRY ADDRESS: ffffffff80200000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 132kB init memory. Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800 @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) } dma_msi_data_init(iommu, iommu->vector); dma_msi_addr_init(iommu, cpu_physical_id(first_cpu(cpu_online_map))); - clear_fault_bits(iommu); + //clear_fault_bits(iommu); dmar_writel(iommu->reg, DMAR_FECTL_REG, 0); /* initialize flush functions */ And, can you comment the invocation of clear_fault_bits() in init_vtd_hw() like above and see if the host can boot fine? Thanks, -- Dexuan -----Original Message----- From: Cui, Dexuan Sent: 2009?6?29? 14:30 To: ''Tom Rotenberg''; xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800 @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm { u64 val; + printk("clear_fault_bits: CAP_REG=0x%llx\n", + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); val = dmar_readq( iommu->reg, cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8); Hi Tom, Can you please apply the above patch and send us the entire log? (I don''t have the same host, so I can''t try it myself.) Thanks, -- Dexuan ________________________________ From: xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com> [mailto:xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>] On Behalf Of Tom Rotenberg Sent: 2009?6?28? 22:56 To: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. I have also tried to boot with and without the ''iommu_inclusive_mapping'' boot option, but it didn''t seem to help at all. Does anyone knows how to solve this issue? These are the last messages in the UART: (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fda0: (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978 (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40 (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Cui, Dexuan
2009-Jun-30 03:03 UTC
RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
Tim, Can you tell me the BIOS version of your Dell E6400 host? BTW, please try to use the latest BIOS if possible, and please check your BIOS document to see if there is any known IOMMU-related issue. Thanks, -- Dexuan ________________________________ From: Cui, Dexuan Sent: 2009年6月30日 10:55 To: ''Tom Rotenberg'' Cc: xen-devel@lists.xensource.com Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1> CAP_REG=0xffffffffffffffffHW obviously doesn''t report the value correctly. So, looks the issue is: when we change the iommu option in BIOS (e.g., from disabled to enabled), to make it really take effect, we have to make a "cold shutdown (shutdown the power supply throughly)" to the host ; or else, though Xen can see the ACPI DMAR tables，the HW doesn''t respond to the MMIO accesses to the iommu registers at all -- such read access would return -1 (all-''1''s) and write access would be ignored. I remember you said after you commented the clear_fault_bits(), the host could boot fine with iommu=1 --- I guess in this case actually the DMA remapping is not enabled at all though Xen thinks HW tells it DMA remapping is enabled (Xen writes DMA_GCMD_TE into GCMD (this write is ignored) and polls the GSTS; since reading GSTS returns -1, Xen thinks TE is on, however the underlying HW doesn''t enable DMA remapping!). Please note: this is only my deduction, anyway. :-) You can try to create some hvm guests with devices assigned, I think the host would die weirdly since the DMAs of the assigned devices of the hvm guests are corrupting the host memory! This can be a way to verify my deduction. And I remember you said after you commented the clear_fault_bits() and booted the host fine, you tried to uncomment the line and the host still boots fine! -- this is strange to me, I guess you might have done a cold shutdown before your uncommenting the line? One more thing: I think your HW doesn''t support SnoopControl/QueuedInvalidation/InterruptRemapping, but after you commented the clear_fault_bits(), LOOKS the host could boot fine even after Xen tries to enable them! Maybe the specific HW implementation of your host could account for this... I think changing the iommu option in BIOS should not require a cold shutdown to the host. I''ll try to raise this issue to the BIOS people. For now, you could work around the issue by remember doing a cold shutdown every time you change the iommu BIOS option. Hope this is acceptable to you for now. :-) Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com] Sent: 2009年6月29日 22:15 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 in steps 1 & 4, i mean disabling and enabling IOMMU in the BIOS (and not in Xen boot parameter). And, this is what probably caused all of this. I am using my own version (above Xen 3.4.0), but i haven''t made any changes in the hyervisor itself (but rather in the hvmloader and rombios), so it''s not related to this. I prefer not to try the latest Xen, unless there is a very good reason for this, as it take long time to compile, and i can''t do it right now. I have reproduced the panic with the print, and here is the relevant part of the log: (XEN) clear_fault_bits: CAP_REG=0xffffffffffffffff (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000000 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff828c801f0564 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fd98 r8: 0000000000004000 (XEN) r9: 0000000000003fff r10: ffff828c80288360 r11: 0000000080287ea0 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fd98: (XEN) ffff830078f82f90 ffff828c8012d35d ffff830078f824b0 ffff828c801e4257 (XEN) ffff828c80204490 ffff828c801e4257 ffff828c802044c0 ffff828c8026fdf8 (XEN) 0000000000000000 ffff828c8012deae ffff830078f82d40 ffff828c80228100 (XEN) 000000000000001f 00000000ffffffed ffff828c80227a78 0000000000000001 (XEN) ffff828c80205f60 ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 (XEN) ffff828c80227978 ffff828c8021a589 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 00000000002b23c0 ffffff8c80274000 0008bf3000000000 (XEN) 000000000008bf30 0000000000000000 ffff83000008bfc0 0000000001ce08a0 (XEN) 0000000000000000 ffffffff00000000 0000000800000000 000000010000006e (XEN) 0000000000000003 00000000000002f8 0000000000000000 0000000000000000 (XEN) 0000000000067e9c 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffff828c801000b5 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0 (XEN) [<ffff828c8012d35d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012deae>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> When you meet with the panic, can you apply the printk log to print CAP_REG? In your step 1 and 4, you just mean using the xen parameter iommu=0 and iommu=1, correct? During theses steps, you don''t change any BIOS setting and there is not any HW change, correct? I see you''re using "3.4.0 19629:b2128a88e3dd", but 19629''s checksum should be 69293fc2096f (http://xenbits.xensource.com/xen-3.4-testing.hg?rev/69293fc2096f). So I assume you made your own changes to the code? Can you try the latest changeset 19677: b2ddeb734ca7(http://xenbits.xensource.com/xen-3.4-testing.hg)? Sorry, I don''t have the same type of host, so I can''t try it myself. :-( Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 21:09 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Well, it looks like my analysis was right. This is how i reproduced it: 1. Disable the IOMMU 2. Turn off the laptop, and leave it turned off for a few seconds 3. Turn on the laptop 4. Enable the IOMMU (*DON''T* turn off the machine) 5. Continue 6. Xen crashes and, now the log lines are: (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. It looks like there is a problem when enabling IOMMU, without cold boot of the machine. What do u say? Here is the full log: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:40:11 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 00B0 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.037 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fda0: (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 (XEN) ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 ffff828c80227978 (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 00000000002b23c0 ffffff8c80274000 0008bf3000000000 000000000008bf30 (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... 2009/6/29 Tom Rotenberg <tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>> I''m almost 100% sure, that this was from the same machine (since i have only 1 such Dell currently on my desk :). However, this log was generated just after turning on the IOMMU (without cold boot). So, let me check if it happened because of this. I''ll check it now... 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> I have no idea about it.> (XEN) Intel VT-d Snoop Control supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported.This is from the log of your original mail. We can see SC/QI/IR are supported, but in your later log, we can see they are not supported. I''m curious where the first log comes from. :-) -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 20:38 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 1) Yep, i can''t reproduce it anymore. Maybe i''ll try it with another Dell R6400 i have here. 2) On my Lenovo T400, i don''t remember the exact crash, and i don''t have the log of that crash, i just remember something about "DMAR ... failure", which happened, after i turned on the IOMMU, and didn''t perform a cold reboot on that machine. Do u have any idea why these problems happen? Does anyone else working with Dell E6400 had this problem? Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> It''s really weird... :-) If I understand correctly, with iommu=1 in Xen: 1) on your "Dell E6400 machine", now you can''t reproduce the panic any longer; 2) on your "Lenovo T400", you can reproduce the panic sometimes -- if so, can you post the log (with the printk patch applied) when you meet with the panic? Can you find out under what condition it''s easy to reproduce the panic? Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 20:15 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, Tried again, this time with cold reboot, it worked again. I also checked the lines you were talking about, and now they are: (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. Could it be that there is some kind of problem with the hardware, which makes it sometimes report invalid values? (with the Lenovo T400 i''m testing, if you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on boot, so you must do a cold reboot). Here is the log: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.058 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 (XEN) TOTAL: ffffffff80000000->ffffffff82000000 (XEN) ENTRY ADDRESS: ffffffff80200000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 132kB init memory. Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> So, if I understand correctly, it''s really strange you can''t reproduce the panic now. With only the first printk patch applied (and don''t comment the clear_fault_bits()), can you try a cold rebooting the host (plug the power cord off from the host; wait several minutes; plug the cord in and tunn on the host)? And the log? BTW, in your original mail, the log shows: (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. But in your this new log, it shows: (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. Are you sure you have been operating on the same host? :-) -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 17:01 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, Well, just tried what you suggested, and here are the results: * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with iommu=1 The strange thing, is that after it booted one time ok (with the second patch), then it continued to boot ok, even if i reverted the second patch. The relevant lines in the log are: (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 Anyway, here is the full Xen log, with the first patch (print cap''s) applied: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.068 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 (XEN) TOTAL: ffffffff80000000->ffffffff82000000 (XEN) ENTRY ADDRESS: ffffffff80200000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 132kB init memory. Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800 @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) } dma_msi_data_init(iommu, iommu->vector); dma_msi_addr_init(iommu, cpu_physical_id(first_cpu(cpu_online_map))); - clear_fault_bits(iommu); + //clear_fault_bits(iommu); dmar_writel(iommu->reg, DMAR_FECTL_REG, 0); /* initialize flush functions */ And, can you comment the invocation of clear_fault_bits() in init_vtd_hw() like above and see if the host can boot fine? Thanks, -- Dexuan -----Original Message----- From: Cui, Dexuan Sent: 2009?6?29? 14:30 To: ''Tom Rotenberg''; xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800 @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm { u64 val; + printk("clear_fault_bits: CAP_REG=0x%llx\n", + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); val = dmar_readq( iommu->reg, cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8); Hi Tom, Can you please apply the above patch and send us the entire log? (I don''t have the same host, so I can''t try it myself.) Thanks, -- Dexuan ________________________________ From: xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com> [mailto:xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>] On Behalf Of Tom Rotenberg Sent: 2009?6?28? 22:56 To: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. I have also tried to boot with and without the ''iommu_inclusive_mapping'' boot option, but it didn''t seem to help at all. Does anyone knows how to solve this issue? These are the last messages in the UART: (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fda0: (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978 (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40 (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Sander Eikelenboom
2009-Jun-30 06:55 UTC
Re[2]: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
Hmm i remember experiencing also problems with not cold booting after changing the VT-D setting in BIOS. Since then i allways cold boot after changing the setting (which is not that often) Tuesday, June 30, 2009, 4:54:40 AM, you wrote:>> CAP_REG=0xffffffffffffffff > HW obviously doesn''t report the value correctly.> So, looks the issue is: when we change the iommu option in BIOS (e.g., > from disabled to enabled), to make it really take effect, we have to make > a "cold shutdown (shutdown the power supply throughly)" to the host ; or > else, though Xen can see the ACPI DMAR tables，the HW doesn''t respond to > the MMIO accesses to the iommu registers at all -- such read access would > return -1 (all-''1''s) and write access would be ignored.> I remember you said after you commented the clear_fault_bits(), the host > could boot fine with iommu=1 --- I guess in this case actually the DMA > remapping is not enabled at all though Xen thinks HW tells it DMA > remapping is enabled (Xen writes DMA_GCMD_TE into GCMD (this write is > ignored) and polls the GSTS; since reading GSTS returns -1, Xen thinks > TE is on, however the underlying HW doesn''t enable DMA remapping!). > Please note: this is only my deduction, anyway. :-) You can try to > create some hvm guests with devices assigned, I think the host would die > weirdly since the DMAs of the assigned devices of the hvm guests are > corrupting the host memory! This can be a way to verify my deduction. > And I remember you said after you commented the clear_fault_bits() and > booted the host fine, you tried to uncomment the line and the host still > boots fine! -- this is strange to me, I guess you might have done a cold > shutdown before your uncommenting the line? > One more thing: I think your HW doesn''t support > SnoopControl/QueuedInvalidation/InterruptRemapping, but after you > commented the clear_fault_bits(), LOOKS the host could boot fine even > after Xen tries to enable them! Maybe the specific HW implementation of your host could account for this...> I think changing the iommu option in BIOS should not require a cold > shutdown to the host. I''ll try to raise this issue to the BIOS people. > For now, you could work around the issue by remember doing a cold > shutdown every time you change the iommu BIOS option. Hope this is acceptable to you for now. :-)> Thanks, > -- Dexuan> ________________________________ > From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com] > Sent: 2009年6月29日 22:15 > To: Cui, Dexuan > Cc: xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1> in steps 1 & 4, i mean disabling and enabling IOMMU in the BIOS (and not > in Xen boot parameter). And, this is what probably caused all of this.> I am using my own version (above Xen 3.4.0), but i haven''t made any > changes in the hyervisor itself (but rather in the hvmloader and > rombios), so it''s not related to this. I prefer not to try the latest > Xen, unless there is a very good reason for this, as it take long time to > compile, and i can''t do it right now.> I have reproduced the panic with the print, and here is the relevant part of the log:> (XEN) clear_fault_bits: CAP_REG=0xffffffffffffffff > (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- > (XEN) CPU: 0 > (XEN) RIP: e008:[<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0 > (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor > (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000000 > (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff828c801f0564 > (XEN) rbp: 0000000000000090 rsp: ffff828c8026fd98 r8: 0000000000004000 > (XEN) r9: 0000000000003fff r10: ffff828c80288360 r11: 0000000080287ea0 > (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc > (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 > (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 > (XEN) Xen stack trace from rsp=ffff828c8026fd98: > (XEN) ffff830078f82f90 ffff828c8012d35d ffff830078f824b0 ffff828c801e4257 > (XEN) ffff828c80204490 ffff828c801e4257 ffff828c802044c0 ffff828c8026fdf8 > (XEN) 0000000000000000 ffff828c8012deae ffff830078f82d40 ffff828c80228100 > (XEN) 000000000000001f 00000000ffffffed ffff828c80227a78 0000000000000001 > (XEN) ffff828c80205f60 ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 > (XEN) ffff828c80227978 ffff828c8021a589 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 00000000002b23c0 ffffff8c80274000 0008bf3000000000 > (XEN) 000000000008bf30 0000000000000000 ffff83000008bfc0 0000000001ce08a0 > (XEN) 0000000000000000 ffffffff00000000 0000000800000000 000000010000006e > (XEN) 0000000000000003 00000000000002f8 0000000000000000 0000000000000000 > (XEN) 0000000000067e9c 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 ffff828c801000b5 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) ffff830079f18000 > (XEN) Xen call trace: > (XEN) [<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0 > (XEN) [<ffff828c8012d35d>] init_vtd_hw+0x15d/0x3e0 > (XEN) [<ffff828c8012deae>] intel_vtd_setup+0x5ae/0x610 > (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 > (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 > (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 > (XEN) > (XEN) Pagetable walk from ffff828bfff5aff8: > (XEN) L4[0x105] = 000000007be7a027 5555555555555555 > (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 > (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 > (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff > (XEN) > (XEN) **************************************** > (XEN) Panic on CPU 0: > (XEN) FATAL PAGE FAULT > (XEN) [error_code=0000] > (XEN) Faulting linear address: ffff828bfff5aff8 > (XEN) **************************************** > (XEN) > (XEN) Reboot in five seconds...> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> > When you meet with the panic, can you apply the printk log to print CAP_REG?> In your step 1 and 4, you just mean using the xen parameter iommu=0 and iommu=1, correct? > During theses steps, you don''t change any BIOS setting and there is not any HW change, correct?> I see you''re using "3.4.0 19629:b2128a88e3dd", but 19629''s checksum > should be 69293fc2096f > (http://xenbits.xensource.com/xen-3.4-testing.hg?rev/69293fc2096f). > So I assume you made your own changes to the code? > Can you try the latest changeset 19677: > b2ddeb734ca7(http://xenbits.xensource.com/xen-3.4-testing.hg)?> Sorry, I don''t have the same type of host, so I can''t try it myself. :-(> Thanks, > -- Dexuan> ________________________________ > From: Tom Rotenberg > [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] > Sent: 2009年6月29日 21:09> To: Cui, Dexuan > Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> > Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1> Well, it looks like my analysis was right. > This is how i reproduced it: > 1. Disable the IOMMU > 2. Turn off the laptop, and leave it turned off for a few seconds > 3. Turn on the laptop > 4. Enable the IOMMU (*DON''T* turn off the machine) > 5. Continue > 6. Xen crashes> and, now the log lines are: > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported.> It looks like there is a problem when enabling IOMMU, without cold boot of the machine.> What do u say?> Here is the full log: > __ __ _____ _ _ ___ > \ \/ /___ _ __ |___ /| || | / _ \ > \ // _ \ ''_ \ |_ \| || |_| | | | > / \ __/ | | | ___) |__ _| |_| | > /_/\_\___|_| |_| |____(_) |_|(_)___/> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian > 4.3.2-1.1) ) Mon Jun 29 07:40:11 EDT 2009 > (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd > (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 > iommu_inclusive_mapping loglvl=all guest_loglvl=all > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 1 MBR signatures > (XEN) Found 1 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009f000 (usable) > (XEN) 000000000009f000 - 00000000000a0000 (reserved) > (XEN) 0000000000100000 - 000000007c04d400 (usable) > (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) > (XEN) 000000007c04f400 - 000000007f000000 (reserved) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec10000 (reserved) > (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) > (XEN) 00000000fed20000 - 00000000fed90000 (reserved) > (XEN) 00000000feda0000 - 00000000feda6000 (reserved) > (XEN) 00000000fee00000 - 00000000fee10000 (reserved) > (XEN) 00000000ffe60000 - 0000000100000000 (reserved) > (XEN) System RAM: 1983MB (2031536kB) > (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) > (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) > (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) > (XEN) ACPI: FACS 7C060C00, 0040 > (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) > (XEN) ACPI: DMAR 7C060400, 00B0 (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) > (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) > (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) > (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) > (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) > (XEN) NUMA turned off > (XEN) Faking a node at 0000000000000000-000000007c04d000 > (XEN) Domain heap initialised > (XEN) DMI 2.4 present. > (XEN) Using APIC driver default > (XEN) ACPI: PM-Timer IO Port: 0x1008 > (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] > (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] > (XEN) ACPI: Local APIC address 0xfee00000 > (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) > (XEN) Processor #0 7:7 APIC version 20 > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) > (XEN) Processor #1 7:7 APIC version 20 > (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) > (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) > (XEN) ACPI: IRQ0 used by override. > (XEN) ACPI: IRQ2 used by override. > (XEN) ACPI: IRQ9 used by override. > (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs > (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 > (XEN) [VT-D]dmar.c:485: Host address width 36 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 > (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL > (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 > (XEN) Intel VT-d DMAR tables have been parsed. > (XEN) Using ACPI (MADT) for SMP configuration information > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Initializing CPU#0 > (XEN) Detected 2527.037 MHz processor. > (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K > (XEN) CPU: L2 cache: 6144K > (XEN) CPU: Physical Processor ID: 0 > (XEN) CPU: Processor Core ID: 0 > (XEN) VMX: Supported advanced features: > (XEN) - APIC MMIO access virtualisation > (XEN) - APIC TPR shadow > (XEN) - Virtual NMI > (XEN) - MSR direct-access bitmap > (XEN) HVM: VMX enabled > (XEN) Intel machine check reporting enabled on CPU#0. > (XEN) mce_init: init bank1 > (XEN) mce_init: init bank2 > (XEN) mce_init: init bank3 > (XEN) mce_init: init bank4 > (XEN) mce_init: init bank5 > (XEN) CPU0: Thermal monitoring enabled (TM2) > (XEN) CMCI: CPU0 has no CMCI support > (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 > (XEN) Booting processor 1/1 eip 8c000 > (XEN) Initializing CPU#1 > (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K > (XEN) CPU: L2 cache: 6144K > (XEN) CPU: Physical Processor ID: 0 > (XEN) CPU: Processor Core ID: 1 > (XEN) Intel machine check reporting enabled on CPU#1. > (XEN) mce_init: init bank1 > (XEN) mce_init: init bank2 > (XEN) mce_init: init bank4 > (XEN) mce_init: init bank5 > (XEN) CPU1: Thermal monitoring enabled (TM2) > (XEN) CMCI: CPU1 has no CMCI support > (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 > (XEN) Total of 2 processors activated. > (XEN) ENABLING IO-APIC IRQs(XEN) ->> Using new ACK method> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 > (XEN) checking TSC synchronization across 2 CPUs: passed. > (XEN) Platform timer is 14.318MHz HPET > (XEN) Brought up 2 CPUs > (XEN) microcode.c:73:d32767 microcode: CPU1 resumed > (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush > (XEN) Intel VT-d Snoop Control supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported. > (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- > (XEN) CPU: 0 > (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 > (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor > (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 > (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 > (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 > (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 > (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc > (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 > (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 > (XEN) Xen stack trace from rsp=ffff828c8026fda0: > (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 > (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 > (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f > (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 > (XEN) ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 ffff828c80227978 > (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 00000000002b23c0 ffffff8c80274000 0008bf3000000000 000000000008bf30 > (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 > (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 > (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 > (XEN) Xen call trace: > (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 > (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 > (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 > (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 > (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 > (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 > (XEN) > (XEN) Pagetable walk from ffff828bfff5aff8: > (XEN) L4[0x105] = 000000007be7a027 5555555555555555 > (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 > (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 > (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff > (XEN) > (XEN) **************************************** > (XEN) Panic on CPU 0: > (XEN) FATAL PAGE FAULT > (XEN) [error_code=0000] > (XEN) Faulting linear address: ffff828bfff5aff8 > (XEN) **************************************** > (XEN) > (XEN) Reboot in five seconds...> 2009/6/29 Tom Rotenberg > <tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>> > I''m almost 100% sure, that this was from the same machine (since i have > only 1 such Dell currently on my desk :). > However, this log was generated just after turning on the IOMMU (without > cold boot). So, let me check if it happened because of this.> I''ll check it now...> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> > I have no idea about it.>> (XEN) Intel VT-d Snoop Control supported. >> (XEN) Intel VT-d DMA Passthrough not supported. >> (XEN) Intel VT-d Queued Invalidation supported. >> (XEN) Intel VT-d Interrupt Remapping supported. > This is from the log of your original mail. We can see SC/QI/IR are > supported, but in your later log, we can see they are not supported. > I''m curious where the first log comes from. :-)> -- Dexuan> ________________________________ > From: Tom Rotenberg > [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] > Sent: 2009年6月29日 20:38> To: Cui, Dexuan > Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> > Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1> 1) Yep, i can''t reproduce it anymore. Maybe i''ll try it with another Dell R6400 i have here. > 2) On my Lenovo T400, i don''t remember the exact crash, and i don''t have > the log of that crash, i just remember something about "DMAR ... > failure", which happened, after i turned on the IOMMU, and didn''t perform a cold reboot on that machine.> Do u have any idea why these problems happen? > Does anyone else working with Dell E6400 had this problem?> Tom> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> > It''s really weird... :-)> If I understand correctly, with iommu=1 in Xen: > 1) on your "Dell E6400 machine", now you can''t reproduce the panic any longer; > 2) on your "Lenovo T400", you can reproduce the panic sometimes -- if > so, can you post the log (with the printk patch applied) when you meet > with the panic? Can you find out under what condition it''s easy to reproduce the panic?> Thanks, > -- Dexuan> ________________________________ > From: Tom Rotenberg > [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] > Sent: 2009年6月29日 20:15> To: Cui, Dexuan > Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> > Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1> Hi,> Tried again, this time with cold reboot, it worked again. I also checked > the lines you were talking about, and now they are: > (XEN) Intel VT-d Snoop Control not supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation not supported. > (XEN) Intel VT-d Interrupt Remapping not supported.> Could it be that there is some kind of problem with the hardware, which > makes it sometimes report invalid values? (with the Lenovo T400 i''m > testing, if you enable the IOMMU, without a cold-reboot, it may cause Xen > to crash on boot, so you must do a cold reboot).> Here is the log: > __ __ _____ _ _ ___ > \ \/ /___ _ __ |___ /| || | / _ \ > \ // _ \ ''_ \ |_ \| || |_| | | | > / \ __/ | | | ___) |__ _| |_| | > /_/\_\___|_| |_| |____(_) |_|(_)___/> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian > 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009 > (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd > (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 > iommu_inclusive_mapping loglvl=all guest_loglvl=all > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 1 MBR signatures > (XEN) Found 1 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009f000 (usable) > (XEN) 000000000009f000 - 00000000000a0000 (reserved) > (XEN) 0000000000100000 - 000000007c04d400 (usable) > (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) > (XEN) 000000007c04f400 - 000000007f000000 (reserved) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec10000 (reserved) > (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) > (XEN) 00000000fed20000 - 00000000fed90000 (reserved) > (XEN) 00000000feda0000 - 00000000feda6000 (reserved) > (XEN) 00000000fee00000 - 00000000fee10000 (reserved) > (XEN) 00000000ffe60000 - 0000000100000000 (reserved) > (XEN) System RAM: 1983MB (2031536kB) > (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) > (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) > (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) > (XEN) ACPI: FACS 7C060C00, 0040 > (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) > (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) > (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) > (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) > (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) > (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) > (XEN) NUMA turned off > (XEN) Faking a node at 0000000000000000-000000007c04d000 > (XEN) Domain heap initialised > (XEN) DMI 2.4 present. > (XEN) Using APIC driver default > (XEN) ACPI: PM-Timer IO Port: 0x1008 > (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] > (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] > (XEN) ACPI: Local APIC address 0xfee00000 > (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) > (XEN) Processor #0 7:7 APIC version 20 > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) > (XEN) Processor #1 7:7 APIC version 20 > (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) > (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) > (XEN) ACPI: IRQ0 used by override. > (XEN) ACPI: IRQ2 used by override. > (XEN) ACPI: IRQ9 used by override. > (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs > (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 > (XEN) [VT-D]dmar.c:485: Host address width 36 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 > (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL > (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 > (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 > (XEN) Intel VT-d DMAR tables have been parsed. > (XEN) Using ACPI (MADT) for SMP configuration information > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Initializing CPU#0 > (XEN) Detected 2527.058 MHz processor. > (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K > (XEN) CPU: L2 cache: 6144K > (XEN) CPU: Physical Processor ID: 0 > (XEN) CPU: Processor Core ID: 0 > (XEN) VMX: Supported advanced features: > (XEN) - APIC MMIO access virtualisation > (XEN) - APIC TPR shadow > (XEN) - Virtual NMI > (XEN) - MSR direct-access bitmap > (XEN) HVM: VMX enabled > (XEN) Intel machine check reporting enabled on CPU#0. > (XEN) mce_init: init bank1 > (XEN) mce_init: init bank2 > (XEN) mce_init: init bank3 > (XEN) mce_init: init bank4 > (XEN) mce_init: init bank5 > (XEN) CPU0: Thermal monitoring enabled (TM2) > (XEN) CMCI: CPU0 has no CMCI support > (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 > (XEN) Booting processor 1/1 eip 8c000 > (XEN) Initializing CPU#1 > (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K > (XEN) CPU: L2 cache: 6144K > (XEN) CPU: Physical Processor ID: 0 > (XEN) CPU: Processor Core ID: 1 > (XEN) Intel machine check reporting enabled on CPU#1. > (XEN) mce_init: init bank1 > (XEN) mce_init: init bank2 > (XEN) mce_init: init bank4 > (XEN) mce_init: init bank5 > (XEN) CPU1: Thermal monitoring enabled (TM2) > (XEN) CMCI: CPU1 has no CMCI support > (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 > (XEN) Total of 2 processors activated. > (XEN) ENABLING IO-APIC IRQs(XEN) ->> Using new ACK method> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 > (XEN) checking TSC synchronization across 2 CPUs: passed. > (XEN) Platform timer is 14.318MHz HPET > (XEN) Brought up 2 CPUs > (XEN) microcode.c:73:d32767 microcode: CPU1 resumed > (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush > (XEN) Intel VT-d Snoop Control not supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation not supported. > (XEN) Intel VT-d Interrupt Remapping not supported. > (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 > (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 > (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 > (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 > (XEN) I/O virtualisation enabled > (XEN) I/O virtualisation for PV guests disabled > (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast > (XEN) ACPI sleep modes: S3 > (XEN) mcheck_poll: Machine check polling timer started. > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 > (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 > (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000 > (XEN) *** LOADING DOMAIN 0 *** > (XEN) Xen kernel: 64-bit, lsb, compat32 > (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 > (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 > (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 > (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 > (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 > (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 > (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 > (XEN) TOTAL: ffffffff80000000->ffffffff82000000 > (XEN) ENTRY ADDRESS: ffffffff80200000 > (XEN) Dom0 has maximum 2 VCPUs > (XEN) Scrubbing Free RAM: .done. > (XEN) Xen trace buffers: disabled > (XEN) Std. Loglevel: All > (XEN) Guest Loglevel: All > (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) > (XEN) Freed 132kB init memory.> Tom> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> > So, if I understand correctly, it''s really strange you can''t reproduce the panic now.> With only the first printk patch applied (and don''t comment the > clear_fault_bits()), can you try a cold rebooting the host (plug the > power cord off from the host; wait several minutes; plug the cord in and tunn on the host)? And the log?> BTW, in your original mail, the log shows: > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported.> But in your this new log, it shows: > (XEN) Intel VT-d Queued Invalidation not supported. > (XEN) Intel VT-d Interrupt Remapping not supported. > Are you sure you have been operating on the same host? :-)> -- Dexuan> ________________________________ > From: Tom Rotenberg > [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] > Sent: 2009年6月29日 17:01 > To: Cui, Dexuan > Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> > Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1> Hi,> Well, just tried what you suggested, and here are the results: > * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with iommu=1 > The strange thing, is that after it booted one time ok (with the second > patch), then it continued to boot ok, even if i reverted the second patch.> The relevant lines in the log are: > (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 > (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 > (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 > (XEN) clear_fault_bits: CAP_REG=0xc9008020630260> Anyway, here is the full Xen log, with the first patch (print cap''s) applied:> __ __ _____ _ _ ___ > \ \/ /___ _ __ |___ /| || | / _ \ > \ // _ \ ''_ \ |_ \| || |_| | | | > / \ __/ | | | ___) |__ _| |_| | > /_/\_\___|_| |_| |____(_) |_|(_)___/> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian > 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 > (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd > (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 > iommu_inclusive_mapping loglvl=all guest_loglvl=all > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 1 MBR signatures > (XEN) Found 1 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 000000000009f000 (usable) > (XEN) 000000000009f000 - 00000000000a0000 (reserved) > (XEN) 0000000000100000 - 000000007c04d400 (usable) > (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) > (XEN) 000000007c04f400 - 000000007f000000 (reserved) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec10000 (reserved) > (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) > (XEN) 00000000fed20000 - 00000000fed90000 (reserved) > (XEN) 00000000feda0000 - 00000000feda6000 (reserved) > (XEN) 00000000fee00000 - 00000000fee10000 (reserved) > (XEN) 00000000ffe60000 - 0000000100000000 (reserved) > (XEN) System RAM: 1983MB (2031536kB) > (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) > (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) > (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) > (XEN) ACPI: FACS 7C060C00, 0040 > (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) > (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) > (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) > (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) > (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) > (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) > (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) > (XEN) NUMA turned off > (XEN) Faking a node at 0000000000000000-000000007c04d000 > (XEN) Domain heap initialised > (XEN) DMI 2.4 present. > (XEN) Using APIC driver default > (XEN) ACPI: PM-Timer IO Port: 0x1008 > (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] > (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] > (XEN) ACPI: Local APIC address 0xfee00000 > (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) > (XEN) Processor #0 7:7 APIC version 20 > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) > (XEN) Processor #1 7:7 APIC version 20 > (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) > (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) > (XEN) ACPI: IRQ0 used by override. > (XEN) ACPI: IRQ2 used by override. > (XEN) ACPI: IRQ9 used by override. > (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs > (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 > (XEN) [VT-D]dmar.c:485: Host address width 36 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 > (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD > (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 > (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL > (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 > (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 > (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 > (XEN) Intel VT-d DMAR tables have been parsed. > (XEN) Using ACPI (MADT) for SMP configuration information > (XEN) Using scheduler: SMP Credit Scheduler (credit) > (XEN) Initializing CPU#0 > (XEN) Detected 2527.068 MHz processor. > (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K > (XEN) CPU: L2 cache: 6144K > (XEN) CPU: Physical Processor ID: 0 > (XEN) CPU: Processor Core ID: 0 > (XEN) VMX: Supported advanced features: > (XEN) - APIC MMIO access virtualisation > (XEN) - APIC TPR shadow > (XEN) - Virtual NMI > (XEN) - MSR direct-access bitmap > (XEN) HVM: VMX enabled > (XEN) Intel machine check reporting enabled on CPU#0. > (XEN) mce_init: init bank1 > (XEN) mce_init: init bank2 > (XEN) mce_init: init bank3 > (XEN) mce_init: init bank4 > (XEN) mce_init: init bank5 > (XEN) CPU0: Thermal monitoring enabled (TM2) > (XEN) CMCI: CPU0 has no CMCI support > (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 > (XEN) Booting processor 1/1 eip 8c000 > (XEN) Initializing CPU#1 > (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K > (XEN) CPU: L2 cache: 6144K > (XEN) CPU: Physical Processor ID: 0 > (XEN) CPU: Processor Core ID: 1 > (XEN) Intel machine check reporting enabled on CPU#1. > (XEN) mce_init: init bank1 > (XEN) mce_init: init bank2 > (XEN) mce_init: init bank4 > (XEN) mce_init: init bank5 > (XEN) CPU1: Thermal monitoring enabled (TM2) > (XEN) CMCI: CPU1 has no CMCI support > (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 > (XEN) Total of 2 processors activated. > (XEN) ENABLING IO-APIC IRQs(XEN) ->> Using new ACK method> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 > (XEN) checking TSC synchronization across 2 CPUs: passed. > (XEN) Platform timer is 14.318MHz HPET > (XEN) Brought up 2 CPUs > (XEN) microcode.c:73:d32767 microcode: CPU1 resumed > (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush > (XEN) Intel VT-d Snoop Control not supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation not supported. > (XEN) Intel VT-d Interrupt Remapping not supported. > (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 > (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 > (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 > (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 > (XEN) I/O virtualisation enabled > (XEN) I/O virtualisation for PV guests disabled > (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast > (XEN) ACPI sleep modes: S3 > (XEN) mcheck_poll: Machine check polling timer started. > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 > (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 > (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 > (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000 > (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000 > (XEN) *** LOADING DOMAIN 0 *** > (XEN) Xen kernel: 64-bit, lsb, compat32 > (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 > (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 > (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 > (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 > (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 > (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 > (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 > (XEN) TOTAL: ffffffff80000000->ffffffff82000000 > (XEN) ENTRY ADDRESS: ffffffff80200000 > (XEN) Dom0 has maximum 2 VCPUs > (XEN) Scrubbing Free RAM: .done. > (XEN) Xen trace buffers: disabled > (XEN) Std. Loglevel: All > (XEN) Guest Loglevel: All > (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) > (XEN) Freed 132kB init memory.> Tom> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> > diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c > --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 > +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800 > @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) > } > dma_msi_data_init(iommu, iommu->vector); > dma_msi_addr_init(iommu, > cpu_physical_id(first_cpu(cpu_online_map))); > - clear_fault_bits(iommu); > + //clear_fault_bits(iommu); > dmar_writel(iommu->reg, DMAR_FECTL_REG, 0);> /* initialize flush functions */> And, can you comment the invocation of clear_fault_bits() in > init_vtd_hw() like above and see if the host can boot fine?> Thanks, > -- Dexuan> -----Original Message----- > From: Cui, Dexuan > Sent: 2009?6?29? 14:30 > To: ''Tom Rotenberg''; > xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> > Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c > --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 > +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800 > @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm > { > u64 val;> + printk("clear_fault_bits: CAP_REG=0x%llx\n", > + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); > val = dmar_readq( > iommu->reg, > cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8);> Hi Tom, > Can you please apply the above patch and send us the entire log? > (I don''t have the same host, so I can''t try it myself.)> Thanks, > -- Dexuan> ________________________________> From: > xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com> > [mailto:xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>] On Behalf Of Tom Rotenberg > Sent: 2009?6?28? 22:56 > To: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> > Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1> Hi,> I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to > boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. > I have also tried to boot with and without the ''iommu_inclusive_mapping'' > boot option, but it didn''t seem to help at all.> Does anyone knows how to solve this issue?> These are the last messages in the UART:> (XEN) Brought up 2 CPUs > (XEN) microcode.c:73:d32767 microcode: CPU1 resumed > (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush > (XEN) Intel VT-d Snoop Control supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported. > (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- > (XEN) CPU: 0 > (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 > (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor > (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 > (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 > (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 > (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 > (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc > (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 > (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 > (XEN) Xen stack trace from rsp=ffff828c8026fda0: > (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 > (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 > (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f > (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 > (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978 > (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40 > (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 > (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 > (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 > (XEN) Xen call trace: > (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 > (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 > (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 > (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 > (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 > (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 > (XEN) > (XEN) Pagetable walk from ffff828bfff5aff8: > (XEN) L4[0x105] = 000000007be7a027 5555555555555555 > (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 > (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 > (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff > (XEN) > (XEN) **************************************** > (XEN) Panic on CPU 0: > (XEN) FATAL PAGE FAULT > (XEN) [error_code=0000] > (XEN) Faulting linear address: ffff828bfff5aff8 > (XEN) **************************************** > (XEN) > (XEN) Reboot in five seconds...-- Best regards, Sander mailto:linux@eikelenboom.it _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Jun-30 07:41 UTC
Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
Dell E6400, BIOS version: A12 I hope this issue will be solved by the BIOS people, since i think i saw the same issue with my Lenovo T400, and it looks like you *must* do a cold boot, on both Lenovo & Dell. Please update me on this issue, if you''ll have any news. Thanks for your help! 2009/6/30 Cui, Dexuan <dexuan.cui@intel.com>> Tim, > Can you tell me the BIOS version of your Dell E6400 host? > BTW, please try to use the latest BIOS if possible, and please check your > BIOS document to see if there is any known IOMMU-related issue. > > Thanks, > -- Dexuan > > > ------------------------------ > *From:* Cui, Dexuan > *Sent:* 2009年6月30日 10:55 > *To:* ''Tom Rotenberg'' > *Cc:* xen-devel@lists.xensource.com > *Subject:* RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, > with iommu=1 > > > CAP_REG=0xffffffffffffffff > HW obviously doesn''t report the value correctly. > > So, looks the issue is: when we change the iommu option in BIOS (e.g., from > disabled to enabled), to make it really take effect, we have to make a "cold > shutdown (shutdown the power supply throughly)" to the host ; or else, > though Xen can see the ACPI DMAR tables，the HW doesn''t respond to the MMIO > accesses to the iommu registers at all -- such read access would return -1 > (all-''1''s) and write access would be ignored. > > I remember you said after you commented the clear_fault_bits(), the host > could boot fine with iommu=1 --- I guess in this case actually the DMA > remapping is not enabled at all though Xen thinks HW tells it DMA remapping > is enabled (Xen writes DMA_GCMD_TE into GCMD (this write is ignored) and > polls the GSTS; since reading GSTS returns -1, Xen thinks TE is on, however > the underlying HW doesn''t enable DMA remapping!). > Please note: this is only my deduction, anyway. :-) You can try to > create some hvm guests with devices assigned, I think the host would die > weirdly since the DMAs of the assigned devices of the hvm guests are > corrupting the host memory! This can be a way to verify my deduction. > And I remember you said after you commented the clear_fault_bits() and > booted the host fine, you tried to uncomment the line and the host still > boots fine! -- this is strange to me, I guess you might have done a cold > shutdown before your uncommenting the line? > One more thing: I think your HW doesn''t support > SnoopControl/QueuedInvalidation/InterruptRemapping, but after you commented > the clear_fault_bits(), LOOKS the host could boot fine even after Xen tries > to enable them! Maybe the specific HW implementation of your host could > account for this... > > I think changing the iommu option in BIOS should not require a cold > shutdown to the host. I''ll try to raise this issue to the BIOS people. > For now, you could work around the issue by remember doing a cold shutdown > every time you change the iommu BIOS option. Hope this is acceptable to you > for now. :-) > > Thanks, > -- Dexuan > > > ------------------------------ > *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] > *Sent:* 2009年6月29日 22:15 > > *To:* Cui, Dexuan > *Cc:* xen-devel@lists.xensource.com > *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, > with iommu=1 > > in steps 1 & 4, i mean disabling and enabling IOMMU in the BIOS (and not > in Xen boot parameter). And, this is what probably caused all of this. > > I am using my own version (above Xen 3.4.0), but i haven''t made any changes > in the hyervisor itself (but rather in the hvmloader and rombios), so it''s > not related to this. I prefer not to try the latest Xen, unless there is a > very good reason for this, as it take long time to compile, and i can''t do > it right now. > > I have reproduced the panic with the print, and here is the relevant part > of the log: > > (XEN) clear_fault_bits: CAP_REG=0xffffffffffffffff > (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- > (XEN) CPU: 0 > (XEN) RIP: e008:[<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0 > (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor > (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000000 > (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff828c801f0564 > (XEN) rbp: 0000000000000090 rsp: ffff828c8026fd98 r8: 0000000000004000 > (XEN) r9: 0000000000003fff r10: ffff828c80288360 r11: 0000000080287ea0 > (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc > (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 > (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 > (XEN) Xen stack trace from rsp=ffff828c8026fd98: > (XEN) ffff830078f82f90 ffff828c8012d35d ffff830078f824b0 > ffff828c801e4257 > (XEN) ffff828c80204490 ffff828c801e4257 ffff828c802044c0 > ffff828c8026fdf8 > (XEN) 0000000000000000 ffff828c8012deae ffff830078f82d40 > ffff828c80228100 > (XEN) 000000000000001f 00000000ffffffed ffff828c80227a78 > 0000000000000001 > (XEN) ffff828c80205f60 ffff828c801f3400 ffff83000008bf30 > ffff828c8012a9d4 > (XEN) ffff828c80227978 ffff828c8021a589 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 00000000002b23c0 ffffff8c80274000 > 0008bf3000000000 > (XEN) 000000000008bf30 0000000000000000 ffff83000008bfc0 > 0000000001ce08a0 > (XEN) 0000000000000000 ffffffff00000000 0000000800000000 > 000000010000006e > (XEN) 0000000000000003 00000000000002f8 0000000000000000 > 0000000000000000 > (XEN) 0000000000067e9c 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 ffff828c801000b5 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) ffff830079f18000 > (XEN) Xen call trace: > (XEN) [<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0 > (XEN) [<ffff828c8012d35d>] init_vtd_hw+0x15d/0x3e0 > (XEN) [<ffff828c8012deae>] intel_vtd_setup+0x5ae/0x610 > (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 > (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 > (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 > (XEN) > (XEN) Pagetable walk from ffff828bfff5aff8: > (XEN) L4[0x105] = 000000007be7a027 5555555555555555 > (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 > (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 > (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff > (XEN) > (XEN) **************************************** > (XEN) Panic on CPU 0: > (XEN) FATAL PAGE FAULT > (XEN) [error_code=0000] > (XEN) Faulting linear address: ffff828bfff5aff8 > (XEN) **************************************** > (XEN) > (XEN) Reboot in five seconds... > > 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> > >> When you meet with the panic, can you apply the printk log to print >> CAP_REG? >> >> In your step 1 and 4, you just mean using the xen parameter iommu=0 and >> iommu=1, correct? >> During theses steps, you don''t change any BIOS setting and there is not >> any HW change, correct? >> >> I see you''re using "3.4.0 19629:b2128a88e3dd", but 19629''s checksum should >> be 69293fc2096f ( >> http://xenbits.xensource.com/xen-3.4-testing.hg?rev/69293fc2096f). >> So I assume you made your own changes to the code? >> Can you try the latest changeset 19677: b2ddeb734ca7( >> http://xenbits.xensource.com/xen-3.4-testing.hg)? >> >> Sorry, I don''t have the same type of host, so I can''t try it myself. :-( >> >> Thanks, >> -- Dexuan >> >> >> ------------------------------ >> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >> *Sent:* 2009年6月29日 21:09 >> >> *To:* Cui, Dexuan >> *Cc:* xen-devel@lists.xensource.com >> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >> with iommu=1 >> >> Well, it looks like my analysis was right. >> This is how i reproduced it: >> 1. Disable the IOMMU >> 2. Turn off the laptop, and leave it turned off for a few seconds >> 3. Turn on the laptop >> 4. Enable the IOMMU (*DON''T* turn off the machine) >> 5. Continue >> 6. Xen crashes >> >> and, now the log lines are: >> (XEN) Intel VT-d Queued Invalidation supported. >> (XEN) Intel VT-d Interrupt Remapping supported. >> >> It looks like there is a problem when enabling IOMMU, without cold boot of >> the machine. >> >> What do u say? >> >> Here is the full log: >> __ __ _____ _ _ ___ >> \ \/ /___ _ __ |___ /| || | / _ \ >> \ // _ \ ''_ \ |_ \| || |_| | | | >> / \ __/ | | | ___) |__ _| |_| | >> /_/\_\___|_| |_| |____(_) |_|(_)___/ >> >> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian >> 4.3.2-1.1) ) Mon Jun 29 07:40:11 EDT 2009 >> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd >> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 >> iommu_inclusive_mapping loglvl=all guest_loglvl=all >> (XEN) Video information: >> (XEN) VGA is text mode 80x25, font 8x16 >> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >> (XEN) Disc information: >> (XEN) Found 1 MBR signatures >> (XEN) Found 1 EDD information structures >> (XEN) Xen-e820 RAM map: >> (XEN) 0000000000000000 - 000000000009f000 (usable) >> (XEN) 000000000009f000 - 00000000000a0000 (reserved) >> (XEN) 0000000000100000 - 000000007c04d400 (usable) >> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) >> (XEN) 000000007c04f400 - 000000007f000000 (reserved) >> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >> (XEN) 00000000fec00000 - 00000000fec10000 (reserved) >> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) >> (XEN) 00000000fed20000 - 00000000fed90000 (reserved) >> (XEN) 00000000feda0000 - 00000000feda6000 (reserved) >> (XEN) 00000000fee00000 - 00000000fee10000 (reserved) >> (XEN) 00000000ffe60000 - 0000000100000000 (reserved) >> (XEN) System RAM: 1983MB (2031536kB) >> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) >> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL >> 20050624) >> (XEN) ACPI: FACS 7C060C00, 0040 >> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL >> 61) >> (XEN) ACPI: DMAR 7C060400, 00B0 (r1 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL >> 47) >> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL >> 0) >> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL >> 61) >> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL >> 20050624) >> (XEN) NUMA turned off >> (XEN) Faking a node at 0000000000000000-000000007c04d000 >> (XEN) Domain heap initialised >> (XEN) DMI 2.4 present. >> (XEN) Using APIC driver default >> (XEN) ACPI: PM-Timer IO Port: 0x1008 >> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] >> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] >> (XEN) ACPI: Local APIC address 0xfee00000 >> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) >> (XEN) Processor #0 7:7 APIC version 20 >> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) >> (XEN) Processor #1 7:7 APIC version 20 >> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) >> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) >> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) >> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 >> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) >> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) >> (XEN) ACPI: IRQ0 used by override. >> (XEN) ACPI: IRQ2 used by override. >> (XEN) ACPI: IRQ9 used by override. >> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs >> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 >> (XEN) [VT-D]dmar.c:485: Host address width 36 >> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 >> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 >> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL >> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 >> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 >> (XEN) Intel VT-d DMAR tables have been parsed. >> (XEN) Using ACPI (MADT) for SMP configuration information >> (XEN) Using scheduler: SMP Credit Scheduler (credit) >> (XEN) Initializing CPU#0 >> (XEN) Detected 2527.037 MHz processor. >> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >> (XEN) CPU: L2 cache: 6144K >> (XEN) CPU: Physical Processor ID: 0 >> (XEN) CPU: Processor Core ID: 0 >> (XEN) VMX: Supported advanced features: >> (XEN) - APIC MMIO access virtualisation >> (XEN) - APIC TPR shadow >> (XEN) - Virtual NMI >> (XEN) - MSR direct-access bitmap >> (XEN) HVM: VMX enabled >> (XEN) Intel machine check reporting enabled on CPU#0. >> (XEN) mce_init: init bank1 >> (XEN) mce_init: init bank2 >> (XEN) mce_init: init bank3 >> (XEN) mce_init: init bank4 >> (XEN) mce_init: init bank5 >> (XEN) CPU0: Thermal monitoring enabled (TM2) >> (XEN) CMCI: CPU0 has no CMCI support >> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >> (XEN) Booting processor 1/1 eip 8c000 >> (XEN) Initializing CPU#1 >> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >> (XEN) CPU: L2 cache: 6144K >> (XEN) CPU: Physical Processor ID: 0 >> (XEN) CPU: Processor Core ID: 1 >> (XEN) Intel machine check reporting enabled on CPU#1. >> (XEN) mce_init: init bank1 >> (XEN) mce_init: init bank2 >> (XEN) mce_init: init bank4 >> (XEN) mce_init: init bank5 >> (XEN) CPU1: Thermal monitoring enabled (TM2) >> (XEN) CMCI: CPU1 has no CMCI support >> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >> (XEN) Total of 2 processors activated. >> (XEN) ENABLING IO-APIC IRQs >> (XEN) -> Using new ACK method >> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 >> (XEN) checking TSC synchronization across 2 CPUs: passed. >> (XEN) Platform timer is 14.318MHz HPET >> (XEN) Brought up 2 CPUs >> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >> (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush >> (XEN) Intel VT-d Snoop Control supported. >> (XEN) Intel VT-d DMA Passthrough not supported. >> (XEN) Intel VT-d Queued Invalidation supported. >> (XEN) Intel VT-d Interrupt Remapping supported. >> (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- >> (XEN) CPU: 0 >> (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >> (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor >> (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: >> 0000000000000020 >> (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: >> ffff830078f82f90 >> (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: >> ffff83007be2c110 >> (XEN) r9: 000000000000001a r10: 0000000000000005 r11: >> 0000000000000005 >> (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: >> ffff830078f82fcc >> (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: >> 00000000000026f0 >> (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 >> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 >> (XEN) Xen stack trace from rsp=ffff828c8026fda0: >> (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f >> ffff828c80204490 >> (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 >> 0000000000000000 >> (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 >> 000000000000001f >> (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 >> ffff828c80205f60 >> (XEN) ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 >> ffff828c80227978 >> (XEN) ffff828c8021a589 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 00000000002b23c0 ffffff8c80274000 0008bf3000000000 >> 000000000008bf30 >> (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 >> 0000000000000000 >> (XEN) ffffffff00000000 0000000800000000 000000010000006e >> 0000000000000003 >> (XEN) 00000000000002f8 0000000000000000 0000000000000000 >> 0000000000067e2c >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) ffff828c801000b5 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 0000000000000000 >> ffff830079f18000 >> (XEN) Xen call trace: >> (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >> (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 >> (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 >> (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 >> (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 >> (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 >> (XEN) >> (XEN) Pagetable walk from ffff828bfff5aff8: >> (XEN) L4[0x105] = 000000007be7a027 5555555555555555 >> (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 >> (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 >> (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff >> (XEN) >> (XEN) **************************************** >> (XEN) Panic on CPU 0: >> (XEN) FATAL PAGE FAULT >> (XEN) [error_code=0000] >> (XEN) Faulting linear address: ffff828bfff5aff8 >> (XEN) **************************************** >> (XEN) >> (XEN) Reboot in five seconds... >> >> >> 2009/6/29 Tom Rotenberg <tom.rotenberg@gmail.com> >> >>> I''m almost 100% sure, that this was from the same machine (since i have >>> only 1 such Dell currently on my desk :). >>> However, this log was generated just after turning on the IOMMU (without >>> cold boot). So, let me check if it happened because of this. >>> >>> I''ll check it now... >>> >>> >>> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >>> >>>> I have no idea about it. >>>> >>>> > (XEN) Intel VT-d Snoop Control supported. >>>> > (XEN) Intel VT-d DMA Passthrough not supported. >>>> > (XEN) Intel VT-d Queued Invalidation supported. >>>> > (XEN) Intel VT-d Interrupt Remapping supported. >>>> This is from the log of your original mail. We can see SC/QI/IR are >>>> supported, but in your later log, we can see they are not supported. >>>> I''m curious where the first log comes from. :-) >>>> >>>> -- Dexuan >>>> >>>> ------------------------------ >>>> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >>>> *Sent:* 2009年6月29日 20:38 >>>> >>>> *To:* Cui, Dexuan >>>> *Cc:* xen-devel@lists.xensource.com >>>> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell >>>> e6400, with iommu=1 >>>> >>>> >>>> 1) Yep, i can''t reproduce it anymore. Maybe i''ll try it with another >>>> Dell R6400 i have here. >>>> 2) On my Lenovo T400, i don''t remember the exact crash, and i don''t have >>>> the log of that crash, i just remember something about "DMAR ... failure", >>>> which happened, after i turned on the IOMMU, and didn''t perform a cold >>>> reboot on that machine. >>>> >>>> Do u have any idea why these problems happen? >>>> Does anyone else working with Dell E6400 had this problem? >>>> >>>> Tom >>>> >>>> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >>>> >>>>> It''s really weird... :-) >>>>> >>>>> If I understand correctly, with iommu=1 in Xen: >>>>> 1) on your "Dell E6400 machine", now you can''t reproduce the panic any >>>>> longer; >>>>> 2) on your "Lenovo T400", you can reproduce the panic sometimes -- if >>>>> so, can you post the log (with the printk patch applied) when you meet with >>>>> the panic? Can you find out under what condition it''s easy to reproduce the >>>>> panic? >>>>> >>>>> >>>>> Thanks, >>>>> -- Dexuan >>>>> >>>>> >>>>> ------------------------------ >>>>> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >>>>> *Sent:* 2009年6月29日 20:15 >>>>> >>>>> *To:* Cui, Dexuan >>>>> *Cc:* xen-devel@lists.xensource.com >>>>> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell >>>>> e6400, with iommu=1 >>>>> >>>>> Hi, >>>>> >>>>> Tried again, this time with cold reboot, it worked again. I also >>>>> checked the lines you were talking about, and now they are: >>>>> (XEN) Intel VT-d Snoop Control not supported. >>>>> (XEN) Intel VT-d DMA Passthrough not supported. >>>>> (XEN) Intel VT-d Queued Invalidation not supported. >>>>> (XEN) Intel VT-d Interrupt Remapping not supported. >>>>> >>>>> Could it be that there is some kind of problem with the hardware, which >>>>> makes it sometimes report invalid values? (with the Lenovo T400 i''m testing, >>>>> if you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on >>>>> boot, so you must do a cold reboot). >>>>> >>>>> Here is the log: >>>>> __ __ _____ _ _ ___ >>>>> \ \/ /___ _ __ |___ /| || | / _ \ >>>>> \ // _ \ ''_ \ |_ \| || |_| | | | >>>>> / \ __/ | | | ___) |__ _| |_| | >>>>> /_/\_\___|_| |_| |____(_) |_|(_)___/ >>>>> >>>>> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian >>>>> 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009 >>>>> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 >>>>> 19629:b2128a88e3dd >>>>> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 >>>>> iommu_inclusive_mapping loglvl=all guest_loglvl=all >>>>> (XEN) Video information: >>>>> (XEN) VGA is text mode 80x25, font 8x16 >>>>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >>>>> (XEN) Disc information: >>>>> (XEN) Found 1 MBR signatures >>>>> (XEN) Found 1 EDD information structures >>>>> (XEN) Xen-e820 RAM map: >>>>> (XEN) 0000000000000000 - 000000000009f000 (usable) >>>>> (XEN) 000000000009f000 - 00000000000a0000 (reserved) >>>>> (XEN) 0000000000100000 - 000000007c04d400 (usable) >>>>> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) >>>>> (XEN) 000000007c04f400 - 000000007f000000 (reserved) >>>>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >>>>> (XEN) 00000000fec00000 - 00000000fec10000 (reserved) >>>>> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) >>>>> (XEN) 00000000fed20000 - 00000000fed90000 (reserved) >>>>> (XEN) 00000000feda0000 - 00000000feda6000 (reserved) >>>>> (XEN) 00000000fee00000 - 00000000fee10000 (reserved) >>>>> (XEN) 00000000ffe60000 - 0000000100000000 (reserved) >>>>> (XEN) System RAM: 1983MB (2031536kB) >>>>> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) >>>>> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL >>>>> 61) >>>>> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL >>>>> 61) >>>>> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL >>>>> 20050624) >>>>> (XEN) ACPI: FACS 7C060C00, 0040 >>>>> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL >>>>> 61) >>>>> (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL >>>>> 61) >>>>> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL >>>>> 47) >>>>> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D >>>>> ASL 61) >>>>> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D >>>>> ASL 61) >>>>> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 >>>>> ASL 0) >>>>> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL >>>>> 61) >>>>> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL >>>>> 20050624) >>>>> (XEN) NUMA turned off >>>>> (XEN) Faking a node at 0000000000000000-000000007c04d000 >>>>> (XEN) Domain heap initialised >>>>> (XEN) DMI 2.4 present. >>>>> (XEN) Using APIC driver default >>>>> (XEN) ACPI: PM-Timer IO Port: 0x1008 >>>>> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] >>>>> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] >>>>> (XEN) ACPI: Local APIC address 0xfee00000 >>>>> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) >>>>> (XEN) Processor #0 7:7 APIC version 20 >>>>> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) >>>>> (XEN) Processor #1 7:7 APIC version 20 >>>>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) >>>>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) >>>>> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) >>>>> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 >>>>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) >>>>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) >>>>> (XEN) ACPI: IRQ0 used by override. >>>>> (XEN) ACPI: IRQ2 used by override. >>>>> (XEN) ACPI: IRQ9 used by override. >>>>> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs >>>>> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 >>>>> (XEN) [VT-D]dmar.c:485: Host address width 36 >>>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 >>>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 >>>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 >>>>> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL >>>>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 >>>>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>>>> (XEN) Intel VT-d DMAR tables have been parsed. >>>>> (XEN) Using ACPI (MADT) for SMP configuration information >>>>> (XEN) Using scheduler: SMP Credit Scheduler (credit) >>>>> (XEN) Initializing CPU#0 >>>>> (XEN) Detected 2527.058 MHz processor. >>>>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>>>> (XEN) CPU: L2 cache: 6144K >>>>> (XEN) CPU: Physical Processor ID: 0 >>>>> (XEN) CPU: Processor Core ID: 0 >>>>> (XEN) VMX: Supported advanced features: >>>>> (XEN) - APIC MMIO access virtualisation >>>>> (XEN) - APIC TPR shadow >>>>> (XEN) - Virtual NMI >>>>> (XEN) - MSR direct-access bitmap >>>>> (XEN) HVM: VMX enabled >>>>> (XEN) Intel machine check reporting enabled on CPU#0. >>>>> (XEN) mce_init: init bank1 >>>>> (XEN) mce_init: init bank2 >>>>> (XEN) mce_init: init bank3 >>>>> (XEN) mce_init: init bank4 >>>>> (XEN) mce_init: init bank5 >>>>> (XEN) CPU0: Thermal monitoring enabled (TM2) >>>>> (XEN) CMCI: CPU0 has no CMCI support >>>>> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >>>>> (XEN) Booting processor 1/1 eip 8c000 >>>>> (XEN) Initializing CPU#1 >>>>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>>>> (XEN) CPU: L2 cache: 6144K >>>>> (XEN) CPU: Physical Processor ID: 0 >>>>> (XEN) CPU: Processor Core ID: 1 >>>>> (XEN) Intel machine check reporting enabled on CPU#1. >>>>> (XEN) mce_init: init bank1 >>>>> (XEN) mce_init: init bank2 >>>>> (XEN) mce_init: init bank4 >>>>> (XEN) mce_init: init bank5 >>>>> (XEN) CPU1: Thermal monitoring enabled (TM2) >>>>> (XEN) CMCI: CPU1 has no CMCI support >>>>> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 >>>>> (XEN) Total of 2 processors activated. >>>>> (XEN) ENABLING IO-APIC IRQs >>>>> (XEN) -> Using new ACK method >>>>> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 >>>>> (XEN) checking TSC synchronization across 2 CPUs: passed. >>>>> (XEN) Platform timer is 14.318MHz HPET >>>>> (XEN) Brought up 2 CPUs >>>>> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >>>>> (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush >>>>> (XEN) Intel VT-d Snoop Control not supported. >>>>> (XEN) Intel VT-d DMA Passthrough not supported. >>>>> (XEN) Intel VT-d Queued Invalidation not supported. >>>>> (XEN) Intel VT-d Interrupt Remapping not supported. >>>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >>>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >>>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >>>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >>>>> (XEN) I/O virtualisation enabled >>>>> (XEN) I/O virtualisation for PV guests disabled >>>>> (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast >>>>> (XEN) ACPI sleep modes: S3 >>>>> (XEN) mcheck_poll: Machine check polling timer started. >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:19.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.1 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.2 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.7 >>>>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf >>>>> 0:1b.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.1 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.2 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.7 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1f.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1f.2 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1f.3 >>>>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf >>>>> c:0.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.1 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.2 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1d.7 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.0 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.1 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.2 >>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>> 0:1a.7 >>>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>>> ffff828bfff57000 >>>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>>> ffff828bfff56000 >>>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>>> ffff828bfff55000 >>>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>>> ffff828bfff54000 >>>>> (XEN) *** LOADING DOMAIN 0 *** >>>>> (XEN) Xen kernel: 64-bit, lsb, compat32 >>>>> (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> >>>>> 0xffffffff8070ab48 >>>>> (XEN) PHYSICAL MEMORY ARRANGEMENT: >>>>> (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages >>>>> to be allocated) >>>>> (XEN) VIRTUAL MEMORY ARRANGEMENT: >>>>> (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 >>>>> (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 >>>>> (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 >>>>> (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 >>>>> (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 >>>>> (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 >>>>> (XEN) TOTAL: ffffffff80000000->ffffffff82000000 >>>>> (XEN) ENTRY ADDRESS: ffffffff80200000 >>>>> (XEN) Dom0 has maximum 2 VCPUs >>>>> (XEN) Scrubbing Free RAM: .done. >>>>> (XEN) Xen trace buffers: disabled >>>>> (XEN) Std. Loglevel: All >>>>> (XEN) Guest Loglevel: All >>>>> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch >>>>> input to Xen) >>>>> (XEN) Freed 132kB init memory. >>>>> >>>>> Tom >>>>> >>>>> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >>>>> >>>>>> So, if I understand correctly, it''s really strange you can''t >>>>>> reproduce the panic now. >>>>>> >>>>>> With only the first printk patch applied (and don''t comment the >>>>>> clear_fault_bits()), can you try a cold rebooting the host (plug the power >>>>>> cord off from the host; wait several minutes; plug the cord in and tunn on >>>>>> the host)? And the log? >>>>>> >>>>>> BTW, in your original mail, the log shows: >>>>>> (XEN) Intel VT-d Queued Invalidation supported. >>>>>> (XEN) Intel VT-d Interrupt Remapping supported. >>>>>> >>>>>> But in your this new log, it shows: >>>>>> (XEN) Intel VT-d Queued Invalidation* not* supported. >>>>>> (XEN) Intel VT-d Interrupt Remapping* not* supported. >>>>>> Are you sure you have been operating on the same host? :-) >>>>>> >>>>>> -- Dexuan >>>>>> >>>>>> ------------------------------ >>>>>> *From:* Tom Rotenberg [mailto:tom.rotenberg@gmail.com] >>>>>> *Sent:* 2009年6月29日 17:01 >>>>>> *To:* Cui, Dexuan >>>>>> *Cc:* xen-devel@lists.xensource.com >>>>>> *Subject:* Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell >>>>>> e6400, with iommu=1 >>>>>> >>>>>> Hi, >>>>>> >>>>>> Well, just tried what you suggested, and here are the results: >>>>>> * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with >>>>>> iommu=1 >>>>>> The strange thing, is that after it booted one time ok (with the >>>>>> second patch), then it continued to boot ok, even if i reverted the second >>>>>> patch. >>>>>> >>>>>> The relevant lines in the log are: >>>>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >>>>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >>>>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >>>>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >>>>>> >>>>>> Anyway, here is the full Xen log, with the first patch (print cap''s) >>>>>> applied: >>>>>> >>>>>> __ __ _____ _ _ ___ >>>>>> \ \/ /___ _ __ |___ /| || | / _ \ >>>>>> \ // _ \ ''_ \ |_ \| || |_| | | | >>>>>> / \ __/ | | | ___) |__ _| |_| | >>>>>> /_/\_\___|_| |_| |____(_) |_|(_)___/ >>>>>> >>>>>> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian >>>>>> 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 >>>>>> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 >>>>>> 19629:b2128a88e3dd >>>>>> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 >>>>>> iommu_inclusive_mapping loglvl=all guest_loglvl=all >>>>>> (XEN) Video information: >>>>>> (XEN) VGA is text mode 80x25, font 8x16 >>>>>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >>>>>> (XEN) Disc information: >>>>>> (XEN) Found 1 MBR signatures >>>>>> (XEN) Found 1 EDD information structures >>>>>> (XEN) Xen-e820 RAM map: >>>>>> (XEN) 0000000000000000 - 000000000009f000 (usable) >>>>>> (XEN) 000000000009f000 - 00000000000a0000 (reserved) >>>>>> (XEN) 0000000000100000 - 000000007c04d400 (usable) >>>>>> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) >>>>>> (XEN) 000000007c04f400 - 000000007f000000 (reserved) >>>>>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >>>>>> (XEN) 00000000fec00000 - 00000000fec10000 (reserved) >>>>>> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) >>>>>> (XEN) 00000000fed20000 - 00000000fed90000 (reserved) >>>>>> (XEN) 00000000feda0000 - 00000000feda6000 (reserved) >>>>>> (XEN) 00000000fee00000 - 00000000fee10000 (reserved) >>>>>> (XEN) 00000000ffe60000 - 0000000100000000 (reserved) >>>>>> (XEN) System RAM: 1983MB (2031536kB) >>>>>> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) >>>>>> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D >>>>>> ASL 61) >>>>>> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D >>>>>> ASL 61) >>>>>> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL >>>>>> 20050624) >>>>>> (XEN) ACPI: FACS 7C060C00, 0040 >>>>>> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 >>>>>> ASL 61) >>>>>> (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D >>>>>> ASL 61) >>>>>> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D >>>>>> ASL 47) >>>>>> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D >>>>>> ASL 61) >>>>>> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D >>>>>> ASL 61) >>>>>> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 >>>>>> ASL 0) >>>>>> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D >>>>>> ASL 61) >>>>>> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL >>>>>> 20050624) >>>>>> (XEN) NUMA turned off >>>>>> (XEN) Faking a node at 0000000000000000-000000007c04d000 >>>>>> (XEN) Domain heap initialised >>>>>> (XEN) DMI 2.4 present. >>>>>> (XEN) Using APIC driver default >>>>>> (XEN) ACPI: PM-Timer IO Port: 0x1008 >>>>>> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] >>>>>> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] >>>>>> (XEN) ACPI: Local APIC address 0xfee00000 >>>>>> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) >>>>>> (XEN) Processor #0 7:7 APIC version 20 >>>>>> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) >>>>>> (XEN) Processor #1 7:7 APIC version 20 >>>>>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) >>>>>> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) >>>>>> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) >>>>>> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 >>>>>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) >>>>>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) >>>>>> (XEN) ACPI: IRQ0 used by override. >>>>>> (XEN) ACPI: IRQ2 used by override. >>>>>> (XEN) ACPI: IRQ9 used by override. >>>>>> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs >>>>>> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 >>>>>> (XEN) [VT-D]dmar.c:485: Host address width 36 >>>>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 >>>>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>>>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 >>>>>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD >>>>>> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 >>>>>> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL >>>>>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 >>>>>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 >>>>>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 >>>>>> (XEN) Intel VT-d DMAR tables have been parsed. >>>>>> (XEN) Using ACPI (MADT) for SMP configuration information >>>>>> (XEN) Using scheduler: SMP Credit Scheduler (credit) >>>>>> (XEN) Initializing CPU#0 >>>>>> (XEN) Detected 2527.068 MHz processor. >>>>>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>>>>> (XEN) CPU: L2 cache: 6144K >>>>>> (XEN) CPU: Physical Processor ID: 0 >>>>>> (XEN) CPU: Processor Core ID: 0 >>>>>> (XEN) VMX: Supported advanced features: >>>>>> (XEN) - APIC MMIO access virtualisation >>>>>> (XEN) - APIC TPR shadow >>>>>> (XEN) - Virtual NMI >>>>>> (XEN) - MSR direct-access bitmap >>>>>> (XEN) HVM: VMX enabled >>>>>> (XEN) Intel machine check reporting enabled on CPU#0. >>>>>> (XEN) mce_init: init bank1 >>>>>> (XEN) mce_init: init bank2 >>>>>> (XEN) mce_init: init bank3 >>>>>> (XEN) mce_init: init bank4 >>>>>> (XEN) mce_init: init bank5 >>>>>> (XEN) CPU0: Thermal monitoring enabled (TM2) >>>>>> (XEN) CMCI: CPU0 has no CMCI support >>>>>> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping >>>>>> 06 >>>>>> (XEN) Booting processor 1/1 eip 8c000 >>>>>> (XEN) Initializing CPU#1 >>>>>> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K >>>>>> (XEN) CPU: L2 cache: 6144K >>>>>> (XEN) CPU: Physical Processor ID: 0 >>>>>> (XEN) CPU: Processor Core ID: 1 >>>>>> (XEN) Intel machine check reporting enabled on CPU#1. >>>>>> (XEN) mce_init: init bank1 >>>>>> (XEN) mce_init: init bank2 >>>>>> (XEN) mce_init: init bank4 >>>>>> (XEN) mce_init: init bank5 >>>>>> (XEN) CPU1: Thermal monitoring enabled (TM2) >>>>>> (XEN) CMCI: CPU1 has no CMCI support >>>>>> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping >>>>>> 06 >>>>>> (XEN) Total of 2 processors activated. >>>>>> (XEN) ENABLING IO-APIC IRQs >>>>>> (XEN) -> Using new ACK method >>>>>> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 >>>>>> (XEN) checking TSC synchronization across 2 CPUs: passed. >>>>>> (XEN) Platform timer is 14.318MHz HPET >>>>>> (XEN) Brought up 2 CPUs >>>>>> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >>>>>> (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush >>>>>> (XEN) Intel VT-d Snoop Control not supported. >>>>>> (XEN) Intel VT-d DMA Passthrough not supported. >>>>>> (XEN) Intel VT-d Queued Invalidation not supported. >>>>>> (XEN) Intel VT-d Interrupt Remapping not supported. >>>>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 >>>>>> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 >>>>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 >>>>>> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 >>>>>> (XEN) I/O virtualisation enabled >>>>>> (XEN) I/O virtualisation for PV guests disabled >>>>>> (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast >>>>>> (XEN) ACPI sleep modes: S3 >>>>>> (XEN) mcheck_poll: Machine check polling timer started. >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:0.0 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:2.0 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:2.1 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:3.0 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:3.2 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:3.3 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:19.0 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1a.0 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1a.1 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1a.2 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1a.7 >>>>>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf >>>>>> 0:1b.0 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1d.0 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1d.1 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1d.2 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1d.7 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1f.0 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1f.2 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1f.3 >>>>>> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf >>>>>> c:0.0 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:2.0 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:2.1 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1d.0 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1d.1 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1d.2 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1d.7 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1a.0 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1a.1 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1a.2 >>>>>> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf >>>>>> 0:1a.7 >>>>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>>>> ffff828bfff57000 >>>>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>>>> ffff828bfff56000 >>>>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>>>> ffff828bfff55000 >>>>>> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg >>>>>> ffff828bfff54000 >>>>>> (XEN) *** LOADING DOMAIN 0 *** >>>>>> (XEN) Xen kernel: 64-bit, lsb, compat32 >>>>>> (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> >>>>>> 0xffffffff8070ab48 >>>>>> (XEN) PHYSICAL MEMORY ARRANGEMENT: >>>>>> (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages >>>>>> to be allocated) >>>>>> (XEN) VIRTUAL MEMORY ARRANGEMENT: >>>>>> (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 >>>>>> (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 >>>>>> (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 >>>>>> (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 >>>>>> (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 >>>>>> (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 >>>>>> (XEN) TOTAL: ffffffff80000000->ffffffff82000000 >>>>>> (XEN) ENTRY ADDRESS: ffffffff80200000 >>>>>> (XEN) Dom0 has maximum 2 VCPUs >>>>>> (XEN) Scrubbing Free RAM: .done. >>>>>> (XEN) Xen trace buffers: disabled >>>>>> (XEN) Std. Loglevel: All >>>>>> (XEN) Guest Loglevel: All >>>>>> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch >>>>>> input to Xen) >>>>>> (XEN) Freed 132kB init memory. >>>>>> >>>>>> Tom >>>>>> >>>>>> 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com> >>>>>> >>>>>>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c >>>>>>> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 >>>>>>> +0100 >>>>>>> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 >>>>>>> +0800 >>>>>>> @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) >>>>>>> } >>>>>>> dma_msi_data_init(iommu, iommu->vector); >>>>>>> dma_msi_addr_init(iommu, >>>>>>> cpu_physical_id(first_cpu(cpu_online_map))); >>>>>>> - clear_fault_bits(iommu); >>>>>>> + //clear_fault_bits(iommu); >>>>>>> dmar_writel(iommu->reg, DMAR_FECTL_REG, 0); >>>>>>> >>>>>>> /* initialize flush functions */ >>>>>>> >>>>>>> And, can you comment the invocation of clear_fault_bits() in >>>>>>> init_vtd_hw() like above and see if the host can boot fine? >>>>>>> >>>>>>> Thanks, >>>>>>> -- Dexuan >>>>>>> >>>>>>> >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Cui, Dexuan >>>>>>> Sent: 2009?6?29? 14:30 >>>>>>> To: ''Tom Rotenberg''; xen-devel@lists.xensource.com >>>>>>> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell >>>>>>> e6400, with iommu=1 >>>>>>> >>>>>>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c >>>>>>> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 >>>>>>> +0100 >>>>>>> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 >>>>>>> +0800 >>>>>>> @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm >>>>>>> { >>>>>>> u64 val; >>>>>>> >>>>>>> + printk("clear_fault_bits: CAP_REG=0x%llx\n", >>>>>>> + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); >>>>>>> val = dmar_readq( >>>>>>> iommu->reg, >>>>>>> >>>>>>> cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8); >>>>>>> >>>>>>> Hi Tom, >>>>>>> Can you please apply the above patch and send us the entire log? >>>>>>> (I don''t have the same host, so I can''t try it myself.) >>>>>>> >>>>>>> Thanks, >>>>>>> -- Dexuan >>>>>>> >>>>>>> ________________________________ >>>>>>> >>>>>>> From: xen-devel-bounces@lists.xensource.com [mailto: >>>>>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Tom Rotenberg >>>>>>> Sent: 2009?6?28? 22:56 >>>>>>> To: xen-devel@lists.xensource.com >>>>>>> Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, >>>>>>> with iommu=1 >>>>>>> >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try >>>>>>> to boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. >>>>>>> I have also tried to boot with and without the ''iommu_inclusive_mapping'' >>>>>>> boot option, but it didn''t seem to help at all. >>>>>>> >>>>>>> Does anyone knows how to solve this issue? >>>>>>> >>>>>>> These are the last messages in the UART: >>>>>>> >>>>>>> (XEN) Brought up 2 CPUs >>>>>>> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed >>>>>>> (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush >>>>>>> (XEN) Intel VT-d Snoop Control supported. >>>>>>> (XEN) Intel VT-d DMA Passthrough not supported. >>>>>>> (XEN) Intel VT-d Queued Invalidation supported. >>>>>>> (XEN) Intel VT-d Interrupt Remapping supported. >>>>>>> (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- >>>>>>> (XEN) CPU: 0 >>>>>>> (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >>>>>>> (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor >>>>>>> (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: >>>>>>> 0000000000000020 >>>>>>> (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: >>>>>>> ffff830078f82f90 >>>>>>> (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: >>>>>>> ffff83007be2c110 >>>>>>> (XEN) r9: 000000000000001a r10: 0000000000000005 r11: >>>>>>> 0000000000000005 >>>>>>> (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: >>>>>>> ffff830078f82fcc >>>>>>> (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: >>>>>>> 00000000000026f0 >>>>>>> (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 >>>>>>> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 >>>>>>> (XEN) Xen stack trace from rsp=ffff828c8026fda0: >>>>>>> (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f >>>>>>> ffff828c80204490 >>>>>>> (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 >>>>>>> 0000000000000000 >>>>>>> (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 >>>>>>> 000000000000001f >>>>>>> (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 >>>>>>> ffff828c80205f60 >>>>>>> (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 >>>>>>> ffff828c80227978 >>>>>>> (XEN) ffff828c8021a589 0000000000000000 0000000000000000 >>>>>>> 0000000000000000 >>>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>>> 0000000000000000 >>>>>>> (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 >>>>>>> 000000000008bf40 >>>>>>> (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 >>>>>>> 0000000000000000 >>>>>>> (XEN) ffffffff00000000 0000000800000000 000000010000006e >>>>>>> 0000000000000003 >>>>>>> (XEN) 00000000000002f8 0000000000000000 0000000000000000 >>>>>>> 0000000000067e2c >>>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>>> 0000000000000000 >>>>>>> (XEN) ffff828c801000b5 0000000000000000 0000000000000000 >>>>>>> 0000000000000000 >>>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>>> 0000000000000000 >>>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>>> 0000000000000000 >>>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>>> 0000000000000000 >>>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>>> 0000000000000000 >>>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>>> 0000000000000000 >>>>>>> (XEN) 0000000000000000 0000000000000000 0000000000000000 >>>>>>> ffff830079f18000 >>>>>>> (XEN) Xen call trace: >>>>>>> (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 >>>>>>> (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 >>>>>>> (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 >>>>>>> (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 >>>>>>> (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 >>>>>>> (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 >>>>>>> (XEN) >>>>>>> (XEN) Pagetable walk from ffff828bfff5aff8: >>>>>>> (XEN) L4[0x105] = 000000007be7a027 5555555555555555 >>>>>>> (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 >>>>>>> (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 >>>>>>> (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff >>>>>>> (XEN) >>>>>>> (XEN) **************************************** >>>>>>> (XEN) Panic on CPU 0: >>>>>>> (XEN) FATAL PAGE FAULT >>>>>>> (XEN) [error_code=0000] >>>>>>> (XEN) Faulting linear address: ffff828bfff5aff8 >>>>>>> (XEN) **************************************** >>>>>>> (XEN) >>>>>>> (XEN) Reboot in five seconds... >>>>>>> >>>>>>> >>>>>>>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Cui, Dexuan
2009-Jun-30 07:47 UTC
RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1
Sure, I''ll keep you updated once I get any news. :-) Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com] Sent: 2009年6月30日 15:41 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Dell E6400, BIOS version: A12 I hope this issue will be solved by the BIOS people, since i think i saw the same issue with my Lenovo T400, and it looks like you *must* do a cold boot, on both Lenovo & Dell. Please update me on this issue, if you''ll have any news. Thanks for your help! 2009/6/30 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> Tim, Can you tell me the BIOS version of your Dell E6400 host? BTW, please try to use the latest BIOS if possible, and please check your BIOS document to see if there is any known IOMMU-related issue. Thanks, -- Dexuan ________________________________ From: Cui, Dexuan Sent: 2009年6月30日 10:55 To: ''Tom Rotenberg'' Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1> CAP_REG=0xffffffffffffffffHW obviously doesn''t report the value correctly. So, looks the issue is: when we change the iommu option in BIOS (e.g., from disabled to enabled), to make it really take effect, we have to make a "cold shutdown (shutdown the power supply throughly)" to the host ; or else, though Xen can see the ACPI DMAR tables，the HW doesn''t respond to the MMIO accesses to the iommu registers at all -- such read access would return -1 (all-''1''s) and write access would be ignored. I remember you said after you commented the clear_fault_bits(), the host could boot fine with iommu=1 --- I guess in this case actually the DMA remapping is not enabled at all though Xen thinks HW tells it DMA remapping is enabled (Xen writes DMA_GCMD_TE into GCMD (this write is ignored) and polls the GSTS; since reading GSTS returns -1, Xen thinks TE is on, however the underlying HW doesn''t enable DMA remapping!). Please note: this is only my deduction, anyway. :-) You can try to create some hvm guests with devices assigned, I think the host would die weirdly since the DMAs of the assigned devices of the hvm guests are corrupting the host memory! This can be a way to verify my deduction. And I remember you said after you commented the clear_fault_bits() and booted the host fine, you tried to uncomment the line and the host still boots fine! -- this is strange to me, I guess you might have done a cold shutdown before your uncommenting the line? One more thing: I think your HW doesn''t support SnoopControl/QueuedInvalidation/InterruptRemapping, but after you commented the clear_fault_bits(), LOOKS the host could boot fine even after Xen tries to enable them! Maybe the specific HW implementation of your host could account for this... I think changing the iommu option in BIOS should not require a cold shutdown to the host. I''ll try to raise this issue to the BIOS people. For now, you could work around the issue by remember doing a cold shutdown every time you change the iommu BIOS option. Hope this is acceptable to you for now. :-) Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 22:15 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 in steps 1 & 4, i mean disabling and enabling IOMMU in the BIOS (and not in Xen boot parameter). And, this is what probably caused all of this. I am using my own version (above Xen 3.4.0), but i haven''t made any changes in the hyervisor itself (but rather in the hvmloader and rombios), so it''s not related to this. I prefer not to try the latest Xen, unless there is a very good reason for this, as it take long time to compile, and i can''t do it right now. I have reproduced the panic with the print, and here is the relevant part of the log: (XEN) clear_fault_bits: CAP_REG=0xffffffffffffffff (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000000 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff828c801f0564 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fd98 r8: 0000000000004000 (XEN) r9: 0000000000003fff r10: ffff828c80288360 r11: 0000000080287ea0 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fd98: (XEN) ffff830078f82f90 ffff828c8012d35d ffff830078f824b0 ffff828c801e4257 (XEN) ffff828c80204490 ffff828c801e4257 ffff828c802044c0 ffff828c8026fdf8 (XEN) 0000000000000000 ffff828c8012deae ffff830078f82d40 ffff828c80228100 (XEN) 000000000000001f 00000000ffffffed ffff828c80227a78 0000000000000001 (XEN) ffff828c80205f60 ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 (XEN) ffff828c80227978 ffff828c8021a589 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 00000000002b23c0 ffffff8c80274000 0008bf3000000000 (XEN) 000000000008bf30 0000000000000000 ffff83000008bfc0 0000000001ce08a0 (XEN) 0000000000000000 ffffffff00000000 0000000800000000 000000010000006e (XEN) 0000000000000003 00000000000002f8 0000000000000000 0000000000000000 (XEN) 0000000000067e9c 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffff828c801000b5 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0 (XEN) [<ffff828c8012d35d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012deae>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> When you meet with the panic, can you apply the printk log to print CAP_REG? In your step 1 and 4, you just mean using the xen parameter iommu=0 and iommu=1, correct? During theses steps, you don''t change any BIOS setting and there is not any HW change, correct? I see you''re using "3.4.0 19629:b2128a88e3dd", but 19629''s checksum should be 69293fc2096f (http://xenbits.xensource.com/xen-3.4-testing.hg?rev/69293fc2096f). So I assume you made your own changes to the code? Can you try the latest changeset 19677: b2ddeb734ca7(http://xenbits.xensource.com/xen-3.4-testing.hg)? Sorry, I don''t have the same type of host, so I can''t try it myself. :-( Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 21:09 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Well, it looks like my analysis was right. This is how i reproduced it: 1. Disable the IOMMU 2. Turn off the laptop, and leave it turned off for a few seconds 3. Turn on the laptop 4. Enable the IOMMU (*DON''T* turn off the machine) 5. Continue 6. Xen crashes and, now the log lines are: (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. It looks like there is a problem when enabling IOMMU, without cold boot of the machine. What do u say? Here is the full log: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:40:11 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 00B0 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.037 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fda0: (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 (XEN) ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 ffff828c80227978 (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 00000000002b23c0 ffffff8c80274000 0008bf3000000000 000000000008bf30 (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... 2009/6/29 Tom Rotenberg <tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>> I''m almost 100% sure, that this was from the same machine (since i have only 1 such Dell currently on my desk :). However, this log was generated just after turning on the IOMMU (without cold boot). So, let me check if it happened because of this. I''ll check it now... 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> I have no idea about it.> (XEN) Intel VT-d Snoop Control supported. > (XEN) Intel VT-d DMA Passthrough not supported. > (XEN) Intel VT-d Queued Invalidation supported. > (XEN) Intel VT-d Interrupt Remapping supported.This is from the log of your original mail. We can see SC/QI/IR are supported, but in your later log, we can see they are not supported. I''m curious where the first log comes from. :-) -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 20:38 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 1) Yep, i can''t reproduce it anymore. Maybe i''ll try it with another Dell R6400 i have here. 2) On my Lenovo T400, i don''t remember the exact crash, and i don''t have the log of that crash, i just remember something about "DMAR ... failure", which happened, after i turned on the IOMMU, and didn''t perform a cold reboot on that machine. Do u have any idea why these problems happen? Does anyone else working with Dell E6400 had this problem? Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> It''s really weird... :-) If I understand correctly, with iommu=1 in Xen: 1) on your "Dell E6400 machine", now you can''t reproduce the panic any longer; 2) on your "Lenovo T400", you can reproduce the panic sometimes -- if so, can you post the log (with the printk patch applied) when you meet with the panic? Can you find out under what condition it''s easy to reproduce the panic? Thanks, -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 20:15 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, Tried again, this time with cold reboot, it worked again. I also checked the lines you were talking about, and now they are: (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. Could it be that there is some kind of problem with the hardware, which makes it sometimes report invalid values? (with the Lenovo T400 i''m testing, if you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on boot, so you must do a cold reboot). Here is the log: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.058 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 (XEN) TOTAL: ffffffff80000000->ffffffff82000000 (XEN) ENTRY ADDRESS: ffffffff80200000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 132kB init memory. Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> So, if I understand correctly, it''s really strange you can''t reproduce the panic now. With only the first printk patch applied (and don''t comment the clear_fault_bits()), can you try a cold rebooting the host (plug the power cord off from the host; wait several minutes; plug the cord in and tunn on the host)? And the log? BTW, in your original mail, the log shows: (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. But in your this new log, it shows: (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. Are you sure you have been operating on the same host? :-) -- Dexuan ________________________________ From: Tom Rotenberg [mailto:tom.rotenberg@gmail.com<mailto:tom.rotenberg@gmail.com>] Sent: 2009年6月29日 17:01 To: Cui, Dexuan Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, Well, just tried what you suggested, and here are the results: * If i comment the ''clear_fault_bits()'' call - then Xen boots ok, with iommu=1 The strange thing, is that after it booted one time ok (with the second patch), then it continued to boot ok, even if i reverted the second patch. The relevant lines in the log are: (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 Anyway, here is the full Xen log, with the first patch (print cap''s) applied: __ __ _____ _ _ ___ \ \/ /___ _ __ |___ /| || | / _ \ \ // _ \ ''_ \ |_ \| || |_| | | | / \ __/ | | | ___) |__ _| |_| | /_/\_\___|_| |_| |____(_) |_|(_)___/ (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009 (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd (XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f000 (usable) (XEN) 000000000009f000 - 00000000000a0000 (reserved) (XEN) 0000000000100000 - 000000007c04d400 (usable) (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS) (XEN) 000000007c04f400 - 000000007f000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec10000 (reserved) (XEN) 00000000fed18000 - 00000000fed1c000 (reserved) (XEN) 00000000fed20000 - 00000000fed90000 (reserved) (XEN) 00000000feda0000 - 00000000feda6000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffe60000 - 0000000100000000 (reserved) (XEN) System RAM: 1983MB (2031536kB) (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61) (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624) (XEN) ACPI: FACS 7C060C00, 0040 (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61) (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47) (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61) (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61) (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0) (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61) (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000007c04d000 (XEN) Domain heap initialised (XEN) DMI 2.4 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x1008 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0] (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:485: Host address width 36 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000 (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1 (XEN) Intel VT-d DMAR tables have been parsed. (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2527.068 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) Intel machine check reporting enabled on CPU#0. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank3 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) Intel machine check reporting enabled on CPU#1. (XEN) mce_init: init bank1 (XEN) mce_init: init bank2 (XEN) mce_init: init bank4 (XEN) mce_init: init bank5 (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation not supported. (XEN) Intel VT-d Interrupt Remapping not supported. (XEN) clear_fault_bits: CAP_REG=0xc0000020630270 (XEN) clear_fault_bits: CAP_REG=0xc0000020630260 (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260 (XEN) clear_fault_bits: CAP_REG=0xc9008020630260 (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2 (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000 (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48 (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400 (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18 (XEN) Start info: ffffffff81f28000->ffffffff81f284b4 (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000 (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000 (XEN) TOTAL: ffffffff80000000->ffffffff82000000 (XEN) ENTRY ADDRESS: ffffffff80200000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 132kB init memory. Tom 2009/6/29 Cui, Dexuan <dexuan.cui@intel.com<mailto:dexuan.cui@intel.com>> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800 @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void) } dma_msi_data_init(iommu, iommu->vector); dma_msi_addr_init(iommu, cpu_physical_id(first_cpu(cpu_online_map))); - clear_fault_bits(iommu); + //clear_fault_bits(iommu); dmar_writel(iommu->reg, DMAR_FECTL_REG, 0); /* initialize flush functions */ And, can you comment the invocation of clear_fault_bits() in init_vtd_hw() like above and see if the host can boot fine? Thanks, -- Dexuan -----Original Message----- From: Cui, Dexuan Sent: 2009?6?29? 14:30 To: ''Tom Rotenberg''; xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100 +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800 @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm { u64 val; + printk("clear_fault_bits: CAP_REG=0x%llx\n", + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG)); val = dmar_readq( iommu->reg, cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8); Hi Tom, Can you please apply the above patch and send us the entire log? (I don''t have the same host, so I can''t try it myself.) Thanks, -- Dexuan ________________________________ From: xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com> [mailto:xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>] On Behalf Of Tom Rotenberg Sent: 2009?6?28? 22:56 To: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1 Hi, I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to boot xen with a ''iommu=1'' parameter in the boot line, Xen fails to boot. I have also tried to boot with and without the ''iommu_inclusive_mapping'' boot option, but it didn''t seem to help at all. Does anyone knows how to solve this issue? These are the last messages in the UART: (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush (XEN) Intel VT-d Snoop Control supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020 (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90 (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110 (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005 (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8026fda0: (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490 (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000 (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60 (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978 (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40 (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000 (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003 (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000 (XEN) Xen call trace: (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0 (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0 (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610 (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110 (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0 (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3 (XEN) (XEN) Pagetable walk from ffff828bfff5aff8: (XEN) L4[0x105] = 000000007be7a027 5555555555555555 (XEN) L3[0x02f] = 000000007c04b063 5555555555555555 (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555 (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: ffff828bfff5aff8 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel