Jan,
I was given a machine to diagnose a boot problem on xen 4.3, that used
to work on 4.2, and have bisected to the following changeset:
commit d0d4635d034f202bb401a6efa3ba61530f3854ab
Author: Jan Beulich <jbeulich@suse.com>
Date: Thu Nov 22 10:47:58 2012 +0100
implement vmap()
... and use it as basis for a proper ioremap() on x86.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
Full diff:
http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=d0d4635d034f202bb401a6efa3ba61530f3854ab
While this is pre-release hardware from an OEM, it did work on 4.2, so
I am not (yet) inclined to blame it on being pre-release hardware.
This is very early in boot, and so does not have a proper stack.
Do you have any thoughts as to where this might be going awry?
The Xen boot log looks like the following in the bad case:
\ \/ /___ _ __ | || | |___ / _ _ _ __ ___| |_ __ _| |__ | | ___
\ // _ \ ''_ \ | || |_ |_ \ __| | | | ''_ \/ __| __/ _` |
''_ \| |/ _ \
/ \ __/ | | | |__ _| ___) |__| |_| | | | \__ \ || (_| | |_) | | __/
/_/\_\___|_| |_| |_|(_)____/ \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
(XEN) Xen version 4.3-unstable (@oldroadcomputing.net) (gcc
(Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3) debug=y Mon Jul 8 15:09:16 EDT
2013
(XEN) Latest ChangeSet: Mon Jul 8 15:08:40 2013 -0400 git:6a62f63
(XEN) Console output is synchronous.
(XEN) Bootloader: COM32 Multiboot loader v0.2
(XEN) Command line: dom0_mem=max:2048M dbgp=ehci@pci00:1d.0
console=dbgp loglvl=all sync_console
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 0000000000091800 (usable)
(XEN) 0000000000091800 - 00000000000a0000 (reserved)
(XEN) 00000000000e0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000aad0b000 (usable)
(XEN) 00000000aad0b000 - 00000000aad12000 (ACPI NVS)
(XEN) 00000000aad12000 - 00000000ab917000 (usable)
(XEN) 00000000ab917000 - 00000000abdd4000 (reserved)
(XEN) 00000000abdd4000 - 00000000b5db4000 (usable)
(XEN) 00000000b5db4000 - 00000000b6000000 (reserved)
(XEN) 00000000b6000000 - 00000000b675c000 (usable)
(XEN) 00000000b675c000 - 00000000b6800000 (reserved)
(XEN) 00000000b6800000 - 00000000b6fb0000 (usable)
(XEN) 00000000b6fb0000 - 00000000b7000000 (ACPI data)
(XEN) 00000000b7000000 - 00000000b871c000 (usable)
(XEN) 00000000b871c000 - 00000000b8800000 (ACPI NVS)
(XEN) 00000000b8800000 - 00000000b9765000 (usable)
(XEN) 00000000b9765000 - 00000000ba000000 (reserved)
(XEN) 00000000bd000000 - 00000000bf200000 (reserved)
(XEN) 00000000f8000000 - 00000000fc000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec01000 (reserved)
(XEN) 00000000fed00000 - 00000000fed04000 (reserved)
(XEN) 00000000fed1c000 - 00000000fed20000 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000ff000000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 000000023fe00000 (usable)
(XEN) ACPI: RSDP 000EEE50, 0024 (r2 DELL )
(XEN) ACPI: XSDT B6FF1088, 0094 (r1 DELL CBX3 1072009 AMI 10013)
(XEN) ACPI: FACP B6FFDDE8, 010C (r5 DELL CBX3 1072009 AMI 10013)
(XEN) ACPI: DSDT B6FF11B0, CC33 (r2 DELL CBX3 14 INTL 20091112)
(XEN) ACPI: FACS B87FE080, 0040
(XEN) ACPI: APIC B6FFDEF8, 0072 (r3 DELL CBX3 1072009 AMI 10013)
(XEN) ACPI: FPDT B6FFDF70, 0044 (r1 DELL CBX3 1072009 AMI 10013)
(XEN) ACPI: ASF! B6FFDFB8, 00A5 (r32 INTEL HCG 1 TFSM F4240)
(XEN) ACPI: SSDT B6FFE060, 01F8 (r1 INTEL sensrhub 0 INTL 20120711)
(XEN) ACPI: SSDT B6FFE258, 01BE (r1 Intel zpoddult 1000 INTL 20091112)
(XEN) ACPI: SSDT B6FFE418, 018E (r1 AcpiRe Ult0Rtd3 1000 INTL 20051117)
(XEN) ACPI: SSDT B6FFE5A8, 0548 (r1 PmRef Cpu0Ist 3000 INTL 20051117)
(XEN) ACPI: SSDT B6FFEAF0, 0AD8 (r1 PmRef CpuPm 3000 INTL 20051117)
(XEN) ACPI: HPET B6FFF5C8, 0038 (r1 DELL CBX3 1072009 AMI. 5)
(XEN) ACPI: SSDT B6FFF600, 03BD (r1 SataRe SataTabl 1000 INTL 20091112)
(XEN) ACPI: MCFG B6FFF9C0, 003C (r1 DELL CBX3 1072009 MSFT 97)
(XEN) ACPI: CSRT B6FFFA00, 00C4 (r1 INTL HSWULT-R 1 INTL 20100528)
(XEN) ACPI: DMAR B6FFFAC8, 0088 (r1 INTEL HSW 1 INTL 1)
(XEN) System RAM: 8076MB (8269880kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000023fe00000
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000fd990
(XEN) DMI 2.7 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1808
(XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0]
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1804,0], pm1x_evt[1800,0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
b87fe080/0000000000000000, using 32
(XEN) ACPI: wakeup_vec[b87fe08c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 6:5 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) Processor #2 6:5 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)
(XEN) Processor #1 6:5 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
(XEN) Processor #3 6:5 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-39
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) [VT-D]iommu.c:1168: IOMMU: unsupported
(XEN) ---- print_iommu_regs ----
(XEN) drhd->address = 0
(XEN) VER = f0004800
(XEN) CAP = f000eef3f000e2c3
(XEN) n_fault_reg = ef
(XEN) fault_recording_offset = 3f00
(XEN) ECAP = f000ff54f000eef5
(XEN) GCMD = f0003182
(XEN) GSTS = f000312a
(XEN) RTADDR = e8000304f000fea5
(XEN) CCMD = f000eef3f000eef3
(XEN) FSTS = f000eef3
(XEN) FECTL = f000ef57
(XEN) FEDATA = f000ff53
(XEN) FEADDR = c0000014
(XEN) FEUADDR = f000f84d
(XEN) Unknown interrupt (cr2=0000000000000000)
(XEN) ffff82c3ffd77ac8 ffff82c3ffd77ac8 ffff83023c4e5530
ffff82c3ffd54000 ffff82c4c02d7c28 0000000000000000 0000000000000a0d
0000000000000096 ffff82c3ffffc0a0 0000000050010412 0000000000000000
ffff82c008000000 ffff82c3ffd54000 0000000000000030 ffff82c3ffd54000
ffff82c4c012d525 000000000000e008 0000000000010086 ffff82c4c02d7c18
0000000000000000 ffff82c4c012d51a ffff82c3ffd54000 0000000000000000
ffff82c4c02d7c48 ffff82c4c012d5f9 ffff83023c4e55c0 00000000ffffffed
ffff82c4c02d7c68 ffff82c4c029978f ffff82c3ffd77af8 ffff82c3ffd77af8
ffff82c4c02d7cf8 ffff82c4c029a63a ffff82c3ffd77000 ffff8300b97641e0
ffff82c4c02d7d10 ffff82c4c029a251 ffff82c4c02d7cb8 ffff82c4c029ea42
ffff82c4c02d7cb8 ffff8300b97641e0 ffff82c4c02d7cd8 ffff82c4c029f2ff
0000000000000000 ffff82c4c029a251 ffff82c4c024dbd5 ffff82c4c029a251
0000000000000000 0000000000002515 ffff82c4c02d7d28 ffff82c4c02a0607
ffff82c3ffd78000 ffff82c3ffd77ac8 0000000000000000 0000000000000000
ffff82c4c02d7df8 ffff82c4c02acd1f ffff82c4c02d7d68 ffff82c4c0225f56
ffff82c4c02d7d58 0000000000000000 0000000000000163 000ffffffffff000
ffff82c4c02d7df8 ffff82c4c0174cdc ffff82c4c02d7e08 00000000000001e3
00000000000b6dff 0000000000000163 00000000b6dff000 00000000000001e3
0000000000000000 00000000000b7000 ffff8300b7000ac8 ffff82c4c02d7e10
ffff82c4c02d7df8 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 ffff82c4c02d7e18 ffff82c4c029af1f
0000000000000088 00000000b6fffac8 ffff82c4c02d7e48 ffff82c4c02ae4b7
ffff82c4c02d7e48 ffff830000080f20 000000000023fdff ffff830000080fb0
ffff82c4c02d7f08 ffff82c4c02aa6de 0000000000000000 0000000000000000
0000000000100000 ffff830000080be0 ffff830000080f20 0000000000000000
00000000aac00000 ffff830000080f01 ffff830000080fb0 0000000000fa4000
000000000000001c ffff82c4c02c3c34 0000000800000000 000000010000006e
0000000000000003 00000000000002f8 0000000000000000 0000000000000000
000000000000180a 00000000a8ef4988 0000000000000001 00000000afbd9018
0000000000000000 ffff82c4c01000b5 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
00000000fffff000 0000000000000000 0000000000000000
>>> On 09.07.13 at 14:55, Ben Guthro <ben@guthro.net> wrote: > Jan, > > I was given a machine to diagnose a boot problem on xen 4.3, that used > to work on 4.2, and have bisected to the following changeset: > > commit d0d4635d034f202bb401a6efa3ba61530f3854ab > Author: Jan Beulich <jbeulich@suse.com> > Date: Thu Nov 22 10:47:58 2012 +0100 > > implement vmap() > > ... and use it as basis for a proper ioremap() on x86. > > Signed-off-by: Jan Beulich <jbeulich@suse.com> > Acked-by: Keir Fraser <keir@xen.org> > > > Full diff: > http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=d0d4635d034f202bb401a6ef > a3ba61530f3854ab > > > While this is pre-release hardware from an OEM, it did work on 4.2, so > I am not (yet) inclined to blame it on being pre-release hardware. > > This is very early in boot, and so does not have a proper stack. > Do you have any thoughts as to where this might be going awry?If you could make the xen-syms available matching the xen.gz that was used when the below log was obtained, I could try to take a look. Perhaps repeating the attempt with "iommu=debug" could reveal further useful details. Jan> The Xen boot log looks like the following in the bad case: > > \ \/ /___ _ __ | || | |___ / _ _ _ __ ___| |_ __ _| |__ | | ___ > \ // _ \ ''_ \ | || |_ |_ \ __| | | | ''_ \/ __| __/ _` | ''_ \| |/ _ \ > / \ __/ | | | |__ _| ___) |__| |_| | | | \__ \ || (_| | |_) | | __/ > /_/\_\___|_| |_| |_|(_)____/ \__,_|_| |_|___/\__\__,_|_.__/|_|\___| > > (XEN) Xen version 4.3-unstable (@oldroadcomputing.net) (gcc > (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3) debug=y Mon Jul 8 15:09:16 EDT > 2013 > (XEN) Latest ChangeSet: Mon Jul 8 15:08:40 2013 -0400 git:6a62f63 > (XEN) Console output is synchronous. > (XEN) Bootloader: COM32 Multiboot loader v0.2 > (XEN) Command line: dom0_mem=max:2048M dbgp=ehci@pci00:1d.0 > console=dbgp loglvl=all sync_console > (XEN) Video information: > (XEN) VGA is text mode 80x25, font 8x16 > (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds > (XEN) Disc information: > (XEN) Found 1 MBR signatures > (XEN) Found 1 EDD information structures > (XEN) Xen-e820 RAM map: > (XEN) 0000000000000000 - 0000000000091800 (usable) > (XEN) 0000000000091800 - 00000000000a0000 (reserved) > (XEN) 00000000000e0000 - 0000000000100000 (reserved) > (XEN) 0000000000100000 - 00000000aad0b000 (usable) > (XEN) 00000000aad0b000 - 00000000aad12000 (ACPI NVS) > (XEN) 00000000aad12000 - 00000000ab917000 (usable) > (XEN) 00000000ab917000 - 00000000abdd4000 (reserved) > (XEN) 00000000abdd4000 - 00000000b5db4000 (usable) > (XEN) 00000000b5db4000 - 00000000b6000000 (reserved) > (XEN) 00000000b6000000 - 00000000b675c000 (usable) > (XEN) 00000000b675c000 - 00000000b6800000 (reserved) > (XEN) 00000000b6800000 - 00000000b6fb0000 (usable) > (XEN) 00000000b6fb0000 - 00000000b7000000 (ACPI data) > (XEN) 00000000b7000000 - 00000000b871c000 (usable) > (XEN) 00000000b871c000 - 00000000b8800000 (ACPI NVS) > (XEN) 00000000b8800000 - 00000000b9765000 (usable) > (XEN) 00000000b9765000 - 00000000ba000000 (reserved) > (XEN) 00000000bd000000 - 00000000bf200000 (reserved) > (XEN) 00000000f8000000 - 00000000fc000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec01000 (reserved) > (XEN) 00000000fed00000 - 00000000fed04000 (reserved) > (XEN) 00000000fed1c000 - 00000000fed20000 (reserved) > (XEN) 00000000fee00000 - 00000000fee01000 (reserved) > (XEN) 00000000ff000000 - 0000000100000000 (reserved) > (XEN) 0000000100000000 - 000000023fe00000 (usable) > (XEN) ACPI: RSDP 000EEE50, 0024 (r2 DELL ) > (XEN) ACPI: XSDT B6FF1088, 0094 (r1 DELL CBX3 1072009 AMI 10013) > (XEN) ACPI: FACP B6FFDDE8, 010C (r5 DELL CBX3 1072009 AMI 10013) > (XEN) ACPI: DSDT B6FF11B0, CC33 (r2 DELL CBX3 14 INTL 20091112) > (XEN) ACPI: FACS B87FE080, 0040 > (XEN) ACPI: APIC B6FFDEF8, 0072 (r3 DELL CBX3 1072009 AMI 10013) > (XEN) ACPI: FPDT B6FFDF70, 0044 (r1 DELL CBX3 1072009 AMI 10013) > (XEN) ACPI: ASF! B6FFDFB8, 00A5 (r32 INTEL HCG 1 TFSM F4240) > (XEN) ACPI: SSDT B6FFE060, 01F8 (r1 INTEL sensrhub 0 INTL 20120711) > (XEN) ACPI: SSDT B6FFE258, 01BE (r1 Intel zpoddult 1000 INTL 20091112) > (XEN) ACPI: SSDT B6FFE418, 018E (r1 AcpiRe Ult0Rtd3 1000 INTL 20051117) > (XEN) ACPI: SSDT B6FFE5A8, 0548 (r1 PmRef Cpu0Ist 3000 INTL 20051117) > (XEN) ACPI: SSDT B6FFEAF0, 0AD8 (r1 PmRef CpuPm 3000 INTL 20051117) > (XEN) ACPI: HPET B6FFF5C8, 0038 (r1 DELL CBX3 1072009 AMI. 5) > (XEN) ACPI: SSDT B6FFF600, 03BD (r1 SataRe SataTabl 1000 INTL 20091112) > (XEN) ACPI: MCFG B6FFF9C0, 003C (r1 DELL CBX3 1072009 MSFT 97) > (XEN) ACPI: CSRT B6FFFA00, 00C4 (r1 INTL HSWULT-R 1 INTL 20100528) > (XEN) ACPI: DMAR B6FFFAC8, 0088 (r1 INTEL HSW 1 INTL 1) > (XEN) System RAM: 8076MB (8269880kB) > (XEN) No NUMA configuration found > (XEN) Faking a node at 0000000000000000-000000023fe00000 > (XEN) Domain heap initialised > (XEN) found SMP MP-table at 000fd990 > (XEN) DMI 2.7 present. > (XEN) Using APIC driver default > (XEN) ACPI: PM-Timer IO Port: 0x1808 > (XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0] > (XEN) ACPI: SLEEP INFO: pm1x_cnt[1804,0], pm1x_evt[1800,0] > (XEN) ACPI: 32/64X FACS address mismatch in FADT - > b87fe080/0000000000000000, using 32 > (XEN) ACPI: wakeup_vec[b87fe08c], vec_size[20] > (XEN) ACPI: Local APIC address 0xfee00000 > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) > (XEN) Processor #0 6:5 APIC version 21 > (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) > (XEN) Processor #2 6:5 APIC version 21 > (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled) > (XEN) Processor #1 6:5 APIC version 21 > (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled) > (XEN) Processor #3 6:5 APIC version 21 > (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1]) > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-39 > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) > (XEN) ACPI: IRQ0 used by override. > (XEN) ACPI: IRQ2 used by override. > (XEN) ACPI: IRQ9 used by override. > (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs > (XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000 > (XEN) [VT-D]iommu.c:1168: IOMMU: unsupported > (XEN) ---- print_iommu_regs ---- > (XEN) drhd->address = 0 > (XEN) VER = f0004800 > (XEN) CAP = f000eef3f000e2c3 > (XEN) n_fault_reg = ef > (XEN) fault_recording_offset = 3f00 > (XEN) ECAP = f000ff54f000eef5 > (XEN) GCMD = f0003182 > (XEN) GSTS = f000312a > (XEN) RTADDR = e8000304f000fea5 > (XEN) CCMD = f000eef3f000eef3 > (XEN) FSTS = f000eef3 > (XEN) FECTL = f000ef57 > (XEN) FEDATA = f000ff53 > (XEN) FEADDR = c0000014 > (XEN) FEUADDR = f000f84d > (XEN) Unknown interrupt (cr2=0000000000000000) > (XEN) ffff82c3ffd77ac8 ffff82c3ffd77ac8 ffff83023c4e5530 > ffff82c3ffd54000 ffff82c4c02d7c28 0000000000000000 0000000000000a0d > 0000000000000096 ffff82c3ffffc0a0 0000000050010412 0000000000000000 > ffff82c008000000 ffff82c3ffd54000 0000000000000030 ffff82c3ffd54000 > ffff82c4c012d525 000000000000e008 0000000000010086 ffff82c4c02d7c18 > 0000000000000000 ffff82c4c012d51a ffff82c3ffd54000 0000000000000000 > ffff82c4c02d7c48 ffff82c4c012d5f9 ffff83023c4e55c0 00000000ffffffed > ffff82c4c02d7c68 ffff82c4c029978f ffff82c3ffd77af8 ffff82c3ffd77af8 > ffff82c4c02d7cf8 ffff82c4c029a63a ffff82c3ffd77000 ffff8300b97641e0 > ffff82c4c02d7d10 ffff82c4c029a251 ffff82c4c02d7cb8 ffff82c4c029ea42 > ffff82c4c02d7cb8 ffff8300b97641e0 ffff82c4c02d7cd8 ffff82c4c029f2ff > 0000000000000000 ffff82c4c029a251 ffff82c4c024dbd5 ffff82c4c029a251 > 0000000000000000 0000000000002515 ffff82c4c02d7d28 ffff82c4c02a0607 > ffff82c3ffd78000 ffff82c3ffd77ac8 0000000000000000 0000000000000000 > ffff82c4c02d7df8 ffff82c4c02acd1f ffff82c4c02d7d68 ffff82c4c0225f56 > ffff82c4c02d7d58 0000000000000000 0000000000000163 000ffffffffff000 > ffff82c4c02d7df8 ffff82c4c0174cdc ffff82c4c02d7e08 00000000000001e3 > 00000000000b6dff 0000000000000163 00000000b6dff000 00000000000001e3 > 0000000000000000 00000000000b7000 ffff8300b7000ac8 ffff82c4c02d7e10 > ffff82c4c02d7df8 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 0000000000000000 ffff82c4c02d7e18 ffff82c4c029af1f > 0000000000000088 00000000b6fffac8 ffff82c4c02d7e48 ffff82c4c02ae4b7 > ffff82c4c02d7e48 ffff830000080f20 000000000023fdff ffff830000080fb0 > ffff82c4c02d7f08 ffff82c4c02aa6de 0000000000000000 0000000000000000 > 0000000000100000 ffff830000080be0 ffff830000080f20 0000000000000000 > 00000000aac00000 ffff830000080f01 ffff830000080fb0 0000000000fa4000 > 000000000000001c ffff82c4c02c3c34 0000000800000000 000000010000006e > 0000000000000003 00000000000002f8 0000000000000000 0000000000000000 > 000000000000180a 00000000a8ef4988 0000000000000001 00000000afbd9018 > 0000000000000000 ffff82c4c01000b5 0000000000000000 0000000000000000 > 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > 00000000fffff000 0000000000000000 0000000000000000
On Tue, Jul 9, 2013 at 9:18 AM, Jan Beulich <JBeulich@suse.com> wrote:>>>> On 09.07.13 at 14:55, Ben Guthro <ben@guthro.net> wrote: >> Jan, >> >> I was given a machine to diagnose a boot problem on xen 4.3, that used >> to work on 4.2, and have bisected to the following changeset: >> >> commit d0d4635d034f202bb401a6efa3ba61530f3854ab >> Author: Jan Beulich <jbeulich@suse.com> >> Date: Thu Nov 22 10:47:58 2012 +0100 >> >> implement vmap() >> >> ... and use it as basis for a proper ioremap() on x86. >> >> Signed-off-by: Jan Beulich <jbeulich@suse.com> >> Acked-by: Keir Fraser <keir@xen.org> >> >> >> Full diff: >> http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=d0d4635d034f202bb401a6ef >> a3ba61530f3854ab >> >> >> While this is pre-release hardware from an OEM, it did work on 4.2, so >> I am not (yet) inclined to blame it on being pre-release hardware. >> >> This is very early in boot, and so does not have a proper stack. >> Do you have any thoughts as to where this might be going awry? > > If you could make the xen-syms available matching the xen.gz that > was used when the below log was obtained, I could try to take a > look. > > Perhaps repeating the attempt with "iommu=debug" could reveal > further useful details.I have put xen.gz, the xen-syms, and an updated boot log, with iommu=debug at the following location that you may download: https://citrix.sharefile.com/d/safbf0b05aac4233a This is against the following commit on the stable-4.3 tree: commit 442ba0b2f6c16945ad496b31bf18a9e85d199d18 Author: Ian Jackson <ian.jackson@eu.citrix.com> Date: Thu Jun 27 11:08:07 2013 +0100 QEMU_UPSTREAM_REVISION update (which was the tip until a day, or so ago) Thanks in advance for taking a look. Ben> > Jan > >> The Xen boot log looks like the following in the bad case: >> >> \ \/ /___ _ __ | || | |___ / _ _ _ __ ___| |_ __ _| |__ | | ___ >> \ // _ \ ''_ \ | || |_ |_ \ __| | | | ''_ \/ __| __/ _` | ''_ \| |/ _ \ >> / \ __/ | | | |__ _| ___) |__| |_| | | | \__ \ || (_| | |_) | | __/ >> /_/\_\___|_| |_| |_|(_)____/ \__,_|_| |_|___/\__\__,_|_.__/|_|\___| >> >> (XEN) Xen version 4.3-unstable (@oldroadcomputing.net) (gcc >> (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3) debug=y Mon Jul 8 15:09:16 EDT >> 2013 >> (XEN) Latest ChangeSet: Mon Jul 8 15:08:40 2013 -0400 git:6a62f63 >> (XEN) Console output is synchronous. >> (XEN) Bootloader: COM32 Multiboot loader v0.2 >> (XEN) Command line: dom0_mem=max:2048M dbgp=ehci@pci00:1d.0 >> console=dbgp loglvl=all sync_console >> (XEN) Video information: >> (XEN) VGA is text mode 80x25, font 8x16 >> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >> (XEN) Disc information: >> (XEN) Found 1 MBR signatures >> (XEN) Found 1 EDD information structures >> (XEN) Xen-e820 RAM map: >> (XEN) 0000000000000000 - 0000000000091800 (usable) >> (XEN) 0000000000091800 - 00000000000a0000 (reserved) >> (XEN) 00000000000e0000 - 0000000000100000 (reserved) >> (XEN) 0000000000100000 - 00000000aad0b000 (usable) >> (XEN) 00000000aad0b000 - 00000000aad12000 (ACPI NVS) >> (XEN) 00000000aad12000 - 00000000ab917000 (usable) >> (XEN) 00000000ab917000 - 00000000abdd4000 (reserved) >> (XEN) 00000000abdd4000 - 00000000b5db4000 (usable) >> (XEN) 00000000b5db4000 - 00000000b6000000 (reserved) >> (XEN) 00000000b6000000 - 00000000b675c000 (usable) >> (XEN) 00000000b675c000 - 00000000b6800000 (reserved) >> (XEN) 00000000b6800000 - 00000000b6fb0000 (usable) >> (XEN) 00000000b6fb0000 - 00000000b7000000 (ACPI data) >> (XEN) 00000000b7000000 - 00000000b871c000 (usable) >> (XEN) 00000000b871c000 - 00000000b8800000 (ACPI NVS) >> (XEN) 00000000b8800000 - 00000000b9765000 (usable) >> (XEN) 00000000b9765000 - 00000000ba000000 (reserved) >> (XEN) 00000000bd000000 - 00000000bf200000 (reserved) >> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >> (XEN) 00000000fec00000 - 00000000fec01000 (reserved) >> (XEN) 00000000fed00000 - 00000000fed04000 (reserved) >> (XEN) 00000000fed1c000 - 00000000fed20000 (reserved) >> (XEN) 00000000fee00000 - 00000000fee01000 (reserved) >> (XEN) 00000000ff000000 - 0000000100000000 (reserved) >> (XEN) 0000000100000000 - 000000023fe00000 (usable) >> (XEN) ACPI: RSDP 000EEE50, 0024 (r2 DELL ) >> (XEN) ACPI: XSDT B6FF1088, 0094 (r1 DELL CBX3 1072009 AMI 10013) >> (XEN) ACPI: FACP B6FFDDE8, 010C (r5 DELL CBX3 1072009 AMI 10013) >> (XEN) ACPI: DSDT B6FF11B0, CC33 (r2 DELL CBX3 14 INTL 20091112) >> (XEN) ACPI: FACS B87FE080, 0040 >> (XEN) ACPI: APIC B6FFDEF8, 0072 (r3 DELL CBX3 1072009 AMI 10013) >> (XEN) ACPI: FPDT B6FFDF70, 0044 (r1 DELL CBX3 1072009 AMI 10013) >> (XEN) ACPI: ASF! B6FFDFB8, 00A5 (r32 INTEL HCG 1 TFSM F4240) >> (XEN) ACPI: SSDT B6FFE060, 01F8 (r1 INTEL sensrhub 0 INTL 20120711) >> (XEN) ACPI: SSDT B6FFE258, 01BE (r1 Intel zpoddult 1000 INTL 20091112) >> (XEN) ACPI: SSDT B6FFE418, 018E (r1 AcpiRe Ult0Rtd3 1000 INTL 20051117) >> (XEN) ACPI: SSDT B6FFE5A8, 0548 (r1 PmRef Cpu0Ist 3000 INTL 20051117) >> (XEN) ACPI: SSDT B6FFEAF0, 0AD8 (r1 PmRef CpuPm 3000 INTL 20051117) >> (XEN) ACPI: HPET B6FFF5C8, 0038 (r1 DELL CBX3 1072009 AMI. 5) >> (XEN) ACPI: SSDT B6FFF600, 03BD (r1 SataRe SataTabl 1000 INTL 20091112) >> (XEN) ACPI: MCFG B6FFF9C0, 003C (r1 DELL CBX3 1072009 MSFT 97) >> (XEN) ACPI: CSRT B6FFFA00, 00C4 (r1 INTL HSWULT-R 1 INTL 20100528) >> (XEN) ACPI: DMAR B6FFFAC8, 0088 (r1 INTEL HSW 1 INTL 1) >> (XEN) System RAM: 8076MB (8269880kB) >> (XEN) No NUMA configuration found >> (XEN) Faking a node at 0000000000000000-000000023fe00000 >> (XEN) Domain heap initialised >> (XEN) found SMP MP-table at 000fd990 >> (XEN) DMI 2.7 present. >> (XEN) Using APIC driver default >> (XEN) ACPI: PM-Timer IO Port: 0x1808 >> (XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0] >> (XEN) ACPI: SLEEP INFO: pm1x_cnt[1804,0], pm1x_evt[1800,0] >> (XEN) ACPI: 32/64X FACS address mismatch in FADT - >> b87fe080/0000000000000000, using 32 >> (XEN) ACPI: wakeup_vec[b87fe08c], vec_size[20] >> (XEN) ACPI: Local APIC address 0xfee00000 >> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) >> (XEN) Processor #0 6:5 APIC version 21 >> (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) >> (XEN) Processor #2 6:5 APIC version 21 >> (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled) >> (XEN) Processor #1 6:5 APIC version 21 >> (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled) >> (XEN) Processor #3 6:5 APIC version 21 >> (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1]) >> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) >> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-39 >> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) >> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) >> (XEN) ACPI: IRQ0 used by override. >> (XEN) ACPI: IRQ2 used by override. >> (XEN) ACPI: IRQ9 used by override. >> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs >> (XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000 >> (XEN) [VT-D]iommu.c:1168: IOMMU: unsupported >> (XEN) ---- print_iommu_regs ---- >> (XEN) drhd->address = 0 >> (XEN) VER = f0004800 >> (XEN) CAP = f000eef3f000e2c3 >> (XEN) n_fault_reg = ef >> (XEN) fault_recording_offset = 3f00 >> (XEN) ECAP = f000ff54f000eef5 >> (XEN) GCMD = f0003182 >> (XEN) GSTS = f000312a >> (XEN) RTADDR = e8000304f000fea5 >> (XEN) CCMD = f000eef3f000eef3 >> (XEN) FSTS = f000eef3 >> (XEN) FECTL = f000ef57 >> (XEN) FEDATA = f000ff53 >> (XEN) FEADDR = c0000014 >> (XEN) FEUADDR = f000f84d >> (XEN) Unknown interrupt (cr2=0000000000000000) >> (XEN) ffff82c3ffd77ac8 ffff82c3ffd77ac8 ffff83023c4e5530 >> ffff82c3ffd54000 ffff82c4c02d7c28 0000000000000000 0000000000000a0d >> 0000000000000096 ffff82c3ffffc0a0 0000000050010412 0000000000000000 >> ffff82c008000000 ffff82c3ffd54000 0000000000000030 ffff82c3ffd54000 >> ffff82c4c012d525 000000000000e008 0000000000010086 ffff82c4c02d7c18 >> 0000000000000000 ffff82c4c012d51a ffff82c3ffd54000 0000000000000000 >> ffff82c4c02d7c48 ffff82c4c012d5f9 ffff83023c4e55c0 00000000ffffffed >> ffff82c4c02d7c68 ffff82c4c029978f ffff82c3ffd77af8 ffff82c3ffd77af8 >> ffff82c4c02d7cf8 ffff82c4c029a63a ffff82c3ffd77000 ffff8300b97641e0 >> ffff82c4c02d7d10 ffff82c4c029a251 ffff82c4c02d7cb8 ffff82c4c029ea42 >> ffff82c4c02d7cb8 ffff8300b97641e0 ffff82c4c02d7cd8 ffff82c4c029f2ff >> 0000000000000000 ffff82c4c029a251 ffff82c4c024dbd5 ffff82c4c029a251 >> 0000000000000000 0000000000002515 ffff82c4c02d7d28 ffff82c4c02a0607 >> ffff82c3ffd78000 ffff82c3ffd77ac8 0000000000000000 0000000000000000 >> ffff82c4c02d7df8 ffff82c4c02acd1f ffff82c4c02d7d68 ffff82c4c0225f56 >> ffff82c4c02d7d58 0000000000000000 0000000000000163 000ffffffffff000 >> ffff82c4c02d7df8 ffff82c4c0174cdc ffff82c4c02d7e08 00000000000001e3 >> 00000000000b6dff 0000000000000163 00000000b6dff000 00000000000001e3 >> 0000000000000000 00000000000b7000 ffff8300b7000ac8 ffff82c4c02d7e10 >> ffff82c4c02d7df8 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 0000000000000000 ffff82c4c02d7e18 ffff82c4c029af1f >> 0000000000000088 00000000b6fffac8 ffff82c4c02d7e48 ffff82c4c02ae4b7 >> ffff82c4c02d7e48 ffff830000080f20 000000000023fdff ffff830000080fb0 >> ffff82c4c02d7f08 ffff82c4c02aa6de 0000000000000000 0000000000000000 >> 0000000000100000 ffff830000080be0 ffff830000080f20 0000000000000000 >> 00000000aac00000 ffff830000080f01 ffff830000080fb0 0000000000fa4000 >> 000000000000001c ffff82c4c02c3c34 0000000800000000 000000010000006e >> 0000000000000003 00000000000002f8 0000000000000000 0000000000000000 >> 000000000000180a 00000000a8ef4988 0000000000000001 00000000afbd9018 >> 0000000000000000 ffff82c4c01000b5 0000000000000000 0000000000000000 >> 0000000000000000 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 0000000000000000 0000000000000000 0000000000000000 >> 0000000000000000 0000000000000000 0000000000000000 0000000000000000 >> 00000000fffff000 0000000000000000 0000000000000000 > >
On Tue, Jul 9, 2013 at 12:34 PM, Ben Guthro <ben@guthro.net> wrote:> On Tue, Jul 9, 2013 at 9:18 AM, Jan Beulich <JBeulich@suse.com> wrote: >>>>> On 09.07.13 at 14:55, Ben Guthro <ben@guthro.net> wrote: >>> Jan, >>> >>> I was given a machine to diagnose a boot problem on xen 4.3, that used >>> to work on 4.2, and have bisected to the following changeset: >>> >>> commit d0d4635d034f202bb401a6efa3ba61530f3854ab >>> Author: Jan Beulich <jbeulich@suse.com> >>> Date: Thu Nov 22 10:47:58 2012 +0100 >>> >>> implement vmap() >>> >>> ... and use it as basis for a proper ioremap() on x86. >>> >>> Signed-off-by: Jan Beulich <jbeulich@suse.com> >>> Acked-by: Keir Fraser <keir@xen.org> >>> >>> >>> Full diff: >>> http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=d0d4635d034f202bb401a6ef >>> a3ba61530f3854ab >>> >>> >>> While this is pre-release hardware from an OEM, it did work on 4.2, so >>> I am not (yet) inclined to blame it on being pre-release hardware. >>> >>> This is very early in boot, and so does not have a proper stack. >>> Do you have any thoughts as to where this might be going awry? >> >> If you could make the xen-syms available matching the xen.gz that >> was used when the below log was obtained, I could try to take a >> look. >> >> Perhaps repeating the attempt with "iommu=debug" could reveal >> further useful details. > > I have put xen.gz, the xen-syms, and an updated boot log, with > iommu=debug at the following location that you may download: > https://citrix.sharefile.com/d/safbf0b05aac4233a > > This is against the following commit on the stable-4.3 tree: > > commit 442ba0b2f6c16945ad496b31bf18a9e85d199d18 > Author: Ian Jackson <ian.jackson@eu.citrix.com> > Date: Thu Jun 27 11:08:07 2013 +0100 > > QEMU_UPSTREAM_REVISION update > > > (which was the tip until a day, or so ago) > > > Thanks in advance for taking a look. > > BenAlso - FWIW, this has been seen on at least 2 platforms here. They are both pre-production Haswell systems - though I''m not sure if that is coincidence, or not. (but they are from two different OEMs)> > >> >> Jan >> >>> The Xen boot log looks like the following in the bad case: >>> >>> \ \/ /___ _ __ | || | |___ / _ _ _ __ ___| |_ __ _| |__ | | ___ >>> \ // _ \ ''_ \ | || |_ |_ \ __| | | | ''_ \/ __| __/ _` | ''_ \| |/ _ \ >>> / \ __/ | | | |__ _| ___) |__| |_| | | | \__ \ || (_| | |_) | | __/ >>> /_/\_\___|_| |_| |_|(_)____/ \__,_|_| |_|___/\__\__,_|_.__/|_|\___| >>> >>> (XEN) Xen version 4.3-unstable (@oldroadcomputing.net) (gcc >>> (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3) debug=y Mon Jul 8 15:09:16 EDT >>> 2013 >>> (XEN) Latest ChangeSet: Mon Jul 8 15:08:40 2013 -0400 git:6a62f63 >>> (XEN) Console output is synchronous. >>> (XEN) Bootloader: COM32 Multiboot loader v0.2 >>> (XEN) Command line: dom0_mem=max:2048M dbgp=ehci@pci00:1d.0 >>> console=dbgp loglvl=all sync_console >>> (XEN) Video information: >>> (XEN) VGA is text mode 80x25, font 8x16 >>> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds >>> (XEN) Disc information: >>> (XEN) Found 1 MBR signatures >>> (XEN) Found 1 EDD information structures >>> (XEN) Xen-e820 RAM map: >>> (XEN) 0000000000000000 - 0000000000091800 (usable) >>> (XEN) 0000000000091800 - 00000000000a0000 (reserved) >>> (XEN) 00000000000e0000 - 0000000000100000 (reserved) >>> (XEN) 0000000000100000 - 00000000aad0b000 (usable) >>> (XEN) 00000000aad0b000 - 00000000aad12000 (ACPI NVS) >>> (XEN) 00000000aad12000 - 00000000ab917000 (usable) >>> (XEN) 00000000ab917000 - 00000000abdd4000 (reserved) >>> (XEN) 00000000abdd4000 - 00000000b5db4000 (usable) >>> (XEN) 00000000b5db4000 - 00000000b6000000 (reserved) >>> (XEN) 00000000b6000000 - 00000000b675c000 (usable) >>> (XEN) 00000000b675c000 - 00000000b6800000 (reserved) >>> (XEN) 00000000b6800000 - 00000000b6fb0000 (usable) >>> (XEN) 00000000b6fb0000 - 00000000b7000000 (ACPI data) >>> (XEN) 00000000b7000000 - 00000000b871c000 (usable) >>> (XEN) 00000000b871c000 - 00000000b8800000 (ACPI NVS) >>> (XEN) 00000000b8800000 - 00000000b9765000 (usable) >>> (XEN) 00000000b9765000 - 00000000ba000000 (reserved) >>> (XEN) 00000000bd000000 - 00000000bf200000 (reserved) >>> (XEN) 00000000f8000000 - 00000000fc000000 (reserved) >>> (XEN) 00000000fec00000 - 00000000fec01000 (reserved) >>> (XEN) 00000000fed00000 - 00000000fed04000 (reserved) >>> (XEN) 00000000fed1c000 - 00000000fed20000 (reserved) >>> (XEN) 00000000fee00000 - 00000000fee01000 (reserved) >>> (XEN) 00000000ff000000 - 0000000100000000 (reserved) >>> (XEN) 0000000100000000 - 000000023fe00000 (usable) >>> (XEN) ACPI: RSDP 000EEE50, 0024 (r2 DELL ) >>> (XEN) ACPI: XSDT B6FF1088, 0094 (r1 DELL CBX3 1072009 AMI 10013) >>> (XEN) ACPI: FACP B6FFDDE8, 010C (r5 DELL CBX3 1072009 AMI 10013) >>> (XEN) ACPI: DSDT B6FF11B0, CC33 (r2 DELL CBX3 14 INTL 20091112) >>> (XEN) ACPI: FACS B87FE080, 0040 >>> (XEN) ACPI: APIC B6FFDEF8, 0072 (r3 DELL CBX3 1072009 AMI 10013) >>> (XEN) ACPI: FPDT B6FFDF70, 0044 (r1 DELL CBX3 1072009 AMI 10013) >>> (XEN) ACPI: ASF! B6FFDFB8, 00A5 (r32 INTEL HCG 1 TFSM F4240) >>> (XEN) ACPI: SSDT B6FFE060, 01F8 (r1 INTEL sensrhub 0 INTL 20120711) >>> (XEN) ACPI: SSDT B6FFE258, 01BE (r1 Intel zpoddult 1000 INTL 20091112) >>> (XEN) ACPI: SSDT B6FFE418, 018E (r1 AcpiRe Ult0Rtd3 1000 INTL 20051117) >>> (XEN) ACPI: SSDT B6FFE5A8, 0548 (r1 PmRef Cpu0Ist 3000 INTL 20051117) >>> (XEN) ACPI: SSDT B6FFEAF0, 0AD8 (r1 PmRef CpuPm 3000 INTL 20051117) >>> (XEN) ACPI: HPET B6FFF5C8, 0038 (r1 DELL CBX3 1072009 AMI. 5) >>> (XEN) ACPI: SSDT B6FFF600, 03BD (r1 SataRe SataTabl 1000 INTL 20091112) >>> (XEN) ACPI: MCFG B6FFF9C0, 003C (r1 DELL CBX3 1072009 MSFT 97) >>> (XEN) ACPI: CSRT B6FFFA00, 00C4 (r1 INTL HSWULT-R 1 INTL 20100528) >>> (XEN) ACPI: DMAR B6FFFAC8, 0088 (r1 INTEL HSW 1 INTL 1) >>> (XEN) System RAM: 8076MB (8269880kB) >>> (XEN) No NUMA configuration found >>> (XEN) Faking a node at 0000000000000000-000000023fe00000 >>> (XEN) Domain heap initialised >>> (XEN) found SMP MP-table at 000fd990 >>> (XEN) DMI 2.7 present. >>> (XEN) Using APIC driver default >>> (XEN) ACPI: PM-Timer IO Port: 0x1808 >>> (XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0] >>> (XEN) ACPI: SLEEP INFO: pm1x_cnt[1804,0], pm1x_evt[1800,0] >>> (XEN) ACPI: 32/64X FACS address mismatch in FADT - >>> b87fe080/0000000000000000, using 32 >>> (XEN) ACPI: wakeup_vec[b87fe08c], vec_size[20] >>> (XEN) ACPI: Local APIC address 0xfee00000 >>> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) >>> (XEN) Processor #0 6:5 APIC version 21 >>> (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) >>> (XEN) Processor #2 6:5 APIC version 21 >>> (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled) >>> (XEN) Processor #1 6:5 APIC version 21 >>> (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled) >>> (XEN) Processor #3 6:5 APIC version 21 >>> (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1]) >>> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) >>> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-39 >>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) >>> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) >>> (XEN) ACPI: IRQ0 used by override. >>> (XEN) ACPI: IRQ2 used by override. >>> (XEN) ACPI: IRQ9 used by override. >>> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs >>> (XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000 >>> (XEN) [VT-D]iommu.c:1168: IOMMU: unsupported >>> (XEN) ---- print_iommu_regs ---- >>> (XEN) drhd->address = 0 >>> (XEN) VER = f0004800 >>> (XEN) CAP = f000eef3f000e2c3 >>> (XEN) n_fault_reg = ef >>> (XEN) fault_recording_offset = 3f00 >>> (XEN) ECAP = f000ff54f000eef5 >>> (XEN) GCMD = f0003182 >>> (XEN) GSTS = f000312a >>> (XEN) RTADDR = e8000304f000fea5 >>> (XEN) CCMD = f000eef3f000eef3 >>> (XEN) FSTS = f000eef3 >>> (XEN) FECTL = f000ef57 >>> (XEN) FEDATA = f000ff53 >>> (XEN) FEADDR = c0000014 >>> (XEN) FEUADDR = f000f84d >>> (XEN) Unknown interrupt (cr2=0000000000000000) >>> (XEN) ffff82c3ffd77ac8 ffff82c3ffd77ac8 ffff83023c4e5530 >>> ffff82c3ffd54000 ffff82c4c02d7c28 0000000000000000 0000000000000a0d >>> 0000000000000096 ffff82c3ffffc0a0 0000000050010412 0000000000000000 >>> ffff82c008000000 ffff82c3ffd54000 0000000000000030 ffff82c3ffd54000 >>> ffff82c4c012d525 000000000000e008 0000000000010086 ffff82c4c02d7c18 >>> 0000000000000000 ffff82c4c012d51a ffff82c3ffd54000 0000000000000000 >>> ffff82c4c02d7c48 ffff82c4c012d5f9 ffff83023c4e55c0 00000000ffffffed >>> ffff82c4c02d7c68 ffff82c4c029978f ffff82c3ffd77af8 ffff82c3ffd77af8 >>> ffff82c4c02d7cf8 ffff82c4c029a63a ffff82c3ffd77000 ffff8300b97641e0 >>> ffff82c4c02d7d10 ffff82c4c029a251 ffff82c4c02d7cb8 ffff82c4c029ea42 >>> ffff82c4c02d7cb8 ffff8300b97641e0 ffff82c4c02d7cd8 ffff82c4c029f2ff >>> 0000000000000000 ffff82c4c029a251 ffff82c4c024dbd5 ffff82c4c029a251 >>> 0000000000000000 0000000000002515 ffff82c4c02d7d28 ffff82c4c02a0607 >>> ffff82c3ffd78000 ffff82c3ffd77ac8 0000000000000000 0000000000000000 >>> ffff82c4c02d7df8 ffff82c4c02acd1f ffff82c4c02d7d68 ffff82c4c0225f56 >>> ffff82c4c02d7d58 0000000000000000 0000000000000163 000ffffffffff000 >>> ffff82c4c02d7df8 ffff82c4c0174cdc ffff82c4c02d7e08 00000000000001e3 >>> 00000000000b6dff 0000000000000163 00000000b6dff000 00000000000001e3 >>> 0000000000000000 00000000000b7000 ffff8300b7000ac8 ffff82c4c02d7e10 >>> ffff82c4c02d7df8 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 0000000000000000 ffff82c4c02d7e18 ffff82c4c029af1f >>> 0000000000000088 00000000b6fffac8 ffff82c4c02d7e48 ffff82c4c02ae4b7 >>> ffff82c4c02d7e48 ffff830000080f20 000000000023fdff ffff830000080fb0 >>> ffff82c4c02d7f08 ffff82c4c02aa6de 0000000000000000 0000000000000000 >>> 0000000000100000 ffff830000080be0 ffff830000080f20 0000000000000000 >>> 00000000aac00000 ffff830000080f01 ffff830000080fb0 0000000000fa4000 >>> 000000000000001c ffff82c4c02c3c34 0000000800000000 000000010000006e >>> 0000000000000003 00000000000002f8 0000000000000000 0000000000000000 >>> 000000000000180a 00000000a8ef4988 0000000000000001 00000000afbd9018 >>> 0000000000000000 ffff82c4c01000b5 0000000000000000 0000000000000000 >>> 0000000000000000 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 0000000000000000 0000000000000000 0000000000000000 >>> 0000000000000000 0000000000000000 0000000000000000 0000000000000000 >>> 00000000fffff000 0000000000000000 0000000000000000 >> >>
>>> On 09.07.13 at 18:34, Ben Guthro <ben@guthro.net> wrote: > On Tue, Jul 9, 2013 at 9:18 AM, Jan Beulich <JBeulich@suse.com> wrote: >>>>> On 09.07.13 at 14:55, Ben Guthro <ben@guthro.net> wrote: >>> While this is pre-release hardware from an OEM, it did work on 4.2, so >>> I am not (yet) inclined to blame it on being pre-release hardware. >>> >>> This is very early in boot, and so does not have a proper stack. >>> Do you have any thoughts as to where this might be going awry? >> >> If you could make the xen-syms available matching the xen.gz that >> was used when the below log was obtained, I could try to take a >> look. >> >> Perhaps repeating the attempt with "iommu=debug" could reveal >> further useful details. > > I have put xen.gz, the xen-syms, and an updated boot log, with > iommu=debug at the following location that you may download: > https://citrix.sharefile.com/d/safbf0b05aac4233aThis (XEN) [VT-D]dmar.c:767: Host address width 39 (XEN) [VT-D]dmar.c:781: found ACPI_DMAR_DRHD: (XEN) [VT-D]dmar.c:461: dmaru->address = 0 is clearly a BIOS bug, and I don''t see how 4.2 would get along with that. It may fail in different ways, but that''s not directly related to how ioremap() is implemented. That said, the changeset you pointed at uncovered an always existing bug in the VT-d code: The mapping of the IOMMU MMIO register space is being established through map_to_nocache_virt() (resolving to set_fixmap_nocache()), but iommu_free() calls iounmap() to undo it (which prior to said changeset wasn''t doing anything). Bottom line - this box wants to be booted with "iommu=off" until you have a fixed BIOS. Jan
On Jul 10, 2013, at 5:09 AM, Jan Beulich <JBeulich@suse.com> wrote:>>>> On 09.07.13 at 18:34, Ben Guthro <ben@guthro.net> wrote: >> On Tue, Jul 9, 2013 at 9:18 AM, Jan Beulich <JBeulich@suse.com> wrote: >>>>>> On 09.07.13 at 14:55, Ben Guthro <ben@guthro.net> wrote: >>>> While this is pre-release hardware from an OEM, it did work on 4.2, so >>>> I am not (yet) inclined to blame it on being pre-release hardware. >>>> >>>> This is very early in boot, and so does not have a proper stack. >>>> Do you have any thoughts as to where this might be going awry? >>> >>> If you could make the xen-syms available matching the xen.gz that >>> was used when the below log was obtained, I could try to take a >>> look. >>> >>> Perhaps repeating the attempt with "iommu=debug" could reveal >>> further useful details. >> >> I have put xen.gz, the xen-syms, and an updated boot log, with >> iommu=debug at the following location that you may download: >> https://citrix.sharefile.com/d/safbf0b05aac4233a > > This > > (XEN) [VT-D]dmar.c:767: Host address width 39 > (XEN) [VT-D]dmar.c:781: found ACPI_DMAR_DRHD: > (XEN) [VT-D]dmar.c:461: dmaru->address = 0 > > is clearly a BIOS bug, and I don''t see how 4.2 would get along > with that. It may fail in different ways, but that''s not directly > related to how ioremap() is implemented. > > That said, the changeset you pointed at uncovered an always > existing bug in the VT-d code: The mapping of the IOMMU MMIO > register space is being established through map_to_nocache_virt() > (resolving to set_fixmap_nocache()), but iommu_free() calls > iounmap() to undo it (which prior to said changeset wasn''t > doing anything). > > Bottom line - this box wants to be booted with "iommu=off" until > you have a fixed BIOS.It would be better if we could detect such a buggy bios, and continue to boot without iommu capability, rather than crash, no? Thank you for the analysis. Is there a wiki page, or the like that might give me a pointer on how to do this analysis for myself in the future, so as not to bother you? (Eg, how to work with Xen-syms)> > Jan >
>>> On 10.07.13 at 12:17, Ben Guthro <ben.guthro@gmail.com> wrote: > On Jul 10, 2013, at 5:09 AM, Jan Beulich <JBeulich@suse.com> wrote: >> This >> >> (XEN) [VT-D]dmar.c:767: Host address width 39 >> (XEN) [VT-D]dmar.c:781: found ACPI_DMAR_DRHD: >> (XEN) [VT-D]dmar.c:461: dmaru->address = 0 >> >> is clearly a BIOS bug, and I don''t see how 4.2 would get along >> with that. It may fail in different ways, but that''s not directly >> related to how ioremap() is implemented. >> >> That said, the changeset you pointed at uncovered an always >> existing bug in the VT-d code: The mapping of the IOMMU MMIO >> register space is being established through map_to_nocache_virt() >> (resolving to set_fixmap_nocache()), but iommu_free() calls >> iounmap() to undo it (which prior to said changeset wasn''t >> doing anything). >> >> Bottom line - this box wants to be booted with "iommu=off" until >> you have a fixed BIOS. > > It would be better if we could detect such a buggy bios, and continue > to boot without iommu capability, rather than crash, no?Just sent a patch to that effect.> Thank you for the analysis. Is there a wiki page, or the like that > might give me a pointer on how to do this analysis for myself in the > future, so as not to bother you? (Eg, how to work with Xen-syms)There''s no magic here: Disassemble xen-syms, and work your way through the raw stack dump. Jan
Possibly Parallel Threads
- Re: DomU: piix:not 100% native mode: will probe irq later
- Asus 990FX chip xen iommu not found?
- Bug#797205: xen-hypervisor-4.4-amd64: Xen live migration dom0 memory swap
- Bug#706543: xen-hypervisor-4.1-amd64: HVM PCI Passthrough not working any more after last upgrade
- 3.5.0 dom0 crash on boot