Simon Horman
2009-Feb-16 07:07 UTC
[Xen-devel] [patch] fix iommu interrupt setup: setup_vector() may fail
Since changeset 19175 (Cleanup IOMMU interrupt setup) I am experiencing a
boot failure on my HP dc7800 running in x86_64 mode. The most recent
version that I have tested is 19210 (x86: Clean up vpt-align patch.), which
is after 19207 (iommu: Fix interrupt setup.) which Shimura-san created to
fix some of the problems caused by 19175.
After investigating this problem I found that the cause is that
for some vectors setup_vector() fails (returning -EBUSY).
setup_vector() is called by request_irq_vector().
And request_irq_vector() is called by iommu_set_interrupt().
Which is in turn called by init_vtd_hw().
And 19175 essentailly adds error checking code which now
checks for errors returned by iommu_set_interrupt() and
thus indirectly the error returned by setup_vector().
Unfortunately as things stands, the way that init_vtd_hw()
handles such errors causes my dc7800 not to boot.
I am not entirely sure of the correctness of my proposed change,
but by calling iommu_set_interrupt() before iommu_set_root_entry()
in init_vtd_hw() my system boots correctly.
In particularl I''m not sure:
a) if it is always valid to call iommu_set_interrupt() before
iommu_set_root_entry()
b) if this would adversely affect a situation where
iommu_set_root_entry() fails.
- This seems to be ok, I tested replacing the body of
iommu_set_root_entry() with return -EINVAL;
Signed-off-by: Simon Horman <horms@verge.net.au>
---
(XEN) Xen version 3.4-unstable (horms@kent.sydney.vergenet.net) (gcc version
4.0.2) Mon Feb 16 14:00:20 EST 2009
(XEN) Latest ChangeSet: Fri Feb 13 09:48:56 2009 +0000 19210:09a6fa059b37
(XEN) Command line: com1=115200,8n1,0x1248,0 console=com1 loglvl=all
guest_loglvl=all vtd=1 iommu=1
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN) EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009fc00 (usable)
(XEN) 000000000009fc00 - 00000000000a0000 (reserved)
(XEN) 00000000000e8000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000defafe00 (usable)
(XEN) 00000000defafe00 - 00000000defb1ea0 (ACPI NVS)
(XEN) 00000000defb1ea0 - 00000000e0000000 (reserved)
(XEN) 00000000f4000000 - 00000000f8000000 (reserved)
(XEN) 00000000fec00000 - 00000000fed40000 (reserved)
(XEN) 00000000fed45000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 000000011c000000 (usable)
(XEN) System RAM: 4015MB (4111672kB)
(XEN) ACPI: RSDP 000E5C10, 0014 (r0 COMPAQ)
(XEN) ACPI: RSDT DEFC1E40, 0044 (r1 HPQOEM SLIC-BPC 20080318 0)
(XEN) ACPI: FACP DEFC1EE8, 0074 (r1 COMPAQ BEARLAKE 1 0)
(XEN) ACPI: DSDT DEFC2427, A370 (r1 COMPAQ DSDT_PRJ 1 MSFT 100000E)
(XEN) ACPI: FACS DEFC1E00, 0040
(XEN) ACPI: APIC DEFC1F5C, 0084 (r1 COMPAQ BEARLAKE 1 0)
(XEN) ACPI: ASF! DEFC1FE0, 0063 (r32 COMPAQ BEARLAKE 1 0)
(XEN) ACPI: MCFG DEFC2043, 003C (r1 COMPAQ BEARLAKE 1 0)
(XEN) ACPI: TCPA DEFC207F, 0032 (r1 COMPAQ BEARLAKE 1 0)
(XEN) ACPI: SLIC DEFC20B1, 0176 (r1 HPQOEM SLIC-BPC 1 0)
(XEN) ACPI: HPET DEFC2227, 0038 (r1 COMPAQ BEARLAKE 1 0)
(XEN) ACPI: DMAR DEFC225F, 0198 (r1 COMPAQ BEARLAKE 1 0)
(XEN) NUMA turned off
(XEN) Faking a node at 0000000000000000-000000011c000000
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f9bf0
(XEN) DMI 2.5 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0xf808
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[f804,460], pm1x_evt[f800,0]
(XEN) ACPI: wakeup_vec[defc1e0c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
(XEN) Processor #1 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x00] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x00] disabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
(XEN) [VT-D]dmar.c:460: Host address width 36
(XEN) [VT-D]dmar.c:469: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:337: dmaru->address = fed90000
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1b.0
(XEN) [VT-D]dmar.c:469: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:337: dmaru->address = fed91000
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:469: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:337: dmaru->address = fed92000
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:3.0
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:3.2
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:3.3
(XEN) [VT-D]dmar.c:469: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:337: dmaru->address = fed93000
(XEN) [VT-D]dmar.c:346: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1d.7
(XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1a.7
(XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1d.0
(XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1d.1
(XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1d.2
(XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1a.0
(XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1a.1
(XEN) Intel VT-d has been enabled
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2826.324 MHz processor.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) HVM: VMX enabled
(XEN) MCE: Intel newly family MC Init
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) CPU0: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU0 has no CMCI support
(XEN) CPU0: Intel(R) Core(TM)2 Duo CPU E8300 @ 2.83GHz stepping 06
(XEN) Booting processor 1/1 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) MCE: Intel newly family MC Init
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) CPU1: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel(R) Core(TM)2 Duo CPU E8300 @ 2.83GHz stepping 06
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1707: Queued Invalidation hardware not found
(XEN) [VT-D]iommu.c:1707: Queued Invalidation hardware not found
(XEN) [VT-D]iommu.c:1707: Queued Invalidation hardware not found
(XEN) [VT-D]iommu.c:1707: Queued Invalidation hardware not found
(XEN) [VT-D]iommu.c:1715: Interrupt Remapping hardware not found
(XEN) [VT-D]iommu.c:1715: Interrupt Remapping hardware not found
(XEN) [VT-D]iommu.c:1715: Interrupt Remapping hardware not found
(XEN) [VT-D]iommu.c:1715: Interrupt Remapping hardware not found
(XEN) Intel VT-d snoop control disabled
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Init_mcheck_timer
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:0.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:3.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:3.2
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:3.3
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:19.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1246:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1f.5
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:618: iommu_enable_translation: iommu->reg =
ffff828bfff57000
(XEN) [VT-D]iommu.c:618: iommu_enable_translation: iommu->reg =
ffff828bfff56000
(XEN) [VT-D]iommu.c:618: iommu_enable_translation: iommu->reg =
ffff828bfff55000
(XEN) [VT-D]iommu.c:618: iommu_enable_translation: iommu->reg =
ffff828bfff54000
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_init: not an ELF binary
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Could not set up DOM0 guest OS
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
(XEN) [VT-D]iommu.c:721: iommu_page_fault: iommu->reg = ffff828bfff54000
(XEN) [VT-D]iommu.c:690: iommu_fault_status: Fault Overflow
(XEN) [VT-D]iommu.c:693: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]iommu.c:675: iommu_fault:DMA Read: 0:1d.2 addr defce000 REASON 6
iommu->reg = ffff828bfff54000
(XEN) print_vtd_entries: iommu = ffff83011bfd4310 bdf = 0:1d:2 gmfn = defce
(XEN) root_entry = ffff83011bfcb000
(XEN) root_entry[0] = 118453001
(XEN) context = ffff830118453000
(XEN) context[ea] = 101_11bfb1001
(XEN) l3 = ffff83011bfb1000
(XEN) l3_index = 3
(XEN) l3[3] = 11862d003
(XEN) l2 = ffff83011862d000
(XEN) l2_index = f7
(XEN) l2[f7] = 118535003
(XEN) l1 = ffff830118535000
(XEN) l1_index = 1ce
(XEN) l1[1ce] = 0
(XEN) l1[1ce] not present
Index: xen-unstable.hg/xen/drivers/passthrough/vtd/iommu.c
==================================================================---
xen-unstable.hg.orig/xen/drivers/passthrough/vtd/iommu.c 2009-02-16
17:14:35.000000000 +1100
+++ xen-unstable.hg/xen/drivers/passthrough/vtd/iommu.c 2009-02-16
18:06:09.000000000 +1100
@@ -1674,12 +1674,6 @@ static int init_vtd_hw(void)
for_each_drhd_unit ( drhd )
{
iommu = drhd->iommu;
- ret = iommu_set_root_entry(iommu);
- if ( ret )
- {
- gdprintk(XENLOG_ERR VTDPREFIX, "IOMMU: set root entry
failed\n");
- return -EIO;
- }
vector = iommu_set_interrupt(iommu);
if ( vector < 0 )
@@ -1687,6 +1681,12 @@ static int init_vtd_hw(void)
gdprintk(XENLOG_ERR VTDPREFIX, "IOMMU: interrupt setup
failed\n");
return vector;
}
+ ret = iommu_set_root_entry(iommu);
+ if ( ret )
+ {
+ gdprintk(XENLOG_ERR VTDPREFIX, "IOMMU: set root entry
failed\n");
+ return -EIO;
+ }
dma_msi_data_init(iommu, vector);
dma_msi_addr_init(iommu, cpu_physical_id(first_cpu(cpu_online_map)));
iommu->vector = vector;
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Simon Horman
2009-Feb-16 07:53 UTC
Re: [Xen-devel] [patch] fix iommu interrupt setup: setup_vector() may fail
On Mon, Feb 16, 2009 at 06:07:33PM +1100, Simon Horman wrote:> Since changeset 19175 (Cleanup IOMMU interrupt setup) I am experiencing a > boot failure on my HP dc7800 running in x86_64 mode. The most recent > version that I have tested is 19210 (x86: Clean up vpt-align patch.), which > is after 19207 (iommu: Fix interrupt setup.) which Shimura-san created to > fix some of the problems caused by 19175.Despite my long-winded explanation, there seems to be an error in my testing and this patch not work. I hope to have a revised version available shortly. -- Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel