thr3ads.net - Xen devel - [Xen-devel] [vt-d][xen4-rc6] Hangs on startup [Mar 2010]

If this information is useful, please help other people find it:
Share via:

Łukasz Oleś

2010-Mar-16 23:41 UTC

[Xen-devel] [vt-d][xen4-rc6] Hangs on startup

Hi,

I''m trying to run xen4-rc6 with iommu=1 option, but it hangs during the
startup.

When I run it with option iommu=0, it starts ok, and I''m able to run
HVM guests(didn''t try PV yet).
I have Intel Q45 chipset on the motherboard so it should have vt-d
support. As dom0 I tried jeremy''s 2.6.31 and 2.6.32 kernels.
The same thing happens on two identical machines. All vt and vt-d
options are enabled in BIOS.

Here is Serial Consol log:


(XEN) Xen version 4.0.0-rc6 (root@(none)) (gcc version 4.4.3 (Debian
4.4.3-2) ) Sun Mar 14 15:51:45 UTC 2010
(XEN) Latest ChangeSet: Thu Mar 11 17:40:35 2010 +0000
21028:4152a3ce90a7
(XEN) Console output is synchronous.
(XEN) Command line: iommu=1 dom0_mem=1024288 loglvl=all
guest_loglvl=all sync_console console_to_ring com1=115200,8n1,0xe1c0,0
console=com1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN)  EDID info not retrieved because no DDC retrieval method
detected
(XEN) Disc information:
(XEN)  Found 3 MBR signatures
(XEN)  Found 3 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009fc00 (usable)
(XEN)  000000000009fc00 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000bd3e2000 (usable)
(XEN)  00000000bd3e2000 - 00000000bd424000 (ACPI NVS)
(XEN)  00000000bd424000 - 00000000bd542000 (reserved)
(XEN)  00000000bd542000 - 00000000bd545000 (ACPI NVS)
(XEN)  00000000bd545000 - 00000000bd645000 (reserved)
(XEN)  00000000bd645000 - 00000000bd646000 (ACPI NVS)
(XEN)  00000000bd646000 - 00000000bd64b000 (reserved)
(XEN)  00000000bd64b000 - 00000000bd653000 (ACPI data)
(XEN)  00000000bd653000 - 00000000bd66e000 (ACPI NVS)
(XEN)  00000000bd66e000 - 00000000bd68c000 (reserved)
(XEN)  00000000bd68c000 - 00000000bd692000 (ACPI NVS)
(XEN)  00000000bd692000 - 00000000bd800000 (usable)
(XEN)  00000000bd800000 - 00000000bdb00000 (reserved)
(XEN)  00000000bdc00000 - 00000000c0000000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000013c000000 (usable)
(XEN) ACPI: RSDP 000F03C0, 0024 (r2  INTEL)
(XEN) ACPI: XSDT BD651E18, 005C (r1 INTEL  DQ45CB         3B MSFT
10013)
(XEN) ACPI: FACP BD650D98, 00F4 (r4  INTEL    A M I  6222004 MSFT
10013)
(XEN) ACPI: DSDT BD64B018, 4E98 (r1 INTEL  DQ45CB         3B INTL
20051117)
(XEN) ACPI: FACS BD659F40, 0040
(XEN) ACPI: APIC BD650F18, 006C (r2 INTEL  DQ45CB         3B MSFT
10013)
(XEN) ACPI: MCFG BD65BE18, 003C (r1 INTEL  DQ45CB         3B MSFT
 97)
(XEN) ACPI: ASF! BD65AD18, 00A0 (r32 INTEL  DQ45CB         3B TFSM
F4240)
(XEN) ACPI: SPCR BD65BD18, 0050 (r1 INTEL  DQ45CB         3B AMI.
  3)
(XEN) ACPI: TCPA BD65BC98, 0032 (r2 INTEL  DQ45CB         3B MSFT
1000013)
(XEN) ACPI: DMAR BD645C18, 0120 (r1 INTEL  DQ45CB         3B INTL
  1)
(XEN) System RAM: 3949MB (4044048kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000013c000000
(XEN) Domain heap initialised
(XEN) DMI 2.4 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
bd654f40/00000000bd659f40, using 32
(XEN) ACPI:                  wakeup_vec[bd654f4c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
(XEN) Processor #1 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] disabled)
(XEN) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) [VT-D]dmar.c:643: Host address width 36
(XEN) [VT-D]dmar.c:652: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fed90000
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1b.0
(XEN) [VT-D]dmar.c:652: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fed91000
(XEN) [VT-D]dmar.c:331:   endpoint: 0:2.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:2.1
(XEN) [VT-D]dmar.c:652: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fed92000
(XEN) [VT-D]dmar.c:331:   endpoint: 0:3.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:3.2
(XEN) [VT-D]dmar.c:331:   endpoint: 0:3.3
(XEN) [VT-D]dmar.c:652: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fed93000
(XEN) [VT-D]dmar.c:391:   flags: INCLUDE_ALL
(XEN) [VT-D]dmar.c:656: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.1
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.2
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.7
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.1
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.2
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.7
(XEN) [VT-D]dmar.c:546:   RMRR region: base_addr bd62f000 end_address
bd642fff
(XEN) [VT-D]dmar.c:656: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:481:   RMRR address range not in reserved memory
base = bdc00000 end = ffffffffbfffffff; iommu_inclusive_mapping=1
parameter may be needed.
(XEN) [VT-D]dmar.c:331:   endpoint: 0:2.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:2.1
(XEN) [VT-D]dmar.c:546:   RMRR region: base_addr bdc00000 end_address
ffffffffbfffffff
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
(XEN) PCI: Not using MMCONFIG.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2800.206 MHz processor.
(XEN) Initing memory sharing.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 2048K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN) HVM: ASIDs disabled.
(XEN) HVM: VMX enabled
(XEN) xsave_init: cpu0: cntxt_max_size: 0x240 and states:
00000000:00000003
(XEN) xsave_init: using cntxt_size: 0x240 and states:
00000000:00000003
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) CPU0: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU0 has no CMCI support
(XEN) [VT-D]iommu.c:1075: drhd->address = fed92000 iommu->reg
ffff82c3fff57000
(XEN) [VT-D]iommu.c:1077: cap = c0000020630272 ecap = 1000
(XEN) [VT-D]iommu.c:1075: drhd->address = fed91000 iommu->reg
ffff82c3fff56000
(XEN) [VT-D]iommu.c:1077: cap = c0000020630272 ecap = 1000
(XEN) [VT-D]iommu.c:1075: drhd->address = fed90000 iommu->reg
ffff82c3fff55000
(XEN) [VT-D]iommu.c:1077: cap = c9008020e30272 ecap = 1000
(XEN) [VT-D]iommu.c:1075: drhd->address = fed93000 iommu->reg
ffff82c3fff54000
(XEN) [VT-D]iommu.c:1077: cap = c9008020630272 ecap = 1000
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) CPU0: Intel Pentium(R) Dual-Core  CPU      E6300  @ 2.80GHz
stepping 0a
(XEN) Booting processor 1/1 eip 88000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 2048K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) HVM: ASIDs disabled.
(XEN) xsave_init: cpu1: cntxt_max_size: 0x240 and states:
00000000:00000003
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) CPU1: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel Pentium(R) Dual-Core  CPU      E6300  @ 2.80GHz
stepping 0a
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) CPUIDLE: disabled due to no HPET. Force enable with
''cpuidle''.
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:0.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:0.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:2.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:2.1:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:3.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:3.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:3.2
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:3.2:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:3.3
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:3.3:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:19.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:19.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1a.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1a.1:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1a.2:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1a.7:  no extended config
(XEN) [VT-D]iommu.c:1321:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1b.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1d.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1d.1:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1d.2:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1d.7:  no extended config
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1e.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1f.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1f.2:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1f.3
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1f.3:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1f.5
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1f.5:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 1:0.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:1:0.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 1:1.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:1:1.0:  no extended config


Regards
Łukasz Oleś

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Mar-16 23:55 UTC

head link

Re: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

On Wed, Mar 17, 2010 at 12:41:50AM +0100, Łukasz Oleś
wrote:> Hi,
> 
> I''m trying to run xen4-rc6 with iommu=1 option, but it hangs
during the startup.
.. snip ..> (XEN) [VT-D]dmar.c:481:   RMRR address range not in reserved memory
> base = bdc00000 end = ffffffffbfffffff; iommu_inclusive_mapping=1
> parameter may be needed.
Did you try that?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Cui, Dexuan

2010-Mar-17 08:02 UTC

head link

RE: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

This issue was reported last year in this mailing list:
http://lists.xensource.com/archives/html/xen-devel/2009-05/msg00917.html
It turned out to be a BIOS issue.
Please update to the latest BIOS to see if the issue is still there or not. The
link above also supplies a workaround patch.

Thanks,
-- Dexuan
 

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of ?ukasz Ole?
Sent: 2010年3月17日 7:42
To: xen-devel@lists.xensource.com
Subject: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

Hi,

I'm trying to run xen4-rc6 with iommu=1 option, but it hangs during the
startup.

When I run it with option iommu=0, it starts ok, and I'm able to run
HVM guests(didn't try PV yet).
I have Intel Q45 chipset on the motherboard so it should have vt-d
support. As dom0 I tried jeremy's 2.6.31 and 2.6.32 kernels.
The same thing happens on two identical machines. All vt and vt-d
options are enabled in BIOS.

Here is Serial Consol log:


(XEN) Xen version 4.0.0-rc6 (root@(none)) (gcc version 4.4.3 (Debian
4.4.3-2) ) Sun Mar 14 15:51:45 UTC 2010
(XEN) Latest ChangeSet: Thu Mar 11 17:40:35 2010 +0000
21028:4152a3ce90a7
(XEN) Console output is synchronous.
(XEN) Command line: iommu=1 dom0_mem=1024288 loglvl=all
guest_loglvl=all sync_console console_to_ring com1=115200,8n1,0xe1c0,0
console=com1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN)  EDID info not retrieved because no DDC retrieval method
detected
(XEN) Disc information:
(XEN)  Found 3 MBR signatures
(XEN)  Found 3 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009fc00 (usable)
(XEN)  000000000009fc00 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000bd3e2000 (usable)
(XEN)  00000000bd3e2000 - 00000000bd424000 (ACPI NVS)
(XEN)  00000000bd424000 - 00000000bd542000 (reserved)
(XEN)  00000000bd542000 - 00000000bd545000 (ACPI NVS)
(XEN)  00000000bd545000 - 00000000bd645000 (reserved)
(XEN)  00000000bd645000 - 00000000bd646000 (ACPI NVS)
(XEN)  00000000bd646000 - 00000000bd64b000 (reserved)
(XEN)  00000000bd64b000 - 00000000bd653000 (ACPI data)
(XEN)  00000000bd653000 - 00000000bd66e000 (ACPI NVS)
(XEN)  00000000bd66e000 - 00000000bd68c000 (reserved)
(XEN)  00000000bd68c000 - 00000000bd692000 (ACPI NVS)
(XEN)  00000000bd692000 - 00000000bd800000 (usable)
(XEN)  00000000bd800000 - 00000000bdb00000 (reserved)
(XEN)  00000000bdc00000 - 00000000c0000000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000013c000000 (usable)
(XEN) ACPI: RSDP 000F03C0, 0024 (r2  INTEL)
(XEN) ACPI: XSDT BD651E18, 005C (r1 INTEL  DQ45CB         3B MSFT
10013)
(XEN) ACPI: FACP BD650D98, 00F4 (r4  INTEL    A M I  6222004 MSFT
10013)
(XEN) ACPI: DSDT BD64B018, 4E98 (r1 INTEL  DQ45CB         3B INTL
20051117)
(XEN) ACPI: FACS BD659F40, 0040
(XEN) ACPI: APIC BD650F18, 006C (r2 INTEL  DQ45CB         3B MSFT
10013)
(XEN) ACPI: MCFG BD65BE18, 003C (r1 INTEL  DQ45CB         3B MSFT
 97)
(XEN) ACPI: ASF! BD65AD18, 00A0 (r32 INTEL  DQ45CB         3B TFSM
F4240)
(XEN) ACPI: SPCR BD65BD18, 0050 (r1 INTEL  DQ45CB         3B AMI.
  3)
(XEN) ACPI: TCPA BD65BC98, 0032 (r2 INTEL  DQ45CB         3B MSFT
1000013)
(XEN) ACPI: DMAR BD645C18, 0120 (r1 INTEL  DQ45CB         3B INTL
  1)
(XEN) System RAM: 3949MB (4044048kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000013c000000
(XEN) Domain heap initialised
(XEN) DMI 2.4 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
bd654f40/00000000bd659f40, using 32
(XEN) ACPI:                  wakeup_vec[bd654f4c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
(XEN) Processor #1 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] disabled)
(XEN) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) [VT-D]dmar.c:643: Host address width 36
(XEN) [VT-D]dmar.c:652: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fed90000
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1b.0
(XEN) [VT-D]dmar.c:652: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fed91000
(XEN) [VT-D]dmar.c:331:   endpoint: 0:2.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:2.1
(XEN) [VT-D]dmar.c:652: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fed92000
(XEN) [VT-D]dmar.c:331:   endpoint: 0:3.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:3.2
(XEN) [VT-D]dmar.c:331:   endpoint: 0:3.3
(XEN) [VT-D]dmar.c:652: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fed93000
(XEN) [VT-D]dmar.c:391:   flags: INCLUDE_ALL
(XEN) [VT-D]dmar.c:656: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.1
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.2
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.7
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.1
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.2
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.7
(XEN) [VT-D]dmar.c:546:   RMRR region: base_addr bd62f000 end_address
bd642fff
(XEN) [VT-D]dmar.c:656: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:481:   RMRR address range not in reserved memory
base = bdc00000 end = ffffffffbfffffff; iommu_inclusive_mapping=1
parameter may be needed.
(XEN) [VT-D]dmar.c:331:   endpoint: 0:2.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:2.1
(XEN) [VT-D]dmar.c:546:   RMRR region: base_addr bdc00000 end_address
ffffffffbfffffff
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
(XEN) PCI: Not using MMCONFIG.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2800.206 MHz processor.
(XEN) Initing memory sharing.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 2048K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN) HVM: ASIDs disabled.
(XEN) HVM: VMX enabled
(XEN) xsave_init: cpu0: cntxt_max_size: 0x240 and states:
00000000:00000003
(XEN) xsave_init: using cntxt_size: 0x240 and states:
00000000:00000003
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) CPU0: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU0 has no CMCI support
(XEN) [VT-D]iommu.c:1075: drhd->address = fed92000 iommu->reg
ffff82c3fff57000
(XEN) [VT-D]iommu.c:1077: cap = c0000020630272 ecap = 1000
(XEN) [VT-D]iommu.c:1075: drhd->address = fed91000 iommu->reg
ffff82c3fff56000
(XEN) [VT-D]iommu.c:1077: cap = c0000020630272 ecap = 1000
(XEN) [VT-D]iommu.c:1075: drhd->address = fed90000 iommu->reg
ffff82c3fff55000
(XEN) [VT-D]iommu.c:1077: cap = c9008020e30272 ecap = 1000
(XEN) [VT-D]iommu.c:1075: drhd->address = fed93000 iommu->reg
ffff82c3fff54000
(XEN) [VT-D]iommu.c:1077: cap = c9008020630272 ecap = 1000
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) CPU0: Intel Pentium(R) Dual-Core  CPU      E6300  @ 2.80GHz
stepping 0a
(XEN) Booting processor 1/1 eip 88000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 2048K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) HVM: ASIDs disabled.
(XEN) xsave_init: cpu1: cntxt_max_size: 0x240 and states:
00000000:00000003
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) CPU1: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel Pentium(R) Dual-Core  CPU      E6300  @ 2.80GHz
stepping 0a
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) CPUIDLE: disabled due to no HPET. Force enable with 'cpuidle'.
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:0.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:0.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:2.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:2.1:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:3.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:3.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:3.2
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:3.2:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:3.3
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:3.3:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:19.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:19.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1a.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1a.1:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1a.2:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1a.7:  no extended config
(XEN) [VT-D]iommu.c:1321:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1b.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1d.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1d.1:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1d.2:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1d.7:  no extended config
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1e.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1f.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1f.2:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1f.3
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1f.3:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1f.5
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1f.5:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 1:0.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:1:0.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 1:1.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:1:1.0:  no extended config


Regards
Łukasz Oleś

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2010-Mar-17 08:21 UTC

head link

Re: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

On 16/3/10 23:55, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com> wrote:
> On Wed, Mar 17, 2010 at 12:41:50AM +0100, Łukasz Oleś wrote:
>> Hi,
>> 
>> I''m trying to run xen4-rc6 with iommu=1 option, but it hangs
during the
>> startup.
> 
> .. snip ..
>> (XEN) [VT-D]dmar.c:481:   RMRR address range not in reserved memory
>> base = bdc00000 end = ffffffffbfffffff; iommu_inclusive_mapping=1
>> parameter may be needed.
> 
> Did you try that?
Actually iommu_inclusive_mapping=1 is the default now. The above warning
appears to print even when iommu_inclusive_mapping is set. :-)

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Cui, Dexuan

2010-Mar-17 08:43 UTC

head link

RE: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

BTW, this is the version of the workaround patch for the latest xen-untable:

diff --git a/xen/drivers/passthrough/vtd/dmar.c
b/xen/drivers/passthrough/vtd/dmar.c
--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -480,6 +480,8 @@ acpi_parse_one_rmrr(struct acpi_dmar_ent
                 "iommu_inclusive_mapping=1 parameter may be
needed.\n",
                 base_addr, end_addr);
     }
+    if ( (end_addr >> 32) & 0x80000000 )
+        end_addr &= 0xFFFFFFFF;
 #endif

     rmrru = xmalloc(struct acpi_rmrr_unit); 

Thanks,
-- Dexuan

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Cui, Dexuan
Sent: 2010年3月17日 16:02
To: Łukasz Oleś; xen-devel@lists.xensource.com
Subject: RE: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

This issue was reported last year in this mailing list:
http://lists.xensource.com/archives/html/xen-devel/2009-05/msg00917.html
It turned out to be a BIOS issue.
Please update to the latest BIOS to see if the issue is still there or not. The
link above also supplies a workaround patch.

Thanks,
-- Dexuan
 

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of ?ukasz Ole?
Sent: 2010年3月17日 7:42
To: xen-devel@lists.xensource.com
Subject: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

Hi,

I'm trying to run xen4-rc6 with iommu=1 option, but it hangs during the
startup.

When I run it with option iommu=0, it starts ok, and I'm able to run
HVM guests(didn't try PV yet).
I have Intel Q45 chipset on the motherboard so it should have vt-d
support. As dom0 I tried jeremy's 2.6.31 and 2.6.32 kernels.
The same thing happens on two identical machines. All vt and vt-d
options are enabled in BIOS.

Here is Serial Consol log:


(XEN) Xen version 4.0.0-rc6 (root@(none)) (gcc version 4.4.3 (Debian
4.4.3-2) ) Sun Mar 14 15:51:45 UTC 2010
(XEN) Latest ChangeSet: Thu Mar 11 17:40:35 2010 +0000
21028:4152a3ce90a7
(XEN) Console output is synchronous.
(XEN) Command line: iommu=1 dom0_mem=1024288 loglvl=all
guest_loglvl=all sync_console console_to_ring com1=115200,8n1,0xe1c0,0
console=com1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN)  EDID info not retrieved because no DDC retrieval method
detected
(XEN) Disc information:
(XEN)  Found 3 MBR signatures
(XEN)  Found 3 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009fc00 (usable)
(XEN)  000000000009fc00 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000bd3e2000 (usable)
(XEN)  00000000bd3e2000 - 00000000bd424000 (ACPI NVS)
(XEN)  00000000bd424000 - 00000000bd542000 (reserved)
(XEN)  00000000bd542000 - 00000000bd545000 (ACPI NVS)
(XEN)  00000000bd545000 - 00000000bd645000 (reserved)
(XEN)  00000000bd645000 - 00000000bd646000 (ACPI NVS)
(XEN)  00000000bd646000 - 00000000bd64b000 (reserved)
(XEN)  00000000bd64b000 - 00000000bd653000 (ACPI data)
(XEN)  00000000bd653000 - 00000000bd66e000 (ACPI NVS)
(XEN)  00000000bd66e000 - 00000000bd68c000 (reserved)
(XEN)  00000000bd68c000 - 00000000bd692000 (ACPI NVS)
(XEN)  00000000bd692000 - 00000000bd800000 (usable)
(XEN)  00000000bd800000 - 00000000bdb00000 (reserved)
(XEN)  00000000bdc00000 - 00000000c0000000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000013c000000 (usable)
(XEN) ACPI: RSDP 000F03C0, 0024 (r2  INTEL)
(XEN) ACPI: XSDT BD651E18, 005C (r1 INTEL  DQ45CB         3B MSFT
10013)
(XEN) ACPI: FACP BD650D98, 00F4 (r4  INTEL    A M I  6222004 MSFT
10013)
(XEN) ACPI: DSDT BD64B018, 4E98 (r1 INTEL  DQ45CB         3B INTL
20051117)
(XEN) ACPI: FACS BD659F40, 0040
(XEN) ACPI: APIC BD650F18, 006C (r2 INTEL  DQ45CB         3B MSFT
10013)
(XEN) ACPI: MCFG BD65BE18, 003C (r1 INTEL  DQ45CB         3B MSFT
 97)
(XEN) ACPI: ASF! BD65AD18, 00A0 (r32 INTEL  DQ45CB         3B TFSM
F4240)
(XEN) ACPI: SPCR BD65BD18, 0050 (r1 INTEL  DQ45CB         3B AMI.
  3)
(XEN) ACPI: TCPA BD65BC98, 0032 (r2 INTEL  DQ45CB         3B MSFT
1000013)
(XEN) ACPI: DMAR BD645C18, 0120 (r1 INTEL  DQ45CB         3B INTL
  1)
(XEN) System RAM: 3949MB (4044048kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000013c000000
(XEN) Domain heap initialised
(XEN) DMI 2.4 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
bd654f40/00000000bd659f40, using 32
(XEN) ACPI:                  wakeup_vec[bd654f4c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
(XEN) Processor #1 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] disabled)
(XEN) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) [VT-D]dmar.c:643: Host address width 36
(XEN) [VT-D]dmar.c:652: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fed90000
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1b.0
(XEN) [VT-D]dmar.c:652: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fed91000
(XEN) [VT-D]dmar.c:331:   endpoint: 0:2.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:2.1
(XEN) [VT-D]dmar.c:652: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fed92000
(XEN) [VT-D]dmar.c:331:   endpoint: 0:3.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:3.2
(XEN) [VT-D]dmar.c:331:   endpoint: 0:3.3
(XEN) [VT-D]dmar.c:652: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fed93000
(XEN) [VT-D]dmar.c:391:   flags: INCLUDE_ALL
(XEN) [VT-D]dmar.c:656: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.1
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.2
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.7
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.1
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.2
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.7
(XEN) [VT-D]dmar.c:546:   RMRR region: base_addr bd62f000 end_address
bd642fff
(XEN) [VT-D]dmar.c:656: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:481:   RMRR address range not in reserved memory
base = bdc00000 end = ffffffffbfffffff; iommu_inclusive_mapping=1
parameter may be needed.
(XEN) [VT-D]dmar.c:331:   endpoint: 0:2.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:2.1
(XEN) [VT-D]dmar.c:546:   RMRR region: base_addr bdc00000 end_address
ffffffffbfffffff
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
(XEN) PCI: Not using MMCONFIG.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2800.206 MHz processor.
(XEN) Initing memory sharing.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 2048K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN) HVM: ASIDs disabled.
(XEN) HVM: VMX enabled
(XEN) xsave_init: cpu0: cntxt_max_size: 0x240 and states:
00000000:00000003
(XEN) xsave_init: using cntxt_size: 0x240 and states:
00000000:00000003
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) CPU0: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU0 has no CMCI support
(XEN) [VT-D]iommu.c:1075: drhd->address = fed92000 iommu->reg
ffff82c3fff57000
(XEN) [VT-D]iommu.c:1077: cap = c0000020630272 ecap = 1000
(XEN) [VT-D]iommu.c:1075: drhd->address = fed91000 iommu->reg
ffff82c3fff56000
(XEN) [VT-D]iommu.c:1077: cap = c0000020630272 ecap = 1000
(XEN) [VT-D]iommu.c:1075: drhd->address = fed90000 iommu->reg
ffff82c3fff55000
(XEN) [VT-D]iommu.c:1077: cap = c9008020e30272 ecap = 1000
(XEN) [VT-D]iommu.c:1075: drhd->address = fed93000 iommu->reg
ffff82c3fff54000
(XEN) [VT-D]iommu.c:1077: cap = c9008020630272 ecap = 1000
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) CPU0: Intel Pentium(R) Dual-Core  CPU      E6300  @ 2.80GHz
stepping 0a
(XEN) Booting processor 1/1 eip 88000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 2048K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) HVM: ASIDs disabled.
(XEN) xsave_init: cpu1: cntxt_max_size: 0x240 and states:
00000000:00000003
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) CPU1: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel Pentium(R) Dual-Core  CPU      E6300  @ 2.80GHz
stepping 0a
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) CPUIDLE: disabled due to no HPET. Force enable with 'cpuidle'.
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:0.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:0.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:2.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:2.1:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:3.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:3.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:3.2
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:3.2:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:3.3
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:3.3:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:19.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:19.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1a.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1a.1:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1a.2:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1a.7:  no extended config
(XEN) [VT-D]iommu.c:1321:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1b.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1d.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1d.1:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1d.2:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1d.7:  no extended config
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1e.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1f.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1f.2:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1f.3
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1f.3:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 0:1f.5
(XEN) [VT-D]mmconfig-shared.c:460: next cap:0:1f.5:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 1:0.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:1:0.0:  no extended config
(XEN) [VT-D]iommu.c:1328:d32767 domain_context_mapping:PCI: bdf = 1:1.0
(XEN) [VT-D]mmconfig-shared.c:460: next cap:1:1.0:  no extended config


Regards
Łukasz Oleś

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ross Philipson

2010-Mar-17 12:46 UTC

head link

RE: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

Actually I think I ran into this once before. Someone sent me a screen shot of
their e820 mappings from the actual BIOS setup program clearly showing the
memory range was reserved yet the check that generated the message below was
triggered. I could not reproduce it but I think I tracked it down to changes
that xen was making to the e820 mappings in sanitize_e820_map() (that is IIRC,
it was a year ago or more).

Thanks
Ross

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Keir Fraser
Sent: Wednesday, March 17, 2010 4:22 AM
To: Konrad Rzeszutek Wilk; Łukasz Oleś 
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

On 16/3/10 23:55, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com> wrote:
> On Wed, Mar 17, 2010 at 12:41:50AM +0100, Łukasz Oleś wrote:
>> Hi,
>> 
>> I''m trying to run xen4-rc6 with iommu=1 option, but it hangs
during the
>> startup.
> 
> .. snip ..
>> (XEN) [VT-D]dmar.c:481:   RMRR address range not in reserved memory
>> base = bdc00000 end = ffffffffbfffffff; iommu_inclusive_mapping=1
>> parameter may be needed.
> 
> Did you try that?
Actually iommu_inclusive_mapping=1 is the default now. The above warning
appears to print even when iommu_inclusive_mapping is set. :-)

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Łukasz Oleś

2010-Mar-17 18:20 UTC

head link

[Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Hi,

I tried iommu_inclusive_mapping=1 but as Keir said it was alredy
enabled so it didn''t help.
Next I applied patch from Cui and Dom0 started and I was able to run HVM guest.

I will update bios later in this week and test it again without patch.

After dom0 started i tried to pass-through two devices using pci-stub,
but it didn''t work.

Heres some log files:

commands: http://pastebin.com/ccjCNR9s
xen log: http://pastebin.com/fNaXLfbX
dom0 dmesg:  http://pastebin.com/AQAbZV4x
HVM config(based on xmexample.hvm): http://pastebin.com/cSLmxWsP

In dmesg there are some network errors, I think you can ignore them. I
need to change network-script


Regards
Łukasz Oleś

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Mar-17 20:14 UTC

head link

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

On Wed, Mar 17, 2010 at 07:20:19PM +0100, Łukasz Oleś
wrote:> Hi,
> 
> I tried iommu_inclusive_mapping=1 but as Keir said it was alredy
> enabled so it didn''t help.
> Next I applied patch from Cui and Dom0 started and I was able to run HVM
guest.
> 
> I will update bios later in this week and test it again without patch.
> 
> After dom0 started i tried to pass-through two devices using pci-stub,
> but it didn''t work.
> 
> Heres some log files:
> 
> commands: http://pastebin.com/ccjCNR9s
> xen log: http://pastebin.com/fNaXLfbX
This is troubling:
# (XEN) mm.c:859:d0 Error getting mfn 40100 (pfn 5555555555555555) from L1
entry 8000000040100473 for l1e_owner=0, pg_owner=32753

You need to run Linux kernel with more debugging. Use ''debug
loglevel=10'' to see it conjoined with Xen.

Also can you try to boot your machine _without_ using X?

When is this happening?
# (XEN) mm.c:859:d2 Error getting mfn 40010 (pfn 5555555555555555) from L1
entry 0000000040010037 for l1e_owner=2, pg_owner=2

Is this when you start your guest? At what point does this happen in the
guest? Can you attach the screenshot of your guest when this is
happening?
> dom0 dmesg:  http://pastebin.com/AQAbZV4x
> HVM config(based on xmexample.hvm): http://pastebin.com/cSLmxWsP
> 
> In dmesg there are some network errors, I think you can ignore them. I
> need to change network-script
> 
> 
> Regards
> Łukasz Oleś
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Łukasz Oleś

2010-Mar-17 21:48 UTC

head link

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

2010/3/17 Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com>:> On Wed, Mar 17, 2010 at 07:20:19PM +0100, Łukasz Oleś wrote:
> This is troubling:
> # (XEN) mm.c:859:d0 Error getting mfn 40100 (pfn 5555555555555555) from L1
> entry 8000000040100473 for l1e_owner=0, pg_owner=32753
It''s happening durring dom0 start with and without iommu enabled.
> You need to run Linux kernel with more debugging. Use ''debug
> loglevel=10'' to see it conjoined with Xen.
I need to recompile kernel, then I will post new logs.
> Also can you try to boot your machine _without_ using X?
You mean dom0 without XWindow? I don''t have XWindow.
Or you mean dom0 without Xen? It works ok.
Or you mean domU without XWindow? I will try.
> When is this happening?
> # (XEN) mm.c:859:d2 Error getting mfn 40010 (pfn 5555555555555555) from L1
> entry 0000000040010037 for l1e_owner=2, pg_owner=2
It starts to show after xm create and stops after xm destroy when I
try to pass-through devices. Without pci=[ <devices>] option there is
no such error.
> At what point does this happen in the
> guest? Can you attach the screenshot of your guest when this is
> happening?
My vnc shows only white screen. xm console doesn''t work. I have no
access to guest.

Without pci option guest works fine and I can connect to it.

Regards,
Łukasz Oleś

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Mar-18 14:19 UTC

head link

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

On Wed, Mar 17, 2010 at 10:48:28PM +0100, Łukasz Oleś
wrote:> 2010/3/17 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>:
> > On Wed, Mar 17, 2010 at 07:20:19PM +0100, Łukasz Oleś wrote:
> > This is troubling:
> > # (XEN) mm.c:859:d0 Error getting mfn 40100 (pfn 5555555555555555)
from L1
> > entry 8000000040100473 for l1e_owner=0, pg_owner=32753
> 
> It''s happening durring dom0 start with and without iommu enabled.
> 
> > You need to run Linux kernel with more debugging. Use ''debug
> > loglevel=10'' to see it conjoined with Xen.
> 
> I need to recompile kernel, then I will post new logs.
> 
> > Also can you try to boot your machine _without_ using X?
> 
> Or you mean domU without XWindow? I will try.
Please do.> 
> > When is this happening?
> > # (XEN) mm.c:859:d2 Error getting mfn 40010 (pfn 5555555555555555)
from L1
> > entry 0000000040010037 for l1e_owner=2, pg_owner=2
> 
> It starts to show after xm create and stops after xm destroy when I
> try to pass-through devices. Without pci=[ <devices>] option there is
What version of Linux kernel do you have in your DomU? There was a
serious problem in 2.6.32 with the Xen fb that causes this type of
failure that was fixed at some point (don''t remember the time-frame
thought :-().

I haven''t seen this with PCI passthrough on my test-bed - but I do see
that
with DRM, which you don''t have enabled. Can you make sure you have
the latest version of xen/next from Jeremy''s git tree?
> no such error.
> 
> > At what point does this happen in the
> > guest? Can you attach the screenshot of your guest when this is
> > happening?
> 
> My vnc shows only white screen. xm console doesn''t work. I have no
> access to guest.
> 
> Without pci option guest works fine and I can connect to it.
Did you try other PCI devices and seen if they do the same
thing?> 
> Regards,
> Łukasz Oleś
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Łukasz Oleś

2010-Mar-21 22:30 UTC

head link

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Hi,

During earlier testst I used on dom0 xen/stable and on domU 2.6.32.9
from debian testing.

Then I switched to xen/next on dom0 and domU. And now I have new problem :)

Here is the full log from the boot: http://pastebin.com/e8hUeQUs
I''m still getting:
(XEN) mm.c:859:d0 Error getting mfn 40100 (pfn 5555555555555555) from
L1 entry 8000000040100473 for l1e_owner=0, pg_owner=32753

Now there are two ways:

FIRST:
as you sugested I tried pass-through another device (usb controller)

I added it to pci-stub:
root@debian:~# echo "8086 3a67" >
/sys/bus/pci/drivers/pci-stub/new_id
root@debian:~# echo
"0000:00:1a.0">/sys/bus/pci/devices/0000:00:1a.0/driver/unbind
root@debian:~# echo
"0000:00:1a.0">/sys/bus/pci/drivers/pci-stub/bind
root@debian:~# echo "8086 3a68" >
/sys/bus/pci/drivers/pci-stub/new_id
root@debian:~# echo
"0000:00:1a.1">/sys/bus/pci/devices/0000:00:1a.1/driver/unbind
root@debian:~# echo
"0000:00:1a.1">/sys/bus/pci/drivers/pci-stub/bind

on console i got:

[  253.516238] uhci_hcd 0000:00:1a.0: remove, state 4
[  253.518622] usb usb3: USB disconnect, address 1
[  253.521197] uhci_hcd 0000:00:1a.0: USB bus 3 deregistered
[  253.523799] uhci_hcd 0000:00:1a.0: PCI INT A disabled
[  269.544219] pci-stub 0000:00:1a.0: claimed by stub
[  319.424304] uhci_hcd 0000:00:1a.1: remove, state 4
[  319.427776] usb usb4: USB disconnect, address 1
[  319.430305] uhci_hcd 0000:00:1a.1: USB bus 4 deregistered
[  319.433006] uhci_hcd 0000:00:1a.1: PCI INT B disabled
[  339.692207] pci-stub 0000:00:1a.1: claimed by stub


Then I launched guest and it worked, guest could see usb controller. I
disabled guest, enabled it again, disabled it again and enabled it
again and got call trace:


(XEN) HVM1: HVM Loader
(XEN) HVM1: Detected Xen v4.0.0-rc6
(XEN) HVM1: CPU speed is 2800 MHz
(XEN) irq.c:243: Dom1 PCI link 0 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:243: Dom1 PCI link 1 changed 0 -> 10
(XEN) HVM1: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:243: Dom1 PCI link 2 changed 0 -> 11
(XEN) HVM1: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:243: Dom1 PCI link 3 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 3 routed to IRQ5
(XEN) HVM1: pci dev 01:3 INTA->IRQ10
(XEN) HVM1: pci dev 03:0 INTA->IRQ5
(XEN) HVM1: pci dev 04:0 INTA->IRQ5
(XEN) HVM1: pci dev 02:0 bar 10 size 02000000: f0000008
(XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f2000008
(XEN) HVM1: pci dev 02:0 bar 14 size 00001000: f3000000
(XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001
(XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=e0c0 np=20
(XEN) HVM1: pci dev 04:0 bar 20 size 00000020: 0000c101
(XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c121
(XEN) HVM1: Multiprocessor initialisation:
(XEN) HVM1:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM1: Writing SMBIOS tables ...
(XEN) HVM1: Loading ROMBIOS ...
(XEN) HVM1: 10556 bytes of ROMBIOS high-memory extensions:
(XEN) HVM1:   Relocating to 0xfc000000-0xfc00293c ... done
(XEN) HVM1: Creating MP tables ...
(XEN) HVM1: Loading Cirrus VGABIOS ...
(XEN) HVM1: Loading ACPI ...
(XEN) HVM1:  - Lo data: 000ea020-000ea04f
(XEN) HVM1:  - Hi data: fc002c00-fc012d1f
(XEN) HVM1: vm86 TSS at fc013000
(XEN) HVM1: BIOS map:
(XEN) HVM1:  c0000-c8fff: VGA BIOS
(XEN) HVM1:  eb000-eb148: SMBIOS tables
(XEN) HVM1:  f0000-fffff: Main BIOS
(XEN) HVM1: Invoking ROMBIOS ...
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) stdvga.c:147:d1 entering stdvga and caching modes
(XEN) HVM1: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert
Exp $
(XEN) HVM1: Bochs BIOS - build: 06/23/99
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) HVM1: Options: apmbios pcibios eltorito PMM
(XEN) HVM1:
(XEN) HVM1: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM1: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (20480 MBytes)
(XEN) HVM1: IDE time out
(XEN) HVM1: ata1 master: QEMU DVD-ROM ATAPI-4 CD-Rom/DVD-Rom
(XEN) HVM1: IDE time out
(XEN) HVM1:
(XEN) HVM1:
(XEN) HVM1:
(XEN) HVM1: Press F12 for boot menu.
(XEN) HVM1:
(XEN) HVM1: Booting from Hard Disk...
(XEN) HVM1: Booting from 0000:7c00
(XEN) stdvga.c:151:d1 leaving stdvga
(XEN) stdvga.c:147:d1 entering stdvga and caching modes
(XEN) vlapic.c:702:d1 Local APIC Write to read-only register 0x30
(XEN) vlapic.c:702:d1 Local APIC Write to read-only register 0x20
(XEN) vlapic.c:702:d1 Local APIC Write to read-only register 0x20
(XEN) domctl.c:1071:d0 ioport_map:remove f_gport=c100 f_mport=e0c0
np=20
(XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=e0c0 np=20
(XEN) irq.c:243: Dom1 PCI link 0 changed 5 -> 0
(XEN) irq.c:243: Dom1 PCI link 1 changed 10 -> 0
(XEN) irq.c:243: Dom1 PCI link 2 changed 11 -> 0
(XEN) irq.c:243: Dom1 PCI link 3 changed 5 -> 0
[  679.719530] pciback pci-2-0: 22 Couldn''t locate PCI device
(0000:00:1a.0)! perhaps already in-use?
(XEN) HVM2: HVM Loader
(XEN) HVM2: Detected Xen v4.0.0-rc6
(XEN) HVM2: CPU speed is 2800 MHz
(XEN) irq.c:243: Dom2 PCI link 0 changed 0 -> 5
(XEN) HVM2: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:243: Dom2 PCI link 1 changed 0 -> 10
(XEN) HVM2: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:243: Dom2 PCI link 2 changed 0 -> 11
(XEN) HVM2: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:243: Dom2 PCI link 3 changed 0 -> 5
(XEN) HVM2: PCI-ISA link 3 routed to IRQ5
(XEN) HVM2: pci dev 01:3 INTA->IRQ10
(XEN) HVM2: pci dev 03:0 INTA->IRQ5
(XEN) HVM2: pci dev 04:0 INTA->IRQ5
(XEN) HVM2: pci dev 02:0 bar 10 size 02000000: f0000008
(XEN) HVM2: pci dev 03:0 bar 14 size 01000000: f2000008
(XEN) HVM2: pci dev 02:0 bar 14 size 00001000: f3000000
(XEN) HVM2: pci dev 03:0 bar 10 size 00000100: 0000c001
(XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=e0c0 np=20
(XEN) HVM2: pci dev 04:0 bar 20 size 00000020: 0000c101
(XEN) HVM2: pci dev 01:1 bar 20 size 00000010: 0000c121
(XEN) HVM2: Multiprocessor initialisation:
(XEN) HVM2:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM2: Writing SMBIOS tables ...
(XEN) HVM2: Loading ROMBIOS ...
(XEN) HVM2: 10556 bytes of ROMBIOS high-memory extensions:
(XEN) HVM2:   Relocating to 0xfc000000-0xfc00293c ... done
(XEN) HVM2: Creating MP tables ...
(XEN) HVM2: Loading Cirrus VGABIOS ...
(XEN) HVM2: Loading ACPI ...
(XEN) HVM2:  - Lo data: 000ea020-000ea04f
(XEN) HVM2:  - Hi data: fc002c00-fc012d1f
(XEN) HVM2: vm86 TSS at fc013000
(XEN) HVM2: BIOS map:
(XEN) HVM2:  c0000-c8fff: VGA BIOS
(XEN) HVM2:  eb000-eb148: SMBIOS tables
(XEN) HVM2:  f0000-fffff: Main BIOS
(XEN) HVM2: Invoking ROMBIOS ...
(XEN) HVM2: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) stdvga.c:147:d2 entering stdvga and caching modes
(XEN) HVM2: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert
Exp $
(XEN) HVM2: Bochs BIOS - build: 06/23/99
(XEN) HVM2: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) HVM2: Options: apmbios pcibios eltorito PMM
(XEN) HVM2:
(XEN) HVM2: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM2: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (20480 MBytes)
(XEN) HVM2: IDE time out
(XEN) HVM2: ata1 master: QEMU DVD-ROM ATAPI-4 CD-Rom/DVD-Rom
(XEN) HVM2: IDE time out
(XEN) HVM2:
(XEN) HVM2:
(XEN) HVM2:
(XEN) HVM2: Press F12 for boot menu.
(XEN) HVM2:
(XEN) HVM2: Booting from Hard Disk...
(XEN) HVM2: Booting from 0000:7c00
(XEN) stdvga.c:151:d2 leaving stdvga
(XEN) stdvga.c:147:d2 entering stdvga and caching modes
(XEN) vlapic.c:702:d2 Local APIC Write to read-only register 0x30
(XEN) vlapic.c:702:d2 Local APIC Write to read-only register 0x20
(XEN) vlapic.c:702:d2 Local APIC Write to read-only register 0x20
(XEN) domctl.c:1071:d0 ioport_map:remove f_gport=c100 f_mport=e0c0
np=20
(XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=e0c0 np=20
(XEN) irq.c:243: Dom2 PCI link 0 changed 5 -> 0
(XEN) irq.c:243: Dom2 PCI link 1 changed 10 -> 0
(XEN) irq.c:243: Dom2 PCI link 2 changed 11 -> 0
(XEN) irq.c:243: Dom2 PCI link 3 changed 5 -> 0
[  733.248052] pciback pci-3-0: 22 Couldn''t locate PCI device
(0000:00:1a.0)! perhaps already in-use?
[  733.253535] BUG: spinlock wrong CPU on CPU#1, xenwatch/16
[  733.256210]  lock: ffff880006792208, .magic: dead4ead, .owner:
xenwatch/16, .owner_cpu: 0
[  733.257409] Pid: 16, comm: xenwatch Not tainted 2.6.32-amd64 #1
[  733.257409] Call Trace:
[  733.257409]  [<ffffffff81191b29>] ? _raw_spin_unlock+0x6e/0x82
[  733.257409]  [<ffffffff811ef98f>] ?
pciback_setup_backend+0x244/0x27e
[  733.257409]  [<ffffffff8100ecef>] ?
xen_restore_fl_direct_end+0x0/0x1
[  733.257409]  [<ffffffff811ead2d>] ? xenbus_gather+0xda/0x125
[  733.257409]  [<ffffffff812f8fcb>] ?
_spin_unlock_irqrestore+0x34/0x36
[  733.257409]  [<ffffffff811efec3>] ?
pciback_xenbus_probe+0x122/0x147
[  733.257409]  [<ffffffff811eb773>] ? xenbus_dev_probe+0x8b/0x125
[  733.257409]  [<ffffffff81221905>] ? driver_probe_device+0x9f/0x126
[  733.257409]  [<ffffffff812219fb>] ? __device_attach+0x0/0x39
[  733.257409]  [<ffffffff81220d27>] ? bus_for_each_drv+0x47/0x7b
[  733.257409]  [<ffffffff81221ab8>] ? device_attach+0x5e/0x7e
[  733.257409]  [<ffffffff81220b99>] ? bus_probe_device+0x22/0x3c
[  733.257409]  [<ffffffff8121f8c7>] ? device_add+0x372/0x519
[  733.257409]  [<ffffffff8121ec3b>] ? dev_set_name+0x3f/0x44
[  733.257409]  [<ffffffff811eba89>] ? xenbus_probe_node+0x136/0x1dd
[  733.317104]  [<ffffffff811ebc8a>] ? xenbus_dev_changed+0x15a/0x18c
[  733.317104]  [<ffffffff811ea0f3>] ? xenwatch_thread+0x11f/0x15c
[  733.317104]  [<ffffffff8105ea7c>] ?
autoremove_wake_function+0x0/0x2a
[  733.317104]  [<ffffffff811e9fd4>] ? xenwatch_thread+0x0/0x15c
[  733.317104]  [<ffffffff8105e7cf>] ? kthread+0x75/0x7d
[  733.317104]  [<ffffffff81012baa>] ? child_rip+0xa/0x20
[  733.317104]  [<ffffffff81011d61>] ? int_ret_from_sys_call+0x7/0x1b
[  733.317104]  [<ffffffff8101251d>] ? retint_restore_args+0x5/0x6
[  733.317104]  [<ffffffff81012ba0>] ? child_rip+0x0/0x20
(XEN) HVM3: HVM Loader
(XEN) HVM3: Detected Xen v4.0.0-rc6
(XEN) HVM3: CPU speed is 2800 MHz
(XEN) irq.c:243: Dom3 PCI link 0 changed 0 -> 5
(XEN) HVM3: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:243: Dom3 PCI link 1 changed 0 -> 10
(XEN) HVM3: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:243: Dom3 PCI link 2 changed 0 -> 11
(XEN) HVM3: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:243: Dom3 PCI link 3 changed 0 -> 5
(XEN) HVM3: PCI-ISA link 3 routed to IRQ5
(XEN) HVM3: pci dev 01:3 INTA->IRQ10
(XEN) HVM3: pci dev 03:0 INTA->IRQ5
(XEN) HVM3: pci dev 04:0 INTA->IRQ5
(XEN) HVM3: pci dev 02:0 bar 10 size 02000000: f0000008
(XEN) HVM3: pci dev 03:0 bar 14 size 01000000: f2000008
(XEN) HVM3: pci dev 02:0 bar 14 size 00001000: f3000000
(XEN) HVM3: pci dev 03:0 bar 10 size 00000100: 0000c001
(XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=e0c0 np=20
(XEN) HVM3: pci dev 04:0 bar 20 size 00000020: 0000c101
(XEN) HVM3: pci dev 01:1 bar 20 size 00000010: 0000c121
(XEN) HVM3: Multiprocessor initialisation:
(XEN) HVM3:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM3: Writing SMBIOS tables ...
(XEN) HVM3: Loading ROMBIOS ...
(XEN) HVM3: 10556 bytes of ROMBIOS high-memory extensions:
(XEN) HVM3:   Relocating to 0xfc000000-0xfc00293c ... done
(XEN) HVM3: Creating MP tables ...
(XEN) HVM3: Loading Cirrus VGABIOS ...
(XEN) HVM3: Loading ACPI ...
(XEN) HVM3:  - Lo data: 000ea020-000ea04f
(XEN) HVM3:  - Hi data: fc002c00-fc012d1f
(XEN) HVM3: vm86 TSS at fc013000
(XEN) HVM3: BIOS map:
(XEN) HVM3:  c0000-c8fff: VGA BIOS
(XEN) HVM3:  eb000-eb148: SMBIOS tables
(XEN) HVM3:  f0000-fffff: Main BIOS
(XEN) HVM3: Invoking ROMBIOS ...
(XEN) HVM3: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) stdvga.c:147:d3 entering stdvga and caching modes
(XEN) HVM3: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $
(XEN) HVM3: Bochs BIOS - build: 06/23/99
(XEN) HVM3: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) HVM3: Options: apmbios pcibios eltorito PMM
(XEN) HVM3:
(XEN) HVM3: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM3: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (20480 MBytes)
(XEN) HVM3: IDE time out
(XEN) HVM3: ata1 master: QEMU DVD-ROM ATAPI-4 CD-Rom/DVD-Rom
(XEN) HVM3: IDE time out
(XEN) HVM3:
(XEN) HVM3:
(XEN) HVM3:
(XEN) HVM3: Press F12 for boot menu.
(XEN) HVM3:
(XEN) HVM3: Booting from Hard Disk...
(XEN) HVM3: Booting from 0000:7c00
(XEN) stdvga.c:151:d3 leaving stdvga
(XEN) stdvga.c:147:d3 entering stdvga and caching modes
(XEN) vlapic.c:702:d3 Local APIC Write to read-only register 0x30
(XEN) vlapic.c:702:d3 Local APIC Write to read-only register 0x20
(XEN) vlapic.c:702:d3 Local APIC Write to read-only register 0x20
(XEN) domctl.c:1071:d0 ioport_map:remove f_gport=c100 f_mport=e0c0 np=20
(XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=e0c0 np=20
(XEN) irq.c:243: Dom3 PCI link 0 changed 5 -> 0
(XEN) irq.c:243: Dom3 PCI link 1 changed 10 -> 0
(XEN) irq.c:243: Dom3 PCI link 2 changed 11 -> 0
(XEN) irq.c:243: Dom3 PCI link 3 changed 5 -> 0


SECOND:

 I used FC card to pass-through(it uses ql2axxx module).

And I got:
[  188.166084] pciback pci-1-0: 22 Couldn''t locate PCI device
(0000:01:00.0)! perhaps already in-use?
(XEN) HVM1: HVM Loader
(XEN) HVM1: Detected Xen v4.0.0-rc6
(XEN) HVM1: CPU speed is 2800 MHz
(XEN) irq.c:243: Dom1 PCI link 0 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:243: Dom1 PCI link 1 changed 0 -> 10
(XEN) HVM1: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:243: Dom1 PCI link 2 changed 0 -> 11
(XEN) HVM1: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:243: Dom1 PCI link 3 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 3 routed to IRQ5
(XEN) HVM1: pci dev 01:3 INTA->IRQ10
(XEN) HVM1: pci dev 03:0 INTA->IRQ5
(XEN) HVM1: pci dev 04:0 INTA->IRQ5
(XEN) HVM1: pci dev 02:0 bar 10 size 02000000: f0000008
(XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f2000008
(XEN) HVM1: pci dev 04:0 bar 30 size 00020000: f3000000
(XEN) HVM1: pci dev 02:0 bar 14 size 00001000: f3020000
(XEN) domctl.c:991:d0 memory_map:add: gfn=f3021 mfn=fe921 nr_mfns=1
(XEN) HVM1: pci dev 04:0 bar 14 size 00001000: f3021000
(XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001
(XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=d000 np=100
(XEN) HVM1: pci dev 04:0 bar 10 size 00000100: 0000c101
(XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c201
(XEN) HVM1: Multiprocessor initialisation:
(XEN) HVM1:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM1: Writing SMBIOS tables ...
(XEN) HVM1: Loading ROMBIOS ...
(XEN) HVM1: 10556 bytes of ROMBIOS high-memory extensions:
(XEN) HVM1:   Relocating to 0xfc000000-0xfc00293c ... done
(XEN) HVM1: Creating MP tables ...
(XEN) HVM1: Loading Cirrus VGABIOS ...
(XEN) domctl.c:991:d0 memory_map:add: gfn=f3000 mfn=40000 nr_mfns=20
(XEN) mm.c:859:d1 Error getting mfn 40000 (pfn 5555555555555555) from
L1 entry 0000000040000037 for l1e_owner=1, pg_owner=1
(XEN) mm.c:859:d1 Error getting mfn 40001 (pfn 5555555555555555) from
L1 entry 0000000040001037 for l1e_owner=1, pg_owner=1
(XEN) mm.c:859:d1 Error getting mfn 40002 (pfn 5555555555555555) from
L1 entry 0000000040002037 for l1e_owner=1, pg_owner=1
(XEN) mm.c:859:d1 Error getting mfn 40003 (pfn 5555555555555555) from
L1 entry 0000000040003037 for l1e_owner=1, pg_owner=1
(XEN) mm.c:859:d1 Error getting mfn 40004 (pfn 5555555555555555) from
L1 entry 0000000040004037 for l1e_owner=1, pg_owner=1
...


And guest never starts.  Perhaps this is related to the firmware which
is used by qla2xxx?

Regards,
Łukasz Oleś

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Mar-22 19:50 UTC

head link

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

On Sun, Mar 21, 2010 at 11:30:57PM +0100, Łukasz Oleś
wrote:> Hi,
> 
> During earlier testst I used on dom0 xen/stable and on domU 2.6.32.9
> from debian testing.
> 
> Then I switched to xen/next on dom0 and domU. And now I have new problem :)
What is the version of Xen tools you have? Did you build them from
xen-unstable?
> 
> Here is the full log from the boot: http://pastebin.com/e8hUeQUs
> I''m still getting:
> (XEN) mm.c:859:d0 Error getting mfn 40100 (pfn 5555555555555555) from
> L1 entry 8000000040100473 for l1e_owner=0, pg_owner=32753
> 
> Now there are two ways:
> 
> FIRST:
> as you sugested I tried pass-through another device (usb controller)
> 
> I added it to pci-stub:
> root@debian:~# echo "8086 3a67" >
/sys/bus/pci/drivers/pci-stub/new_id
> root@debian:~# echo
> "0000:00:1a.0">/sys/bus/pci/devices/0000:00:1a.0/driver/unbind
> root@debian:~# echo
"0000:00:1a.0">/sys/bus/pci/drivers/pci-stub/bind
> root@debian:~# echo "8086 3a68" >
/sys/bus/pci/drivers/pci-stub/new_id
> root@debian:~# echo
> "0000:00:1a.1">/sys/bus/pci/devices/0000:00:1a.1/driver/unbind
> root@debian:~# echo
"0000:00:1a.1">/sys/bus/pci/drivers/pci-stub/bind
Where did you get the documentation for those steps? I would suggest you use
the ''pciback'' instead of ''pci-stub'' module
for this just to see if that
makes the ''pciback'' later error go
away.> 
> on console i got:
> 
> [  253.516238] uhci_hcd 0000:00:1a.0: remove, state 4
> [  253.518622] usb usb3: USB disconnect, address 1
> [  253.521197] uhci_hcd 0000:00:1a.0: USB bus 3 deregistered
> [  253.523799] uhci_hcd 0000:00:1a.0: PCI INT A disabled
> [  269.544219] pci-stub 0000:00:1a.0: claimed by stub
> [  319.424304] uhci_hcd 0000:00:1a.1: remove, state 4
> [  319.427776] usb usb4: USB disconnect, address 1
> [  319.430305] uhci_hcd 0000:00:1a.1: USB bus 4 deregistered
> [  319.433006] uhci_hcd 0000:00:1a.1: PCI INT B disabled
> [  339.692207] pci-stub 0000:00:1a.1: claimed by stub
> 
> 
> Then I launched guest and it worked, guest could see usb controller. I
> disabled guest, enabled it again, disabled it again and enabled it
> again and got call trace:
> 
> 
> (XEN) HVM1: HVM Loader
> (XEN) HVM1: Detected Xen v4.0.0-rc6
> (XEN) HVM1: CPU speed is 2800 MHz
> (XEN) irq.c:243: Dom1 PCI link 0 changed 0 -> 5
> (XEN) HVM1: PCI-ISA link 0 routed to IRQ5
> (XEN) irq.c:243: Dom1 PCI link 1 changed 0 -> 10
> (XEN) HVM1: PCI-ISA link 1 routed to IRQ10
> (XEN) irq.c:243: Dom1 PCI link 2 changed 0 -> 11
> (XEN) HVM1: PCI-ISA link 2 routed to IRQ11
> (XEN) irq.c:243: Dom1 PCI link 3 changed 0 -> 5
> (XEN) HVM1: PCI-ISA link 3 routed to IRQ5
> (XEN) HVM1: pci dev 01:3 INTA->IRQ10
> (XEN) HVM1: pci dev 03:0 INTA->IRQ5
> (XEN) HVM1: pci dev 04:0 INTA->IRQ5
> (XEN) HVM1: pci dev 02:0 bar 10 size 02000000: f0000008
> (XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f2000008
> (XEN) HVM1: pci dev 02:0 bar 14 size 00001000: f3000000
> (XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001
> (XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=e0c0 np=20
> (XEN) HVM1: pci dev 04:0 bar 20 size 00000020: 0000c101
> (XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c121
> (XEN) HVM1: Multiprocessor initialisation:
> (XEN) HVM1:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs
> [2/8] ... done.
> (XEN) HVM1: Writing SMBIOS tables ...
> (XEN) HVM1: Loading ROMBIOS ...
> (XEN) HVM1: 10556 bytes of ROMBIOS high-memory extensions:
> (XEN) HVM1:   Relocating to 0xfc000000-0xfc00293c ... done
> (XEN) HVM1: Creating MP tables ...
> (XEN) HVM1: Loading Cirrus VGABIOS ...
> (XEN) HVM1: Loading ACPI ...
> (XEN) HVM1:  - Lo data: 000ea020-000ea04f
> (XEN) HVM1:  - Hi data: fc002c00-fc012d1f
> (XEN) HVM1: vm86 TSS at fc013000
> (XEN) HVM1: BIOS map:
> (XEN) HVM1:  c0000-c8fff: VGA BIOS
> (XEN) HVM1:  eb000-eb148: SMBIOS tables
> (XEN) HVM1:  f0000-fffff: Main BIOS
> (XEN) HVM1: Invoking ROMBIOS ...
> (XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
> (XEN) stdvga.c:147:d1 entering stdvga and caching modes
> (XEN) HVM1: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert
> Exp $
> (XEN) HVM1: Bochs BIOS - build: 06/23/99
> (XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
> (XEN) HVM1: Options: apmbios pcibios eltorito PMM
> (XEN) HVM1:
> (XEN) HVM1: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
> (XEN) HVM1: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (20480 MBytes)
> (XEN) HVM1: IDE time out
> (XEN) HVM1: ata1 master: QEMU DVD-ROM ATAPI-4 CD-Rom/DVD-Rom
> (XEN) HVM1: IDE time out
> (XEN) HVM1:
> (XEN) HVM1:
> (XEN) HVM1:
> (XEN) HVM1: Press F12 for boot menu.
> (XEN) HVM1:
> (XEN) HVM1: Booting from Hard Disk...
> (XEN) HVM1: Booting from 0000:7c00
> (XEN) stdvga.c:151:d1 leaving stdvga
> (XEN) stdvga.c:147:d1 entering stdvga and caching modes
> (XEN) vlapic.c:702:d1 Local APIC Write to read-only register 0x30
> (XEN) vlapic.c:702:d1 Local APIC Write to read-only register 0x20
> (XEN) vlapic.c:702:d1 Local APIC Write to read-only register 0x20
> (XEN) domctl.c:1071:d0 ioport_map:remove f_gport=c100 f_mport=e0c0
> np=20
> (XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=e0c0 np=20
> (XEN) irq.c:243: Dom1 PCI link 0 changed 5 -> 0
> (XEN) irq.c:243: Dom1 PCI link 1 changed 10 -> 0
> (XEN) irq.c:243: Dom1 PCI link 2 changed 11 -> 0
> (XEN) irq.c:243: Dom1 PCI link 3 changed 5 -> 0
> [  679.719530] pciback pci-2-0: 22 Couldn''t locate PCI device
> (0000:00:1a.0)! perhaps already in-use?

Uhh. So you assigned the device to ''pci-stub'', and somehow
''pciback'' gets
involved. That is not good. PCI back is to be used _only_ with
PV guests - you on the other hand are running an HVM guest.

Per the earlier statement, I would recommend you use the
''pciback''
instead of ''pci-stub'' or just not compile pciback in and see
what
happens. Keep in mind: PCI back module is only needed when you want to do PV PCI
passthrough, which is not what you are doing.


> (XEN) HVM2: HVM Loader
> (XEN) HVM2: Detected Xen v4.0.0-rc6
> (XEN) HVM2: CPU speed is 2800 MHz
> (XEN) irq.c:243: Dom2 PCI link 0 changed 0 -> 5
> (XEN) HVM2: PCI-ISA link 0 routed to IRQ5
> (XEN) irq.c:243: Dom2 PCI link 1 changed 0 -> 10
> (XEN) HVM2: PCI-ISA link 1 routed to IRQ10
> (XEN) irq.c:243: Dom2 PCI link 2 changed 0 -> 11
> (XEN) HVM2: PCI-ISA link 2 routed to IRQ11
> (XEN) irq.c:243: Dom2 PCI link 3 changed 0 -> 5
> (XEN) HVM2: PCI-ISA link 3 routed to IRQ5
> (XEN) HVM2: pci dev 01:3 INTA->IRQ10
> (XEN) HVM2: pci dev 03:0 INTA->IRQ5
> (XEN) HVM2: pci dev 04:0 INTA->IRQ5
> (XEN) HVM2: pci dev 02:0 bar 10 size 02000000: f0000008
> (XEN) HVM2: pci dev 03:0 bar 14 size 01000000: f2000008
> (XEN) HVM2: pci dev 02:0 bar 14 size 00001000: f3000000
> (XEN) HVM2: pci dev 03:0 bar 10 size 00000100: 0000c001
> (XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=e0c0 np=20
> (XEN) HVM2: pci dev 04:0 bar 20 size 00000020: 0000c101
> (XEN) HVM2: pci dev 01:1 bar 20 size 00000010: 0000c121
> (XEN) HVM2: Multiprocessor initialisation:
> (XEN) HVM2:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs
> [2/8] ... done.
> (XEN) HVM2: Writing SMBIOS tables ...
> (XEN) HVM2: Loading ROMBIOS ...
> (XEN) HVM2: 10556 bytes of ROMBIOS high-memory extensions:
> (XEN) HVM2:   Relocating to 0xfc000000-0xfc00293c ... done
> (XEN) HVM2: Creating MP tables ...
> (XEN) HVM2: Loading Cirrus VGABIOS ...
> (XEN) HVM2: Loading ACPI ...
> (XEN) HVM2:  - Lo data: 000ea020-000ea04f
> (XEN) HVM2:  - Hi data: fc002c00-fc012d1f
> (XEN) HVM2: vm86 TSS at fc013000
> (XEN) HVM2: BIOS map:
> (XEN) HVM2:  c0000-c8fff: VGA BIOS
> (XEN) HVM2:  eb000-eb148: SMBIOS tables
> (XEN) HVM2:  f0000-fffff: Main BIOS
> (XEN) HVM2: Invoking ROMBIOS ...
> (XEN) HVM2: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
> (XEN) stdvga.c:147:d2 entering stdvga and caching modes
> (XEN) HVM2: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert
> Exp $
> (XEN) HVM2: Bochs BIOS - build: 06/23/99
> (XEN) HVM2: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
> (XEN) HVM2: Options: apmbios pcibios eltorito PMM
> (XEN) HVM2:
> (XEN) HVM2: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
> (XEN) HVM2: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (20480 MBytes)
> (XEN) HVM2: IDE time out
> (XEN) HVM2: ata1 master: QEMU DVD-ROM ATAPI-4 CD-Rom/DVD-Rom
> (XEN) HVM2: IDE time out
> (XEN) HVM2:
> (XEN) HVM2:
> (XEN) HVM2:
> (XEN) HVM2: Press F12 for boot menu.
> (XEN) HVM2:
> (XEN) HVM2: Booting from Hard Disk...
> (XEN) HVM2: Booting from 0000:7c00
> (XEN) stdvga.c:151:d2 leaving stdvga
> (XEN) stdvga.c:147:d2 entering stdvga and caching modes
> (XEN) vlapic.c:702:d2 Local APIC Write to read-only register 0x30
> (XEN) vlapic.c:702:d2 Local APIC Write to read-only register 0x20
> (XEN) vlapic.c:702:d2 Local APIC Write to read-only register 0x20
> (XEN) domctl.c:1071:d0 ioport_map:remove f_gport=c100 f_mport=e0c0
> np=20
> (XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=e0c0 np=20
> (XEN) irq.c:243: Dom2 PCI link 0 changed 5 -> 0
> (XEN) irq.c:243: Dom2 PCI link 1 changed 10 -> 0
> (XEN) irq.c:243: Dom2 PCI link 2 changed 11 -> 0
> (XEN) irq.c:243: Dom2 PCI link 3 changed 5 -> 0
> [  733.248052] pciback pci-3-0: 22 Couldn''t locate PCI device
> (0000:00:1a.0)! perhaps already in-use?
Can you paste your ''xenstore-ls /local/domain/0'' output when
this
happening?
> [  733.253535] BUG: spinlock wrong CPU on CPU#1, xenwatch/16
> [  733.256210]  lock: ffff880006792208, .magic: dead4ead, .owner:
> xenwatch/16, .owner_cpu: 0
> [  733.257409] Pid: 16, comm: xenwatch Not tainted 2.6.32-amd64 #1
> [  733.257409] Call Trace:
> [  733.257409]  [<ffffffff81191b29>] ? _raw_spin_unlock+0x6e/0x82
> [  733.257409]  [<ffffffff811ef98f>] ?
> pciback_setup_backend+0x244/0x27e
> [  733.257409]  [<ffffffff8100ecef>] ?
> xen_restore_fl_direct_end+0x0/0x1
> [  733.257409]  [<ffffffff811ead2d>] ? xenbus_gather+0xda/0x125
> [  733.257409]  [<ffffffff812f8fcb>] ?
> _spin_unlock_irqrestore+0x34/0x36
> [  733.257409]  [<ffffffff811efec3>] ?
> pciback_xenbus_probe+0x122/0x147
> [  733.257409]  [<ffffffff811eb773>] ? xenbus_dev_probe+0x8b/0x125
> [  733.257409]  [<ffffffff81221905>] ? driver_probe_device+0x9f/0x126
> [  733.257409]  [<ffffffff812219fb>] ? __device_attach+0x0/0x39
> [  733.257409]  [<ffffffff81220d27>] ? bus_for_each_drv+0x47/0x7b
> [  733.257409]  [<ffffffff81221ab8>] ? device_attach+0x5e/0x7e
> [  733.257409]  [<ffffffff81220b99>] ? bus_probe_device+0x22/0x3c
> [  733.257409]  [<ffffffff8121f8c7>] ? device_add+0x372/0x519
> [  733.257409]  [<ffffffff8121ec3b>] ? dev_set_name+0x3f/0x44
> [  733.257409]  [<ffffffff811eba89>] ? xenbus_probe_node+0x136/0x1dd
> [  733.317104]  [<ffffffff811ebc8a>] ? xenbus_dev_changed+0x15a/0x18c
> [  733.317104]  [<ffffffff811ea0f3>] ? xenwatch_thread+0x11f/0x15c
> [  733.317104]  [<ffffffff8105ea7c>] ?
> autoremove_wake_function+0x0/0x2a
> [  733.317104]  [<ffffffff811e9fd4>] ? xenwatch_thread+0x0/0x15c
> [  733.317104]  [<ffffffff8105e7cf>] ? kthread+0x75/0x7d
> [  733.317104]  [<ffffffff81012baa>] ? child_rip+0xa/0x20
> [  733.317104]  [<ffffffff81011d61>] ? int_ret_from_sys_call+0x7/0x1b
> [  733.317104]  [<ffffffff8101251d>] ? retint_restore_args+0x5/0x6
> [  733.317104]  [<ffffffff81012ba0>] ? child_rip+0x0/0x20
> (XEN) HVM3: HVM Loader
> (XEN) HVM3: Detected Xen v4.0.0-rc6
> (XEN) HVM3: CPU speed is 2800 MHz
> (XEN) irq.c:243: Dom3 PCI link 0 changed 0 -> 5
> (XEN) HVM3: PCI-ISA link 0 routed to IRQ5
> (XEN) irq.c:243: Dom3 PCI link 1 changed 0 -> 10
> (XEN) HVM3: PCI-ISA link 1 routed to IRQ10
> (XEN) irq.c:243: Dom3 PCI link 2 changed 0 -> 11
> (XEN) HVM3: PCI-ISA link 2 routed to IRQ11
> (XEN) irq.c:243: Dom3 PCI link 3 changed 0 -> 5
> (XEN) HVM3: PCI-ISA link 3 routed to IRQ5
> (XEN) HVM3: pci dev 01:3 INTA->IRQ10
> (XEN) HVM3: pci dev 03:0 INTA->IRQ5
> (XEN) HVM3: pci dev 04:0 INTA->IRQ5
> (XEN) HVM3: pci dev 02:0 bar 10 size 02000000: f0000008
> (XEN) HVM3: pci dev 03:0 bar 14 size 01000000: f2000008
> (XEN) HVM3: pci dev 02:0 bar 14 size 00001000: f3000000
> (XEN) HVM3: pci dev 03:0 bar 10 size 00000100: 0000c001
> (XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=e0c0 np=20
> (XEN) HVM3: pci dev 04:0 bar 20 size 00000020: 0000c101
> (XEN) HVM3: pci dev 01:1 bar 20 size 00000010: 0000c121
> (XEN) HVM3: Multiprocessor initialisation:
> (XEN) HVM3:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs
> [2/8] ... done.
> (XEN) HVM3: Writing SMBIOS tables ...
> (XEN) HVM3: Loading ROMBIOS ...
> (XEN) HVM3: 10556 bytes of ROMBIOS high-memory extensions:
> (XEN) HVM3:   Relocating to 0xfc000000-0xfc00293c ... done
> (XEN) HVM3: Creating MP tables ...
> (XEN) HVM3: Loading Cirrus VGABIOS ...
> (XEN) HVM3: Loading ACPI ...
> (XEN) HVM3:  - Lo data: 000ea020-000ea04f
> (XEN) HVM3:  - Hi data: fc002c00-fc012d1f
> (XEN) HVM3: vm86 TSS at fc013000
> (XEN) HVM3: BIOS map:
> (XEN) HVM3:  c0000-c8fff: VGA BIOS
> (XEN) HVM3:  eb000-eb148: SMBIOS tables
> (XEN) HVM3:  f0000-fffff: Main BIOS
> (XEN) HVM3: Invoking ROMBIOS ...
> (XEN) HVM3: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
> (XEN) stdvga.c:147:d3 entering stdvga and caching modes
> (XEN) HVM3: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp
$
> (XEN) HVM3: Bochs BIOS - build: 06/23/99
> (XEN) HVM3: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
> (XEN) HVM3: Options: apmbios pcibios eltorito PMM
> (XEN) HVM3:
> (XEN) HVM3: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
> (XEN) HVM3: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (20480 MBytes)
> (XEN) HVM3: IDE time out
> (XEN) HVM3: ata1 master: QEMU DVD-ROM ATAPI-4 CD-Rom/DVD-Rom
> (XEN) HVM3: IDE time out
> (XEN) HVM3:
> (XEN) HVM3:
> (XEN) HVM3:
> (XEN) HVM3: Press F12 for boot menu.
> (XEN) HVM3:
> (XEN) HVM3: Booting from Hard Disk...
> (XEN) HVM3: Booting from 0000:7c00
> (XEN) stdvga.c:151:d3 leaving stdvga
> (XEN) stdvga.c:147:d3 entering stdvga and caching modes
> (XEN) vlapic.c:702:d3 Local APIC Write to read-only register 0x30
> (XEN) vlapic.c:702:d3 Local APIC Write to read-only register 0x20
> (XEN) vlapic.c:702:d3 Local APIC Write to read-only register 0x20
> (XEN) domctl.c:1071:d0 ioport_map:remove f_gport=c100 f_mport=e0c0 np=20
> (XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=e0c0 np=20
> (XEN) irq.c:243: Dom3 PCI link 0 changed 5 -> 0
> (XEN) irq.c:243: Dom3 PCI link 1 changed 10 -> 0
> (XEN) irq.c:243: Dom3 PCI link 2 changed 11 -> 0
> (XEN) irq.c:243: Dom3 PCI link 3 changed 5 -> 0
> 
> 
> SECOND:
> 
>  I used FC card to pass-through(it uses ql2axxx module).
> 
> And I got:
> [  188.166084] pciback pci-1-0: 22 Couldn''t locate PCI device
> (0000:01:00.0)! perhaps already in-use?
> (XEN) HVM1: HVM Loader
> (XEN) HVM1: Detected Xen v4.0.0-rc6
> (XEN) HVM1: CPU speed is 2800 MHz
> (XEN) irq.c:243: Dom1 PCI link 0 changed 0 -> 5
> (XEN) HVM1: PCI-ISA link 0 routed to IRQ5
> (XEN) irq.c:243: Dom1 PCI link 1 changed 0 -> 10
> (XEN) HVM1: PCI-ISA link 1 routed to IRQ10
> (XEN) irq.c:243: Dom1 PCI link 2 changed 0 -> 11
> (XEN) HVM1: PCI-ISA link 2 routed to IRQ11
> (XEN) irq.c:243: Dom1 PCI link 3 changed 0 -> 5
> (XEN) HVM1: PCI-ISA link 3 routed to IRQ5
> (XEN) HVM1: pci dev 01:3 INTA->IRQ10
> (XEN) HVM1: pci dev 03:0 INTA->IRQ5
> (XEN) HVM1: pci dev 04:0 INTA->IRQ5
> (XEN) HVM1: pci dev 02:0 bar 10 size 02000000: f0000008
> (XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f2000008
> (XEN) HVM1: pci dev 04:0 bar 30 size 00020000: f3000000
> (XEN) HVM1: pci dev 02:0 bar 14 size 00001000: f3020000
> (XEN) domctl.c:991:d0 memory_map:add: gfn=f3021 mfn=fe921 nr_mfns=1
> (XEN) HVM1: pci dev 04:0 bar 14 size 00001000: f3021000
> (XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001
> (XEN) domctl.c:1047:d0 ioport_map:add f_gport=c100 f_mport=d000 np=100
> (XEN) HVM1: pci dev 04:0 bar 10 size 00000100: 0000c101
> (XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c201
> (XEN) HVM1: Multiprocessor initialisation:
> (XEN) HVM1:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs
> [2/8] ... done.
> (XEN) HVM1: Writing SMBIOS tables ...
> (XEN) HVM1: Loading ROMBIOS ...
> (XEN) HVM1: 10556 bytes of ROMBIOS high-memory extensions:
> (XEN) HVM1:   Relocating to 0xfc000000-0xfc00293c ... done
> (XEN) HVM1: Creating MP tables ...
> (XEN) HVM1: Loading Cirrus VGABIOS ...
> (XEN) domctl.c:991:d0 memory_map:add: gfn=f3000 mfn=40000 nr_mfns=20
> (XEN) mm.c:859:d1 Error getting mfn 40000 (pfn 5555555555555555) from
> L1 entry 0000000040000037 for l1e_owner=1, pg_owner=1
> (XEN) mm.c:859:d1 Error getting mfn 40001 (pfn 5555555555555555) from
> L1 entry 0000000040001037 for l1e_owner=1, pg_owner=1
> (XEN) mm.c:859:d1 Error getting mfn 40002 (pfn 5555555555555555) from
> L1 entry 0000000040002037 for l1e_owner=1, pg_owner=1
> (XEN) mm.c:859:d1 Error getting mfn 40003 (pfn 5555555555555555) from
> L1 entry 0000000040003037 for l1e_owner=1, pg_owner=1
> (XEN) mm.c:859:d1 Error getting mfn 40004 (pfn 5555555555555555) from
> L1 entry 0000000040004037 for l1e_owner=1, pg_owner=1
> ...
> 
> 
> And guest never starts.  Perhaps this is related to the firmware which
> is used by qla2xxx?
So you see this _only_ when you pass in QLA2XXX card, right? When you
don''t pass in a PCI card, you don''t see this, correct?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Mar-22 20:54 UTC

head link

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

On Mon, Mar 22, 2010 at 03:06:48PM -0600, Nadolski, Ed
wrote:> 
> 
> > -----Original Message-----
>  
> > PCI back is to be used _only_ with PV guests - you on the other
> > hand are running an HVM guest.
> > 
> > Per the earlier statement, I would recommend you use the
''pciback''
> > instead of ''pci-stub'' or just not compile pciback in
and see what
> > happens. Keep in mind: PCI back module is only needed when you want to
> > do PV PCI passthrough, which is not what you are doing.
> 
> I''m confused - does that mean pci-stub must be used for device
assignment to an HVM guest?  The VTdHowTo isn''t clear on that.
Not per say.

xen-pciback can be used for both PV and HVM.
pci-stub can only be used for HVM guests.

But there seems to be a bug somewhere that when the PCI device is
assigned to pci-stub, pciback tries to seize it and can''t find it and
somehow is stuck in a spin-lock.  That shouldn''t be happening.

Right now I am trying to figure out if we remove from Lukasza system
pciback and only use pci-stub whether he still gets those MFN lookup
errors with his QLogic card. Those are, I believe, a seperate issue from
the pciback spinlock failure.
> 
> Is there a way to bind the PCI devices to pci-stub at boot time, like with
pciback?
No.> Also, do the ''xm pci-list-assignable-devices'' and
''xm pci-list'' commands work the same for both pciback and
pci-stub bindings?
Yes.> 
> It seems odd to have different ways to do bindings for PV and for HVM. For
example, would I have to switch the bindings if I want to first assign a device
to a PV guest, then stop the PV guest and re-assign the device to an HVM guest?
No if you use ''xen-pciback''. Yes if you use
''pci-stub''.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Nadolski, Ed

2010-Mar-22 21:06 UTC

head link

RE: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

> -----Original Message-----
 > PCI back is to be used _only_ with PV guests - you on the other
> hand are running an HVM guest.
> 
> Per the earlier statement, I would recommend you use the 'pciback'
> instead of 'pci-stub' or just not compile pciback in and see what
> happens. Keep in mind: PCI back module is only needed when you want to
> do PV PCI passthrough, which is not what you are doing.
I'm confused - does that mean pci-stub must be used for device assignment to
an HVM guest?  The VTdHowTo isn't clear on that.

Is there a way to bind the PCI devices to pci-stub at boot time, like with
pciback?  Also, do the 'xm pci-list-assignable-devices' and 'xm
pci-list' commands work the same for both pciback and pci-stub bindings?

It seems odd to have different ways to do bindings for PV and for HVM. For
example, would I have to switch the bindings if I want to first assign a device
to a PV guest, then stop the PV guest and re-assign the device to an HVM guest?

Thanks,
Ed







_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Mar-22 21:54 UTC

head link

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

> >>
"0000:00:1a.1">/sys/bus/pci/devices/0000:00:1a.1/driver/unbind
> >> root@debian:~# echo
"0000:00:1a.1">/sys/bus/pci/drivers/pci-stub/bind
> >
> > Where did you get the documentation for those steps? I would suggest
you use
> > the ''pciback'' instead of
''pci-stub'' module for this just to see if that
> > makes the ''pciback'' later error go away.
> 
> Well, I missread http://wiki.xensource.com/xenwiki/VTdHowTo  I
> switched to pciback(I used
>
http://wiki.xensource.com/xenwiki/Assign_hardware_to_DomU_with_PCIBack_as_module)
>  and this error no longer occurs.
.. snip ..> >> (XEN) mm.c:859:d1 Error getting mfn 40004 (pfn 5555555555555555)
from
> >> L1 entry 0000000040004037 for l1e_owner=1, pg_owner=1
> >> ...
> >> And guest never starts.  Perhaps this is related to the firmware
which
> >> is used by qla2xxx?
> >
> > So you see this _only_ when you pass in QLA2XXX card, right? When you
> > don''t pass in a PCI card, you don''t see this,
correct?
> 
> Exactly. This happens with both pciback and pci-stub.
You are hitting two bugs.

1). pciback doesn''t release the spinlock properly, spinning forever and
causing havoc when pcistub is used for the device.

2). qemu/vt-d/bochs bios/hvmload isn''t setting up the HVM PCI
passthrough
properly for your QLogic card.

For the 1) there isn''t much there except to find the culprit and fix
it,
should be pretty easy - probably some mutex/dead-lock condition. But
there is a work-around which you found, and that is to use _only_ pciback for
HVM and PV guests.

For the 2) problem I''ve no idea. I haven''t dug through the
QEMU PCI
passthrough code that much to figure this out. Depending on your needs
we can fix this the easy or the hard way. The easy way is for you to try
PV pass-through instead of HVM pass-through. It requires taking your
kernel and initramfs image from your DomU, and referencing them direclty
in your .cfg file. There should be an example of this in /etc/xen (look
for pv-grub). Also you might need to pass in ''iommu=0'' on your
Xen command line.

The hard way is to try to figure this one out. There are four pieces of
technology working here: IOMMU (VT-d chipset), the QEMU
PCI pass-through code, the hvmloader code, and the Bochs BIOS. First step is
to instrument your QEMU to print all manners of debug information and do the
same with
the hvmloader (grep for ''debug'').  Also adding in
''loglvl=all'' on your Xen
command line is a good idea (and compiling Xen with
''debug=y''). For the
Bochs BIOS, there are some options in it where you can tweak it to print
more details.

Oh, and also would need to instrument DomU to figure out when that
''Error getting mfn'' is triggered. You can do that easily by
using
''xenctx'' and seeing the stack trace of the guest (And with
your
System.map table pin-point what is happening).


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Łukasz Oleś

2010-Mar-22 21:56 UTC

head link

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Hi,

2010/3/22 Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com>:> What is the version of Xen tools you have? Did you build them from
> xen-unstable?
Yes, there are from xen-unstable
>>
"0000:00:1a.1">/sys/bus/pci/devices/0000:00:1a.1/driver/unbind
>> root@debian:~# echo
"0000:00:1a.1">/sys/bus/pci/drivers/pci-stub/bind
>
> Where did you get the documentation for those steps? I would suggest you
use
> the ''pciback'' instead of ''pci-stub''
module for this just to see if that
> makes the ''pciback'' later error go away.
Well, I missread http://wiki.xensource.com/xenwiki/VTdHowTo  I
switched to pciback(I used
http://wiki.xensource.com/xenwiki/Assign_hardware_to_DomU_with_PCIBack_as_module)
 and this error no longer occurs.
> Uhh. So you assigned the device to ''pci-stub'', and
somehow ''pciback'' gets
> involved. That is not good. PCI back is to be used _only_ with
> PV guests - you on the other hand are running an HVM guest.
>
> Per the earlier statement, I would recommend you use the
''pciback''
> instead of ''pci-stub'' or just not compile pciback in and
see what
> happens. Keep in mind: PCI back module is only needed when you want to do
PV PCI
> passthrough, which is not what you are doing.
>
Wiki says something else (http://wiki.xensource.com/xenwiki/VTdHowTo)
So pciback or pci-stub for HVM?
from wiki: "If using pv-ops dom0, also can use pci-stub to hide
devices for assignment"
>> [  733.248052] pciback pci-3-0: 22 Couldn''t locate PCI device
>> (0000:00:1a.0)! perhaps already in-use?
>
> Can you paste your ''xenstore-ls /local/domain/0'' output
when this
> happening?
When using pci-stub:

xenstore-ls /local/domain/0
vm = "/vm/00000000-0000-0000-0000-000000000000"
device = ""
control = ""
 platform-feature-multiprocessor-suspend = "1"
error = ""
 backend = ""
  pci = ""
   1 = ""
    0 = ""
     error = "22 Couldn''t locate PCI device (0000:01:01.0)!
perhaps
already in-use?"
memory = ""
 target = "1017632"
guest = ""
hvmpv = ""
data = ""
cpu = ""
 1 = ""
  availability = "online"
 0 = ""
  availability = "online"
description = ""
console = ""
 limit = "1048576"
 type = "xenconsoled"
domid = "0"
name = "Domain-0"
device-model = ""
 1 = ""
  disable_pf = "0"
  state = "running"
  parameter = "0x20"
backend = ""
 vfb = ""
  1 = ""
   0 = ""
    vncunused = "1"
    domain = "ExampleHVMDomain"
    vnc = "1"
    uuid = "e4f967e1-9ab9-6170-9392-711b7eee638d"
    vnclisten = "0.0.0.0"
    frontend = "/local/domain/1/device/vfb/0"
    state = "1"
    online = "1"
    frontend-id = "1"
    location = "0.0.0.0:5900"
 vbd = ""
  1 = ""
   768 = ""
    domain = "ExampleHVMDomain"
    frontend = "/local/domain/1/device/vbd/768"
    uuid = "f51680ce-7057-ed25-3a4f-aacd78e5f2b5"
    bootable = "1"
    dev = "hda"
    state = "2"
    params = "/media/debian.img"
    mode = "w"
    online = "1"
    frontend-id = "1"
    type = "file"
    node = "/dev/loop1"
    physical-device = "7:1"
    hotplug-status = "connected"
   5632 = ""
    domain = "ExampleHVMDomain"
    frontend = "/local/domain/1/device/vbd/5632"
    uuid = "44367e07-409f-d676-9047-d84b1f2189e2"
    bootable = "0"
    dev = "hdc"
    state = "2"
    params = "/media/debian.iso"
    mode = "r"
    online = "1"
    frontend-id = "1"
    type = "file"
    node = "/dev/loop2"
    physical-device = "7:2"
    hotplug-status = "connected"
 pci = ""
  1 = ""
   0 = ""
    domain = "ExampleHVMDomain"
    key-0 = "01:01.0"
    vdevfn-0 = "20"
    uuid = "01ad7a64-bf43-b782-b4ad-bbcb9060c36a"
    dev-0 = "0000:01:01.0"
    frontend = "/local/domain/1/device/pci/0"
    state = "7"
    online = "1"
    frontend-id = "1"
    opts-0 = "msitranslate=1,power_mgmt=0"
    uuid-0 = "9db6e99f-31c4-8a7a-96c4-dd88e130abbe"
    num_devs = "1"
    state-0 = "1"
 console = ""
  1 = ""
   0 = ""
    domain = "ExampleHVMDomain"
    protocol = "vt100"
    uuid = "ae8d643f-cd4d-2256-2f55-c79736f96311"
    frontend = "/local/domain/1/device/console/0"
    state = "1"
    location = "3"
    online = "1"
    frontend-id = "1"
>> (XEN) mm.c:859:d1 Error getting mfn 40004 (pfn 5555555555555555) from
>> L1 entry 0000000040004037 for l1e_owner=1, pg_owner=1
>> ...
>> And guest never starts.  Perhaps this is related to the firmware which
>> is used by qla2xxx?
>
> So you see this _only_ when you pass in QLA2XXX card, right? When you
> don''t pass in a PCI card, you don''t see this, correct?
Exactly. This happens with both pciback and pci-stub.

Regards,
Łukasz Oleś

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Mar-22 22:05 UTC

head link

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

On Mon, Mar 22, 2010 at 04:11:34PM -0600, Nadolski, Ed
wrote:> > -----Original Message-----
> > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> > Sent: Monday, March 22, 2010 2:54 PM
> > To: Nadolski, Ed
> > Cc: Łukasz Oleś; xen-devel@lists.xensource.com
> > Subject: Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup
> > 
> > On Mon, Mar 22, 2010 at 03:06:48PM -0600, Nadolski, Ed wrote:
> > >
> > >
> > > > -----Original Message-----
> > >
> > > > PCI back is to be used _only_ with PV guests - you on the
other
> > > > hand are running an HVM guest.
> > > >
> > > > Per the earlier statement, I would recommend you use the
''pciback''
> > > > instead of ''pci-stub'' or just not compile
pciback in and see what
> > > > happens. Keep in mind: PCI back module is only needed when
you want
> > to
> > > > do PV PCI passthrough, which is not what you are doing.
> > >
> > > I''m confused - does that mean pci-stub must be used for
device
> > assignment to an HVM guest?  The VTdHowTo isn''t clear on
that.
> > 
> > Not per say.
> > 
> > xen-pciback can be used for both PV and HVM.
> 
> OK then what is the difference between pciback and xen-pciback?  I thought
they were just a re-name, but there must be more than that if one supports HVM
and the other doesn''t.
That is is. Just a rename.> 
> 
> > pci-stub can only be used for HVM guests.
> 
> OK, but then why would I ever want to use pci-stub, if xen-pciback already
does both?
Uhh.. you probably wouldn''t. But the pci-stub code has been in
existence
in the Linux kernel for some time..> 
> 
> 
> > But there seems to be a bug somewhere that when the PCI device is
> > assigned to pci-stub, pciback tries to seize it and can''t
find it and
> > somehow is stuck in a spin-lock.  That shouldn''t be
happening.
> > 
> > Right now I am trying to figure out if we remove from Lukasza system
> > pciback and only use pci-stub whether he still gets those MFN lookup
> > errors with his QLogic card. Those are, I believe, a seperate issue
> > from the pciback spinlock failure.
> 
> Yes, those sound like two different things.
> 
> I too am seeing problems with a Qlogic card in an HVM.  It looks like my
card''s BIOS makes an INT1A call (FIND PCI DEVICE function) that fails
when the HVM is booting, and the code hangs forever.
I would think this would work as the Bochs BIOS is actually being used
with real hardware. Is it possible that some data-structure that Bochs
BIOS has for the PCI enumeration is not up-to-date with the hvmloader
sets up? 

What about the BDF numbers? Any chance of trying to make them 1-to-1 (so
what you see in Dom0 is what you would see in DomU)?

Maybe there is an option in QEMU to do that?

What about passing in the PCI device _after_ the OS has booted. You can
do that by using ''pci-attach'' (or something) flags. That will
mean no
BIOS initialization, but the qla2xxx driver might be OK with that.
> But AFAICT it looks like Lukasz''s latest trace hasn''t
gotten to the point where it tries to load the PCI option ROM from the card.  I
would be interested to see if his card sees that error or not, once the MFN is
resolved.
<nods>> 
> Has anyone used these Qlogic cards successfully under VT-d in a Xen HVM?  I
don''t know of any reason why they should not work.  I''m
assuming (hoping) that they are virtualization-friendly enough to work under
VT-d.
I think you and Lukasz are the first ones.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Nadolski, Ed

2010-Mar-22 22:11 UTC

head link

RE: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

> -----Original Message-----
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> Sent: Monday, March 22, 2010 2:54 PM
> To: Nadolski, Ed
> Cc: Łukasz Oleś; xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup
> 
> On Mon, Mar 22, 2010 at 03:06:48PM -0600, Nadolski, Ed wrote:
> >
> >
> > > -----Original Message-----
> >
> > > PCI back is to be used _only_ with PV guests - you on the other
> > > hand are running an HVM guest.
> > >
> > > Per the earlier statement, I would recommend you use the
''pciback''
> > > instead of ''pci-stub'' or just not compile
pciback in and see what
> > > happens. Keep in mind: PCI back module is only needed when you
want
> to
> > > do PV PCI passthrough, which is not what you are doing.
> >
> > I''m confused - does that mean pci-stub must be used for
device
> assignment to an HVM guest?  The VTdHowTo isn''t clear on that.
> 
> Not per say.
> 
> xen-pciback can be used for both PV and HVM.
OK then what is the difference between pciback and xen-pciback?  I thought they
were just a re-name, but there must be more than that if one supports HVM and
the other doesn''t.

> pci-stub can only be used for HVM guests.
OK, but then why would I ever want to use pci-stub, if xen-pciback already does
both?


> But there seems to be a bug somewhere that when the PCI device is
> assigned to pci-stub, pciback tries to seize it and can''t find it
and
> somehow is stuck in a spin-lock.  That shouldn''t be happening.
> 
> Right now I am trying to figure out if we remove from Lukasza system
> pciback and only use pci-stub whether he still gets those MFN lookup
> errors with his QLogic card. Those are, I believe, a seperate issue
> from the pciback spinlock failure.
Yes, those sound like two different things.

I too am seeing problems with a Qlogic card in an HVM.  It looks like my
card''s BIOS makes an INT1A call (FIND PCI DEVICE function) that fails
when the HVM is booting, and the code hangs forever. But AFAICT it looks like
Lukasz''s latest trace hasn''t gotten to the point where it
tries to load the PCI option ROM from the card.  I would be interested to see if
his card sees that error or not, once the MFN is resolved.

Has anyone used these Qlogic cards successfully under VT-d in a Xen HVM?  I
don''t know of any reason why they should not work.  I''m
assuming (hoping) that they are virtualization-friendly enough to work under
VT-d.

Thanks again,
Ed



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Sander Eikelenboom

2010-Mar-22 22:57 UTC

head link

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Hello Ed,

I have put a note on the VT-D howto wiki page at the example using pci-stub.
That pci-stub is only working for HVM, and xen-pciback works for both PV and HVM
and is thus preferred.

--

Sander


Monday, March 22, 2010, 11:11:34 PM, you wrote:
>> -----Original Message-----
>> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
>> Sent: Monday, March 22, 2010 2:54 PM
>> To: Nadolski, Ed
>> Cc: Łukasz Oleś; xen-devel@lists.xensource.com
>> Subject: Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup
>> 
>> On Mon, Mar 22, 2010 at 03:06:48PM -0600, Nadolski, Ed wrote:
>> >
>> >
>> > > -----Original Message-----
>> >
>> > > PCI back is to be used _only_ with PV guests - you on the
other
>> > > hand are running an HVM guest.
>> > >
>> > > Per the earlier statement, I would recommend you use the
''pciback''
>> > > instead of ''pci-stub'' or just not compile
pciback in and see what
>> > > happens. Keep in mind: PCI back module is only needed when
you want
>> to
>> > > do PV PCI passthrough, which is not what you are doing.
>> >
>> > I''m confused - does that mean pci-stub must be used for
device
>> assignment to an HVM guest?  The VTdHowTo isn''t clear on that.
>> 
>> Not per say.
>> 
>> xen-pciback can be used for both PV and HVM.
> OK then what is the difference between pciback and xen-pciback?  I thought
they were just a re-name, but there must be more than that if one supports HVM
and the other doesn''t.
>> pci-stub can only be used for HVM guests.
> OK, but then why would I ever want to use pci-stub, if xen-pciback already
does both?







_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Mar 2010 - [vt-d][xen4-rc6] Hangs on startup

[Xen-devel] [vt-d][xen4-rc6] Hangs on startup

Re: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

RE: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

Re: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

RE: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

RE: [Xen-devel] [vt-d][xen4-rc6] Hangs on startup

[Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

RE: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

RE: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup

Re: [Xen-devel] Re: [vt-d][xen4-rc6] Hangs on startup