On Wed, Apr 11, 2012 at 04:31:46PM -0400, Aleksandr Tarutin
wrote:> New findings!
> Passthrough worked for a PV domU!
> *And*... It worked if i pci-attach the controller to the same PVHVM!
> Any suggestions?
>
The difference is that when you use a PV domU or pci-attach with PVHVM guest
the LSI Boot/Option ROM is not executed at all.
PV domUs don''t execute Option ROMS, because they don''t have a
BIOS.
Also when you pci-attach to a running HVM guest, the LSI BIOS/ROM is not
executed.
However when you do a full boot of HVM guest with PCI passthru the
LSI MPT Boot/Option ROM gets executed by Xen HVM ROMBIOS,
and that seems to fail for some reason, so I assume the LSI HBA
is left in some weird semi-initialized or broken state,
and thus the mpt2sas driver fails to work..
I picked the interesting entries from your logs:
Error on the HVM guest VNC console during boot (before grub is executed):
MPT BIOS Fault 09h encountered at adapter PCI(00h,05h,00h)
Xen dmesg:
Xen version 4.1.3-rc1-pre
(XEN) HVM1: Writing SMBIOS tables ...
(XEN) HVM1: Loading ROMBIOS ...
(XEN) HVM1: Loading PCI Option ROM ...
(XEN) HVM1: - Manufacturer: LSI Corporation
(XEN) HVM1: - Product name: LSI MPI Boot Support
(XEN) domctl.c:995:d0 memory_map:remove: gfn=f3000 mfn=fba00 nr_mfns=80
(XEN) HVM1: Invoking ROMBIOS ...
dunno if this is related?:
(XEN) traps.c:451:d0 Unhandled nmi fault/trap [#2] on VCPU 0 [ec=0000]
dom0 lspci:
01:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008
PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
Subsystem: LSI Logic / Symbios Logic Device 3020
Flags: fast devsel, IRQ 16
I/O ports at e000 [size=256]
Memory at fbac0000 (64-bit, non-prefetchable) [size=16K]
Memory at fba80000 (64-bit, non-prefetchable) [size=256K]
Expansion ROM at fba00000 [disabled] [size=512K]
Capabilities: [50] Power Management version 3
Capabilities: [68] Express Endpoint, MSI 00
Capabilities: [d0] Vital Product Data
Capabilities: [a8] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [c0] MSI-X: Enable- Count=15 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [138] Power Budgeting <?>
Kernel driver in use: pciback
Kernel modules: mpt2sas
HVM guest kernel dmesg:
mpt2sas version 09.101.00.00 loaded
scsi0 : Fusion MPT SAS Host
alloc irq_desc for 36 on node -1
alloc kstat_irqs on node -1
mpt2sas 0000:00:05.0: PCI INT A -> GSI 36 (level, low) -> IRQ 36
mpt2sas 0000:00:05.0: setting latency timer to 64
mpt2sas0: 32 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (2053376 kB)
alloc irq_desc for 48 on node -1
alloc kstat_irqs on node -1
mpt2sas 0000:00:05.0: irq 48 for MSI/MSI-X
mpt2sas0: PCI-MSI-X enabled: IRQ 48
mpt2sas0: iomem(0x00000000f30e0000), mapped(0xffffc900009e8000), size(16384)
mpt2sas0: ioport(0x000000000000c100), size(256)
mpt2sas0: Allocated physical memory: size(2688 kB)
mpt2sas0: Current Controller Queue Depth(1754), Max Controller Queue Depth(2015)
mpt2sas0: Scatter Gather Elements per IO(128)
mpt2sas0: _base_event_notification: timeout
mf:
07000000 00000000 00000000 00000000 00000000 0f2f7fff fffffffc ffffffff
ffffffff 00000000 00000000
mpt2sas0: sending diag reset !!
mpt2sas0: diag reset: SUCCESS
mpt2sas 0000:00:05.0: PCI INT A disabled
mpt2sas0: failure at drivers/scsi/mpt2sas/mpt2sas_scsih.c:7628/_scsih_probe()!
Hopefully someone has more ideas about why the LSI MPT Boot/Option ROM fails..
-- Pasi