Hi,
We've been successfully using Xen with PCI pass-through of disk and
network cards for a couple of years now. We have this architecture working
successfully with a wide range of hardware from HP and Dell but recently
have been getting problems with the HP ProLiant DL380p Gen8 and its
P420i RAID controller.
When we bind the RAID controller to pciback and issue the appropriate
'xl pci-attach' command the controller appears on the driver domain but
fails
during probing by the hpsa driver. Interestingly, in about 1 in every 10
test runs the hpsa driver probes successfully, the controller appears in the
driver domain and the disks appear to function correctly.
In the failure cases the syslog output from the hpsa driver is as follows:
Jul 7 13:30:01 zylevel0-HVM-domU kernel: [ 54.855597] bus: 'pci':
driver_prob
e_device: matched device 0000:00:00.0 with driver hpsa
Jul 7 13:30:01 zylevel0-HVM-domU kernel: [ 54.855599] bus: 'pci':
really_prob
e: probing driver hpsa with device 0000:00:00.0
Jul 7 13:30:01 zylevel0-HVM-domU kernel: [ 54.855605] HP HPSA Driver (v 3.4.4
-1)
Jul 7 13:30:01 zylevel0-HVM-domU kernel: [ 54.855684] hpsa 0000:00:00.0: enab
ling device (0000 -> 0003)
Jul 7 13:30:01 zylevel0-HVM-domU kernel: [ 54.855983] hpsa 0000:00:00.0: Xen
PCI mapped GSI34 to IRQ29
Jul 7 13:30:01 zylevel0-HVM-domU kernel: [ 54.856089] hpsa 0000:00:00.0: enab
ling bus mastering
Jul 7 13:30:01 zylevel0-HVM-domU kernel: [ 54.856659] hpsa 0000:00:00.0: MSIX
Jul 7 13:30:01 zylevel0-HVM-domU kernel: [ 54.857384] hpsa 0000:00:00.0: Logi
cal aborts not supported
Jul 7 13:40:00 zylevel0-HVM-domU kernel: [ 654.318297] hpsa 0000:00:00.0:
unable
to get board into simple mode
Jul 7 13:40:00 zylevel0-HVM-domU kernel: [ 654.318478] hpsa: probe of
0000:00:00.0
rejects match -19
The error seems to originate from the hpsa_enter_simple_mode function and looks
like the first point where an interaction with the board via IO mapped memory is
expected to result in a response from the board.
We are using Xen 4.4 and a newly built Linux 3.15.0 kernel with Ubuntu trusty.
VTd is
disabled in the BIOS of the Dom-0. We use the following Grub commands for the
Dom-0 boot:
multiboot /boot/xen.gz placeholder dom0_mem=max:2048m iommu=0
loglvl=all guest_loglvl=all
module /boot/vmlinuz-3.15.0 placeholder
root=UUID=f56cf02b-d46f-4e2a-b667-242f97ec82f5
ro debug
module --nounzip /boot/initrd.img-3.15.0
The driver domain is a PV guest running the same kernel version as the Dom-0.
The Xen configuration file contains the following kernel boot parameters:
extra="earlyprintk=xen debug max_loop=64 iommu=soft"
After we issue the pci-attach command the following additional lines appear in
the
'xl dmesg' log:
(XEN) grant_table.c:289:d0 Increased maptrack size to 2 frames
(XEN) mm.c:809: d2: Forcing read-only access to MFN f7f02
(XEN) mm.c:809: d0: Forcing read-only access to MFN f7f02
We were wondering whether anyone in the community had got PCI pass-through of
the P420i working, or any pointers of where to look to further diagnose the
problem.
Best wishes,
Simon