Kristoffer Egefelt
2013-Oct-04 07:44 UTC
[BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
Hi,
I''m trying to pass through a NIC (intel X520 with ixgbevf driver) to
domU, but since kernel 3.8 this has not worked.
The dom0 kernel seems to cause the problem.
Xen version, domU kernel version and driver version seems to be unrelated to
this bug, meaning
it works as long as dom0 kernel is 3.8.
I tried kernel version 3.9, 3.10 and 3.11 - all show the same bug pattern when
used as dom0.
The BUG appears on xl pci attach.
On pci detach the dom0 panics.
I have attached logs from a working setup (kernel 3.8) and from a setup not
working (kernel 3.11) and also the kernel config for 3.11.
In short, this is what domU logs after pci attach:
BUG: unable to handle kernel paging request at ffffc9000030200c
IP: [<ffffffff81205812>] __msix_mask_irq+0x21/0x24
PGD 75a40067 PUD 75a41067 PMD 75b44067 PTE 8010000000000464
Oops: 0003 [#1] SMP
Modules linked in: ixgbevf(+) xen_pcifront nfnetlink_log nfnetlink ipt_ULOG
x_tables x86_pkg_temp_thermal thermal_sys coretemp crc32c_intel
ghash_clmulni_intel aesni_intel aes_x86_64 ablk_helper cryptd lrw gf128mul
glue_helper microcode ext4 crc16 jbd2 mbcache xen_blkfront
CPU: 0 PID: 2122 Comm: modprobe Not tainted 3.11.3-kernel-v1.0.0.21+ #1
And this is dom0 on pci detach:
(XEN) Assertion ''_raw_spin_is_locked(lock)'' failed at
/usr/src/xen/xen/include/asm/spinlock.h:16402
(XEN) ----[ Xen-4.4-unstable x86_64 debug=y Not tainted ]----
(XEN) CPU: 1
(XEN) RIP: e008:[<ffff82d0801258ef>] _spin_unlock_irqrestore+0x27/0x32
(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor
(XEN) rax: 0000000000000001 rbx: ffff83201ba07724 rcx: 0000000000000001
(XEN) rdx: ffff83201bb97020 rsi: 0000000000000286 rdi: ffff83201ba07724
(XEN) rbp: ffff83203ffcfdd8 rsp: ffff83203ffcfdd8 r8: ffff8141002000e0
(XEN) r9: 000000000000001c r10: 0000000000000082 r11: 0000000000000001
(XEN) r12: 0000000000000000 r13: ffff8320e13c8240 r14: ffff880148047df4
(XEN) r15: 0000000000000286 cr0: 0000000080050033 cr4: 00000000000426f0
(XEN) cr3: 000000206f3ff000 cr2: 00007fa5ec560c49
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83203ffcfdd8:
(XEN) ffff83203ffcfe68 ffff82d080166a4f ffff83203ffcfe18 0000000280118988
(XEN) 0000000000000cfe 0000000000000cfe ffff832015d3b8a0 ffff8320e13c83f0
(XEN) ffff832015d3b880 0000000000000001 00000000fee00678 0000000000000000
(XEN) ffff83200000f800 000000000000001b ffff8300bcef5000 ffffffffffffffed
(XEN) ffff880148047df4 ffffffff814530e0 ffff83203ffcfef8 ffff82d08017dee4
(XEN) ffff832000000002 0000000000000008 ffff83203ffcfef8 ffff82d000a0fb00
(XEN) 0000000000000000 ffffffff93010000 ffff82d0802e8000 ffff83203ffc80ef
(XEN) 82d080222c00b948 c390ef66d1ffffff ffff83203ffcfef8 ffff8300bcef5000
(XEN) ffff880145951868 ffff880145bb2a60 ffff880148047f50 ffffffff814530e0
(XEN) 00007cdfc00300c7 ffff82d08022213b ffffffff8100142a 0000000000000021
(XEN) ffffffff814530e0 ffff880148047f50 000000000000c002 0000000000009300
(XEN) ffff88013faf1a80 ffff880145951000 0000000000000202 0000000000000093
(XEN) ffff880148047df4 0000000000000002 0000000000000021 ffffffff8100142a
(XEN) 0000000000000000 ffff880148047df4 000000000000001b 0001010000000000
(XEN) ffffffff8100142a 000000000000e033 0000000000000202 ffff880148047dc8
(XEN) 000000000000e02b 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000001 ffff8300bcef5000 0000004f9b885e00
(XEN) 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82d0801258ef>] _spin_unlock_irqrestore+0x27/0x32
(XEN) [<ffff82d080166a4f>] pci_restore_msi_state+0x1c9/0x2f0
(XEN) [<ffff82d08017dee4>] do_physdev_op+0xe4f/0x114f
(XEN) [<ffff82d08022213b>] syscall_enter+0xeb/0x145
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 1:
(XEN) Assertion ''_raw_spin_is_locked(lock)'' failed at
/usr/src/xen/xen/include/asm/spinlock.h:16402
(XEN) ****************************************
(XEN)
(XEN) Manual reset required (''noreboot'' specified)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
David Vrabel
2013-Oct-04 09:30 UTC
Re: [BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
On 04/10/13 08:44, Kristoffer Egefelt wrote:> Hi, > > I''m trying to pass through a NIC (intel X520 with ixgbevf driver) to domU, but since kernel 3.8 this has not worked. > > The dom0 kernel seems to cause the problem. > Xen version, domU kernel version and driver version seems to be unrelated to this bug, meaning > it works as long as dom0 kernel is 3.8. > I tried kernel version 3.9, 3.10 and 3.11 - all show the same bug pattern when used as dom0.Can you do a git bisect on the kernel to find which kernel commit triggers this crash? This is clearly a Xen bug but it would be useful to know what triggered it. Thanks. David
Andrew Cooper
2013-Oct-04 09:53 UTC
Re: [BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
On 04/10/13 08:44, Kristoffer Egefelt wrote:> Hi, > > I''m trying to pass through a NIC (intel X520 with ixgbevf driver) to > domU, but since kernel 3.8 this has not worked. > > The dom0 kernel seems to cause the problem. > Xen version, domU kernel version and driver version seems to be > unrelated to this bug, meaning > it works as long as dom0 kernel is 3.8. > I tried kernel version 3.9, 3.10 and 3.11 - all show the same bug > pattern when used as dom0. > > The BUG appears on xl pci attach. > On pci detach the dom0 panics. > > I have attached logs from a working setup (kernel 3.8) and from a > setup not working (kernel 3.11) and also the kernel config for 3.11. > > In short, this is what domU logs after pci attach: > > BUG: unable to handle kernel paging request at ffffc9000030200c > IP: [<ffffffff81205812>] __msix_mask_irq+0x21/0x24 > PGD 75a40067 PUD 75a41067 PMD 75b44067 PTE 8010000000000464 > Oops: 0003 [#1] SMP > Modules linked in: ixgbevf(+) xen_pcifront nfnetlink_log nfnetlink > ipt_ULOG x_tables x86_pkg_temp_thermal thermal_sys coretemp > crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 ablk_helper > cryptd lrw gf128mul glue_helper microcode ext4 crc16 jbd2 mbcache > xen_blkfront > CPU: 0 PID: 2122 Comm: modprobe Not tainted 3.11.3-kernel-v1.0.0.21+ #1 > > And this is dom0 on pci detach: > > (XEN) Assertion ''_raw_spin_is_locked(lock)'' failed at > /usr/src/xen/xen/include/asm/spinlock.h:16402 > (XEN) ----[ Xen-4.4-unstable x86_64 debug=y Not tainted ]---- > (XEN) CPU: 1 > (XEN) RIP: e008:[<ffff82d0801258ef>] _spin_unlock_irqrestore+0x27/0x32 > (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor > (XEN) rax: 0000000000000001 rbx: ffff83201ba07724 rcx: > 0000000000000001 > (XEN) rdx: ffff83201bb97020 rsi: 0000000000000286 rdi: > ffff83201ba07724 > (XEN) rbp: ffff83203ffcfdd8 rsp: ffff83203ffcfdd8 r8: > ffff8141002000e0 > (XEN) r9: 000000000000001c r10: 0000000000000082 r11: > 0000000000000001 > (XEN) r12: 0000000000000000 r13: ffff8320e13c8240 r14: > ffff880148047df4 > (XEN) r15: 0000000000000286 cr0: 0000000080050033 cr4: > 00000000000426f0 > (XEN) cr3: 000000206f3ff000 cr2: 00007fa5ec560c49 > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 > (XEN) Xen stack trace from rsp=ffff83203ffcfdd8: > (XEN) ffff83203ffcfe68 ffff82d080166a4f ffff83203ffcfe18 > 0000000280118988 > (XEN) 0000000000000cfe 0000000000000cfe ffff832015d3b8a0 > ffff8320e13c83f0 > (XEN) ffff832015d3b880 0000000000000001 00000000fee00678 > 0000000000000000 > (XEN) ffff83200000f800 000000000000001b ffff8300bcef5000 > ffffffffffffffed > (XEN) ffff880148047df4 ffffffff814530e0 ffff83203ffcfef8 > ffff82d08017dee4 > (XEN) ffff832000000002 0000000000000008 ffff83203ffcfef8 > ffff82d000a0fb00 > (XEN) 0000000000000000 ffffffff93010000 ffff82d0802e8000 > ffff83203ffc80ef > (XEN) 82d080222c00b948 c390ef66d1ffffff ffff83203ffcfef8 > ffff8300bcef5000 > (XEN) ffff880145951868 ffff880145bb2a60 ffff880148047f50 > ffffffff814530e0 > (XEN) 00007cdfc00300c7 ffff82d08022213b ffffffff8100142a > 0000000000000021 > (XEN) ffffffff814530e0 ffff880148047f50 000000000000c002 > 0000000000009300 > (XEN) ffff88013faf1a80 ffff880145951000 0000000000000202 > 0000000000000093 > (XEN) ffff880148047df4 0000000000000002 0000000000000021 > ffffffff8100142a > (XEN) 0000000000000000 ffff880148047df4 000000000000001b > 0001010000000000 > (XEN) ffffffff8100142a 000000000000e033 0000000000000202 > ffff880148047dc8 > (XEN) 000000000000e02b 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000001 ffff8300bcef5000 > 0000004f9b885e00 > (XEN) 0000000000000000 > (XEN) Xen call trace: > (XEN) [<ffff82d0801258ef>] _spin_unlock_irqrestore+0x27/0x32 > (XEN) [<ffff82d080166a4f>] pci_restore_msi_state+0x1c9/0x2f0 > (XEN) [<ffff82d08017dee4>] do_physdev_op+0xe4f/0x114f > (XEN) [<ffff82d08022213b>] syscall_enter+0xeb/0x145 > (XEN) > (XEN) > (XEN) **************************************** > (XEN) Panic on CPU 1: > (XEN) Assertion ''_raw_spin_is_locked(lock)'' failed at > /usr/src/xen/xen/include/asm/spinlock.h:16402 > (XEN) **************************************** > (XEN) > (XEN) Manual reset required (''noreboot'' specified)This Xen crash appears to be as a result of d1b6d0a0248 "x86: enable multi-vector MSI", and is likely a mishanding of some error state caused by the initial pci attach failure/partial setup. Can you printk() in xen/arch/x86/msi.c:pci_restore_msi_state() just before the list_for_each_entry_safe() and work out which is the problematic pci device, then run `xl debug-keys iQM` and provide the output. Perhaps an lspci of the affected device might also help. ~Andrew _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Jan Beulich
2013-Oct-04 10:05 UTC
Re: [BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
>>> On 04.10.13 at 09:44, Kristoffer Egefelt <kristoffer@itoc.dk> wrote: > Hi, > > I''m trying to pass through a NIC (intel X520 with ixgbevf driver) to domU, > but since kernel 3.8 this has not worked. > > The dom0 kernel seems to cause the problem. > Xen version, domU kernel version and driver version seems to be unrelated to > this bug, meaning > it works as long as dom0 kernel is 3.8. > I tried kernel version 3.9, 3.10 and 3.11 - all show the same bug pattern > when used as dom0. > > The BUG appears on xl pci attach. > On pci detach the dom0 panics. > > I have attached logs from a working setup (kernel 3.8) and from a setup not > working (kernel 3.11) and also the kernel config for 3.11. > > In short, this is what domU logs after pci attach: > > BUG: unable to handle kernel paging request at ffffc9000030200c > IP: [<ffffffff81205812>] __msix_mask_irq+0x21/0x24 > PGD 75a40067 PUD 75a41067 PMD 75b44067 PTE 8010000000000464 > Oops: 0003 [#1] SMP > Modules linked in: ixgbevf(+) xen_pcifront nfnetlink_log nfnetlink ipt_ULOG > x_tables x86_pkg_temp_thermal thermal_sys coretemp crc32c_intel > ghash_clmulni_intel aesni_intel aes_x86_64 ablk_helper cryptd lrw gf128mul > glue_helper microcode ext4 crc16 jbd2 mbcache xen_blkfront > CPU: 0 PID: 2122 Comm: modprobe Not tainted 3.11.3-kernel-v1.0.0.21+ #1Are you certain this is kernel (rather than hypervisor) version dependent? Iirc this is a manifestation of a guest kernel not being permitted to write to the MSI-X mask bit.> And this is dom0 on pci detach: > > (XEN) Assertion ''_raw_spin_is_locked(lock)'' failed at > /usr/src/xen/xen/include/asm/spinlock.h:16402 > (XEN) ----[ Xen-4.4-unstable x86_64 debug=y Not tainted ]---- > (XEN) CPU: 1 > (XEN) RIP: e008:[<ffff82d0801258ef>] _spin_unlock_irqrestore+0x27/0x32 > (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor > (XEN) rax: 0000000000000001 rbx: ffff83201ba07724 rcx: 0000000000000001 > (XEN) rdx: ffff83201bb97020 rsi: 0000000000000286 rdi: ffff83201ba07724 > (XEN) rbp: ffff83203ffcfdd8 rsp: ffff83203ffcfdd8 r8: ffff8141002000e0 > (XEN) r9: 000000000000001c r10: 0000000000000082 r11: 0000000000000001 > (XEN) r12: 0000000000000000 r13: ffff8320e13c8240 r14: ffff880148047df4 > (XEN) r15: 0000000000000286 cr0: 0000000080050033 cr4: 00000000000426f0 > (XEN) cr3: 000000206f3ff000 cr2: 00007fa5ec560c49 > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 > (XEN) Xen stack trace from rsp=ffff83203ffcfdd8: > (XEN) ffff83203ffcfe68 ffff82d080166a4f ffff83203ffcfe18 0000000280118988 > (XEN) 0000000000000cfe 0000000000000cfe ffff832015d3b8a0 ffff8320e13c83f0 > (XEN) ffff832015d3b880 0000000000000001 00000000fee00678 0000000000000000 > (XEN) ffff83200000f800 000000000000001b ffff8300bcef5000 ffffffffffffffed > (XEN) ffff880148047df4 ffffffff814530e0 ffff83203ffcfef8 ffff82d08017dee4 > (XEN) ffff832000000002 0000000000000008 ffff83203ffcfef8 ffff82d000a0fb00 > (XEN) 0000000000000000 ffffffff93010000 ffff82d0802e8000 ffff83203ffc80ef > (XEN) 82d080222c00b948 c390ef66d1ffffff ffff83203ffcfef8 ffff8300bcef5000 > (XEN) ffff880145951868 ffff880145bb2a60 ffff880148047f50 ffffffff814530e0 > (XEN) 00007cdfc00300c7 ffff82d08022213b ffffffff8100142a 0000000000000021 > (XEN) ffffffff814530e0 ffff880148047f50 000000000000c002 0000000000009300 > (XEN) ffff88013faf1a80 ffff880145951000 0000000000000202 0000000000000093 > (XEN) ffff880148047df4 0000000000000002 0000000000000021 ffffffff8100142a > (XEN) 0000000000000000 ffff880148047df4 000000000000001b 0001010000000000 > (XEN) ffffffff8100142a 000000000000e033 0000000000000202 ffff880148047dc8 > (XEN) 000000000000e02b 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000001 ffff8300bcef5000 0000004f9b885e00 > (XEN) 0000000000000000 > (XEN) Xen call trace: > (XEN) [<ffff82d0801258ef>] _spin_unlock_irqrestore+0x27/0x32 > (XEN) [<ffff82d080166a4f>] pci_restore_msi_state+0x1c9/0x2f0 > (XEN) [<ffff82d08017dee4>] do_physdev_op+0xe4f/0x114f > (XEN) [<ffff82d08022213b>] syscall_enter+0xeb/0x145 > (XEN) > (XEN) > (XEN) **************************************** > (XEN) Panic on CPU 1: > (XEN) Assertion ''_raw_spin_is_locked(lock)'' failed at > /usr/src/xen/xen/include/asm/spinlock.h:16402 > (XEN) **************************************** > (XEN) > (XEN) Manual reset required (''noreboot'' specified)This, otoh, is clearly a hypervisor bug. Afaict the patch below should help. But - this code is supposed to be executed on host S3 resume only (i.e. there might also be some kernel flaw involved here). Jan --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -1158,11 +1158,11 @@ int pci_restore_msi_state(struct pci_dev for ( i = 0; ; ) { msi_set_mask_bit(desc, entry[i].msi_attrib.masked); - spin_unlock_irqrestore(&desc->lock, flags); if ( !--nr ) break; + spin_unlock_irqrestore(&desc->lock, flags); desc = &irq_desc[entry[++i].irq]; spin_lock_irqsave(&desc->lock, flags); if ( desc->msi_desc != entry + i )
Kristoffer Egefelt
2013-Oct-04 12:03 UTC
Re: [BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
I''m not sure what I should printk() to get the device - could you show me ? Ouput from xl debug-keys iQM and lspci attached. The devices I''m trying to pass through is all of the "Ethernet Controller Virtual Function"s - from 01:10.0 to 01:12.3 (nodev20)root@h9:/usr/src# lspci -s 01: 01:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) 01:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) 01:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:10.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:10.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:10.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:11.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:11.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:11.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:11.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:11.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:11.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:12.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:12.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:12.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 01:12.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) (nodev20)root@h9:/usr/src# lspci -s 01: -vv 01:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) Subsystem: Dell Ethernet 10G 4P X520/I350 rNDC Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin B routed to IRQ 36 Region 0: Memory at d8e80000 (64-bit, non-prefetchable) [size=512K] Region 2: I/O ports at fcc0 [size=32] Region 4: Memory at d8ff8000 (64-bit, non-prefetchable) [size=16K] Expansion ROM at d8000000 [disabled] [size=512K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME- Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ Address: 0000000000000000 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [70] MSI-X: Enable+ Count=64 Masked- Vector table: BAR=4 offset=00000000 PBA: BAR=4 offset=00002000 Capabilities: [a0] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ DevCtl: Report errors: Correctable- Non-Fatal+ Fatal+ Unsupported+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset- MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend- LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s, Latency L0 <1us, L1 <8us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [e0] Vital Product Data Product Name: X520 10GbE Controller Read-only fields: [PN] Part number: G61346 [MN] Manufacture ID: 31 30 32 38 [V0] Vendor specific: FFV14.5.9 [V1] Vendor specific: DSV1028VPDR.VER1.0 [V3] Vendor specific: DTINIC [V4] Vendor specific: DCM10010081D521010081D5 [V5] Vendor specific: NPY2 [V6] Vendor specific: PMT12345678 [V7] Vendor specific: NMVIntel Corp [RV] Reserved: checksum good, 4 byte(s) reserved End Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+ AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [140 v1] Device Serial Number 36-a0-ff-ff-ff-9f-00-00 Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 1 ARICtl: MFVC- ACS-, Function Group: 0 Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV) IOVCap: Migration-, Interrupt Message Number: 000 IOVCtl: Enable+ Migration- Interrupt- MSE+ ARIHierarchy+ IOVSta: Migration- Initial VFs: 64, Total VFs: 64, Number of VFs: 10, Function Dependency Link: 00 VF offset: 128, stride: 2, Device ID: 10ed Supported Page Size: 00000553, System Page Size: 00000001 Region 0: Memory at 00000000d5000000 (64-bit, prefetchable) Region 3: Memory at 00000000d5100000 (64-bit, prefetchable) VF Migration: offset: 00000000, BIR: 0 Kernel driver in use: ixgbe 01:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) Subsystem: Dell Ethernet 10G 4P X520/I350 rNDC Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 34 Region 0: Memory at d8f00000 (64-bit, non-prefetchable) [size=512K] Region 2: I/O ports at fce0 [size=32] Region 4: Memory at d8ffc000 (64-bit, non-prefetchable) [size=16K] Expansion ROM at d8080000 [disabled] [size=512K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME- Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ Address: 0000000000000000 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [70] MSI-X: Enable+ Count=64 Masked- Vector table: BAR=4 offset=00000000 PBA: BAR=4 offset=00002000 Capabilities: [a0] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ DevCtl: Report errors: Correctable- Non-Fatal+ Fatal+ Unsupported+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset- MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend- LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s, Latency L0 <1us, L1 <8us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [e0] Vital Product Data Product Name: X520 10GbE Controller Read-only fields: [PN] Part number: G61346 [MN] Manufacture ID: 31 30 32 38 [V0] Vendor specific: FFV14.5.9 [V1] Vendor specific: DSV1028VPDR.VER1.0 [V3] Vendor specific: DTINIC [V4] Vendor specific: DCM10010081D521010081D5 [V5] Vendor specific: NPY2 [V6] Vendor specific: PMT12345678 [V7] Vendor specific: NMVIntel Corp [RV] Reserved: checksum good, 4 byte(s) reserved End Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+ AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [140 v1] Device Serial Number 36-a0-ff-ff-ff-9f-00-00 Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV) IOVCap: Migration-, Interrupt Message Number: 000 IOVCtl: Enable+ Migration- Interrupt- MSE+ ARIHierarchy- IOVSta: Migration- Initial VFs: 64, Total VFs: 64, Number of VFs: 10, Function Dependency Link: 01 VF offset: 128, stride: 2, Device ID: 10ed Supported Page Size: 00000553, System Page Size: 00000001 Region 0: Memory at 00000000d5200000 (64-bit, prefetchable) Region 3: Memory at 00000000d5300000 (64-bit, prefetchable) VF Migration: offset: 00000000, BIR: 0 Kernel driver in use: ixgbe 01:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5000000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5100000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 01:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5200000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5300000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 01:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5004000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5104000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 01:10.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5204000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5304000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 01:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5008000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5108000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 01:10.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5208000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5308000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 01:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d500c000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d510c000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:10.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d520c000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d530c000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5010000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5110000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:11.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5210000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5310000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:11.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5014000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5114000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:11.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5214000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5314000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5018000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5118000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:11.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5218000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5318000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:11.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d501c000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d511c000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:11.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d521c000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d531c000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:12.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5020000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5120000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:12.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5220000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5320000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:12.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5024000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5124000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback 01:12.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Subsystem: Dell Device 1f72 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Region 0: [virtual] Memory at d5224000 (64-bit, prefetchable) [size=16K] Region 3: [virtual] Memory at d5324000 (64-bit, prefetchable) [size=16K] Capabilities: [70] MSI-X: Enable- Count=3 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v0) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Kernel driver in use: pciback (nodev20)root@h9:/usr/src# xl debug-keys iQM (XEN) Guest interrupt information: (XEN) IRQ: 0 affinity:00000001 vec:f0 type=IO-APIC-edge status=00000000 timer_interrupt+0/0x11f (XEN) IRQ: 1 affinity:00000001 vec:38 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 3 affinity:ffffffff vec:40 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 4 affinity:00000001 vec:f1 type=IO-APIC-edge status=00000000 ns16550_interrupt+0/0x68 (XEN) IRQ: 5 affinity:00000001 vec:48 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 6 affinity:00000001 vec:50 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 7 affinity:00000001 vec:58 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 8 affinity:00000001 vec:60 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 8(----), (XEN) IRQ: 9 affinity:00000001 vec:68 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 9(----), (XEN) IRQ: 10 affinity:ffffffff vec:70 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 11 affinity:00000001 vec:78 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 12 affinity:00000001 vec:88 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 13 affinity:ffffffff vec:90 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 14 affinity:00000001 vec:98 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 15 affinity:00000001 vec:a0 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 16 affinity:ffffffff vec:b0 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 19 affinity:ffffffff vec:b8 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 20 affinity:ffffffff vec:b9 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 22 affinity:ffffffff vec:d0 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 23 affinity:ffffffff vec:c8 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 34 affinity:ffffffff vec:d9 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 36 affinity:ffffffff vec:69 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 42 affinity:ffffffff vec:71 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 53 affinity:ffffffff vec:a8 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 85 affinity:ffffffff vec:c0 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 88 affinity:ffffffff vec:28 type=DMA_MSI status=00000000 iommu_page_fault+0/0x12 (XEN) IRQ: 89 affinity:ffffffff vec:30 type=DMA_MSI status=00000000 iommu_page_fault+0/0x12 (XEN) IRQ: 90 affinity:00000001 vec:d8 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:343(----), (XEN) IRQ: 91 affinity:00000001 vec:21 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:342(----), (XEN) IRQ: 92 affinity:00000001 vec:29 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:341(----), (XEN) IRQ: 93 affinity:00000001 vec:31 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:340(----), (XEN) IRQ: 94 affinity:00000001 vec:39 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:339(----), (XEN) IRQ: 95 affinity:00000001 vec:41 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:338(----), (XEN) IRQ: 96 affinity:00000001 vec:49 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:337(----), (XEN) IRQ: 97 affinity:00000001 vec:51 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:336(----), (XEN) IRQ: 98 affinity:00000001 vec:59 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:335(----), (XEN) IRQ: 99 affinity:00000001 vec:61 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:334(----), (XEN) IRQ: 100 affinity:00000001 vec:79 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:333(----), (XEN) IRQ: 101 affinity:00000001 vec:81 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:332(----), (XEN) IRQ: 102 affinity:00000001 vec:89 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:331(----), (XEN) IRQ: 103 affinity:00000001 vec:91 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:330(----), (XEN) IRQ: 104 affinity:00000001 vec:99 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:329(----), (XEN) IRQ: 105 affinity:00000001 vec:a1 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:328(----), (XEN) IRQ: 106 affinity:00000001 vec:a9 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:327(----), (XEN) IRQ: 107 affinity:00000001 vec:b1 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:326(----), (XEN) IRQ: 108 affinity:00000001 vec:c1 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:325(----), (XEN) IRQ: 109 affinity:00000001 vec:ba type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:324(----), (XEN) IRQ: 110 affinity:00000001 vec:c2 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:323(----), (XEN) IRQ: 111 affinity:00000001 vec:7a type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:322(----), (XEN) IRQ: 112 affinity:00000001 vec:8a type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:321(----), (XEN) IRQ: 113 affinity:00000001 vec:ca type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:320(----), (XEN) IRQ: 114 affinity:00000001 vec:d2 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:319(----), (XEN) IRQ: 115 affinity:00000001 vec:da type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:318(----), (XEN) IRQ: 120 affinity:00000001 vec:92 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:313(----), (XEN) IRQ: 121 affinity:00000001 vec:9a type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:312(----), (XEN) IRQ: 122 affinity:00000001 vec:a2 type=PCI-MSI/-X status=00000010 in-flight=0 domain-list=0:311(----), (XEN) IO-APIC interrupt information: (XEN) IRQ 0 Vec240: (XEN) Apic 0x00, Pin 2: vec=f0 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 1 Vec 56: (XEN) Apic 0x00, Pin 1: vec=38 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 3 Vec 64: (XEN) Apic 0x00, Pin 3: vec=40 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=1 dest_id:15 (XEN) IRQ 4 Vec241: (XEN) Apic 0x00, Pin 4: vec=f1 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 5 Vec 72: (XEN) Apic 0x00, Pin 5: vec=48 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 6 Vec 80: (XEN) Apic 0x00, Pin 6: vec=50 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 7 Vec 88: (XEN) Apic 0x00, Pin 7: vec=58 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 8 Vec 96: (XEN) Apic 0x00, Pin 8: vec=60 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 9 Vec104: (XEN) Apic 0x00, Pin 9: vec=68 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=L mask=0 dest_id:0 (XEN) IRQ 10 Vec112: (XEN) Apic 0x00, Pin 10: vec=70 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=1 dest_id:3 (XEN) IRQ 11 Vec120: (XEN) Apic 0x00, Pin 11: vec=78 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 12 Vec136: (XEN) Apic 0x00, Pin 12: vec=88 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 13 Vec144: (XEN) Apic 0x00, Pin 13: vec=90 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=1 dest_id:34 (XEN) IRQ 14 Vec152: (XEN) Apic 0x00, Pin 14: vec=98 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 15 Vec160: (XEN) Apic 0x00, Pin 15: vec=a0 delivery=Fixed dest=P status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 16 Vec176: (XEN) Apic 0x00, Pin 16: vec=b0 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=1 dest_id:40 (XEN) IRQ 19 Vec184: (XEN) Apic 0x00, Pin 19: vec=b8 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=1 dest_id:41 (XEN) IRQ 20 Vec185: (XEN) Apic 0x00, Pin 20: vec=b9 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=1 dest_id:36 (XEN) IRQ 22 Vec208: (XEN) Apic 0x00, Pin 22: vec=d0 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=1 dest_id:10 (XEN) IRQ 23 Vec200: (XEN) Apic 0x00, Pin 23: vec=c8 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=1 dest_id:6 (XEN) IRQ 34 Vec217: (XEN) Apic 0x01, Pin 2: vec=d9 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=1 dest_id:37 (XEN) IRQ 36 Vec105: (XEN) Apic 0x01, Pin 4: vec=69 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=1 dest_id:34 (XEN) IRQ 42 Vec113: (XEN) Apic 0x01, Pin 10: vec=71 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=1 dest_id:9 (XEN) IRQ 53 Vec168: (XEN) Apic 0x01, Pin 21: vec=a8 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=1 dest_id:11 (XEN) IRQ 85 Vec192: (XEN) Apic 0x02, Pin 21: vec=c0 delivery=Fixed dest=P status=0 polarity=1 irr=0 trig=L mask=1 dest_id:15 (XEN) ==== PCI devices ===(XEN) ==== segment 0000 ===(XEN) 0000:01:12.3 - dom 0 - MSIs < > (XEN) 0000:01:12.1 - dom 0 - MSIs < > (XEN) 0000:01:11.7 - dom 0 - MSIs < > (XEN) 0000:01:11.5 - dom 0 - MSIs < > (XEN) 0000:01:11.3 - dom 0 - MSIs < > (XEN) 0000:01:11.1 - dom 0 - MSIs < > (XEN) 0000:01:10.7 - dom 0 - MSIs < > (XEN) 0000:01:10.5 - dom 0 - MSIs < > (XEN) 0000:01:10.3 - dom 0 - MSIs < > (XEN) 0000:01:10.1 - dom 0 - MSIs < > (XEN) 0000:01:12.2 - dom 0 - MSIs < > (XEN) 0000:01:12.0 - dom 0 - MSIs < > (XEN) 0000:01:11.6 - dom 0 - MSIs < > (XEN) 0000:01:11.4 - dom 0 - MSIs < > (XEN) 0000:01:11.2 - dom 0 - MSIs < > (XEN) 0000:01:11.0 - dom 0 - MSIs < > (XEN) 0000:01:10.6 - dom 0 - MSIs < > (XEN) 0000:01:10.4 - dom 0 - MSIs < > (XEN) 0000:01:10.2 - dom 0 - MSIs < > (XEN) 0000:01:10.0 - dom 0 - MSIs < > (XEN) 0000:7f:13.6 - dom 0 - MSIs < > (XEN) 0000:7f:13.5 - dom 0 - MSIs < > (XEN) 0000:7f:13.4 - dom 0 - MSIs < > (XEN) 0000:7f:13.1 - dom 0 - MSIs < > (XEN) 0000:7f:13.0 - dom 0 - MSIs < > (XEN) 0000:7f:11.0 - dom 0 - MSIs < > (XEN) 0000:7f:10.7 - dom 0 - MSIs < > (XEN) 0000:7f:10.6 - dom 0 - MSIs < > (XEN) 0000:7f:10.5 - dom 0 - MSIs < > (XEN) 0000:7f:10.4 - dom 0 - MSIs < > (XEN) 0000:7f:10.3 - dom 0 - MSIs < > (XEN) 0000:7f:10.2 - dom 0 - MSIs < > (XEN) 0000:7f:10.1 - dom 0 - MSIs < > (XEN) 0000:7f:10.0 - dom 0 - MSIs < > (XEN) 0000:7f:0f.6 - dom 0 - MSIs < > (XEN) 0000:7f:0f.5 - dom 0 - MSIs < > (XEN) 0000:7f:0f.4 - dom 0 - MSIs < > (XEN) 0000:7f:0f.3 - dom 0 - MSIs < > (XEN) 0000:7f:0f.2 - dom 0 - MSIs < > (XEN) 0000:7f:0f.1 - dom 0 - MSIs < > (XEN) 0000:7f:0f.0 - dom 0 - MSIs < > (XEN) 0000:7f:0e.1 - dom 0 - MSIs < > (XEN) 0000:7f:0e.0 - dom 0 - MSIs < > (XEN) 0000:7f:0d.6 - dom 0 - MSIs < > (XEN) 0000:7f:0d.3 - dom 0 - MSIs < > (XEN) 0000:7f:0d.2 - dom 0 - MSIs < > (XEN) 0000:7f:0d.1 - dom 0 - MSIs < > (XEN) 0000:7f:0d.0 - dom 0 - MSIs < > (XEN) 0000:7f:0c.7 - dom 0 - MSIs < > (XEN) 0000:7f:0c.6 - dom 0 - MSIs < > (XEN) 0000:7f:0c.3 - dom 0 - MSIs < > (XEN) 0000:7f:0c.2 - dom 0 - MSIs < > (XEN) 0000:7f:0c.1 - dom 0 - MSIs < > (XEN) 0000:7f:0c.0 - dom 0 - MSIs < > (XEN) 0000:7f:0b.3 - dom 0 - MSIs < > (XEN) 0000:7f:0b.0 - dom 0 - MSIs < > (XEN) 0000:7f:0a.3 - dom 0 - MSIs < > (XEN) 0000:7f:0a.2 - dom 0 - MSIs < > (XEN) 0000:7f:0a.1 - dom 0 - MSIs < > (XEN) 0000:7f:0a.0 - dom 0 - MSIs < > (XEN) 0000:7f:09.0 - dom 0 - MSIs < > (XEN) 0000:7f:08.0 - dom 0 - MSIs < > (XEN) 0000:40:05.2 - dom 0 - MSIs < > (XEN) 0000:40:05.0 - dom 0 - MSIs < > (XEN) 0000:40:03.0 - dom 0 - MSIs < 99 > (XEN) 0000:40:01.0 - dom 0 - MSIs < 98 > (XEN) 0000:3f:13.6 - dom 0 - MSIs < > (XEN) 0000:3f:13.5 - dom 0 - MSIs < > (XEN) 0000:3f:13.4 - dom 0 - MSIs < > (XEN) 0000:3f:13.1 - dom 0 - MSIs < > (XEN) 0000:3f:13.0 - dom 0 - MSIs < > (XEN) 0000:3f:11.0 - dom 0 - MSIs < > (XEN) 0000:3f:10.7 - dom 0 - MSIs < > (XEN) 0000:3f:10.6 - dom 0 - MSIs < > (XEN) 0000:3f:10.5 - dom 0 - MSIs < > (XEN) 0000:3f:10.4 - dom 0 - MSIs < > (XEN) 0000:3f:10.3 - dom 0 - MSIs < > (XEN) 0000:3f:10.2 - dom 0 - MSIs < > (XEN) 0000:3f:10.1 - dom 0 - MSIs < > (XEN) 0000:3f:10.0 - dom 0 - MSIs < > (XEN) 0000:3f:0f.6 - dom 0 - MSIs < > (XEN) 0000:3f:0f.5 - dom 0 - MSIs < > (XEN) 0000:3f:0f.4 - dom 0 - MSIs < > (XEN) 0000:3f:0f.3 - dom 0 - MSIs < > (XEN) 0000:3f:0f.2 - dom 0 - MSIs < > (XEN) 0000:3f:0f.1 - dom 0 - MSIs < > (XEN) 0000:3f:0f.0 - dom 0 - MSIs < > (XEN) 0000:3f:0e.1 - dom 0 - MSIs < > (XEN) 0000:3f:0e.0 - dom 0 - MSIs < > (XEN) 0000:3f:0d.6 - dom 0 - MSIs < > (XEN) 0000:3f:0d.3 - dom 0 - MSIs < > (XEN) 0000:3f:0d.2 - dom 0 - MSIs < > (XEN) 0000:3f:0d.1 - dom 0 - MSIs < > (XEN) 0000:3f:0d.0 - dom 0 - MSIs < > (XEN) 0000:3f:0c.7 - dom 0 - MSIs < > (XEN) 0000:3f:0c.6 - dom 0 - MSIs < > (XEN) 0000:3f:0c.3 - dom 0 - MSIs < > (XEN) 0000:3f:0c.2 - dom 0 - MSIs < > (XEN) 0000:3f:0c.1 - dom 0 - MSIs < > (XEN) 0000:3f:0c.0 - dom 0 - MSIs < > (XEN) 0000:3f:0b.3 - dom 0 - MSIs < > (XEN) 0000:3f:0b.0 - dom 0 - MSIs < > (XEN) 0000:3f:0a.3 - dom 0 - MSIs < > (XEN) 0000:3f:0a.2 - dom 0 - MSIs < > (XEN) 0000:3f:0a.1 - dom 0 - MSIs < > (XEN) 0000:3f:0a.0 - dom 0 - MSIs < > (XEN) 0000:3f:09.0 - dom 0 - MSIs < > (XEN) 0000:3f:08.0 - dom 0 - MSIs < > (XEN) 0000:0b:00.0 - dom 0 - MSIs < > (XEN) 0000:0a:00.0 - dom 0 - MSIs < > (XEN) 0000:09:01.0 - dom 0 - MSIs < > (XEN) 0000:09:00.0 - dom 0 - MSIs < > (XEN) 0000:08:00.0 - dom 0 - MSIs < > (XEN) 0000:07:00.1 - dom 0 - MSIs < > (XEN) 0000:07:00.0 - dom 0 - MSIs < > (XEN) 0000:02:00.0 - dom 0 - MSIs < 100 101 102 103 104 105 106 107 > (XEN) 0000:01:00.1 - dom 0 - MSIs < 111 112 120 121 122 > (XEN) 0000:01:00.0 - dom 0 - MSIs < 109 110 113 114 115 > (XEN) 0000:00:1f.2 - dom 0 - MSIs < 108 > (XEN) 0000:00:1f.0 - dom 0 - MSIs < > (XEN) 0000:00:1e.0 - dom 0 - MSIs < > (XEN) 0000:00:1d.0 - dom 0 - MSIs < > (XEN) 0000:00:1c.7 - dom 0 - MSIs < 97 > (XEN) 0000:00:1c.4 - dom 0 - MSIs < 96 > (XEN) 0000:00:1c.0 - dom 0 - MSIs < 95 > (XEN) 0000:00:1a.0 - dom 0 - MSIs < > (XEN) 0000:00:16.1 - dom 0 - MSIs < > (XEN) 0000:00:16.0 - dom 0 - MSIs < > (XEN) 0000:00:11.0 - dom 0 - MSIs < 94 > (XEN) 0000:00:05.2 - dom 0 - MSIs < > (XEN) 0000:00:05.0 - dom 0 - MSIs < > (XEN) 0000:00:03.0 - dom 0 - MSIs < 93 > (XEN) 0000:00:02.2 - dom 0 - MSIs < 92 > (XEN) 0000:00:02.0 - dom 0 - MSIs < 91 > (XEN) 0000:00:01.0 - dom 0 - MSIs < 90 > (XEN) 0000:00:00.0 - dom 0 - MSIs < > (XEN) MSI information: (XEN) IOMMU 88 vec=28 fixed edge assert phys cpu dest=00000000 mask=1/0/? (XEN) IOMMU 89 vec=30 fixed edge assert phys cpu dest=00000000 mask=1/0/? (XEN) MSI 90 vec=d8 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI 91 vec=21 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI 92 vec=29 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI 93 vec=31 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI 94 vec=39 fixed edge assert phys cpu dest=00000000 mask=0/1/? (XEN) MSI 95 vec=41 fixed edge assert phys cpu dest=00000000 mask=0/1/? (XEN) MSI 96 vec=49 fixed edge assert phys cpu dest=00000000 mask=0/1/? (XEN) MSI 97 vec=51 fixed edge assert phys cpu dest=00000000 mask=0/1/? (XEN) MSI 98 vec=59 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI 99 vec=61 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 100 vec=79 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 101 vec=81 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 102 vec=89 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 103 vec=91 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 104 vec=99 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 105 vec=a1 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 106 vec=a9 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 107 vec=b1 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI 108 vec=c1 fixed edge assert phys cpu dest=00000000 mask=0/1/? (XEN) MSI-X 109 vec=ba fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 110 vec=c2 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 111 vec=7a fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 112 vec=8a fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 113 vec=ca fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 114 vec=d2 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 115 vec=da fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 120 vec=92 fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 121 vec=9a fixed edge assert phys cpu dest=00000000 mask=1/0/0 (XEN) MSI-X 122 vec=a2 fixed edge assert phys cpu dest=00000000 mask=1/0/0 On 04/10/2013, at 11.53.41, Andrew Cooper <andrew.cooper3@citrix.com> wrote:> On 04/10/13 08:44, Kristoffer Egefelt wrote: > > Can you printk() in xen/arch/x86/msi.c:pci_restore_msi_state() just before the list_for_each_entry_safe() and work out which is the problematic pci device, then run `xl debug-keys iQM` and provide the output. Perhaps an lspci of the affected device might also help. > > ~Andrew > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Kristoffer Egefelt
2013-Oct-07 10:19 UTC
Re: [BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
Do you still need a bisect, or is Andrews:> This Xen crash appears to be as a result of d1b6d0a0248 "x86: enable multi-vector MSI", and is likely a mishanding of some error state caused by the initial pci attach failure/partial setup.enough ? Thanks Regards Kristoffer On 04/10/2013, at 11.30.38, David Vrabel <david.vrabel@citrix.com> wrote:> On 04/10/13 08:44, Kristoffer Egefelt wrote: >> Hi, >> >> I''m trying to pass through a NIC (intel X520 with ixgbevf driver) to domU, but since kernel 3.8 this has not worked. >> >> The dom0 kernel seems to cause the problem. >> Xen version, domU kernel version and driver version seems to be unrelated to this bug, meaning >> it works as long as dom0 kernel is 3.8. >> I tried kernel version 3.9, 3.10 and 3.11 - all show the same bug pattern when used as dom0. > > Can you do a git bisect on the kernel to find which kernel commit > triggers this crash? > > This is clearly a Xen bug but it would be useful to know what triggered it. > > Thanks. > > David > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
David Vrabel
2013-Oct-07 11:59 UTC
Re: [BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
On 07/10/13 11:19, Kristoffer Egefelt wrote:> Do you still need a bisect, or is Andrews: > >> This Xen crash appears to be as a result of d1b6d0a0248 "x86: enable >> multi-vector MSI", and is likely a mishanding of some error state >> caused by the initial pci attach failure/partial setup. > > enough ?The Xen bug has been identified but as Jan notes, this shouldn''t have been triggered by the kernel so it would be useful to know why. David
Kristoffer Egefelt
2013-Oct-09 14:06 UTC
Re: [BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
Here''s the bisect (my first - hope it looks allright) - let me know if
you need anything else.
root@builder:/k/linux-stable# git bisect log
git bisect start
# bad: [00cfbb8ad0a6419f40660362b4d8b5baa30d3efe] Linux 3.8.6
git bisect bad 00cfbb8ad0a6419f40660362b4d8b5baa30d3efe
# good: [aa4cfdeb17e3559fe1e80175c7338e065553dce5] Linux 3.8.5
git bisect good aa4cfdeb17e3559fe1e80175c7338e065553dce5
# bad: [dfa7981463bbf902057c7c2b36c1e9e98f1651c4] USB: mos7840: fix broken
TIOCMIWAIT
git bisect bad dfa7981463bbf902057c7c2b36c1e9e98f1651c4
# bad: [f80725b652aca0871ea2cc8b2e037edf67de7630] usb: ftdi_sio: Add support for
Mitsubishi FX-USB-AW/-BD
git bisect bad f80725b652aca0871ea2cc8b2e037edf67de7630
# good: [4911ece4056be5d62a8a3db13d80bf86166bcf7a] HID: usbhid: quirk for
Realtek Multi-card reader
git bisect good 4911ece4056be5d62a8a3db13d80bf86166bcf7a
# good: [a3f6fd302bab90e11c1d7952c3565e3f3c244824] tg3: fix length overflow in
VPD firmware parsing
git bisect good a3f6fd302bab90e11c1d7952c3565e3f3c244824
# bad: [ccaa606d2301cb33da35a463951e1ed117d927cc] xen/blkback: correctly respond
to unknown, non-native requests
git bisect bad ccaa606d2301cb33da35a463951e1ed117d927cc
# good: [e131628113867748a05c9080b9aaf67740d24057] iommu/amd: Make sure dma_ops
are set for hotplug devices
git bisect good e131628113867748a05c9080b9aaf67740d24057
# bad: [a420a7e3a40a8396104d9a62f676205f2db1a671] xen-pciback: notify hypervisor
about devices intended to be assigned to guests
git bisect bad a420a7e3a40a8396104d9a62f676205f2db1a671
# bad: [a420a7e3a40a8396104d9a62f676205f2db1a671] xen-pciback: notify hypervisor
about devices intended to be assigned to guests
git bisect bad a420a7e3a40a8396104d9a62f676205f2db1a671
# bad: [a420a7e3a40a8396104d9a62f676205f2db1a671] xen-pciback: notify hypervisor
about devices intended to be assigned to guests
git bisect bad a420a7e3a40a8396104d9a62f676205f2db1a671
# bad: [a420a7e3a40a8396104d9a62f676205f2db1a671] xen-pciback: notify hypervisor
about devices intended to be assigned to guests
git bisect bad a420a7e3a40a8396104d9a62f676205f2db1a671
root@builder:/k/linux-stable# git bisect bad
a420a7e3a40a8396104d9a62f676205f2db1a671 is the first bad commit
commit a420a7e3a40a8396104d9a62f676205f2db1a671
Author: Jan Beulich <JBeulich@suse.com>
Date: Tue Mar 12 15:06:23 2013 +0000
xen-pciback: notify hypervisor about devices intended to be assigned to
guests
commit 909b3fdb0dd4f3db07b2d75425a00a2adb551383 upstream.
For MSI-X capable devices the hypervisor wants to write protect the
MSI-X table and PBA, yet it can''t assume that resources have been
assigned to their final values at device enumeration time. Thus have
pciback do that notification, as having the device controlled by it is
a prerequisite to assigning the device to guests anyway.
This is the kernel part of hypervisor side commit 4245d33 ("x86/MSI:
add mechanism to fully protect MSI-X table from PV guest accesses") on
the master branch of git://xenbits.xen.org/xen.git.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
:040000 040000 431896b0135448c76f8787f6cd31596228db71f0
2b62653093dbf5de13f7c33ea65b61506fb8a557 M arch
:040000 040000 e6b4f73c9d28c8912870e3a5e271b9dc45c2a4b4
d042d0455b9ae00401cb6307f6ed5bfe28d98611 M drivers
:040000 040000 ffe88a72f395700149e7f6d2b982d91331a7dd78
11b587a7d3b26f6f443fd44f36a9c2be8bc464f1 M include
On 07/10/2013, at 13.59.13, David Vrabel <david.vrabel@citrix.com> wrote:
> On 07/10/13 11:19, Kristoffer Egefelt wrote:
>> Do you still need a bisect, or is Andrews:
>>
>>> This Xen crash appears to be as a result of d1b6d0a0248 "x86:
enable
>>> multi-vector MSI", and is likely a mishanding of some error
state
>>> caused by the initial pci attach failure/partial setup.
>>
>> enough ?
>
> The Xen bug has been identified but as Jan notes, this shouldn''t
have
> been triggered by the kernel so it would be useful to know why.
>
> David
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>
David Vrabel
2013-Oct-09 14:39 UTC
Re: [BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
On 04/10/13 11:05, Jan Beulich wrote:>>>> On 04.10.13 at 09:44, Kristoffer Egefelt <kristoffer@itoc.dk> wrote: >> Hi, >> >> I''m trying to pass through a NIC (intel X520 with ixgbevf driver) to domU, >> but since kernel 3.8 this has not worked. >> >> The dom0 kernel seems to cause the problem. >> Xen version, domU kernel version and driver version seems to be unrelated to >> this bug, meaning >> it works as long as dom0 kernel is 3.8. >> I tried kernel version 3.9, 3.10 and 3.11 - all show the same bug pattern >> when used as dom0. >> >> The BUG appears on xl pci attach. >> On pci detach the dom0 panics. >> >> I have attached logs from a working setup (kernel 3.8) and from a setup not >> working (kernel 3.11) and also the kernel config for 3.11. >> >> In short, this is what domU logs after pci attach: >> >> BUG: unable to handle kernel paging request at ffffc9000030200c >> IP: [<ffffffff81205812>] __msix_mask_irq+0x21/0x24 >> PGD 75a40067 PUD 75a41067 PMD 75b44067 PTE 8010000000000464 >> Oops: 0003 [#1] SMP >> Modules linked in: ixgbevf(+) xen_pcifront nfnetlink_log nfnetlink ipt_ULOG >> x_tables x86_pkg_temp_thermal thermal_sys coretemp crc32c_intel >> ghash_clmulni_intel aesni_intel aes_x86_64 ablk_helper cryptd lrw gf128mul >> glue_helper microcode ext4 crc16 jbd2 mbcache xen_blkfront >> CPU: 0 PID: 2122 Comm: modprobe Not tainted 3.11.3-kernel-v1.0.0.21+ #1 > > Are you certain this is kernel (rather than hypervisor) version > dependent? Iirc this is a manifestation of a guest kernel not being > permitted to write to the MSI-X mask bit. > >> And this is dom0 on pci detach: >> >> (XEN) Assertion ''_raw_spin_is_locked(lock)'' failed at >> /usr/src/xen/xen/include/asm/spinlock.h:16402 >> (XEN) ----[ Xen-4.4-unstable x86_64 debug=y Not tainted ]---- >> (XEN) CPU: 1 >> (XEN) RIP: e008:[<ffff82d0801258ef>] _spin_unlock_irqrestore+0x27/0x32 >> (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor >> (XEN) rax: 0000000000000001 rbx: ffff83201ba07724 rcx: 0000000000000001 >> (XEN) rdx: ffff83201bb97020 rsi: 0000000000000286 rdi: ffff83201ba07724 >> (XEN) rbp: ffff83203ffcfdd8 rsp: ffff83203ffcfdd8 r8: ffff8141002000e0 >> (XEN) r9: 000000000000001c r10: 0000000000000082 r11: 0000000000000001 >> (XEN) r12: 0000000000000000 r13: ffff8320e13c8240 r14: ffff880148047df4 >> (XEN) r15: 0000000000000286 cr0: 0000000080050033 cr4: 00000000000426f0 >> (XEN) cr3: 000000206f3ff000 cr2: 00007fa5ec560c49 >> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 >> (XEN) Xen stack trace from rsp=ffff83203ffcfdd8: >> (XEN) ffff83203ffcfe68 ffff82d080166a4f ffff83203ffcfe18 0000000280118988 >> (XEN) 0000000000000cfe 0000000000000cfe ffff832015d3b8a0 ffff8320e13c83f0 >> (XEN) ffff832015d3b880 0000000000000001 00000000fee00678 0000000000000000 >> (XEN) ffff83200000f800 000000000000001b ffff8300bcef5000 ffffffffffffffed >> (XEN) ffff880148047df4 ffffffff814530e0 ffff83203ffcfef8 ffff82d08017dee4 >> (XEN) ffff832000000002 0000000000000008 ffff83203ffcfef8 ffff82d000a0fb00 >> (XEN) 0000000000000000 ffffffff93010000 ffff82d0802e8000 ffff83203ffc80ef >> (XEN) 82d080222c00b948 c390ef66d1ffffff ffff83203ffcfef8 ffff8300bcef5000 >> (XEN) ffff880145951868 ffff880145bb2a60 ffff880148047f50 ffffffff814530e0 >> (XEN) 00007cdfc00300c7 ffff82d08022213b ffffffff8100142a 0000000000000021 >> (XEN) ffffffff814530e0 ffff880148047f50 000000000000c002 0000000000009300 >> (XEN) ffff88013faf1a80 ffff880145951000 0000000000000202 0000000000000093 >> (XEN) ffff880148047df4 0000000000000002 0000000000000021 ffffffff8100142a >> (XEN) 0000000000000000 ffff880148047df4 000000000000001b 0001010000000000 >> (XEN) ffffffff8100142a 000000000000e033 0000000000000202 ffff880148047dc8 >> (XEN) 000000000000e02b 0000000000000000 0000000000000000 0000000000000000 >> (XEN) 0000000000000000 0000000000000001 ffff8300bcef5000 0000004f9b885e00 >> (XEN) 0000000000000000 >> (XEN) Xen call trace: >> (XEN) [<ffff82d0801258ef>] _spin_unlock_irqrestore+0x27/0x32 >> (XEN) [<ffff82d080166a4f>] pci_restore_msi_state+0x1c9/0x2f0 >> (XEN) [<ffff82d08017dee4>] do_physdev_op+0xe4f/0x114f >> (XEN) [<ffff82d08022213b>] syscall_enter+0xeb/0x145 >> (XEN) >> (XEN) >> (XEN) **************************************** >> (XEN) Panic on CPU 1: >> (XEN) Assertion ''_raw_spin_is_locked(lock)'' failed at >> /usr/src/xen/xen/include/asm/spinlock.h:16402 >> (XEN) **************************************** >> (XEN) >> (XEN) Manual reset required (''noreboot'' specified) > > This, otoh, is clearly a hypervisor bug. Afaict the patch below > should help. > > But - this code is supposed to be executed on host S3 resume only > (i.e. there might also be some kernel flaw involved here).It''s called from pci_restore_state() which is called from pciback when a device is released. This doesn''t seem unreasonable to me. David
Kristoffer Egefelt
2013-Oct-09 21:18 UTC
Re: [BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
Let me know if I can do anything to assist. Thanks On 09/10/2013, at 16.39.42, David Vrabel <david.vrabel@citrix.com> wrote:> On 04/10/13 11:05, Jan Beulich wrote: >>>>> On 04.10.13 at 09:44, Kristoffer Egefelt <kristoffer@itoc.dk> wrote: >>> Hi, >>> >>> I''m trying to pass through a NIC (intel X520 with ixgbevf driver) to domU, >>> but since kernel 3.8 this has not worked. >>> >>> The dom0 kernel seems to cause the problem. >>> Xen version, domU kernel version and driver version seems to be unrelated to >>> this bug, meaning >>> it works as long as dom0 kernel is 3.8. >>> I tried kernel version 3.9, 3.10 and 3.11 - all show the same bug pattern >>> when used as dom0. >>> >>> The BUG appears on xl pci attach. >>> On pci detach the dom0 panics. >>> >>> I have attached logs from a working setup (kernel 3.8) and from a setup not >>> working (kernel 3.11) and also the kernel config for 3.11. >>> >>> In short, this is what domU logs after pci attach: >>> >>> BUG: unable to handle kernel paging request at ffffc9000030200c >>> IP: [<ffffffff81205812>] __msix_mask_irq+0x21/0x24 >>> PGD 75a40067 PUD 75a41067 PMD 75b44067 PTE 8010000000000464 >>> Oops: 0003 [#1] SMP >>> Modules linked in: ixgbevf(+) xen_pcifront nfnetlink_log nfnetlink ipt_ULOG >>> x_tables x86_pkg_temp_thermal thermal_sys coretemp crc32c_intel >>> ghash_clmulni_intel aesni_intel aes_x86_64 ablk_helper cryptd lrw gf128mul >>> glue_helper microcode ext4 crc16 jbd2 mbcache xen_blkfront >>> CPU: 0 PID: 2122 Comm: modprobe Not tainted 3.11.3-kernel-v1.0.0.21+ #1 >> >> Are you certain this is kernel (rather than hypervisor) version >> dependent? Iirc this is a manifestation of a guest kernel not being >> permitted to write to the MSI-X mask bit. >> >>> And this is dom0 on pci detach: >>> >>> (XEN) Assertion ''_raw_spin_is_locked(lock)'' failed at >>> /usr/src/xen/xen/include/asm/spinlock.h:16402 >>> (XEN) ----[ Xen-4.4-unstable x86_64 debug=y Not tainted ]---- >>> (XEN) CPU: 1 >>> (XEN) RIP: e008:[<ffff82d0801258ef>] _spin_unlock_irqrestore+0x27/0x32 >>> (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor >>> (XEN) rax: 0000000000000001 rbx: ffff83201ba07724 rcx: 0000000000000001 >>> (XEN) rdx: ffff83201bb97020 rsi: 0000000000000286 rdi: ffff83201ba07724 >>> (XEN) rbp: ffff83203ffcfdd8 rsp: ffff83203ffcfdd8 r8: ffff8141002000e0 >>> (XEN) r9: 000000000000001c r10: 0000000000000082 r11: 0000000000000001 >>> (XEN) r12: 0000000000000000 r13: ffff8320e13c8240 r14: ffff880148047df4 >>> (XEN) r15: 0000000000000286 cr0: 0000000080050033 cr4: 00000000000426f0 >>> (XEN) cr3: 000000206f3ff000 cr2: 00007fa5ec560c49 >>> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 >>> (XEN) Xen stack trace from rsp=ffff83203ffcfdd8: >>> (XEN) ffff83203ffcfe68 ffff82d080166a4f ffff83203ffcfe18 0000000280118988 >>> (XEN) 0000000000000cfe 0000000000000cfe ffff832015d3b8a0 ffff8320e13c83f0 >>> (XEN) ffff832015d3b880 0000000000000001 00000000fee00678 0000000000000000 >>> (XEN) ffff83200000f800 000000000000001b ffff8300bcef5000 ffffffffffffffed >>> (XEN) ffff880148047df4 ffffffff814530e0 ffff83203ffcfef8 ffff82d08017dee4 >>> (XEN) ffff832000000002 0000000000000008 ffff83203ffcfef8 ffff82d000a0fb00 >>> (XEN) 0000000000000000 ffffffff93010000 ffff82d0802e8000 ffff83203ffc80ef >>> (XEN) 82d080222c00b948 c390ef66d1ffffff ffff83203ffcfef8 ffff8300bcef5000 >>> (XEN) ffff880145951868 ffff880145bb2a60 ffff880148047f50 ffffffff814530e0 >>> (XEN) 00007cdfc00300c7 ffff82d08022213b ffffffff8100142a 0000000000000021 >>> (XEN) ffffffff814530e0 ffff880148047f50 000000000000c002 0000000000009300 >>> (XEN) ffff88013faf1a80 ffff880145951000 0000000000000202 0000000000000093 >>> (XEN) ffff880148047df4 0000000000000002 0000000000000021 ffffffff8100142a >>> (XEN) 0000000000000000 ffff880148047df4 000000000000001b 0001010000000000 >>> (XEN) ffffffff8100142a 000000000000e033 0000000000000202 ffff880148047dc8 >>> (XEN) 000000000000e02b 0000000000000000 0000000000000000 0000000000000000 >>> (XEN) 0000000000000000 0000000000000001 ffff8300bcef5000 0000004f9b885e00 >>> (XEN) 0000000000000000 >>> (XEN) Xen call trace: >>> (XEN) [<ffff82d0801258ef>] _spin_unlock_irqrestore+0x27/0x32 >>> (XEN) [<ffff82d080166a4f>] pci_restore_msi_state+0x1c9/0x2f0 >>> (XEN) [<ffff82d08017dee4>] do_physdev_op+0xe4f/0x114f >>> (XEN) [<ffff82d08022213b>] syscall_enter+0xeb/0x145 >>> (XEN) >>> (XEN) >>> (XEN) **************************************** >>> (XEN) Panic on CPU 1: >>> (XEN) Assertion ''_raw_spin_is_locked(lock)'' failed at >>> /usr/src/xen/xen/include/asm/spinlock.h:16402 >>> (XEN) **************************************** >>> (XEN) >>> (XEN) Manual reset required (''noreboot'' specified) >> >> This, otoh, is clearly a hypervisor bug. Afaict the patch below >> should help. >> >> But - this code is supposed to be executed on host S3 resume only >> (i.e. there might also be some kernel flaw involved here). > > It''s called from pci_restore_state() which is called from pciback when a > device is released. This doesn''t seem unreasonable to me. > > David_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Jan Beulich
2013-Oct-10 10:39 UTC
Re: [BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
>>> On 09.10.13 at 23:18, Kristoffer Egefelt <kristoffer@itoc.dk> wrote: > Let me know if I can do anything to assist.I am still awaiting your feedback on the effect the hypervisor side patch (which I sent you almost a week ago) had on the hypervisor crash you had observed in this context. Jan
Kristoffer Egefelt
2013-Oct-11 08:22 UTC
Re: [BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
Ah, I thought you meant that it worked, not that it needed testing ;-) It works with your patch, dom0 do not crash anymore - it looks like this: root@builder:/k# xl pci-attach vm ''0000:01:12.3,permissive=1'' [ 1363.048348] pciback 0000:01:12.3: enabling permissive mode configuration space accesses! [ 1363.229314] pciback 0000:01:12.3: permissive mode is potentially unsafe! [ 1363.877218] xen_pciback: vpci: 0000:01:12.3: assign to virtual slot 0 root@builder:/k# [ 1364.077427] pciback 0000:01:12.3: enabling device (0000 -> 0002) [ 1364.255582] ixgbe 0000:01:00.1 eth1: VF Reset msg received from vf 9 [ 1364.256144] ixgbe 0000:01:00.1: VF 9 has no MAC address assigned, you may have to assign one manually (XEN) mm.c:807: d1: Forcing read-only access to MFN d5324 (XEN) mm.c:807: d0: Forcing read-only access to MFN d5324 (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 (XEN) mm.c:618:d1 Could not get page ref for pfn d5324 root@builder:/k# xl destroy vm (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 libxl: error: libxl_device.c:1127:libxl__wait_for_backend: Backend /local/domain/0/backend/pci/1/0 not ready (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 On 10/10/2013, at 12.39.37, Jan Beulich <JBeulich@suse.com> wrote:>>>> On 09.10.13 at 23:18, Kristoffer Egefelt <kristoffer@itoc.dk> wrote: >> Let me know if I can do anything to assist. > > I am still awaiting your feedback on the effect the hypervisor side > patch (which I sent you almost a week ago) had on the hypervisor > crash you had observed in this context. > > Jan > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Kristoffer Egefelt
2013-Oct-14 10:13 UTC
Re: [BUG] pci-passthrough on dom0 kernel versions above 3.8 crashes dom0
Hi Jan, about the kernel bug - I''d be happy to help testing this :-) Thanks Kristoffer On 11/10/2013, at 10.22.07, Kristoffer Egefelt <kristoffer@itoc.dk> wrote:> Ah, I thought you meant that it worked, not that it needed testing ;-) > It works with your patch, dom0 do not crash anymore - it looks like this: > > root@builder:/k# xl pci-attach vm ''0000:01:12.3,permissive=1'' > [ 1363.048348] pciback 0000:01:12.3: enabling permissive mode configuration space accesses! > [ 1363.229314] pciback 0000:01:12.3: permissive mode is potentially unsafe! > [ 1363.877218] xen_pciback: vpci: 0000:01:12.3: assign to virtual slot 0 > root@builder:/k# [ 1364.077427] pciback 0000:01:12.3: enabling device (0000 -> 0002) > [ 1364.255582] ixgbe 0000:01:00.1 eth1: VF Reset msg received from vf 9 > [ 1364.256144] ixgbe 0000:01:00.1: VF 9 has no MAC address assigned, you may have to assign one manually > (XEN) mm.c:807: d1: Forcing read-only access to MFN d5324 > (XEN) mm.c:807: d0: Forcing read-only access to MFN d5324 > (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 > (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 > (XEN) mm.c:618:d1 Could not get page ref for pfn d5324 > > > > root@builder:/k# xl destroy vm > (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 > (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 > libxl: error: libxl_device.c:1127:libxl__wait_for_backend: Backend /local/domain/0/backend/pci/1/0 not ready > (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 > (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 > (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 > (XEN) mm.c:618:d0 Could not get page ref for pfn d5324 > > > > > > > > > On 10/10/2013, at 12.39.37, Jan Beulich <JBeulich@suse.com> wrote: > >>>>> On 09.10.13 at 23:18, Kristoffer Egefelt <kristoffer@itoc.dk> wrote: >>> Let me know if I can do anything to assist. >> >> I am still awaiting your feedback on the effect the hypervisor side >> patch (which I sent you almost a week ago) had on the hypervisor >> crash you had observed in this context. >> >> Jan >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel