Masroor Vettuparambil
2010-Jun-11 13:10 UTC
[Xen-devel] MSIX allocation failure on 2.6.31.13 PV domU
Hi, I am trying pci pass-through on PV DomU with xen4.0+Dom0 2.6.31.13. While loading the driver on DomU with MSIX, I am getting the below oops message. I have passed "iommu=soft" and "swiotlb=force" to DomU kernel. I can see many related patches applied to 2.6.32.x, but not to 2.6.31.x. On my setup I can run the driver with MSIX/INTA on Dom0 and with INTA on DomU. So can someone confirm that MSIX is working with 2.6.31.x? Also I gave a try on 2.6.32.x Dom0 as well, but unfortunately I am stuck with xend failure which I can see a discussion going on a different thread.( http://lists.xensource.com/archives/html/xen-devel/2010-06/msg00530.html ) The Oops message is, [ 3548.303800] BUG: unable to handle kernel paging request at bfabdca0 [ 3548.303899] IP: [<bfabdca0>] 0xbfabdca0 [ 3548.303964] *pdpt = 000000001f2de001 [ 3548.304026] Oops: 0010 [#1] SMP [ 3548.304108] last sysfs file: /sys/devices/pci-0/pci0000:00/0000:00:00.0/class [ 3548.304150] Modules linked in: vxge [ 3548.304233] [ 3548.304269] Pid: 3089, comm: ip Not tainted (2.6.31.13 #1) [ 3548.304309] EIP: 0061:[<bfabdca0>] EFLAGS: 00010286 CPU: 0 [ 3548.304346] EIP is at 0xbfabdca0 [ 3548.304381] EAX: ccd69400 EBX: ccd69400 ECX: ccd69400 EDX: bfabdca0 [ 3548.304419] ESI: ccd69400 EDI: ccd69400 EBP: c90c56a0 ESP: c90c569c [ 3548.304457] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0069 [ 3548.304495] Process ip (pid: 3089, ti=c90c4000 task=cf961900 task.ti=c90c4000) [ 3548.304534] Stack: [ 3548.304567] c1025106 c90c56b8 c103beef c103c017 ccd6976c ccd69400 ccd69400 c90c56d4 [ 3548.304712] <0> c11c90af c11c93aa ccd6977c ccde51d0 ccd69400 ffffffff c90c5714 c11c9662 [ 3548.304712] <0> 00000003 cccd3ac0 000280d0 00000044 d0a56000 000000a0 ccd6977c 000000a2 [ 3548.304712] Call Trace: [ 3548.304712] [<c1025106>] ? xen_teardown_msi_dev+0x42/0x44 [ 3548.304712] [<c103beef>] ? arch_teardown_msi_irqs+0x19/0x6f [ 3548.304712] [<c103c017>] ? arch_setup_msi_irqs+0x30/0x106 [ 3548.304712] [<c11c90af>] ? msi_free_irqs+0x70/0xcc [ 3548.304712] [<c11c93aa>] ? alloc_msi_entry+0x1b/0x71 [ 3548.304712] [<c11c9662>] ? pci_enable_msix+0x262/0x354 [ 3548.304712] [<d0b3053a>] ? vxge_open+0x264/0x1483 [vxge] [ 3548.304712] [<c1024dde>] ? __xen_spin_lock+0xc4/0xd8 [ 3548.304712] [<c1023b03>] ? xen_force_evtchn_callback+0xf/0x14 [ 3548.304712] [<c1023b03>] ? xen_force_evtchn_callback+0xf/0x14 [ 3548.304712] [<c1021982>] ? __raw_callee_save_xen_pmd_val+0x6/0x8 [ 3548.304712] [<c1043791>] ? lookup_address+0xea/0x10b [ 3548.304712] [<c1023b03>] ? xen_force_evtchn_callback+0xf/0x14 [ 3548.304712] [<c10241e0>] ? check_events+0x8/0xc [ 3548.304712] [<c10241d7>] ? xen_restore_fl_direct_end+0x0/0x1 [ 3548.304712] [<c102180e>] ? xen_mc_flush+0x1fa/0x215 [ 3548.304712] [<c10241e0>] ? check_events+0x8/0xc [ 3548.304712] [<c10241d7>] ? xen_restore_fl_direct_end+0x0/0x1 [ 3548.304712] [<c1020d73>] ? xen_end_context_switch+0x12/0x15 [ 3548.304712] [<c10267ae>] ? __switch_to+0x113/0x144 [ 3548.304712] [<c105226e>] ? finish_task_switch+0x77/0xe7 [ 3548.304712] [<c14d9f69>] ? schedule+0x7b5/0x842 [ 3548.304712] [<c104d2a2>] ? update_curr+0x94/0x1bb [ 3548.304712] [<c104f326>] ? enqueue_entity+0x25e/0x266 [ 3548.304712] [<c1023b03>] ? xen_force_evtchn_callback+0xf/0x14 [ 3548.304712] [<c10241e0>] ? check_events+0x8/0xc [ 3548.304712] [<c10241d7>] ? xen_restore_fl_direct_end+0x0/0x1 [ 3548.304712] [<c14db424>] ? _spin_unlock_irqrestore+0x43/0x48 [ 3548.304712] [<c1054278>] ? try_to_wake_up+0x298/0x2a2 [ 3548.304712] [<c105428d>] ? default_wake_function+0xb/0xd [ 3548.304712] [<c10e5882>] ? pollwake+0x59/0x63 [ 3548.304712] [<c1024dde>] ? __xen_spin_lock+0xc4/0xd8 [ 3548.304712] [<c1023b03>] ? xen_force_evtchn_callback+0xf/0x14 [ 3548.304712] [<c10241d7>] ? xen_restore_fl_direct_end+0x0/0x1 [ 3548.304712] [<c14db424>] ? _spin_unlock_irqrestore+0x43/0x48 [ 3548.304712] [<c104ce3c>] ? __wake_up+0x31/0x3b [ 3548.304712] [<c13f6462>] ? netlink_broadcast+0x221/0x252 [ 3548.304712] [<c13f6952>] ? nlmsg_notify+0x3f/0x78 [ 3548.304712] [<c1429c07>] ? inetdev_event+0x1c/0x39f [ 3548.304712] [<c147738d>] ? packet_notifier+0x21/0x16c [ 3548.304712] [<c14dd475>] ? notifier_call_chain+0x2b/0x4a [ 3548.304712] [<c13e1707>] ? dev_open+0x93/0xce [ 3548.304712] [<c13e0f7b>] ? dev_change_flags+0xa5/0x158 [ 3548.304712] [<c13e8149>] ? do_setlink+0x262/0x31a [ 3548.304712] [<c13e9293>] ? rtnl_newlink+0x27e/0x3f9 [ 3548.304712] [<c13e9050>] ? rtnl_newlink+0x3b/0x3f9 [ 3548.304712] [<c13e909a>] ? rtnl_newlink+0x85/0x3f9 [ 3548.304712] [<c1181df3>] ? selinux_netlink_recv+0x46/0x50 [ 3548.304712] [<c13e9015>] ? rtnl_newlink+0x0/0x3f9 [ 3548.304712] [<c13e8ffb>] ? rtnetlink_rcv_msg+0x1a0/0x1ba [ 3548.304712] [<c13e8e5b>] ? rtnetlink_rcv_msg+0x0/0x1ba [ 3548.304712] [<c13f6a89>] ? netlink_rcv_skb+0x30/0x77 [ 3548.304712] [<c13e8e53>] ? rtnetlink_rcv+0x1c/0x24 [ 3548.304712] [<c13f666d>] ? netlink_unicast+0xc4/0x11a [ 3548.304712] [<c13f6906>] ? netlink_sendmsg+0x243/0x250 [ 3548.304712] [<c13d327f>] ? __sock_sendmsg+0x45/0x4e [ 3548.304712] [<c13d3a07>] ? sock_sendmsg+0xb8/0xce [ 3548.304712] [<c106b3df>] ? autoremove_wake_function+0x0/0x33 [ 3548.304712] [<c106b3df>] ? autoremove_wake_function+0x0/0x33 [ 3548.304712] [<c1023b03>] ? xen_force_evtchn_callback+0xf/0x14 [ 3548.304712] [<c10ae581>] ? find_get_page+0x1d/0x81 [ 3548.304712] [<c11b6ff1>] ? copy_from_user+0x2a/0x111 [ 3548.304712] [<c13dad63>] ? verify_iovec+0x40/0x71 [ 3548.304712] [<c13d3b5c>] ? sys_sendmsg+0x13f/0x192 [ 3548.304712] [<c10c2974>] ? handle_mm_fault+0x481/0xa8e [ 3548.304712] [<c10c63a1>] ? vma_link+0x53/0x94 [ 3548.304712] [<c13d4b7c>] ? sys_socketcall+0x153/0x183 [ 3548.304712] [<c1027ac9>] ? syscall_call+0x7/0xb [ 3548.304712] Code: Bad EIP value. [ 3548.304712] EIP: [<bfabdca0>] 0xbfabdca0 SS:ESP 0069:c90c569c [ 3548.304712] CR2: 00000000bfabdca0 [ 3548.304712] ---[ end trace 79e784140187edbe ]--- _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Masroor Vettuparambil
2010-Jun-12 08:33 UTC
[Xen-devel] MSIX allocation failure on 2.6.31.13 PV domU
Hi, I am trying pci pass-through on PV DomU with xen4.0+Dom0 2.6.31.13. While loading the driver on DomU with MSIX, I am getting the below oops message. I have passed "iommu=soft" and "swiotlb=force" to DomU kernel. I can see many related patches applied to 2.6.32.x, but not to 2.6.31.x. On my setup I can run the driver with MSIX/INTA on Dom0 and with INTA on DomU. Is MSIX support broken in 2.6.31.x? Also I gave a try on 2.6.32.x Dom0 as well, but unfortunately I am stuck with xend failure which I can see a discussion going on a different thread.(http://lists.xensource.com/archives/html/xen-devel/2010-06/msg00 530.html ) The Oops message is, [ 3548.303800] BUG: unable to handle kernel paging request at bfabdca0 [ 3548.303899] IP: [<bfabdca0>] 0xbfabdca0 [ 3548.303964] *pdpt = 000000001f2de001 [ 3548.304026] Oops: 0010 [#1] SMP [ 3548.304108] last sysfs file: /sys/devices/pci-0/pci0000:00/0000:00:00.0/class [ 3548.304150] Modules linked in: vxge [ 3548.304233] [ 3548.304269] Pid: 3089, comm: ip Not tainted (2.6.31.13 #1) [ 3548.304309] EIP: 0061:[<bfabdca0>] EFLAGS: 00010286 CPU: 0 [ 3548.304346] EIP is at 0xbfabdca0 [ 3548.304381] EAX: ccd69400 EBX: ccd69400 ECX: ccd69400 EDX: bfabdca0 [ 3548.304419] ESI: ccd69400 EDI: ccd69400 EBP: c90c56a0 ESP: c90c569c [ 3548.304457] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0069 [ 3548.304495] Process ip (pid: 3089, ti=c90c4000 task=cf961900 task.ti=c90c4000) [ 3548.304534] Stack: [ 3548.304567] c1025106 c90c56b8 c103beef c103c017 ccd6976c ccd69400 ccd69400 c90c56d4 [ 3548.304712] <0> c11c90af c11c93aa ccd6977c ccde51d0 ccd69400 ffffffff c90c5714 c11c9662 [ 3548.304712] <0> 00000003 cccd3ac0 000280d0 00000044 d0a56000 000000a0 ccd6977c 000000a2 [ 3548.304712] Call Trace: [ 3548.304712] [<c1025106>] ? xen_teardown_msi_dev+0x42/0x44 [ 3548.304712] [<c103beef>] ? arch_teardown_msi_irqs+0x19/0x6f [ 3548.304712] [<c103c017>] ? arch_setup_msi_irqs+0x30/0x106 [ 3548.304712] [<c11c90af>] ? msi_free_irqs+0x70/0xcc [ 3548.304712] [<c11c93aa>] ? alloc_msi_entry+0x1b/0x71 [ 3548.304712] [<c11c9662>] ? pci_enable_msix+0x262/0x354 [ 3548.304712] [<d0b3053a>] ? vxge_open+0x264/0x1483 [vxge] [ 3548.304712] [<c1024dde>] ? __xen_spin_lock+0xc4/0xd8 [ 3548.304712] [<c1023b03>] ? xen_force_evtchn_callback+0xf/0x14 [ 3548.304712] [<c1023b03>] ? xen_force_evtchn_callback+0xf/0x14 [ 3548.304712] [<c1021982>] ? __raw_callee_save_xen_pmd_val+0x6/0x8 [ 3548.304712] [<c1043791>] ? lookup_address+0xea/0x10b [ 3548.304712] [<c1023b03>] ? xen_force_evtchn_callback+0xf/0x14 [ 3548.304712] [<c10241e0>] ? check_events+0x8/0xc [ 3548.304712] [<c10241d7>] ? xen_restore_fl_direct_end+0x0/0x1 [ 3548.304712] [<c102180e>] ? xen_mc_flush+0x1fa/0x215 [ 3548.304712] [<c10241e0>] ? check_events+0x8/0xc [ 3548.304712] [<c10241d7>] ? xen_restore_fl_direct_end+0x0/0x1 [ 3548.304712] [<c1020d73>] ? xen_end_context_switch+0x12/0x15 [ 3548.304712] [<c10267ae>] ? __switch_to+0x113/0x144 [ 3548.304712] [<c105226e>] ? finish_task_switch+0x77/0xe7 [ 3548.304712] [<c14d9f69>] ? schedule+0x7b5/0x842 [ 3548.304712] [<c104d2a2>] ? update_curr+0x94/0x1bb [ 3548.304712] [<c104f326>] ? enqueue_entity+0x25e/0x266 [ 3548.304712] [<c1023b03>] ? xen_force_evtchn_callback+0xf/0x14 [ 3548.304712] [<c10241e0>] ? check_events+0x8/0xc [ 3548.304712] [<c10241d7>] ? xen_restore_fl_direct_end+0x0/0x1 [ 3548.304712] [<c14db424>] ? _spin_unlock_irqrestore+0x43/0x48 [ 3548.304712] [<c1054278>] ? try_to_wake_up+0x298/0x2a2 [ 3548.304712] [<c105428d>] ? default_wake_function+0xb/0xd [ 3548.304712] [<c10e5882>] ? pollwake+0x59/0x63 [ 3548.304712] [<c1024dde>] ? __xen_spin_lock+0xc4/0xd8 [ 3548.304712] [<c1023b03>] ? xen_force_evtchn_callback+0xf/0x14 [ 3548.304712] [<c10241d7>] ? xen_restore_fl_direct_end+0x0/0x1 [ 3548.304712] [<c14db424>] ? _spin_unlock_irqrestore+0x43/0x48 [ 3548.304712] [<c104ce3c>] ? __wake_up+0x31/0x3b [ 3548.304712] [<c13f6462>] ? netlink_broadcast+0x221/0x252 [ 3548.304712] [<c13f6952>] ? nlmsg_notify+0x3f/0x78 [ 3548.304712] [<c1429c07>] ? inetdev_event+0x1c/0x39f [ 3548.304712] [<c147738d>] ? packet_notifier+0x21/0x16c [ 3548.304712] [<c14dd475>] ? notifier_call_chain+0x2b/0x4a [ 3548.304712] [<c13e1707>] ? dev_open+0x93/0xce [ 3548.304712] [<c13e0f7b>] ? dev_change_flags+0xa5/0x158 [ 3548.304712] [<c13e8149>] ? do_setlink+0x262/0x31a [ 3548.304712] [<c13e9293>] ? rtnl_newlink+0x27e/0x3f9 [ 3548.304712] [<c13e9050>] ? rtnl_newlink+0x3b/0x3f9 [ 3548.304712] [<c13e909a>] ? rtnl_newlink+0x85/0x3f9 [ 3548.304712] [<c1181df3>] ? selinux_netlink_recv+0x46/0x50 [ 3548.304712] [<c13e9015>] ? rtnl_newlink+0x0/0x3f9 [ 3548.304712] [<c13e8ffb>] ? rtnetlink_rcv_msg+0x1a0/0x1ba [ 3548.304712] [<c13e8e5b>] ? rtnetlink_rcv_msg+0x0/0x1ba [ 3548.304712] [<c13f6a89>] ? netlink_rcv_skb+0x30/0x77 [ 3548.304712] [<c13e8e53>] ? rtnetlink_rcv+0x1c/0x24 [ 3548.304712] [<c13f666d>] ? netlink_unicast+0xc4/0x11a [ 3548.304712] [<c13f6906>] ? netlink_sendmsg+0x243/0x250 [ 3548.304712] [<c13d327f>] ? __sock_sendmsg+0x45/0x4e [ 3548.304712] [<c13d3a07>] ? sock_sendmsg+0xb8/0xce [ 3548.304712] [<c106b3df>] ? autoremove_wake_function+0x0/0x33 [ 3548.304712] [<c106b3df>] ? autoremove_wake_function+0x0/0x33 [ 3548.304712] [<c1023b03>] ? xen_force_evtchn_callback+0xf/0x14 [ 3548.304712] [<c10ae581>] ? find_get_page+0x1d/0x81 [ 3548.304712] [<c11b6ff1>] ? copy_from_user+0x2a/0x111 [ 3548.304712] [<c13dad63>] ? verify_iovec+0x40/0x71 [ 3548.304712] [<c13d3b5c>] ? sys_sendmsg+0x13f/0x192 [ 3548.304712] [<c10c2974>] ? handle_mm_fault+0x481/0xa8e [ 3548.304712] [<c10c63a1>] ? vma_link+0x53/0x94 [ 3548.304712] [<c13d4b7c>] ? sys_socketcall+0x153/0x183 [ 3548.304712] [<c1027ac9>] ? syscall_call+0x7/0xb [ 3548.304712] Code: Bad EIP value. [ 3548.304712] EIP: [<bfabdca0>] 0xbfabdca0 SS:ESP 0069:c90c569c [ 3548.304712] CR2: 00000000bfabdca0 [ 3548.304712] ---[ end trace 79e784140187edbe ]--- _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Jun-12 12:41 UTC
Re: [Xen-devel] MSIX allocation failure on 2.6.31.13 PV domU
On Sat, Jun 12, 2010 at 04:33:15AM -0400, Masroor Vettuparambil wrote:> Hi, > > I am trying pci pass-through on PV DomU with xen4.0+Dom0 2.6.31.13. > While loading the driver on DomU with MSIX, I am getting the below oops > message. I have passed "iommu=soft" and "swiotlb=force" to DomU kernel. > > I can see many related patches applied to 2.6.32.x, but not to 2.6.31.x.Ugh, I actually didn''t backport those patches to 2.6.31.x. I can take a swing at it next week.> On my setup I can run the driver with MSIX/INTA on Dom0 and with INTA on > DomU. Is MSIX support broken in 2.6.31.x?Yes it is broken. Interestingly you are also using the 32-bit version of kernel which had light testing when it comes to PCI front. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Jun-15 21:36 UTC
Re: [Xen-devel] MSIX allocation failure on 2.6.31.13 PV domU
On Sat, Jun 12, 2010 at 08:41:06AM -0400, Konrad Rzeszutek Wilk wrote:> On Sat, Jun 12, 2010 at 04:33:15AM -0400, Masroor Vettuparambil wrote: > > Hi, > > > > I am trying pci pass-through on PV DomU with xen4.0+Dom0 2.6.31.13. > > While loading the driver on DomU with MSIX, I am getting the below oops > > message. I have passed "iommu=soft" and "swiotlb=force" to DomU kernel. > > > > I can see many related patches applied to 2.6.32.x, but not to 2.6.31.x. > > Ugh, I actually didn''t backport those patches to 2.6.31.x. I can take a > swing at it next week.Ok. just finished with the DRM/KMS fixes so will start dealing with the backlog of Xen PCI issues. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel