If I build 2.6.31.1 using latest git add just add infiniband I am able to use infiniband uverbs without any issues. If I turn on XEN with dom0 in the config and run that kernel infiniband uverbs does not pass data, ibv_rc_pingpong just sits there. Any ideas? Anyone get infiniband and xen working in a kernel? I talked to the QFED guys and they say that since infiniband works in the 2.6.31.1 kernel just fine that I have to talk to the xen guys. :(><>Nathan Stratton CTO, BlinkMind, Inc. nathan at robotics.net nathan at blinkmind.com http://www.robotics.net http://www.blinkmind.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Thu, 8 Oct 2009, Nathan Stratton wrote:> If I build 2.6.31.1 using latest git add just add infiniband I am able to use > infiniband uverbs without any issues. If I turn on XEN with dom0 in the > config and run that kernel infiniband uverbs does not pass data, > ibv_rc_pingpong just sits there. > > Any ideas? Anyone get infiniband and xen working in a kernel? I talked to the > QFED guys and they say that since infiniband works in the 2.6.31.1 kernel > just fine that I have to talk to the xen guys. :(Interesting, the same kernel works if I load it without xen. Working Infiniband: title Fedora (2.6.31.1) root (hd0,0) kernel /boot/vmlinuz-2.6.31.1 ro root=UUID=b770a383-39e0-48c5-91f5-6c0630f42410 rhgb quiet initrd /boot/initrd-2.6.31.1.img Broken Infiniband: title Fedora (2.6.31.1) root (hd0,0) kernel /boot/xen-3.4.1.gz module /boot/vmlinuz-2.6.31.1 ro root=UUID=b770a383-39e0-48c5-91f5-6c0630f42410 rhgb quiet module /boot/initrd-2.6.31.1.img><>Nathan Stratton CTO, BlinkMind, Inc. nathan at robotics.net nathan at blinkmind.com http://www.robotics.net http://www.blinkmind.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen
2009-Oct-09 07:21 UTC
Re: [Xen-users] Re: 2.6.31.1 xen and infiniband uverbs
On Thu, Oct 08, 2009 at 07:29:03PM -0500, Nathan Stratton wrote:> > On Thu, 8 Oct 2009, Nathan Stratton wrote: > > >If I build 2.6.31.1 using latest git add just add infiniband I am able to > >use infiniband uverbs without any issues. If I turn on XEN with dom0 in > >the config and run that kernel infiniband uverbs does not pass data, > >ibv_rc_pingpong just sits there. > > > >Any ideas? Anyone get infiniband and xen working in a kernel? I talked to > >the QFED guys and they say that since infiniband works in the 2.6.31.1 > >kernel just fine that I have to talk to the xen guys. :( > > Interesting, the same kernel works if I load it without xen. >Do you get any errors in the dom0 kernel dmesg? Or in Xen hypervisor dmesg/logs? All the drivers load correctly? Try these commands: dmesg xm dmesg xm log Also add "loglvl=all guest_loglvl=all" parameters for xen-3.4.1.gz. You might also want to add "dom0_mem=2G" for xen-3.4.1.gz. Or something that is less than 4G.. just in case there''s some 32bit counter/variable bug. -- Pasi> Working Infiniband: > title Fedora (2.6.31.1) > root (hd0,0) > kernel /boot/vmlinuz-2.6.31.1 ro > root=UUID=b770a383-39e0-48c5-91f5-6c0630f42410 rhgb quiet > initrd /boot/initrd-2.6.31.1.img > > Broken Infiniband: > title Fedora (2.6.31.1) > root (hd0,0) > kernel /boot/xen-3.4.1.gz > module /boot/vmlinuz-2.6.31.1 ro > root=UUID=b770a383-39e0-48c5-91f5-6c0630f42410 rhgb quiet > module /boot/initrd-2.6.31.1.img > > > ><> > Nathan Stratton CTO, BlinkMind, Inc. > nathan at robotics.net nathan at blinkmind.com > http://www.robotics.net http://www.blinkmind.com > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Nathan Stratton
2009-Oct-09 12:55 UTC
Re: [Xen-users] Re: 2.6.31.1 xen and infiniband uverbs
On Fri, 9 Oct 2009, Pasi Kärkkäinen wrote:> On Thu, Oct 08, 2009 at 07:29:03PM -0500, Nathan Stratton wrote: >> >> On Thu, 8 Oct 2009, Nathan Stratton wrote: >> >>> If I build 2.6.31.1 using latest git add just add infiniband I am able to >>> use infiniband uverbs without any issues. If I turn on XEN with dom0 in >>> the config and run that kernel infiniband uverbs does not pass data, >>> ibv_rc_pingpong just sits there. >>> >>> Any ideas? Anyone get infiniband and xen working in a kernel? I talked to >>> the QFED guys and they say that since infiniband works in the 2.6.31.1 >>> kernel just fine that I have to talk to the xen guys. :( >> >> Interesting, the same kernel works if I load it without xen. > > Do you get any errors in the dom0 kernel dmesg? Or in Xen hypervisor dmesg/logs?No errors.> All the drivers load correctly?Yes, also infiniband ipoib works, only having issues with DMA uverbs.> Also add "loglvl=all guest_loglvl=all" parameters for xen-3.4.1.gz. > > You might also want to add "dom0_mem=2G" for xen-3.4.1.gz. > Or something that is less than 4G.. just in case there''s some 32bit counter/variable bug.Tried that, same. -Nathan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users