Andrey Luzgin
2010-Feb-10 19:44 UTC
[Xen-users] Ethernet modules craches in domU with pci passthrough
I do not know where to search, help, please, anybody. This is error part of dmesg from domU (details of system below): udev: starting version 146 Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.0.1 (May 6, 2009) bnx2 0000:02:00.0: enabling device (0000 -> 0002) alloc irq_desc for 32 on node 0 alloc kstat_irqs on node 0 ------------[ cut here ]------------ Kernel BUG at ffffffff8044e6f5 [verbose debug info unavailable] invalid opcode: 0000 [#1] SMP last sysfs file: /sys/devices/xen/vbd-51713/block/xvda1/removable CPU 0 Modules linked in: bnx2(+) Pid: 1271, comm: modprobe Not tainted 2.6.31-xen-r10 #10 RIP: e030:[<ffffffff8044e6f5>] [<ffffffff8044e6f5>] 0xffffffff8044e6f5 RSP: e02b:ffff8801b3d19cc8 EFLAGS: 00010216 RAX: ffff8801b42ff840 RBX: 0000000000000020 RCX: 0000000000000000 RDX: 0000000000007900 RSI: 0000000000000000 RDI: 0000000000000020 RBP: 0000000000000001 R08: 00000000ffffffff R09: 0000000000007900 R10: 00000b0b00000000 R11: 0000000000000020 R12: 00000000fffffff4 R13: ffff8801b5047090 R14: ffff8801b535c000 R15: ffff8801b5047000 FS: 00007fcd29a366f0(0000) GS:ffffc90000000000(0000) knlGS:0000000000000000 CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007fcbb162600f CR3: 00000001b3056000 CR4: 0000000000002660 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process modprobe (pid: 1271, threadinfo ffff8801b3d18000, task ffff8801b4be1d60) Stack: ffff8801b5047000 ffffffff8048f214 ffff8801b535c000 20ffffff80490283 <0> ffff8801b5047000 ffffffff803ea39f ffffffffa000c530 ffff8801b5047000 <0> ffff8801b535c580 ffffffff803ea404 ffffffffa000c590 ffffffffa0009e25 Call Trace: [<ffffffff8048f214>] ? 0xffffffff8048f214 [<ffffffff803ea39f>] ? 0xffffffff803ea39f [<ffffffff803ea404>] ? 0xffffffff803ea404 [<ffffffffa0009e25>] ? 0xffffffffa0009e25 [<ffffffff802ca492>] ? 0xffffffff802ca492 [<ffffffff8030c7ce>] ? 0xffffffff8030c7ce [<ffffffff803eb032>] ? 0xffffffff803eb032 [<ffffffff803eb7f1>] ? 0xffffffff803eb7f1 [<ffffffff80444b7f>] ? 0xffffffff80444b7f [<ffffffff80444c69>] ? 0xffffffff80444c69 [<ffffffff80444c1a>] ? 0xffffffff80444c1a [<ffffffff80444467>] ? 0xffffffff80444467 [<ffffffff80443e2d>] ? 0xffffffff80443e2d [<ffffffff80444f03>] ? 0xffffffff80444f03 [<ffffffffa0010000>] ? 0xffffffffa0010000 [<ffffffff803ebc29>] ? 0xffffffff803ebc29 [<ffffffffa0010000>] ? 0xffffffffa0010000 [<ffffffff80207051>] ? 0xffffffff80207051 [<ffffffff80258fcd>] ? 0xffffffff80258fcd [<ffffffff802093f8>] ? 0xffffffff802093f8 [<ffffffff80209390>] ? 0xffffffff80209390 Code: 44 f8 ff 41 89 c6 3b 05 ea 71 20 00 0f 82 b1 fd ff ff 48 83 c4 68 31 c0 5b 5d 41 5c 41 5d 41 5e 41 5f c3 53 83 ff 0f 89 fb 76 04 <0f> 0b eb fe 7e 55 e8 88 f1 e1 ff 48 85 c0 74 10 48 8b 40 38 48 RIP [<ffffffff8044e6f5>] 0xffffffff8044e6f5 RSP <ffff8801b3d19cc8> ---[ end trace c14fe0c5cc1cd90d ]--- I use in dom0 Gentoo linux with kernel 2.6.31-xen-r10 with CONFIG_XEN_PCIDEV_BACKEND_PASS=y The same kernel in domU with frontends options enabled instead of backends Xen version: xen-3.4.2-r1 (with xsm compile option enabled - gentoo specific) xen linux # xm info host : xen release : 2.6.31-xen-r10 version : #4 SMP Tue Jan 12 20:59:12 IST 2010 machine : x86_64 nr_cpus : 8 nr_nodes : 1 cores_per_socket : 4 threads_per_core : 1 cpu_mhz : 2128 hw_caps : bfebfbff:28100800:00000000:00000340:009ce3bd:00000000:00000001:00000000 virt_caps : hvm hvm_directio total_memory : 8182 free_memory : 14 node_to_cpu : node0:0-7 node_to_memory : node0:14 xen_major : 3 xen_minor : 4 xen_extra : .2 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : unavailable cc_compiler : gcc version 4.3.4 (Gentoo 4.3.4 p1.0, pie-10.1.5) cc_compile_by : root cc_compile_domain : (none) cc_compile_date : Tue Jan 12 21:18:45 IST 2010 xend_config_format : 4 I have Dell R710 with 2 dual NICs Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) (4 ports summary) This is my grub.conf options: title Xen 3.4.2 / Gentoo Linux 2.6.31-gentoo-r6-dom0 root (hd0,0) kernel /xen.gz iommu=1 msi=1 module /vmlinuz.xen root=/dev/sda3 xencons=xvc pciback.hide=(02:00.0)(02:00.1) This is my domU config: xen linux # cat /home/VM/proxy/proxy.cfg #kernel = "/boot/vmlinuz-2.6.30.10" bootloader = "/usr/bin/pygrub" name="proxy" builder=''linux'' memory = ''7000'' vcpus = ''8'' disk = [''tap:aio:/home/VM/proxy/proxy.img,xvda1,w''] root = "/dev/xvda1" hostname="proxy" vif = [ ''bridge=eth0'', ''bridge=eth1'' ] extra="vga=791 xencons=xvc" pci=[''02:00.0'',''02:00.1''] In dom0: xen xen # dmesg | grep pciback pciback 0000:02:00.0: seizing device pciback 0000:02:00.1: seizing device pciback 0000:02:00.1: PCI INT B -> GSI 42 (level, low) -> IRQ 42 pciback 0000:02:00.1: PCI INT B disabled pciback 0000:02:00.0: PCI INT A -> GSI 32 (level, low) -> IRQ 32 pciback 0000:02:00.0: PCI INT A disabled pciback 0000:02:00.0: enabling permissive mode configuration space accesses! pciback 0000:02:00.0: permissive mode is potentially unsafe! pciback 0000:02:00.1: enabling permissive mode configuration space accesses! pciback 0000:02:00.1: permissive mode is potentially unsafe! pciback 0000:02:00.0: enabling device (0000 -> 0002) pciback 0000:02:00.0: PCI INT A -> GSI 32 (level, low) -> IRQ 32 In domU proxy ~ # dmesg | grep pcifront pcifront pci-0: Installing PCI frontend pcifront pci-0: Creating PCI Frontend Bus 0000:02 I tried other cards with other modules and is always broken with same crach error. -- Andrey Luzgin <andrey@icomsw.com> Icom software _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen
2010-Feb-10 20:48 UTC
Re: [Xen-users] Ethernet modules craches in domU with pci passthrough
On Wed, Feb 10, 2010 at 09:44:08PM +0200, Andrey Luzgin wrote:> I do not know where to search, help, please, anybody. > > This is error part of dmesg from domU (details of system below): > > udev: starting version 146 > Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.0.1 (May 6, 2009) > bnx2 0000:02:00.0: enabling device (0000 -> 0002) > alloc irq_desc for 32 on node 0 > alloc kstat_irqs on node 0 > ------------[ cut here ]------------ > Kernel BUG at ffffffff8044e6f5 [verbose debug info unavailable] > invalid opcode: 0000 [#1] SMP > last sysfs file: /sys/devices/xen/vbd-51713/block/xvda1/removable > CPU 0 > Modules linked in: bnx2(+) > Pid: 1271, comm: modprobe Not tainted 2.6.31-xen-r10 #10 > RIP: e030:[<ffffffff8044e6f5>] [<ffffffff8044e6f5>] 0xffffffff8044e6f5 > RSP: e02b:ffff8801b3d19cc8 EFLAGS: 00010216 > RAX: ffff8801b42ff840 RBX: 0000000000000020 RCX: 0000000000000000 > RDX: 0000000000007900 RSI: 0000000000000000 RDI: 0000000000000020 > RBP: 0000000000000001 R08: 00000000ffffffff R09: 0000000000007900 > R10: 00000b0b00000000 R11: 0000000000000020 R12: 00000000fffffff4 > R13: ffff8801b5047090 R14: ffff8801b535c000 R15: ffff8801b5047000 > FS: 00007fcd29a366f0(0000) GS:ffffc90000000000(0000) > knlGS:0000000000000000 > CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b > CR2: 00007fcbb162600f CR3: 00000001b3056000 CR4: 0000000000002660 > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > Process modprobe (pid: 1271, threadinfo ffff8801b3d18000, task > ffff8801b4be1d60) > Stack: > ffff8801b5047000 ffffffff8048f214 ffff8801b535c000 20ffffff80490283 > <0> ffff8801b5047000 ffffffff803ea39f ffffffffa000c530 ffff8801b5047000 > <0> ffff8801b535c580 ffffffff803ea404 ffffffffa000c590 ffffffffa0009e25 > Call Trace: > [<ffffffff8048f214>] ? 0xffffffff8048f214 > [<ffffffff803ea39f>] ? 0xffffffff803ea39f > [<ffffffff803ea404>] ? 0xffffffff803ea404 > [<ffffffffa0009e25>] ? 0xffffffffa0009e25 > [<ffffffff802ca492>] ? 0xffffffff802ca492 > [<ffffffff8030c7ce>] ? 0xffffffff8030c7ce > [<ffffffff803eb032>] ? 0xffffffff803eb032 > [<ffffffff803eb7f1>] ? 0xffffffff803eb7f1 > [<ffffffff80444b7f>] ? 0xffffffff80444b7f > [<ffffffff80444c69>] ? 0xffffffff80444c69 > [<ffffffff80444c1a>] ? 0xffffffff80444c1a > [<ffffffff80444467>] ? 0xffffffff80444467 > [<ffffffff80443e2d>] ? 0xffffffff80443e2d > [<ffffffff80444f03>] ? 0xffffffff80444f03 > [<ffffffffa0010000>] ? 0xffffffffa0010000 > [<ffffffff803ebc29>] ? 0xffffffff803ebc29 > [<ffffffffa0010000>] ? 0xffffffffa0010000 > [<ffffffff80207051>] ? 0xffffffff80207051 > [<ffffffff80258fcd>] ? 0xffffffff80258fcd > [<ffffffff802093f8>] ? 0xffffffff802093f8 > [<ffffffff80209390>] ? 0xffffffff80209390 > Code: 44 f8 ff 41 89 c6 3b 05 ea 71 20 00 0f 82 b1 fd ff ff 48 83 c4 68 31 > c0 5b 5d 41 5c 41 5d 41 5e 41 5f c3 53 83 ff 0f 89 fb 76 04 <0f> 0b eb fe > 7e 55 e8 88 f1 e1 ff 48 85 c0 74 10 48 8b 40 38 48 > RIP [<ffffffff8044e6f5>] 0xffffffff8044e6f5 > RSP <ffff8801b3d19cc8> > ---[ end trace c14fe0c5cc1cd90d ]--- >You need to recompile the kernel with debug information so you can actually see the function names in the stack trace above. Also you could use xenctx from dom0 do get a stack trace, see: http://wiki.xensource.com/xenwiki/XenCommonProblems There''s a chapter about getting a stacktrace with xenctx. Also, do you have errors in "xm dmesg" ? -- Pasi _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users