Vasudevan, Swaminathan (PNB Roseville)
2009-Sep-24 18:04 UTC
[Xen-devel] Issues with the PCI Passthrough and swiotlb on xen 3.3.1
Hi Folks,
I have been trying to do a PCI Passthrough of one of my PCI device.
The Pass through works fine initially.
But I found that after couple of reboots the kernal crashes with the message
below:
I am running xen 3.3.1 64-bit on CentOS 5.3.
My DomU is pv-kernel CentOS 5.3 (32-bit)
Fatal DMA error! Please use ''swiotlb=force''
------------[ cut here ]------------
kernel BUG at arch/i386/kernel/pci-dma-xen.c:395!
invalid opcode: 0000 [#1]
SMP
last sysfs file: /class/dahdi/dahdi!24/dev
Modules linked in: wanec(U) af_wanpipe(U) wanpipe(U) wanrouter(U) dahdi(U)
crc_ccitt sdladrv(U) dm_mirror dm_multipath scsi_dh scsi_mod parport_pc lp
parport xennet pcspkr dm_raid45 dm_message dm_region_hash dm_log dm_mod
dm_mem_cache xenblk ext3 jbd uhci_hcd ohci_hcd ehci_hcd
CPU: 0
EIP: 0061:[<c040a37c>] Tainted: G VLI
EFLAGS: 00010086 (2.6.18-128.7.1.el5.centos.plusxen #1)
EIP is at dma_map_single+0x12d/0x147
eax: 0000002f ebx: 00000000 ecx: 00000001 edx: f5416000
esi: 348d1010 edi: 00000001 ebp: 00000400 esp: df984db0
ds: 007b es: 007b ss: 0069
Process wanconfig (pid: 890, ti=df984000 task=c0322aa0 task.ti=df984000)
Stack: c06292b6 c0c55848 01548010 00000000 00000002 00000000 c0298f48 00000400
e1109d99 00000002 00000000 00000400 00000400 c1548010 fffffff0 c0298000
ddc10000 c0298f48 e1783d63 00000400 00000400 00000002 ddc10000 00002034
Call Trace:
[<e1109d99>] sdla_busdma_map+0xdc/0x140 [sdladrv]
[<e1783d63>] aft_dma_rx+0x131/0x551 [wanpipe]
[<e1785243>] enable_data_error_intr+0x3c7/0x787 [wanpipe]
[<e17858a1>] handle_front_end_state+0x124/0x1df [wanpipe]
[<e178c683>] new_if+0x615/0x837 [wanpipe]
[<e113a000>] wan_netif_fake_init+0x0/0x29 [wanrouter]
[<c04589bb>] __kzalloc+0xd/0x34
[<e113a710>] wan_device_new_if+0x101/0x1e3 [wanrouter]
[<e113ab70>] wanrouter_ioctl+0x37e/0x887 [wanrouter]
[<e109ae91>] ext3_file_write+0x19/0x83 [ext3]
[<c046ed76>] do_sync_write+0xb6/0xf1
[<c042fe57>] autoremove_wake_function+0x0/0x2d
[<c047ef03>] do_ioctl+0x47/0x5d
[<c047f163>] vfs_ioctl+0x24a/0x25c
[<c047f1bd>] sys_ioctl+0x48/0x5f
[<c0405413>] syscall_call+0x7/0xb
======================Code: c0 74 07 8b 08 8b 58 04 eb 05 83 c9 ff 31 db 89 da
89 c8 f7 d2 f7 d0 21 fa 21 f0 89 d1 09 c1 74 13 68 b6 92 62 c0 e8 0e 5e 01 00
<0f> 0b 8b 01 97 92 62 c0 5d f0 83 04 24 00 83 c4 0c 89 f0 89 fa
EIP: [<c040a37c>] dma_map_single+0x12d/0x147 SS:ESP 0069:df984db0
<0>Kernel panic - not syncing: Fatal exception
Then I read some of the postings and found that I need to include the
"swiotlb=force" as a kernel boot option and I did.
I included "swiotlb=128" in the dom0 kernel boot and in the domU I
included "swiotlb=force".
After setting this up, the clean domU came up fine, but again when the domU for
some reason crashed and after that, when I try to boot to the domU I get this
kernel panic as shown below:
------------[ cut here ]------------
kernel BUG at lib/../arch/i386/kernel/swiotlb.c:160!
invalid opcode: 0000 [#1]
SMP
last sysfs file:
Modules linked in:
CPU: 0
EIP: e019:[<c04ebc14>] Not tainted VLI
EFLAGS: 00010282 (2.6.18-128.7.1.el5.centos.plusxen #1)
EIP is at swiotlb_init_with_default_size+0x88/0x14e
eax: fffffff4 ebx: 00000c80 ecx: 00000000 edx: f5416000
esi: c071fc64 edi: 04000000 ebp: c06effc0 esp: c06effb4
ds: e021 es: e021 ss: e021
Process swapper (pid: 0, ti=c06ef000 task=c06762c0 task.ti=c06ef000)
Stack: c1417b64 c071fc64 c1415b64 00000020 c06ff84d 0000000e 00000002 c1417b64
c071fc64 c1415b64 00000020 c06f486f 00004b64 c0765800 01020800 c0d6a000
00000000 00000000 c040006f
Call Trace:
[<c06ff84d>] mem_init+0xc/0x343
[<c06f486f>] start_kernel+0x1f4/0x381
======================Code: ea 3a f3 ff 83 c8 ff 40 83 f8 06 75 fa 89 d8 b9 1e
00 00 00 c1 e0 0b ba 06 00 00 00 03 05 34 dc 7a c0 e8 c6 c2 f2 ff 85 c0 74 08
<0f> 0b a0 00 87 af 63 c0 83 eb 80 a1 38 dc 7a c0 39 c3 72 c5 b9
EIP: [<c04ebc14>] swiotlb_init_with_default_size+0x88/0x14e SS:ESP
e021:c06effb4
<0>Kernel panic - not syncing: Fatal exception
[root@cvanzante xen]#
Is there any permanent solution for this errors. Or what might have caused this
problem.
If this problem has been addressed earlier by this community please someone
provide me the link to this solution.
Thanks.
Swaminathan Vasudevan
Systems Software Engineer (TC)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Jan Beulich
2009-Sep-25 07:48 UTC
Re: [Xen-devel] Issues with the PCI Passthrough and swiotlb on xen 3.3.1
This must be a really old kernel code base: Neither does Xen''s swiotlb code live at the indicated position anymore, nor is there any BUG() in today''s swiotlb_init_with_default_size(). So you may want to look at the differences of your and the current file, or you would at least need to indicate what BUG() statement it is that you''re (apparently) hitting. Jan>>> "Vasudevan, Swaminathan (PNB Roseville)" <swaminathan.vasudevan@hp.com> 24.09.09 20:04 >>>Hi Folks, I have been trying to do a PCI Passthrough of one of my PCI device. The Pass through works fine initially. But I found that after couple of reboots the kernal crashes with the message below: I am running xen 3.3.1 64-bit on CentOS 5.3. My DomU is pv-kernel CentOS 5.3 (32-bit) Fatal DMA error! Please use ''swiotlb=force'' ------------[ cut here ]------------ kernel BUG at arch/i386/kernel/pci-dma-xen.c:395! invalid opcode: 0000 [#1] SMP last sysfs file: /class/dahdi/dahdi!24/dev Modules linked in: wanec(U) af_wanpipe(U) wanpipe(U) wanrouter(U) dahdi(U) crc_ccitt sdladrv(U) dm_mirror dm_multipath scsi_dh scsi_mod parport_pc lp parport xennet pcspkr dm_raid45 dm_message dm_region_hash dm_log dm_mod dm_mem_cache xenblk ext3 jbd uhci_hcd ohci_hcd ehci_hcd CPU: 0 EIP: 0061:[<c040a37c>] Tainted: G VLI EFLAGS: 00010086 (2.6.18-128.7.1.el5.centos.plusxen #1) EIP is at dma_map_single+0x12d/0x147 eax: 0000002f ebx: 00000000 ecx: 00000001 edx: f5416000 esi: 348d1010 edi: 00000001 ebp: 00000400 esp: df984db0 ds: 007b es: 007b ss: 0069 Process wanconfig (pid: 890, ti=df984000 task=c0322aa0 task.ti=df984000) Stack: c06292b6 c0c55848 01548010 00000000 00000002 00000000 c0298f48 00000400 e1109d99 00000002 00000000 00000400 00000400 c1548010 fffffff0 c0298000 ddc10000 c0298f48 e1783d63 00000400 00000400 00000002 ddc10000 00002034 Call Trace: [<e1109d99>] sdla_busdma_map+0xdc/0x140 [sdladrv] [<e1783d63>] aft_dma_rx+0x131/0x551 [wanpipe] [<e1785243>] enable_data_error_intr+0x3c7/0x787 [wanpipe] [<e17858a1>] handle_front_end_state+0x124/0x1df [wanpipe] [<e178c683>] new_if+0x615/0x837 [wanpipe] [<e113a000>] wan_netif_fake_init+0x0/0x29 [wanrouter] [<c04589bb>] __kzalloc+0xd/0x34 [<e113a710>] wan_device_new_if+0x101/0x1e3 [wanrouter] [<e113ab70>] wanrouter_ioctl+0x37e/0x887 [wanrouter] [<e109ae91>] ext3_file_write+0x19/0x83 [ext3] [<c046ed76>] do_sync_write+0xb6/0xf1 [<c042fe57>] autoremove_wake_function+0x0/0x2d [<c047ef03>] do_ioctl+0x47/0x5d [<c047f163>] vfs_ioctl+0x24a/0x25c [<c047f1bd>] sys_ioctl+0x48/0x5f [<c0405413>] syscall_call+0x7/0xb ======================Code: c0 74 07 8b 08 8b 58 04 eb 05 83 c9 ff 31 db 89 da 89 c8 f7 d2 f7 d0 21 fa 21 f0 89 d1 09 c1 74 13 68 b6 92 62 c0 e8 0e 5e 01 00 <0f> 0b 8b 01 97 92 62 c0 5d f0 83 04 24 00 83 c4 0c 89 f0 89 fa EIP: [<c040a37c>] dma_map_single+0x12d/0x147 SS:ESP 0069:df984db0 <0>Kernel panic - not syncing: Fatal exception Then I read some of the postings and found that I need to include the "swiotlb=force" as a kernel boot option and I did. I included "swiotlb=128" in the dom0 kernel boot and in the domU I included "swiotlb=force". After setting this up, the clean domU came up fine, but again when the domU for some reason crashed and after that, when I try to boot to the domU I get this kernel panic as shown below: ------------[ cut here ]------------ kernel BUG at lib/../arch/i386/kernel/swiotlb.c:160! invalid opcode: 0000 [#1] SMP last sysfs file: Modules linked in: CPU: 0 EIP: e019:[<c04ebc14>] Not tainted VLI EFLAGS: 00010282 (2.6.18-128.7.1.el5.centos.plusxen #1) EIP is at swiotlb_init_with_default_size+0x88/0x14e eax: fffffff4 ebx: 00000c80 ecx: 00000000 edx: f5416000 esi: c071fc64 edi: 04000000 ebp: c06effc0 esp: c06effb4 ds: e021 es: e021 ss: e021 Process swapper (pid: 0, ti=c06ef000 task=c06762c0 task.ti=c06ef000) Stack: c1417b64 c071fc64 c1415b64 00000020 c06ff84d 0000000e 00000002 c1417b64 c071fc64 c1415b64 00000020 c06f486f 00004b64 c0765800 01020800 c0d6a000 00000000 00000000 c040006f Call Trace: [<c06ff84d>] mem_init+0xc/0x343 [<c06f486f>] start_kernel+0x1f4/0x381 ======================Code: ea 3a f3 ff 83 c8 ff 40 83 f8 06 75 fa 89 d8 b9 1e 00 00 00 c1 e0 0b ba 06 00 00 00 03 05 34 dc 7a c0 e8 c6 c2 f2 ff 85 c0 74 08 <0f> 0b a0 00 87 af 63 c0 83 eb 80 a1 38 dc 7a c0 39 c3 72 c5 b9 EIP: [<c04ebc14>] swiotlb_init_with_default_size+0x88/0x14e SS:ESP e021:c06effb4 <0>Kernel panic - not syncing: Fatal exception [root@cvanzante xen]# Is there any permanent solution for this errors. Or what might have caused this problem. If this problem has been addressed earlier by this community please someone provide me the link to this solution. Thanks. Swaminathan Vasudevan Systems Software Engineer (TC) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Vasudevan, Swaminathan (PNB Roseville)
2009-Sep-25 17:11 UTC
RE: [Xen-devel] Issues with the PCI Passthrough and swiotlb on xen 3.3.1
Hi Jan, You mentioned that it might be an old kernel. Are you talking about the domU kernel or the dom0 kernel. Because dom0 kernel was build from the xen-3.3.1 sources. For the domU I have used the kernel from CentOS. Thanks. Swaminathan Vasudevan Systems Software Engineer (TC) HP ProCurve Hewlett-Packard 8000 Foothills Blvd M/S 5541 Roseville, CA - 95747 tel: 916.785.0937 fax: 916.785.1815 email: swaminathan.vasudevan@hp.com -----Original Message----- From: Jan Beulich [mailto:JBeulich@novell.com] Sent: Friday, September 25, 2009 12:49 AM To: Vasudevan, Swaminathan (PNB Roseville) Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Issues with the PCI Passthrough and swiotlb on xen 3.3.1 This must be a really old kernel code base: Neither does Xen''s swiotlb code live at the indicated position anymore, nor is there any BUG() in today''s swiotlb_init_with_default_size(). So you may want to look at the differences of your and the current file, or you would at least need to indicate what BUG() statement it is that you''re (apparently) hitting. Jan>>> "Vasudevan, Swaminathan (PNB Roseville)" >>> <swaminathan.vasudevan@hp.com> 24.09.09 20:04 >>>Hi Folks, I have been trying to do a PCI Passthrough of one of my PCI device. The Pass through works fine initially. But I found that after couple of reboots the kernal crashes with the message below: I am running xen 3.3.1 64-bit on CentOS 5.3. My DomU is pv-kernel CentOS 5.3 (32-bit) Fatal DMA error! Please use ''swiotlb=force'' ------------[ cut here ]------------ kernel BUG at arch/i386/kernel/pci-dma-xen.c:395! invalid opcode: 0000 [#1] SMP last sysfs file: /class/dahdi/dahdi!24/dev Modules linked in: wanec(U) af_wanpipe(U) wanpipe(U) wanrouter(U) dahdi(U) crc_ccitt sdladrv(U) dm_mirror dm_multipath scsi_dh scsi_mod parport_pc lp parport xennet pcspkr dm_raid45 dm_message dm_region_hash dm_log dm_mod dm_mem_cache xenblk ext3 jbd uhci_hcd ohci_hcd ehci_hcd CPU: 0 EIP: 0061:[<c040a37c>] Tainted: G VLI EFLAGS: 00010086 (2.6.18-128.7.1.el5.centos.plusxen #1) EIP is at dma_map_single+0x12d/0x147 eax: 0000002f ebx: 00000000 ecx: 00000001 edx: f5416000 esi: 348d1010 edi: 00000001 ebp: 00000400 esp: df984db0 ds: 007b es: 007b ss: 0069 Process wanconfig (pid: 890, ti=df984000 task=c0322aa0 task.ti=df984000) Stack: c06292b6 c0c55848 01548010 00000000 00000002 00000000 c0298f48 00000400 e1109d99 00000002 00000000 00000400 00000400 c1548010 fffffff0 c0298000 ddc10000 c0298f48 e1783d63 00000400 00000400 00000002 ddc10000 00002034 Call Trace: [<e1109d99>] sdla_busdma_map+0xdc/0x140 [sdladrv] [<e1783d63>] aft_dma_rx+0x131/0x551 [wanpipe] [<e1785243>] enable_data_error_intr+0x3c7/0x787 [wanpipe] [<e17858a1>] handle_front_end_state+0x124/0x1df [wanpipe] [<e178c683>] new_if+0x615/0x837 [wanpipe] [<e113a000>] wan_netif_fake_init+0x0/0x29 [wanrouter] [<c04589bb>] __kzalloc+0xd/0x34 [<e113a710>] wan_device_new_if+0x101/0x1e3 [wanrouter] [<e113ab70>] wanrouter_ioctl+0x37e/0x887 [wanrouter] [<e109ae91>] ext3_file_write+0x19/0x83 [ext3] [<c046ed76>] do_sync_write+0xb6/0xf1 [<c042fe57>] autoremove_wake_function+0x0/0x2d [<c047ef03>] do_ioctl+0x47/0x5d [<c047f163>] vfs_ioctl+0x24a/0x25c [<c047f1bd>] sys_ioctl+0x48/0x5f [<c0405413>] syscall_call+0x7/0xb ======================Code: c0 74 07 8b 08 8b 58 04 eb 05 83 c9 ff 31 db 89 da 89 c8 f7 d2 f7 d0 21 fa 21 f0 89 d1 09 c1 74 13 68 b6 92 62 c0 e8 0e 5e 01 00 <0f> 0b 8b 01 97 92 62 c0 5d f0 83 04 24 00 83 c4 0c 89 f0 89 fa EIP: [<c040a37c>] dma_map_single+0x12d/0x147 SS:ESP 0069:df984db0 <0>Kernel panic - not syncing: Fatal exception Then I read some of the postings and found that I need to include the "swiotlb=force" as a kernel boot option and I did. I included "swiotlb=128" in the dom0 kernel boot and in the domU I included "swiotlb=force". After setting this up, the clean domU came up fine, but again when the domU for some reason crashed and after that, when I try to boot to the domU I get this kernel panic as shown below: ------------[ cut here ]------------ kernel BUG at lib/../arch/i386/kernel/swiotlb.c:160! invalid opcode: 0000 [#1] SMP last sysfs file: Modules linked in: CPU: 0 EIP: e019:[<c04ebc14>] Not tainted VLI EFLAGS: 00010282 (2.6.18-128.7.1.el5.centos.plusxen #1) EIP is at swiotlb_init_with_default_size+0x88/0x14e eax: fffffff4 ebx: 00000c80 ecx: 00000000 edx: f5416000 esi: c071fc64 edi: 04000000 ebp: c06effc0 esp: c06effb4 ds: e021 es: e021 ss: e021 Process swapper (pid: 0, ti=c06ef000 task=c06762c0 task.ti=c06ef000) Stack: c1417b64 c071fc64 c1415b64 00000020 c06ff84d 0000000e 00000002 c1417b64 c071fc64 c1415b64 00000020 c06f486f 00004b64 c0765800 01020800 c0d6a000 00000000 00000000 c040006f Call Trace: [<c06ff84d>] mem_init+0xc/0x343 [<c06f486f>] start_kernel+0x1f4/0x381 ======================Code: ea 3a f3 ff 83 c8 ff 40 83 f8 06 75 fa 89 d8 b9 1e 00 00 00 c1 e0 0b ba 06 00 00 00 03 05 34 dc 7a c0 e8 c6 c2 f2 ff 85 c0 74 08 <0f> 0b a0 00 87 af 63 c0 83 eb 80 a1 38 dc 7a c0 39 c3 72 c5 b9 EIP: [<c04ebc14>] swiotlb_init_with_default_size+0x88/0x14e SS:ESP e021:c06effb4 <0>Kernel panic - not syncing: Fatal exception [root@cvanzante xen]# Is there any permanent solution for this errors. Or what might have caused this problem. If this problem has been addressed earlier by this community please someone provide me the link to this solution. Thanks. Swaminathan Vasudevan Systems Software Engineer (TC) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2009-Sep-28 07:20 UTC
RE: [Xen-devel] Issues with the PCI Passthrough and swiotlb on xen 3.3.1
>>> "Vasudevan, Swaminathan (PNB Roseville)" <swaminathan.vasudevan@hp.com> 25.09.09 19:11 >>> >You mentioned that it might be an old kernel. >Are you talking about the domU kernel or the dom0 kernel.The one that''s crashing, i.e. the DomU one afaics. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel