Peter Braun
2010-Nov-09 23:02 UTC
[Xen-users] Xen 4.0.1 - Ubuntu 10.04 LTS - multipath bug
Hi, just today after 30+ days of uptime we experienced following bug related to iscsi+multipathing: [2750787.672441] BUG: unable to handle kernel paging request at ffff88001d379010 [2750787.672869] IP: [<ffffffff8100e1c6>] xen_set_pmd+0x1d/0x31 [2750787.673090] PGD 1002067 PUD 1006067 PMD 1d5067 PTE 801000001d379065 [2750787.673324] Oops: 0003 [#1] SMP [2750787.673541] last sysfs file: /sys/devices/virtual/block/dm-1/dm/suspended [2750787.673757] CPU 7 [2750787.673969] Modules linked in: crc32c iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi dm_round_robin dm_multipath 8021q [last unloaded: scsi_transport_iscsi] [2750787.674642] Pid: 21406, comm: dmsetup_env Not tainted 2.6.32.24 #8 X8DTT [2750787.674857] RIP: e030:[<ffffffff8100e1c6>] [<ffffffff8100e1c6>] xen_set_pmd+0x1d/0x31 [2750787.675278] RSP: e02b:ffff8800161b1d08 EFLAGS: 00010246 [2750787.675492] RAX: 0000000000000000 RBX: ffff88001d379010 RCX: ffffc00000000f98 [2750787.675910] RDX: 0000000000000003 RSI: 0000000000000000 RDI: 0000000000000003 [2750787.676326] RBP: ffff8800161b1d28 R08: 0000000002067000 R09: dead000000100100 [2750787.676747] R10: ffff880073ecd068 R11: 0000000000000003 R12: 000000000061c000 [2750787.677178] R13: ffff880004ae20a0 R14: ffff88001d379010 R15: ffff880041d86000 [2750787.677601] FS: 00007fa021e18700(0000) GS:ffff880004ad5000(0000) knlGS:0000000000000000 [2750787.678018] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [2750787.678229] CR2: ffff88001d379010 CR3: 0000000001001000 CR4: 0000000000002660 [2750787.678652] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [2750787.679065] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [2750787.679484] Process dmsetup_env (pid: 21406, threadinfo ffff8800161b0000, task ffff88007cc9d070) [2750787.679910] Stack: [2750787.680116] 0000000000000000 0000000000000000 ffff880028000000 0000000000600000 [2750787.680341] <0> ffff8800161b1de8 ffffffff810bb124 ffffffff8100eaf5 ffff8800161b1dc8 [2750787.680773] <0> 0000000000400000 0000000000000000 0000000000000000 000000000061bfff [2750787.681410] Call Trace: [2750787.681620] [<ffffffff810bb124>] free_pgd_range+0x22c/0x3aa [2750787.681835] [<ffffffff8100eaf5>] ? xen_force_evtchn_callback+0xd/0xf [2750787.682053] [<ffffffff810bb346>] free_pgtables+0xa4/0xc7 [2750787.682268] [<ffffffff810be853>] exit_mmap+0xf3/0x12b [2750787.682484] [<ffffffff81042341>] mmput+0x39/0xda [2750787.682698] [<ffffffff81045e90>] exit_mm+0xfb/0x106 [2750787.682911] [<ffffffff8104785a>] do_exit+0x1e8/0x6c9 [2750787.683125] [<ffffffff8148e934>] ? do_page_fault+0x243/0x270 [2750787.683338] [<ffffffff81047db6>] do_group_exit+0x7b/0xa5 [2750787.683552] [<ffffffff81047df2>] sys_exit_group+0x12/0x16 [2750787.683767] [<ffffffff81011a82>] system_call_fastpath+0x16/0x1b [2750787.683981] Code: 00 00 e8 07 ed ff ff 48 83 c4 18 5b c9 c3 55 48 89 e5 53 48 89 fb 48 83 ec 18 48 89 75 e8 e8 79 e3 ff ff 48 8b 75 e8 84 c0 75 05 <48> 89 33 eb 08 48 89 df e8 79 ff ff ff 48 83 c4 18 5b c9 c3 55 [2750787.684846] RIP [<ffffffff8100e1c6>] xen_set_pmd+0x1d/0x31 [2750787.685062] RSP <ffff8800161b1d08> [2750787.685272] CR2: ffff88001d379010 [2750787.685666] ---[ end trace ca5d469d9ad43c7a ]--- [2750787.685901] Fixing recursive fault but reboot is needed! [2750797.519472] connection2:0: ping timeout of 5 secs expired, recv timeout 5, last rx 7045410728, last ping 7045415728, now 7045420728 [2750797.519936] connection2:0: detected conn error (1011) [2750797.521469] connection3:0: ping timeout of 5 secs expired, recv timeout 5, last rx 7045410730, last ping 7045415730, now 7045420730 [2750797.521920] connection3:0: detected conn error (1011) Ubuntu 10.04 x86_64 Xen 4.0.1 from sources Kernel 2.6.32.24 pv_ops connected to SAN with iscsi and using multipathing Host had to be rebooted to fix the situation. Any idea? Br Peter _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users