Radu Spineanu
2012-Feb-19 13:17 UTC
kernel panic with 2.6.32-5-xen-amd64 alignment check: 0000 [#1] SMP
Hi. Since a couple of days ago on high loads a domU is constantly rebooting itself after a kernel panic. Below you can find the console logs. There are multiple domUs and only one has this problem. It''s running 8 vcpus, on a Intel(R) Xeon(R) CPU E5504, Debian stable, kernel 2.6.32-5-xen-amd64. If there''s anything else I can provide let me now. [ 58.806593] alignment check: 0000 [#1] SMP [ 58.806610] last sysfs file: /sys/module/x_tables/initstate [ 58.806615] CPU 4 [ 58.806620] Modules linked in: xt_tcpudp iptable_filter ip_tables x_tables fuse nfsd exportfs nfs lockd fscache nfs_acl auth_rpcgss sunrpc snd_pcm snd_timer snd soundcore snd_page_alloc pcspkr evdev ext3 jbd mbcache dm_mod xen_netfront xen_blkfront [ 58.806668] Pid: 1561, comm: node Not tainted 2.6.32-5-xen-amd64 #1 [ 58.806673] RIP: e030:[<ffffffff81270c0b>] [<ffffffff81270c0b>] eth_type_trans+0x3d/0xae [ 58.806690] RSP: e02b:ffff8801cd07f988 EFLAGS: 00050246 [ 58.806695] RAX: ffff8801fb29ecfc RBX: ffff8801fbb44ce8 RCX: ffff8801fb29ecee [ 58.806702] RDX: 0000000000000042 RSI: 000000000000000e RDI: ffff8801fbb44ce8 [ 58.806709] RBP: ffff8801ff056000 R08: 0000000000000002 R09: ffffffff814eeab0 [ 58.806715] R10: 000000000000000e R11: ffffffff8125fbaf R12: ffff8801fd6a5b00 [ 58.806721] R13: ffff8801ff056000 R14: ffff8801ff150b00 R15: ffffffff81350270 [ 58.806732] FS: 00007f7e5b149700(0000) GS:ffff88000bb45000(0000) knlGS:0000000000000000 [ 58.806740] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 58.806745] CR2: 00007f7e59f9ffd0 CR3: 0000000001001000 CR4: 0000000000002660 [ 58.806752] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 58.806758] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 58.806765] Process node (pid: 1561, threadinfo ffff8801cd07e000, task ffff8801fdc0c6a0) [ 58.806772] Stack: [ 58.806775] 0000000000000002 ffff8801ff056000 ffff8801fbb44ce8 ffffffff812398d0 [ 58.806785] <0> 0000000000000004 ffff8801fbb44ce8 ffff8801fd6a5b00 ffffffff8125f9e4 [ 58.806797] <0> ffffffff81285100 0000000000000004 ffff8801ff150b00 ffff8801fd6a5b00 [ 58.806810] Call Trace: [ 58.806818] [<ffffffff812398d0>] ? loopback_xmit+0x36/0x7a [ 58.806827] [<ffffffff8125f9e4>] ? dev_hard_start_xmit+0x211/0x2db [ 58.806837] [<ffffffff81285100>] ? dst_output+0x0/0xb [ 58.806843] [<ffffffff8125fe8c>] ? dev_queue_xmit+0x2dd/0x38d [ 58.806850] [<ffffffff81287483>] ? ip_queue_xmit+0x311/0x386 [ 58.806861] [<ffffffff8100ecf2>] ? check_events+0x12/0x20 [ 58.806869] [<ffffffff81297e33>] ? tcp_transmit_skb+0x648/0x687 [ 58.806876] [<ffffffff8100ecf2>] ? check_events+0x12/0x20 [ 58.806887] [<ffffffff810fdfef>] ? d_kill+0x58/0x61 [ 58.806894] [<ffffffff8129a2b5>] ? tcp_write_xmit+0x874/0x96c [ 58.806902] [<ffffffff8129a3fa>] ? __tcp_push_pending_frames+0x22/0x53 [ 58.806910] [<ffffffff8128d7fd>] ? tcp_close+0x176/0x3d0 [ 58.806916] [<ffffffff810fdfef>] ? d_kill+0x58/0x61 [ 58.806924] [<ffffffff812aa2f8>] ? inet_release+0x4e/0x54 [ 58.806935] [<ffffffff81251121>] ? sock_release+0x19/0x66 [ 58.806942] [<ffffffff81251190>] ? sock_close+0x22/0x26 [ 58.806951] [<ffffffff810f09c9>] ? __fput+0x100/0x1af [ 58.806958] [<ffffffff810ede06>] ? filp_close+0x5b/0x62 [ 58.806967] [<ffffffff810508a0>] ? put_files_struct+0x64/0xc1 [ 58.806974] [<ffffffff8105215d>] ? do_exit+0x22e/0x6c6 [ 58.806981] [<ffffffff81052165>] ? do_exit+0x236/0x6c6 [ 58.806988] [<ffffffff8105266b>] ? do_group_exit+0x76/0x9d [ 58.806997] [<ffffffff8105eef7>] ? get_signal_to_deliver+0x310/0x339 [ 58.807005] [<ffffffff8101104f>] ? do_notify_resume+0x87/0x73f [ 58.807012] [<ffffffff8100ecf2>] ? check_events+0x12/0x20 [ 58.807020] [<ffffffff8100ecdf>] ? xen_restore_fl_direct_end+0x0/0x1 [ 58.807031] [<ffffffff8130cb3a>] ? _spin_unlock_irqrestore+0xd/0xe [ 58.807038] [<ffffffff813105cc>] ? kprobe_flush_task+0x9f/0xac [ 58.807048] [<ffffffff81049097>] ? finish_task_switch+0x96/0xaf [ 58.807056] [<ffffffff81011e0e>] ? int_signal+0x12/0x17 [ 58.807061] Code: 87 d8 00 00 00 2b 87 d0 00 00 00 be 0e 00 00 00 89 87 c4 00 00 00 e8 68 48 fe ff 8b 8b c4 00 00 00 48 03 8b d0 00 00 00 f6 01 01 <48> 8b 11 74 20 48 33 95 40 02 00 00 8a 43 7d 48 c1 e2 10 75 08 [ 58.807143] RIP [<ffffffff81270c0b>] eth_type_trans+0x3d/0xae [ 58.807151] RSP <ffff8801cd07f988> [ 58.807158] ---[ end trace e0ab4c430984e308 ]--- [ 58.807162] Kernel panic - not syncing: Fatal exception in interrupt [ 58.807169] Pid: 1561, comm: node Tainted: G D 2.6.32-5-xen-amd64 #1 [ 58.807175] Call Trace: [ 58.807181] [<ffffffff8130ac81>] ? panic+0x86/0x143 [ 58.807188] [<ffffffff8100ecdf>] ? xen_restore_fl_direct_end+0x0/0x1 [ 58.807197] [<ffffffff811ba120>] ? dummycon_dummy+0x0/0x3 [ 58.807204] [<ffffffff8100e63d>] ? xen_force_evtchn_callback+0x9/0xa [ 58.807211] [<ffffffff8100ecf2>] ? check_events+0x12/0x20 [ 58.807217] [<ffffffff811ba120>] ? dummycon_dummy+0x0/0x3 [ 58.807225] [<ffffffff8130d9d5>] ? oops_end+0xa7/0xb4 [ 58.807233] [<ffffffff81013416>] ? do_alignment_check+0x88/0x92 [ 58.807240] [<ffffffff81012a75>] ? alignment_check+0x25/0x30 [ 58.807247] [<ffffffff8125fbaf>] ? dev_queue_xmit+0x0/0x38d [ 58.807254] [<ffffffff81270c0b>] ? eth_type_trans+0x3d/0xae [ 58.807261] [<ffffffff81270bfb>] ? eth_type_trans+0x2d/0xae [ 58.807267] [<ffffffff812398d0>] ? loopback_xmit+0x36/0x7a [ 58.807274] [<ffffffff8125f9e4>] ? dev_hard_start_xmit+0x211/0x2db [ 58.807281] [<ffffffff81285100>] ? dst_output+0x0/0xb [ 58.807287] [<ffffffff8125fe8c>] ? dev_queue_xmit+0x2dd/0x38d [ 58.807294] [<ffffffff81287483>] ? ip_queue_xmit+0x311/0x386 [ 58.807301] [<ffffffff8100ecf2>] ? check_events+0x12/0x20 [ 58.807308] [<ffffffff81297e33>] ? tcp_transmit_skb+0x648/0x687 [ 58.807315] [<ffffffff8100ecf2>] ? check_events+0x12/0x20 [ 58.807322] [<ffffffff810fdfef>] ? d_kill+0x58/0x61 [ 58.807329] [<ffffffff8129a2b5>] ? tcp_write_xmit+0x874/0x96c [ 58.807336] [<ffffffff8129a3fa>] ? __tcp_push_pending_frames+0x22/0x53 [ 58.807343] [<ffffffff8128d7fd>] ? tcp_close+0x176/0x3d0 [ 58.807350] [<ffffffff810fdfef>] ? d_kill+0x58/0x61 [ 58.807356] [<ffffffff812aa2f8>] ? inet_release+0x4e/0x54 [ 58.807363] [<ffffffff81251121>] ? sock_release+0x19/0x66 [ 58.807370] [<ffffffff81251190>] ? sock_close+0x22/0x26 [ 58.807376] [<ffffffff810f09c9>] ? __fput+0x100/0x1af [ 58.807382] [<ffffffff810ede06>] ? filp_close+0x5b/0x62 [ 58.807389] [<ffffffff810508a0>] ? put_files_struct+0x64/0xc1 [ 58.807396] [<ffffffff8105215d>] ? do_exit+0x22e/0x6c6 [ 58.807402] [<ffffffff81052165>] ? do_exit+0x236/0x6c6 [ 58.807409] [<ffffffff8105266b>] ? do_group_exit+0x76/0x9d [ 58.807415] [<ffffffff8105eef7>] ? get_signal_to_deliver+0x310/0x339 [ 58.807422] [<ffffffff8101104f>] ? do_notify_resume+0x87/0x73f [ 58.807429] [<ffffffff8100ecf2>] ? check_events+0x12/0x20 [ 58.807436] [<ffffffff8100ecdf>] ? xen_restore_fl_direct_end+0x0/0x1 [ 58.807443] [<ffffffff8130cb3a>] ? _spin_unlock_irqrestore+0xd/0xe [ 58.807450] [<ffffffff813105cc>] ? kprobe_flush_task+0x9f/0xac [ 58.807457] [<ffffffff81049097>] ? finish_task_switch+0x96/0xaf [ 58.807463] [<ffffffff81011e0e>] ? int_signal+0x12/0x17 -R.