Hi,
This problem just occurs on a small ubuntu 8.04 LTS host.
On this host, there is just 1 VM running.
The problem always occurs, when the VM is up for about 2-3days.
Suddenly, the machine hangs and you cannot connect to it using ssh.
( Just a xm destroy and xm create helps to get the VM running again. )
when i open the xen console of the vm, i start seeing such strange error
messages:
[148030.316941] [<c011cdb3>] dequeue_entity+0x13/0x40
[148030.316952] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[148030.316963] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[148030.316974] [<c0167ecf>] shrink_inactive_list+0x11f/
0x3b0
[148030.316991] [<c01681fc>] shrink_zone+0x9c/0x100
[148030.317001] [<c016889c>] kswapd+0x44c/0x490
[148030.317014] [<c013bad0>] autoremove_wake_function+0x0/0x40
[148030.317027] [<c011e270>] complete+0x40/0x60
[148030.317037] [<c0168450>] kswapd+0x0/0x490
[148030.317042] [<c013b812>] kthread+0x42/0x70
[148030.317046] [<c013b7d0>] kthread+0x0/0x70
[148030.317051] [<c0105bb7>] kernel_thread_helper+0x7/0x10
[148030.317057] ======================[148041.948368] BUG: soft lockup - CPU#0
stuck for 11s! [kswapd0:95]
[148041.948377]
[148041.948383] Pid: 95, comm: kswapd0 Tainted: G D (2.6.24-24-xen #1)
[148041.948390] EIP: 0061:[<c03285f7>] EFLAGS: 00000286 CPU: 0
[148041.948399] EIP is at _spin_lock+0x7/0x10
[148041.948405] EAX: cf802dd8 EBX: cf802d94 ECX: c17a33c0 EDX: 00000000
[148041.948410] ESI: c17a33c0 EDI: 00000000 EBP: cfdeff7c ESP: cfdefdcc
[148041.948416] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
[148041.948423] CR0: 8005003b CR2: 00100100 CR3: 0f5b7000 CR4: 00000660
[148041.948430] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[148041.948435] DR6: ffff0ff0 DR7: 00000400
[148041.948441] [<c01a6ebd>] try_to_free_buffers+0x2d/0x90
[148041.948456] [<c0167c75>] shrink_page_list+0x4c5/0x600
[148041.948471] [<c011cdb3>] dequeue_entity+0x13/0x40
[148041.948481] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[148041.948492] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[148041.948504] [<c0167ecf>] shrink_inactive_list+0x11f/0x3b0
[148041.948520] [<c01681fc>] shrink_zone+0x9c/0x100
[148041.948531] [<c016889c>] kswapd+0x44c/0x490
[148041.948544] [<c013bad0>] autoremove_wake_function+0x0/0x40
[148041.948557] [<c011e270>] complete+0x40/0x60
[148041.948566] [<c0168450>] kswapd+0x0/0x490
[148041.948571] [<c013b812>] kthread+0x42/0x70
[148041.948576] [<c013b7d0>] kthread+0x0/0x70
[148041.948581] [<c0105bb7>] kernel_thread_helper+0x7/0x10
[148041.948587] ======================[148053.543628] BUG: soft lockup - CPU#0
stuck for 11s! [kswapd0:95]
[148053.543638]
[148053.543644] Pid: 95, comm: kswapd0 Tainted: G D (2.6.24-24-xen #1)
[148053.543650] EIP: 0061:[<c03285f7>] EFLAGS: 00000286 CPU: 0
[148053.543660] EIP is at _spin_lock+0x7/0x10
[148053.543665] EAX: cf802dd8 EBX: cf802d94 ECX: c17a33c0 EDX: 00000000
[148053.543671] ESI: c17a33c0 EDI: 00000000 EBP: cfdeff7c ESP: cfdefdcc
[148053.543676] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
[148053.543684] CR0: 8005003b CR2: 00100100 CR3: 0f5b7000 CR4: 00000660
[148053.543690] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[148053.543696] DR6: ffff0ff0 DR7: 00000400
[148053.543701] [<c01a6ebd>] try_to_free_buffers+0x2d/0x90
[148053.543719] [<c0167c75>] shrink_page_list+0x4c5/0x600
[148053.543734] [<c011cdb3>] dequeue_entity+0x13/0x40
[148053.543745] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[148053.543755] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[148053.543767] [<c0167ecf>] shrink_inactive_list+0x11f/0x3b0
[148053.543784] [<c01681fc>] shrink_zone+0x9c/0x100
[148053.543795] [<c016889c>] kswapd+0x44c/0x490
[148053.543808] [<c013bad0>] autoremove_wake_function+0x0/0x40
[148053.543820] [<c011e270>] complete+0x40/0x60
[148053.543828] [<c0168450>] kswapd+0x0/0x490
[148053.543833] [<c013b812>] kthread+0x42/0x70
[148053.543837] [<c013b7d0>] kthread+0x0/0x70
[148053.543842] [<c0105bb7>] kernel_thread_helper+0x7/0x10
[148053.543849] ======================[148065.165553] BUG: soft lockup - CPU#0
stuck for 11s! [kswapd0:95]
[148065.165563]
[148065.165569] Pid: 95, comm: kswapd0 Tainted: G D (2.6.24-24-xen #1)
[148065.165575] EIP: 0061:[<c03285f5>] EFLAGS: 00000286 CPU: 0
[148065.165585] EIP is at _spin_lock+0x5/0x10
[148065.165590] EAX: cf802dd8 EBX: cf802d94 ECX: c17a33c0 EDX: 00000000
[148065.165596] ESI: c17a33c0 EDI: 00000000 EBP: cfdeff7c ESP: cfdefdcc
[148065.165601] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
[148065.165609] CR0: 8005003b CR2: 00100100 CR3: 0f5b7000 CR4: 00000660
[148065.165615] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[148065.165621] DR6: ffff0ff0 DR7: 00000400
[148065.165626] [<c01a6ebd>] try_to_free_buffers+0x2d/0x90
[148065.165642] [<c0167c75>] shrink_page_list+0x4c5/0x600
[148065.165656] [<c011cdb3>] dequeue_entity+0x13/0x40
[148065.165667] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[148065.165678] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[148065.165690] [<c0167ecf>] shrink_inactive_list+0x11f/0x3b0
[148065.165706] [<c01681fc>] shrink_zone+0x9c/0x100
[148065.165717] [<c016889c>] kswapd+0x44c/0x490
[148065.165730] [<c013bad0>] autoremove_wake_function+0x0/0x40
[148065.165742] [<c011e270>] complete+0x40/0x60
[148065.165752] [<c0168450>] kswapd+0x0/0x490
[148065.165757] [<c013b812>] kthread+0x42/0x70
[148065.165762] [<c013b7d0>] kthread+0x0/0x70
[148065.165767] [<c0105bb7>] kernel_thread_helper+0x7/0x10
[148065.165773] ======================[148076.738934] BUG: soft lockup - CPU#0
stuck for 11s! [kswapd0:95]
[148076.738943]
[148076.738950] Pid: 95, comm: kswapd0 Tainted: G D (2.6.24-24-xen #1)
[148076.738956] EIP: 0061:[<c03285f5>] EFLAGS: 00000286 CPU: 0
[148076.738965] EIP is at _spin_lock+0x5/0x10
[148076.738971] EAX: cf802dd8 EBX: cf802d94 ECX: c17a33c0 EDX: 00000000
[148076.738976] ESI: c17a33c0 EDI: 00000000 EBP: cfdeff7c ESP: cfdefdcc
[148076.738982] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
[148076.738989] CR0: 8005003b CR2: 00100100 CR3: 0f5b7000 CR4: 00000660
[148076.738996] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[148076.739001] DR6: ffff0ff0 DR7: 00000400
[148076.739007] [<c01a6ebd>] try_to_free_buffers+0x2d/0x90
[148076.739022] [<c0167c75>] shrink_page_list+0x4c5/0x600
[148076.739037] [<c011cdb3>] dequeue_entity+0x13/0x40
[148076.739048] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[148076.739059] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[148076.739070] [<c0167ecf>] shrink_inactive_list+0x11f/0x3b0
[148076.739087] [<c01681fc>] shrink_zone+0x9c/0x100
[148076.739097] [<c016889c>] kswapd+0x44c/0x490
[148076.739110] [<c013bad0>] autoremove_wake_function+0x0/0x40
[148076.739123] [<c011e270>] complete+0x40/0x60
[148076.739133] [<c0168450>] kswapd+0x0/0x490
[148076.739138] [<c013b812>] kthread+0x42/0x70
[148076.739142] [<c013b7d0>] kthread+0x0/0x70
[148076.739147] [<c0105bb7>] kernel_thread_helper+0x7/0x10
[148076.739154] ======================[148088.362709] BUG: soft lockup - CPU#0
stuck for 11s! [kswapd0:95]
[148088.362719]
[148088.362725] Pid: 95, comm: kswapd0 Tainted: G D (2.6.24-24-xen #1)
[148088.362731] EIP: 0061:[<c03285f7>] EFLAGS: 00000286 CPU: 0
[148088.362741] EIP is at _spin_lock+0x7/0x10
[148088.362746] EAX: cf802dd8 EBX: cf802d94 ECX: c17a33c0 EDX: 00000000
[148088.362754] ESI: c17a33c0 EDI__rmqueue_smallest+0xb7/0x130
[178204.806034] [<c02de99b>] __inet_lookup_listener+0x1b/0x140
[178204.806041] [<c02f589d>] tcp_v4_rcv+0x7bd/0x9a0
[178204.806049] [<c02d72b9>] ip_local_deliver_finish+0xf9/0x210
[178204.806055] [<c02d6f4f>] ip_rcv_finish+0xff/0x370
[178204.806061] [<c01082e3>] sched_clock+0x23/0x70
[178204.806067] [<c02d74c0>] ip_rcv+0x0/0x290
[178204.806072] [<c02b4419>] netif_receive_skb+0x399/0x480
[178204.806079] [<c01623d9>] free_hot_cold_page+0x139/0x220
[178204.806085] [<c026bd97>] netif_poll+0xa67/0[187595.019177] BUG: soft
lockup - CPU#0 stuck for 11s! [kswapd0:95]
[187595.019185]
[187595.019189] Pid: 95, comm: kswapd0 Tainted: G D (2.6.24-24-xen #1)
[187595.019193] EIP: 0061:[<c03285f7>] EFLAGS: 00000286 CPU: 0
[187595.019200] EIP is at _spin_lock+0x7/0x10
[187595.019203] EAX: cf802dd8 EBX: cf802d94 ECX: c17a33c0 EDX: 00000000
[187595.019206] ESI: c17a33c0 EDI: 00000000 EBP: cfdeff7c ESP: cfdefdcc
[187595.019210] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
[187595.019215] CR0: 8005003b CR2: 00100100 CR3: 0f5b7000 CR4: 00000660
[187595.019219] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[187595.019223] DR6: ffff0ff0 DR7: 00000400
[187595.019226] [<c01a6ebd>] try_to_free_buffers+0x2d/0x90
[187595.019236] [<c0167c75>] shrink_page_list+0x4c5/0x600
[187595.019245] [<c011cdb3>] dequeue_entity+0x13/0x40
[187595.019250] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[187595.019256] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[187595.019262] [<c0167ecf>] shrink_inactive_list+0x11f/0x3b0
[187595.019274] [<c01681fc>] shrink_zone+0x9c/0x100
[187595.019279] [<c016889c>] kswapd+0x44c/0x490
[187595.019287] [<c013bad0>] autoremove_wake_function+0x0/0x40
[187595.019294] [<c011e270>] complete+0x40/0x60
[187699.479450] [<c0168450>] kswapd+0x0/0x490
[187699.479454] [<c013b812>] kthread+0x42/0x70
[187699.479459] [<c013b7d0>] kthread+0x0/0x70
[187699.479464] [<c0105bb7>] kernel_thread_helper+0x7/0x10
[187699.479470] ======================[187711.037292] BUG: soft lockup - CPU#0
stuck for 11s! [kswapd0:95]
[187711.037301]
[187711.037308] Pid: 95, comm: kswapd0 Tainted: G D (2.6.24-24-xen #1)
[187711.037315] EIP: 0061:[<c03285f5>] EFLAGS: 00000286 CPU: 0
[187711.037325] EIP is at _spin_lock+0x5/0x10
[187711.037332] EAX: cf802dd8 EBX: cf802d94 ECX: c17a33c0 EDX: 00000000
[187711.037338] ESI: c17a33c0 EDI: 00000000 EBP: cfdeff7c ESP: cfdefdcc
[187711.037345] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
[187711.037353] CR0: 8005003b CR2: 00100100 CR3: 0f5b7000 CR4: 00000660
[187711.037360] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[187711.037367] DR6: ffff0ff0 DR7: 00000400
[187711.037373] [<c01a6ebd>] try_to_free_buffers+0x2d/0x90
[187711.037390] [<c0167c75>] shrink_page_list+0x4c5/0x600
[187711.037407] [<c011cdb3>] dequeue_entity+0x13/0x40
[187711.037420] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[187711.037433] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[187711.037448] [<c0167ecf>] shrink_inactive_list+0x11f/0x3b0
[187711.037467] [<c01681fc>] shrink_zone+0x9c/0x100
[187711.037480] [<c016889c>] kswapd+0x44c/0x490
[187711.037496] [<c013bad0>] autoremove_wake_function+0x0/0x40
[187711.037511] [<c011e270>] complete+0x40/0x60
[187711.037524] [<c0168450>] kswapd+0x0/0x490
[187711.037529] [<c013b812>] kthread+0x42/0x70
[187711.037534] [<c013b7d0>] kthread+0x0/0x70
[187711.037539] [<c0105bb7>] kernel_thread_helper+0x7/0x10
[187711.037545] ======================[187722.652134] BUG: soft lockup - CPU#0
stuck for 11s! [kswapd0:95]
[187722.652143]
[187722.652150] Pid: 95, comm: kswapd0 Tainted: G D (2.6.24-24-xen #1)
[187722.652157] EIP: 0061:[<c03285f5>] EFLAGS: 00000286 CPU: 0
[187722.652168] EIP is at _spin_lock+0x5/0x10
[187722.652174] EAX: cf802dd8 EBX: cf802d94 ECX: c17a33c0 EDX: 00000000
[187722.652180] ESI: c17a33c0 EDI: 00000000 EBP: cfdeff7c ESP: cfdefdcc
[187722.652187] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
[187722.652195] CR0: 8005003b CR2: 00100100 CR3: 0f5b7000 CR4: 00000660
[187722.652202] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[187722.652209] DR6: ffff0ff0 DR7: 00000400
[187722.652215] [<c01a6ebd>] try_to_free_buffers+0x2d/0x90
[187722.652233] [<c0167c75>] shrink_page_list+0x4c5/0x600
[187722.652252] [<c011cdb3>] dequeue_entity+0x13/0x40
[187722.652265] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[187722.652284] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[187722.652306] [<c0167ecf>] shrink_inactive_list+0x11f/0x3b0
[187722.652329] [<c01681fc>] shrink_zone+0x9c/0x100
[187722.652347] [<c016889c>] kswapd+0x44c/0x490
[187722.652372] [<c013bad0>] autoremove_wake_function+0x0/0x40
[187722.652387] [<c011e270>] complete+0x40/0x60
[187722.652400] [<c0168450>] kswapd+0x0/0x490
[187722.652406] [<c013b812>] kthread+0x42/0x70
[187722.652410] [<c013b7d0>] kthread+0x0/0x70
[187722.652415] [<c0105bb7>] kernel_thread_helper+0x7/0x10
[187722.652426] ======================[187734.268601] BUG: soft lockup - CPU#0
stuck for 11s! [kswapd0:95]
[187734.268610]
[187734.268617] Pid: 95, comm: kswapd0 Tainted: G D (2.6.24-24-xen #1)
[187734.268624] EIP: 0061:[<c03285f5>] EFLAGS: 00000286 CPU: 0
[187734.268635] EIP is at _spin_lock+0x5/0x10
[187734.268641] EAX: cf802dd8 EBX: cf802d94 ECX: c17a33c0 EDX: 00000000
[187734.268647] ESI: c17a33c0 EDI: 00000000 EBP: cfdeff7c ESP: cfdefdcc
[187734.268654] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
[187734.268662] CR0: 8005003b CR2: 00100100 CR3: 0f5b7000 CR4: 00000660
[187734.268669] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[187734.268676] DR6: ffff0ff0 DR7: 00000400
[187734.268682] [<c01a6ebd>] try_to_free_buffers+0x2d/0x90
[187734.268700] [<c0167c75>] shrink_page_list+0x4c5/0x600
[187734.268717] [<c011cdb3>] dequeue_entity+0x13/0x40
[187734.268730] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[187734.268744] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[187734.268758] [<c0167ecf>] shrink_inactive_list+0x11f/0x3b0
[187734.268777] [<c01681fc>] shrink_zone+0x9c/0x100
[187734.268795] [<c016889c>] kswapd+0x44c/0x490
[187734.268811] [<c013bad0>] autoremove_wake_function+0x0/0x40
[187734.268826] [<c011e270>] complete+0x40/0x60
[187734.268839] [<c0168450>] kswapd+0x0/0x490
[187734.268845] [<c013b812>] kthread+0x42/0x70
[187734.268850] [<c013b7d0>] kthread+0x0/0x70
[187734.268854] [<c0105bb7>] kernel_thread_helper+0x7/0x10
[187734.268861] ======================[187745.853420] BUG: soft lockup - CPU#0
stuck for 11s! [kswapd0:95]
[187745.853429]
[187745.853436] Pid: 95, comm: kswapd0 Tainted: G D (2.6.24-24-xen #1)
[187745.853443] EIP: 0061:[<c03285f7>] EFLAGS: 00000286 CPU: 0
[187745.853454] EIP is at _spin_lock+0x7/0x10
[187745.853460] EAX: cf802dd8 EBX: cf802d94 ECX: c17a33c0 EDX: 00000000
[187745.853466] ESI: c17a33c0 EDI: 00000000 EBP: cfdeff7c ESP: cfdefdcc
[187745.853473] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
[187745.853482] CR0: 8005003b CR2: 00100100 CR3: 0f5b7000 CR4: 00000660
[187745.853489] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[187745.853496] DR6: ffff0ff0 DR7: 00000400
[187745.853502] [<c01a6ebd>] try_to_free_buffers+0x2d/0x90
[187745.853520] [<c0167c75>] shrink_page_list+0x4c5/0x600
[187745.853537] [<c011cdb3>] dequeue_entity+0x13/0x40
[187745.853555] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[187745.853569] [<c0166e0f>] isolate_lru_pages+0x5f/0x1c0
[187745.853583] [<c0167ecf>] shrink_inactive_list+0x11f/0x3b0
[187745.853605] [<c01681fc>] shrink_zone+0x9c/0x100
[187745.853619] [<c016889c>] kswapd+0x44c/0x490
[187745.853636] [<c013bad0>] autoremove_wake_function+0x0/0x40
[187745.853654] [<c011e270>] complete+0x40/0x60
[187745.853667] [<c0168450>] kswapd+0x0/0x490
[187745.853672] [<c013b812>] kthread+0x42/0x70
[187745.853677] [<c013b7d0>] kthread+0x0/0x70
[187745.853682] [<c0105bb7>] kernel_thread_helper+0x7/0x10
[187745.853689] ======================
Does anyone know what to do there?
Thanks a lot in advance for any reply.
Marco Weber
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users