Weird issue where on one of my hosts dom0 partially locks up. Whats weird
is that alot of things work still and all domU''s are running fine.
SSHD
locks up on dom0 and cannot be restarted a xm works fine too. Here is a
dump from dmesg that i think is what started the whole mess.
bwmonitor is a perl script i wrote that basically loops through all chains
(2 chains per domU) running ebtables -L <chain> grabs the rates and writes
them to a database, simple script... From what i can tell this dump looks
like it may be a issue with xen and ebtables?
------------[ cut here ]------------
kernel BUG at arch/i386/mm/pgtable-xen.c:306!
invalid opcode: 0000 [#88]
SMP
Modules linked in: ebt_arp ebt_ip xt_physdev iptable_filter ip_tables
x_tables ebtable_filter ebtables ipv6 binfmt_misc video thermal processor
fan button tery ac ehci_hcd e1000 floppy dm_snapshot dm_zero dm_mirror ext3
mbcache jbd dm_mod 3w_9xxx sd_mod scsi_mod
CPU: 0
EIP: 0061:[<c0111ab5>] Not tainted VLI
EFLAGS: 00010282 (2.6.16-xen0 #2)
EIP is at pgd_ctor+0x30/0x3a
eax: fffffff4 ebx: 00000001 ecx: 00000001 edx: 00000000
esi: c0741ec0 edi: dbde65bc ebp: eaad7dc8 esp: eaad7dbc
ds: 007b es: 007b ss: 0069
Process bwmonitor (pid: 25477, threadinfo=eaad6000 task=e25f4a90)
Stack: <0>d37f0000 00000000 00000020 eaad7de8 c015b53e d37f0000 c0741ec0
00000001
dbde65a0 d37f0000 c073ff00 eaad7e1c c015b749 c0741ec0 dbde65a0
00000001
000000d0 c073ff2c 00000001 000000d0 00000000 ebc30c80 c073ff10
0000000c
Call Trace:
[<c01051c8>] show_stack_log_lvl+0xaa/0xdf
[<c01053d2>] show_registers+0x193/0x20b
[<c01055db>] die+0x10e/0x1a4
[<c01056ef>] do_trap+0x7e/0xb4
[<c0105a25>] do_invalid_op+0xb5/0xbf
[<c0104e37>] error_code+0x2b/0x30
[<c015b53e>] cache_init_objs+0x4f/0x77
[<c015b749>] cache_grow+0x133/0x1ce
[<c015b9ba>] cache_alloc_refill+0x1d6/0x21b
[<c015bc18>] kmem_cache_alloc+0x86/0x8a
[<c0111b03>] pgd_alloc+0x1e/0x315
[<c011ae6d>] mm_init+0x88/0xb3
[<c011b0b8>] dup_mm+0x58/0x311
[<c011b3d4>] copy_mm+0x63/0x82
[<c011bce0>] copy_process+0x3d9/0xe11
[<c011c812>] do_fork+0x69/0x195
[<c010387b>] sys_clone+0x3b/0x3d
[<c0104ca5>] syscall_call+0x7/0xb
Code: 0c 80 3d 84 f3 39 c0 00 74 02 c9 c3 c7 44 24 08 20 00 00 00 c7 44 24
04 00 00 00 00 8b 45 08 89 04 24 e8 ae 39 00 00 85 c0 74 df
c0 c9 c3 55 89 e5 83 ec 08 80 3d 84 f3 39
<0>------------[ cut here ]------------
kernel BUG at arch/i386/mm/pgtable-xen.c:306!
invalid opcode: 0000 [#89]
SMP
Modules linked in: ebt_arp ebt_ip xt_physdev iptable_filter ip_tables
x_tables ebtable_filter ebtables ipv6 binfmt_misc video thermal p
tery ac ehci_hcd e1000 floppy dm_snapshot dm_zero dm_mirror ext3 mbcache jbd
dm_mod 3w_9xxx sd_mod scsi_mod
CPU: 0
EIP: 0061:[<c0111ab5>] Not tainted VLI
EFLAGS: 00010282 (2.6.16-xen0 #2)
EIP is at pgd_ctor+0x30/0x3a
eax: fffffff4 ebx: 00000001 ecx: 00000001 edx: 00000000
esi: c0741ec0 edi: eb7c1e9c ebp: d2ea3dc8 esp: d2ea3dbc
ds: 007b es: 007b ss: 0069
Process bwmonitor (pid: 25480, threadinfo=d2ea2000 task=e99dca50)
Stack: <0>d03b0000 00000000 00000020 d2ea3de8 c015b53e d03b0000 c0741ec0
00000001
eb7c1e80 d03b0000 c073ff00 d2ea3e1c c015b749 c0741ec0 eb7c1e80
00000001
000000d0 c073ff2c 00000001 000000d0 00000000 ebc30040 c073ff10
0000000c
Call Trace:
[<c01051c8>] show_stack_log_lvl+0xaa/0xdf
[<c01053d2>] show_registers+0x193/0x20b
[<c01055db>] die+0x10e/0x1a4
[<c01056ef>] do_trap+0x7e/0xb4
[<c0105a25>] do_invalid_op+0xb5/0xbf
[<c0104e37>] error_code+0x2b/0x30
[<c015b53e>] cache_init_objs+0x4f/0x77
[<c015b749>] cache_grow+0x133/0x1ce
[<c015b9ba>] cache_alloc_refill+0x1d6/0x21b
[<c015bc18>] kmem_cache_alloc+0x86/0x8a
[<c0111b03>] pgd_alloc+0x1e/0x315
[<c011ae6d>] mm_init+0x88/0xb3
[<c011b0b8>] dup_mm+0x58/0x311
[<c011b3d4>] copy_mm+0x63/0x82
[<c011bce0>] copy_process+0x3d9/0xe11
[<c011c812>] do_fork+0x69/0x195
[<c010387b>] sys_clone+0x3b/0x3d
[<c0104ca5>] syscall_call+0x7/0xb
Code: 0c 80 3d 84 f3 39 c0 00 74 02 c9 c3 c7 44 24 08 20 00 00 00 c7 44 24
04 00
c0 c9 c3 55 89 e5 83 ec 08 80 3d 84 f3 39
--
~Shaun
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On 18 Jun 2006, at 22:21, Shaun wrote:> Weird issue where on one of my hosts dom0 partially locks up. Whats > weird > is that alot of things work still and all domU''s are running fine. > SSHD > locks up on dom0 and cannot be restarted a xm works fine too. Here is > a > dump from dmesg that i think is what started the whole mess. > > bwmonitor is a perl script i wrote that basically loops through all > chains > (2 chains per domU) running ebtables -L <chain> grabs the rates and > writes > them to a database, simple script... From what i can tell this dump > looks > like it may be a issue with xen and ebtables?Which Xen version are you running? You probably need to specify lowmem_emergency_pool=16M, or similar, on the Xen command line. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Here''s a xm info, hope that gives you everything you need. What does this lowmem_emergency_pool=16M do and why is it needed? Is the problem a memory issue? This host has 8GB, 1GB allocated to dom0 and about 6GB allocated to DomU''s release : 2.6.16-xen0 version : #2 SMP Wed Apr 12 15:23:41 PDT 2006 machine : i686 nr_cpus : 2 nr_nodes : 1 sockets_per_node : 2 cores_per_socket : 1 threads_per_core : 1 cpu_mhz : 3000 hw_caps : bfebfbff:20000000:00000000:00000180:0000641d total_memory : 8192 free_memory : 1840 xen_major : 3 xen_minor : 0 xen_extra : -unstable xen_caps : xen-3.0-x86_32p platform_params : virt_start=0xf5800000 xen_changeset : Thu Mar 30 17:50:44 2006 +0100 9487:9316fe0c9c4f cc_compiler : gcc version 3.4.5 20051201 (Red Hat 3.4.5-2) cc_compile_by : root cc_compile_domain : localdomain cc_compile_date : Mon Apr 10 22:44:44 PDT 2006 -- ~Shaun "Keir Fraser" <Keir.Fraser@cl.cam.ac.uk> wrote in message news:2111d9e07819fa86d5dcf001c649026d@cl.cam.ac.uk...> > On 18 Jun 2006, at 22:21, Shaun wrote: > >> Weird issue where on one of my hosts dom0 partially locks up. Whats >> weird >> is that alot of things work still and all domU''s are running fine. SSHD >> locks up on dom0 and cannot be restarted a xm works fine too. Here is a >> dump from dmesg that i think is what started the whole mess. >> >> bwmonitor is a perl script i wrote that basically loops through all >> chains >> (2 chains per domU) running ebtables -L <chain> grabs the rates and >> writes >> them to a database, simple script... From what i can tell this dump >> looks >> like it may be a issue with xen and ebtables? > > Which Xen version are you running? You probably need to specify > lowmem_emergency_pool=16M, or similar, on the Xen command line. > > -- Keir_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 18 Jun 2006, at 22:44, Shaun wrote:> Here''s a xm info, hope that gives you everything you need. What does > this > lowmem_emergency_pool=16M do and why is it needed? Is the problem a > memory > issue? This host has 8GB, 1GB allocated to dom0 and about 6GB > allocated to > DomU''sWhen running in PAE mode you need a supply of memory below 4GB as each process needs its root page table to be located below 4GB. Xen isn''t very careful about ensuring it doesn''t give away all that memory and, if you run out, you get crashes of the type you''ve been seeing. The fix is to specifically reserve some of that low memory so that it is only given out when that type of memory is specifically requested. This limitation is gone from the xen-unstable tree (future Xen 3.0.3). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel