Pasi Kärkkäinen
2009-Mar-02 19:53 UTC
[Xen-devel] latest pv_ops dom0 (2.6.29-rc6) crashes / unhandled page fault
Hello. log of the crashing boot process: http://pasik.reaktio.net/xen/pv_ops-dom0-debug/pv_ops-dom0-bootlog-14-xen331-linux-2.6.29-rc6-crash.txt 2.6.29-rc5 based pv_ops dom0 works on the same computer.. with the same Xen version. I tried with and without pci=nomsi. adding ''noapic'' doesn''t seem to help (I need that with 2.6.29-rc5 to make it work). Any ideas what''s going on? (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 112kB init memory. mapping kernel into physical memory (XEN) d0:v0: unhandled page fault (ec=0000) (XEN) Pagetable walk from bffff000: (XEN) L3[0x002] = 000000003d204001 00001204 (XEN) L2[0x1ff] = 0000000000000000 ffffffff (XEN) domain_crash_sync called from entry.S (ff1a2c9e) (XEN) Domain 0 (vcpu#0) crashed on cpu#0: (XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) EIP: e019:[<c0879716>] (XEN) EFLAGS: 00000283 EM: 1 CONTEXT: pv guest (XEN) eax: fffff000 ebx: 00000000 ecx: 00000016 edx: 00000000 (XEN) esi: 00035800 edi: bffff000 ebp: c086efd4 esp: c086ef9c (XEN) cr0: 8005003b cr4: 000006f0 cr3: 3d201000 cr2: bffff000 (XEN) ds: e021 es: e021 fs: e021 gs: e021 ss: e021 cs: e019 (XEN) Guest stack trace from esp=c086ef9c: (XEN) 00000000 c0879716 0001e019 00010083 00000000 c086efb8 c1201000 c1222000 (XEN) 000c1690 000001ac 00035800 c1201000 c08a69f8 c082fa64 c086effc c0878c2e (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 c1200000 00000000 (XEN) 00000000 c087322e c06d7f7b c06d7f83 c0404677 c0879211 c0404ee4 c0405051 (XEN) c04051a0 c0405418 c04054c7 c0405522 c0405787 c0405824 c0405c6f c0879ac6 (XEN) c0879ace c06d7fe5 c0879b2f c06d8087 c06d80b8 c0879bb3 c0879bfe c040715d (XEN) c04072ae c04077ed c04078a9 c04078d9 c0407901 c0407a27 c0407f03 c0407f59 (XEN) c0407fd9 c04085a8 c0408a4f c0408a6b c0408a82 c0408efa c087a383 c087a433 (XEN) c06e1057 c06e113d c06e1367 c040b187 c087a9d8 c087a9e0 c040bb72 c040bcc6 (XEN) c087b157 c087beb4 c087bebc c040d0e0 c040d0f3 c040d15e c040d171 c087d670 (XEN) c087d686 c040e16a c040e1e8 c040e6eb c040e752 c040e77f c040e834 c040f789 (XEN) c040fc2c c040fc36 c0410162 c0410377 c04103e3 c04105e2 c0410a60 c0410ae6 (XEN) c0410aef c0410b55 c0410b5e c0410c5d c0410dc6 c0410dcf c0410e48 c0410e51 (XEN) c0410f36 c0410f3f c0410fcd c0410fd6 c0411655 c0411660 c041166b c041174b (XEN) c04117d0 c04117e2 c04117f3 c0411861 c06d8d59 c06d9021 c06d909e c06d91eb (XEN) c06d923d c06d9269 c06d9300 c06d94af c087dd1c c087dd29 c087dd31 c087dd39 (XEN) c087dd4b c087dd53 c087dd79 c087dd81 c087ddf6 c087ddfe c06d970b c06d99b3 (XEN) c06d9a4b c06da007 c06da07f c06da192 c06da19a c06da25a c06da283 c06da28b (XEN) c06da293 c06da2b0 c06da33e c06da3be c06da424 c06da435 c06da461 c06da46e (XEN) c06da494 c06da505 c06da512 c06da578 c06da57d c06da582 c06da587 c06da58c (XEN) Domain 0 crashed: rebooting machine in 5 seconds. I''m using 32bit PAE, as both dom0 kernel and Xen hypervisor. -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2009-Mar-02 19:55 UTC
Re: [Xen-devel] latest pv_ops dom0 (2.6.29-rc6) crashes / unhandled page fault
On Mon, Mar 02, 2009 at 09:53:17PM +0200, Pasi Kärkkäinen wrote:> Hello. > > log of the crashing boot process: > http://pasik.reaktio.net/xen/pv_ops-dom0-debug/pv_ops-dom0-bootlog-14-xen331-linux-2.6.29-rc6-crash.txt > > 2.6.29-rc5 based pv_ops dom0 works on the same computer.. with the same Xen version. > > I tried with and without pci=nomsi. > > adding ''noapic'' doesn''t seem to help (I need that with 2.6.29-rc5 to make it work). > > Any ideas what''s going on? >Oh, and forgot to mention that the same kernel works on baremetal without Xen just fine.. -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
M A Young
2009-Mar-02 20:07 UTC
Re: [Xen-devel] latest pv_ops dom0 (2.6.29-rc6) crashes / unhandled page fault
On Mon, 2 Mar 2009, Pasi Kärkkäinen wrote:> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) > (XEN) Freed 112kB init memory. > mapping kernel into physical memory > (XEN) d0:v0: unhandled page fault (ec=0000) > (XEN) Pagetable walk from bffff000: > (XEN) L3[0x002] = 000000003d204001 00001204 > (XEN) L2[0x1ff] = 0000000000000000 ffffffff > (XEN) domain_crash_sync called from entry.S (ff1a2c9e) > (XEN) Domain 0 (vcpu#0) crashed on cpu#0: > (XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]----... What is your setting for CONFIG_CC_STACKPROTECTOR in your compile? There are bits of xen_kernel_start that break if you have this enabled. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2009-Mar-02 20:11 UTC
Re: [Xen-devel] latest pv_ops dom0 (2.6.29-rc6) crashes / unhandled page fault
On Mon, Mar 02, 2009 at 08:07:56PM +0000, M A Young wrote:> On Mon, 2 Mar 2009, Pasi Kärkkäinen wrote: > > >(XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input > >to Xen) > >(XEN) Freed 112kB init memory. > >mapping kernel into physical memory > >(XEN) d0:v0: unhandled page fault (ec=0000) > >(XEN) Pagetable walk from bffff000: > >(XEN) L3[0x002] = 000000003d204001 00001204 > >(XEN) L2[0x1ff] = 0000000000000000 ffffffff > >(XEN) domain_crash_sync called from entry.S (ff1a2c9e) > >(XEN) Domain 0 (vcpu#0) crashed on cpu#0: > >(XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]---- > ... > > What is your setting for CONFIG_CC_STACKPROTECTOR in your compile? There > are bits of xen_kernel_start that break if you have this enabled. >[root@dom0test linux-2.6-xen]# grep CONFIG_CC_STACKPROTECTOR .config # CONFIG_CC_STACKPROTECTOR is not set So it''s not that.. -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
M A Young
2009-Mar-02 20:36 UTC
Re: [Xen-devel] latest pv_ops dom0 (2.6.29-rc6) crashes / unhandled page fault
On Mon, 2 Mar 2009, Pasi Kärkkäinen wrote:> On Mon, Mar 02, 2009 at 08:07:56PM +0000, M A Young wrote: >> On Mon, 2 Mar 2009, Pasi Kärkkäinen wrote: >> >>> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input >>> to Xen) >>> (XEN) Freed 112kB init memory. >>> mapping kernel into physical memory >>> (XEN) d0:v0: unhandled page fault (ec=0000) >>> (XEN) Pagetable walk from bffff000: >>> (XEN) L3[0x002] = 000000003d204001 00001204 >>> (XEN) L2[0x1ff] = 0000000000000000 ffffffff >>> (XEN) domain_crash_sync called from entry.S (ff1a2c9e) >>> (XEN) Domain 0 (vcpu#0) crashed on cpu#0: >>> (XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]---- >> ... >> >> What is your setting for CONFIG_CC_STACKPROTECTOR in your compile? There >> are bits of xen_kernel_start that break if you have this enabled. >> > > [root@dom0test linux-2.6-xen]# grep CONFIG_CC_STACKPROTECTOR .config > # CONFIG_CC_STACKPROTECTOR is not set > > So it''s not that..Strange, because it looked pretty similar to the second crash of http://lists.xensource.com/archives/html/xen-devel/2009-02/msg00852.html and the followup http://lists.xensource.com/archives/html/xen-devel/2009-02/msg00875.html It might be worth double-checking the state of -fstack-protector -fno-stack-protector -fstack-protector-all etc. in your Makefile. Michael Young _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2009-Mar-03 01:38 UTC
Re: [Xen-devel] latest pv_ops dom0 (2.6.29-rc6) crashes / unhandled page fault
Pasi Kärkkäinen wrote:> Hello. > > log of the crashing boot process: > http://pasik.reaktio.net/xen/pv_ops-dom0-debug/pv_ops-dom0-bootlog-14-xen331-linux-2.6.29-rc6-crash.txt > > 2.6.29-rc5 based pv_ops dom0 works on the same computer.. with the same Xen version. > > I tried with and without pci=nomsi. > > adding ''noapic'' doesn''t seem to help (I need that with 2.6.29-rc5 to make it work). >Yeah, something got broken. Tracking it down now.> Any ideas what''s going on? > > (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) > (XEN) Freed 112kB init memory. > mapping kernel into physical memory > (XEN) d0:v0: unhandled page fault (ec=0000) > (XEN) Pagetable walk from bffff000: > (XEN) L3[0x002] = 000000003d204001 00001204 > (XEN) L2[0x1ff] = 0000000000000000 ffffffff > (XEN) domain_crash_sync called from entry.S (ff1a2c9e) > (XEN) Domain 0 (vcpu#0) crashed on cpu#0: > (XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]---- > (XEN) CPU: 0 > (XEN) EIP: e019:[<c0879716>] >What does this map to in your vmlinux? J> (XEN) EFLAGS: 00000283 EM: 1 CONTEXT: pv guest > (XEN) eax: fffff000 ebx: 00000000 ecx: 00000016 edx: 00000000 > (XEN) esi: 00035800 edi: bffff000 ebp: c086efd4 esp: c086ef9c > (XEN) cr0: 8005003b cr4: 000006f0 cr3: 3d201000 cr2: bffff000 > (XEN) ds: e021 es: e021 fs: e021 gs: e021 ss: e021 cs: e019 > (XEN) Guest stack trace from esp=c086ef9c: > (XEN) 00000000 c0879716 0001e019 00010083 00000000 c086efb8 c1201000 c1222000 > (XEN) 000c1690 000001ac 00035800 c1201000 c08a69f8 c082fa64 c086effc c0878c2e > (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 c1200000 00000000 > (XEN) 00000000 c087322e c06d7f7b c06d7f83 c0404677 c0879211 c0404ee4 c0405051 > (XEN) c04051a0 c0405418 c04054c7 c0405522 c0405787 c0405824 c0405c6f c0879ac6 > (XEN) c0879ace c06d7fe5 c0879b2f c06d8087 c06d80b8 c0879bb3 c0879bfe c040715d > (XEN) c04072ae c04077ed c04078a9 c04078d9 c0407901 c0407a27 c0407f03 c0407f59 > (XEN) c0407fd9 c04085a8 c0408a4f c0408a6b c0408a82 c0408efa c087a383 c087a433 > (XEN) c06e1057 c06e113d c06e1367 c040b187 c087a9d8 c087a9e0 c040bb72 c040bcc6 > (XEN) c087b157 c087beb4 c087bebc c040d0e0 c040d0f3 c040d15e c040d171 c087d670 > (XEN) c087d686 c040e16a c040e1e8 c040e6eb c040e752 c040e77f c040e834 c040f789 > (XEN) c040fc2c c040fc36 c0410162 c0410377 c04103e3 c04105e2 c0410a60 c0410ae6 > (XEN) c0410aef c0410b55 c0410b5e c0410c5d c0410dc6 c0410dcf c0410e48 c0410e51 > (XEN) c0410f36 c0410f3f c0410fcd c0410fd6 c0411655 c0411660 c041166b c041174b > (XEN) c04117d0 c04117e2 c04117f3 c0411861 c06d8d59 c06d9021 c06d909e c06d91eb > (XEN) c06d923d c06d9269 c06d9300 c06d94af c087dd1c c087dd29 c087dd31 c087dd39 > (XEN) c087dd4b c087dd53 c087dd79 c087dd81 c087ddf6 c087ddfe c06d970b c06d99b3 > (XEN) c06d9a4b c06da007 c06da07f c06da192 c06da19a c06da25a c06da283 c06da28b > (XEN) c06da293 c06da2b0 c06da33e c06da3be c06da424 c06da435 c06da461 c06da46e > (XEN) c06da494 c06da505 c06da512 c06da578 c06da57d c06da582 c06da587 c06da58c > (XEN) Domain 0 crashed: rebooting machine in 5 seconds. > > I''m using 32bit PAE, as both dom0 kernel and Xen hypervisor. > > -- Pasi > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2009-Mar-03 10:16 UTC
Re: [Xen-devel] latest pv_ops dom0 (2.6.29-rc6) crashes / unhandled page fault
On Mon, 2009-03-02 at 20:38 -0500, Jeremy Fitzhardinge wrote:> What does this map to in your vmlinux?I get the same sort of thing, this particular one is from a domU, the EIP is (gdb) list *0xc0103f06 0xc0103f06 is in get_phys_to_machine (/local/scratch/ianc/devel/kernels/linux-2.6/arch/x86/xen/mmu.c:253). 248 if (unlikely(pfn >= MAX_DOMAIN_PAGES)) 249 return INVALID_P2M_ENTRY; 250 251 topidx = p2m_top_index(pfn); 252 idx = p2m_index(pfn); 253 return p2m_top[topidx][idx]; 254 } 255 EXPORT_SYMBOL_GPL(get_phys_to_machine); 256 257 /* install a new p2m_top page */ (gdb) disas 0xc0103f06 Dump of assembler code for function get_phys_to_machine: 0xc0103ee0 <get_phys_to_machine+0>: mov %eax,%ecx 0xc0103ee2 <get_phys_to_machine+2>: mov $0xffffffff,%eax 0xc0103ee7 <get_phys_to_machine+7>: push %ebp 0xc0103ee8 <get_phys_to_machine+8>: cmp $0x1fffff,%ecx 0xc0103eee <get_phys_to_machine+14>: mov %esp,%ebp 0xc0103ef0 <get_phys_to_machine+16>: ja 0xc0103f09 <get_phys_to_machine+41> 0xc0103ef2 <get_phys_to_machine+18>: mov 0xc0449a88,%edx 0xc0103ef8 <get_phys_to_machine+24>: mov %ecx,%eax 0xc0103efa <get_phys_to_machine+26>: and $0x3ff,%ecx 0xc0103f00 <get_phys_to_machine+32>: shr $0xa,%eax 0xc0103f03 <get_phys_to_machine+35>: mov (%edx,%eax,4),%eax 0xc0103f06 <get_phys_to_machine+38>: mov (%eax,%ecx,4),%eax 0xc0103f09 <get_phys_to_machine+41>: pop %ebp 0xc0103f0a <get_phys_to_machine+42>: ret It bisects down to: commit 962342dabe7a71946d9a5187083577a37dcbe205 Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Date: Thu Feb 26 17:40:47 2009 -0800 xen: dynamically allocate p2m tables Saves about 128k static object size. Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Crash:1 (XEN) d20:v0: unhandled page fault (ec=0000) (XEN) Pagetable walk from 0000000000041b8d: (XEN) L4[0x000] = 000000004149c027 00000000000003dd (XEN) L3[0x000] = 0000000000000000 ffffffffffffffff (XEN) domain_crash_sync called from entry.S (XEN) Domain 20 (vcpu#0) crashed on cpu#1: (XEN) ----[ Xen-3.4-unstable x86_64 debug=y Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: e019:[<00000000c0103f06>] (XEN) RFLAGS: 0000000000000202 EM: 1 CONTEXT: pv guest (XEN) rax: 000000000004193d rbx: 0000000000001008 rcx: 0000000000000094 (XEN) rdx: 00000000c0495000 rsi: 0000000000100800 rdi: 00000000c0494000 (XEN) rbp: 00000000c03efeb4 rsp: 00000000c03efea4 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000006f0 (XEN) cr3: 00000000dfbd0000 cr2: 0000000000041b8d (XEN) ds: e021 es: e021 fs: e021 gs: e021 ss: e021 cs: e019 (XEN) Guest stack trace from esp=c03efea4: (XEN) 00000000 c0103f06 0001e019 00010002 c03efec8 c01058a3 00000000 c0100000 (XEN) c03ecd00 c03efed8 c010308f 00000163 80000000 c03efee0 c03fa748 c03eff9c (XEN) c03fbb6e c03eff88 c03eff80 c032679e 00003fff c03eff18 c01352af 00000000 (XEN) 000000f0 ffffff10 c03effb0 c03eff88 c0105dbe c0106f10 c0135732 00000000 (XEN) 00000000 00000003 000371b5 00000000 00000004 00000000 00000000 0000000f (XEN) 0000009b 205bff90 33202020 06000000 00000000 5d370020 c03e0020 c0105dbe (XEN) c03eff84 00006000 00000000 00007000 00000000 c03c71d8 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 c03effe0 c03c1e80 c03effb8 c03f37d1 (XEN) c0329020 0048e000 00000000 c041c520 c036a0ca c03effd0 c03f30a9 00494000 (XEN) 00000000 c036a0ca c03effec c03efffc c03fa24c 00000000 00000000 1f898975 (XEN) 80000401 00020800 00000f44 00000000 c048b000 00000000 00000000 quick gdb decoded stack trace: <<< (XEN) 00000000 c0103f06 0001e019 00010002 c03efec8 c01058a3 00000000 c0100000 c0103f06: get_phys_to_machine + 38 in section .text c03efec8: init_thread_union + 7880 in section .data.init_task c01058a3: xen_setup_mfn_list_list + 115 in section .text c0100000: startup_32 in section .text.head <<< (XEN) c03ecd00 c03efed8 c010308f 00000163 80000000 c03efee0 c03fa748 c03eff9c c03ecd00: boot_cpu_id in section .data.read_mostly c03efed8: init_thread_union + 7896 in section .data.init_task c010308f: xen_setup_shared_info + 95 in section .text c03efee0: init_thread_union + 7904 in section .data.init_task c03fa748: xen_pagetable_setup_done + 8 in section .init.text c03eff9c: init_thread_union + 8092 in section .data.init_task <<< (XEN) c03fbb6e c03eff88 c03eff80 c032679e 00003fff c03eff18 c01352af 00000000 c03fbb6e: setup_arch + 1150 in section .init.text c03eff88: init_thread_union + 8072 in section .data.init_task c03eff80: init_thread_union + 8064 in section .data.init_task c032679e: _spin_unlock_irqrestore + 30 in section .text c03eff18: init_thread_union + 7960 in section .data.init_task c01352af: release_console_sem + 447 in section .text <<< (XEN) 000000f0 ffffff10 c03effb0 c03eff88 c0105dbe c0106f10 c0135732 00000000 c03effb0: init_thread_union + 8112 in section .data.init_task c03eff88: init_thread_union + 8072 in section .data.init_task c0105dbe: __raw_callee_save_xen_restore_fl + 6 in section .text c0106f10: xen_spin_unlock in section .text c0135732: vprintk + 530 in section .text <<< (XEN) 00000000 00000003 000371b5 00000000 00000004 00000000 00000000 0000000f <<< (XEN) 0000009b 205bff90 33202020 06000000 00000000 5d370020 c03e0020 c0105dbe c03e0020: doublefault_tss + 6176 in section .data.cacheline_aligned c0105dbe: __raw_callee_save_xen_restore_fl + 6 in section .text <<< (XEN) c03eff84 00006000 00000000 00007000 00000000 c03c71d8 00000000 00000000 c03eff84: init_thread_union + 8068 in section .data.init_task c03c71d8: tick_notifier in section .data <<< (XEN) 00000000 00000000 00000000 00000000 c03effe0 c03c1e80 c03effb8 c03f37d1 c03effe0: init_thread_union + 8160 in section .data.init_task c03c1e80: reboot_cpu in section .data c03effb8: init_thread_union + 8120 in section .data.init_task c03f37d1: start_kernel + 113 in section .init.text <<< (XEN) c0329020 0048e000 00000000 c041c520 c036a0ca c03effd0 c03f30a9 00494000 c0329020: linux_banner in section .rodata c041c520: command_line in section .init.data c036a0ca: kallsyms_token_index + 526 in section .rodata c03effd0: init_thread_union + 8144 in section .data.init_task c03f30a9: i386_start_kernel + 137 in section .init.text <<< (XEN) 00000000 c036a0ca c03effec c03efffc c03fa24c 00000000 00000000 1f898975 c036a0ca: kallsyms_token_index + 526 in section .rodata c03effec: init_thread_union + 8172 in section .data.init_task c03efffc: init_thread_union + 8188 in section .data.init_task c03fa24c: xen_start_kernel + 828 in section .init.text> > -- Pasi > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2009-Mar-03 17:41 UTC
Re: [Xen-devel] latest pv_ops dom0 (2.6.29-rc6) crashes / unhandled page fault
On Mon, Mar 02, 2009 at 05:38:54PM -0800, Jeremy Fitzhardinge wrote:> Pasi Kärkkäinen wrote: > >Hello. > > > >log of the crashing boot process: > >http://pasik.reaktio.net/xen/pv_ops-dom0-debug/pv_ops-dom0-bootlog-14-xen331-linux-2.6.29-rc6-crash.txt > > > >2.6.29-rc5 based pv_ops dom0 works on the same computer.. with the same > >Xen version. > > > >I tried with and without pci=nomsi. > > > >adding ''noapic'' doesn''t seem to help (I need that with 2.6.29-rc5 to make > >it work). > > > > Yeah, something got broken. Tracking it down now. > > >Any ideas what''s going on? > > > >(XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input > >to Xen) > >(XEN) Freed 112kB init memory. > >mapping kernel into physical memory > >(XEN) d0:v0: unhandled page fault (ec=0000) > >(XEN) Pagetable walk from bffff000: > >(XEN) L3[0x002] = 000000003d204001 00001204 > >(XEN) L2[0x1ff] = 0000000000000000 ffffffff > >(XEN) domain_crash_sync called from entry.S (ff1a2c9e) > >(XEN) Domain 0 (vcpu#0) crashed on cpu#0: > >(XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]---- > >(XEN) CPU: 0 > >(XEN) EIP: e019:[<c0879716>] > > > > What does this map to in your vmlinux? >Unfortunately I had already ran "make clean" on that tree, and upgraded my distro packages up-to-date, so can''t check.. But I rebuilt the latest tree now, and verified it crashes aswell.. different memory address this time though (probably because of different git tree + different gcc version). Now it crashes at: (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen) (XEN) Freed 112kB init memory. mapping kernel into physical memory (XEN) d0:v0: unhandled page fault (ec=0000) (XEN) Pagetable walk from bffff000: (XEN) L3[0x002] = 000000003d203001 00001203 (XEN) L2[0x1ff] = 0000000000000000 ffffffff (XEN) domain_crash_sync called from entry.S (ff1a2c9e) (XEN) Domain 0 (vcpu#0) crashed on cpu#0: (XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) EIP: e019:[<c0876b43>] (XEN) EFLAGS: 00000297 EM: 1 CONTEXT: pv guest # gdb vmlinux GNU gdb Fedora (6.8-29.fc10) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... (gdb) x/i 0xc0876b43 0xc0876b43 <xen_map_identity_early+157>: mov (%edi,%ebx,8),%edx (gdb) Hopefully that helps.. -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2009-Mar-04 20:41 UTC
Re: [Xen-devel] latest pv_ops dom0 (2.6.29-rc6) crashes / unhandled page fault
On Tue, Mar 03, 2009 at 07:41:49PM +0200, Pasi Kärkkäinen wrote:> > > > > > > Yeah, something got broken. Tracking it down now. > > > > >Any ideas what''s going on? > > > > > >(XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input > > >to Xen) > > >(XEN) Freed 112kB init memory. > > >mapping kernel into physical memory > > >(XEN) d0:v0: unhandled page fault (ec=0000) > > >(XEN) Pagetable walk from bffff000: > > >(XEN) L3[0x002] = 000000003d204001 00001204 > > >(XEN) L2[0x1ff] = 0000000000000000 ffffffff > > >(XEN) domain_crash_sync called from entry.S (ff1a2c9e) > > >(XEN) Domain 0 (vcpu#0) crashed on cpu#0: > > >(XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]---- > > >(XEN) CPU: 0 > > >(XEN) EIP: e019:[<c0879716>] > > > > > > > What does this map to in your vmlinux? > > >Just upgraded to latest tree (as of some hours ago), and I still get the same crash: http://pasik.reaktio.net/xen/pv_ops-dom0-debug/pv_ops-dom0-bootlog-16-xen331-linux-2.6.29-rc6-crash.txt (XEN) Freed 112kB init memory. mapping kernel into physical memory (XEN) d0:v0: unhandled page fault (ec=0000) (XEN) Pagetable walk from bffff000: (XEN) L3[0x002] = 000000003d204001 00001204 (XEN) L2[0x1ff] = 0000000000000000 ffffffff (XEN) domain_crash_sync called from entry.S (ff1a2c9e) (XEN) Domain 0 (vcpu#0) crashed on cpu#0: (XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) EIP: e019:[<c0877902>] [root@dom0test linux-2.6-xen]# gdb vmlinux GNU gdb Fedora (6.8-29.fc10) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... (gdb) x/i 0xc0877902 0xc0877902 <xen_map_identity_early+157>: mov (%edi,%ebx,8),%edx (gdb) -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2009-Mar-04 20:44 UTC
Re: [Xen-devel] latest pv_ops dom0 (2.6.29-rc6) crashes / unhandled page fault
Pasi Kärkkäinen wrote:> On Tue, Mar 03, 2009 at 07:41:49PM +0200, Pasi Kärkkäinen wrote: > >>>> >>>> >>> Yeah, something got broken. Tracking it down now. >>> >>> >>>> Any ideas what''s going on? >>>> >>>> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input >>>> to Xen) >>>> (XEN) Freed 112kB init memory. >>>> mapping kernel into physical memory >>>> (XEN) d0:v0: unhandled page fault (ec=0000) >>>> (XEN) Pagetable walk from bffff000: >>>> (XEN) L3[0x002] = 000000003d204001 00001204 >>>> (XEN) L2[0x1ff] = 0000000000000000 ffffffff >>>> (XEN) domain_crash_sync called from entry.S (ff1a2c9e) >>>> (XEN) Domain 0 (vcpu#0) crashed on cpu#0: >>>> (XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]---- >>>> (XEN) CPU: 0 >>>> (XEN) EIP: e019:[<c0879716>] >>>> >>>> >>> What does this map to in your vmlinux? >>> >>> > > Just upgraded to latest tree (as of some hours ago), and I still get the > same crash: > > http://pasik.reaktio.net/xen/pv_ops-dom0-debug/pv_ops-dom0-bootlog-16-xen331-linux-2.6.29-rc6-crash.txt > > (XEN) Freed 112kB init memory. > mapping kernel into physical memory > (XEN) d0:v0: unhandled page fault (ec=0000) > (XEN) Pagetable walk from bffff000: > (XEN) L3[0x002] = 000000003d204001 00001204 > (XEN) L2[0x1ff] = 0000000000000000 ffffffff > (XEN) domain_crash_sync called from entry.S (ff1a2c9e) > (XEN) Domain 0 (vcpu#0) crashed on cpu#0: > (XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]---- > (XEN) CPU: 0 > (XEN) EIP: e019:[<c0877902>] > > > [root@dom0test linux-2.6-xen]# gdb vmlinux > GNU gdb Fedora (6.8-29.fc10) > Copyright (C) 2008 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > <http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "i386-redhat-linux-gnu"... > (gdb) x/i 0xc0877902 > 0xc0877902 <xen_map_identity_early+157>: mov (%edi,%ebx,8),%edx >Yep, working on it at the moment. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2009-Mar-05 17:50 UTC
Re: [Xen-devel] latest pv_ops dom0 (2.6.29-rc7) crashes
On Wed, Mar 04, 2009 at 12:44:09PM -0800, Jeremy Fitzhardinge wrote:> Pasi Kärkkäinen wrote: > >On Tue, Mar 03, 2009 at 07:41:49PM +0200, Pasi Kärkkäinen wrote: > > > >>>> > >>>> > >>>Yeah, something got broken. Tracking it down now. > >>> > >>> > >>>>Any ideas what''s going on? > >>>> > >>>>(XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch > >>>>input to Xen) > >>>>(XEN) Freed 112kB init memory. > >>>>mapping kernel into physical memory > >>>>(XEN) d0:v0: unhandled page fault (ec=0000) > >>>>(XEN) Pagetable walk from bffff000: > >>>>(XEN) L3[0x002] = 000000003d204001 00001204 > >>>>(XEN) L2[0x1ff] = 0000000000000000 ffffffff > >>>>(XEN) domain_crash_sync called from entry.S (ff1a2c9e) > >>>>(XEN) Domain 0 (vcpu#0) crashed on cpu#0: > >>>>(XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]---- > >>>>(XEN) CPU: 0 > >>>>(XEN) EIP: e019:[<c0879716>] > >>>> > >>>> > >>>What does this map to in your vmlinux? > >>> > >>> > > > >Just upgraded to latest tree (as of some hours ago), and I still get the > >same crash: > > > >http://pasik.reaktio.net/xen/pv_ops-dom0-debug/pv_ops-dom0-bootlog-16-xen331-linux-2.6.29-rc6-crash.txt > > > >(XEN) Freed 112kB init memory. > >mapping kernel into physical memory > >(XEN) d0:v0: unhandled page fault (ec=0000) > >(XEN) Pagetable walk from bffff000: > >(XEN) L3[0x002] = 000000003d204001 00001204 > >(XEN) L2[0x1ff] = 0000000000000000 ffffffff > >(XEN) domain_crash_sync called from entry.S (ff1a2c9e) > >(XEN) Domain 0 (vcpu#0) crashed on cpu#0: > >(XEN) ----[ Xen-3.3.1 x86_32p debug=n Not tainted ]---- > >(XEN) CPU: 0 > >(XEN) EIP: e019:[<c0877902>] > > > > > >[root@dom0test linux-2.6-xen]# gdb vmlinux > >GNU gdb Fedora (6.8-29.fc10) > >Copyright (C) 2008 Free Software Foundation, Inc. > >License GPLv3+: GNU GPL version 3 or later > ><http://gnu.org/licenses/gpl.html> > >This is free software: you are free to change and redistribute it. > >There is NO WARRANTY, to the extent permitted by law. Type "show copying" > >and "show warranty" for details. > >This GDB was configured as "i386-redhat-linux-gnu"... > >(gdb) x/i 0xc0877902 > >0xc0877902 <xen_map_identity_early+157>: mov (%edi,%ebx,8),%edx > > > > Yep, working on it at the moment. >Wow, latest tree (2.6.29-rc7) _almost_ boots now! First time ever (on my testing box) disk attached to ata_piix is now actually detected and seems to work.. at least partly! See: http://pasik.reaktio.net/xen/pv_ops-dom0-debug/pv_ops-dom0-bootlog-17-xen331-linux-2.6.29-rc7-crash.txt But now I get crash when init is started.. Starting udev: BUG: unable to handle kernel paging request at c1106840 IP: [<c04045ef>] get_phys_to_machine+0x24/0x29 *pdpt = 000000005fa7a001 Thread overran stack, or stack corrupted [root@dom0test linux-2.6-xen]# gdb vmlinux GNU gdb Fedora (6.8-29.fc10) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... (gdb) x/i 0xc04045ef 0xc04045ef <get_phys_to_machine+36>: mov (%eax,%ecx,4),%eax (gdb) -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2009-Mar-05 20:10 UTC
Re: [Xen-devel] latest pv_ops dom0 (2.6.29-rc7) crashes
Pasi Kärkkäinen wrote:> First time ever (on my testing box) disk attached to ata_piix is now actually > detected and seems to work.. at least partly! >Good!> See: > http://pasik.reaktio.net/xen/pv_ops-dom0-debug/pv_ops-dom0-bootlog-17-xen331-linux-2.6.29-rc7-crash.txt > > But now I get crash when init is started.. > > Starting udev: BUG: unable to handle kernel paging request at c1106840 > IP: [<c04045ef>] get_phys_to_machine+0x24/0x29 > *pdpt = 000000005fa7a001 > Thread overran stack, or stack corrupted >Try again. I don''t know if I specifically fixed this one, but I noticed that it wasn''t reserving the 32-bit Xen pagetables, so it might have tried to reallocate them for the swiotlb buffer... J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2009-Mar-05 21:16 UTC
Re: [Xen-devel] latest pv_ops dom0 (2.6.29-rc7) works!!
On Thu, Mar 05, 2009 at 12:10:01PM -0800, Jeremy Fitzhardinge wrote:> Pasi Kärkkäinen wrote: > >First time ever (on my testing box) disk attached to ata_piix is now > >actually detected and seems to work.. at least partly! > > > > Good! > > >See: > >http://pasik.reaktio.net/xen/pv_ops-dom0-debug/pv_ops-dom0-bootlog-17-xen331-linux-2.6.29-rc7-crash.txt > > > >But now I get crash when init is started.. > > > >Starting udev: BUG: unable to handle kernel paging request at c1106840 > >IP: [<c04045ef>] get_phys_to_machine+0x24/0x29 > >*pdpt = 000000005fa7a001 > >Thread overran stack, or stack corrupted > > > > Try again. I don''t know if I specifically fixed this one, but I noticed > that it wasn''t reserving the 32-bit Xen pagetables, so it might have > tried to reallocate them for the swiotlb buffer... >And now it boots all the way to login prompt!! Congratulations! http://pasik.reaktio.net/xen/pv_ops-dom0-debug/pv_ops-dom0-bootlog-18-xen331-linux-2.6.29-rc7.txt grub.conf: title Xen pv_ops dom0-test (2.6.29-rc7-tip) / serial console root (hd0,0) kernel /xen-3.3.gz dom0_mem=1024M loglvl=all guest_loglvl=all com1=19200,8n1 console=com1 module /vmlinuz-2.6.29-rc7-tip ro root=/dev/vg00/lv01 console=hvc0 earlyprintk=xen module /initrd-2.6.29-rc7-tip.img Network seems to be OK, I can ssh into the box: [root@dom0test ~]# uname -a Linux dom0test.localdomain 2.6.29-rc7-tip #7 SMP Thu Mar 5 22:33:17 EET 2009 i686 i686 i386 GNU/Linux [root@dom0test ~]# ethtool -i eth0 driver: tg3 version: 3.97 firmware-version: 5721-v3.29a, ASFIPMI v6.09 bus-info: 0000:04:00.0 [root@dom0test ~]# cat /proc/interrupts CPU0 CPU1 1: 36 0 xen-pirq-pirq i8042 3: 2 0 xen-pirq-pirq 6: 5 0 xen-pirq-pirq floppy 7: 0 0 xen-pirq-pirq parport0 8: 33 0 xen-pirq-pirq rtc0 9: 0 0 xen-pirq-pirq acpi 14: 3868 0 xen-pirq-pirq ata_piix 15: 0 0 xen-pirq-pirq ata_piix 16: 0 0 xen-pirq-pirq uhci_hcd:usb5, pata_pdc2027x 18: 1454 0 xen-pirq-pirq uhci_hcd:usb4, eth0 19: 1 0 xen-pirq-pirq ahci, uhci_hcd:usb3, eth1 23: 0 0 xen-pirq-pirq ehci_hcd:usb1, uhci_hcd:usb2 24: 0 0 xen-pirq-pirq sata_promise 48: 38110 0 xen-dyn-virq timer0 49: 2 0 xen-dyn-ipi spinlock0 50: 3280 0 xen-dyn-ipi resched0 51: 0 0 xen-dyn-ipi callfunc0 52: 0 0 xen-dyn-virq debug0 53: 118 0 xen-dyn-ipi callfuncsingle0 54: 0 34153 xen-dyn-virq timer1 55: 0 7 xen-dyn-ipi spinlock1 56: 0 4402 xen-dyn-ipi resched1 57: 0 0 xen-dyn-ipi callfunc1 58: 0 0 xen-dyn-virq debug1 59: 0 368 xen-dyn-ipi callfuncsingle1 60: 0 0 xen-dyn-event xenbus 61: 16 0 xen-dyn-virq hvc_console NMI: 0 0 Non-maskable interrupts LOC: 0 0 Local timer interrupts CNT: 0 0 Performance counter interrupts RES: 3280 4402 Rescheduling interrupts CAL: 118 368 Function call interrupts TLB: 0 0 TLB shootdowns TRM: 0 0 Thermal event interrupts SPU: 0 0 Spurious interrupts ERR: 0 MIS: 0 [root@dom0test ~]# lsmod Module Size Used by bridge 40160 0 stp 2032 1 bridge bnep 10852 2 sco 9036 2 l2cap 17616 3 bnep bluetooth 44556 5 bnep,sco,l2cap sunrpc 149716 1 ipv6 226944 26 dm_multipath 13520 0 uinput 6612 0 ppdev 6376 0 floppy 47220 0 i2c_i801 8228 0 i2c_core 17668 1 i2c_i801 iTCO_wdt 10456 0 pcspkr 2248 0 iTCO_vendor_support 2828 1 iTCO_wdt pata_pdc2027x 6968 0 tg3 91468 0 libphy 16496 1 tg3 sata_promise 10024 0 parport_pc 22096 0 parport 27956 2 ppdev,parport_pc ata_generic 4224 0 pata_acpi 3508 0 [root@dom0test ~]# mount | grep xen none on /proc/xen type xenfs (rw) [root@dom0test ~]# /etc/init.d/xend start Starting xend daemon: [FAILED] I''ll start playing with that stuff tomorrow, now I need to get some sleep :) -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Possibly Parallel Threads
- (XEN) mm.c:707:d49 Error getting mfn 2fef3 (pfn 6aef3) from L1 entry
- Xen 3.2.1-rc1: ptwr_emulate: could not get_page_from_l1e()
- debian lenny domU installation
- 2.6.26-rc8 pv_ops causes Unhandled invalid opcode fault/trap
- Loading FreeBSD in an HVM-domain on Intel-based host with GRUB