[ I''m not subscribed to the list, please keep me in Cc: ]
Hi,
I''m trying to upgrade the NetBSD xen packages to xen-3.1.2 and
I''m having
troubles loading a x86_64 NetBSD kernel. The hypervisor crashes very
early in the bootstrap setup, when trying to unping its bootstrap page table
after switching to the new page table. Basically NetBSD does the following:
build temporary new pages tables mapping the kernel in the free memory area
after the Xen page tables, pin and switch to the new temporary L4 page,
unpin the old boot L4 page provided by Xen. After that it should map the
boostrap pages provided by Xen R/W and build the final pages tables
for the kernel in place of the Xen-provided bootstrap pages and switch to
it, but it doesn''t get there: the hypervisor panics with:
(XEN) Assertion ''(x & ((1U<<16)-1)) != 0'' failed at
mm.c:1670
I added a MEM_LOG() in put_page_from_l4e and here''s what I get (there
is also some debug infos from the NetBSD kernel, especially about the
temporary pages tables being built):
(XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times
to switch input to Xen).
(XEN) Freed 116kB init memory.
xen_arch_pmap_bootstrap init_tables=0xffffffff80caa000
xen_bootstrap_tables(0xffffffff80caa000, 0xffffffff80d16000, 11, 9)
xen_bootstrap_tables text_end 0xffffffff80a10000 map_end 0xffffffff80d28000
L3 va 0xffffffff80d17000 pa 0xd17000 entry 0x3ad17007 -> L4[0x1ff]
L2 va 0xffffffff80d18000 pa 0xd18000 entry 0x3ad18007 -> L3[0x1fe]
L1 va 0xffffffff80d19000 pa 0xd19000 entry 0x3ad19007 -> L2[0x0]
L1 va 0xffffffff80d1a000 pa 0xd1a000 entry 0x3ad1a007 -> L2[0x1]
L1 va 0xffffffff80d1b000 pa 0xd1b000 entry 0x3ad1b007 -> L2[0x2]
L1 va 0xffffffff80d1c000 pa 0xd1c000 entry 0x3ad1c007 -> L2[0x3]
L1 va 0xffffffff80d1d000 pa 0xd1d000 entry 0x3ad1d007 -> L2[0x4]
L1 va 0xffffffff80d1e000 pa 0xd1e000 entry 0x3ad1e007 -> L2[0x5]
va 0xffffffff80caa000 pa 0xcaa000 entry 0x3acaa005 -> L1[0xaa]
L1 va 0xffffffff80d1f000 pa 0xd1f000 entry 0x3ad1f007 -> L2[0x6]
L1 va 0xffffffff80d20000 pa 0xd20000 entry 0x3ad20007 -> L2[0x7]
L1 va 0xffffffff80d21000 pa 0xd21000 entry 0x3ad21007 -> L2[0x8]
bt_pgd[PDIR_SLOT_PTE] va 0xffffffff80d16000 pa 0xd16000 entry 0x3ad16005
pin PDG
switch to PDG
bt_pgd[PDIR_SLOT_PTE] now entry 0x3ad16005
L4_BASE va 0x7fbfdfeff000
value 0x0
[PDIR_SLOT_PTE] 0x3ad16025
unpin old PDG
(XEN) mm.c:881:d0 put_page_from_l4e entry 0x3e3bf063 pfn 0x3acaa type 0x0
(XEN)
(XEN) Assertion ''(x & ((1U<<16)-1)) != 0'' failed at
mm.c:1670
(XEN) ----[ Xen-3.1.2 x86_64 debug=y Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e008:[<ffff828c80138e34>] put_page_type+0x29/0xfc
(XEN) RFLAGS: 0000000000010246 CONTEXT: hypervisor
(XEN) rax: 0000000000000000 rbx: ffff8284009b95d8 rcx: 0000000000001ee7
(XEN) rdx: 000000000000000a rsi: 0000000000000000 rdi: ffff8284009b95d8
(XEN) rbp: ffff828c80217db8 rsp: ffff828c80217d98 r8: 0000000000000004
(XEN) r9: 0000000000000004 r10: 0000000000000004 r11: 0000000000000004
(XEN) r12: ffffffffffffffff r13: ffff8284009b95d8 r14: 00000000009b95d8
(XEN) r15: 000000000003acaa cr0: 000000008005003b cr4: 00000000000006b0
(XEN) cr3: 000000003ad16000 cr2: 0000000000000000
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff828c80217d98:
(XEN) ffff83003d2fa100 ffff8284009b95d8 000000003e3bf063 000000000003e3bf
(XEN) ffff828c80217df8 ffff828c8013897e 0000000000000000 ffff83003acaa000
(XEN) 0000000000000000 ffff83003d2fa100 000000000003acaa ffff82840092fa90
(XEN) ffff828c80217e38 ffff828c80138da4 000000000000000e 0000000080000001
(XEN) 0000000088000000 ffff82840092fa90 ffffffff80a10368 ffff83003d2fa100
(XEN) ffff828c80217e68 ffff828c80138e9b 000000000003acaa 0000000000000004
(XEN) 000000000003acaa ffff82840092fa90 ffff828c80217f08 ffff828c8013e01d
(XEN) 0000000100007ff0 0000000000000000 ffff83003d2f6100 0000000000000000
(XEN) 0000000000000002 ffffffff80d16000 ffff83003d2fa100 0000000000000002
(XEN) 0000003000000004 000000000003acaa ffffffff80a10388 000000000003acb3
(XEN) 0000000000000000 ffff83003d2f6100 ffffffff81200000 ffffffff80d1f938
(XEN) 0000000000000127 0000000080000000 00007d737fde80b7 ffff828c80199157
(XEN) ffffffff8010134a 000000000000001a 0000000080000000 0000000000000127
(XEN) ffffffff80d1f938 ffffffff81200000 ffffffff80a10398 000000000003acaa
(XEN) 0000000000000246 0000000000007ff0 ffffffff80803947 ffffffff80a102c8
(XEN) 000000000000001a ffffffff8010134a 0000000000000000 0000000000000001
(XEN) ffffffff80a10368 0000010000000000 ffffffff8010134a 000000000000e033
(XEN) 0000000000000246 ffffffff80a10350 000000000000e02b 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) ffff83003d2f6100
(XEN) Xen call trace:
(XEN) [<ffff828c80138e34>] put_page_type+0x29/0xfc
(XEN) [<ffff828c8013897e>] put_page_from_l4e+0x94/0xd9
(XEN) [<ffff828c80138da4>] free_page_type+0x3e1/0x448
(XEN) [<ffff828c80138e9b>] put_page_type+0x90/0xfc
(XEN) [<ffff828c8013e01d>] do_mmuext_op+0x764/0xe72
(XEN) [<ffff828c80199157>] syscall_enter+0xa7/0x101
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion ''(x & ((1U<<16)-1)) != 0'' failed at
mm.c:1670
(XEN) ****************************************
Any idea or comment welcome ...
--
Manuel Bouyer, LIP6, Universite Paris VI. Manuel.Bouyer@lip6.fr
NetBSD: 26 ans d''experience feront toujours la difference
--
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Manuel Bouyer
2007-Dec-03 20:44 UTC
[Xen-devel] Re: xen 3.1.2: can''t unpin bootstrap page
On Mon, Dec 03, 2007 at 07:12:43PM +0100, Manuel Bouyer wrote:> [ I''m not subscribed to the list, please keep me in Cc: ] > Hi, > I''m trying to upgrade the NetBSD xen packages to xen-3.1.2 and I''m having > troubles loading a x86_64 NetBSD kernel. The hypervisor crashes very > early in the bootstrap setup, when trying to unping its bootstrap page table > after switching to the new page table. Basically NetBSD does the following: > build temporary new pages tables mapping the kernel in the free memory area > after the Xen page tables, pin and switch to the new temporary L4 page, > unpin the old boot L4 page provided by Xen. After that it should map the > boostrap pages provided by Xen R/W and build the final pages tables > for the kernel in place of the Xen-provided bootstrap pages and switch to > it, but it doesn''t get there: the hypervisor panics with: > (XEN) Assertion ''(x & ((1U<<16)-1)) != 0'' failed at mm.c:1670 > > I added a MEM_LOG() in put_page_from_l4e and here''s what I get (there > is also some debug infos from the NetBSD kernel, especially about the > temporary pages tables being built): > (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen). > (XEN) Freed 116kB init memory. > xen_arch_pmap_bootstrap init_tables=0xffffffff80caa000 > xen_bootstrap_tables(0xffffffff80caa000, 0xffffffff80d16000, 11, 9) > xen_bootstrap_tables text_end 0xffffffff80a10000 map_end 0xffffffff80d28000 > L3 va 0xffffffff80d17000 pa 0xd17000 entry 0x3ad17007 -> L4[0x1ff] > L2 va 0xffffffff80d18000 pa 0xd18000 entry 0x3ad18007 -> L3[0x1fe] > L1 va 0xffffffff80d19000 pa 0xd19000 entry 0x3ad19007 -> L2[0x0] > L1 va 0xffffffff80d1a000 pa 0xd1a000 entry 0x3ad1a007 -> L2[0x1] > L1 va 0xffffffff80d1b000 pa 0xd1b000 entry 0x3ad1b007 -> L2[0x2] > L1 va 0xffffffff80d1c000 pa 0xd1c000 entry 0x3ad1c007 -> L2[0x3] > L1 va 0xffffffff80d1d000 pa 0xd1d000 entry 0x3ad1d007 -> L2[0x4] > L1 va 0xffffffff80d1e000 pa 0xd1e000 entry 0x3ad1e007 -> L2[0x5] > va 0xffffffff80caa000 pa 0xcaa000 entry 0x3acaa005 -> L1[0xaa] > L1 va 0xffffffff80d1f000 pa 0xd1f000 entry 0x3ad1f007 -> L2[0x6] > L1 va 0xffffffff80d20000 pa 0xd20000 entry 0x3ad20007 -> L2[0x7] > L1 va 0xffffffff80d21000 pa 0xd21000 entry 0x3ad21007 -> L2[0x8] > bt_pgd[PDIR_SLOT_PTE] va 0xffffffff80d16000 pa 0xd16000 entry 0x3ad16005 > pin PDG > switch to PDG > bt_pgd[PDIR_SLOT_PTE] now entry 0x3ad16005 > L4_BASE va 0x7fbfdfeff000 > value 0x0 > [PDIR_SLOT_PTE] 0x3ad16025 > unpin old PDG > (XEN) mm.c:881:d0 put_page_from_l4e entry 0x3e3bf063 pfn 0x3acaa type 0x0I tracked it down to an entry in the L4 page provided by Xen: slot 0 has 0x3e3bf063 in it, which isn''t even in the domains''s memory range. Trying to clear this entry from the dom0 kernel leads to the same hypervisor panic. Here''s what Xen says about this dom0: (XEN) *** LOADING DOMAIN 0 *** (XEN) elf_parse_binary: phdr: paddr=0xffffffff80100000 memsz=0x810328 (XEN) elf_parse_binary: phdr: paddr=0xffffffff80a10340 memsz=0xedf98 (XEN) elf_parse_binary: memory: 0xffffffff80100000 -> 0xffffffff80afe2d8 (XEN) elf_xen_parse: __xen_guest: "GUEST_OS=NetBSD,GUEST_VER=4.99,XEN_VER=xen-3. 0,LOADER=generic,VIRT_BASE=0xffffffff80000000,ELF_PADDR_OFFSET=0xffffffff8000000 0,VIRT_ENTRY=0xffffffff80100000,HYPERCALL_PAGE=0x00000101,BSD_SYMTAB=yes" (XEN) elf_xen_parse_guest_info: GUEST_OS="NetBSD" (XEN) elf_xen_parse_guest_info: GUEST_VER="4.99" (XEN) elf_xen_parse_guest_info: XEN_VER="xen-3.0" (XEN) elf_xen_parse_guest_info: LOADER="generic" (XEN) elf_xen_parse_guest_info: VIRT_BASE="0xffffffff80000000" (XEN) elf_xen_parse_guest_info: ELF_PADDR_OFFSET="0xffffffff80000000" (XEN) elf_xen_parse_guest_info: VIRT_ENTRY="0xffffffff80100000" (XEN) elf_xen_parse_guest_info: HYPERCALL_PAGE="0x00000101" (XEN) elf_xen_parse_guest_info: BSD_SYMTAB="yes" (XEN) elf_xen_addr_calc_check: addresses: (XEN) virt_base = 0xffffffff80000000 (XEN) elf_paddr_offset = 0xffffffff80000000 (XEN) virt_offset = 0x0 (XEN) virt_kstart = 0xffffffff80100000 (XEN) virt_kend = 0xffffffff80be8c70 (XEN) virt_entry = 0xffffffff80100000 (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80100000 -> 0xffffffff80afe2d8 (XEN) Dom0 symbol map 0xffffffff80afe2d8 -> 0xffffffff80be8c70 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 000000003a000000->000000003b000000 (94208 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80100000->ffffffff80be8c70 (XEN) Init. ramdisk: ffffffff80be9000->ffffffff80be9000 (XEN) Phys-Mach map: ffffffff80be9000->ffffffff80ca9000 (XEN) Start info: ffffffff80ca9000->ffffffff80ca949c (XEN) Page tables: ffffffff80caa000->ffffffff80cb5000 (XEN) Boot stack: ffffffff80cb5000->ffffffff80cb6000 (XEN) TOTAL: ffffffff80000000->ffffffff81000000 (XEN) ENTRY ADDRESS: ffffffff80100000 (XEN) Dom0 has maximum 2 VCPUs (XEN) elf_load_binary: phdr 0 at 0xffffffff80100000 -> 0xffffffff80910328 (XEN) elf_load_binary: phdr 1 at 0xffffffff80a10340 -> 0xffffffff80a7dae0 (XEN) elf_load_bsdsyms: shdr 22 at 0xffff83003d1153a4 -> 0xffffffff80afe960 (XEN) elf_load_bsdsyms: shdr 23 at 0xffff83003d115b38 -> 0xffffffff80afeab8 (XEN) elf_load_bsdsyms: shdr 24 at 0xffff83003d1a2e68 -> 0xffffffff80b8bde8 (XEN) Scrubbing Free RAM: .....done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen). (XEN) Freed 116kB init memory. -- Manuel Bouyer, LIP6, Universite Paris VI. Manuel.Bouyer@lip6.fr NetBSD: 26 ans d''experience feront toujours la difference -- _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Manuel Bouyer
2007-Dec-03 21:30 UTC
[Xen-devel] Re: xen 3.1.2: can''t unpin bootstrap page
On Mon, Dec 03, 2007 at 09:44:36PM +0100, Manuel Bouyer wrote:> I tracked it down to an entry in the L4 page provided by Xen: > slot 0 has 0x3e3bf063 in it, which isn''t even in the domains''s memory range. > Trying to clear this entry from the dom0 kernel leads to the same hypervisor > panic.It seems this entry comes from the idle_pg_table page. Clearing the user slots in construct_dom0() fixes it for me, I can again load a NetBSD kernel as dom0. See the attached patch -- Manuel Bouyer, LIP6, Universite Paris VI. Manuel.Bouyer@lip6.fr NetBSD: 26 ans d''experience feront toujours la difference -- _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel