thr3ads.net - Xen devel - Kernel panic in pin_pagetable

If this information is useful, please help other people find it:
Share via:

Joe Cao

2013-Jun-17 22:57 UTC

Kernel panic in pin_pagetable_pfn

Hi Xen developers,

We are hitting a DomU Linux kernel panic and we''d like to find a
solution for it.

We are porting our virtual appliance to Amazon''s AWS.  Our kernel is
2.6.27.57.  We know it is old but we have to stay with it for now.  The problem
is when the kernel (64-bit) boots up in EC2 ( a large instance with 7.5 GB
memory), the kernel crashes when hitting the BUG in
enlighten.c:pin_pagetable_pfn():

static void pin_pagetable_pfn(unsigned cmd, unsigned long pfn)
{
        struct mmuext_op op;
        op.cmd = cmd;
        op.arg1.mfn = pfn_to_mfn(pfn);
        if (HYPERVISOR_mmuext_op(&op, 1, NULL, DOMID_SELF))
                BUG();
}

The complete kernel log is attached below.  We noticed we don''t see
this panic with 2.6.32 kernel.  But we''d like to keep with the 2.6.27
kernel if possible.  Does anyone know if there''s a patch that can fix
this problem?

Here are some additional information about this panic:

1.       In pin_pagetable_pfn, before it hits BUG,   the pfn is 0x180011 and the
mfn is 0x6c7b1.

2.       It doesn''t crash when the total memory is less than 4GB.

Does anything know the cause and if there''s any solution to it?

Thanks,
Joe

--
[    0.000000] Linux version 2.6.27 (root@dev64-build1) (gcc version 4.1.1) #3
SMP Sun Jun 9 05:11:27 IST 2013
[    0.000000] Command line: root=/dev/ram0 rw load_ramdisk=1 
ramdisk_size=350000 console=tty0,9600n8 console=ttyS0,9600n8 bfile=disk_pri
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000008cc2000 (usable)
[    0.000000]  Xen: 0000000008cc2000 - 0000000009bc5000 (reserved)
[    0.000000]  Xen: 0000000009bc5000 - 00000001e0000000 (usable)
[    0.000000] last_pfn = 0x1e0000 max_arch_pfn = 0x3ffffffff
[    0.000000] last_pfn = 0x100000 max_arch_pfn = 0x3ffffffff
[    0.000000] init_memory_mapping
[    0.000000] last_map_addr: 100000000 end: 100000000
[    0.000000] init_memory_mapping
[    0.000000] last_map_addr: 1e0000000 end: 1e0000000
[    0.000000] RAMDISK: 009b9000 - 08cc2000
[    0.000000] DMI not present or invalid.
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-00000001e0000000
[    0.000000] Bootmem setup node 0 0000000000000000-00000001e0000000
[    0.000000]   NODE_DATA [0000000000001000 - 0000000000005fff]
[    0.000000]   bootmap [0000000000008000 -  0000000000043fff] pages 3c
[    0.000000] (7 early reservations) ==> bootmem [0000000000 - 01e0000000]
[    0.000000]   #0 [0000000000 - 0000001000]   BIOS data page ==>
[0000000000 - 0000001000]
[    0.000000]   #1 [0000006000 - 0000008000]       TRAMPOLINE ==>
[0000006000 - 0000008000]
[    0.000000]   #2 [0009bc5000 - 0009c18000]   XEN PAGETABLES ==>
[0009bc5000 - 0009c18000]
[    0.000000]   #3 [0000200000 - 00009b8568]    TEXT DATA BSS ==>
[0000200000 - 00009b8568]
[    0.000000]   #4 [00009b9000 - 0008cc2000]          RAMDISK ==>
[00009b9000 - 0008cc2000]
[    0.000000]   #5 [0009c18000 - 000a3c7000]          PGTABLE ==>
[0009c18000 - 000a3c7000]
[    0.000000]   #6 [000a3c7000 - 000aacb000]          PGTABLE ==>
[000a3c7000 - 000aacb000]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000000 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x001e0000
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[3] active PFN ranges
[    0.000000]     0: 0x00000000 -> 0x000000a0
[    0.000000]     0: 0x00000100 -> 0x00008cc2
[    0.000000]     0: 0x00009bc5 -> 0x001e0000
[    0.000000] SMP: Allowing 2 CPUs, 0 hotplug CPUs
[    0.000000] No local APIC present
[    0.000000] PCI: Warning: Cannot find a gap in the 32bit address range
[    0.000000] PCI: Unassigned devices with 32bit resource registers may break!
[    0.000000] Allocating PCI resources starting at 1e0200000 (gap:
1e0100000:400000)
[    0.000000] PERCPU: Allocating 59040 bytes of per cpu data
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total
pages: 1933282
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: root=/dev/ram0 rw load_ramdisk=1 
ramdisk_size=350000 console=tty0,9600n8 console=ttyS0,9600n8 bfile=disk_pri
[    0.000000] Initializing CPU#0
[    0.000000] PID hash table entries: 4096 (order: 12, 32768 bytes)
[    0.000000] Detected 1999.975 MHz processor.
[    0.004000] Console: colour dummy device 80x25
[    0.004000] console [tty0] enabled
[    0.004000] console [hvc0] enabled
[    0.004000] console [ttyS0] enabled
[    0.000000] Linux version 2.6.27 (root@dev64-build1) (gcc version 4.1.1) #3
SMP Sun Jun 9 05:11:27 IST 2013
[    0.000000] Command line: root=/dev/ram0 rw load_ramdisk=1 
ramdisk_size=350000 console=tty0,9600n8 console=ttyS0,9600n8 bfile=disk_pri
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000008cc2000 (usable)
[    0.000000]  Xen: 0000000008cc2000 - 0000000009bc5000 (reserved)
[    0.000000]  Xen: 0000000009bc5000 - 00000001e0000000 (usable)
[    0.000000] last_pfn = 0x1e0000 max_arch_pfn = 0x3ffffffff
[    0.000000] last_pfn = 0x100000 max_arch_pfn = 0x3ffffffff
[    0.000000] init_memory_mapping
[    0.000000] last_map_addr: 100000000 end: 100000000
[    0.000000] init_memory_mapping
[    0.000000] last_map_addr: 1e0000000 end: 1e0000000
[    0.000000] RAMDISK: 009b9000 - 08cc2000
[    0.000000] DMI not present or invalid.
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-00000001e0000000
[    0.000000] Bootmem setup node 0 0000000000000000-00000001e0000000
[    0.000000]   NODE_DATA [0000000000001000 - 0000000000005fff]
[    0.000000]   bootmap [0000000000008000 -  0000000000043fff] pages 3c
[    0.000000] (7 early reservations) ==> bootmem [0000000000 - 01e0000000]
[    0.000000]   #0 [0000000000 - 0000001000]   BIOS data page ==>
[0000000000 - 0000001000]
[    0.000000]   #1 [0000006000 - 0000008000]       TRAMPOLINE ==>
[0000006000 - 0000008000]
[    0.000000]   #2 [0009bc5000 - 0009c18000]   XEN PAGETABLES ==>
[0009bc5000 - 0009c18000]
[    0.000000]   #3 [0000200000 - 00009b8568]    TEXT DATA BSS ==>
[0000200000 - 00009b8568]
[    0.000000]   #4 [00009b9000 - 0008cc2000]          RAMDISK ==>
[00009b9000 - 0008cc2000]
[    0.000000]   #5 [0009c18000 - 000a3c7000]          PGTABLE ==>
[0009c18000 - 000a3c7000]
[    0.000000]   #6 [000a3c7000 - 000aacb000]          PGTABLE ==>
[000a3c7000 - 000aacb000] [    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000000 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x001e0000
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[3] active PFN ranges
[    0.000000]     0: 0x00000000 -> 0x000000a0
[    0.000000]     0: 0x00000100 -> 0x00008cc2
[    0.000000]     0: 0x00009bc5 -> 0x001e0000
[    0.000000] SMP: Allowing 2 CPUs, 0 hotplug CPUs
[    0.000000] No local APIC present
[    0.000000] PCI: Warning: Cannot find a gap in the 32bit address range
[    0.000000] PCI: Unassigned devices with 32bit resource registers may break!
[    0.000000] Allocating PCI resources starting at 1e0200000 (gap:
1e0100000:400000)
[    0.000000] PERCPU: Allocating 59040 bytes of per cpu data
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total
pages: 1933282
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: root=/dev/ram0 rw load_ramdisk=1 
ramdisk_size=350000 console=tty0,9600n8 console=ttyS0,9600n8 bfile=disk_pri
[    0.000000] Initializing CPU#0
[    0.000000] PID hash table entries: 4096 (order: 12, 32768 bytes)
[    0.000000] Detected 1999.975 MHz processor.
[    0.004000] Console: colour dummy device 80x25
[    0.004000] console [tty0] enabled
[    0.004000] console [hvc0] enabled
[    0.004000] console [ttyS0] enabled
[    0.004000] Checking aperture...
[    0.004000] No AGP bridge found
[    0.004000] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    0.004000] Placing software IO TLB between 0x20000000 - 0x24000000
[    0.004000] Memory: 7509788k/7864320k available (4055k kernel code, 338776k
reserved, 2016k data, 544k init)
[    0.004000] SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=2,
Nodes=1
[    0.004000] installing Xen timer for CPU 0
[    0.004000] Calibrating delay loop (skipped), value calculated using timer
frequency.. 3999.95 BogoMIPS (lpj=7999900)
[    0.004000] Security Framework initialized
[    0.004000] SELinux:  Disabled at boot.
[    0.004000] ------------[ cut here ]------------
[    0.004000] kernel BUG at arch/x86/xen/enlighten.c:836!
[    0.004000] invalid opcode: 0000 [1] SMP
[    0.004000] CPU 0
[    0.004000] Modules linked in:
[    0.004000] Pid: 0, comm: swapper Not tainted 2.6.27 #3
[    0.004000] RIP: e030:[<ffffffff8020ab69>]  [<ffffffff8020ab69>]
pin_pagetable_pfn+0x49/0x60
[    0.004000] RSP: e02b:ffffffff8082bdc8  EFLAGS: 00010282
[    0.004000] RAX: 00000000ffffffea RBX: ffff880180011000 RCX: 0000000080011000
[    0.004000] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff8082bdc8
[    0.004000] RBP: ffff880180010020 R08: ffff880180002134 R09: 0000000000000000
[    0.004000] R10: 0000000000007ff0 R11: 0000000000000002 R12: 00000000ffffffff
[    0.004000] R13: ffffc20000805000 R14: ffffffff8082beb8 R15: ffffc20000805000
[    0.004000] FS:  0000000000000000(0000) GS:ffffffff80819c80(0000)
knlGS:0000000000000000
[    0.004000] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[    0.004000] CR2: 00000021f1e4d000 CR3: 0000000000201000 CR4: 0000000000002660
[    0.004000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.004000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[    0.004000] Process swapper (pid: 0, threadinfo ffffffff8082a000, task
ffffffff8077c340)
[    0.004000] Stack:  ffffffff00000000 00000000007993b8 ffff880180011000
ffff880180010020
[    0.004000]  00000000ffffffff ffffffff8028d372 0000000180010000
ffffc20000801000
[    0.004000]  00000000ffffffff ffffffff802976a9 0000000000000000
0000000200000000
[    0.004000] Call Trace:
[    0.004000]  [<ffffffff8028d372>] ? __pte_alloc_kernel+0x82/0xb0
[    0.004000]  [<ffffffff802976a9>] ? map_vm_area+0x309/0x340
[    0.004000]  [<ffffffff80297db8>] ? __vmalloc_area_node+0x118/0x170
[    0.004000]  [<ffffffff80874bd0>] ? alloc_large_system_hash+0x1f0/0x2c0
[    0.004000]  [<ffffffff80297cf9>] ? __vmalloc_area_node+0x59/0x170
[    0.004000]  [<ffffffff80874bd0>] ? alloc_large_system_hash+0x1f0/0x2c0
[    0.004000]  [<ffffffff8087619d>] ? vfs_caches_init+0x10d/0x160
[    0.004000]  [<ffffffff80856d31>] ? start_kernel+0x2a1/0x3b0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Jan Beulich

2013-Jun-19 08:01 UTC

head link

Re: Kernel panic in pin_pagetable_pfn

>>> Joe Cao <JCao@a10networks.com> 06/18/13 3:13 PM >>>
>static void pin_pagetable_pfn(unsigned cmd, unsigned long pfn)
>{        >struct mmuext_op op;
        >op.cmd = cmd;
        >op.arg1.mfn = pfn_to_mfn(pfn);
        >if (HYPERVISOR_mmuext_op(&op, 1, NULL, DOMID_SELF))
                >BUG();>}
This is almost certainly associated with some hypervisor messages, which may
help you
debugging this.
>The complete kernel log is attached below.  We noticed we don''t see
this panic with
>2.6.32 kernel.  But we''d like to keep with the 2.6.27 kernel if
possible.  Does anyone
>know if there''s a patch that can fix this problem?
With this old a kernel you''re unlikely to get much help, so
you''ll need to do some
debugging on your own and/or go through the respective changes between the
known non-working and known working kernel versions.

Jan

Possibly Parallel Threads

Search for more apparently analagous threads

Xen devel - Jun 2013 - Kernel panic in pin_pagetable_pfn

Kernel panic in pin_pagetable_pfn

Re: Kernel panic in pin_pagetable_pfn

Possibly Parallel Threads