thr3ads.net - Xen devel - [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request [Nov 2010]

If this information is useful, please help other people find it:
Share via:

Bruce Edge

2010-Nov-10 22:15 UTC

[Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build
22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.

The domU console output hung up prior to spitting out a prompt post root
login. I logged into the domU via ssh and ran an ''ls'' of an
NFS mounted dir
and
this immediately came out on the domU console:

[  480.467525] BUG: unable to handle kernel paging request at
ffff880077e9f030
[  480.467547] IP: [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
[  480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
[  480.467587] Oops: 0003 [#1] SMP
[  480.467601] last sysfs file: /sys/kernel/uevent_seqnum
[  480.467611] CPU 3
[  480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
[  480.467676]
[  480.467684] Pid: 3957, comm: zsh Not tainted
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
[  480.467695] RIP: e030:[<ffffffff810355f3>]  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
[  480.467710] RSP: e02b:ffff880077ebfbd8  EFLAGS: 00010202
[  480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX:
800000004d686167
[  480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI:
ffff8800763ddb80
[  480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09:
ffffea0001a3b2d8
[  480.467748] R10: 0000000000000100 R11: 0000000000000000 R12:
00007f5cb26069d0
[  480.467759] R13: 0000000000000001 R14: 0000000000000000 R15:
ffff8800763ddb80
[  480.467772] FS:  00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
knlGS:0000000000000000
[  480.467783] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4:
0000000000002660
[  480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[  480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task
ffff880078b80000)
[  480.467832] Stack:
[  480.467838]  0000000000000100 ffffea0001a3b2d8 7000000000000000
0000000000000000
[  480.467862]  ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
ffffffff810e2200
[  480.467886]  ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
0000000000000001
[  480.467910] Call Trace:
[  480.467921]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
[  480.467934]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[  480.467946]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
[  480.467958]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
[  480.467972]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
[  480.467984]  [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
[  480.467996]  [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
[  480.468009]  [<ffffffff810072a2>] ? check_events+0x12/0x20
[  480.468020]  [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1
[  480.468034]  [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
[  480.468045]  [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
[  480.468059]  [<ffffffff810072a2>] ? check_events+0x12/0x20
[  480.468072]  [<ffffffff81443615>] page_fault+0x25/0x30
[  480.468085]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
[  480.468096]  [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
[  480.468108]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
[  480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89 f4
53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18
<48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
[  480.468323] RIP  [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
[  480.468323]  RSP <ffff880077ebfbd8>
[  480.468323] CR2: ffff880077e9f030
[  480.468323] ---[ end trace 6d1623321ecd50ac ]---

The domU config is attached as config.2.6.37_domU_debug


-Bruce



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Nov-10 22:30 UTC

head link

[Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com>
wrote:>
> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build
22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
>
> The domU console output hung up prior to spitting out a prompt post root
login. I logged into the domU via ssh and ran an ''ls'' of an
NFS mounted dir and
> this immediately came out on the domU console:
>
> [  480.467525] BUG: unable to handle kernel paging request at
ffff880077e9f030
> [  480.467547] IP: [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> [  480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
> [  480.467587] Oops: 0003 [#1] SMP
> [  480.467601] last sysfs file: /sys/kernel/uevent_seqnum
> [  480.467611] CPU 3
> [  480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
> [  480.467676]
> [  480.467684] Pid: 3957, comm: zsh Not tainted
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [  480.467695] RIP: e030:[<ffffffff810355f3>] 
[<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
> [  480.467710] RSP: e02b:ffff880077ebfbd8  EFLAGS: 00010202
> [  480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX:
800000004d686167
> [  480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI:
ffff8800763ddb80
> [  480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09:
ffffea0001a3b2d8
> [  480.467748] R10: 0000000000000100 R11: 0000000000000000 R12:
00007f5cb26069d0
> [  480.467759] R13: 0000000000000001 R14: 0000000000000000 R15:
ffff8800763ddb80
> [  480.467772] FS:  00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
knlGS:0000000000000000
> [  480.467783] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4:
0000000000002660
> [  480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
> [  480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
> [  480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task
ffff880078b80000)
> [  480.467832] Stack:
> [  480.467838]  0000000000000100 ffffea0001a3b2d8 7000000000000000
0000000000000000
> [  480.467862]  ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
ffffffff810e2200
> [  480.467886]  ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
0000000000000001
> [  480.467910] Call Trace:
> [  480.467921]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
> [  480.467934]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [  480.467946]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [  480.467958]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
> [  480.467972]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [  480.467984]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.467996]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.468009]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> [  480.468020]  [<ffffffff8100728f>] ?
xen_restore_fl_direct_end+0x0/0x1
> [  480.468034]  [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
> [  480.468045]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.468059]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> [  480.468072]  [<ffffffff81443615>] page_fault+0x25/0x30
> [  480.468085]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
> [  480.468096]  [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
> [  480.468108]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
> [  480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89 f4
53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18
<48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
> [  480.468323] RIP  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> [  480.468323]  RSP <ffff880077ebfbd8>
> [  480.468323] CR2: ffff880077e9f030
> [  480.468323] ---[ end trace 6d1623321ecd50ac ]---
>
> The domU config is attached as config.2.6.37_domU_debug
>
>
> -Bruce
Here''s another attempt from the same config with the full domU boot
logs:

Started domain dpm (id=4)
                         [    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug (root@opal) (gcc version
4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Tue Nov 9 20:26:03 PST 2010
[    0.000000] Command line:
root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro   swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] released 0 pages of unused memory
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000080800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[    0.000000] Scanning 0 areas for low memory corruption
[    0.000000] init_memory_mapping: 0000000000000000-0000000080800000
[    0.000000] RAMDISK: 01d95000 - 02fa9000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-0000000080800000
[    0.000000] Initmem setup node 0 0000000000000000-0000000080800000
[    0.000000]   NODE_DATA [000000007fffb000 - 000000007fffffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x000000a0
[    0.000000]     0: 0x00000100 -> 0x00080800
[    0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
[    0.000000] No local APIC present
[    0.000000] APIC: disable apic facility
[    0.000000] APIC: switched to apic NOOP
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
[    0.000000] Allocating PCI resources starting at 80800000 (gap:
80800000:7f800000)
[    0.000000] Booting paravirtualized kernel on Xen
[    0.000000] Xen version: 4.1-unstable (preserve-AD)
[    0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
nr_cpu_ids:6 nr_node_ids:1
[    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
r8192 d23744 u114688
[    0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.
Total pages: 519022
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:
root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro   swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Placing 64MB software IO TLB between ffff880079400000 -
ffff88007d400000
[    0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
[    0.000000] Memory: 1959896k/2105344k available (4398k kernel code,
448k absent, 145000k reserved, 6327k data, 756k init)
[    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=6, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU-based detection of stalled CPUs is disabled.
[    0.000000] NR_IRQS:4352 nr_irqs:320 16
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [hvc0] enabled
[    0.000000] installing Xen timer for CPU 0
[    0.000000] Detected 2660.064 MHz processor.
[    0.000999] Calibrating delay loop (skipped), value calculated
using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
[    0.000999] pid_max: default: 32768 minimum: 301
[    0.000999] Security Framework initialized
[    0.000999] SELinux:  Initializing.
[    0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.001408] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.001635] Mount-cache hash table entries: 256
[    0.002195] Initializing cgroup subsys ns
[    0.002208] ns_cgroup deprecated: consider using the
''clone_children'' flag without the ns_cgroup.
[    0.002220] Initializing cgroup subsys cpuacct
[    0.002231] Initializing cgroup subsys freezer
[    0.002356] CPU: Unsupported number of siblings 16
[    0.002362] Performance Events: unsupported p6 CPU model 26 no PMU
driver, software events only.
[    0.002483] SMP alternatives: switching to UP code
[    0.004119] cpu 0 spinlock event irq 318
[    0.004765] NMI watchdog failed to create perf event on cpu0:
fffffffffffffffe
[    0.005050] installing Xen timer for CPU 1
[    0.005157] cpu 1 spinlock event irq 312
[    0.005194] SMP alternatives: switching to SMP code
[    0.000999] CPU: Unsupported number of siblings 16
[    0.006158] NMI watchdog failed to create perf event on cpu1:
fffffffffffffffe
[    0.006474] installing Xen timer for CPU 2
[    0.006585] cpu 2 spinlock event irq 306
[    0.000999] CPU: Unsupported number of siblings 16
[    0.006816] NMI watchdog failed to create perf event on cpu2:
fffffffffffffffe
[    0.006999] installing Xen timer for CPU 3
[    0.007037] cpu 3 spinlock event irq 300
[    0.000999] CPU: Unsupported number of siblings 16
[    0.007300] NMI watchdog failed to create perf event on cpu3:
fffffffffffffffe
[    0.007534] installing Xen timer for CPU 4
[    0.007636] cpu 4 spinlock event irq 294
[    0.000999] CPU: Unsupported number of siblings 16
[    0.007874] NMI watchdog failed to create perf event on cpu4:
fffffffffffffffe
[    0.008041] installing Xen timer for CPU 5
[    0.008083] cpu 5 spinlock event irq 288
[    0.000999] CPU: Unsupported number of siblings 16
[    0.008344] NMI watchdog failed to create perf event on cpu5:
fffffffffffffffe
[    0.008356] Brought up 6 CPUs
[    0.009181] kworker/u:0 used greatest stack depth: 5696 bytes left
[    0.012051] Grant table initialized
[    0.030978] Time: 165:165:165  Date: 165/165/65
[    0.031229] NET: Registered protocol family 16
[    0.036672] PCI: setting up Xen PCI frontend stub
[    0.037225] kworker/u:0 used greatest stack depth: 5528 bytes left
[    0.074302] bio: create slab <bio-0> at 0
[    0.075385] ACPI: Interpreter disabled.
[    0.077056] xen_balloon: Initialising balloon driver.
[    0.079323] vgaarb: loaded
[    0.081270] usbcore: registered new interface driver usbfs
[    0.081283] usbcore: registered new interface driver hub
[    0.081283] usbcore: registered new device driver usb
[    0.083183] PCI: System does not support PCI
[    0.084007] PCI: System does not support PCI
[    0.085236] NetLabel: Initializing
[    0.085236] NetLabel:  domain hash size = 128
[    0.085236] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.085313] NetLabel:  unlabeled traffic allowed by default
[    0.086544] Switching to clocksource xen
[    0.106256] pnp: PnP ACPI: disabled
[    0.129799] NET: Registered protocol family 2
[    0.130174] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.131882] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[    0.132891] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.133179] TCP: Hash tables configured (established 262144 bind 65536)
[    0.133191] TCP reno registered
[    0.133301] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[    0.133407] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[    0.133914] NET: Registered protocol family 1
[    0.134582] RPC: Registered udp transport module.
[    0.134590] RPC: Registered tcp transport module.
[    0.134595] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.134892] Trying to unpack rootfs image as initramfs...
[    0.178824] Freeing initrd memory: 18512k freed
[    0.268798] DMA-API: preallocated 32768 debug entries
[    0.268810] DMA-API: debugging enabled by kernel config
[    0.271181] platform rtc_cmos: registered platform RTC device (no
PNP device found)
[    0.285410] Machine check injector initialized
[    0.294732] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
[    0.294803] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
[    0.294893] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
[    0.294944] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
[    0.294998] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
[    0.295043] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
[    0.295590] microcode: Microcode Update Driver: v2.00
<tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    0.295607] Scanning for low memory corruption every 60 seconds
[    0.298813] audit: initializing netlink socket (disabled)
[    0.298940] type=2000 audit(1289403082.795:1): initialized
[    0.333762] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.376985] VFS: Disk quotas dquot_6.5.2
[    0.377853] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.383121] NTFS driver 2.1.29 [Flags: R/W].
[    0.385014] msgmni has been set to 3864
[    0.389953] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[    0.389970] io scheduler noop registered
[    0.389979] io scheduler deadline registered
[    0.390666] io scheduler cfq registered (default)
[    0.392486] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.406238] pcifront pci-0: Installing PCI frontend
[    0.406967] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/0
[    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/2
[    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/3
[    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/4
[    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/0
[    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/2
[    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/3
[    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/4
[    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/0
[    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/2
[    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/3
[    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/4
[    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/0
[    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/2
[    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/3
[    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/4
[    0.731775] Non-volatile memory driver v1.3
[    0.731790] Linux agpgart interface v0.103
[    0.733731] [drm] Initialized drm 1.1.0 20060810
[    0.733752] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.827202] brd: module loaded
[    0.840788] loop: module loaded
[    0.841965] Fixed MDIO Bus: probed
[    0.844025] ehci_hcd: USB 2.0 ''Enhanced'' Host Controller
(EHCI) Driver
[    0.844651] ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI)
Driver
[    0.845194] uhci_hcd: USB Universal Host Controller Interface driver
[    0.846985] usbcore: registered new interface driver usblp
[    0.847763] usbcore: registered new interface driver libusual
[    0.849433] PNP: No PS/2 controller found. Probing ports directly.
[    0.850279] i8042.c: No controller found.
[    0.850883] mice: PS/2 mouse device common for all mice
[    0.853446] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[    0.854066] rtc_cmos: probe of rtc_cmos failed with error -38
[    0.855928] cpuidle: using governor ladder
[    0.855941] cpuidle: using governor menu
[    0.856029] Netfilter messages via NETLINK v0.30.
[    0.856136] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    0.857888] ctnetlink v0.93: registering with nfnetlink.
[    0.860285] ip_tables: (C) 2000-2006 Netfilter Core Team
[    0.860677] TCP cubic registered
[    0.860687] Initializing XFRM netlink socket
[    0.863286] NET: Registered protocol family 10
[    0.867216] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    0.867612] IPv6 over IPv4 tunneling driver
[    0.871326] NET: Registered protocol family 17
[    0.871566] Registering the dns_resolver key type
[    0.873668] registered taskstats version 1
[    0.873912] kmemleak: Kernel memory leak detector initialized
[    0.873928] kmemleak: Automatic memory scanning thread started
[    0.875057] XENBUS: Device with no driver: device/vbd/51712
[    0.875057] XENBUS: Device with no driver: device/vif/0
[    0.875057] XENBUS: Device with no driver: device/console/0
[    0.875057]   Magic number: 1:252:3141
[   13.388958] Freeing unused kernel memory: 756k freed
[   13.389190] Write protecting the kernel read-only data: 10240k
[   13.398774] Freeing unused kernel memory: 1724k freed
[   13.400663] Freeing unused kernel memory: 1992k freed
[   13.494573] all_generic_ide used greatest stack depth: 5472 bytes left
[   13.519388] udevd (1111): /proc/1111/oom_adj is deprecated, please
use /proc/1111/oom_score_adj instead.
Begin: Loading essential drivers... ...
[   13.559435] blkid used greatest stack depth: 5408 bytes left
[   13.568391] blkfront: xvda: barriers disabled
[   13.574762] blkid used greatest stack depth: 5264 bytes left
[   13.580813]  xvda: xvda1 xvda2 < xvda5 >
[   13.645562] modprobe used greatest stack depth: 5088 bytes left
[   13.653160] Initialising Xen virtual ethernet driver.
[   13.659540] blkid used greatest stack depth: 4960 bytes left
[   13.735467] blkid used greatest stack depth: 4752 bytes left
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
[   13.903774] EXT4-fs (xvda1): INFO: recovery required on readonly filesystem
[   13.903799] EXT4-fs (xvda1): write access will be enabled during recovery
[   14.033719] EXT4-fs (xvda1): recovery complete
[   14.034966] EXT4-fs (xvda1): mounted filesystem with ordered data
mode. Opts: (null)
[   14.035510] exe used greatest stack depth: 4112 bytes left
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
init: ureadahead main process (1214) terminated with status 5
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
/dev/xvda1: clean, 56314/288576 files, 287743/1153280 blocks
 * Enable login on console                                               [ OK ]
 * Check for hostname changes...                                         [ OK ]
   ...done.
 * Setting sensors limits
         No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
                                                                         [ OK ]
 * Initializing random number generator...                               [ OK ]
 * Setting up X server socket directory /tmp/.X11-unix...                [ OK ]
 * Setting up ICE socket directory /tmp/.ICE-unix...                     [ OK ]
 * Not starting fancontrol; run pwmconfig first.
 * Starting internet superserver inetd                                   [ OK ]
Starting openntpd: ntpd.
 * Starting network management services:
 * Setting core file location...                                         [ OK ]
 * Restoring resolver state...                                           [ OK ]
 * Checking tomcat configuration
   ...done.
 * Starting Tomcat servlet engine tomcat6
         Using CATALINA_BASE:   /var/lib/tomcat6
Using CATALINA_HOME:   /usr/share/tomcat6
Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
Using JRE_HOME:        /usr/lib/jvm/java-6-openjdk
Using CLASSPATH:       /usr/share/tomcat6/bin/bootstrap.jar
                                                                         [ OK ]
collectd has been disabled - see /etc/default/collectd.
 * Recording successful boot for GRUB
   ...done.
 * Running local boot scripts (/etc/rc.local)                            [ OK ]
mountall: Plymouth command failed
mountall: Disconnected from Plymouth

6.0.16-01
kaan-04-dpm login: root
Password:
[   42.646538] ------------[ cut here ]------------
[   42.646556] kernel BUG at arch/x86/xen/mmu.c:1831!
[   42.646566] invalid opcode: 0000 [#1] SMP
[   42.646582] last sysfs file: /sys/kernel/uevent_seqnum
[   42.646592] CPU 0
[   42.646598] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
xen_netfront xen_blkfront
[   42.646665]
[   42.646674] Pid: 3916, comm: lsb_release Not tainted
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
[   42.646688] RIP: e030:[<ffffffff81004bce>]  [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[   42.646711] RSP: e02b:ffff88007595bce8  EFLAGS: 00010282
[   42.646721] RAX: 00000000ffffffea RBX: 000000000007751e RCX: 0000000000000001
[   42.646732] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
[   42.646744] RBP: ffff88007595bd08 R08: 00003ffffffff000 R09: ffff880000000000
[   42.646756] R10: 00000000deadbeef R11: dead000000200200 R12: 0000000000000003
[   42.646768] R13: ffff880075c90150 R14: ffff8800781d9398 R15: ffff880075161d40
[   42.646784] FS:  00007fee63b01700(0000) GS:ffff88007ff40000(0000)
knlGS:0000000000000000
[   42.646797] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[   42.646807] CR2: 00007faac54182b8 CR3: 00000000771dd000 CR4: 0000000000002660
[   42.646819] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   42.646832] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[   42.646844] Process lsb_release (pid: 3916, threadinfo
ffff88007595a000, task ffff880075161d40)
[   42.646857] Stack:
[   42.646864]  ffff880000000000 00000000000484f0 ffff8800777c6d80
000000000007751e
[   42.646892]  ffff88007595bd28 ffffffff810066a8 000000000007751e
ffff8800777c6d80
[   42.646919]  ffff88007595bd38 ffffffff810066db ffff88007595bd78
ffffffff810e048f
[   42.646946] Call Trace:
[   42.646959]  [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[   42.646973]  [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[   42.646989]  [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[   42.647003]  [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
[   42.647017]  [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
[   42.647033]  [<ffffffff8100eb84>] ?
arch_get_unmapped_area_topdown+0x4a/0x28a
[   42.647052]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
[   42.647059]  [<ffffffff8102b78a>] ? pvclock_clocksource_read+0x4b/0xb4
[   42.647059]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[   42.647059]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
[   42.647059]  [<ffffffff81443615>] page_fault+0x25/0x30
[   42.647059] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff
85 c0 74 04 <0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48
8b 3d
[   42.647059] RIP  [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[   42.647059]  RSP <ffff88007595bce8>
[   42.647059] ---[ end trace 3fbbffee414a5166 ]---

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Nov-10 22:39 UTC

head link

[Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Wed, Nov 10, 2010 at 02:15:25PM -0800, Bruce Edge
wrote:> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build
> 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
I need the full output.> 
> The domU console output hung up prior to spitting out a prompt post root
> login. I logged into the domU via ssh and ran an ''ls'' of
an NFS mounted dir
> and
> this immediately came out on the domU console:
> 
> [  480.467525] BUG: unable to handle kernel paging request at
> ffff880077e9f030
> [  480.467547] IP: [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> [  480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
> [  480.467587] Oops: 0003 [#1] SMP
> [  480.467601] last sysfs file: /sys/kernel/uevent_seqnum
> [  480.467611] CPU 3
> [  480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
> xen_blkfront
> [  480.467676]
> [  480.467684] Pid: 3957, comm: zsh Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [  480.467695] RIP: e030:[<ffffffff810355f3>] 
[<ffffffff810355f3>]
> ptep_set_access_flags+0x27/0x4d
> [  480.467710] RSP: e02b:ffff880077ebfbd8  EFLAGS: 00010202
> [  480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX:
> 800000004d686167
> [  480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI:
> ffff8800763ddb80
> [  480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09:
> ffffea0001a3b2d8
> [  480.467748] R10: 0000000000000100 R11: 0000000000000000 R12:
> 00007f5cb26069d0
> [  480.467759] R13: 0000000000000001 R14: 0000000000000000 R15:
> ffff8800763ddb80
> [  480.467772] FS:  00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
> knlGS:0000000000000000
> [  480.467783] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4:
> 0000000000002660
> [  480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [  480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [  480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task
> ffff880078b80000)
> [  480.467832] Stack:
> [  480.467838]  0000000000000100 ffffea0001a3b2d8 7000000000000000
> 0000000000000000
> [  480.467862]  ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
> ffffffff810e2200
> [  480.467886]  ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
> 0000000000000001
> [  480.467910] Call Trace:
> [  480.467921]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
> [  480.467934]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [  480.467946]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [  480.467958]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
> [  480.467972]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [  480.467984]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.467996]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.468009]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> [  480.468020]  [<ffffffff8100728f>] ?
xen_restore_fl_direct_end+0x0/0x1
> [  480.468034]  [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
> [  480.468045]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.468059]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> [  480.468072]  [<ffffffff81443615>] page_fault+0x25/0x30
> [  480.468085]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
> [  480.468096]  [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
> [  480.468108]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
> [  480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89 f4
> 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18
> <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
> [  480.468323] RIP  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> [  480.468323]  RSP <ffff880077ebfbd8>
> [  480.468323] CR2: ffff880077e9f030
> [  480.468323] ---[ end trace 6d1623321ecd50ac ]---
> 
> The domU config is attached as config.2.6.37_domU_debug
> 
> 
> -Bruce


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

M A Young

2010-Nov-10 22:50 UTC

head link

Re: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Wed, 10 Nov 2010, Bruce Edge wrote:
> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build
> 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
> 
> The domU console output hung up prior to spitting out a prompt post root
> login. I logged into the domU via ssh and ran an ''ls'' of
an NFS mounted dir
> and
> this immediately came out on the domU console:
> 
> [  480.467525] BUG: unable to handle kernel paging request at
> ffff880077e9f030
> [  480.467547] IP: [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
I had a very similar crash, that I reported earlier 
http://lists.xensource.com/archives/html/xen-devel/2010-10/msg01647.html
But this was in a 2.6.37 dom0 kernel when trying to start a guest.

 	Michael Young
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Sander Eikelenboom

2010-Nov-10 22:51 UTC

head link

Re: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Hmm i''m just running domU''s with 2.6.37-rc1 and pci
passthrough from today.
Will see how (un)stable it is.

Stacktrace seems slightly related to the one trying to start a domU on a
2.6.37-rc1 dom0
(http://lists.xensource.com/archives/html/xen-devel/2010-11/msg00154.html)

--
Sander

Wednesday, November 10, 2010, 11:15:25 PM, you wrote:
> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build
> 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
> The domU console output hung up prior to spitting out a prompt post root
> login. I logged into the domU via ssh and ran an ''ls'' of
an NFS mounted dir
> and
> this immediately came out on the domU console:
> [  480.467525] BUG: unable to handle kernel paging request at
> ffff880077e9f030
> [  480.467547] IP: [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> [  480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
> [  480.467587] Oops: 0003 [#1] SMP
> [  480.467601] last sysfs file: /sys/kernel/uevent_seqnum
> [  480.467611] CPU 3
> [  480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
> xen_blkfront
> [  480.467676]
> [  480.467684] Pid: 3957, comm: zsh Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [  480.467695] RIP: e030:[<ffffffff810355f3>] 
[<ffffffff810355f3>]
> ptep_set_access_flags+0x27/0x4d
> [  480.467710] RSP: e02b:ffff880077ebfbd8  EFLAGS: 00010202
> [  480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX:
> 800000004d686167
> [  480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI:
> ffff8800763ddb80
> [  480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09:
> ffffea0001a3b2d8
> [  480.467748] R10: 0000000000000100 R11: 0000000000000000 R12:
> 00007f5cb26069d0
> [  480.467759] R13: 0000000000000001 R14: 0000000000000000 R15:
> ffff8800763ddb80
> [  480.467772] FS:  00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
> knlGS:0000000000000000
> [  480.467783] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4:
> 0000000000002660
> [  480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [  480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [  480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task
> ffff880078b80000)
> [  480.467832] Stack:
> [  480.467838]  0000000000000100 ffffea0001a3b2d8 7000000000000000
> 0000000000000000
> [  480.467862]  ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
> ffffffff810e2200
> [  480.467886]  ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
> 0000000000000001
> [  480.467910] Call Trace:
> [  480.467921]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
> [  480.467934]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [  480.467946]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [  480.467958]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
> [  480.467972]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [  480.467984]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.467996]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.468009]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> [  480.468020]  [<ffffffff8100728f>] ?
xen_restore_fl_direct_end+0x0/0x1
> [  480.468034]  [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
> [  480.468045]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.468059]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> [  480.468072]  [<ffffffff81443615>] page_fault+0x25/0x30
> [  480.468085]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
> [  480.468096]  [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
> [  480.468108]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
> [  480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89 f4
> 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18
> <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
> [  480.468323] RIP  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> [  480.468323]  RSP <ffff880077ebfbd8>
> [  480.468323] CR2: ffff880077e9f030
> [  480.468323] ---[ end trace 6d1623321ecd50ac ]---
> The domU config is attached as config.2.6.37_domU_debug
> -Bruce


-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Nov-10 23:03 UTC

head link

[Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Wed, Nov 10, 2010 at 2:30 PM, Bruce Edge <bruce.edge@gmail.com>
wrote:> On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com>
wrote:
>>
>> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build
22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
>>
>> The domU console output hung up prior to spitting out a prompt post
root login. I logged into the domU via ssh and ran an ''ls'' of
an NFS mounted dir and
>> this immediately came out on the domU console:
>>
>> [  480.467525] BUG: unable to handle kernel paging request at
ffff880077e9f030
>> [  480.467547] IP: [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
>> [  480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
>> [  480.467587] Oops: 0003 [#1] SMP
>> [  480.467601] last sysfs file: /sys/kernel/uevent_seqnum
>> [  480.467611] CPU 3
>> [  480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
>> [  480.467676]
>> [  480.467684] Pid: 3957, comm: zsh Not tainted
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
>> [  480.467695] RIP: e030:[<ffffffff810355f3>] 
[<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
>> [  480.467710] RSP: e02b:ffff880077ebfbd8  EFLAGS: 00010202
>> [  480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX:
800000004d686167
>> [  480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI:
ffff8800763ddb80
>> [  480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09:
ffffea0001a3b2d8
>> [  480.467748] R10: 0000000000000100 R11: 0000000000000000 R12:
00007f5cb26069d0
>> [  480.467759] R13: 0000000000000001 R14: 0000000000000000 R15:
ffff8800763ddb80
>> [  480.467772] FS:  00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
knlGS:0000000000000000
>> [  480.467783] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [  480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4:
0000000000002660
>> [  480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
>> [  480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
>> [  480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000,
task ffff880078b80000)
>> [  480.467832] Stack:
>> [  480.467838]  0000000000000100 ffffea0001a3b2d8 7000000000000000
0000000000000000
>> [  480.467862]  ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
ffffffff810e2200
>> [  480.467886]  ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
0000000000000001
>> [  480.467910] Call Trace:
>> [  480.467921]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
>> [  480.467934]  [<ffffffff81007c09>] ?
__spin_time_accum+0x21/0x37
>> [  480.467946]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
>> [  480.467958]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
>> [  480.467972]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
>> [  480.467984]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
>> [  480.467996]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
>> [  480.468009]  [<ffffffff810072a2>] ? check_events+0x12/0x20
>> [  480.468020]  [<ffffffff8100728f>] ?
xen_restore_fl_direct_end+0x0/0x1
>> [  480.468034]  [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
>> [  480.468045]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
>> [  480.468059]  [<ffffffff810072a2>] ? check_events+0x12/0x20
>> [  480.468072]  [<ffffffff81443615>] page_fault+0x25/0x30
>> [  480.468085]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
>> [  480.468096]  [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
>> [  480.468108]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
>> [  480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89
f4 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18
<48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
>> [  480.468323] RIP  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
>> [  480.468323]  RSP <ffff880077ebfbd8>
>> [  480.468323] CR2: ffff880077e9f030
>> [  480.468323] ---[ end trace 6d1623321ecd50ac ]---
>>
>> The domU config is attached as config.2.6.37_domU_debug
>>
>>
>> -Bruce
>
> Here''s another attempt from the same config with the full domU
boot logs:
>
> Started domain dpm (id=4)
>                         [    0.000000] Initializing cgroup subsys cpuset
> [    0.000000] Initializing cgroup subsys cpu
> [    0.000000] Linux version
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug (root@opal) (gcc version
> 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Tue Nov 9 20:26:03 PST 2010
> [    0.000000] Command line:
> root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro   swiotlb=force
> noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
> ip=:127.0.255.255::::eth0:dhcp
> [    0.000000] ACPI in unprivileged domain disabled
> [    0.000000] released 0 pages of unused memory
> [    0.000000] BIOS-provided physical RAM map:
> [    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
> [    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
> [    0.000000]  Xen: 0000000000100000 - 0000000080800000 (usable)
> [    0.000000] NX (Execute Disable) protection: active
> [    0.000000] DMI not present or invalid.
> [    0.000000] No AGP bridge found
> [    0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
> [    0.000000] Scanning 0 areas for low memory corruption
> [    0.000000] init_memory_mapping: 0000000000000000-0000000080800000
> [    0.000000] RAMDISK: 01d95000 - 02fa9000
> [    0.000000] No NUMA configuration found
> [    0.000000] Faking a node at 0000000000000000-0000000080800000
> [    0.000000] Initmem setup node 0 0000000000000000-0000000080800000
> [    0.000000]   NODE_DATA [000000007fffb000 - 000000007fffffff]
> [    0.000000] Zone PFN ranges:
> [    0.000000]   DMA      0x00000010 -> 0x00001000
> [    0.000000]   DMA32    0x00001000 -> 0x00100000
> [    0.000000]   Normal   empty
> [    0.000000] Movable zone start PFN for each node
> [    0.000000] early_node_map[2] active PFN ranges
> [    0.000000]     0: 0x00000010 -> 0x000000a0
> [    0.000000]     0: 0x00000100 -> 0x00080800
> [    0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
> [    0.000000] No local APIC present
> [    0.000000] APIC: disable apic facility
> [    0.000000] APIC: switched to apic NOOP
> [    0.000000] PM: Registered nosave memory: 00000000000a0000 -
0000000000100000
> [    0.000000] Allocating PCI resources starting at 80800000 (gap:
> 80800000:7f800000)
> [    0.000000] Booting paravirtualized kernel on Xen
> [    0.000000] Xen version: 4.1-unstable (preserve-AD)
> [    0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
> nr_cpu_ids:6 nr_node_ids:1
> [    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
> r8192 d23744 u114688
> [    0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
> [    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
> [    0.000000] Built 1 zonelists in Node order, mobility grouping on.
> Total pages: 519022
> [    0.000000] Policy zone: DMA32
> [    0.000000] Kernel command line:
> root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro   swiotlb=force
> noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
> ip=:127.0.255.255::::eth0:dhcp
> [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
> [    0.000000] Placing 64MB software IO TLB between ffff880079400000 -
> ffff88007d400000
> [    0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
> [    0.000000] Memory: 1959896k/2105344k available (4398k kernel code,
> 448k absent, 145000k reserved, 6327k data, 756k init)
> [    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
> CPUs=6, Nodes=1
> [    0.000000] Hierarchical RCU implementation.
> [    0.000000]  RCU-based detection of stalled CPUs is disabled.
> [    0.000000] NR_IRQS:4352 nr_irqs:320 16
> [    0.000000] Console: colour dummy device 80x25
> [    0.000000] console [tty0] enabled
> [    0.000000] console [hvc0] enabled
> [    0.000000] installing Xen timer for CPU 0
> [    0.000000] Detected 2660.064 MHz processor.
> [    0.000999] Calibrating delay loop (skipped), value calculated
> using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
> [    0.000999] pid_max: default: 32768 minimum: 301
> [    0.000999] Security Framework initialized
> [    0.000999] SELinux:  Initializing.
> [    0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152
bytes)
> [    0.001408] Inode-cache hash table entries: 131072 (order: 8, 1048576
bytes)
> [    0.001635] Mount-cache hash table entries: 256
> [    0.002195] Initializing cgroup subsys ns
> [    0.002208] ns_cgroup deprecated: consider using the
> ''clone_children'' flag without the ns_cgroup.
> [    0.002220] Initializing cgroup subsys cpuacct
> [    0.002231] Initializing cgroup subsys freezer
> [    0.002356] CPU: Unsupported number of siblings 16
> [    0.002362] Performance Events: unsupported p6 CPU model 26 no PMU
> driver, software events only.
> [    0.002483] SMP alternatives: switching to UP code
> [    0.004119] cpu 0 spinlock event irq 318
> [    0.004765] NMI watchdog failed to create perf event on cpu0:
> fffffffffffffffe
> [    0.005050] installing Xen timer for CPU 1
> [    0.005157] cpu 1 spinlock event irq 312
> [    0.005194] SMP alternatives: switching to SMP code
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.006158] NMI watchdog failed to create perf event on cpu1:
> fffffffffffffffe
> [    0.006474] installing Xen timer for CPU 2
> [    0.006585] cpu 2 spinlock event irq 306
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.006816] NMI watchdog failed to create perf event on cpu2:
> fffffffffffffffe
> [    0.006999] installing Xen timer for CPU 3
> [    0.007037] cpu 3 spinlock event irq 300
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.007300] NMI watchdog failed to create perf event on cpu3:
> fffffffffffffffe
> [    0.007534] installing Xen timer for CPU 4
> [    0.007636] cpu 4 spinlock event irq 294
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.007874] NMI watchdog failed to create perf event on cpu4:
> fffffffffffffffe
> [    0.008041] installing Xen timer for CPU 5
> [    0.008083] cpu 5 spinlock event irq 288
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.008344] NMI watchdog failed to create perf event on cpu5:
> fffffffffffffffe
> [    0.008356] Brought up 6 CPUs
> [    0.009181] kworker/u:0 used greatest stack depth: 5696 bytes left
> [    0.012051] Grant table initialized
> [    0.030978] Time: 165:165:165  Date: 165/165/65
> [    0.031229] NET: Registered protocol family 16
> [    0.036672] PCI: setting up Xen PCI frontend stub
> [    0.037225] kworker/u:0 used greatest stack depth: 5528 bytes left
> [    0.074302] bio: create slab <bio-0> at 0
> [    0.075385] ACPI: Interpreter disabled.
> [    0.077056] xen_balloon: Initialising balloon driver.
> [    0.079323] vgaarb: loaded
> [    0.081270] usbcore: registered new interface driver usbfs
> [    0.081283] usbcore: registered new interface driver hub
> [    0.081283] usbcore: registered new device driver usb
> [    0.083183] PCI: System does not support PCI
> [    0.084007] PCI: System does not support PCI
> [    0.085236] NetLabel: Initializing
> [    0.085236] NetLabel:  domain hash size = 128
> [    0.085236] NetLabel:  protocols = UNLABELED CIPSOv4
> [    0.085313] NetLabel:  unlabeled traffic allowed by default
> [    0.086544] Switching to clocksource xen
> [    0.106256] pnp: PnP ACPI: disabled
> [    0.129799] NET: Registered protocol family 2
> [    0.130174] IP route cache hash table entries: 65536 (order: 7, 524288
bytes)
> [    0.131882] TCP established hash table entries: 262144 (order: 10,
> 4194304 bytes)
> [    0.132891] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
> [    0.133179] TCP: Hash tables configured (established 262144 bind 65536)
> [    0.133191] TCP reno registered
> [    0.133301] UDP hash table entries: 1024 (order: 3, 32768 bytes)
> [    0.133407] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
> [    0.133914] NET: Registered protocol family 1
> [    0.134582] RPC: Registered udp transport module.
> [    0.134590] RPC: Registered tcp transport module.
> [    0.134595] RPC: Registered tcp NFSv4.1 backchannel transport module.
> [    0.134892] Trying to unpack rootfs image as initramfs...
> [    0.178824] Freeing initrd memory: 18512k freed
> [    0.268798] DMA-API: preallocated 32768 debug entries
> [    0.268810] DMA-API: debugging enabled by kernel config
> [    0.271181] platform rtc_cmos: registered platform RTC device (no
> PNP device found)
> [    0.285410] Machine check injector initialized
> [    0.294732] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
> [    0.294803] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
> [    0.294893] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
> [    0.294944] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
> [    0.294998] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
> [    0.295043] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
> [    0.295590] microcode: Microcode Update Driver: v2.00
> <tigran@aivazian.fsnet.co.uk>, Peter Oruba
> [    0.295607] Scanning for low memory corruption every 60 seconds
> [    0.298813] audit: initializing netlink socket (disabled)
> [    0.298940] type=2000 audit(1289403082.795:1): initialized
> [    0.333762] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [    0.376985] VFS: Disk quotas dquot_6.5.2
> [    0.377853] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> [    0.383121] NTFS driver 2.1.29 [Flags: R/W].
> [    0.385014] msgmni has been set to 3864
> [    0.389953] Block layer SCSI generic (bsg) driver version 0.4
> loaded (major 253)
> [    0.389970] io scheduler noop registered
> [    0.389979] io scheduler deadline registered
> [    0.390666] io scheduler cfq registered (default)
> [    0.392486] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> [    0.406238] pcifront pci-0: Installing PCI frontend
> [    0.406967] pcifront pci-0: Creating PCI Frontend Bus 0000:00
> [    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/0
> [    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/2
> [    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/3
> [    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/4
> [    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/0
> [    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/2
> [    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/3
> [    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/4
> [    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/0
> [    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/2
> [    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/3
> [    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/4
> [    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/0
> [    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/2
> [    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/3
> [    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/4
> [    0.731775] Non-volatile memory driver v1.3
> [    0.731790] Linux agpgart interface v0.103
> [    0.733731] [drm] Initialized drm 1.1.0 20060810
> [    0.733752] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [    0.827202] brd: module loaded
> [    0.840788] loop: module loaded
> [    0.841965] Fixed MDIO Bus: probed
> [    0.844025] ehci_hcd: USB 2.0 ''Enhanced'' Host
Controller (EHCI) Driver
> [    0.844651] ohci_hcd: USB 1.1 ''Open'' Host Controller
(OHCI) Driver
> [    0.845194] uhci_hcd: USB Universal Host Controller Interface driver
> [    0.846985] usbcore: registered new interface driver usblp
> [    0.847763] usbcore: registered new interface driver libusual
> [    0.849433] PNP: No PS/2 controller found. Probing ports directly.
> [    0.850279] i8042.c: No controller found.
> [    0.850883] mice: PS/2 mouse device common for all mice
> [    0.853446] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
> [    0.854066] rtc_cmos: probe of rtc_cmos failed with error -38
> [    0.855928] cpuidle: using governor ladder
> [    0.855941] cpuidle: using governor menu
> [    0.856029] Netfilter messages via NETLINK v0.30.
> [    0.856136] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
> [    0.857888] ctnetlink v0.93: registering with nfnetlink.
> [    0.860285] ip_tables: (C) 2000-2006 Netfilter Core Team
> [    0.860677] TCP cubic registered
> [    0.860687] Initializing XFRM netlink socket
> [    0.863286] NET: Registered protocol family 10
> [    0.867216] ip6_tables: (C) 2000-2006 Netfilter Core Team
> [    0.867612] IPv6 over IPv4 tunneling driver
> [    0.871326] NET: Registered protocol family 17
> [    0.871566] Registering the dns_resolver key type
> [    0.873668] registered taskstats version 1
> [    0.873912] kmemleak: Kernel memory leak detector initialized
> [    0.873928] kmemleak: Automatic memory scanning thread started
> [    0.875057] XENBUS: Device with no driver: device/vbd/51712
> [    0.875057] XENBUS: Device with no driver: device/vif/0
> [    0.875057] XENBUS: Device with no driver: device/console/0
> [    0.875057]   Magic number: 1:252:3141
> [   13.388958] Freeing unused kernel memory: 756k freed
> [   13.389190] Write protecting the kernel read-only data: 10240k
> [   13.398774] Freeing unused kernel memory: 1724k freed
> [   13.400663] Freeing unused kernel memory: 1992k freed
> [   13.494573] all_generic_ide used greatest stack depth: 5472 bytes left
> [   13.519388] udevd (1111): /proc/1111/oom_adj is deprecated, please
> use /proc/1111/oom_score_adj instead.
> Begin: Loading essential drivers... ...
> [   13.559435] blkid used greatest stack depth: 5408 bytes left
> [   13.568391] blkfront: xvda: barriers disabled
> [   13.574762] blkid used greatest stack depth: 5264 bytes left
> [   13.580813]  xvda: xvda1 xvda2 < xvda5 >
> [   13.645562] modprobe used greatest stack depth: 5088 bytes left
> [   13.653160] Initialising Xen virtual ethernet driver.
> [   13.659540] blkid used greatest stack depth: 4960 bytes left
> [   13.735467] blkid used greatest stack depth: 4752 bytes left
> Done.
> Begin: Running /scripts/init-premount ...
> Done.
> Begin: Mounting root file system... ...
> Begin: Running /scripts/local-top ...
> Done.
> Begin: Running /scripts/local-premount ...
> Done.
> [   13.903774] EXT4-fs (xvda1): INFO: recovery required on readonly
filesystem
> [   13.903799] EXT4-fs (xvda1): write access will be enabled during
recovery
> [   14.033719] EXT4-fs (xvda1): recovery complete
> [   14.034966] EXT4-fs (xvda1): mounted filesystem with ordered data
> mode. Opts: (null)
> [   14.035510] exe used greatest stack depth: 4112 bytes left
> Begin: Running /scripts/local-bottom ...
> Done.
> Done.
> Begin: Running /scripts/init-bottom ...
> Done.
> init: ureadahead main process (1214) terminated with status 5
> fsck from util-linux-ng 2.17.2
> e2fsck 1.41.11 (14-Mar-2010)
> /dev/xvda1: clean, 56314/288576 files, 287743/1153280 blocks
>  * Enable login on console                                               [
OK ]
>  * Check for hostname changes...                                         [
OK ]
>   ...done.
>  * Setting sensors limits
>         No sensors found!
> Make sure you loaded all the kernel drivers you need.
> Try sensors-detect to find out which these are.
> No sensors found!
> Make sure you loaded all the kernel drivers you need.
> Try sensors-detect to find out which these are.
>                                                                         [
OK ]
>  * Initializing random number generator...                               [
OK ]
>  * Setting up X server socket directory /tmp/.X11-unix...                [
OK ]
>  * Setting up ICE socket directory /tmp/.ICE-unix...                     [
OK ]
>  * Not starting fancontrol; run pwmconfig first.
>  * Starting internet superserver inetd                                   [
OK ]
> Starting openntpd: ntpd.
>  * Starting network management services:
>  * Setting core file location...                                         [
OK ]
>  * Restoring resolver state...                                           [
OK ]
>  * Checking tomcat configuration
>   ...done.
>  * Starting Tomcat servlet engine tomcat6
>         Using CATALINA_BASE:   /var/lib/tomcat6
> Using CATALINA_HOME:   /usr/share/tomcat6
> Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
> Using JRE_HOME:        /usr/lib/jvm/java-6-openjdk
> Using CLASSPATH:       /usr/share/tomcat6/bin/bootstrap.jar
>                                                                         [
OK ]
> collectd has been disabled - see /etc/default/collectd.
>  * Recording successful boot for GRUB
>   ...done.
>  * Running local boot scripts (/etc/rc.local)                            [
OK ]
> mountall: Plymouth command failed
> mountall: Disconnected from Plymouth
>
> 6.0.16-01
> kaan-04-dpm login: root
> Password:
> [   42.646538] ------------[ cut here ]------------
> [   42.646556] kernel BUG at arch/x86/xen/mmu.c:1831!
> [   42.646566] invalid opcode: 0000 [#1] SMP
> [   42.646582] last sysfs file: /sys/kernel/uevent_seqnum
> [   42.646592] CPU 0
> [   42.646598] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
> xen_netfront xen_blkfront
> [   42.646665]
> [   42.646674] Pid: 3916, comm: lsb_release Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [   42.646688] RIP: e030:[<ffffffff81004bce>]
 [<ffffffff81004bce>]
> pin_pagetable_pfn+0x31/0x37
> [   42.646711] RSP: e02b:ffff88007595bce8  EFLAGS: 00010282
> [   42.646721] RAX: 00000000ffffffea RBX: 000000000007751e RCX:
0000000000000001
> [   42.646732] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
00000000deadbeef
> [   42.646744] RBP: ffff88007595bd08 R08: 00003ffffffff000 R09:
ffff880000000000
> [   42.646756] R10: 00000000deadbeef R11: dead000000200200 R12:
0000000000000003
> [   42.646768] R13: ffff880075c90150 R14: ffff8800781d9398 R15:
ffff880075161d40
> [   42.646784] FS:  00007fee63b01700(0000) GS:ffff88007ff40000(0000)
> knlGS:0000000000000000
> [   42.646797] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [   42.646807] CR2: 00007faac54182b8 CR3: 00000000771dd000 CR4:
0000000000002660
> [   42.646819] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
> [   42.646832] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
> [   42.646844] Process lsb_release (pid: 3916, threadinfo
> ffff88007595a000, task ffff880075161d40)
> [   42.646857] Stack:
> [   42.646864]  ffff880000000000 00000000000484f0 ffff8800777c6d80
> 000000000007751e
> [   42.646892]  ffff88007595bd28 ffffffff810066a8 000000000007751e
> ffff8800777c6d80
> [   42.646919]  ffff88007595bd38 ffffffff810066db ffff88007595bd78
> ffffffff810e048f
> [   42.646946] Call Trace:
> [   42.646959]  [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
> [   42.646973]  [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
> [   42.646989]  [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
> [   42.647003]  [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
> [   42.647017]  [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
> [   42.647033]  [<ffffffff8100eb84>] ?
arch_get_unmapped_area_topdown+0x4a/0x28a
> [   42.647052]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [   42.647059]  [<ffffffff8102b78a>] ?
pvclock_clocksource_read+0x4b/0xb4
> [   42.647059]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [   42.647059]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [   42.647059]  [<ffffffff81443615>] page_fault+0x25/0x30
> [   42.647059] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
> e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff
> 85 c0 74 04 <0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db
48
> 8b 3d
> [   42.647059] RIP  [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
> [   42.647059]  RSP <ffff88007595bce8>
> [   42.647059] ---[ end trace 3fbbffee414a5166 ]---
>
Once more, this time with CONFIG_XEN_PVHVM=n just in case it was
related to Stefano''s PVonHVM work.

[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version
2.6.37-rc1-kernel.org-domu-6.0.nopvhvm.bedge-debug (root@opal) (gcc
version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Wed Nov 10 12:01:55 PST
2010
[    0.000000] Command line:
root=UUID=e4e223aa-080e-421b-98ac-ab0b3e0e57e5 ro   swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] released 0 pages of unused memory
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000080800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[    0.000000] Scanning 0 areas for low memory corruption
[    0.000000] init_memory_mapping: 0000000000000000-0000000080800000
[    0.000000] RAMDISK: 01d95000 - 02faa000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-0000000080800000
[    0.000000] Initmem setup node 0 0000000000000000-0000000080800000
[    0.000000]   NODE_DATA [000000007fffb000 - 000000007fffffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x000000a0
[    0.000000]     0: 0x00000100 -> 0x00080800
[    0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
[    0.000000] No local APIC present
[    0.000000] APIC: disable apic facility
[    0.000000] APIC: switched to apic NOOP
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
[    0.000000] Booting paravirtualized kernel on Xen
[    0.000000] Xen version: 4.1-unstable (preserve-AD)
[    0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
nr_cpu_ids:6 nr_node_ids:1
[    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
r8192 d23744 u114688
[    0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.
Total pages: 519022
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:
root=UUID=e4e223aa-080e-421b-98ac-ab0b3e0e57e5 ro   swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Placing 64MB software IO TLB between ffff880079400000 -
ffff88007d400000
[    0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
[    0.000000] Memory: 1959892k/2105344k available (4398k kernel code,
448k absent, 145004k reserved, 6327k data, 756k init)
[    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=6, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU-based detection of stalled CPUs is disabled.
[    0.000000] NR_IRQS:4352 nr_irqs:320 16
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [hvc0] enabled
[    0.000000] installing Xen timer for CPU 0
[    0.000000] Detected 2660.064 MHz processor.
[    0.000999] Calibrating delay loop (skipped), value calculated
using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
[    0.000999] pid_max: default: 32768 minimum: 301
[    0.000999] Security Framework initialized
[    0.000999] SELinux:  Initializing.
[    0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.001365] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.001631] Mount-cache hash table entries: 256
[    0.002177] Initializing cgroup subsys ns
[    0.002191] ns_cgroup deprecated: consider using the
''clone_children'' flag without the ns_cgroup.
[    0.002203] Initializing cgroup subsys cpuacct
[    0.002214] Initializing cgroup subsys freezer
[    0.002342] CPU: Unsupported number of siblings 16
[    0.002348] Performance Events: unsupported p6 CPU model 26 no PMU
driver, software events only.
[    0.004120] cpu 0 spinlock event irq 318
[    0.004755] NMI watchdog failed to create perf event on cpu0:
fffffffffffffffe
[    0.004999] installing Xen timer for CPU 1
[    0.005056] cpu 1 spinlock event irq 312
[    0.005093] SMP alternatives: switching to SMP code
[    0.000999] CPU: Unsupported number of siblings 16
[    0.006155] NMI watchdog failed to create perf event on cpu1:
fffffffffffffffe
[    0.006471] installing Xen timer for CPU 2
[    0.006582] cpu 2 spinlock event irq 306
[    0.000999] CPU: Unsupported number of siblings 16
[    0.006810] NMI watchdog failed to create perf event on cpu2:
fffffffffffffffe
[    0.006999] installing Xen timer for CPU 3
[    0.007033] cpu 3 spinlock event irq 300
[    0.000999] CPU: Unsupported number of siblings 16
[    0.007353] NMI watchdog failed to create perf event on cpu3:
fffffffffffffffe
[    0.007587] installing Xen timer for CPU 4
[    0.007626] cpu 4 spinlock event irq 294
[    0.000999] CPU: Unsupported number of siblings 16
[    0.007854] NMI watchdog failed to create perf event on cpu4:
fffffffffffffffe
[    0.008032] installing Xen timer for CPU 5
[    0.008072] cpu 5 spinlock event irq 288
[    0.000999] CPU: Unsupported number of siblings 16
[    0.008391] NMI watchdog failed to create perf event on cpu5:
fffffffffffffffe
[    0.008402] Brought up 6 CPUs
[    0.009156] kworker/u:0 used greatest stack depth: 5696 bytes left
[    0.011444] Grant table initialized
[    0.030860] Time: 165:165:165  Date: 165/165/65
[    0.031124] NET: Registered protocol family 16
[    0.036023] PCI: setting up Xen PCI frontend stub
[    0.037149] kworker/u:0 used greatest stack depth: 5528 bytes left
[    0.063224] bio: create slab <bio-0> at 0
[    0.064258] ACPI: Interpreter disabled.
[    0.065066] xen_balloon: Initialising balloon driver.
[    0.066380] vgaarb: loaded
[    0.067219] usbcore: registered new interface driver usbfs
[    0.068076] usbcore: registered new interface driver hub
[    0.069160] PCI: System does not support PCI
[    0.070062] PCI: System does not support PCI
[    0.071229] NetLabel: Initializing
[    0.071229] NetLabel:  domain hash size = 128
[    0.071229] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.071271] NetLabel:  unlabeled traffic allowed by default
[    0.072589] Switching to clocksource xen
[    0.086952] pnp: PnP ACPI: disabled
[    0.110928] NET: Registered protocol family 2
[    0.111286] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.112870] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[    0.113959] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.114223] TCP: Hash tables configured (established 262144 bind 65536)
[    0.114235] TCP reno registered
[    0.114348] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[    0.114454] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[    0.115033] NET: Registered protocol family 1
[    0.139648] RPC: Registered udp transport module.
[    0.139656] RPC: Registered tcp transport module.
[    0.139662] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.139898] Trying to unpack rootfs image as initramfs...
[    0.179608] Freeing initrd memory: 18516k freed
[    0.267847] DMA-API: preallocated 32768 debug entries
[    0.267860] DMA-API: debugging enabled by kernel config
[    0.270280] platform rtc_cmos: registered platform RTC device (no
PNP device found)
[    0.284949] Machine check injector initialized
[    0.294390] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
[    0.294463] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
[    0.294516] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
[    0.294584] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
[    0.294680] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
[    0.294768] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
[    0.295370] microcode: Microcode Update Driver: v2.00
<tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    0.295387] Scanning for low memory corruption every 60 seconds
[    0.297945] audit: initializing netlink socket (disabled)
[    0.298034] type=2000 audit(1289404539.466:1): initialized
[    0.371706] VFS: Disk quotas dquot_6.5.2
[    0.372818] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.378043] NTFS driver 2.1.29 [Flags: R/W].
[    0.379946] msgmni has been set to 3864
[    0.384845] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[    0.384862] io scheduler noop registered
[    0.384871] io scheduler deadline registered
[    0.385499] io scheduler cfq registered (default)
[    0.387281] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.401306] pcifront pci-0: Installing PCI frontend
[    0.402048] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[    0.411423] pcifront pci-0: claiming resource 0000:00:00.0/0
[    0.411423] pcifront pci-0: claiming resource 0000:00:00.0/2
[    0.411423] pcifront pci-0: claiming resource 0000:00:00.0/3
[    0.411423] pcifront pci-0: claiming resource 0000:00:00.0/4
[    0.411423] pcifront pci-0: claiming resource 0000:00:01.0/0
[    0.411423] pcifront pci-0: claiming resource 0000:00:01.0/2
[    0.411423] pcifront pci-0: claiming resource 0000:00:01.0/3
[    0.411423] pcifront pci-0: claiming resource 0000:00:01.0/4
[    0.411423] pcifront pci-0: claiming resource 0000:00:02.0/0
[    0.411423] pcifront pci-0: claiming resource 0000:00:02.0/2
[    0.411423] pcifront pci-0: claiming resource 0000:00:02.0/3
[    0.411423] pcifront pci-0: claiming resource 0000:00:02.0/4
[    0.411423] pcifront pci-0: claiming resource 0000:00:03.0/0
[    0.411423] pcifront pci-0: claiming resource 0000:00:03.0/2
[    0.411423] pcifront pci-0: claiming resource 0000:00:03.0/3
[    0.411423] pcifront pci-0: claiming resource 0000:00:03.0/4
[    0.718875] Non-volatile memory driver v1.3
[    0.718890] Linux agpgart interface v0.103
[    0.720286] [drm] Initialized drm 1.1.0 20060810
[    0.720307] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.816097] brd: module loaded
[    0.829618] loop: module loaded
[    0.830819] Fixed MDIO Bus: probed
[    0.833373] ehci_hcd: USB 2.0 ''Enhanced'' Host Controller
(EHCI) Driver
[    0.833977] ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI)
Driver
[    0.836341] usbcore: registered new interface driver usblp
[    0.836850] usbcore: registered new interface driver libusual
[    0.838519] PNP: No PS/2 controller found. Probing ports directly.
[    0.839365] i8042.c: No controller found.
[    0.840021] mice: PS/2 mouse device common for all mice
[    0.842821] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[    0.843646] rtc_cmos: probe of rtc_cmos failed with error -38
[    0.845545] cpuidle: using governor ladder
[    0.845558] cpuidle: using governor menu
[    0.845645] Netfilter messages via NETLINK v0.30.
[    0.845732] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    0.847527] ctnetlink v0.93: registering with nfnetlink.
[    0.849858] ip_tables: (C) 2000-2006 Netfilter Core Team
[    0.850277] TCP cubic registered
[    0.850288] Initializing XFRM netlink socket
[    0.852995] NET: Registered protocol family 10
[    0.856904] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    0.857303] IPv6 over IPv4 tunneling driver
[    0.861148] NET: Registered protocol family 17
[    0.861376] Registering the dns_resolver key type
[    0.863749] registered taskstats version 1
[    0.864020] kmemleak: Kernel memory leak detector initialized
[    0.864035] kmemleak: Automatic memory scanning thread started
[    0.866079] XENBUS: Device with no driver: device/vbd/51712
[    0.866079] XENBUS: Device with no driver: device/vif/0
[    0.866079] XENBUS: Device with no driver: device/console/0
[    0.866079]   Magic number: 1:252:3141

[   13.380040] Freeing unused kernel memory: 756k freed
[   13.380152] Write protecting the kernel read-only data: 10240k
[   13.390107] Freeing unused kernel memory: 1724k freed
[   13.391580] Freeing unused kernel memory: 1992k freed


Loading, please wait...
[   13.484058] exe used greatest stack depth: 5360 bytes left
[   13.510024] udevd (1111): /proc/1111/oom_adj is deprecated, please
use /proc/1111/oom_score_adj instead.
Begin: Loading essential drivers... ...
[   13.533300] blkid used greatest stack depth: 5088 bytes left
[   13.564959] blkfront: xvda: barriers disabled
[   13.576723]  xvda: xvda1 xvda2 < xvda5 >
[   13.639471] blkid used greatest stack depth: 4752 bytes left
[   13.662716] Initialising Xen virtual ethernet driver.
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
[   13.927847] EXT4-fs (xvda1): mounted filesystem with ordered data
mode. Opts: (null)
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
[   14.608840] hostname used greatest stack depth: 4464 bytes left

fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
Superblock last write time is in the future.
        (by less than a day, probably due to the hardware clock being
incorrectly set).  Fix? yes

/dev/xvda1: clean, 55961/288576 files, 289520/1153280 blocks
 * Enable login on console                                               [ OK ]
 * Check for hostname changes...                                         [ OK ]
   ...done.
 * Setting sensors limits
         No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
                                                                         [ OK ]
 * Initializing random number generator...                               [ OK ]
 * Setting up X server socket directory /tmp/.X11-unix...                [ OK ]
 * Not starting fancontrol; run pwmconfig first.
 * Starting internet superserver inetd                                   [ OK ]
Starting openntpd: ntpd.
 * Starting network management services:
          * Starting the system activity data collector sadc

Cannot open /var/log/sysstat/sa10: No such file or directory
                                                                         [fail]
 * Setting core file location...                                         [ OK ]
 * Restoring resolver state...                                           [ OK ]
 * Checking tomcat configuration
   ...done.
 * Starting Tomcat servlet engine tomcat6
         Using CATALINA_BASE:   /var/lib/tomcat6
Using CATALINA_HOME:   /usr/share/tomcat6
Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
Using JRE_HOME:        /usr/lib/jvm/java-6-openjdk
Using CLASSPATH:       /usr/share/tomcat6/bin/bootstrap.jar
                                                                         [ OK ]
collectd has been disabled - see /etc/default/collectd.
 * Recording successful boot for GRUB
   ...done.
 * Running local boot scripts (/etc/rc.local)                            [ OK ]
mountall: Plymouth command failed
mountall: Disconnected from Plymouth

6.0.16-01
kaan-04-dpm login: [  144.548118] BUG: unable to handle kernel paging
request at ffff88007733d508
[  144.548145] IP: [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
[  144.548167] PGD 1a04067 PUD 3dd7067 PMD 3f91067 PTE 801000007733d065
[  144.548194] Oops: 0003 [#1] SMP
[  144.548211] last sysfs file: /sys/kernel/uevent_seqnum
[  144.548222] CPU 3
[  144.548228] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
xen_netfront xen_blkfront
[  144.548345]
[  144.548354] Pid: 3938, comm: sh Not tainted
2.6.37-rc1-kernel.org-domu-6.0.nopvhvm.bedge-debug #1 /
[  144.548367] RIP: e030:[<ffffffff810355f3>]  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
[  144.548395] RAX: 800000005adc6167 RBX: ffff880077403b80 RCX: 800000005adc6167
[  144.548408] RDX: ffff88007733d508 RSI: 00007f52324a19d0 RDI: ffff880077403b80
[  144.548420] RBP: ffff880077cabc08 R08: 0000000000000001 R09: ffffea0001a13568
[  144.548431] R10: 0000000000000100 R11: ffffffff00000001 R12: 00007f52324a19d0
[  144.548443] R13: 0000000000000001 R14: 0000000000000000 R15: ffff880077403b80
[  144.548495] FS:  00007f52324a1700(0000) GS:ffff88007ff94000(0000)
knlGS:0000000000000000
[  144.548509] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  144.548520] CR2: ffff88007733d508 CR3: 00000000783ad000 CR4: 0000000000002660
[  144.548531] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  144.548543] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  144.548556] Process sh (pid: 3938, threadinfo ffff880077caa000,
task ffff880077a615f0)
[  144.548568] Stack:
[  144.548575]  0000000000000100 ffffea0001a13568 f000000000000000
0000000000000000
[  144.548602]  ffff88007733d508 ffffea0001a13568 ffff880077cabca8
ffffffff810e2200
[  144.548630]  ffff880077cabc38 ffffffff81007c09 ffffea0001a13568
0000000000000001
[  144.548657] Call Trace:
[  144.548672]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
[  144.548688]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[  144.548702]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
[  144.548716]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
[  144.548732]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
[  144.548747]  [<ffffffff8100ba0b>] ? xen_hypervisor_callback+0x1b/0x20
[  144.548762]  [<ffffffff8144384a>] ? error_exit+0x2a/0x60
[  144.548776]  [<ffffffff8144339d>] ? retint_restore_args+0x5/0x6
[  144.548792]  [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1006
[  144.548806]  [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1006
[  144.548821]  [<ffffffff81443615>] page_fault+0x25/0x30
[  144.548837]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
[  144.548851]  [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
[  144.548865]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
[  144.548875] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49
89 f4 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45
85 ed 74 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df
e8 b2
[  144.549091] RIP  [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
[  144.549091]  RSP <ffff880077cabbd8>
[  144.549091] CR2: ffff88007733d508
[  144.549091] ---[ end trace e2156ffb7bd19b0d ]---


This time it also generated a stack dump on dom0

BUG: scheduling while atomic: xenwatch/12/0x00000002
Modules linked in: xt_physdev ipmi_msghandler ipv6 osa_mfgdom0
xen_evtchn tun bridge stp serio_raw joydev llc ioatdma dca usb_storage
e1000e
Pid: 12, comm: xenwatch Not tainted
2.6.32.24-pvops-stable-dom0-5.3.bedge-debug #1
Call Trace:
 [<ffffffff810424d7>] __schedule_bug+0x5c/0x61
 [<ffffffff814964ac>] schedule+0xea/0x84d
 [<ffffffff8100f5dd>] ? xen_force_evtchn_callback+0xd/0xf
 [<ffffffff8100fdff>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff814989e5>] ? _spin_unlock_irqrestore+0x1d/0x40
 [<ffffffff812b76bb>] read_reply+0x9c/0x13a
 [<ffffffff8105ea57>] ? autoremove_wake_function+0x0/0x34
 [<ffffffff812b78bf>] xs_talkv+0xab/0x16d
 [<ffffffff81010be2>] ? xen_spin_lock+0xc7/0xda
 [<ffffffff812b7c9a>] unregister_xenbus_watch+0xdb/0x1e7
 [<ffffffff812bd303>] pciback_xenbus_remove+0x33/0x75
 [<ffffffff812b8bba>] xenbus_dev_remove+0x44/0x5b
 [<ffffffff812fead4>] __device_release_driver+0x70/0xc3
 [<ffffffff812febf5>] device_release_driver+0x1e/0x2b
 [<ffffffff812fe17c>] bus_remove_device+0x96/0xa8
 [<ffffffff812fc8ca>] device_del+0x135/0x1a2
 [<ffffffff812fc948>] device_unregister+0x11/0x1e
 [<ffffffff812b89c8>] xenbus_dev_changed+0x92/0x186
 [<ffffffff812b8f0a>] backend_changed+0x16/0x18
 [<ffffffff812b75e4>] xenwatch_thread+0x123/0x15e
 [<ffffffff8105ea57>] ? autoremove_wake_function+0x0/0x34
 [<ffffffff812b74c1>] ? xenwatch_thread+0x0/0x15e
 [<ffffffff8105e7c5>] kthread+0x7a/0x82
 [<ffffffff81014bda>] child_rip+0xa/0x20
 [<ffffffff81013d61>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff81014521>] ? retint_restore_args+0x5/0x6
 [<ffffffff81014bd0>] ? child_rip+0x0/0x20
pciback 0000:01:00.0: free-ing dynamically allocated virtual
configuration space fields
pciback 0000:01:00.0: resetting virtual configuration space
pciback 0000:01:00.1: free-ing dynamically allocated virtual
configuration space fields
pciback 0000:01:00.1: resetting virtual configuration space
pciback 0000:04:00.0: free-ing dynamically allocated virtual
configuration space fields
pciback 0000:04:00.0: resetting virtual configuration space
pciback 0000:04:00.1: free-ing dynamically allocated virtual
configuration space fields
pciback 0000:04:00.1: resetting virtual configuration space

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Nov-11 06:13 UTC

head link

[Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Wed, Nov 10, 2010 at 2:30 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
> On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com>
wrote:
> >
> > This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable
build
> 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
> >
> > The domU console output hung up prior to spitting out a prompt post
root
> login. I logged into the domU via ssh and ran an ''ls'' of
an NFS mounted dir
> and
> > this immediately came out on the domU console:
> >
> > [  480.467525] BUG: unable to handle kernel paging request at
> ffff880077e9f030
> > [  480.467547] IP: [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> > [  480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE
8010000077e9f065
> > [  480.467587] Oops: 0003 [#1] SMP
> > [  480.467601] last sysfs file: /sys/kernel/uevent_seqnum
> > [  480.467611] CPU 3
> > [  480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
> xen_blkfront
> > [  480.467676]
> > [  480.467684] Pid: 3957, comm: zsh Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> > [  480.467695] RIP: e030:[<ffffffff810355f3>] 
[<ffffffff810355f3>]
> ptep_set_access_flags+0x27/0x4d
> > [  480.467710] RSP: e02b:ffff880077ebfbd8  EFLAGS: 00010202
> > [  480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX:
> 800000004d686167
> > [  480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI:
> ffff8800763ddb80
> > [  480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09:
> ffffea0001a3b2d8
> > [  480.467748] R10: 0000000000000100 R11: 0000000000000000 R12:
> 00007f5cb26069d0
> > [  480.467759] R13: 0000000000000001 R14: 0000000000000000 R15:
> ffff8800763ddb80
> > [  480.467772] FS:  00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
> knlGS:0000000000000000
> > [  480.467783] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> > [  480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4:
> 0000000000002660
> > [  480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> > [  480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> > [  480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000,
task
> ffff880078b80000)
> > [  480.467832] Stack:
> > [  480.467838]  0000000000000100 ffffea0001a3b2d8 7000000000000000
> 0000000000000000
> > [  480.467862]  ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
> ffffffff810e2200
> > [  480.467886]  ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
> 0000000000000001
> > [  480.467910] Call Trace:
> > [  480.467921]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
> > [  480.467934]  [<ffffffff81007c09>] ?
__spin_time_accum+0x21/0x37
> > [  480.467946]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> > [  480.467958]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
> > [  480.467972]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
> > [  480.467984]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> > [  480.467996]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> > [  480.468009]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> > [  480.468020]  [<ffffffff8100728f>] ?
xen_restore_fl_direct_end+0x0/0x1
> > [  480.468034]  [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
> > [  480.468045]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> > [  480.468059]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> > [  480.468072]  [<ffffffff81443615>] page_fault+0x25/0x30
> > [  480.468085]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
> > [  480.468096]  [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
> > [  480.468108]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
> > [  480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49
89
> f4 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74
> 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
> > [  480.468323] RIP  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> > [  480.468323]  RSP <ffff880077ebfbd8>
> > [  480.468323] CR2: ffff880077e9f030
> > [  480.468323] ---[ end trace 6d1623321ecd50ac ]---
> >
> > The domU config is attached as config.2.6.37_domU_debug
> >
> >
> > -Bruce
>
> Here''s another attempt from the same config with the full domU
boot logs:
>
> Started domain dpm (id=4)
>                         [    0.000000] Initializing cgroup subsys cpuset
> [    0.000000] Initializing cgroup subsys cpu
> [    0.000000] Linux version
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug (root@opal) (gcc version
> 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Tue Nov 9 20:26:03 PST 2010
> [    0.000000] Command line:
> root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro   swiotlb=force
> noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
> ip=:127.0.255.255::::eth0:dhcp
> [    0.000000] ACPI in unprivileged domain disabled
> [    0.000000] released 0 pages of unused memory
> [    0.000000] BIOS-provided physical RAM map:
> [    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
> [    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
> [    0.000000]  Xen: 0000000000100000 - 0000000080800000 (usable)
> [    0.000000] NX (Execute Disable) protection: active
> [    0.000000] DMI not present or invalid.
> [    0.000000] No AGP bridge found
> [    0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
> [    0.000000] Scanning 0 areas for low memory corruption
> [    0.000000] init_memory_mapping: 0000000000000000-0000000080800000
> [    0.000000] RAMDISK: 01d95000 - 02fa9000
> [    0.000000] No NUMA configuration found
> [    0.000000] Faking a node at 0000000000000000-0000000080800000
> [    0.000000] Initmem setup node 0 0000000000000000-0000000080800000
> [    0.000000]   NODE_DATA [000000007fffb000 - 000000007fffffff]
> [    0.000000] Zone PFN ranges:
> [    0.000000]   DMA      0x00000010 -> 0x00001000
> [    0.000000]   DMA32    0x00001000 -> 0x00100000
> [    0.000000]   Normal   empty
> [    0.000000] Movable zone start PFN for each node
> [    0.000000] early_node_map[2] active PFN ranges
> [    0.000000]     0: 0x00000010 -> 0x000000a0
> [    0.000000]     0: 0x00000100 -> 0x00080800
> [    0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
> [    0.000000] No local APIC present
> [    0.000000] APIC: disable apic facility
> [    0.000000] APIC: switched to apic NOOP
> [    0.000000] PM: Registered nosave memory: 00000000000a0000 -
> 0000000000100000
> [    0.000000] Allocating PCI resources starting at 80800000 (gap:
> 80800000:7f800000)
> [    0.000000] Booting paravirtualized kernel on Xen
> [    0.000000] Xen version: 4.1-unstable (preserve-AD)
> [    0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
> nr_cpu_ids:6 nr_node_ids:1
> [    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
> r8192 d23744 u114688
> [    0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
> [    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
> [    0.000000] Built 1 zonelists in Node order, mobility grouping on.
> Total pages: 519022
> [    0.000000] Policy zone: DMA32
> [    0.000000] Kernel command line:
> root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro   swiotlb=force
> noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
> ip=:127.0.255.255::::eth0:dhcp
> [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
> [    0.000000] Placing 64MB software IO TLB between ffff880079400000 -
> ffff88007d400000
> [    0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
> [    0.000000] Memory: 1959896k/2105344k available (4398k kernel code,
> 448k absent, 145000k reserved, 6327k data, 756k init)
> [    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
> CPUs=6, Nodes=1
> [    0.000000] Hierarchical RCU implementation.
> [    0.000000]  RCU-based detection of stalled CPUs is disabled.
> [    0.000000] NR_IRQS:4352 nr_irqs:320 16
> [    0.000000] Console: colour dummy device 80x25
> [    0.000000] console [tty0] enabled
> [    0.000000] console [hvc0] enabled
> [    0.000000] installing Xen timer for CPU 0
> [    0.000000] Detected 2660.064 MHz processor.
> [    0.000999] Calibrating delay loop (skipped), value calculated
> using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
> [    0.000999] pid_max: default: 32768 minimum: 301
> [    0.000999] Security Framework initialized
> [    0.000999] SELinux:  Initializing.
> [    0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152
> bytes)
> [    0.001408] Inode-cache hash table entries: 131072 (order: 8, 1048576
> bytes)
> [    0.001635] Mount-cache hash table entries: 256
> [    0.002195] Initializing cgroup subsys ns
> [    0.002208] ns_cgroup deprecated: consider using the
> ''clone_children'' flag without the ns_cgroup.
> [    0.002220] Initializing cgroup subsys cpuacct
> [    0.002231] Initializing cgroup subsys freezer
> [    0.002356] CPU: Unsupported number of siblings 16
> [    0.002362] Performance Events: unsupported p6 CPU model 26 no PMU
> driver, software events only.
> [    0.002483] SMP alternatives: switching to UP code
> [    0.004119] cpu 0 spinlock event irq 318
> [    0.004765] NMI watchdog failed to create perf event on cpu0:
> fffffffffffffffe
> [    0.005050] installing Xen timer for CPU 1
> [    0.005157] cpu 1 spinlock event irq 312
> [    0.005194] SMP alternatives: switching to SMP code
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.006158] NMI watchdog failed to create perf event on cpu1:
> fffffffffffffffe
> [    0.006474] installing Xen timer for CPU 2
> [    0.006585] cpu 2 spinlock event irq 306
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.006816] NMI watchdog failed to create perf event on cpu2:
> fffffffffffffffe
> [    0.006999] installing Xen timer for CPU 3
> [    0.007037] cpu 3 spinlock event irq 300
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.007300] NMI watchdog failed to create perf event on cpu3:
> fffffffffffffffe
> [    0.007534] installing Xen timer for CPU 4
> [    0.007636] cpu 4 spinlock event irq 294
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.007874] NMI watchdog failed to create perf event on cpu4:
> fffffffffffffffe
> [    0.008041] installing Xen timer for CPU 5
> [    0.008083] cpu 5 spinlock event irq 288
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.008344] NMI watchdog failed to create perf event on cpu5:
> fffffffffffffffe
> [    0.008356] Brought up 6 CPUs
> [    0.009181] kworker/u:0 used greatest stack depth: 5696 bytes left
> [    0.012051] Grant table initialized
> [    0.030978] Time: 165:165:165  Date: 165/165/65
> [    0.031229] NET: Registered protocol family 16
> [    0.036672] PCI: setting up Xen PCI frontend stub
> [    0.037225] kworker/u:0 used greatest stack depth: 5528 bytes left
> [    0.074302] bio: create slab <bio-0> at 0
> [    0.075385] ACPI: Interpreter disabled.
> [    0.077056] xen_balloon: Initialising balloon driver.
> [    0.079323] vgaarb: loaded
> [    0.081270] usbcore: registered new interface driver usbfs
> [    0.081283] usbcore: registered new interface driver hub
> [    0.081283] usbcore: registered new device driver usb
> [    0.083183] PCI: System does not support PCI
> [    0.084007] PCI: System does not support PCI
> [    0.085236] NetLabel: Initializing
> [    0.085236] NetLabel:  domain hash size = 128
> [    0.085236] NetLabel:  protocols = UNLABELED CIPSOv4
> [    0.085313] NetLabel:  unlabeled traffic allowed by default
> [    0.086544] Switching to clocksource xen
> [    0.106256] pnp: PnP ACPI: disabled
> [    0.129799] NET: Registered protocol family 2
> [    0.130174] IP route cache hash table entries: 65536 (order: 7, 524288
> bytes)
> [    0.131882] TCP established hash table entries: 262144 (order: 10,
> 4194304 bytes)
> [    0.132891] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
> [    0.133179] TCP: Hash tables configured (established 262144 bind 65536)
> [    0.133191] TCP reno registered
> [    0.133301] UDP hash table entries: 1024 (order: 3, 32768 bytes)
> [    0.133407] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
> [    0.133914] NET: Registered protocol family 1
> [    0.134582] RPC: Registered udp transport module.
> [    0.134590] RPC: Registered tcp transport module.
> [    0.134595] RPC: Registered tcp NFSv4.1 backchannel transport module.
> [    0.134892] Trying to unpack rootfs image as initramfs...
> [    0.178824] Freeing initrd memory: 18512k freed
> [    0.268798] DMA-API: preallocated 32768 debug entries
> [    0.268810] DMA-API: debugging enabled by kernel config
> [    0.271181] platform rtc_cmos: registered platform RTC device (no
> PNP device found)
> [    0.285410] Machine check injector initialized
> [    0.294732] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
> [    0.294803] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
> [    0.294893] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
> [    0.294944] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
> [    0.294998] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
> [    0.295043] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
> [    0.295590] microcode: Microcode Update Driver: v2.00
> <tigran@aivazian.fsnet.co.uk>, Peter Oruba
> [    0.295607] Scanning for low memory corruption every 60 seconds
> [    0.298813] audit: initializing netlink socket (disabled)
> [    0.298940] type=2000 audit(1289403082.795:1): initialized
> [    0.333762] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [    0.376985] VFS: Disk quotas dquot_6.5.2
> [    0.377853] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> [    0.383121] NTFS driver 2.1.29 [Flags: R/W].
> [    0.385014] msgmni has been set to 3864
> [    0.389953] Block layer SCSI generic (bsg) driver version 0.4
> loaded (major 253)
> [    0.389970] io scheduler noop registered
> [    0.389979] io scheduler deadline registered
> [    0.390666] io scheduler cfq registered (default)
> [    0.392486] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> [    0.406238] pcifront pci-0: Installing PCI frontend
> [    0.406967] pcifront pci-0: Creating PCI Frontend Bus 0000:00
> [    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/0
> [    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/2
> [    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/3
> [    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/4
> [    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/0
> [    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/2
> [    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/3
> [    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/4
> [    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/0
> [    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/2
> [    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/3
> [    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/4
> [    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/0
> [    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/2
> [    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/3
> [    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/4
> [    0.731775] Non-volatile memory driver v1.3
> [    0.731790] Linux agpgart interface v0.103
> [    0.733731] [drm] Initialized drm 1.1.0 20060810
> [    0.733752] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [    0.827202] brd: module loaded
> [    0.840788] loop: module loaded
> [    0.841965] Fixed MDIO Bus: probed
> [    0.844025] ehci_hcd: USB 2.0 ''Enhanced'' Host
Controller (EHCI) Driver
> [    0.844651] ohci_hcd: USB 1.1 ''Open'' Host Controller
(OHCI) Driver
> [    0.845194] uhci_hcd: USB Universal Host Controller Interface driver
> [    0.846985] usbcore: registered new interface driver usblp
> [    0.847763] usbcore: registered new interface driver libusual
> [    0.849433] PNP: No PS/2 controller found. Probing ports directly.
> [    0.850279] i8042.c: No controller found.
> [    0.850883] mice: PS/2 mouse device common for all mice
> [    0.853446] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
> [    0.854066] rtc_cmos: probe of rtc_cmos failed with error -38
> [    0.855928] cpuidle: using governor ladder
> [    0.855941] cpuidle: using governor menu
> [    0.856029] Netfilter messages via NETLINK v0.30.
> [    0.856136] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
> [    0.857888] ctnetlink v0.93: registering with nfnetlink.
> [    0.860285] ip_tables: (C) 2000-2006 Netfilter Core Team
> [    0.860677] TCP cubic registered
> [    0.860687] Initializing XFRM netlink socket
> [    0.863286] NET: Registered protocol family 10
> [    0.867216] ip6_tables: (C) 2000-2006 Netfilter Core Team
> [    0.867612] IPv6 over IPv4 tunneling driver
> [    0.871326] NET: Registered protocol family 17
> [    0.871566] Registering the dns_resolver key type
> [    0.873668] registered taskstats version 1
> [    0.873912] kmemleak: Kernel memory leak detector initialized
> [    0.873928] kmemleak: Automatic memory scanning thread started
> [    0.875057] XENBUS: Device with no driver: device/vbd/51712
> [    0.875057] XENBUS: Device with no driver: device/vif/0
> [    0.875057] XENBUS: Device with no driver: device/console/0
> [    0.875057]   Magic number: 1:252:3141
> [   13.388958] Freeing unused kernel memory: 756k freed
> [   13.389190] Write protecting the kernel read-only data: 10240k
> [   13.398774] Freeing unused kernel memory: 1724k freed
> [   13.400663] Freeing unused kernel memory: 1992k freed
> [   13.494573] all_generic_ide used greatest stack depth: 5472 bytes left
> [   13.519388] udevd (1111): /proc/1111/oom_adj is deprecated, please
> use /proc/1111/oom_score_adj instead.
> Begin: Loading essential drivers... ...
> [   13.559435] blkid used greatest stack depth: 5408 bytes left
> [   13.568391] blkfront: xvda: barriers disabled
> [   13.574762] blkid used greatest stack depth: 5264 bytes left
> [   13.580813]  xvda: xvda1 xvda2 < xvda5 >
> [   13.645562] modprobe used greatest stack depth: 5088 bytes left
> [   13.653160] Initialising Xen virtual ethernet driver.
> [   13.659540] blkid used greatest stack depth: 4960 bytes left
> [   13.735467] blkid used greatest stack depth: 4752 bytes left
> Done.
> Begin: Running /scripts/init-premount ...
> Done.
> Begin: Mounting root file system... ...
> Begin: Running /scripts/local-top ...
> Done.
> Begin: Running /scripts/local-premount ...
> Done.
> [   13.903774] EXT4-fs (xvda1): INFO: recovery required on readonly
> filesystem
> [   13.903799] EXT4-fs (xvda1): write access will be enabled during
> recovery
> [   14.033719] EXT4-fs (xvda1): recovery complete
> [   14.034966] EXT4-fs (xvda1): mounted filesystem with ordered data
> mode. Opts: (null)
> [   14.035510] exe used greatest stack depth: 4112 bytes left
> Begin: Running /scripts/local-bottom ...
> Done.
> Done.
> Begin: Running /scripts/init-bottom ...
> Done.
> init: ureadahead main process (1214) terminated with status 5
> fsck from util-linux-ng 2.17.2
> e2fsck 1.41.11 (14-Mar-2010)
> /dev/xvda1: clean, 56314/288576 files, 287743/1153280 blocks
>  * Enable login on console                                               [
> OK ]
>  * Check for hostname changes...                                         [
> OK ]
>   ...done.
>  * Setting sensors limits
>         No sensors found!
> Make sure you loaded all the kernel drivers you need.
> Try sensors-detect to find out which these are.
> No sensors found!
> Make sure you loaded all the kernel drivers you need.
> Try sensors-detect to find out which these are.
>                                                                         [
> OK ]
>  * Initializing random number generator...                               [
> OK ]
>  * Setting up X server socket directory /tmp/.X11-unix...                [
> OK ]
>  * Setting up ICE socket directory /tmp/.ICE-unix...                     [
> OK ]
>  * Not starting fancontrol; run pwmconfig first.
>  * Starting internet superserver inetd                                   [
> OK ]
> Starting openntpd: ntpd.
>  * Starting network management services:
>  * Setting core file location...                                         [
> OK ]
>  * Restoring resolver state...                                           [
> OK ]
>  * Checking tomcat configuration
>   ...done.
>  * Starting Tomcat servlet engine tomcat6
>         Using CATALINA_BASE:   /var/lib/tomcat6
> Using CATALINA_HOME:   /usr/share/tomcat6
> Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
> Using JRE_HOME:        /usr/lib/jvm/java-6-openjdk
> Using CLASSPATH:       /usr/share/tomcat6/bin/bootstrap.jar
>                                                                         [
> OK ]
> collectd has been disabled - see /etc/default/collectd.
>  * Recording successful boot for GRUB
>   ...done.
>  * Running local boot scripts (/etc/rc.local)                            [
> OK ]
> mountall: Plymouth command failed
> mountall: Disconnected from Plymouth
>
> 6.0.16-01
> kaan-04-dpm login: root
> Password:
>

This same mmu.c:1831 is also generated by the stable/xen-pcifront-fixes
branch.
Identical symptoms.

-Bruce


> [   42.646538] ------------[ cut here ]------------
> [   42.646556] kernel BUG at arch/x86/xen/mmu.c:1831!
> [   42.646566] invalid opcode: 0000 [#1] SMP
> [   42.646582] last sysfs file: /sys/kernel/uevent_seqnum
> [   42.646592] CPU 0
> [   42.646598] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
> xen_netfront xen_blkfront
> [   42.646665]
> [   42.646674] Pid: 3916, comm: lsb_release Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [   42.646688] RIP: e030:[<ffffffff81004bce>] 
[<ffffffff81004bce>]
> pin_pagetable_pfn+0x31/0x37
> [   42.646711] RSP: e02b:ffff88007595bce8  EFLAGS: 00010282
> [   42.646721] RAX: 00000000ffffffea RBX: 000000000007751e RCX:
> 0000000000000001
> [   42.646732] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
> 00000000deadbeef
> [   42.646744] RBP: ffff88007595bd08 R08: 00003ffffffff000 R09:
> ffff880000000000
> [   42.646756] R10: 00000000deadbeef R11: dead000000200200 R12:
> 0000000000000003
> [   42.646768] R13: ffff880075c90150 R14: ffff8800781d9398 R15:
> ffff880075161d40
> [   42.646784] FS:  00007fee63b01700(0000) GS:ffff88007ff40000(0000)
> knlGS:0000000000000000
> [   42.646797] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [   42.646807] CR2: 00007faac54182b8 CR3: 00000000771dd000 CR4:
> 0000000000002660
> [   42.646819] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [   42.646832] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [   42.646844] Process lsb_release (pid: 3916, threadinfo
> ffff88007595a000, task ffff880075161d40)
> [   42.646857] Stack:
> [   42.646864]  ffff880000000000 00000000000484f0 ffff8800777c6d80
> 000000000007751e
> [   42.646892]  ffff88007595bd28 ffffffff810066a8 000000000007751e
> ffff8800777c6d80
> [   42.646919]  ffff88007595bd38 ffffffff810066db ffff88007595bd78
> ffffffff810e048f
> [   42.646946] Call Trace:
> [   42.646959]  [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
> [   42.646973]  [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
> [   42.646989]  [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
> [   42.647003]  [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
> [   42.647017]  [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
> [   42.647033]  [<ffffffff8100eb84>] ?
> arch_get_unmapped_area_topdown+0x4a/0x28a
> [   42.647052]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [   42.647059]  [<ffffffff8102b78a>] ?
pvclock_clocksource_read+0x4b/0xb4
> [   42.647059]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [   42.647059]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [   42.647059]  [<ffffffff81443615>] page_fault+0x25/0x30
> [   42.647059] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
> e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff
> 85 c0 74 04 <0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db
48
> 8b 3d
> [   42.647059] RIP  [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
> [   42.647059]  RSP <ffff88007595bce8>
> [   42.647059] ---[ end trace 3fbbffee414a5166 ]---
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-11 11:01 UTC

head link

Re: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

> The domU console output hung up prior to spitting out a prompt post root
login.
>  I logged into the domU via sshUbuntu 10.10 HVM image has been used to test 2.6.37-rc1 pvops DomU
File /etc/init/hvc0 has been prepared in HVM mode to login from command line.

PV Guest config

root@ServerLLX:/home/boris/ubuntu# cat u10.cfg
name="Ubuntu10PV"
memory = 2048
kernel = "/home/boris/ubuntu/vmlinuz-2.6.37-rc1"
ramdisk = "/home/boris/ubuntu/initrd.img-2.6.37-rc1"
disk = [''phy:/dev/sda7,xvda,w'']
vif  = [''bridge=br0'']
extra = "root=/dev/xvda1 ro console=hvc0"
# vfb =  [''type=vnc,vncunused=1'']

root@ServerLLX:/home/boris/ubuntu# xm create -c  u10.cfg
Using config file "./u10.cfg".
Started domain Ubuntu10PV (id=4)
                                Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.37-rc1 (root@ServerLLX) (gcc version 4.4.3 (Ubuntu
4.4.3-4ubuntu5) ) #1 SMP Fri Nov 5 19:29:09 MSK 2010
Command line: root=/dev/xvda1 ro console=hvc0
ACPI in unprivileged domain disabled
released 0 pages of unused memory
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 00000000000a0000 (usable)
 Xen: 00000000000a0000 - 0000000000100000 (reserved)
 Xen: 0000000000100000 - 0000000080800000 (usable)
NX (Execute Disable) protection: active
DMI not present or invalid.
No AGP bridge found
last_pfn = 0x80800 max_arch_pfn = 0x400000000
init_memory_mapping: 0000000000000000-0000000080800000
RAMDISK: 01e98000 - 0e9fa000
No NUMA configuration found
Faking a node at 0000000000000000-0000000080800000
Initmem setup node 0 0000000000000000-0000000080800000
  NODE_DATA [000000007ffec000 - 000000007fffffff]
Zone PFN ranges:
  DMA      0x00000010 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   empty
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
    0: 0x00000010 -> 0x000000a0
    0: 0x00000100 -> 0x00080800
SMP: Allowing 1 CPUs, 0 hotplug CPUs
No local APIC present
APIC: disable apic facility
APIC: switched to apic NOOP
PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
Allocating PCI resources starting at 80800000 (gap: 80800000:7f800000)
Booting paravirtualized kernel on Xen
Xen version: 4.0.1 (preserve-AD)
setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:1 nr_node_ids:1
PERCPU: Embedded 28 pages/cpu @ffff88007ffbd000 s82752 r8192 d23744 u114688
pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
pcpu-alloc: [0] 0 
Built 1 zonelists in Node order, mobility grouping on.  Total pages: 519022
Policy zone: DMA32
Kernel command line: root=/dev/xvda1 ro console=hvc0
PID hash table entries: 4096 (order: 3, 32768 bytes)
Checking aperture...
No AGP bridge found
Memory: 1835560k/2105344k available (4462k kernel code, 448k absent, 269336k
reserved, 6442k data, 1564k init)
Hierarchical RCU implementation.
    RCU-based detection of stalled CPUs is disabled.
NR_IRQS:16640 nr_irqs:256 16
Console: colour dummy device 80x25
console [tty0] enabled
console [hvc0] enabled
allocated 22282240 bytes of page_cgroup
please try ''cgroup_disable=memory'' option if you
don''t want memory cgroups
installing Xen timer for CPU 0
Detected 2833.068 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency..
5666.13 BogoMIPS (lpj=2833068)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux:  Initializing.
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Mount-cache hash table entries: 256
Initializing cgroup subsys debug
Initializing cgroup subsys ns
ns_cgroup deprecated: consider using the ''clone_children'' flag
without the ns_cgroup.
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
CPU: Unsupported number of siblings 4
Performance Events: unsupported p6 CPU model 23 no PMU driver, software events
only.
SMP alternatives: switching to UP code
Freeing SMP alternatives: 20k freed
cpu 0 spinlock event irq 254
Brought up 1 CPUs
Grant table initialized
regulator: core version 0.5
regulator: dummy: 
Time: 165:165:165  Date: 165/165/65
NET: Registered protocol family 16
PCI: setting up Xen PCI frontend stub
bio: create slab <bio-0> at 0
ACPI: Interpreter disabled.
xen_balloon: Initialising balloon driver.
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: System does not support PCI
PCI: System does not support PCI
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
Switching to clocksource xen
pnp: PnP ACPI: disabled
NET: Registered protocol family 2
IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
UDP hash table entries: 1024 (order: 3, 32768 bytes)
UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
NET: Registered protocol family 1
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 208264k freed
platform rtc_cmos: registered platform RTC device (no PNP device found)
audit: initializing netlink socket (disabled)
type=2000 audit(1289472066.106:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 3991
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
pci-stub: invalid id string ""
Event-channel device installed.
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
brd: module loaded
loop: module loaded
Fixed MDIO Bus: probed
Initialising Xen virtual ethernet driver.
ehci_hcd: USB 2.0 ''Enhanced'' Host Controller (EHCI) Driver
ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
PNP: No PS/2 controller found. Probing ports directly.
i8042.c: No controller found.
mice: PS/2 mouse device common for all mice
rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
rtc_cmos: probe of rtc_cmos failed with error -38
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com
cpuidle: using governor ladder
cpuidle: using governor menu
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
IPVS: Registered protocols (TCP, UDP, AH, ESP)
IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
IPVS: ipvs loaded.
ip_tables: (C) 2000-2006 Netfilter Core Team
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
Bridge firewalling registered
Ebtables v2.0 registered
Registering the dns_resolver key type
registered taskstats version 1
IMA: No TPM chip found, activating TPM-bypass!
blkfront: xvda: barriers disabled
 xvda: xvda1 xvda2
XENBUS: Device with no driver: device/console/0
  Magic number: 1:252:3141
drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
Initalizing network drop monitor service
Freeing unused kernel memory: 1564k freed
Write protecting the kernel read-only data: 10240k
Freeing unused kernel memory: 1664k freed
Freeing unused kernel memory: 1904k freed
Loading, please wait...
mount: mounting none on /dev failed: No such device
udevd (63): /proc/63/oom_adj is deprecated, please use /proc/63/oom_score_adj
instead.
Begin: Loading essential drivers... ...
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.

Ubuntu 10.10 boris-HVM-domU hvc0

boris-HVM-domU login: root
Password: 
Linux boris-HVM-domU 2.6.37-rc1 #1 SMP Fri Nov 5 19:29:09 MSK 2010 x86_64
GNU/Linux
Ubuntu 10.10

Welcome to Ubuntu!
 * Documentation:  https://help.ubuntu.com/

root@boris-HVM-domU:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1             12G  3.9G  7.1G  36% /
none                 1001M  116K 1001M   1% /dev
none                 1001M     0 1001M   0% /dev/shm
none                 1001M   84K 1001M   1% /var/run
none                 1001M     0 1001M   0% /var/lock

Boris.

--- On Wed, 11/10/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel
paging request
To: xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Wednesday, November 10, 2010, 5:15 PM

This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build 22241
and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.

The domU console output hung up prior to spitting out a prompt post root login.
I logged into the domU via ssh and ran an ''ls'' of an NFS
mounted dir and

this immediately came out on the domU console:

[  480.467525] BUG: unable to handle kernel paging request at ffff880077e9f030
[  480.467547] IP: [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
[  480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065

[  480.467587] Oops: 0003 [#1] SMP 
[  480.467601] last sysfs file: /sys/kernel/uevent_seqnum
[  480.467611] CPU 3 
[  480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront

[  480.467676] 
[  480.467684] Pid: 3957, comm: zsh Not tainted
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
[  480.467695] RIP: e030:[<ffffffff810355f3>]  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d

[  480.467710] RSP: e02b:ffff880077ebfbd8  EFLAGS: 00010202
[  480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX: 800000004d686167
[  480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI: ffff8800763ddb80

[  480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09: ffffea0001a3b2d8
[  480.467748] R10: 0000000000000100 R11: 0000000000000000 R12: 00007f5cb26069d0
[  480.467759] R13: 0000000000000001 R14: 0000000000000000 R15: ffff8800763ddb80

[  480.467772] FS:  00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
knlGS:0000000000000000
[  480.467783] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4: 0000000000002660

[  480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task
ffff880078b80000)

[  480.467832] Stack:
[  480.467838]  0000000000000100 ffffea0001a3b2d8 7000000000000000
0000000000000000
[  480.467862]  ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
ffffffff810e2200
[  480.467886]  ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
0000000000000001

[  480.467910] Call Trace:
[  480.467921]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
[  480.467934]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[  480.467946]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd

[  480.467958]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
[  480.467972]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
[  480.467984]  [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf

[  480.467996]  [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
[  480.468009]  [<ffffffff810072a2>] ? check_events+0x12/0x20
[  480.468020]  [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1

[  480.468034]  [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
[  480.468045]  [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
[  480.468059]  [<ffffffff810072a2>] ? check_events+0x12/0x20

[  480.468072]  [<ffffffff81443615>] page_fault+0x25/0x30
[  480.468085]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
[  480.468096]  [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
[  480.468108]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80

[  480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89 f4 53 48
89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18 <48>
89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2

[  480.468323] RIP  [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
[  480.468323]  RSP <ffff880077ebfbd8>
[  480.468323] CR2: ffff880077e9f030
[  480.468323] ---[ end trace 6d1623321ecd50ac ]---

The domU config is attached as config.2.6.37_domU_debug

-Bruce

-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-11 12:01 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

This is a mainline 2.6.37-rc1 pvops domU on top of a xen 4.0.1 and
xen/stable-2.6.32.x 2.6.32.25 pvops dom0.
Now prepare NFS remote share , mount it at DomU on /mnt/tmp
after normal login  and run `ls -l /mnt/tmp`
***********
Report 
***********
root@ServerLLX:/home/boris/ubuntu# xm create -c  u10.cfg
Using config file "./u10.cfg".
Started domain Ubuntu10PV (id=9)
                                Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.37-rc1 (root@ServerLLX) (gcc version 4.4.3 (Ubuntu
4.4.3-4ubuntu5) ) #1 SMP Fri Nov 5 19:29:09 MSK 2010
Command line: root=/dev/xvda1 ro console=hvc0
ACPI in unprivileged domain disabled
released 0 pages of unused memory
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 00000000000a0000 (usable)
 Xen: 00000000000a0000 - 0000000000100000 (reserved)
 Xen: 0000000000100000 - 0000000080800000 (usable)
NX (Execute Disable) protection: active
DMI not present or invalid.
No AGP bridge found
last_pfn = 0x80800 max_arch_pfn = 0x400000000
init_memory_mapping: 0000000000000000-0000000080800000
RAMDISK: 01e98000 - 0e9fa000
No NUMA configuration found
Faking a node at 0000000000000000-0000000080800000
Initmem setup node 0 0000000000000000-0000000080800000
  NODE_DATA [000000007ffec000 - 000000007fffffff]
Zone PFN ranges:
  DMA      0x00000010 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   empty
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
    0: 0x00000010 -> 0x000000a0
    0: 0x00000100 -> 0x00080800
SMP: Allowing 1 CPUs, 0 hotplug CPUs
No local APIC present
APIC: disable apic facility
APIC: switched to apic NOOP
PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
Allocating PCI resources starting at 80800000 (gap: 80800000:7f800000)
Booting paravirtualized kernel on Xen
Xen version: 4.0.1 (preserve-AD)
setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:1 nr_node_ids:1
PERCPU: Embedded 28 pages/cpu @ffff88007ffbd000 s82752 r8192 d23744 u114688
pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
pcpu-alloc: [0] 0 
Built 1 zonelists in Node order, mobility grouping on.  Total pages: 519022
Policy zone: DMA32
Kernel command line: root=/dev/xvda1 ro console=hvc0
PID hash table entries: 4096 (order: 3, 32768 bytes)
Checking aperture...
No AGP bridge found
Memory: 1835560k/2105344k available (4462k kernel code, 448k absent, 269336k
reserved, 6442k data, 1564k init)
Hierarchical RCU implementation.
    RCU-based detection of stalled CPUs is disabled.
NR_IRQS:16640 nr_irqs:256 16
Console: colour dummy device 80x25
console [tty0] enabled
console [hvc0] enabled
allocated 22282240 bytes of page_cgroup
please try ''cgroup_disable=memory'' option if you
don''t want memory cgroups
installing Xen timer for CPU 0
Detected 2833.068 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency..
5666.13 BogoMIPS (lpj=2833068)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux:  Initializing.
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Mount-cache hash table entries: 256
Initializing cgroup subsys debug
Initializing cgroup subsys ns
ns_cgroup deprecated: consider using the ''clone_children'' flag
without the ns_cgroup.
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
CPU: Unsupported number of siblings 4
Performance Events: unsupported p6 CPU model 23 no PMU driver, software events
only.
SMP alternatives: switching to UP code
Freeing SMP alternatives: 20k freed
cpu 0 spinlock event irq 254
Brought up 1 CPUs
Grant table initialized
regulator: core version 0.5
regulator: dummy: 
Time: 165:165:165  Date: 165/165/65
NET: Registered protocol family 16
PCI: setting up Xen PCI frontend stub
bio: create slab <bio-0> at 0
ACPI: Interpreter disabled.
xen_balloon: Initialising balloon driver.
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: System does not support PCI
PCI: System does not support PCI
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
Switching to clocksource xen
pnp: PnP ACPI: disabled
NET: Registered protocol family 2
IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
UDP hash table entries: 1024 (order: 3, 32768 bytes)
UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
NET: Registered protocol family 1
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 208264k freed
platform rtc_cmos: registered platform RTC device (no PNP device found)
audit: initializing netlink socket (disabled)
type=2000 audit(1289476232.768:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 3991
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
pci-stub: invalid id string ""
Event-channel device installed.
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
brd: module loaded
loop: module loaded
Fixed MDIO Bus: probed
Initialising Xen virtual ethernet driver.
ehci_hcd: USB 2.0 ''Enhanced'' Host Controller (EHCI) Driver
ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
PNP: No PS/2 controller found. Probing ports directly.
i8042.c: No controller found.
mice: PS/2 mouse device common for all mice
rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
rtc_cmos: probe of rtc_cmos failed with error -38
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com
cpuidle: using governor ladder
cpuidle: using governor menu
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
IPVS: Registered protocols (TCP, UDP, AH, ESP)
IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
IPVS: ipvs loaded.
ip_tables: (C) 2000-2006 Netfilter Core Team
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
Bridge firewalling registered
Ebtables v2.0 registered
Registering the dns_resolver key type
registered taskstats version 1
IMA: No TPM chip found, activating TPM-bypass!
blkfront: xvda: barriers disabled
 xvda: xvda1 xvda2
XENBUS: Device with no driver: device/console/0
  Magic number: 1:252:3141
drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
Initalizing network drop monitor service
Freeing unused kernel memory: 1564k freed
Write protecting the kernel read-only data: 10240k
Freeing unused kernel memory: 1664k freed
Freeing unused kernel memory: 1904k freed
Loading, please wait...
mount: mounting none on /dev failed: No such device
udevd (63): /proc/63/oom_adj is deprecated, please use /proc/63/oom_score_adj
instead.
Begin: Loading essential drivers... ...
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
EXT4-fs (xvda1): INFO: recovery required on readonly filesystem
EXT4-fs (xvda1): write access will be enabled during recovery
EXT4-fs (xvda1): orphan cleanup on readonly fs
EXT4-fs (xvda1): 5 orphan inodes deleted
EXT4-fs (xvda1): recovery complete
EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
init: ureadahead main process (157) terminated with status 5
fsck from util-linux-ng 2.17.2
/dev/xvda1: clean, 188758/760368 files, 1083747/3041280 blocks
 * Setting sensors limits                                                [ OK ] 
 * Exporting directories for NFS kernel daemon...                        [ OK ] 
 * Starting NFS kernel daemon                                            [ OK ] 
speech-dispatcher disabled; edit /etc/default/speech-dispatcher
 * PulseAudio configured for per-user sessions
saned disabled; edit /etc/default/saned
 * Enabling additional executable binary formats binfmt-support          [ OK ] 
fsck from util-linux-ng 2.17.2
/dev/xvda1: clean, 188758/760368 files, 1083747/3041280 blocks
 * Setting sensors limits                                                [ OK ] 
 * Exporting directories for NFS kernel daemon...                        [ OK ] 
 * Starting NFS kernel daemon                                            [ OK ] 
speech-dispatcher disabled; edit /etc/default/speech-dispatcher
 * PulseAudio configured for per-user sessions
saned disabled; edit /etc/default/saned
 * Enabling additional executable binary formats binfmt-support          [ OK ] 
 * Checking battery state...                                             [ OK ] 

Ubuntu 10.10 boris-HVM-domU hvc0

boris-HVM-domU login: root
Password: 
Last login: Thu Nov 11 12:29:06 CET 2010 on hvc0
Linux boris-HVM-domU 2.6.37-rc1 #1 SMP Fri Nov 5 19:29:09 MSK 2010 x86_64
GNU/Linux
Ubuntu 10.10

Welcome to Ubuntu!
 * Documentation:  https://help.ubuntu.com/

root@boris-HVM-domU:~# mount 192.168.1.11:/home/boris /mnt/tmp
root@boris-HVM-domU:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1             12G  4.0G  6.9G  37% /
none                 1001M  116K 1001M   1% /dev
none                 1001M     0 1001M   0% /dev/shm
none                 1001M  100K 1001M   1% /var/run
none                 1001M     0 1001M   0% /var/lock
192.168.1.11:/home/boris
                       44G   21G   21G  50% /mnt/tmp

*************************************************
RUNNING ''ls -l'' against NFS mounted share
*************************************************

root@boris-HVM-domU:~# ls -l /mnt/tmp
total 693924
-rw-r--r-- 1 root root 709871616 2009-06-07 18:51 osol-0906-x86.iso
drwxr-xr-x 2 root root      4096 2009-06-07 19:06 solaris
root@boris-HVM-domU:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3e:48:b3:3c  
          inet addr:192.168.1.9  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fe48:b33c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:300 errors:0 dropped:0 overruns:0 frame:0
          TX packets:94 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:30391 (30.3 KB)  TX bytes:13562 (13.5 KB)
          Interrupt:246 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:96 errors:0 dropped:0 overruns:0 frame:0
          TX packets:96 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:8996 (8.9 KB)  TX bytes:8996 (8.9 KB)

root@boris-HVM-domU:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1             12G  4.0G  6.9G  37% /
none                 1001M  116K 1001M   1% /dev
none                 1001M     0 1001M   0% /dev/shm
none                 1001M  100K 1001M   1% /var/run
none                 1001M     0 1001M   0% /var/lock
192.168.1.11:/home/boris
                       44G   21G   21G  51% /mnt/tmp

Boris.



--- On Wed, 11/10/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel
paging request
To: xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Wednesday, November 10, 2010, 5:30 PM

On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com>
wrote:>
> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build
22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
>
> The domU console output hung up prior to spitting out a prompt post root
login. I logged into the domU via ssh and ran an ''ls'' of an
NFS mounted dir and
> this immediately came out on the domU console:
>
> [  480.467525] BUG: unable to handle kernel paging request at
ffff880077e9f030
> [  480.467547] IP: [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> [  480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
> [  480.467587] Oops: 0003 [#1] SMP
> [  480.467601] last sysfs file: /sys/kernel/uevent_seqnum
> [  480.467611] CPU 3
> [  480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
> [  480.467676]
> [  480.467684] Pid: 3957, comm: zsh Not tainted
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [  480.467695] RIP: e030:[<ffffffff810355f3>] 
[<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
> [  480.467710] RSP: e02b:ffff880077ebfbd8  EFLAGS: 00010202
> [  480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX:
800000004d686167
> [  480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI:
ffff8800763ddb80
> [  480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09:
ffffea0001a3b2d8
> [  480.467748] R10: 0000000000000100 R11: 0000000000000000 R12:
00007f5cb26069d0
> [  480.467759] R13: 0000000000000001 R14: 0000000000000000 R15:
ffff8800763ddb80
> [  480.467772] FS:  00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
knlGS:0000000000000000
> [  480.467783] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4:
0000000000002660
> [  480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
> [  480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
> [  480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task
ffff880078b80000)
> [  480.467832] Stack:
> [  480.467838]  0000000000000100 ffffea0001a3b2d8 7000000000000000
0000000000000000
> [  480.467862]  ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
ffffffff810e2200
> [  480.467886]  ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
0000000000000001
> [  480.467910] Call Trace:
> [  480.467921]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
> [  480.467934]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [  480.467946]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [  480.467958]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
> [  480.467972]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [  480.467984]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.467996]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.468009]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> [  480.468020]  [<ffffffff8100728f>] ?
xen_restore_fl_direct_end+0x0/0x1
> [  480.468034]  [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
> [  480.468045]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.468059]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> [  480.468072]  [<ffffffff81443615>] page_fault+0x25/0x30
> [  480.468085]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
> [  480.468096]  [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
> [  480.468108]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
> [  480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89 f4
53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18
<48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
> [  480.468323] RIP  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> [  480.468323]  RSP <ffff880077ebfbd8>
> [  480.468323] CR2: ffff880077e9f030
> [  480.468323] ---[ end trace 6d1623321ecd50ac ]---
>
> The domU config is attached as config.2.6.37_domU_debug
>
>
> -Bruce
Here''s another attempt from the same config with the full domU boot
logs:

Started domain dpm (id=4)
                         [    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug (root@opal) (gcc version
4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Tue Nov 9 20:26:03 PST 2010
[    0.000000] Command line:
root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro   swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] released 0 pages of unused memory
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000080800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[    0.000000] Scanning 0 areas for low memory corruption
[    0.000000] init_memory_mapping: 0000000000000000-0000000080800000
[    0.000000] RAMDISK: 01d95000 - 02fa9000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-0000000080800000
[    0.000000] Initmem setup node 0 0000000000000000-0000000080800000
[    0.000000]   NODE_DATA [000000007fffb000 - 000000007fffffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x000000a0
[    0.000000]     0: 0x00000100 -> 0x00080800
[    0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
[    0.000000] No local APIC present
[    0.000000] APIC: disable apic facility
[    0.000000] APIC: switched to apic NOOP
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
[    0.000000] Allocating PCI resources starting at 80800000 (gap:
80800000:7f800000)
[    0.000000] Booting paravirtualized kernel on Xen
[    0.000000] Xen version: 4.1-unstable (preserve-AD)
[    0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
nr_cpu_ids:6 nr_node_ids:1
[    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
r8192 d23744 u114688
[    0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.
Total pages: 519022
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:
root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro   swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Placing 64MB software IO TLB between ffff880079400000 -
ffff88007d400000
[    0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
[    0.000000] Memory: 1959896k/2105344k available (4398k kernel code,
448k absent, 145000k reserved, 6327k data, 756k init)
[    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=6, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU-based detection of stalled CPUs is disabled.
[    0.000000] NR_IRQS:4352 nr_irqs:320 16
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [hvc0] enabled
[    0.000000] installing Xen timer for CPU 0
[    0.000000] Detected 2660.064 MHz processor.
[    0.000999] Calibrating delay loop (skipped), value calculated
using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
[    0.000999] pid_max: default: 32768 minimum: 301
[    0.000999] Security Framework initialized
[    0.000999] SELinux:  Initializing.
[    0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.001408] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.001635] Mount-cache hash table entries: 256
[    0.002195] Initializing cgroup subsys ns
[    0.002208] ns_cgroup deprecated: consider using the
''clone_children'' flag without the ns_cgroup.
[    0.002220] Initializing cgroup subsys cpuacct
[    0.002231] Initializing cgroup subsys freezer
[    0.002356] CPU: Unsupported number of siblings 16
[    0.002362] Performance Events: unsupported p6 CPU model 26 no PMU
driver, software events only.
[    0.002483] SMP alternatives: switching to UP code
[    0.004119] cpu 0 spinlock event irq 318
[    0.004765] NMI watchdog failed to create perf event on cpu0:
fffffffffffffffe
[    0.005050] installing Xen timer for CPU 1
[    0.005157] cpu 1 spinlock event irq 312
[    0.005194] SMP alternatives: switching to SMP code
[    0.000999] CPU: Unsupported number of siblings 16
[    0.006158] NMI watchdog failed to create perf event on cpu1:
fffffffffffffffe
[    0.006474] installing Xen timer for CPU 2
[    0.006585] cpu 2 spinlock event irq 306
[    0.000999] CPU: Unsupported number of siblings 16
[    0.006816] NMI watchdog failed to create perf event on cpu2:
fffffffffffffffe
[    0.006999] installing Xen timer for CPU 3
[    0.007037] cpu 3 spinlock event irq 300
[    0.000999] CPU: Unsupported number of siblings 16
[    0.007300] NMI watchdog failed to create perf event on cpu3:
fffffffffffffffe
[    0.007534] installing Xen timer for CPU 4
[    0.007636] cpu 4 spinlock event irq 294
[    0.000999] CPU: Unsupported number of siblings 16
[    0.007874] NMI watchdog failed to create perf event on cpu4:
fffffffffffffffe
[    0.008041] installing Xen timer for CPU 5
[    0.008083] cpu 5 spinlock event irq 288
[    0.000999] CPU: Unsupported number of siblings 16
[    0.008344] NMI watchdog failed to create perf event on cpu5:
fffffffffffffffe
[    0.008356] Brought up 6 CPUs
[    0.009181] kworker/u:0 used greatest stack depth: 5696 bytes left
[    0.012051] Grant table initialized
[    0.030978] Time: 165:165:165  Date: 165/165/65
[    0.031229] NET: Registered protocol family 16
[    0.036672] PCI: setting up Xen PCI frontend stub
[    0.037225] kworker/u:0 used greatest stack depth: 5528 bytes left
[    0.074302] bio: create slab <bio-0> at 0
[    0.075385] ACPI: Interpreter disabled.
[    0.077056] xen_balloon: Initialising balloon driver.
[    0.079323] vgaarb: loaded
[    0.081270] usbcore: registered new interface driver usbfs
[    0.081283] usbcore: registered new interface driver hub
[    0.081283] usbcore: registered new device driver usb
[    0.083183] PCI: System does not support PCI
[    0.084007] PCI: System does not support PCI
[    0.085236] NetLabel: Initializing
[    0.085236] NetLabel:  domain hash size = 128
[    0.085236] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.085313] NetLabel:  unlabeled traffic allowed by default
[    0.086544] Switching to clocksource xen
[    0.106256] pnp: PnP ACPI: disabled
[    0.129799] NET: Registered protocol family 2
[    0.130174] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.131882] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[    0.132891] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.133179] TCP: Hash tables configured (established 262144 bind 65536)
[    0.133191] TCP reno registered
[    0.133301] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[    0.133407] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[    0.133914] NET: Registered protocol family 1
[    0.134582] RPC: Registered udp transport module.
[    0.134590] RPC: Registered tcp transport module.
[    0.134595] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.134892] Trying to unpack rootfs image as initramfs...
[    0.178824] Freeing initrd memory: 18512k freed
[    0.268798] DMA-API: preallocated 32768 debug entries
[    0.268810] DMA-API: debugging enabled by kernel config
[    0.271181] platform rtc_cmos: registered platform RTC device (no
PNP device found)
[    0.285410] Machine check injector initialized
[    0.294732] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
[    0.294803] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
[    0.294893] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
[    0.294944] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
[    0.294998] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
[    0.295043] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
[    0.295590] microcode: Microcode Update Driver: v2.00
<tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    0.295607] Scanning for low memory corruption every 60 seconds
[    0.298813] audit: initializing netlink socket (disabled)
[    0.298940] type=2000 audit(1289403082.795:1): initialized
[    0.333762] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.376985] VFS: Disk quotas dquot_6.5.2
[    0.377853] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.383121] NTFS driver 2.1.29 [Flags: R/W].
[    0.385014] msgmni has been set to 3864
[    0.389953] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[    0.389970] io scheduler noop registered
[    0.389979] io scheduler deadline registered
[    0.390666] io scheduler cfq registered (default)
[    0.392486] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.406238] pcifront pci-0: Installing PCI frontend
[    0.406967] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/0
[    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/2
[    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/3
[    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/4
[    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/0
[    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/2
[    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/3
[    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/4
[    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/0
[    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/2
[    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/3
[    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/4
[    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/0
[    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/2
[    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/3
[    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/4
[    0.731775] Non-volatile memory driver v1.3
[    0.731790] Linux agpgart interface v0.103
[    0.733731] [drm] Initialized drm 1.1.0 20060810
[    0.733752] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.827202] brd: module loaded
[    0.840788] loop: module loaded
[    0.841965] Fixed MDIO Bus: probed
[    0.844025] ehci_hcd: USB 2.0 ''Enhanced'' Host Controller
(EHCI) Driver
[    0.844651] ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI)
Driver
[    0.845194] uhci_hcd: USB Universal Host Controller Interface driver
[    0.846985] usbcore: registered new interface driver usblp
[    0.847763] usbcore: registered new interface driver libusual
[    0.849433] PNP: No PS/2 controller found. Probing ports directly.
[    0.850279] i8042.c: No controller found.
[    0.850883] mice: PS/2 mouse device common for all mice
[    0.853446] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[    0.854066] rtc_cmos: probe of rtc_cmos failed with error -38
[    0.855928] cpuidle: using governor ladder
[    0.855941] cpuidle: using governor menu
[    0.856029] Netfilter messages via NETLINK v0.30.
[    0.856136] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    0.857888] ctnetlink v0.93: registering with nfnetlink.
[    0.860285] ip_tables: (C) 2000-2006 Netfilter Core Team
[    0.860677] TCP cubic registered
[    0.860687] Initializing XFRM netlink socket
[    0.863286] NET: Registered protocol family 10
[    0.867216] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    0.867612] IPv6 over IPv4 tunneling driver
[    0.871326] NET: Registered protocol family 17
[    0.871566] Registering the dns_resolver key type
[    0.873668] registered taskstats version 1
[    0.873912] kmemleak: Kernel memory leak detector initialized
[    0.873928] kmemleak: Automatic memory scanning thread started
[    0.875057] XENBUS: Device with no driver: device/vbd/51712
[    0.875057] XENBUS: Device with no driver: device/vif/0
[    0.875057] XENBUS: Device with no driver: device/console/0
[    0.875057]   Magic number: 1:252:3141
[   13.388958] Freeing unused kernel memory: 756k freed
[   13.389190] Write protecting the kernel read-only data: 10240k
[   13.398774] Freeing unused kernel memory: 1724k freed
[   13.400663] Freeing unused kernel memory: 1992k freed
[   13.494573] all_generic_ide used greatest stack depth: 5472 bytes left
[   13.519388] udevd (1111): /proc/1111/oom_adj is deprecated, please
use /proc/1111/oom_score_adj instead.
Begin: Loading essential drivers... ...
[   13.559435] blkid used greatest stack depth: 5408 bytes left
[   13.568391] blkfront: xvda: barriers disabled
[   13.574762] blkid used greatest stack depth: 5264 bytes left
[   13.580813]  xvda: xvda1 xvda2 < xvda5 >
[   13.645562] modprobe used greatest stack depth: 5088 bytes left
[   13.653160] Initialising Xen virtual ethernet driver.
[   13.659540] blkid used greatest stack depth: 4960 bytes left
[   13.735467] blkid used greatest stack depth: 4752 bytes left
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
[   13.903774] EXT4-fs (xvda1): INFO: recovery required on readonly filesystem
[   13.903799] EXT4-fs (xvda1): write access will be enabled during recovery
[   14.033719] EXT4-fs (xvda1): recovery complete
[   14.034966] EXT4-fs (xvda1): mounted filesystem with ordered data
mode. Opts: (null)
[   14.035510] exe used greatest stack depth: 4112 bytes left
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
init: ureadahead main process (1214) terminated with status 5
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
/dev/xvda1: clean, 56314/288576 files, 287743/1153280 blocks
 * Enable login on console                                               [ OK ]
 * Check for hostname changes...                                         [ OK ]
   ...done.
 * Setting sensors limits
         No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
                                                                         [ OK ]
 * Initializing random number generator...                               [ OK ]
 * Setting up X server socket directory /tmp/.X11-unix...                [ OK ]
 * Setting up ICE socket directory /tmp/.ICE-unix...                     [ OK ]
 * Not starting fancontrol; run pwmconfig first.
 * Starting internet superserver inetd                                   [ OK ]
Starting openntpd: ntpd.
 * Starting network management services:
 * Setting core file location...                                         [ OK ]
 * Restoring resolver state...                                           [ OK ]
 * Checking tomcat configuration
   ...done.
 * Starting Tomcat servlet engine tomcat6
         Using CATALINA_BASE:   /var/lib/tomcat6
Using CATALINA_HOME:   /usr/share/tomcat6
Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
Using JRE_HOME:        /usr/lib/jvm/java-6-openjdk
Using CLASSPATH:       /usr/share/tomcat6/bin/bootstrap.jar
                                                                         [ OK ]
collectd has been disabled - see /etc/default/collectd.
 * Recording successful boot for GRUB
   ...done.
 * Running local boot scripts (/etc/rc.local)                            [ OK ]
mountall: Plymouth command failed
mountall: Disconnected from Plymouth

6.0.16-01
kaan-04-dpm login: root
Password:
[   42.646538] ------------[ cut here ]------------
[   42.646556] kernel BUG at arch/x86/xen/mmu.c:1831!
[   42.646566] invalid opcode: 0000 [#1] SMP
[   42.646582] last sysfs file: /sys/kernel/uevent_seqnum
[   42.646592] CPU 0
[   42.646598] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
xen_netfront xen_blkfront
[   42.646665]
[   42.646674] Pid: 3916, comm: lsb_release Not tainted
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
[   42.646688] RIP: e030:[<ffffffff81004bce>]  [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[   42.646711] RSP: e02b:ffff88007595bce8  EFLAGS: 00010282
[   42.646721] RAX: 00000000ffffffea RBX: 000000000007751e RCX: 0000000000000001
[   42.646732] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
[   42.646744] RBP: ffff88007595bd08 R08: 00003ffffffff000 R09: ffff880000000000
[   42.646756] R10: 00000000deadbeef R11: dead000000200200 R12: 0000000000000003
[   42.646768] R13: ffff880075c90150 R14: ffff8800781d9398 R15: ffff880075161d40
[   42.646784] FS:  00007fee63b01700(0000) GS:ffff88007ff40000(0000)
knlGS:0000000000000000
[   42.646797] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[   42.646807] CR2: 00007faac54182b8 CR3: 00000000771dd000 CR4: 0000000000002660
[   42.646819] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   42.646832] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[   42.646844] Process lsb_release (pid: 3916, threadinfo
ffff88007595a000, task ffff880075161d40)
[   42.646857] Stack:
[   42.646864]  ffff880000000000 00000000000484f0 ffff8800777c6d80
000000000007751e
[   42.646892]  ffff88007595bd28 ffffffff810066a8 000000000007751e
ffff8800777c6d80
[   42.646919]  ffff88007595bd38 ffffffff810066db ffff88007595bd78
ffffffff810e048f
[   42.646946] Call Trace:
[   42.646959]  [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[   42.646973]  [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[   42.646989]  [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[   42.647003]  [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
[   42.647017]  [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
[   42.647033]  [<ffffffff8100eb84>] ?
arch_get_unmapped_area_topdown+0x4a/0x28a
[   42.647052]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
[   42.647059]  [<ffffffff8102b78a>] ? pvclock_clocksource_read+0x4b/0xb4
[   42.647059]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[   42.647059]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
[   42.647059]  [<ffffffff81443615>] page_fault+0x25/0x30
[   42.647059] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff
85 c0 74 04 <0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48
8b 3d
[   42.647059] RIP  [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[   42.647059]  RSP <ffff88007595bce8>
[   42.647059] ---[ end trace 3fbbffee414a5166 ]---

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-11 12:08 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Now reproduced :-

root@boris-HVM-domU:~# scp  1.log 192.168.1.12:/home/boris/ubuntu
BUG: unable to handle kernel paging request at ffff88000e676930
IP: [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
PGD 1a04067 PUD 1a08067 PMD ee73067 PTE 801000000e676065
Oops: 0003 [#1] SMP 
last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
CPU 0 
Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl auth_rpcgss
sunrpc exportfs ipv6 lp parport

Pid: 3534, comm: scp Not tainted 2.6.37-rc1 #1 /
RIP: e030:[<ffffffff810069c3>]  [<ffffffff810069c3>]
xen_set_pmd+0x24/0x44
RSP: e02b:ffff88006e697b18  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff88000e676930 RCX: ffffc00000000f98
RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff88000e676930
RBP: ffff88006e697b28 R08: 00007fff9d120000 R09: ffff88007ffc80a0
R10: 0000000000000000 R11: ffff88006e697ae8 R12: 0000000000000000
R13: ffff88007ffcb140 R14: ffff88000e676930 R15: ffff88006f08d7f8
FS:  00007fa525896700(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff88000e676930 CR3: 0000000077b0f000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process scp (pid: 3534, threadinfo ffff88006e696000, task ffff88007a32c2c0)
Stack:
 00007fa524e00000 00007fa525b37000 ffff88006e697be8 ffffffff810dc4a4
 ffff88007a26ac88 ffff88006e690298 ffff88007a26ac78 00007fa524a00000
 00007f8000000000 00007fff80000000 00007fa525b36fff 00007fa525b36fff
Call Trace:
 [<ffffffff810dc4a4>] free_pgd_range+0x213/0x397
 [<ffffffff810dc6cc>] free_pgtables+0xa4/0xc7
 [<ffffffff810dfd82>] exit_mmap+0xf2/0x12a
 [<ffffffff8104a7c5>] mmput+0x5e/0xd1
 [<ffffffff8110694b>] flush_old_exec+0x526/0x600
 [<ffffffff8113e006>] load_elf_binary+0x2dc/0x1752
 [<ffffffff8113b687>] ? load_misc_binary+0x6e/0x367
 [<ffffffff811dcee4>] ? ima_must_measure+0x21/0x2b
 [<ffffffff811dc7d6>] ? process_measurement+0x56/0xe4
 [<ffffffff81106195>] search_binary_handler+0xbb/0x24d
 [<ffffffff8113dd2a>] ? load_elf_binary+0x0/0x1752
 [<ffffffff8110785d>] do_execve+0x1c1/0x2ae
 [<ffffffff810119e5>] sys_execve+0x3e/0x55
 [<ffffffff8100af9c>] stub_execve+0x6c/0xc0
Code: 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 b5 e4 ff ff
48 89 df ff 05 3e 55 c3 00 e8 d5 dd ff ff 84 c0 75 05 <4c> 89 23 eb 16 e8
99 e4 ff ff 4c 89 e6 48 89 df ff 05 23 55 c3
RIP  [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
 RSP <ffff88006e697b18>
CR2: ffff88000e676930
---[ end trace 073da74f4ce02315 ]---
lost connection


--- On Thu, 11/11/10, Boris Derzhavets <bderzhavets@yahoo.com> wrote:

From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>, "Bruce Edge"
<bruce.edge@gmail.com>
Date: Thursday, November 11, 2010, 7:01 AM

This is a mainline 2.6.37-rc1 pvops domU on top of a xen 4.0.1 and
xen/stable-2.6.32.x 2.6.32.25 pvops dom0.
Now prepare NFS remote share , mount it at DomU on /mnt/tmp
after normal login  and run `ls -l /mnt/tmp`
***********
Report 
***********
root@ServerLLX:/home/boris/ubuntu# xm create -c  u10.cfg
Using config file "./u10.cfg".
Started domain Ubuntu10PV (id=9)
                                Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.37-rc1 (root@ServerLLX) (gcc version 4.4.3 (Ubuntu
4.4.3-4ubuntu5) ) #1 SMP Fri Nov 5 19:29:09 MSK 2010
Command line: root=/dev/xvda1 ro console=hvc0
ACPI in unprivileged domain
 disabled
released 0 pages of unused memory
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 00000000000a0000 (usable)
 Xen: 00000000000a0000 - 0000000000100000 (reserved)
 Xen: 0000000000100000 - 0000000080800000 (usable)
NX (Execute Disable) protection: active
DMI not present or invalid.
No AGP bridge found
last_pfn = 0x80800 max_arch_pfn = 0x400000000
init_memory_mapping: 0000000000000000-0000000080800000
RAMDISK: 01e98000 - 0e9fa000
No NUMA configuration found
Faking a node at 0000000000000000-0000000080800000
Initmem setup node 0 0000000000000000-0000000080800000
  NODE_DATA [000000007ffec000 - 000000007fffffff]
Zone PFN ranges:
  DMA      0x00000010 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   empty
Movable zone start PFN for each node
early_node_map[2] active PFN
 ranges
    0: 0x00000010 -> 0x000000a0
    0: 0x00000100 -> 0x00080800
SMP: Allowing 1 CPUs, 0 hotplug CPUs
No local APIC present
APIC: disable apic facility
APIC: switched to apic NOOP
PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
Allocating PCI resources starting at 80800000 (gap: 80800000:7f800000)
Booting paravirtualized kernel on Xen
Xen version: 4.0.1 (preserve-AD)
setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:1 nr_node_ids:1
PERCPU: Embedded 28 pages/cpu @ffff88007ffbd000 s82752 r8192 d23744 u114688
pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
pcpu-alloc: [0] 0 
Built 1 zonelists in Node order, mobility grouping on.  Total pages: 519022
Policy zone: DMA32
Kernel command line: root=/dev/xvda1 ro console=hvc0
PID hash table entries: 4096 (order: 3, 32768 bytes)
Checking aperture...
No AGP bridge
 found
Memory: 1835560k/2105344k available (4462k kernel code, 448k absent, 269336k
reserved, 6442k data, 1564k init)
Hierarchical RCU implementation.
    RCU-based detection of stalled CPUs is disabled.
NR_IRQS:16640 nr_irqs:256 16
Console: colour dummy device 80x25
console [tty0] enabled
console [hvc0] enabled
allocated 22282240 bytes of page_cgroup
please try ''cgroup_disable=memory'' option if you
don''t want memory cgroups
installing Xen timer for CPU 0
Detected 2833.068 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency..
5666.13 BogoMIPS (lpj=2833068)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux:  Initializing.
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Mount-cache hash table entries: 256
Initializing cgroup subsys
 debug
Initializing cgroup subsys ns
ns_cgroup deprecated: consider using the ''clone_children'' flag
without the ns_cgroup.
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
CPU: Unsupported number of siblings 4
Performance Events: unsupported p6 CPU model 23 no PMU driver, software events
only.
SMP alternatives: switching to UP code
Freeing SMP alternatives: 20k freed
cpu 0 spinlock event irq 254
Brought up 1 CPUs
Grant table initialized
regulator: core version 0.5
regulator: dummy: 
Time: 165:165:165  Date: 165/165/65
NET: Registered protocol family 16
PCI: setting up Xen PCI frontend stub
bio: create slab <bio-0> at 0
ACPI: Interpreter disabled.
xen_balloon: Initialising balloon driver.
vgaarb: loaded
SCSI subsystem initialized
usbcore:
 registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: System does not support PCI
PCI: System does not support PCI
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
Switching to clocksource xen
pnp: PnP ACPI: disabled
NET: Registered protocol family 2
IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
UDP hash table entries: 1024 (order: 3, 32768 bytes)
UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
NET: Registered protocol family 1
Trying to unpack rootfs image as
 initramfs...
Freeing initrd memory: 208264k freed
platform rtc_cmos: registered platform RTC device (no PNP device found)
audit: initializing netlink socket (disabled)
type=2000 audit(1289476232.768:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 3991
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
pci-stub: invalid id string ""
Event-channel device installed.
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
Serial: 8250/16550 driver, 4 ports, IRQ sharing
 enabled
brd: module loaded
loop: module loaded
Fixed MDIO Bus: probed
Initialising Xen virtual ethernet driver.
ehci_hcd: USB 2.0 ''Enhanced'' Host Controller (EHCI) Driver
ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
PNP: No PS/2 controller found. Probing ports directly.
i8042.c: No controller found.
mice: PS/2 mouse device common for all mice
rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
rtc_cmos: probe of rtc_cmos failed with error -38
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com
cpuidle: using governor ladder
cpuidle: using governor menu
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
IPVS: Registered protocols (TCP, UDP, AH, ESP)
IPVS: Connection hash table
 configured (size=4096, memory=64Kbytes)
IPVS: ipvs loaded.
ip_tables: (C) 2000-2006 Netfilter Core Team
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
Bridge firewalling registered
Ebtables v2.0 registered
Registering the dns_resolver key type
registered taskstats version 1
IMA: No TPM chip found, activating TPM-bypass!
blkfront: xvda: barriers disabled
 xvda: xvda1 xvda2
XENBUS: Device with no driver: device/console/0
  Magic number: 1:252:3141
drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
Initalizing network drop monitor service
Freeing unused kernel memory: 1564k freed
Write protecting the kernel read-only data: 10240k
Freeing unused kernel memory: 1664k freed
Freeing unused kernel memory: 1904k freed
Loading, please wait...
mount: mounting none on /dev failed: No such device
udevd (63): /proc/63/oom_adj is deprecated,
 please use /proc/63/oom_score_adj instead.
Begin: Loading essential drivers... ...
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
EXT4-fs (xvda1): INFO: recovery required on readonly filesystem
EXT4-fs (xvda1): write access will be enabled during recovery
EXT4-fs (xvda1): orphan cleanup on readonly fs
EXT4-fs (xvda1): 5 orphan inodes deleted
EXT4-fs (xvda1): recovery complete
EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
init: ureadahead main process (157) terminated with status 5
fsck from util-linux-ng 2.17.2
/dev/xvda1: clean, 188758/760368 files, 1083747/3041280 blocks
 * Setting sensors
 limits                                                [ OK ] 
 * Exporting directories for NFS kernel daemon...                        [ OK ] 
 * Starting NFS kernel daemon                                            [ OK ] 
speech-dispatcher disabled; edit /etc/default/speech-dispatcher
 * PulseAudio configured for per-user sessions
saned disabled; edit
 /etc/default/saned
 * Enabling additional executable binary formats binfmt-support          [ OK ] 
fsck from util-linux-ng 2.17.2
/dev/xvda1: clean, 188758/760368 files, 1083747/3041280 blocks
 * Setting sensors limits                                                [ OK ] 
 * Exporting directories for NFS kernel daemon...                        [ OK ] 
 * Starting NFS kernel
 daemon                                            [ OK ] 
speech-dispatcher disabled; edit /etc/default/speech-dispatcher
 * PulseAudio configured for per-user sessions
saned disabled; edit /etc/default/saned
 * Enabling additional executable binary formats binfmt-support          [ OK ] 
 * Checking battery state...                                             [ OK ] 

Ubuntu 10.10 boris-HVM-domU hvc0

boris-HVM-domU login: root
Password:
 
Last login: Thu Nov 11 12:29:06 CET 2010 on hvc0
Linux boris-HVM-domU 2.6.37-rc1 #1 SMP Fri Nov 5 19:29:09 MSK 2010 x86_64
GNU/Linux
Ubuntu 10.10

Welcome to Ubuntu!
 * Documentation:  https://help.ubuntu.com/

root@boris-HVM-domU:~# mount 192.168.1.11:/home/boris /mnt/tmp
root@boris-HVM-domU:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1             12G  4.0G  6.9G  37% /
none                 1001M  116K 1001M   1% /dev
none                 1001M     0 1001M   0%
 /dev/shm
none                 1001M  100K 1001M   1% /var/run
none                 1001M     0 1001M   0% /var/lock
192.168.1.11:/home/boris
                       44G   21G   21G  50% /mnt/tmp

*************************************************
RUNNING ''ls -l'' against NFS mounted share
*************************************************

root@boris-HVM-domU:~# ls -l /mnt/tmp
total 693924
-rw-r--r-- 1 root root 709871616 2009-06-07 18:51 osol-0906-x86.iso
drwxr-xr-x 2 root root      4096 2009-06-07 19:06 solaris
root@boris-HVM-domU:~#
 ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3e:48:b3:3c  
          inet addr:192.168.1.9  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fe48:b33c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:300 errors:0 dropped:0 overruns:0 frame:0
          TX packets:94 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:30391 (30.3 KB)  TX bytes:13562 (13.5 KB)
          Interrupt:246
 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:96 errors:0 dropped:0 overruns:0 frame:0
          TX packets:96 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:8996 (8.9 KB)  TX bytes:8996 (8.9 KB)

root@boris-HVM-domU:~# df -h
Filesystem            Size  Used Avail Use% Mounted
 on
/dev/xvda1             12G  4.0G  6.9G  37% /
none                 1001M  116K 1001M   1% /dev
none                 1001M     0 1001M   0% /dev/shm
none                 1001M  100K 1001M   1% /var/run
none                 1001M     0 1001M   0% /var/lock
192.168.1.11:/home/boris
                       44G   21G   21G  51%
 /mnt/tmp

Boris.



--- On Wed, 11/10/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel
paging request
To: xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Wednesday, November 10, 2010, 5:30 PM

On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com>
wrote:>
> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build
22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
>
> The domU console output hung up prior to spitting out a prompt post root login. I logged into the domU via ssh and ran an
''ls'' of an NFS mounted dir and> this immediately came out on the domU console:
>
> [  480.467525] BUG: unable to handle kernel paging request at
ffff880077e9f030
> [  480.467547] IP: [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> [  480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
> [  480.467587] Oops: 0003 [#1] SMP
> [  480.467601] last sysfs file: /sys/kernel/uevent_seqnum
> [  480.467611] CPU 3
> [  480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
> [  480.467676]
> [  480.467684] Pid: 3957, comm: zsh Not tainted
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [  480.467695] RIP: e030:[<ffffffff810355f3>]  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d> [  480.467710] RSP: e02b:ffff880077ebfbd8  EFLAGS: 00010202
> [  480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX:
800000004d686167
> [  480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI:
ffff8800763ddb80
> [  480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09:
ffffea0001a3b2d8
> [  480.467748] R10: 0000000000000100 R11: 0000000000000000 R12:
00007f5cb26069d0
> [  480.467759] R13: 0000000000000001 R14: 0000000000000000 R15:
ffff8800763ddb80
> [  480.467772] FS:  00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
knlGS:0000000000000000
> [  480.467783] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4:
0000000000002660
> [  480.467802] DR0: 0000000000000000 DR1: 0000000000000000
 DR2: 0000000000000000> [  480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
> [  480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task
ffff880078b80000)
> [  480.467832] Stack:
> [  480.467838]  0000000000000100 ffffea0001a3b2d8 7000000000000000
0000000000000000
> [  480.467862]  ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
ffffffff810e2200
> [  480.467886]  ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
0000000000000001
> [  480.467910] Call Trace:
> [  480.467921]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
> [  480.467934]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [  480.467946]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [  480.467958]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
> [  480.467972]  [<ffffffff814463fe>]
do_page_fault+0x374/0x396> [  480.467984]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.467996]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.468009]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> [  480.468020]  [<ffffffff8100728f>] ?
xen_restore_fl_direct_end+0x0/0x1
> [  480.468034]  [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
> [  480.468045]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.468059]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> [  480.468072]  [<ffffffff81443615>] page_fault+0x25/0x30
> [  480.468085]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
> [  480.468096]  [<ffffffff810498a6>] ?
 schedule_tail+0x5c/0x60> [  480.468108]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
> [  480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89 f4
53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18
<48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
> [  480.468323] RIP  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> [  480.468323]  RSP <ffff880077ebfbd8>
> [  480.468323] CR2: ffff880077e9f030
> [  480.468323] ---[ end trace 6d1623321ecd50ac ]---
>
> The domU config is attached as config.2.6.37_domU_debug
>
>
> -Bruce
Here''s another attempt from the same config with the full domU boot
logs:

Started domain dpm (id=4)
                         [    0.000000]
 Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug (root@opal) (gcc version
4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Tue Nov 9 20:26:03 PST 2010
[    0.000000] Command line:
root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro   swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] released 0 pages of unused memory
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000080800000 (usable)
[   
 0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[    0.000000] Scanning 0 areas for low memory corruption
[    0.000000] init_memory_mapping: 0000000000000000-0000000080800000
[    0.000000] RAMDISK: 01d95000 - 02fa9000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-0000000080800000
[    0.000000] Initmem setup node 0 0000000000000000-0000000080800000
[    0.000000]   NODE_DATA [000000007fffb000 - 000000007fffffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000
 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x000000a0
[    0.000000]     0: 0x00000100 -> 0x00080800
[    0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
[    0.000000] No local APIC present
[    0.000000] APIC: disable apic facility
[    0.000000] APIC: switched to apic NOOP
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
[    0.000000] Allocating PCI resources starting at 80800000 (gap:
80800000:7f800000)
[    0.000000] Booting paravirtualized kernel on Xen
[    0.000000] Xen version: 4.1-unstable (preserve-AD)
[   
 0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
nr_cpu_ids:6 nr_node_ids:1
[    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
r8192 d23744 u114688
[    0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.
Total pages: 519022
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:
root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro   swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Placing 64MB software IO TLB between ffff880079400000 -
ffff88007d400000
[    0.000000] software IO TLB
 at phys 0x79400000 - 0x7d400000
[    0.000000] Memory: 1959896k/2105344k available (4398k kernel code,
448k absent, 145000k reserved, 6327k data, 756k init)
[    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=6, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU-based detection of stalled CPUs is disabled.
[    0.000000] NR_IRQS:4352 nr_irqs:320 16
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [hvc0] enabled
[    0.000000] installing Xen timer for CPU 0
[    0.000000] Detected 2660.064 MHz processor.
[    0.000999] Calibrating delay loop (skipped), value calculated
using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
[    0.000999] pid_max: default: 32768 minimum:
 301
[    0.000999] Security Framework initialized
[    0.000999] SELinux:  Initializing.
[    0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.001408] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.001635] Mount-cache hash table entries: 256
[    0.002195] Initializing cgroup subsys ns
[    0.002208] ns_cgroup deprecated: consider using the
''clone_children'' flag without the ns_cgroup.
[    0.002220] Initializing cgroup subsys cpuacct
[    0.002231] Initializing cgroup subsys freezer
[    0.002356] CPU: Unsupported number of siblings 16
[    0.002362] Performance Events: unsupported p6 CPU model 26 no PMU
driver, software events only.
[    0.002483] SMP alternatives: switching to UP code
[    0.004119] cpu 0 spinlock
 event irq 318
[    0.004765] NMI watchdog failed to create perf event on cpu0:
fffffffffffffffe
[    0.005050] installing Xen timer for CPU 1
[    0.005157] cpu 1 spinlock event irq 312
[    0.005194] SMP alternatives: switching to SMP code
[    0.000999] CPU: Unsupported number of siblings 16
[    0.006158] NMI watchdog failed to create perf event on cpu1:
fffffffffffffffe
[    0.006474] installing Xen timer for CPU 2
[    0.006585] cpu 2 spinlock event irq 306
[    0.000999] CPU: Unsupported number of siblings 16
[    0.006816] NMI watchdog failed to create perf event on cpu2:
fffffffffffffffe
[    0.006999] installing Xen timer for CPU 3
[    0.007037] cpu 3 spinlock event irq 300
[    0.000999] CPU: Unsupported number of siblings 16
[    0.007300] NMI watchdog
 failed to create perf event on cpu3:
fffffffffffffffe
[    0.007534] installing Xen timer for CPU 4
[    0.007636] cpu 4 spinlock event irq 294
[    0.000999] CPU: Unsupported number of siblings 16
[    0.007874] NMI watchdog failed to create perf event on cpu4:
fffffffffffffffe
[    0.008041] installing Xen timer for CPU 5
[    0.008083] cpu 5 spinlock event irq 288
[    0.000999] CPU: Unsupported number of siblings 16
[    0.008344] NMI watchdog failed to create perf event on cpu5:
fffffffffffffffe
[    0.008356] Brought up 6 CPUs
[    0.009181] kworker/u:0 used greatest stack depth: 5696 bytes left
[    0.012051] Grant table initialized
[    0.030978] Time: 165:165:165  Date: 165/165/65
[    0.031229] NET: Registered protocol family 16
[    0.036672] PCI:
 setting up Xen PCI frontend stub
[    0.037225] kworker/u:0 used greatest stack depth: 5528 bytes left
[    0.074302] bio: create slab <bio-0> at 0
[    0.075385] ACPI: Interpreter disabled.
[    0.077056] xen_balloon: Initialising balloon driver.
[    0.079323] vgaarb: loaded
[    0.081270] usbcore: registered new interface driver usbfs
[    0.081283] usbcore: registered new interface driver hub
[    0.081283] usbcore: registered new device driver usb
[    0.083183] PCI: System does not support PCI
[    0.084007] PCI: System does not support PCI
[    0.085236] NetLabel: Initializing
[    0.085236] NetLabel:  domain hash size = 128
[    0.085236] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.085313] NetLabel:  unlabeled traffic allowed by
 default
[    0.086544] Switching to clocksource xen
[    0.106256] pnp: PnP ACPI: disabled
[    0.129799] NET: Registered protocol family 2
[    0.130174] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.131882] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[    0.132891] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.133179] TCP: Hash tables configured (established 262144 bind 65536)
[    0.133191] TCP reno registered
[    0.133301] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[    0.133407] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[    0.133914] NET: Registered protocol family 1
[    0.134582] RPC: Registered udp transport module.
[    0.134590] RPC: Registered tcp transport
 module.
[    0.134595] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.134892] Trying to unpack rootfs image as initramfs...
[    0.178824] Freeing initrd memory: 18512k freed
[    0.268798] DMA-API: preallocated 32768 debug entries
[    0.268810] DMA-API: debugging enabled by kernel config
[    0.271181] platform rtc_cmos: registered platform RTC device (no
PNP device found)
[    0.285410] Machine check injector initialized
[    0.294732] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
[    0.294803] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
[    0.294893] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
[    0.294944] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
[    0.294998] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
[    0.295043]
 microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
[    0.295590] microcode: Microcode Update Driver: v2.00
<tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    0.295607] Scanning for low memory corruption every 60 seconds
[    0.298813] audit: initializing netlink socket (disabled)
[    0.298940] type=2000 audit(1289403082.795:1): initialized
[    0.333762] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.376985] VFS: Disk quotas dquot_6.5.2
[    0.377853] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.383121] NTFS driver 2.1.29 [Flags: R/W].
[    0.385014] msgmni has been set to 3864
[    0.389953] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[   
 0.389970] io scheduler noop registered
[    0.389979] io scheduler deadline registered
[    0.390666] io scheduler cfq registered (default)
[    0.392486] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.406238] pcifront pci-0: Installing PCI frontend
[    0.406967] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/0
[    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/2
[    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/3
[    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/4
[    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/0
[    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/2
[    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/3
[    0.417634] pcifront
 pci-0: claiming resource 0000:00:01.0/4
[    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/0
[    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/2
[    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/3
[    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/4
[    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/0
[    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/2
[    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/3
[    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/4
[    0.731775] Non-volatile memory driver v1.3
[    0.731790] Linux agpgart interface v0.103
[    0.733731] [drm] Initialized drm 1.1.0 20060810
[    0.733752] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.827202] brd: module
 loaded
[    0.840788] loop: module loaded
[    0.841965] Fixed MDIO Bus: probed
[    0.844025] ehci_hcd: USB 2.0 ''Enhanced'' Host Controller
(EHCI) Driver
[    0.844651] ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI)
Driver
[    0.845194] uhci_hcd: USB Universal Host Controller Interface driver
[    0.846985] usbcore: registered new interface driver usblp
[    0.847763] usbcore: registered new interface driver libusual
[    0.849433] PNP: No PS/2 controller found. Probing ports directly.
[    0.850279] i8042.c: No controller found.
[    0.850883] mice: PS/2 mouse device common for all mice
[    0.853446] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[    0.854066] rtc_cmos: probe of rtc_cmos failed with error -38
[    0.855928] cpuidle: using governor ladder
[   
 0.855941] cpuidle: using governor menu
[    0.856029] Netfilter messages via NETLINK v0.30.
[    0.856136] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    0.857888] ctnetlink v0.93: registering with nfnetlink.
[    0.860285] ip_tables: (C) 2000-2006 Netfilter Core Team
[    0.860677] TCP cubic registered
[    0.860687] Initializing XFRM netlink socket
[    0.863286] NET: Registered protocol family 10
[    0.867216] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    0.867612] IPv6 over IPv4 tunneling driver
[    0.871326] NET: Registered protocol family 17
[    0.871566] Registering the dns_resolver key type
[    0.873668] registered taskstats version 1
[    0.873912] kmemleak: Kernel memory leak detector initialized
[    0.873928] kmemleak: Automatic memory
 scanning thread started
[    0.875057] XENBUS: Device with no driver: device/vbd/51712
[    0.875057] XENBUS: Device with no driver: device/vif/0
[    0.875057] XENBUS: Device with no driver: device/console/0
[    0.875057]   Magic number: 1:252:3141
[   13.388958] Freeing unused kernel memory: 756k freed
[   13.389190] Write protecting the kernel read-only data: 10240k
[   13.398774] Freeing unused kernel memory: 1724k freed
[   13.400663] Freeing unused kernel memory: 1992k freed
[   13.494573] all_generic_ide used greatest stack depth: 5472 bytes left
[   13.519388] udevd (1111): /proc/1111/oom_adj is deprecated, please
use /proc/1111/oom_score_adj instead.
Begin: Loading essential drivers... ...
[   13.559435] blkid used greatest stack depth: 5408 bytes
 left
[   13.568391] blkfront: xvda: barriers disabled
[   13.574762] blkid used greatest stack depth: 5264 bytes left
[   13.580813]  xvda: xvda1 xvda2 < xvda5 >
[   13.645562] modprobe used greatest stack depth: 5088 bytes left
[   13.653160] Initialising Xen virtual ethernet driver.
[   13.659540] blkid used greatest stack depth: 4960 bytes left
[   13.735467] blkid used greatest stack depth: 4752 bytes left
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
[   13.903774] EXT4-fs (xvda1): INFO: recovery required on readonly filesystem
[   13.903799] EXT4-fs (xvda1): write access will be enabled during
 recovery
[   14.033719] EXT4-fs (xvda1): recovery complete
[   14.034966] EXT4-fs (xvda1): mounted filesystem with ordered data
mode. Opts: (null)
[   14.035510] exe used greatest stack depth: 4112 bytes left
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
init: ureadahead main process (1214) terminated with status 5
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
/dev/xvda1: clean, 56314/288576 files, 287743/1153280 blocks
 * Enable login on console                                               [ OK ]
 * Check for hostname changes...                                     
    [ OK ]
   ...done.
 * Setting sensors limits
         No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
                                                                         [ OK ]
 * Initializing random number generator...                               [ OK ]
 * Setting up X server socket directory /tmp/.X11-unix...                [ OK ]
 * Setting up ICE
 socket directory /tmp/.ICE-unix...                     [ OK ]
 * Not starting fancontrol; run pwmconfig first.
 * Starting internet superserver inetd                                   [ OK ]
Starting openntpd: ntpd.
 * Starting network management services:
 * Setting core file location...                                         [ OK ]
 * Restoring resolver state...                                           [ OK ]
 * Checking tomcat configuration
   ...done.
 * Starting Tomcat servlet engine tomcat6
     
    Using CATALINA_BASE:   /var/lib/tomcat6
Using CATALINA_HOME:   /usr/share/tomcat6
Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
Using JRE_HOME:        /usr/lib/jvm/java-6-openjdk
Using CLASSPATH:       /usr/share/tomcat6/bin/bootstrap.jar
                                                                         [ OK ]
collectd has been disabled - see /etc/default/collectd.
 * Recording successful boot for GRUB
   ...done.
 * Running local boot scripts (/etc/rc.local)                            [ OK ]
mountall: Plymouth command failed
mountall: Disconnected from
 Plymouth

6.0.16-01
kaan-04-dpm login: root
Password:
[   42.646538] ------------[ cut here ]------------
[   42.646556] kernel BUG at arch/x86/xen/mmu.c:1831!
[   42.646566] invalid opcode: 0000 [#1] SMP
[   42.646582] last sysfs file: /sys/kernel/uevent_seqnum
[   42.646592] CPU 0
[   42.646598] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
xen_netfront xen_blkfront
[   42.646665]
[   42.646674] Pid: 3916, comm: lsb_release Not tainted
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
[   42.646688] RIP: e030:[<ffffffff81004bce>]  [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[   42.646711] RSP: e02b:ffff88007595bce8  EFLAGS:
 00010282
[   42.646721] RAX: 00000000ffffffea RBX: 000000000007751e RCX: 0000000000000001
[   42.646732] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
[   42.646744] RBP: ffff88007595bd08 R08: 00003ffffffff000 R09: ffff880000000000
[   42.646756] R10: 00000000deadbeef R11: dead000000200200 R12: 0000000000000003
[   42.646768] R13: ffff880075c90150 R14: ffff8800781d9398 R15: ffff880075161d40
[   42.646784] FS:  00007fee63b01700(0000) GS:ffff88007ff40000(0000)
knlGS:0000000000000000
[   42.646797] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[   42.646807] CR2: 00007faac54182b8 CR3: 00000000771dd000 CR4: 0000000000002660
[   42.646819] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   42.646832] DR3: 0000000000000000 DR6:
 00000000ffff0ff0 DR7: 0000000000000400
[   42.646844] Process lsb_release (pid: 3916, threadinfo
ffff88007595a000, task ffff880075161d40)
[   42.646857] Stack:
[   42.646864]  ffff880000000000 00000000000484f0 ffff8800777c6d80
000000000007751e
[   42.646892]  ffff88007595bd28 ffffffff810066a8 000000000007751e
ffff8800777c6d80
[   42.646919]  ffff88007595bd38 ffffffff810066db ffff88007595bd78
ffffffff810e048f
[   42.646946] Call Trace:
[   42.646959]  [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[   42.646973]  [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[   42.646989]  [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[   42.647003]  [<ffffffff810e2717>]
 handle_mm_fault+0x131/0x8c1
[   42.647017]  [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
[   42.647033]  [<ffffffff8100eb84>] ?
arch_get_unmapped_area_topdown+0x4a/0x28a
[   42.647052]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
[   42.647059]  [<ffffffff8102b78a>] ? pvclock_clocksource_read+0x4b/0xb4
[   42.647059]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[   42.647059]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
[   42.647059]  [<ffffffff81443615>] page_fault+0x25/0x30
[   42.647059] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff
85 c0 74 04 <0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48
8b
 3d
[   42.647059] RIP  [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[   42.647059]  RSP <ffff88007595bce8>
[   42.647059] ---[ end trace 3fbbffee414a5166 ]---

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      
-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-11 12:26 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Full stack trace obtained in text console of 2.6.37-rc1 PV guest :-
Attempt to scp generates output ( or maybe just time delay expired)

root@boris-HVM-domU:~# scp  1.log 192.168.1.12:/home/boris/ubuntu
BUG: unable to handle kernel paging request at ffff88000e676930
IP: [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
PGD 1a04067 PUD 1a08067 PMD ee73067 PTE 801000000e676065
Oops: 0003 [#1] SMP 
last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
CPU 0 
Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl auth_rpcgss
sunrpc exportfs ipv6 lp parport

Pid: 3534, comm: scp Not tainted 2.6.37-rc1 #1 /
RIP: e030:[<ffffffff810069c3>]  [<ffffffff810069c3>]
xen_set_pmd+0x24/0x44
RSP: e02b:ffff88006e697b18  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff88000e676930 RCX: ffffc00000000f98
RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff88000e676930
RBP: ffff88006e697b28 R08: 00007fff9d120000 R09: ffff88007ffc80a0
R10: 0000000000000000 R11: ffff88006e697ae8 R12: 0000000000000000
R13: ffff88007ffcb140 R14: ffff88000e676930 R15: ffff88006f08d7f8
FS:  00007fa525896700(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff88000e676930 CR3: 0000000077b0f000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process scp (pid: 3534, threadinfo ffff88006e696000, task ffff88007a32c2c0)
Stack:
 00007fa524e00000 00007fa525b37000 ffff88006e697be8 ffffffff810dc4a4
 ffff88007a26ac88 ffff88006e690298 ffff88007a26ac78 00007fa524a00000
 00007f8000000000 00007fff80000000 00007fa525b36fff 00007fa525b36fff
Call Trace:
 [<ffffffff810dc4a4>] free_pgd_range+0x213/0x397
 [<ffffffff810dc6cc>] free_pgtables+0xa4/0xc7
 [<ffffffff810dfd82>] exit_mmap+0xf2/0x12a
 [<ffffffff8104a7c5>] mmput+0x5e/0xd1
 [<ffffffff8110694b>] flush_old_exec+0x526/0x600
 [<ffffffff8113e006>] load_elf_binary+0x2dc/0x1752
 [<ffffffff8113b687>] ? load_misc_binary+0x6e/0x367
 [<ffffffff811dcee4>] ? ima_must_measure+0x21/0x2b
 [<ffffffff811dc7d6>] ? process_measurement+0x56/0xe4
 [<ffffffff81106195>] search_binary_handler+0xbb/0x24d
 [<ffffffff8113dd2a>] ? load_elf_binary+0x0/0x1752
 [<ffffffff8110785d>] do_execve+0x1c1/0x2ae
 [<ffffffff810119e5>] sys_execve+0x3e/0x55
 [<ffffffff8100af9c>] stub_execve+0x6c/0xc0
Code: 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 b5 e4 ff ff
48 89 df ff 05 3e 55 c3 00 e8 d5 dd ff ff 84 c0 75 05 <4c> 89 23 eb 16 e8
99 e4 ff ff 4c 89 e6 48 89 df ff 05 23 55 c3
RIP  [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
 RSP <ffff88006e697b18>
CR2: ffff88000e676930
---[ end trace 073da74f4ce02315 ]---
lost connection
root@boris-HVM-domU:~# ------------[ cut here ]------------
kernel BUG at arch/x86/xen/mmu.c:1831!
invalid opcode: 0000 [#2] SMP 
last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
CPU 0 
Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl auth_rpcgss
sunrpc exportfs ipv6 lp parport

Pid: 3536, comm: cron Tainted: G      D     2.6.37-rc1 #1 /
RIP: e030:[<ffffffff81004bce>]  [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
RSP: e02b:ffff88006d391cd8  EFLAGS: 00010282
RAX: 00000000ffffffea RBX: 000000000006d0f5 RCX: 0000000000000001
RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88006d391cd8
RBP: ffff88006d391cf8 R08: 00003ffffffff000 R09: ffff880000000000
R10: 0000000000007ff0 R11: ffff880000000041 R12: 0000000000000003
R13: ffff88006f082010 R14: ffff88006e7c1420 R15: ffff88007a266340
FS:  00007f3e0f0f87a0(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000406039 CR3: 000000006d2ad000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process cron (pid: 3536, threadinfo ffff88006d390000, task ffff88007a266340)
Stack:
 ffff880000000000 00000000001411dd ffff88000e598b00 000000000006d0f5
 ffff88006d391d18 ffffffff810066ce 000000000006d0f5 ffff88000e598b00
 ffff88006d391d28 ffffffff81006701 ffff88006d391d68 ffffffff810dafc6
Call Trace:
 [<ffffffff810066ce>] xen_alloc_ptpage+0x68/0x6d
 [<ffffffff81006701>] xen_alloc_pte+0xe/0x10
 [<ffffffff810dafc6>] __pte_alloc+0x72/0xd7
 [<ffffffff810dd355>] handle_mm_fault+0x12f/0xa05
 [<ffffffff8145665a>] do_page_fault+0x387/0x3a9
 [<ffffffff81453715>] ? page_fault+0x25/0x30
 [<ffffffff8145394a>] ? error_exit+0x2a/0x60
 [<ffffffff8145349d>] ? retint_restore_args+0x5/0x6
 [<ffffffff8120510d>] ? __put_user_4+0x1d/0x30
 [<ffffffff81453715>] page_fault+0x25/0x30
Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d e0 48 89 45 e8 be 01 00 00
00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3
55 48 89 e5 41 54 49 89 fc 53 31 db 48 8b 3d
RIP  [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
 RSP <ffff88006d391cd8>
---[ end trace 073da74f4ce02316 ]---

Boris.




      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Nov-11 14:50 UTC

head link

Re: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Thu, Nov 11, 2010 at 3:01 AM, Boris Derzhavets
<bderzhavets@yahoo.com>wrote:
> > The domU console output hung up prior to spitting out a prompt post
root
> login.
> >  I logged into the domU via ssh
> Ubuntu 10.10 HVM image has been used to test 2.6.37-rc1 pvops DomU
>
Hi Boris,

What do you mean by "Ubuntu 10.10 HVM image has been used to test
2.6.37-rc1
pvops DomU"?
Is there an Ubuntu released hvm image that also works as a domU?

If so, could you post details on where this image is from, or if you built
it, could you post your kernel .config?

> File /etc/init/hvc0 has been prepared in HVM mode to login from command
> line.
>
I am able to get a prompt most boots. The hang post login appears a bit
random. Sometimes it''ll show the same BUG if I let it sit at the login
prompt.
After a few tries, I don''t think my login/ssh/ls test case had anything
to
do with it.

I am mounting some NFS shares from the dom0 at boot time, so that might be
the trigger that hits the
    "kernel BUG at arch/x86/xen/mmu.c:1831!..."
problem.

-Bruce
>
> PV Guest config
>
> root@ServerLLX:/home/boris/ubuntu# cat u10.cfg
> name="Ubuntu10PV"
> memory = 2048
> kernel = "/home/boris/ubuntu/vmlinuz-2.6.37-rc1"
> ramdisk = "/home/boris/ubuntu/initrd.img-2.6.37-rc1"
> disk = [''phy:/dev/sda7,xvda,w'']
> vif  = [''bridge=br0'']
> extra = "root=/dev/xvda1 ro console=hvc0"
> # vfb =  [''type=vnc,vncunused=1'']
>
> root@ServerLLX:/home/boris/ubuntu# xm create -c  u10.cfg
> Using config file "./u10.cfg".
> Started domain Ubuntu10PV (id=4)
>                                 Initializing cgroup subsys cpuset
> Initializing cgroup subsys cpu
> Linux version 2.6.37-rc1 (root@ServerLLX) (gcc version 4.4.3 (Ubuntu
> 4.4.3-4ubuntu5) ) #1 SMP Fri Nov 5 19:29:09 MSK 2010
> Command line: root=/dev/xvda1 ro console=hvc0
>
> ACPI in unprivileged domain disabled
> released 0 pages of unused memory
> BIOS-provided physical RAM map:
>  Xen: 0000000000000000 - 00000000000a0000 (usable)
>  Xen: 00000000000a0000 - 0000000000100000 (reserved)
>  Xen: 0000000000100000 - 0000000080800000 (usable)
>
> NX (Execute Disable) protection: active
> DMI not present or invalid.
> No AGP bridge found
>
> last_pfn = 0x80800 max_arch_pfn = 0x400000000
> init_memory_mapping: 0000000000000000-0000000080800000
> RAMDISK: 01e98000 - 0e9fa000
> No NUMA configuration found
>
> Faking a node at 0000000000000000-0000000080800000
> Initmem setup node 0 0000000000000000-0000000080800000
>   NODE_DATA [000000007ffec000 - 000000007fffffff]
> Zone PFN ranges:
>   DMA      0x00000010 -> 0x00001000
>   DMA32    0x00001000 -> 0x00100000
>   Normal   empty
>
> Movable zone start PFN for each node
> early_node_map[2] active PFN ranges
>     0: 0x00000010 -> 0x000000a0
>     0: 0x00000100 -> 0x00080800
> SMP: Allowing 1 CPUs, 0 hotplug CPUs
> No local APIC present
> APIC: disable apic facility
>
> APIC: switched to apic NOOP
> PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
> Allocating PCI resources starting at 80800000 (gap: 80800000:7f800000)
> Booting paravirtualized kernel on Xen
> Xen version: 4.0.1 (preserve-AD)
> setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:1 nr_node_ids:1
> PERCPU: Embedded 28 pages/cpu @ffff88007ffbd000 s82752 r8192 d23744 u114688
>
> pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
> pcpu-alloc: [0] 0
> Built 1 zonelists in Node order, mobility grouping on.  Total pages: 519022
> Policy zone: DMA32
> Kernel command line: root=/dev/xvda1 ro console=hvc0
>
> PID hash table entries: 4096 (order: 3, 32768 bytes)
> Checking aperture...
> No AGP bridge found
> Memory: 1835560k/2105344k available (4462k kernel code, 448k absent,
> 269336k reserved, 6442k data, 1564k init)
> Hierarchical RCU implementation.
>
>     RCU-based detection of stalled CPUs is disabled.
> NR_IRQS:16640 nr_irqs:256 16
>
> Console: colour dummy device 80x25
> console [tty0] enabled
> console [hvc0] enabled
> allocated 22282240 bytes of page_cgroup
> please try ''cgroup_disable=memory'' option if you
don''t want memory cgroups
>
> installing Xen timer for CPU 0
> Detected 2833.068 MHz processor.
> Calibrating delay loop (skipped), value calculated using timer frequency..
> 5666.13 BogoMIPS (lpj=2833068)
>
> pid_max: default: 32768 minimum: 301
> Security Framework initialized
> SELinux:  Initializing.
>
> Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
> Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
> Mount-cache hash table entries: 256
> Initializing cgroup subsys debug
> Initializing cgroup subsys ns
>
> ns_cgroup deprecated: consider using the ''clone_children''
flag without the
> ns_cgroup.
> Initializing cgroup subsys cpuacct
> Initializing cgroup subsys memory
> Initializing cgroup subsys devices
> Initializing cgroup subsys freezer
> Initializing cgroup subsys net_cls
> CPU: Unsupported number of siblings 4
> Performance Events: unsupported p6 CPU model 23 no PMU driver, software
> events only.
>
> SMP alternatives: switching to UP code
> Freeing SMP alternatives: 20k freed
> cpu 0 spinlock event irq 254
> Brought up 1 CPUs
> Grant table initialized
> regulator: core version 0.5
> regulator: dummy:
> Time: 165:165:165  Date: 165/165/65
> NET: Registered protocol family 16
> PCI: setting up Xen PCI frontend stub
> bio: create slab <bio-0> at 0
> ACPI: Interpreter disabled.
>
> xen_balloon: Initialising balloon driver.
> vgaarb: loaded
> SCSI subsystem initialized
>
> usbcore: registered new interface driver usbfs
> usbcore: registered new interface driver hub
> usbcore: registered new device driver usb
> PCI: System does not support PCI
> PCI: System does not support PCI
> NetLabel: Initializing
>
> NetLabel:  domain hash size = 128
> NetLabel:  protocols = UNLABELED CIPSOv4
>
> NetLabel:  unlabeled traffic allowed by default
> Switching to clocksource xen
> pnp: PnP ACPI: disabled
>
> NET: Registered protocol family 2
> IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
> TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
> TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
> TCP: Hash tables configured (established 262144 bind 65536)
> TCP reno registered
>
> UDP hash table entries: 1024 (order: 3, 32768 bytes)
> UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
> NET: Registered protocol family 1
> Trying to unpack rootfs image as initramfs...
> Freeing initrd memory: 208264k freed
>
> platform rtc_cmos: registered platform RTC device (no PNP device found)
> audit: initializing netlink socket (disabled)
> type=2000 audit(1289472066.106:1): initialized
>
> HugeTLB registered 2 MB page size, pre-allocated 0 pages
> VFS: Disk quotas dquot_6.5.2
> Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> msgmni has been set to 3991
>
> Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
> io scheduler noop registered
> io scheduler deadline registered
>
> io scheduler cfq registered (default)
> pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> pciehp: PCI Express Hot Plug Controller Driver version: 0.4
> acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
> pci-stub: invalid id string ""
> Event-channel device installed.
>
> Non-volatile memory driver v1.3
> Linux agpgart interface v0.103
> Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> brd: module loaded
> loop: module loaded
> Fixed MDIO Bus: probed
>
> Initialising Xen virtual ethernet driver.
> ehci_hcd: USB 2.0 ''Enhanced'' Host Controller (EHCI)
Driver
> ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI) Driver
> uhci_hcd: USB Universal Host Controller Interface driver
> PNP: No PS/2 controller found. Probing ports directly.
> i8042.c: No controller found.
> mice: PS/2 mouse device common for all mice
> rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
> rtc_cmos: probe of rtc_cmos failed with error -38
> device-mapper: uevent: version 1.0.3
> device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised:
> dm-devel@redhat.com
> cpuidle: using governor ladder
> cpuidle: using governor menu
> usbcore: registered new interface driver usbhid
> usbhid: USB HID core driver
>
> nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
> IPVS: Registered protocols (TCP, UDP, AH, ESP)
> IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
> IPVS: ipvs loaded.
>
> ip_tables: (C) 2000-2006 Netfilter Core Team
> TCP cubic registered
> Initializing XFRM netlink socket
>
> NET: Registered protocol family 17
> Bridge firewalling registered
> Ebtables v2.0 registered
>
> Registering the dns_resolver key type
> registered taskstats version 1
> IMA: No TPM chip found, activating TPM-bypass!
> blkfront: xvda: barriers disabled
>  xvda: xvda1 xvda2
>
> XENBUS: Device with no driver: device/console/0
>   Magic number: 1:252:3141
> drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
> Initalizing network drop monitor service
> Freeing unused kernel memory: 1564k freed
>
> Write protecting the kernel read-only data: 10240k
> Freeing unused kernel memory: 1664k freed
> Freeing unused kernel memory: 1904k freed
> Loading, please wait...
> mount: mounting none on /dev failed: No such device
> udevd (63): /proc/63/oom_adj is deprecated, please use
> /proc/63/oom_score_adj instead.
> Begin: Loading essential drivers... ...
>
> Done.
> Begin: Running /scripts/init-premount ...
> Done.
> Begin: Mounting root file system... ...
> Begin: Running /scripts/local-top ...
> Done.
> Begin: Running /scripts/local-premount ...
> Done.
> EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
> Begin: Running /scripts/local-bottom ...
> Done.
> Done.
> Begin: Running /scripts/init-bottom ...
> Done.
>
> Ubuntu 10.10 boris-HVM-domU hvc0
>
> boris-HVM-domU login: root
> Password:
> Linux boris-HVM-domU 2.6.37-rc1 #1 SMP Fri Nov 5 19:29:09 MSK 2010 x86_64
> GNU/Linux
> Ubuntu 10.10
>
> Welcome to Ubuntu!
>  * Documentation:  https://help.ubuntu.com/
>
> root@boris-HVM-domU:~# df -h
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/xvda1             12G  3.9G  7.1G  36% /
> none                 1001M  116K 1001M   1% /dev
> none                 1001M     0 1001M   0% /dev/shm
> none                 1001M   84K 1001M   1% /var/run
> none                 1001M     0 1001M   0% /var/lock
>
> Boris.
>
> --- On *Wed, 11/10/10, Bruce Edge <bruce.edge@gmail.com>* wrote:
>
>
> From: Bruce Edge <bruce.edge@gmail.com>
> Subject: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle
> kernel paging request
> To: xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>,
> "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
> Date: Wednesday, November 10, 2010, 5:15 PM
>
>
> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build
> 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
>
> The domU console output hung up prior to spitting out a prompt post root
> login. I logged into the domU via ssh and ran an ''ls'' of
an NFS mounted dir
> and
> this immediately came out on the domU console:
>
> [  480.467525] BUG: unable to handle kernel paging request at
> ffff880077e9f030
> [  480.467547] IP: [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> [  480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
> [  480.467587] Oops: 0003 [#1] SMP
> [  480.467601] last sysfs file: /sys/kernel/uevent_seqnum
> [  480.467611] CPU 3
> [  480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
> xen_blkfront
> [  480.467676]
> [  480.467684] Pid: 3957, comm: zsh Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [  480.467695] RIP: e030:[<ffffffff810355f3>] 
[<ffffffff810355f3>]
> ptep_set_access_flags+0x27/0x4d
> [  480.467710] RSP: e02b:ffff880077ebfbd8  EFLAGS: 00010202
> [  480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX:
> 800000004d686167
> [  480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI:
> ffff8800763ddb80
> [  480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09:
> ffffea0001a3b2d8
> [  480.467748] R10: 0000000000000100 R11: 0000000000000000 R12:
> 00007f5cb26069d0
> [  480.467759] R13: 0000000000000001 R14: 0000000000000000 R15:
> ffff8800763ddb80
> [  480.467772] FS:  00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
> knlGS:0000000000000000
> [  480.467783] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4:
> 0000000000002660
> [  480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [  480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [  480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task
> ffff880078b80000)
> [  480.467832] Stack:
> [  480.467838]  0000000000000100 ffffea0001a3b2d8 7000000000000000
> 0000000000000000
> [  480.467862]  ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
> ffffffff810e2200
> [  480.467886]  ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
> 0000000000000001
> [  480.467910] Call Trace:
> [  480.467921]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
> [  480.467934]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [  480.467946]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [  480.467958]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
> [  480.467972]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [  480.467984]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.467996]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.468009]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> [  480.468020]  [<ffffffff8100728f>] ?
xen_restore_fl_direct_end+0x0/0x1
> [  480.468034]  [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
> [  480.468045]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> [  480.468059]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> [  480.468072]  [<ffffffff81443615>] page_fault+0x25/0x30
> [  480.468085]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
> [  480.468096]  [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
> [  480.468108]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
> [  480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89 f4
> 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18
> <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
> [  480.468323] RIP  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> [  480.468323]  RSP <ffff880077ebfbd8>
> [  480.468323] CR2: ffff880077e9f030
> [  480.468323] ---[ end trace 6d1623321ecd50ac ]---
>
> The domU config is attached as config.2.6.37_domU_debug
>
>
> -Bruce
>
> -----Inline Attachment Follows-----
>
> _______________________________________________
> Xen-devel mailing list
>
Xen-devel@lists.xensource.com<http://mc/compose?to=Xen-devel@lists.xensource.com>
> http://lists.xensource.com/xen-devel
>
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-11 15:06 UTC

head link

Re: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

> What do you mean by "Ubuntu 10.10 HVM image has been 
> used to test 2.6.37-rc1 pvops DomU"?
> Is there an Ubuntu released hvm image that also works as a domU?
Bruce,

I created Ubuntu 10.10 HVM on /dev/sda7 and created 
/etc/init/hvc0.conf as usual then loaded PV DomU via profile

root@ServerLLX:/home/boris/ubuntu# cat u10.cfg
name="Ubuntu10PV"
memory = 2048
kernel = "/home/boris/ubuntu/vmlinuz-2.6.37-rc1"
ramdisk = "/home/boris/ubuntu/initrd.img-2.6.37-rc1"
disk = [''phy:/dev/sda7,xvda,w'']
vif  = [''bridge=br0'']
extra = "root=/dev/xvda1 ro console=hvc0"
# vfb =  [''type=vnc,vncunused=1'']

# xm create -c u10.cfg

and got a login prompt ( vs console=tty0)
It also may be loaded with vfb uncommented
 # vncviewer localhost:0
will provide you a login 

I''ve also mounted remote NFS share at DomU and finally
got stack trace been posted.

Boris

--- On Thu, 11/11/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel
paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Thursday, November 11, 2010, 9:50 AM

On Thu, Nov 11, 2010 at 3:01 AM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:
> The domU console output hung up prior to spitting out a prompt post root
login.
>  I logged into the domU via ssh
Ubuntu 10.10 HVM image has been used to test 2.6.37-rc1 pvops DomU

Hi Boris,

What do you mean by "Ubuntu 10.10 HVM image has been used to test
2.6.37-rc1 pvops DomU"?

Is there an Ubuntu released hvm image that also works as a domU?

If so, could you post details on where this image is from, or if you built it,
could you post your kernel .config?

File /etc/init/hvc0 has been prepared in HVM mode to login from command line.

I am able to get a prompt most boots. The hang post login appears a bit random.
Sometimes it''ll show the same BUG if I let it sit at the login prompt.
After a few tries, I don''t think my login/ssh/ls test case had anything
to do with it.

I am mounting some NFS shares from the dom0 at boot time, so that might be the
trigger that hits the 
    "kernel BUG at arch/x86/xen/mmu.c:1831!..."
problem.

-Bruce

PV Guest config

root@ServerLLX:/home/boris/ubuntu# cat u10.cfg
name="Ubuntu10PV"
memory = 2048

kernel = "/home/boris/ubuntu/vmlinuz-2.6.37-rc1"
ramdisk = "/home/boris/ubuntu/initrd.img-2.6.37-rc1"
disk = [''phy:/dev/sda7,xvda,w'']
vif  = [''bridge=br0'']
extra = "root=/dev/xvda1 ro console=hvc0"

# vfb =  [''type=vnc,vncunused=1'']

root@ServerLLX:/home/boris/ubuntu# xm create -c  u10.cfg
Using config file "./u10.cfg".
Started domain Ubuntu10PV
 (id=4)
                                Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.37-rc1 (root@ServerLLX) (gcc version 4.4.3 (Ubuntu
4.4.3-4ubuntu5) ) #1 SMP Fri Nov 5 19:29:09 MSK 2010

Command line: root=/dev/xvda1 ro console=hvc0
ACPI in unprivileged domain disabled
released 0 pages of unused memory
BIOS-provided physical RAM map:

 Xen: 0000000000000000 - 00000000000a0000 (usable)
 Xen: 00000000000a0000 - 0000000000100000 (reserved)
 Xen: 0000000000100000 - 0000000080800000 (usable)
NX (Execute Disable) protection: active

DMI not present or invalid.
No AGP bridge found
last_pfn = 0x80800 max_arch_pfn = 0x400000000
init_memory_mapping: 0000000000000000-0000000080800000

RAMDISK: 01e98000 -
 0e9fa000
No NUMA configuration found
Faking a node at 0000000000000000-0000000080800000
Initmem setup node 0 0000000000000000-0000000080800000
  NODE_DATA [000000007ffec000 - 000000007fffffff]

Zone PFN ranges:
  DMA      0x00000010 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   empty
Movable zone start PFN for each node
early_node_map[2] active PFN ranges

    0: 0x00000010 -> 0x000000a0
    0: 0x00000100 -> 0x00080800
SMP: Allowing 1 CPUs, 0 hotplug CPUs
No local APIC present
APIC: disable apic facility
APIC: switched to apic NOOP

PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
Allocating PCI resources starting at 80800000 (gap: 80800000:7f800000)
Booting paravirtualized kernel on Xen

Xen version: 4.0.1 (preserve-AD)
setup_percpu: NR_CPUS:256
 nr_cpumask_bits:256 nr_cpu_ids:1 nr_node_ids:1
PERCPU: Embedded 28 pages/cpu @ffff88007ffbd000 s82752 r8192 d23744 u114688
pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
pcpu-alloc: [0] 0 

Built 1 zonelists in Node order, mobility grouping on.  Total pages: 519022
Policy zone: DMA32
Kernel command line: root=/dev/xvda1 ro console=hvc0
PID hash table entries: 4096 (order: 3, 32768 bytes)

Checking aperture...
No AGP bridge found
Memory: 1835560k/2105344k available (4462k kernel code, 448k absent, 269336k
reserved, 6442k data, 1564k init)
Hierarchical RCU implementation.
    RCU-based detection of stalled CPUs is disabled.

NR_IRQS:16640 nr_irqs:256 16
Console: colour dummy device 80x25
console [tty0] enabled
console [hvc0] enabled
allocated 22282240 bytes of page_cgroup
please try ''cgroup_disable=memory'' option if you
don''t want memory cgroups
installing Xen timer for CPU
 0
Detected 2833.068 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency..
5666.13 BogoMIPS (lpj=2833068)
pid_max: default: 32768 minimum: 301
Security Framework initialized

SELinux:  Initializing.
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)

Mount-cache hash table entries: 256
Initializing cgroup subsys debug
Initializing cgroup subsys ns
ns_cgroup deprecated: consider using the ''clone_children'' flag
without the ns_cgroup.

Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
CPU: Unsupported number of siblings 4

Performance Events: unsupported p6 CPU model 23 no PMU driver, software events
only.
SMP alternatives: switching to UP code
Freeing SMP
 alternatives: 20k freed
cpu 0 spinlock event irq 254
Brought up 1 CPUs
Grant table initialized
regulator: core version 0.5
regulator: dummy: 
Time: 165:165:165  Date: 165/165/65

NET: Registered protocol family 16
PCI: setting up Xen PCI frontend stub
bio: create slab <bio-0> at 0
ACPI: Interpreter disabled.
xen_balloon: Initialising balloon driver.
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub

usbcore: registered new device driver usb
PCI: System does not support PCI
PCI: System does not support PCI
NetLabel: Initializing
NetLabel:  domain hash size = 128

NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
Switching to clocksource xen
pnp: PnP ACPI: disabled
NET: Registered protocol family 2

IP route cache hash table entries:
 65536 (order: 7, 524288 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)

TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
UDP hash table entries: 1024 (order: 3, 32768 bytes)
UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)

NET: Registered protocol family 1
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 208264k freed
platform rtc_cmos: registered platform RTC device (no PNP device found)

audit: initializing netlink socket (disabled)
type=2000 audit(1289472066.106:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages

VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 3991
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)

io scheduler noop registered
io
 scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4

acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
pci-stub: invalid id string ""
Event-channel device installed.
Non-volatile memory driver v1.3
Linux agpgart interface v0.103

Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
brd: module loaded
loop: module loaded
Fixed MDIO Bus: probed
Initialising Xen virtual ethernet driver.

ehci_hcd: USB 2.0 ''Enhanced'' Host Controller (EHCI) Driver
ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver

PNP: No PS/2 controller found. Probing ports directly.
i8042.c: No controller found.
mice: PS/2 mouse device common for all mice

rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
rtc_cmos: probe of rtc_cmos
 failed with error -38
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com
cpuidle: using governor ladder

cpuidle: using governor menu
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
IPVS: Registered protocols (TCP, UDP, AH, ESP)

IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
IPVS: ipvs loaded.
ip_tables: (C) 2000-2006 Netfilter Core Team
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
Bridge firewalling registered
Ebtables v2.0 registered
Registering the dns_resolver key type
registered taskstats version 1
IMA: No TPM chip found, activating TPM-bypass!

blkfront: xvda: barriers disabled
 xvda: xvda1 xvda2
XENBUS: Device with no driver: device/console/0
  Magic number:
 1:252:3141
drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
Initalizing network drop monitor service
Freeing unused kernel memory: 1564k freed
Write protecting the kernel read-only data: 10240k

Freeing unused kernel memory: 1664k freed
Freeing unused kernel memory: 1904k freed
Loading, please wait...
mount: mounting none on /dev failed: No such device
udevd (63): /proc/63/oom_adj is deprecated, please use /proc/63/oom_score_adj
instead.

Begin: Loading essential drivers... ...
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...

Done.
EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...

Done.

Ubuntu 10.10 boris-HVM-domU
 hvc0

boris-HVM-domU login: root
Password: 
Linux boris-HVM-domU 2.6.37-rc1 #1 SMP Fri Nov 5 19:29:09 MSK 2010 x86_64
GNU/Linux
Ubuntu 10.10

Welcome to Ubuntu!
 * Documentation:  https://help.ubuntu.com/

root@boris-HVM-domU:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1             12G  3.9G  7.1G  36% /
none                 1001M  116K 1001M   1% /dev
none                 1001M     0 1001M   0% /dev/shm

none                 1001M   84K
 1001M   1% /var/run
none                 1001M     0 1001M   0% /var/lock

Boris.

--- On Wed, 11/10/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel
paging request

To: xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>

Date: Wednesday, November 10, 2010, 5:15 PM

This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build 22241
and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.

The domU console output hung up prior to spitting out a prompt post
 root login. I logged into the domU via ssh and ran an ''ls'' of
an NFS mounted dir and

this immediately came out on the domU console:

[  480.467525] BUG: unable to handle kernel paging request at ffff880077e9f030
[  480.467547] IP: [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
[  480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065

[  480.467587] Oops: 0003 [#1] SMP 
[  480.467601] last sysfs file: /sys/kernel/uevent_seqnum
[  480.467611] CPU 3 
[  480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront

[  480.467676] 
[  480.467684] Pid: 3957, comm: zsh Not tainted
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
[  480.467695] RIP: e030:[<ffffffff810355f3>]  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d

[  480.467710] RSP: e02b:ffff880077ebfbd8  EFLAGS: 00010202
[  480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX: 800000004d686167
[  480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI: ffff8800763ddb80

[  480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09: ffffea0001a3b2d8
[  480.467748] R10: 0000000000000100 R11: 0000000000000000 R12: 00007f5cb26069d0
[  480.467759] R13: 0000000000000001 R14: 0000000000000000 R15: ffff8800763ddb80

[  480.467772] FS:  00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
knlGS:0000000000000000
[  480.467783] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4: 0000000000002660

[  480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task
ffff880078b80000)

[  480.467832] Stack:
[  480.467838]  0000000000000100 ffffea0001a3b2d8 7000000000000000
0000000000000000
[  480.467862]  ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
ffffffff810e2200
[  480.467886]  ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
0000000000000001

[  480.467910] Call Trace:
[  480.467921]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
[  480.467934]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[  480.467946]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd

[  480.467958]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
[  480.467972]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
[  480.467984]  [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf

[  480.467996]  [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
[  480.468009]  [<ffffffff810072a2>] ? check_events+0x12/0x20
[  480.468020]  [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1

[  480.468034]  [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
[  480.468045]  [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
[  480.468059]  [<ffffffff810072a2>] ? check_events+0x12/0x20

[  480.468072]  [<ffffffff81443615>] page_fault+0x25/0x30
[  480.468085]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
[  480.468096]  [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
[  480.468108]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80

[  480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89 f4 53 48
89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18 <48>
89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2

[  480.468323] RIP  [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
[  480.468323]  RSP <ffff880077ebfbd8>
[  480.468323] CR2: ffff880077e9f030
[  480.468323] ---[ end trace 6d1623321ecd50ac ]---

The domU config is attached as config.2.6.37_domU_debug

-Bruce

-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com

http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Nov-11 15:08 UTC

head link

[Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Wed, Nov 10, 2010 at 2:30 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
> On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com>
wrote:
> >
> > This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable
build
> 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
> >
> > The domU console output hung up prior to spitting out a prompt post
root
> login. I logged into the domU via ssh and ran an ''ls'' of
an NFS mounted dir
> and
> > this immediately came out on the domU console:
> >
> > [  480.467525] BUG: unable to handle kernel paging request at
> ffff880077e9f030
> > [  480.467547] IP: [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> > [  480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE
8010000077e9f065
> > [  480.467587] Oops: 0003 [#1] SMP
> > [  480.467601] last sysfs file: /sys/kernel/uevent_seqnum
> > [  480.467611] CPU 3
> > [  480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
> xen_blkfront
> > [  480.467676]
> > [  480.467684] Pid: 3957, comm: zsh Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> > [  480.467695] RIP: e030:[<ffffffff810355f3>] 
[<ffffffff810355f3>]
> ptep_set_access_flags+0x27/0x4d
> > [  480.467710] RSP: e02b:ffff880077ebfbd8  EFLAGS: 00010202
> > [  480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX:
> 800000004d686167
> > [  480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI:
> ffff8800763ddb80
> > [  480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09:
> ffffea0001a3b2d8
> > [  480.467748] R10: 0000000000000100 R11: 0000000000000000 R12:
> 00007f5cb26069d0
> > [  480.467759] R13: 0000000000000001 R14: 0000000000000000 R15:
> ffff8800763ddb80
> > [  480.467772] FS:  00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
> knlGS:0000000000000000
> > [  480.467783] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> > [  480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4:
> 0000000000002660
> > [  480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> > [  480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> > [  480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000,
task
> ffff880078b80000)
> > [  480.467832] Stack:
> > [  480.467838]  0000000000000100 ffffea0001a3b2d8 7000000000000000
> 0000000000000000
> > [  480.467862]  ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
> ffffffff810e2200
> > [  480.467886]  ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
> 0000000000000001
> > [  480.467910] Call Trace:
> > [  480.467921]  [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
> > [  480.467934]  [<ffffffff81007c09>] ?
__spin_time_accum+0x21/0x37
> > [  480.467946]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> > [  480.467958]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
> > [  480.467972]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
> > [  480.467984]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> > [  480.467996]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> > [  480.468009]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> > [  480.468020]  [<ffffffff8100728f>] ?
xen_restore_fl_direct_end+0x0/0x1
> > [  480.468034]  [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
> > [  480.468045]  [<ffffffff81006c3d>] ?
xen_force_evtchn_callback+0xd/0xf
> > [  480.468059]  [<ffffffff810072a2>] ? check_events+0x12/0x20
> > [  480.468072]  [<ffffffff81443615>] page_fault+0x25/0x30
> > [  480.468085]  [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
> > [  480.468096]  [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
> > [  480.468108]  [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
> > [  480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49
89
> f4 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74
> 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
> > [  480.468323] RIP  [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
> > [  480.468323]  RSP <ffff880077ebfbd8>
> > [  480.468323] CR2: ffff880077e9f030
> > [  480.468323] ---[ end trace 6d1623321ecd50ac ]---
> >
> > The domU config is attached as config.2.6.37_domU_debug
> >
> >
> > -Bruce
>
> Here''s another attempt from the same config with the full domU
boot logs:
>
> Started domain dpm (id=4)
>                         [    0.000000] Initializing cgroup subsys cpuset
> [    0.000000] Initializing cgroup subsys cpu
> [    0.000000] Linux version
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug (root@opal) (gcc version
> 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Tue Nov 9 20:26:03 PST 2010
> [    0.000000] Command line:
> root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro   swiotlb=force
> noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
> ip=:127.0.255.255::::eth0:dhcp
> [    0.000000] ACPI in unprivileged domain disabled
> [    0.000000] released 0 pages of unused memory
> [    0.000000] BIOS-provided physical RAM map:
> [    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
> [    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
> [    0.000000]  Xen: 0000000000100000 - 0000000080800000 (usable)
> [    0.000000] NX (Execute Disable) protection: active
> [    0.000000] DMI not present or invalid.
> [    0.000000] No AGP bridge found
> [    0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
> [    0.000000] Scanning 0 areas for low memory corruption
> [    0.000000] init_memory_mapping: 0000000000000000-0000000080800000
> [    0.000000] RAMDISK: 01d95000 - 02fa9000
> [    0.000000] No NUMA configuration found
> [    0.000000] Faking a node at 0000000000000000-0000000080800000
> [    0.000000] Initmem setup node 0 0000000000000000-0000000080800000
> [    0.000000]   NODE_DATA [000000007fffb000 - 000000007fffffff]
> [    0.000000] Zone PFN ranges:
> [    0.000000]   DMA      0x00000010 -> 0x00001000
> [    0.000000]   DMA32    0x00001000 -> 0x00100000
> [    0.000000]   Normal   empty
> [    0.000000] Movable zone start PFN for each node
> [    0.000000] early_node_map[2] active PFN ranges
> [    0.000000]     0: 0x00000010 -> 0x000000a0
> [    0.000000]     0: 0x00000100 -> 0x00080800
> [    0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
> [    0.000000] No local APIC present
> [    0.000000] APIC: disable apic facility
> [    0.000000] APIC: switched to apic NOOP
> [    0.000000] PM: Registered nosave memory: 00000000000a0000 -
> 0000000000100000
> [    0.000000] Allocating PCI resources starting at 80800000 (gap:
> 80800000:7f800000)
> [    0.000000] Booting paravirtualized kernel on Xen
> [    0.000000] Xen version: 4.1-unstable (preserve-AD)
> [    0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
> nr_cpu_ids:6 nr_node_ids:1
> [    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
> r8192 d23744 u114688
> [    0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
> [    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
> [    0.000000] Built 1 zonelists in Node order, mobility grouping on.
> Total pages: 519022
> [    0.000000] Policy zone: DMA32
> [    0.000000] Kernel command line:
> root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro   swiotlb=force
> noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
> ip=:127.0.255.255::::eth0:dhcp
> [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
> [    0.000000] Placing 64MB software IO TLB between ffff880079400000 -
> ffff88007d400000
> [    0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
> [    0.000000] Memory: 1959896k/2105344k available (4398k kernel code,
> 448k absent, 145000k reserved, 6327k data, 756k init)
> [    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
> CPUs=6, Nodes=1
> [    0.000000] Hierarchical RCU implementation.
> [    0.000000]  RCU-based detection of stalled CPUs is disabled.
> [    0.000000] NR_IRQS:4352 nr_irqs:320 16
> [    0.000000] Console: colour dummy device 80x25
> [    0.000000] console [tty0] enabled
> [    0.000000] console [hvc0] enabled
> [    0.000000] installing Xen timer for CPU 0
> [    0.000000] Detected 2660.064 MHz processor.
> [    0.000999] Calibrating delay loop (skipped), value calculated
> using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
> [    0.000999] pid_max: default: 32768 minimum: 301
> [    0.000999] Security Framework initialized
> [    0.000999] SELinux:  Initializing.
> [    0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152
> bytes)
> [    0.001408] Inode-cache hash table entries: 131072 (order: 8, 1048576
> bytes)
> [    0.001635] Mount-cache hash table entries: 256
> [    0.002195] Initializing cgroup subsys ns
> [    0.002208] ns_cgroup deprecated: consider using the
> ''clone_children'' flag without the ns_cgroup.
> [    0.002220] Initializing cgroup subsys cpuacct
> [    0.002231] Initializing cgroup subsys freezer
> [    0.002356] CPU: Unsupported number of siblings 16
> [    0.002362] Performance Events: unsupported p6 CPU model 26 no PMU
> driver, software events only.
> [    0.002483] SMP alternatives: switching to UP code
> [    0.004119] cpu 0 spinlock event irq 318
> [    0.004765] NMI watchdog failed to create perf event on cpu0:
> fffffffffffffffe
> [    0.005050] installing Xen timer for CPU 1
> [    0.005157] cpu 1 spinlock event irq 312
> [    0.005194] SMP alternatives: switching to SMP code
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.006158] NMI watchdog failed to create perf event on cpu1:
> fffffffffffffffe
> [    0.006474] installing Xen timer for CPU 2
> [    0.006585] cpu 2 spinlock event irq 306
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.006816] NMI watchdog failed to create perf event on cpu2:
> fffffffffffffffe
> [    0.006999] installing Xen timer for CPU 3
> [    0.007037] cpu 3 spinlock event irq 300
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.007300] NMI watchdog failed to create perf event on cpu3:
> fffffffffffffffe
> [    0.007534] installing Xen timer for CPU 4
> [    0.007636] cpu 4 spinlock event irq 294
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.007874] NMI watchdog failed to create perf event on cpu4:
> fffffffffffffffe
> [    0.008041] installing Xen timer for CPU 5
> [    0.008083] cpu 5 spinlock event irq 288
> [    0.000999] CPU: Unsupported number of siblings 16
> [    0.008344] NMI watchdog failed to create perf event on cpu5:
> fffffffffffffffe
> [    0.008356] Brought up 6 CPUs
> [    0.009181] kworker/u:0 used greatest stack depth: 5696 bytes left
> [    0.012051] Grant table initialized
> [    0.030978] Time: 165:165:165  Date: 165/165/65
> [    0.031229] NET: Registered protocol family 16
> [    0.036672] PCI: setting up Xen PCI frontend stub
> [    0.037225] kworker/u:0 used greatest stack depth: 5528 bytes left
> [    0.074302] bio: create slab <bio-0> at 0
> [    0.075385] ACPI: Interpreter disabled.
> [    0.077056] xen_balloon: Initialising balloon driver.
> [    0.079323] vgaarb: loaded
> [    0.081270] usbcore: registered new interface driver usbfs
> [    0.081283] usbcore: registered new interface driver hub
> [    0.081283] usbcore: registered new device driver usb
> [    0.083183] PCI: System does not support PCI
> [    0.084007] PCI: System does not support PCI
> [    0.085236] NetLabel: Initializing
> [    0.085236] NetLabel:  domain hash size = 128
> [    0.085236] NetLabel:  protocols = UNLABELED CIPSOv4
> [    0.085313] NetLabel:  unlabeled traffic allowed by default
> [    0.086544] Switching to clocksource xen
> [    0.106256] pnp: PnP ACPI: disabled
> [    0.129799] NET: Registered protocol family 2
> [    0.130174] IP route cache hash table entries: 65536 (order: 7, 524288
> bytes)
> [    0.131882] TCP established hash table entries: 262144 (order: 10,
> 4194304 bytes)
> [    0.132891] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
> [    0.133179] TCP: Hash tables configured (established 262144 bind 65536)
> [    0.133191] TCP reno registered
> [    0.133301] UDP hash table entries: 1024 (order: 3, 32768 bytes)
> [    0.133407] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
> [    0.133914] NET: Registered protocol family 1
> [    0.134582] RPC: Registered udp transport module.
> [    0.134590] RPC: Registered tcp transport module.
> [    0.134595] RPC: Registered tcp NFSv4.1 backchannel transport module.
> [    0.134892] Trying to unpack rootfs image as initramfs...
> [    0.178824] Freeing initrd memory: 18512k freed
> [    0.268798] DMA-API: preallocated 32768 debug entries
> [    0.268810] DMA-API: debugging enabled by kernel config
> [    0.271181] platform rtc_cmos: registered platform RTC device (no
> PNP device found)
> [    0.285410] Machine check injector initialized
> [    0.294732] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
> [    0.294803] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
> [    0.294893] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
> [    0.294944] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
> [    0.294998] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
> [    0.295043] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
> [    0.295590] microcode: Microcode Update Driver: v2.00
> <tigran@aivazian.fsnet.co.uk>, Peter Oruba
> [    0.295607] Scanning for low memory corruption every 60 seconds
> [    0.298813] audit: initializing netlink socket (disabled)
> [    0.298940] type=2000 audit(1289403082.795:1): initialized
> [    0.333762] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [    0.376985] VFS: Disk quotas dquot_6.5.2
> [    0.377853] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> [    0.383121] NTFS driver 2.1.29 [Flags: R/W].
> [    0.385014] msgmni has been set to 3864
> [    0.389953] Block layer SCSI generic (bsg) driver version 0.4
> loaded (major 253)
> [    0.389970] io scheduler noop registered
> [    0.389979] io scheduler deadline registered
> [    0.390666] io scheduler cfq registered (default)
> [    0.392486] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> [    0.406238] pcifront pci-0: Installing PCI frontend
> [    0.406967] pcifront pci-0: Creating PCI Frontend Bus 0000:00
> [    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/0
> [    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/2
> [    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/3
> [    0.417634] pcifront pci-0: claiming resource 0000:00:00.0/4
> [    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/0
> [    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/2
> [    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/3
> [    0.417634] pcifront pci-0: claiming resource 0000:00:01.0/4
> [    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/0
> [    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/2
> [    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/3
> [    0.417634] pcifront pci-0: claiming resource 0000:00:02.0/4
> [    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/0
> [    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/2
> [    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/3
> [    0.417634] pcifront pci-0: claiming resource 0000:00:03.0/4
> [    0.731775] Non-volatile memory driver v1.3
> [    0.731790] Linux agpgart interface v0.103
> [    0.733731] [drm] Initialized drm 1.1.0 20060810
> [    0.733752] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [    0.827202] brd: module loaded
> [    0.840788] loop: module loaded
> [    0.841965] Fixed MDIO Bus: probed
> [    0.844025] ehci_hcd: USB 2.0 ''Enhanced'' Host
Controller (EHCI) Driver
> [    0.844651] ohci_hcd: USB 1.1 ''Open'' Host Controller
(OHCI) Driver
> [    0.845194] uhci_hcd: USB Universal Host Controller Interface driver
> [    0.846985] usbcore: registered new interface driver usblp
> [    0.847763] usbcore: registered new interface driver libusual
> [    0.849433] PNP: No PS/2 controller found. Probing ports directly.
> [    0.850279] i8042.c: No controller found.
> [    0.850883] mice: PS/2 mouse device common for all mice
> [    0.853446] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
> [    0.854066] rtc_cmos: probe of rtc_cmos failed with error -38
> [    0.855928] cpuidle: using governor ladder
> [    0.855941] cpuidle: using governor menu
> [    0.856029] Netfilter messages via NETLINK v0.30.
> [    0.856136] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
> [    0.857888] ctnetlink v0.93: registering with nfnetlink.
> [    0.860285] ip_tables: (C) 2000-2006 Netfilter Core Team
> [    0.860677] TCP cubic registered
> [    0.860687] Initializing XFRM netlink socket
> [    0.863286] NET: Registered protocol family 10
> [    0.867216] ip6_tables: (C) 2000-2006 Netfilter Core Team
> [    0.867612] IPv6 over IPv4 tunneling driver
> [    0.871326] NET: Registered protocol family 17
> [    0.871566] Registering the dns_resolver key type
> [    0.873668] registered taskstats version 1
> [    0.873912] kmemleak: Kernel memory leak detector initialized
> [    0.873928] kmemleak: Automatic memory scanning thread started
> [    0.875057] XENBUS: Device with no driver: device/vbd/51712
> [    0.875057] XENBUS: Device with no driver: device/vif/0
> [    0.875057] XENBUS: Device with no driver: device/console/0
> [    0.875057]   Magic number: 1:252:3141
> [   13.388958] Freeing unused kernel memory: 756k freed
> [   13.389190] Write protecting the kernel read-only data: 10240k
> [   13.398774] Freeing unused kernel memory: 1724k freed
> [   13.400663] Freeing unused kernel memory: 1992k freed
> [   13.494573] all_generic_ide used greatest stack depth: 5472 bytes left
> [   13.519388] udevd (1111): /proc/1111/oom_adj is deprecated, please
> use /proc/1111/oom_score_adj instead.
> Begin: Loading essential drivers... ...
> [   13.559435] blkid used greatest stack depth: 5408 bytes left
> [   13.568391] blkfront: xvda: barriers disabled
> [   13.574762] blkid used greatest stack depth: 5264 bytes left
> [   13.580813]  xvda: xvda1 xvda2 < xvda5 >
> [   13.645562] modprobe used greatest stack depth: 5088 bytes left
> [   13.653160] Initialising Xen virtual ethernet driver.
> [   13.659540] blkid used greatest stack depth: 4960 bytes left
> [   13.735467] blkid used greatest stack depth: 4752 bytes left
> Done.
> Begin: Running /scripts/init-premount ...
> Done.
> Begin: Mounting root file system... ...
> Begin: Running /scripts/local-top ...
> Done.
> Begin: Running /scripts/local-premount ...
> Done.
> [   13.903774] EXT4-fs (xvda1): INFO: recovery required on readonly
> filesystem
> [   13.903799] EXT4-fs (xvda1): write access will be enabled during
> recovery
> [   14.033719] EXT4-fs (xvda1): recovery complete
> [   14.034966] EXT4-fs (xvda1): mounted filesystem with ordered data
> mode. Opts: (null)
> [   14.035510] exe used greatest stack depth: 4112 bytes left
> Begin: Running /scripts/local-bottom ...
> Done.
> Done.
> Begin: Running /scripts/init-bottom ...
> Done.
> init: ureadahead main process (1214) terminated with status 5
> fsck from util-linux-ng 2.17.2
> e2fsck 1.41.11 (14-Mar-2010)
> /dev/xvda1: clean, 56314/288576 files, 287743/1153280 blocks
>  * Enable login on console                                               [
> OK ]
>  * Check for hostname changes...                                         [
> OK ]
>   ...done.
>  * Setting sensors limits
>         No sensors found!
> Make sure you loaded all the kernel drivers you need.
> Try sensors-detect to find out which these are.
> No sensors found!
> Make sure you loaded all the kernel drivers you need.
> Try sensors-detect to find out which these are.
>                                                                         [
> OK ]
>  * Initializing random number generator...                               [
> OK ]
>  * Setting up X server socket directory /tmp/.X11-unix...                [
> OK ]
>  * Setting up ICE socket directory /tmp/.ICE-unix...                     [
> OK ]
>  * Not starting fancontrol; run pwmconfig first.
>  * Starting internet superserver inetd                                   [
> OK ]
> Starting openntpd: ntpd.
>  * Starting network management services:
>  * Setting core file location...                                         [
> OK ]
>  * Restoring resolver state...                                           [
> OK ]
>  * Checking tomcat configuration
>   ...done.
>  * Starting Tomcat servlet engine tomcat6
>         Using CATALINA_BASE:   /var/lib/tomcat6
> Using CATALINA_HOME:   /usr/share/tomcat6
> Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
> Using JRE_HOME:        /usr/lib/jvm/java-6-openjdk
> Using CLASSPATH:       /usr/share/tomcat6/bin/bootstrap.jar
>                                                                         [
> OK ]
> collectd has been disabled - see /etc/default/collectd.
>  * Recording successful boot for GRUB
>   ...done.
>  * Running local boot scripts (/etc/rc.local)                            [
> OK ]
> mountall: Plymouth command failed
> mountall: Disconnected from Plymouth
>
> 6.0.16-01
> kaan-04-dpm login: root
> Password:
> [   42.646538] ------------[ cut here ]------------
> [   42.646556] kernel BUG at arch/x86/xen/mmu.c:1831!
> [   42.646566] invalid opcode: 0000 [#1] SMP
> [   42.646582] last sysfs file: /sys/kernel/uevent_seqnum
> [   42.646592] CPU 0
> [   42.646598] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
> xen_netfront xen_blkfront
> [   42.646665]
> [   42.646674] Pid: 3916, comm: lsb_release Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [   42.646688] RIP: e030:[<ffffffff81004bce>] 
[<ffffffff81004bce>]
> pin_pagetable_pfn+0x31/0x37
> [   42.646711] RSP: e02b:ffff88007595bce8  EFLAGS: 00010282
> [   42.646721] RAX: 00000000ffffffea RBX: 000000000007751e RCX:
> 0000000000000001
> [   42.646732] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
> 00000000deadbeef
> [   42.646744] RBP: ffff88007595bd08 R08: 00003ffffffff000 R09:
> ffff880000000000
> [   42.646756] R10: 00000000deadbeef R11: dead000000200200 R12:
> 0000000000000003
> [   42.646768] R13: ffff880075c90150 R14: ffff8800781d9398 R15:
> ffff880075161d40
> [   42.646784] FS:  00007fee63b01700(0000) GS:ffff88007ff40000(0000)
> knlGS:0000000000000000
> [   42.646797] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [   42.646807] CR2: 00007faac54182b8 CR3: 00000000771dd000 CR4:
> 0000000000002660
> [   42.646819] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [   42.646832] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [   42.646844] Process lsb_release (pid: 3916, threadinfo
> ffff88007595a000, task ffff880075161d40)
> [   42.646857] Stack:
> [   42.646864]  ffff880000000000 00000000000484f0 ffff8800777c6d80
> 000000000007751e
> [   42.646892]  ffff88007595bd28 ffffffff810066a8 000000000007751e
> ffff8800777c6d80
> [   42.646919]  ffff88007595bd38 ffffffff810066db ffff88007595bd78
> ffffffff810e048f
> [   42.646946] Call Trace:
> [   42.646959]  [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
> [   42.646973]  [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
> [   42.646989]  [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
> [   42.647003]  [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
> [   42.647017]  [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
> [   42.647033]  [<ffffffff8100eb84>] ?
> arch_get_unmapped_area_topdown+0x4a/0x28a
> [   42.647052]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [   42.647059]  [<ffffffff8102b78a>] ?
pvclock_clocksource_read+0x4b/0xb4
> [   42.647059]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [   42.647059]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [   42.647059]  [<ffffffff81443615>] page_fault+0x25/0x30
> [   42.647059] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
> e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff
> 85 c0 74 04 <0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db
48
> 8b 3d
> [   42.647059] RIP  [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
> [   42.647059]  RSP <ffff88007595bce8>
> [   42.647059] ---[ end trace 3fbbffee414a5166 ]---
>

Here''s another full boot that I just let sit at the login prompt. This
is
using the pcifront-fixes kernel:

Started domain dpm (id=1)
                         [    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version
2.6.37-rc1-pvops-pcifront-fixes-domu-6.0.bedge-debug (root@opal) (gcc
version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Wed Nov 10 17:11:41 PST 2010
[    0.000000] Command line: root=UUID=1e67e680-4108-4e11-8bc4-844dad448902
ro   swiotlb=force noirqbalance noirqbalance console=hvc0,115200n8
--flow_control=r ip=:127.0.255.255::::eth0:dhcp
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] released 0 pages of unused memory
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000080800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[    0.000000] Scanning 0 areas for low memory corruption
[    0.000000] init_memory_mapping: 0000000000000000-0000000080800000
[    0.000000] RAMDISK: 01d95000 - 02db9000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-0000000080800000
[    0.000000] Initmem setup node 0 0000000000000000-0000000080800000
[    0.000000]   NODE_DATA [000000007fffb000 - 000000007fffffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x000000a0
[    0.000000]     0: 0x00000100 -> 0x00080800
[    0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
[    0.000000] No local APIC present
[    0.000000] APIC: disable apic facility
[    0.000000] APIC: switched to apic NOOP
[    0.000000] PM: Registered nosave memory: 00000000000a0000 -
0000000000100000
[    0.000000] Allocating PCI resources starting at 80800000 (gap:
80800000:7f800000)
[    0.000000] Booting paravirtualized kernel on Xen
[    0.000000] Xen version: 4.1-unstable (preserve-AD)
[    0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:6
nr_node_ids:1
[    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752 r8192
d23744 u114688
[    0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total
pages: 519022
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:
root=UUID=1e67e680-4108-4e11-8bc4-844dad448902 ro   swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Placing 64MB software IO TLB between ffff880079400000 -
ffff88007d400000
[    0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
[    0.000000] Memory: 1961880k/2105344k available (4398k kernel code, 448k
absent, 143016k reserved, 6327k data, 756k init)
[    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=6, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU-based detection of stalled CPUs is disabled.
[    0.000000] NR_IRQS:4352 nr_irqs:320 16
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [hvc0] enabled
[    0.000000] installing Xen timer for CPU 0
[    0.000000] Detected 2660.054 MHz processor.
[    0.000999] Calibrating delay loop (skipped), value calculated using
timer frequency.. 5320.10 BogoMIPS (lpj=2660054)
[    0.000999] pid_max: default: 32768 minimum: 301
[    0.000999] Security Framework initialized
[    0.000999] SELinux:  Initializing.
[    0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152
bytes)
[    0.001410] Inode-cache hash table entries: 131072 (order: 8, 1048576
bytes)
[    0.001641] Mount-cache hash table entries: 256
[    0.002191] Initializing cgroup subsys ns
[    0.002204] ns_cgroup deprecated: consider using the
''clone_children''
flag without the ns_cgroup.
[    0.002216] Initializing cgroup subsys cpuacct
[    0.002227] Initializing cgroup subsys freezer
[    0.002354] CPU: Unsupported number of siblings 16
[    0.002361] Performance Events: unsupported p6 CPU model 26 no PMU
driver, software events only.
[    0.002524] SMP alternatives: switching to UP code
[    0.004910] cpu 0 spinlock event irq 318
[    0.005490] NMI watchdog failed to create perf event on cpu0:
fffffffffffffffe
[    0.005817] installing Xen timer for CPU 1
[    0.005857] cpu 1 spinlock event irq 312
[    0.005893] SMP alternatives: switching to SMP code
[    0.000999] CPU: Unsupported number of siblings 16
[    0.007158] NMI watchdog failed to create perf event on cpu1:
fffffffffffffffe
[    0.007474] installing Xen timer for CPU 2
[    0.007585] cpu 2 spinlock event irq 306
[    0.000999] CPU: Unsupported number of siblings 16
[    0.007909] NMI watchdog failed to create perf event on cpu2:
fffffffffffffffe
[    0.008078] installing Xen timer for CPU 3
[    0.008193] cpu 3 spinlock event irq 300
[    0.000999] CPU: Unsupported number of siblings 16
[    0.008450] NMI watchdog failed to create perf event on cpu3:
fffffffffffffffe
[    0.008685] installing Xen timer for CPU 4
[    0.008789] cpu 4 spinlock event irq 294
[    0.000999] CPU: Unsupported number of siblings 16
[    0.008999] NMI watchdog failed to create perf event on cpu4:
fffffffffffffffe
[    0.009210] installing Xen timer for CPU 5
[    0.009252] cpu 5 spinlock event irq 288
[    0.000999] CPU: Unsupported number of siblings 16
[    0.009530] NMI watchdog failed to create perf event on cpu5:
fffffffffffffffe
[    0.009541] Brought up 6 CPUs
[    0.010295] kworker/u:0 used greatest stack depth: 5696 bytes left
[    0.013138] Grant table initialized
[    0.032051] Time: 165:165:165  Date: 165/165/65
[    0.032378] NET: Registered protocol family 16
[    0.037699] PCI: setting up Xen PCI frontend stub
[    0.039132] kworker/u:0 used greatest stack depth: 5528 bytes left
[    0.067272] bio: create slab <bio-0> at 0
[    0.068203] ACPI: Interpreter disabled.
[    0.069064] xen_balloon: Initialising balloon driver.
[    0.070332] vgaarb: loaded
[    0.072248] usbcore: registered new interface driver usbfs
[    0.072279] usbcore: registered new interface driver hub
[    0.073080] usbcore: registered new device driver usb
[    0.075072] PCI: System does not support PCI
[    0.075137] PCI: System does not support PCI
[    0.076112] NetLabel: Initializing
[    0.076120] NetLabel:  domain hash size = 128
[    0.076126] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.076223] NetLabel:  unlabeled traffic allowed by default
[    0.077209] Switching to clocksource xen
[    0.093954] pnp: PnP ACPI: disabled
[    0.138850] NET: Registered protocol family 2
[    0.139329] IP route cache hash table entries: 65536 (order: 7, 524288
bytes)
[    0.141816] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[    0.143253] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.143474] TCP: Hash tables configured (established 262144 bind 65536)
[    0.143487] TCP reno registered
[    0.143537] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[    0.143593] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[    0.144245] NET: Registered protocol family 1
[    0.145189] RPC: Registered udp transport module.
[    0.145203] RPC: Registered tcp transport module.
[    0.145213] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.145645] Trying to unpack rootfs image as initramfs...
[    0.186054] Freeing initrd memory: 16528k freed
[    0.270158] DMA-API: preallocated 32768 debug entries
[    0.270170] DMA-API: debugging enabled by kernel config
[    0.272584] platform rtc_cmos: registered platform RTC device (no PNP
device found)
[    0.285338] Machine check injector initialized
[    0.294148] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
[    0.294217] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
[    0.294308] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
[    0.294359] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
[    0.294427] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
[    0.294473] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
[    0.294990] microcode: Microcode Update Driver: v2.00 <
tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    0.295008] Scanning for low memory corruption every 60 seconds
[    0.297474] audit: initializing netlink socket (disabled)
[    0.297559] type=2000 audit(1289430805.716:1): initialized
[    0.331826] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.375119] VFS: Disk quotas dquot_6.5.2
[    0.375896] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.381318] NTFS driver 2.1.29 [Flags: R/W].
[    0.383311] msgmni has been set to 3864
[    0.387906] Block layer SCSI generic (bsg) driver version 0.4 loaded
(major 253)
[    0.387924] io scheduler noop registered
[    0.387933] io scheduler deadline registered
[    0.388724] io scheduler cfq registered (default)
[    0.390543] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.403338] pcifront pci-0: Installing PCI frontend
[    0.404082] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[    0.406134] pcifront pci-0: claiming resource 0000:00:00.0/0
[    0.406134] pcifront pci-0: claiming resource 0000:00:00.0/2
[    0.406134] pcifront pci-0: claiming resource 0000:00:00.0/3
[    0.406134] pcifront pci-0: claiming resource 0000:00:00.0/4
[    0.406134] pcifront pci-0: claiming resource 0000:00:01.0/0
[    0.406134] pcifront pci-0: claiming resource 0000:00:01.0/2
[    0.406134] pcifront pci-0: claiming resource 0000:00:01.0/3
[    0.406134] pcifront pci-0: claiming resource 0000:00:01.0/4
[    0.406134] pcifront pci-0: claiming resource 0000:00:02.0/0
[    0.406134] pcifront pci-0: claiming resource 0000:00:02.0/2
[    0.406134] pcifront pci-0: claiming resource 0000:00:02.0/3
[    0.406134] pcifront pci-0: claiming resource 0000:00:02.0/4
[    0.406134] pcifront pci-0: claiming resource 0000:00:03.0/0
[    0.406134] pcifront pci-0: claiming resource 0000:00:03.0/2
[    0.406134] pcifront pci-0: claiming resource 0000:00:03.0/3
[    0.406134] pcifront pci-0: claiming resource 0000:00:03.0/4
[    0.734074] Non-volatile memory driver v1.3
[    0.734089] Linux agpgart interface v0.103
[    0.735535] [drm] Initialized drm 1.1.0 20060810
[    0.735556] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.829048] brd: module loaded
[    0.842342] loop: module loaded
[    0.843465] Fixed MDIO Bus: probed
[    0.845827] ehci_hcd: USB 2.0 ''Enhanced'' Host Controller
(EHCI) Driver
[    0.846398] ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI)
Driver
[    0.846882] uhci_hcd: USB Universal Host Controller Interface driver
[    0.848574] usbcore: registered new interface driver usblp
[    0.849103] usbcore: registered new interface driver libusual
[    0.850786] PNP: No PS/2 controller found. Probing ports directly.
[    0.851626] i8042.c: No controller found.
[    0.852258] mice: PS/2 mouse device common for all mice
[    0.855068] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[    0.855717] rtc_cmos: probe of rtc_cmos failed with error -38
[    0.857602] cpuidle: using governor ladder
[    0.857619] cpuidle: using governor menu
[    0.857713] Netfilter messages via NETLINK v0.30.
[    0.857800] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    0.859562] ctnetlink v0.93: registering with nfnetlink.
[    0.861913] ip_tables: (C) 2000-2006 Netfilter Core Team
[    0.862342] TCP cubic registered
[    0.862354] Initializing XFRM netlink socket
[    0.864995] NET: Registered protocol family 10
[    0.869020] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    0.869455] IPv6 over IPv4 tunneling driver
[    0.873134] NET: Registered protocol family 17
[    0.873381] Registering the dns_resolver key type
[    0.875443] registered taskstats version 1
[    0.875615] kmemleak: Kernel memory leak detector initialized
[    0.875634] kmemleak: Automatic memory scanning thread started
[    0.877640] XENBUS: Device with no driver: device/vbd/51712
[    0.877654] XENBUS: Device with no driver: device/vif/0
[    0.877664] XENBUS: Device with no driver: device/console/0
[    0.877692]   Magic number: 1:252:3141
[   13.391082] Freeing unused kernel memory: 756k freed
[   13.391322] Write protecting the kernel read-only data: 10240k
[   13.401075] Freeing unused kernel memory: 1724k freed
[   13.402500] Freeing unused kernel memory: 1992k freed
Loading, please wait...
mount: mounting none on /dev failed: No such device
[   13.498289] all_generic_ide used greatest stack depth: 5472 bytes left
[   13.523488] udevd (1111): /proc/1111/oom_adj is deprecated, please use
/proc/1111/oom_score_adj instead.
Begin: Loading essential drivers... ...
[   13.546062] udevadm used greatest stack depth: 5088 bytes left
[   13.580639] blkfront: xvda: barriers disabled
[   13.594779]  xvda: xvda1 xvda2 < xvda5 >
[   13.669872] Initialising Xen virtual ethernet driver.
[   13.672323] blkid used greatest stack depth: 4960 bytes left
[   13.756667] blkid used greatest stack depth: 4752 bytes left
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
[   13.903338] EXT4-fs (xvda1): INFO: recovery required on readonly
filesystem
[   13.903363] EXT4-fs (xvda1): write access will be enabled during recovery
[   14.190114] EXT4-fs (xvda1): recovery complete
[   14.191367] EXT4-fs (xvda1): mounted filesystem with ordered data mode.
Opts: (null)
[   14.191951] exe used greatest stack depth: 4112 bytes left
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
init: ureadahead main process (1208) terminated with status 5
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
/dev/xvda1: clean, 56107/288576 files, 296111/1153280 blocks
 * Enable login on console                                               [
OK ]
 * Check for hostname changes...                                         [
OK ]
   ...done.
 * Setting sensors
limits                                                       No sensors
found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
                                                                         [
OK ]
 * Initializing random number generator...                               [
OK ]
 * Setting up X server socket directory /tmp/.X11-unix...                [
OK ]
 * Setting up ICE socket directory /tmp/.ICE-unix...                     [
OK ]
 * Not starting fancontrol; run pwmconfig first.
 * Starting internet superserver inetd                                   [
OK ]
Starting openntpd: ntpd.
 * Starting network management
services:                                         * Starting the system
activity data collector
sadc
Cannot open /var/log/sysstat/sa10: No such file or directory

[fail]
 * Setting core file location...                                         [
OK ]
 * Restoring resolver state...                                           [
OK ]
 * Checking tomcat configuration
   ...done.
 * Starting Tomcat servlet engine
tomcat6                                       Using CATALINA_BASE:
/var/lib/tomcat6
Using CATALINA_HOME:   /usr/share/tomcat6
Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
Using JRE_HOME:        /usr/lib/jvm/java-6-openjdk
Using CLASSPATH:       /usr/share/tomcat6/bin/bootstrap.jar
                                                                         [
OK ]
collectd has been disabled - see /etc/default/collectd.
 * Recording successful boot for GRUB
   ...done.
 * Running local boot scripts (/etc/rc.local)                            [
OK ]
mountall: Plymouth command failed
mountall: Disconnected from Plymouth

6.0.16-01
kaan-04-dpm login: [   47.548041] ------------[ cut here ]------------
[   47.548078] kernel BUG at arch/x86/xen/mmu.c:1831!
[   47.548093] invalid opcode: 0000 [#1] SMP
[   47.548114] last sysfs file: /sys/kernel/uevent_seqnum
[   47.548124] CPU 0
[   47.548130] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
[   47.548199]
[   47.548208] Pid: 3815, comm: java Not tainted
2.6.37-rc1-pvops-pcifront-fixes-domu-6.0.bedge-debug #1 /
[   47.548222] RIP: e030:[<ffffffff81004bce>]  [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[   47.548245] RSP: e02b:ffff880077fadce8  EFLAGS: 00010282
[   47.548255] RAX: 00000000ffffffea RBX: 0000000000078312 RCX:
0000000000000001
[   47.548266] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
00000000deadbeef
[   47.548278] RBP: ffff880077fadd08 R08: 00003ffffffff000 R09:
ffff880000000000
[   47.548290] R10: 00000000deadbeef R11: dead000000200200 R12:
0000000000000003
[   47.548302] R13: ffff880077f5e838 R14: ffff88007629ab80 R15:
ffff880078288750
[   47.548319] FS:  00007f2e8058c710(0000) GS:ffff88007ff40000(0000)
knlGS:0000000000000000
[   47.548332] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[   47.548342] CR2: 00007f2e60fe8728 CR3: 0000000077ec9000 CR4:
0000000000002660
[   47.548354] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[   47.548366] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[   47.548379] Process java (pid: 3815, threadinfo ffff880077fac000, task
ffff880078288750)
[   47.548391] Stack:
[   47.548398]  ffff880000000000 00000000000476fc ffff88007603ea00
0000000000078312
[   47.548426]  ffff880077fadd28 ffffffff810066a8 0000000000078312
ffff88007603ea00
[   47.548454]  ffff880077fadd38 ffffffff810066db ffff880077fadd78
ffffffff810e048f
[   47.548481] Call Trace:
[   47.548494]  [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[   47.548509]  [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[   47.548525]  [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[   47.548539]  [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
[   47.548554]  [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1
[   47.548571]  [<ffffffff81442ef2>] ?
_raw_spin_unlock_irqrestore+0x19/0x1c
[   47.548588]  [<ffffffff81048583>] ? try_to_wake_up+0x207/0x219
[   47.548603]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
[   47.548617]  [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1
[   47.548632]  [<ffffffff81442ef2>] ?
_raw_spin_unlock_irqrestore+0x19/0x1c
[   47.548648]  [<ffffffff81202bce>] ? rwsem_wake+0x39/0x43
[   47.548661]  [<ffffffff81206bc7>] ? call_rwsem_wake+0x17/0x30
[   47.548677]  [<ffffffff8106aa3d>] ? up_write+0x18/0x1a
[   47.548690]  [<ffffffff810e7378>] ? sys_mmap_pgoff+0x126/0x159
[   47.548705]  [<ffffffff81443615>] page_fault+0x25/0x30
[   47.548715] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d e0 48
89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff 85 c0 74 04
<0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48 8b 3d
[   47.548794] RIP  [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[   47.548794]  RSP <ffff880077fadce8>
[   47.548794] ---[ end trace ceae70ee7c69e40a ]---
[   63.942338] kmemleak: 4 new suspected memory leaks (see
/sys/kernel/debug/kmemleak)
[   92.806778] ------------[ cut here ]------------
[   92.806802] kernel BUG at arch/x86/xen/mmu.c:1831!
[   92.806817] invalid opcode: 0000 [#2] SMP
[   92.806841] last sysfs file: /sys/kernel/uevent_seqnum
[   92.806856] CPU 0
[   92.806865] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
[   92.806967]
[   92.806981] Pid: 3914, comm: rsyslogd Tainted: G      D
2.6.37-rc1-pvops-pcifront-fixes-domu-6.0.bedge-debug #1 /
[   92.807005] RIP: e030:[<ffffffff81004bce>]  [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[   92.807036] RSP: e02b:ffff880077869ce8  EFLAGS: 00010282
[   92.807053] RAX: 00000000ffffffea RBX: 0000000000077c61 RCX:
0000000000000001
[   92.807063] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
00000000deadbeef
[   92.807063] RBP: ffff880077869d08 R08: 00003ffffffff000 R09:
ffff880000000000
[   92.807063] R10: 00000000deadbeef R11: dead000000200200 R12:
0000000000000003
[   92.807063] R13: ffff8800760e44e0 R14: ffff880075ce8958 R15:
ffff880077015f10
[   92.807063] FS:  00007ffd53e39700(0000) GS:ffff88007ff40000(0000)
knlGS:0000000000000000
[   92.807063] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[   92.807063] CR2: 00007ffd53801330 CR3: 000000007821e000 CR4:
0000000000002660
[   92.807063] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[   92.807063] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[   92.807063] Process rsyslogd (pid: 3914, threadinfo ffff880077868000,
task ffff880077015f10)
[   92.807063] Stack:
[   92.807063]  ffff880000000000 0000000000047dad ffff880076138700
0000000000077c61
[   92.807063]  ffff880077869d28 ffffffff810066a8 0000000000077c61
ffff880076138700
[   92.807063]  ffff880077869d38 ffffffff810066db ffff880077869d78
ffffffff810e048f
[   92.807063] Call Trace:
[   92.807063]  [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[   92.807063]  [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[   92.807063]  [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[   92.807063]  [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
[   92.807063]  [<ffffffff8102b78a>] ? pvclock_clocksource_read+0x4b/0xb4
[   92.807063]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
[   92.807063]  [<ffffffff81007bc9>] ? xen_spin_unlock+0x11/0x30
[   92.807063]  [<ffffffff8105b253>] ? do_sigaction+0x175/0x18d
[   92.807063]  [<ffffffff811039a7>] ? fput+0x1ed/0x1fc
[   92.807063]  [<ffffffff8105d855>] ? sys_rt_sigaction+0x5e/0x9d
[   92.807063]  [<ffffffff81443615>] page_fault+0x25/0x30
[   92.807063] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d e0 48
89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff 85 c0 74 04
<0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48 8b 3d
[   92.807063] RIP  [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[   92.807063]  RSP <ffff880077869ce8>
[   92.807063] ---[ end trace ceae70ee7c69e40b ]---
[  693.101054] kmemleak: 86 new suspected memory leaks (see
/sys/kernel/debug/kmemleak)
[  695.855546] ------------[ cut here ]------------
[  695.855557] kernel BUG at arch/x86/xen/mmu.c:1831!
[  695.855563] invalid opcode: 0000 [#3] SMP
[  695.855574] last sysfs file: /sys/kernel/uevent_seqnum
[  695.855581] CPU 1
[  695.855585] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
[  695.855629]
[  695.855635] Pid: 3946, comm: cron Tainted: G      D
2.6.37-rc1-pvops-pcifront-fixes-domu-6.0.bedge-debug #1 /
[  695.855645] RIP: e030:[<ffffffff81004bce>]  [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[  695.855664] RSP: e02b:ffff880067a29ce8  EFLAGS: 00010282
[  695.855671] RAX: 00000000ffffffea RBX: 0000000000077c58 RCX:
0000000000000001
[  695.855680] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
00000000deadbeef
[  695.855689] RBP: ffff880067a29d08 R08: 00003ffffffff000 R09:
ffff880000000000
[  695.855696] R10: 00000000deadbeef R11: dead000000200200 R12:
0000000000000003
[  695.855703] R13: ffff880067a2cb40 R14: ffff880077c7e678 R15:
ffff8800682395f0
[  695.855716] FS:  00007fa1afde67a0(0000) GS:ffff88007ff5c000(0000)
knlGS:0000000000000000
[  695.855725] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  695.855734] CR2: 00007fa1ad1ac758 CR3: 0000000067890000 CR4:
0000000000002660
[  695.855742] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  695.855749] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[  695.855759] Process cron (pid: 3946, threadinfo ffff880067a28000, task
ffff8800682395f0)
[  695.855767] Stack:
[  695.855774]  ffff880000000000 0000000000047db6 ffff8800760e3480
0000000000077c58
[  695.855793]  ffff880067a29d28 ffffffff810066a8 0000000000077c58
ffff8800760e3480
[  695.855814]  ffff880067a29d38 ffffffff810066db ffff880067a29d78
ffffffff810e048f
[  695.855834] Call Trace:
[  695.855842]  [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[  695.855852]  [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[  695.855865]  [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[  695.855877]  [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
[  695.855887]  [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
[  695.855900]  [<ffffffff8100eb84>] ?
arch_get_unmapped_area_topdown+0x4a/0x28a
[  695.855915]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
[  695.855925]  [<ffffffff8102b78a>] ? pvclock_clocksource_read+0x4b/0xb4
[  695.855937]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[  695.855949]  [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
[  695.855959]  [<ffffffff81443615>] page_fault+0x25/0x30
[  695.855967] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d e0 48
89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff 85 c0 74 04
<0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48 8b 3d
[  695.856056] RIP  [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[  695.856056]  RSP <ffff880067a29ce8>
[  695.856056] ---[ end trace ceae70ee7c69e40c ]---
[  696.704202] cron[3928] general protection ip:7fa1af4ce588 sp:7ffff1875200
error:0 in libc-2.11.1.so[7fa1af424000+17a000]
[  697.853416] ------------[ cut here ]------------
[  697.853427] kernel BUG at arch/x86/xen/mmu.c:1831!
[  697.853433] invalid opcode: 0000 [#4] SMP
[  697.853443] last sysfs file: /sys/kernel/uevent_seqnum
[  697.853450] CPU 4
[  697.853453] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
[  697.853497]
[  697.853503] Pid: 3947, comm: sh Tainted: G      D
2.6.37-rc1-pvops-pcifront-fixes-domu-6.0.bedge-debug #1 /
[  697.853512] RIP: e030:[<ffffffff81004bce>]  [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[  697.853526] RSP: e02b:ffff880067aa5b38  EFLAGS: 00010282
[  697.853532] RAX: 00000000ffffffea RBX: 0000000000075203 RCX:
0000000000000001
[  697.853540] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
00000000deadbeef
[  697.853547] RBP: ffff880067aa5b58 R08: 00003ffffffff000 R09:
ffff880000000000
[  697.853554] R10: 00000000deadbeef R11: dead000000200200 R12:
0000000000000003
[  697.853561] R13: ffff880075204000 R14: ffff880075204000 R15:
ffff880078a26ff8
[  697.853571] FS:  00007fa1afde67a0(0000) GS:ffff88007ffb0000(0000)
knlGS:0000000000000000
[  697.853580] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  697.853586] CR2: 00007fa1af7bd560 CR3: 0000000077a98000 CR4:
0000000000002660
[  697.853594] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  697.853602] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[  697.853609] Process sh (pid: 3947, threadinfo ffff880067aa4000, task
ffff880068239d40)
[  697.853616] Stack:
[  697.853621]  ffff880000000000 000000000004a80b ffff8800760e1500
0000000000075203
[  697.853640]  ffff880067aa5b78 ffffffff810066a8 0000000000075203
ffff8800760e1500
[  697.853658]  ffff880067aa5b88 ffffffff810066db ffff880067aa5bc8
ffffffff810e048f
[  697.853676] Call Trace:
[  697.853685]  [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[  697.853693]  [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[  697.853704]  [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[  697.853713]  [<ffffffff810e7faa>] move_page_tables+0x1cb/0x4c1
[  697.853722]  [<ffffffff81007bc9>] ? xen_spin_unlock+0x11/0x30
[  697.853732]  [<ffffffff8110870a>] setup_arg_pages+0x1cb/0x355
[  697.853743]  [<ffffffff8113ce7a>] load_elf_binary+0x418/0x16ec
[  697.853752]  [<ffffffff8113b319>] ? load_misc_binary+0x5c/0x33c
[  697.853761]  [<ffffffff810e3336>] ? get_user_pages+0x44/0x46
[  697.853772]  [<ffffffff81107355>] ? get_arg_page+0x50/0xa7
[  697.853781]  [<ffffffff81107793>] search_binary_handler+0xbb/0x24d
[  697.853790]  [<ffffffff8113ca62>] ? load_elf_binary+0x0/0x16ec
[  697.853799]  [<ffffffff81108e2d>] do_execve+0x19e/0x28d
[  697.853809]  [<ffffffff810118dd>] sys_execve+0x3e/0x55
[  697.853818]  [<ffffffff8100af1c>] stub_execve+0x6c/0xc0
[  697.853824] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d e0 48
89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff 85 c0 74 04
<0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48 8b 3d
[  697.854013] RIP  [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[  697.854024]  RSP <ffff880067aa5b38>
[  697.854031] ---[ end trace ceae70ee7c69e40d ]---
[  755.706246] BUG: unable to handle kernel paging request at
ffff880075202b80
[  755.706263] IP: [<ffffffff810e24b3>] do_wp_page+0x58b/0x6be
[  755.706278] PGD 1a04067 PUD 39d9067 PMD 3b83067 PTE 8010000075202065
[  755.706294] Oops: 0003 [#5] SMP
[  755.706305] last sysfs file: /sys/kernel/uevent_seqnum
[  755.706312] CPU 4
[  755.706316] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
[  755.706364]
[  755.706370] Pid: 3948, comm: cron Tainted: G      D
2.6.37-rc1-pvops-pcifront-fixes-domu-6.0.bedge-debug #1 /
[  755.706379] RIP: e030:[<ffffffff810e24b3>]  [<ffffffff810e24b3>]
do_wp_page+0x58b/0x6be
[  755.706391] RSP: e02b:ffff880067439ce8  EFLAGS: 00010246
[  755.706397] RAX: 0000000000000000 RBX: ffffea00016f7d98 RCX:
0000000000000001
[  755.706404] RDX: 6db6db6db6db6db7 RSI: ffffffff81b460c0 RDI:
ffff8800760a1f80
[  755.706411] RBP: ffff880067439d78 R08: 0000000000000000 R09:
0000000000000041
[  755.706419] R10: 0000000000000100 R11: ffff880078277030 R12:
80000000407c3067
[  755.706426] R13: ffffea0001bd0068 R14: ffff880075202b80 R15:
ffff880067a77508
[  755.706436] FS:  00007fa1afde67a0(0000) GS:ffff88007ffb0000(0000)
knlGS:0000000000000000
[  755.706444] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  755.706451] CR2: ffff880075202b80 CR3: 0000000077ea9000 CR4:
0000000000002660
[  755.706458] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  755.706466] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[  755.706474] Process cron (pid: 3948, threadinfo ffff880067438000, task
ffff88006823a490)
[  755.706481] Stack:
[  755.706486]  ffff880067439d08 ffffffff81007c09 ffffea000199f080
0000000000000000
[  755.706504]  01ff880067439d58 ffffea000199f080 ffff8800760a1f80
0000000000d703b0
[  755.706523]  00007fa1afde6a70 ffff880078277030 ffff880067a77508
ffff8800760a1f80
[  755.706542] Call Trace:
[  755.706552]  [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[  755.706562]  [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
[  755.706571]  [<ffffffff8103ed4d>] ? enqueue_entity+0x2ac/0x2b8
[  755.706582]  [<ffffffff814463fe>] do_page_fault+0x374/0x396
[  755.706590]  [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1
[  755.706600]  [<ffffffff81004274>] ? xen_mc_flush+0x1cd/0x1f2
[  755.706609]  [<ffffffff8103ed4d>] ? enqueue_entity+0x2ac/0x2b8
[  755.706618]  [<ffffffff81007bc9>] ? xen_spin_unlock+0x11/0x30
[  755.706628]  [<ffffffff81045b20>] ? finish_task_switch+0x4f/0x94
[  755.706637]  [<ffffffff81443615>] page_fault+0x25/0x30
[  755.706644] Code: 48 0f af fa 48 c1 e7 0c 48 09 c7 ff 14 25 28 13 a1 81
49 89 c4 48 83 c8 42 49 8b 3f 49 83 cc 40 41 f6 47 30 02 4c 0f 45 e0 31 c0
<49> 87 06 48 8b 75 a8 4c 89 f2 0f 1f 80 00 00 00 00 48 8b 75 a8
[  755.706830] RIP  [<ffffffff810e24b3>] do_wp_page+0x58b/0x6be
[  755.706841]  RSP <ffff880067439ce8>
[  755.706847] CR2: ffff880075202b80
[  755.706854] ---[ end trace ceae70ee7c69e40e ]---


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Nov-11 15:56 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Wed, Nov 10, 2010 at 03:03:34PM -0800, Bruce Edge
wrote:> On Wed, Nov 10, 2010 at 2:30 PM, Bruce Edge <bruce.edge@gmail.com>
wrote:
> > On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge
<bruce.edge@gmail.com> wrote:
> >>
> >> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable
build 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
> >>
> >> The domU console output hung up prior to spitting out a prompt
post root login. I logged into the domU via ssh and ran an
''ls'' of an NFS mounted dir and
> >> this immediately came out on the domU console:

So... does this show if you don''t do PCI passthrough? Just as a normal
PV guests with
NFS mount?

I am not going to look at this yet - debugging a bootup issue with the
2.6.37-rc1 having
a fit on AMD CPUs.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Nov-11 16:09 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Thu, Nov 11, 2010 at 04:08:03AM -0800, Boris Derzhavets
wrote:> Now reproduced :-
> 
> root@boris-HVM-domU:~# scp  1.log 192.168.1.12:/home/boris/ubuntu
> BUG: unable to handle kernel paging request at ffff88000e676930
> IP: [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
> PGD 1a04067 PUD 1a08067 PMD ee73067 PTE 801000000e676065
> Oops: 0003 [#1] SMP 
> last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
> CPU 0 
> Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl
auth_rpcgss sunrpc exportfs ipv6 lp parport
> 
> Pid: 3534, comm: scp Not tainted 2.6.37-rc1 #1 /
Yikes. So no PCI passthrough, just a normal PV DomU.

Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36? 2.6.35?
> RIP: e030:[<ffffffff810069c3>]  [<ffffffff810069c3>]
xen_set_pmd+0x24/0x44
> RSP: e02b:ffff88006e697b18  EFLAGS: 00010246
> RAX: 0000000000000000 RBX: ffff88000e676930 RCX: ffffc00000000f98
> RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff88000e676930
> RBP: ffff88006e697b28 R08: 00007fff9d120000 R09: ffff88007ffc80a0
> R10: 0000000000000000 R11: ffff88006e697ae8 R12: 0000000000000000
> R13: ffff88007ffcb140 R14: ffff88000e676930 R15: ffff88006f08d7f8
> FS:  00007fa525896700(0000) GS:ffff88007ffbd000(0000)
knlGS:0000000000000000
> CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: ffff88000e676930 CR3: 0000000077b0f000 CR4: 0000000000002660
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process scp (pid: 3534, threadinfo ffff88006e696000, task ffff88007a32c2c0)
> Stack:
>  00007fa524e00000 00007fa525b37000 ffff88006e697be8 ffffffff810dc4a4
>  ffff88007a26ac88 ffff88006e690298 ffff88007a26ac78 00007fa524a00000
>  00007f8000000000 00007fff80000000 00007fa525b36fff 00007fa525b36fff
> Call Trace:
>  [<ffffffff810dc4a4>] free_pgd_range+0x213/0x397
>  [<ffffffff810dc6cc>] free_pgtables+0xa4/0xc7
>  [<ffffffff810dfd82>] exit_mmap+0xf2/0x12a
>  [<ffffffff8104a7c5>] mmput+0x5e/0xd1
>  [<ffffffff8110694b>] flush_old_exec+0x526/0x600
>  [<ffffffff8113e006>] load_elf_binary+0x2dc/0x1752
>  [<ffffffff8113b687>] ? load_misc_binary+0x6e/0x367
>  [<ffffffff811dcee4>] ? ima_must_measure+0x21/0x2b
>  [<ffffffff811dc7d6>] ? process_measurement+0x56/0xe4
>  [<ffffffff81106195>] search_binary_handler+0xbb/0x24d
>  [<ffffffff8113dd2a>] ? load_elf_binary+0x0/0x1752
>  [<ffffffff8110785d>] do_execve+0x1c1/0x2ae
>  [<ffffffff810119e5>] sys_execve+0x3e/0x55
>  [<ffffffff8100af9c>] stub_execve+0x6c/0xc0
> Code: 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 b5 e4
ff ff 48 89 df ff 05 3e 55 c3 00 e8 d5 dd ff ff 84 c0 75 05 <4c> 89 23 eb
16 e8 99 e4 ff ff 4c 89 e6 48 89 df ff 05 23 55 c3
> RIP  [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
>  RSP <ffff88006e697b18>
> CR2: ffff88000e676930
> ---[ end trace 073da74f4ce02315 ]---
> lost connection
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-11 16:29 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

> Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36?
2.6.35?
Konrad,

The version is 2.6.37-rc1,  been built from upstream source.

Boris.

--- On Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>
Date: Thursday, November 11, 2010, 11:09 AM

On Thu, Nov 11, 2010 at 04:08:03AM -0800, Boris Derzhavets
wrote:> Now reproduced :-
> 
> root@boris-HVM-domU:~# scp  1.log 192.168.1.12:/home/boris/ubuntu
> BUG: unable to handle kernel paging request at ffff88000e676930
> IP: [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
> PGD 1a04067 PUD 1a08067 PMD ee73067 PTE 801000000e676065
> Oops: 0003 [#1] SMP 
> last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
> CPU 0 
> Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl
auth_rpcgss sunrpc exportfs ipv6 lp parport
> 
> Pid: 3534, comm: scp Not tainted 2.6.37-rc1 #1 /
Yikes. So no PCI passthrough, just a normal PV DomU.

Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36? 2.6.35?
> RIP: e030:[<ffffffff810069c3>]  [<ffffffff810069c3>]
xen_set_pmd+0x24/0x44
> RSP: e02b:ffff88006e697b18  EFLAGS: 00010246
> RAX: 0000000000000000 RBX: ffff88000e676930 RCX: ffffc00000000f98
> RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff88000e676930
> RBP: ffff88006e697b28 R08: 00007fff9d120000 R09: ffff88007ffc80a0
> R10: 0000000000000000 R11: ffff88006e697ae8 R12: 0000000000000000
> R13: ffff88007ffcb140 R14: ffff88000e676930 R15: ffff88006f08d7f8
> FS:  00007fa525896700(0000) GS:ffff88007ffbd000(0000)
knlGS:0000000000000000
> CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: ffff88000e676930 CR3: 0000000077b0f000 CR4: 0000000000002660
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process scp (pid: 3534, threadinfo ffff88006e696000, task ffff88007a32c2c0)
> Stack:
>  00007fa524e00000 00007fa525b37000 ffff88006e697be8 ffffffff810dc4a4
>  ffff88007a26ac88 ffff88006e690298 ffff88007a26ac78 00007fa524a00000
>  00007f8000000000 00007fff80000000 00007fa525b36fff 00007fa525b36fff
> Call Trace:
>  [<ffffffff810dc4a4>] free_pgd_range+0x213/0x397
>  [<ffffffff810dc6cc>] free_pgtables+0xa4/0xc7
>  [<ffffffff810dfd82>] exit_mmap+0xf2/0x12a
>  [<ffffffff8104a7c5>] mmput+0x5e/0xd1
>  [<ffffffff8110694b>] flush_old_exec+0x526/0x600
>  [<ffffffff8113e006>] load_elf_binary+0x2dc/0x1752
>  [<ffffffff8113b687>] ? load_misc_binary+0x6e/0x367
>  [<ffffffff811dcee4>] ? ima_must_measure+0x21/0x2b
>  [<ffffffff811dc7d6>] ? process_measurement+0x56/0xe4
>  [<ffffffff81106195>] search_binary_handler+0xbb/0x24d
>  [<ffffffff8113dd2a>] ? load_elf_binary+0x0/0x1752
>  [<ffffffff8110785d>] do_execve+0x1c1/0x2ae
>  [<ffffffff810119e5>] sys_execve+0x3e/0x55
>  [<ffffffff8100af9c>] stub_execve+0x6c/0xc0
> Code: 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 b5 e4
ff ff 48 89 df ff 05 3e 55 c3 00 e8 d5 dd ff ff 84 c0 75 05 <4c> 89 23 eb
16 e8 99 e4 ff ff 4c 89 e6 48 89 df ff 05 23 55 c3
> RIP  [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
>  RSP <ffff88006e697b18>
> CR2: ffff88000e676930
> ---[ end trace 073da74f4ce02315 ]---
> lost connection
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-11 16:32 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

> So... does this show if you don''t do PCI passthrough? 
> Just as a normal PV guests with NFS mount?
Yes.

Boris.

--- On Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Bruce Edge" <bruce.edge@gmail.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com
Date: Thursday, November 11, 2010, 10:56 AM

On Wed, Nov 10, 2010 at 03:03:34PM -0800, Bruce Edge
wrote:> On Wed, Nov 10, 2010 at 2:30 PM, Bruce Edge <bruce.edge@gmail.com>
wrote:
> > On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge
<bruce.edge@gmail.com> wrote:
> >>
> >> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable
build 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
> >>
> >> The domU console output hung up prior to spitting out a prompt
post root login. I logged into the domU via ssh and ran an
''ls'' of an NFS mounted dir and
> >> this immediately came out on the domU console:

So... does this show if you don''t do PCI passthrough? Just as a normal
PV guests with
NFS mount?

I am not going to look at this yet - debugging a bootup issue with the
2.6.37-rc1 having
a fit on AMD CPUs.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Nov-11 16:46 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Thu, Nov 11, 2010 at 08:29:38AM -0800, Boris Derzhavets
wrote:> > Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36?
2.6.35?
> 
> Konrad,
> 
> The version is 2.6.37-rc1,  been built from upstream source.
I know that. I was asking what version worked for you in the past. As in, did
you do
the same tests with older kernels (2.6.36 for example?)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-11 16:46 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

This time to reproduce crash  `ls -l` was not enough , but similar stack trace
comes up sooner or later :

root@boris-HVM-domU:~# mount  192.168.1.9:/home/boris     /mnt/tmp
. . . . . . 

CRASH :-

root@boris-HVM-domU:~# cp /mnt/tmp/*.iso .
BUG: unable to handle kernel paging request at ffff88006f3824f0
IP: [<ffffffff810dd0c3>] do_wp_page+0x597/0x6fa
PGD 1a04067 PUD f57b067 PMD f6f5067 PTE 801000006f382065
Oops: 0003 [#1] SMP 
last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
CPU 0 
Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl auth_rpcgss
sunrpc exportfs ipv6 lp parport

Pid: 3533, comm: bash Not tainted 2.6.37-rc1 #1 /
RIP: e030:[<ffffffff810dd0c3>]  [<ffffffff810dd0c3>]
do_wp_page+0x597/0x6fa
RSP: e02b:ffff880077b25cd8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffea000034dcf8 RCX: 0000000000000001
RDX: 8000000162e88065 RSI: ffffffff81c3d050 RDI: ffff88006d3e6100
RBP: ffff880077b25d68 R08: 0000000000000100 R09: 0000000000000041
R10: ffff880077b25a78 R11: ffff88006d3e6100 R12: ffffea0000277030
R13: ffff88006f3824f0 R14: ffff88006d3e6100 R15: ffff88006f0b34d8
FS:  00007fc18508b700(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff88006f3824f0 CR3: 000000007a103000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process bash (pid: 3533, threadinfo ffff880077b24000, task ffff88006f0ee2c0)
Stack:
 ffff880077b25cf8 ffffffff81007c31 ffffea0001854480 0000000000000000
 01ff880077b25d48 ffffea0001854480 8000000162e88067 00007fc18509e9a8
 00007fc18508b9d0 ffff88006f0b34d8 ffff88006f3a2140 ffff88006d3e6100
Call Trace:
 [<ffffffff81007c31>] ? __spin_time_accum+0x21/0x37
 [<ffffffff810ddbd7>] handle_mm_fault+0x9b1/0xa05
 [<ffffffff8145665a>] do_page_fault+0x387/0x3a9
 [<ffffffff810072af>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff81004274>] ? xen_mc_flush+0x1cd/0x1f2
 [<ffffffff81003e5f>] ? xen_clts+0x33/0x59
 [<ffffffff81007bf1>] ? xen_spin_unlock+0x11/0x30
 [<ffffffff81045a6f>] ? finish_task_switch+0x4f/0xc5
 [<ffffffff8103d0a9>] ? __might_sleep+0x23/0xea
 [<ffffffff81453715>] page_fault+0x25/0x30
Code: 48 c1 e7 0c 48 09 c7 ff 14 25 28 28 a1 81 48 89 c2 48 83 c8 42 49 8b 3f 48
83 ca 40 41 f6 47 30 02 48 0f 44 c2 48 89 45 a0 31 c0 <49> 87 45 00 48 8b
75 a8 4c 89 ea 0f 1f 80 00 00 00 00 48 8b 75
RIP  [<ffffffff810dd0c3>] do_wp_page+0x597/0x6fa
 RSP <ffff880077b25cd8>
CR2: ffff88006f3824f0
---[ end trace 69a76296779698ff ]---

Boris.

--- On Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>
Date: Thursday, November 11, 2010, 11:09 AM

On Thu, Nov 11, 2010 at 04:08:03AM -0800, Boris Derzhavets
wrote:> Now reproduced :-
> 
> root@boris-HVM-domU:~# scp  1.log 192.168.1.12:/home/boris/ubuntu
> BUG: unable to handle kernel paging request at ffff88000e676930
> IP: [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
> PGD 1a04067 PUD 1a08067 PMD ee73067 PTE 801000000e676065
> Oops: 0003 [#1] SMP 
> last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
> CPU 0 
> Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl
auth_rpcgss sunrpc exportfs ipv6 lp parport
> 
> Pid: 3534, comm: scp Not tainted 2.6.37-rc1 #1 /
Yikes. So no PCI passthrough, just a normal PV DomU.

Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36? 2.6.35?
> RIP: e030:[<ffffffff810069c3>]  [<ffffffff810069c3>]
xen_set_pmd+0x24/0x44
> RSP: e02b:ffff88006e697b18  EFLAGS: 00010246
> RAX: 0000000000000000 RBX: ffff88000e676930 RCX: ffffc00000000f98
> RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff88000e676930
> RBP: ffff88006e697b28 R08: 00007fff9d120000 R09: ffff88007ffc80a0
> R10: 0000000000000000 R11: ffff88006e697ae8 R12: 0000000000000000
> R13: ffff88007ffcb140 R14: ffff88000e676930 R15: ffff88006f08d7f8
> FS:  00007fa525896700(0000) GS:ffff88007ffbd000(0000)
knlGS:0000000000000000
> CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: ffff88000e676930 CR3: 0000000077b0f000 CR4: 0000000000002660
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process scp (pid: 3534, threadinfo ffff88006e696000, task ffff88007a32c2c0)
> Stack:
>  00007fa524e00000 00007fa525b37000 ffff88006e697be8 ffffffff810dc4a4
>  ffff88007a26ac88 ffff88006e690298 ffff88007a26ac78 00007fa524a00000
>  00007f8000000000 00007fff80000000 00007fa525b36fff 00007fa525b36fff
> Call Trace:
>  [<ffffffff810dc4a4>] free_pgd_range+0x213/0x397
>  [<ffffffff810dc6cc>] free_pgtables+0xa4/0xc7
>  [<ffffffff810dfd82>] exit_mmap+0xf2/0x12a
>  [<ffffffff8104a7c5>] mmput+0x5e/0xd1
>  [<ffffffff8110694b>] flush_old_exec+0x526/0x600
>  [<ffffffff8113e006>] load_elf_binary+0x2dc/0x1752
>  [<ffffffff8113b687>] ? load_misc_binary+0x6e/0x367
>  [<ffffffff811dcee4>] ? ima_must_measure+0x21/0x2b
>  [<ffffffff811dc7d6>] ? process_measurement+0x56/0xe4
>  [<ffffffff81106195>] search_binary_handler+0xbb/0x24d
>  [<ffffffff8113dd2a>] ? load_elf_binary+0x0/0x1752
>  [<ffffffff8110785d>] do_execve+0x1c1/0x2ae
>  [<ffffffff810119e5>] sys_execve+0x3e/0x55
>  [<ffffffff8100af9c>] stub_execve+0x6c/0xc0
> Code: 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 b5 e4
ff ff 48 89 df ff 05 3e 55 c3 00 e8 d5 dd ff ff 84 c0 75 05 <4c> 89 23 eb
16 e8 99 e4 ff ff 4c 89 e6 48 89 df ff 05 23 55 c3
> RIP  [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
>  RSP <ffff88006e697b18>
> CR2: ffff88000e676930
> ---[ end trace 073da74f4ce02315 ]---
> lost connection
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-11 16:53 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Ubuntu''s  10.10  2.6.35-22-generic (or server)  works fine for PV DomUs
sharing
same image device. I never tested  vanilla 2.6.35 (36) itself for PV DomU
support.

Boris.

--- On Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>
Date: Thursday, November 11, 2010, 11:46 AM

On Thu, Nov 11, 2010 at 08:29:38AM -0800, Boris Derzhavets
wrote:> > Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36?
2.6.35?
> 
> Konrad,
> 
> The version is 2.6.37-rc1,  been built from upstream source.
I know that. I was asking what version worked for you in the past. As in, did
you do
the same tests with older kernels (2.6.36 for example?)



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Nov-12 14:40 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

The git bisecting is slow going. I''ve never tried that before and
I''m a git
rookie.
I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
panic at boot so I''m obviously doing something wrong.
I''ll RTFM a bit more and keep at it.

-Bruce

On Thu, Nov 11, 2010 at 8:53 AM, Boris Derzhavets
<bderzhavets@yahoo.com>wrote:
> Ubuntu''s  10.10  2.6.35-22-generic (or server)  works fine for PV
DomUs
> sharing
> same image device. I never tested  vanilla 2.6.35 (36) itself for PV DomU
> support.
>
>
> Boris.
>
> --- On *Thu, 11/11/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com>*wrote:
>
>
> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com,
> "Bruce Edge" <bruce.edge@gmail.com>
> Date: Thursday, November 11, 2010, 11:46 AM
>
>
> On Thu, Nov 11, 2010 at 08:29:38AM -0800, Boris Derzhavets wrote:
> > > Boris, what version of Linux kernel worked for you as PV DomU?
2.6.36?
> 2.6.35?
> >
> > Konrad,
> >
> > The version is 2.6.37-rc1,  been built from upstream source.
>
> I know that. I was asking what version worked for you in the past. As in,
> did you do
> the same tests with older kernels (2.6.36 for example?)
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-12 16:06 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Bruce,

I just tried 2.6.36 + patch-2.6.36-git13 via patch-2.6.36-git13.bz2 ( i picked
it up from MY''s first kernel-2.6.37.fc15.rc0.git13.src.rpm ). Kernel
2.6.36-git13 already has a BUG .
So , problem is inside 11 MB   patch-2.6.36-git13.bz2 . Might be it gives
37-rc1,
i am not sure. If you want , i can try to send it to you.

Boris.
P.S. Bisecting 36 - 37-rc1 is too much anyway.

--- On Fri, 11/12/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Friday, November 12, 2010, 9:40 AM

The git bisecting is slow going. I''ve never tried that before and
I''m a git rookie.
I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
panic at boot so I''m obviously doing something wrong.

I''ll RTFM a bit more and keep at it.

-Bruce

On Thu, Nov 11, 2010 at 8:53 AM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:

Ubuntu''s  10.10  2.6.35-22-generic (or server)  works fine for PV DomUs
sharing
same image device. I never tested  vanilla 2.6.35 (36) itself for PV DomU
support.

Boris.

--- On Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>

Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>

Date: Thursday, November 11, 2010, 11:46 AM

On Thu, Nov 11, 2010 at 08:29:38AM -0800,
 Boris Derzhavets wrote:> > Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36?
2.6.35?
> 
> Konrad,
> 
> The version is 2.6.37-rc1,  been built from upstream source.
I know that. I was asking what version worked for you in the past. As in, did
you do
the same tests with older kernels (2.6.36 for example?)

_______________________________________________

Xen-devel mailing list

Xen-devel@lists.xensource.com

http://lists.xensource.com/xen-devel

-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Sander Eikelenboom

2010-Nov-12 16:27 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Hi Bruce,

Perhaps handpick some kernels before and after the pulls of the xen patches
(pv-on-hvm etc) to begin with ?
When you let git choose, especially with rc-1 kernels, you will end up with
kernels in between patch series, resulting in panics.

--
Sander

Friday, November 12, 2010, 3:40:11 PM, you wrote:
> The git bisecting is slow going. I''ve never tried that before and
I''m a git
> rookie.
> I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
> panic at boot so I''m obviously doing something wrong.
> I''ll RTFM a bit more and keep at it.
> -Bruce
> On Thu, Nov 11, 2010 at 8:53 AM, Boris Derzhavets
<bderzhavets@yahoo.com>wrote:
>> Ubuntu''s  10.10  2.6.35-22-generic (or server)  works fine for
PV DomUs
>> sharing
>> same image device. I never tested  vanilla 2.6.35 (36) itself for PV
DomU
>> support.
>>
>>
>> Boris.
>>
>> --- On *Thu, 11/11/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com>*wrote:
>>
>>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
>> handle kernel paging request
>> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
>> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com,
>> "Bruce Edge" <bruce.edge@gmail.com>
>> Date: Thursday, November 11, 2010, 11:46 AM
>>
>>
>> On Thu, Nov 11, 2010 at 08:29:38AM -0800, Boris Derzhavets wrote:
>> > > Boris, what version of Linux kernel worked for you as PV
DomU? 2.6.36?
>> 2.6.35?
>> >
>> > Konrad,
>> >
>> > The version is 2.6.37-rc1,  been built from upstream source.
>>
>> I know that. I was asking what version worked for you in the past. As
in,
>> did you do
>> the same tests with older kernels (2.6.36 for example?)
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>


-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Nov-12 17:01 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom
wrote:> Hi Bruce,
> 
> Perhaps handpick some kernels before and after the pulls of the xen patches
(pv-on-hvm etc) to begin with ?
> When you let git choose, especially with rc-1 kernels, you will end up with
kernels in between patch series, resulting in panics.
Well, just the bare-bone boot of PV guests with nothing fancy ought to work.

But that is the theory and ..> > The git bisecting is slow going. I''ve never tried that before
and I''m a git
> > rookie.
> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2
bisects all
> > panic at boot so I''m obviously doing something wrong.
> > I''ll RTFM a bit more and keep at it.
.. as Bruce experiences this is not the case. Hmm..

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-13 08:37 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Same stack trace comes up when i attempted tp "cp" files from NFS
folder (at Dom0)
mounted at DomU ( 2.6.37-rc1-git9 kernel )

root@boris-HVM-domU:~# mount 192.168.1.6:/home/boris /mnt/tmp
root@boris-HVM-domU:~# ls -l /mnt/tmp
total 140300
drwxr-xr-x 2 boris boris     4096 2010-10-14 18:29 Desktop
drwxr-xr-x 2 boris boris     4096 2010-10-14 18:29 Documents
drwxr-xr-x 2 boris boris     4096 2010-10-14 18:29 Downloads
-rw-r--r-- 1 boris boris      179 2010-10-14 18:06 examples.desktop
-rw-r--r-- 1 root  root  70277083 2010-11-12 21:40 linux-2.6.36.tar.bz2
-rw-r--r-- 1 root  root  73335884 2010-11-12 21:42 linux-2.6.37-rc1.tar.bz2
drwxr-xr-x 2 boris boris     4096 2010-10-14 18:29 Music
drwxr-xr-x 2 boris boris     4096 2010-10-14 18:29 Pictures
drwxr-xr-x 2 boris boris     4096 2010-10-14 18:29 Public
drwxr-xr-x 2 root  root      4096 2010-11-05 15:12 solaris
drwxr-xr-x 2 boris boris     4096 2010-10-14 18:29 Templates
drwxr-xr-x 2 root  root      4096 2010-11-13 11:19 ubuntu
drwxr-xr-x 2 boris boris     4096 2010-10-14 18:29 Videos
root@boris-HVM-domU:~# cp /mnt/tmp/linux-2.6.36.tar.bz2 .
root@boris-HVM-domU:~# cp /mnt/tmp/linux-2.6.37-rc1.tar.bz2  .

BUG: unable to handle kernel paging request at ffff880079b96cc0
IP: [<ffffffff810daa31>] do_wp_page+0x597/0x6fa
PGD 1a04067 PUD 41d5067 PMD 43a3067 PTE 8010000079b96065
Oops: 0003 [#1] SMP 
last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
CPU 0 
Modules linked in: parport_pc ppdev ipv6 lp parport

Pid: 3509, comm: bash Not tainted 2.6.37-rc1-git9 #1 /
RIP: e030:[<ffffffff810daa31>]  [<ffffffff810daa31>]
do_wp_page+0x597/0x6fa
RSP: e02b:ffff88007a305cd8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffea0001a0bcd0 RCX: 0000000000000001
RDX: 8000000136d77065 RSI: ffffffff81c47050 RDI: ffff880078adf040
RBP: ffff88007a305d68 R08: 0000000000000100 R09: 0000000000000041
R10: ffff8800798869c0 R11: ffff880078adf040 R12: ffffea0001a1ada8
R13: ffff880079b96cc0 R14: ffff880078adf040 R15: ffff88007a3c8700
FS:  00007f3a44186700(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff880079b96cc0 CR3: 00000000798b8000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process bash (pid: 3509, threadinfo ffff88007a304000, task ffff880078f66680)
Stack:
 ffff88007a305cf8 ffffffff81007c31 ffffea0001aa08e0 0000000000000000
 01ff88007a305d48 ffffea0001aa08e0 8000000136d77067 00007f3a441989a8
 00007f3a441869d0 ffff88007a3c8700 ffff88007988d100 ffff880078adf040
Call Trace:
 [<ffffffff81007c31>] ? __spin_time_accum+0x21/0x37
 [<ffffffff810db54e>] handle_mm_fault+0x9ba/0xa0e
 [<ffffffff814cd5dc>] do_page_fault+0x387/0x3a9
 [<ffffffff810072af>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff81004274>] ? xen_mc_flush+0x1cd/0x1f2
 [<ffffffff81003e5f>] ? xen_clts+0x33/0x59
 [<ffffffff81007bf1>] ? xen_spin_unlock+0x11/0x30
 [<ffffffff81042d24>] ? finish_task_switch+0x4f/0xc5
 [<ffffffff8103c580>] ? __might_sleep+0x23/0xea
 [<ffffffff814ca695>] page_fault+0x25/0x30
Code: 48 c1 e7 0c 48 09 c7 ff 14 25 e8 27 a1 81 48 89 c2 48 83 c8 42 49 8b 3f 48
83 ca 40 41 f6 47 30 02 48 0f 44 c2 48 89 45 a0 31 c0 <49> 87 45 00 48 8b
75 a8 4c 89 ea 0f 1f 80 00 00 00 00 48 8b 75
RIP  [<ffffffff810daa31>] do_wp_page+0x597/0x6fa
 RSP <ffff88007a305cd8>
CR2: ffff880079b96cc0
---[ end trace a83a1add1cfb7333 ]---


--- On Fri, 11/12/10, Boris Derzhavets <bderzhavets@yahoo.com> wrote:

From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Bruce Edge" <bruce.edge@gmail.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Friday, November 12, 2010, 11:06 AM

Bruce,

I just tried 2.6.36 + patch-2.6.36-git13 via patch-2.6.36-git13.bz2 ( i picked
it up from MY''s first kernel-2.6.37.fc15.rc0.git13.src.rpm ). Kernel
2.6.36-git13 already has a BUG .
So , problem is inside 11 MB   patch-2.6.36-git13.bz2 . Might be it gives
37-rc1,
i am not sure. If you want , i can try to send it to you.

Boris.
P.S. Bisecting 36 - 37-rc1 is too much anyway.

--- On Fri, 11/12/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
 xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Friday, November 12, 2010, 9:40 AM

The git bisecting is slow going. I''ve never tried that before and
I''m a git rookie.
I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
panic at boot so I''m obviously doing something wrong.

I''ll RTFM a bit more and keep at it.

-Bruce

On Thu, Nov 11, 2010 at 8:53 AM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:


Ubuntu''s  10.10  2.6.35-22-generic (or server)  works fine for PV DomUs
sharing
same image device. I never tested  vanilla 2.6.35 (36) itself for PV DomU
support.

Boris.

--- On Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>

Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>

Date: Thursday, November 11, 2010, 11:46 AM

On Thu, Nov 11, 2010 at 08:29:38AM -0800,
 Boris Derzhavets wrote:> > Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36?
2.6.35?
> 
> Konrad,
> 
> The version is 2.6.37-rc1,  been built from upstream source.
I know that. I was asking what version worked for you in the past. As in, did
you do
the same tests with older kernels (2.6.36 for example?)









      
_______________________________________________

Xen-devel mailing list

Xen-devel@lists.xensource.com

http://lists.xensource.com/xen-devel



-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      
-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-14 16:37 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)

Boris.

--- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
Date: Friday, November 12, 2010, 12:01 PM

On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom
wrote:> Hi Bruce,
> 
> Perhaps handpick some kernels before and after the pulls of the xen patches
(pv-on-hvm etc) to begin with ?
> When you let git choose, especially with rc-1 kernels, you will end up with
kernels in between patch series, resulting in panics.
Well, just the bare-bone boot of PV guests with nothing fancy ought to work.

But that is the theory and ..> > The git bisecting is slow going. I''ve never tried that before
and I''m a git
> > rookie.
> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2
bisects all
> > panic at boot so I''m obviously doing something wrong.
> > I''ll RTFM a bit more and keep at it.
.. as Bruce experiences this is not the case. Hmm..

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Sander Eikelenboom

2010-Nov-14 16:56 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Hmmm have you tried do do a lot of I/O with something else as NFS ?
That would perhaps pinpoint it to NFS doing something not completely compatible
with Xen.

I''m not using NFS (I still use file: based guests, and i use glusterfs
(fuse based userspace cluster fs) to share diskspace to domU''s via
ethernet).
I tried NFS in the past, but had some troubles setting it up, and even more
problems with disconnects.

I haven''t seen any "unable to handle page request" problems
with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.

--

Sander





Sunday, November 14, 2010, 5:37:59 PM, you wrote:
> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
> Boris.
> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:
> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
> To: "Sander Eikelenboom" <linux@eikelenboom.it>
> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
> Date: Friday, November 12, 2010, 12:01 PM
> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>> Hi Bruce,
>> 
>> Perhaps handpick some kernels before and after the pulls of the xen
patches (pv-on-hvm etc) to begin with ?
>> When you let git choose, especially with rc-1 kernels, you will end up
with kernels in between patch series, resulting in panics.
> Well, just the bare-bone boot of PV guests with nothing fancy ought to
work.
> But that is the theory and ..
>> > The git bisecting is slow going. I''ve never tried that
before and I''m a git
>> > rookie.
>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2
bisects all
>> > panic at boot so I''m obviously doing something wrong.
>> > I''ll RTFM a bit more and keep at it.
> .. as Bruce experiences this is not the case. Hmm..
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

>       


-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-14 17:09 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

> Hmmm have you tried do do a lot of I/O with something else as NFS ?
I''ve tried scp a lot from DomU to Dom0 on F14 no problems, on Ubuntu
10.04 Server
DomU as NFS client cannot survive more then a 2-3 min with I/O generated by scp.
If i unmount NFS share everything goes fine ( Ubuntu 10.04)
> That would perhaps pinpoint it to NFS doing something not completely
compatible withXen.

If DomU ( Ubuntu 10.04 ) is running  by 2.6.36 kernel there is no any NFS or
other network related problems . Issue is specific for 2.6.37-rc1 kernel as
kernel running DomU at Xen 4.0.1 Dom0 ( 2.6.32.25 pvops) on top of Ubuntu Lucid
Server.

Boris.





-- On Sun, 11/14/10, Sander Eikelenboom <linux@eikelenboom.it> wrote:

From: Sander Eikelenboom <linux@eikelenboom.it>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
Date: Sunday, November 14, 2010, 11:56 AM

Hmmm have you tried do do a lot of I/O with something else as NFS ?
That would perhaps pinpoint it to NFS doing something not completely compatible
with Xen.

I''m not using NFS (I still use file: based guests, and i use glusterfs
(fuse based userspace cluster fs) to share diskspace to domU''s via
ethernet).
I tried NFS in the past, but had some troubles setting it up, and even more
problems with disconnects.

I haven''t seen any "unable to handle page request" problems
with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.

--

Sander





Sunday, November 14, 2010, 5:37:59 PM, you wrote:
> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
> Boris.
> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:
> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
> To: "Sander Eikelenboom" <linux@eikelenboom.it>
> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
> Date: Friday, November 12, 2010, 12:01 PM
> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>> Hi Bruce,
>> 
>> Perhaps handpick some kernels before and after the pulls of the xen
patches (pv-on-hvm etc) to begin with ?
>> When you let git choose, especially with rc-1 kernels, you will end up
with kernels in between patch series, resulting in panics.
> Well, just the bare-bone boot of PV guests with nothing fancy ought to
work.
> But that is the theory and ..
>> > The git bisecting is slow going. I''ve never tried that
before and I''m a git
>> > rookie.
>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2
bisects all
>> > panic at boot so I''m obviously doing something wrong.
>> > I''ll RTFM a bit more and keep at it.
> .. as Bruce experiences this is not the case. Hmm..
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

>       


-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it




      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Sander Eikelenboom

2010-Nov-14 17:19 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Perhaps these patches (especially Stefano''s) could make a difference ?

http://lkml.indiana.edu/hypermail/linux/kernel/1011.1/02348.html

I don''t think these are applied to your domU kernel  since
it''s git8, it worth the shot i guess...

--
Sander

Sunday, November 14, 2010, 6:09:14 PM, you wrote:
>> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> I''ve tried scp a lot from DomU to Dom0 on F14 no problems, on
Ubuntu 10.04 Server
> DomU as NFS client cannot survive more then a 2-3 min with I/O generated by
scp.
> If i unmount NFS share everything goes fine ( Ubuntu 10.04)
>> That would perhaps pinpoint it to NFS doing something not completely
compatible with
> Xen.
> If DomU ( Ubuntu 10.04 ) is running  by 2.6.36 kernel there is no any NFS
or other network related problems . Issue is specific for 2.6.37-rc1 kernel as
kernel running DomU at Xen 4.0.1 Dom0 ( 2.6.32.25 pvops) on top of Ubuntu Lucid
Server.
> Boris.



> -- On Sun, 11/14/10, Sander Eikelenboom <linux@eikelenboom.it> wrote:
> From: Sander Eikelenboom <linux@eikelenboom.it>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
> Date: Sunday, November 14, 2010, 11:56 AM
> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely
compatible with Xen.
> I''m not using NFS (I still use file: based guests, and i use
glusterfs (fuse based userspace cluster fs) to share diskspace to
domU''s via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more
problems with disconnects.
> I haven''t seen any "unable to handle page request"
problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
> --
> Sander



> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
>> Boris.
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>> 
>>> Perhaps handpick some kernels before and after the pulls of the xen
patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end
up with kernels in between patch series, resulting in panics.
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to
work.
>> But that is the theory and ..
>>> > The git bisecting is slow going. I''ve never tried
that before and I''m a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first
2 bisects all
>>> > panic at boot so I''m obviously doing something wrong.
>>> > I''ll RTFM a bit more and keep at it.
>> .. as Bruce experiences this is not the case. Hmm..
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel

>>       





-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-14 17:47 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

This is not exactly git8 . Michael Young applied attached patch on top of git8.
It''s  his the most recent submission to fedora-xen.
http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434 


Boris.

--- On Sun, 11/14/10, Sander Eikelenboom <linux@eikelenboom.it> wrote:

From: Sander Eikelenboom <linux@eikelenboom.it>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
Date: Sunday, November 14, 2010, 12:19 PM

Perhaps these patches (especially Stefano''s) could make a difference ?

http://lkml.indiana.edu/hypermail/linux/kernel/1011.1/02348.html

I don''t think these are applied to your domU kernel  since
it''s git8, it worth the shot i guess...

--
Sander

Sunday, November 14, 2010, 6:09:14 PM, you wrote:
>> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> I''ve tried scp a lot from DomU to Dom0 on F14 no problems, on
Ubuntu 10.04 Server
> DomU as NFS client cannot survive more then a 2-3 min with I/O generated by
scp.
> If i unmount NFS share everything goes fine ( Ubuntu 10.04)
>> That would perhaps pinpoint it to NFS doing something not completely
compatible with
> Xen.
> If DomU ( Ubuntu 10.04 ) is running  by 2.6.36 kernel there is no any NFS
or other network related problems . Issue is specific for 2.6.37-rc1 kernel as
kernel running DomU at Xen 4.0.1 Dom0 ( 2.6.32.25 pvops) on top of Ubuntu Lucid
Server.
> Boris.



> -- On Sun, 11/14/10, Sander Eikelenboom <linux@eikelenboom.it> wrote:
> From: Sander Eikelenboom <linux@eikelenboom.it>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
> Date: Sunday, November 14, 2010, 11:56 AM
> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely
compatible with Xen.
> I''m not using NFS (I still use file: based guests, and i use
glusterfs (fuse based userspace cluster fs) to share diskspace to
domU''s via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more
problems with disconnects.
> I haven''t seen any "unable to handle page request"
problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
> --
> Sander



> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
>> Boris.
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>> 
>>> Perhaps handpick some kernels before and after the pulls of the xen
patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end
up with kernels in between patch series, resulting in panics.
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to
work.
>> But that is the theory and ..
>>> > The git bisecting is slow going. I''ve never tried
that before and I''m a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first
2 bisects all
>>> > panic at boot so I''m obviously doing something wrong.
>>> > I''ll RTFM a bit more and keep at it.
>> .. as Bruce experiences this is not the case. Hmm..
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel

>>       





-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it




      


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Sander Eikelenboom

2010-Nov-14 17:52 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Ahh i see that patch contains the patches i mentioned already.


Sunday, November 14, 2010, 6:47:25 PM, you wrote:
> This is not exactly git8 . Michael Young applied attached patch on top of
git8.
> It''s  his the most recent submission to fedora-xen.
> http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434 
> Boris.
> --- On Sun, 11/14/10, Sander Eikelenboom <linux@eikelenboom.it>
wrote:
> From: Sander Eikelenboom <linux@eikelenboom.it>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
> Date: Sunday, November 14, 2010, 12:19 PM
> Perhaps these patches (especially Stefano''s) could make a
difference ?
> http://lkml.indiana.edu/hypermail/linux/kernel/1011.1/02348.html
> I don''t think these are applied to your domU kernel  since
it''s git8, it worth the shot i guess...
> --
> Sander
> Sunday, November 14, 2010, 6:09:14 PM, you wrote:
>>> Hmmm have you tried do do a lot of I/O with something else as NFS ?
>> I''ve tried scp a lot from DomU to Dom0 on F14 no problems, on
Ubuntu 10.04 Server
>> DomU as NFS client cannot survive more then a 2-3 min with I/O
generated by scp.
>> If i unmount NFS share everything goes fine ( Ubuntu 10.04)
>>> That would perhaps pinpoint it to NFS doing something not
completely compatible with
>> Xen.
>> If DomU ( Ubuntu 10.04 ) is running  by 2.6.36 kernel there is no any
NFS or other network related problems . Issue is specific for 2.6.37-rc1 kernel
as kernel running DomU at Xen 4.0.1 Dom0 ( 2.6.32.25 pvops) on top of Ubuntu
Lucid Server.
>> Boris.



>> -- On Sun, 11/14/10, Sander Eikelenboom <linux@eikelenboom.it>
wrote:
>> From: Sander Eikelenboom <linux@eikelenboom.it>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
>> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
>> Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>> Date: Sunday, November 14, 2010, 11:56 AM
>> Hmmm have you tried do do a lot of I/O with something else as NFS ?
>> That would perhaps pinpoint it to NFS doing something not completely
compatible with Xen.
>> I''m not using NFS (I still use file: based guests, and i use
glusterfs (fuse based userspace cluster fs) to share diskspace to
domU''s via ethernet).
>> I tried NFS in the past, but had some troubles setting it up, and even
more problems with disconnects.
>> I haven''t seen any "unable to handle page request"
problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>> --
>> Sander



>> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>>> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
>>> Boris.
>>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable
to handle kernel paging request
>>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>>> Date: Friday, November 12, 2010, 12:01 PM
>>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>>> Hi Bruce,
>>>> 
>>>> Perhaps handpick some kernels before and after the pulls of the
xen patches (pv-on-hvm etc) to begin with ?
>>>> When you let git choose, especially with rc-1 kernels, you will
end up with kernels in between patch series, resulting in panics.
>>> Well, just the bare-bone boot of PV guests with nothing fancy ought
to work.
>>> But that is the theory and ..
>>>> > The git bisecting is slow going. I''ve never tried
that before and I''m a git
>>>> > rookie.
>>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my
first 2 bisects all
>>>> > panic at boot so I''m obviously doing something
wrong.
>>>> > I''ll RTFM a bit more and keep at it.
>>> .. as Bruce experiences this is not the case. Hmm..
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel

>>>       








-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Nov-14 21:35 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely
compatible with Xen.
I have my own suspicions regarding the more recent NFS clients. Post
10.04 Ubuntu variants do not tolerate large NFS transfers even without
Xen. Any more than a few 100 Megs and you start getting ''task blocked
for more than 120 sec..." messages along with stack traces showing
part of the NFS call stack.
Perhaps a parallel effort could be to test the 2.6.37-rc1 kernel with
something other than NFS for remote filesystems. I''ll see if I get the
same problems with glusterfs.

-Bruce
>
> I''m not using NFS (I still use file: based guests, and i use
glusterfs (fuse based userspace cluster fs) to share diskspace to
domU''s via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more
problems with disconnects.
>
> I haven''t seen any "unable to handle page request"
problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen
patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end
up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to
work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I''ve never tried
that before and I''m a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first
2 bisects all
>>> > panic at boot so I''m obviously doing something wrong.
>>> > I''ll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
>  Sander                            mailto:linux@eikelenboom.it
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-15 08:06 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Stack trace on f14 when working with NFS mount

[  218.984818] ------------[ cut here ]------------
[  218.984834] kernel BUG at mm/mmap.c:2399!
[  218.984844] invalid opcode: 0000 [#1] SMP 
[  218.984857] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[  218.984872] CPU 1 
[  218.984879] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia
cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64
twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd
aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4
xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro
xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel
tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ip6t_REJECT
nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipv6 uinput
xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan]
[  218.985011] 
[  218.985011] Pid: 1566, comm: ls Not tainted
2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[  218.985011] RIP: e030:[<ffffffff8110ada1>]  [<ffffffff8110ada1>]
exit_mmap+0x10c/0x119
[  218.985011] RSP: e02b:ffff8800774a9e18  EFLAGS: 00010202
[  218.985011] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0020000000000000
[  218.985011] RDX: 0000000000100004 RSI: ffff8800770ea1b8 RDI: ffffea0001a00230
[  218.985011] RBP: ffff8800774a9e48 R08: ffff88007d045108 R09: 000000000000005a
[  218.985011] R10: ffffffff8100750f R11: ffffea000182b7b0 R12: ffff880077dc6300
[  218.985011] R13: ffff88007fa1b1e0 R14: ffff880077dc6368 R15: 0000000000000001
[  218.985011] FS:  00007f4a38dd17c0(0000) GS:ffff88007fa0d000(0000)
knlGS:0000000000000000
[  218.985011] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  218.985011] CR2: 00007f4a380a1940 CR3: 0000000001a03000 CR4: 0000000000002660
[  218.985011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  218.985011] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  218.985011] Process ls (pid: 1566, threadinfo ffff8800774a8000, task
ffff880003ca47c0)
[  218.985011] Stack:
[  218.985011]  000000000000006b ffff88007fa1b1e0 ffff8800774a9e38
ffff880077dc6300
[  218.985011]  ffff880077dc6440 ffff880003ca4db0 ffff8800774a9e68
ffffffff810505fc
[  218.985011]  ffff880003ca47c0 ffff880077dc6300 ffff8800774a9eb8
ffffffff81056747
[  218.985011] Call Trace:
[  218.985011]  [<ffffffff810505fc>] mmput+0x65/0xd8
[  218.985011]  [<ffffffff81056747>] exit_mm+0x13e/0x14b
[  218.985011]  [<ffffffff81056976>] do_exit+0x222/0x7c6
[  218.985011]  [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[  218.985011]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
[  218.985011]  [<ffffffff814b3949>] ? lockdep_sys_exit_thunk+0x35/0x67
[  218.985011]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
[  218.985011]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
[  218.985011]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
[  218.985011] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8
0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f>
0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
[  218.985011] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[  218.985011]  RSP <ffff8800774a9e18>
[  218.985011] ---[ end trace 99b09fa378e85262 ]---
[  218.985011] Fixing recursive fault but reboot is needed!

Message from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.984818] ------------[ cut here ]------------

Message from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.984844] invalid opcode: 0000 [#1] SMP 

Message from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.984857] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map

Message from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.985011] Stack:

Message from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.985011] Call Trace:

Message from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.985011] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48
89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
<0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
[  259.093423] BUG: unable to handle kernel paging request at ffff880077d352a8
[  259.093441] IP: [<ffffffff81037648>] ptep_set_access_flags+0x2b/0x51
[  259.093456] PGD 1a04067 PUD 59c9067 PMD 5b88067 PTE 8010000077d35065
[  259.093472] Oops: 0003 [#2] SMP 
[  259.093481] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[  259.093493] CPU 1 
[  259.093498] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia
cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64
twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd
aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4
xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro
xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel
tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ip6t_REJECT
nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipv6 uinput
xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan]
[  259.093652] 
[  259.093658] Pid: 1567, comm: abrtd Tainted: G      D    
2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[  259.093669] RIP: e030:[<ffffffff81037648>]  [<ffffffff81037648>]
ptep_set_access_flags+0x2b/0x51
[  259.093683] RSP: e02b:ffff8800770e7bf8  EFLAGS: 00010202
[  259.093690] RAX: 80000001bf75f101 RBX: ffff880077521400 RCX: 80000001bf75f167
[  259.093699] RDX: ffff880077d352a8 RSI: 00007fb9b9255ad0 RDI: ffff880077521400
[  259.093708] RBP: ffff8800770e7c28 R08: 0000000000000001 R09: 1580000000000000
[  259.093717] R10: ffffffff8100750f R11: ffff880077dc5800 R12: 00007fb9b9255ad0
[  259.093726] R13: 0000000000000001 R14: ffff880003f2f9f8 R15: ffff880077d352a8
[  259.093737] FS:  00007fb9b9255800(0000) GS:ffff88007fa0d000(0000)
knlGS:0000000000000000
[  259.093747] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  259.093755] CR2: ffff880077d352a8 CR3: 00000000043c8000 CR4: 0000000000002660
[  259.093764] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  259.093773] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  259.093783] Process abrtd (pid: 1567, threadinfo ffff8800770e6000, task
ffff880003d2c7c0)
[  259.093800] Stack:
[  259.093807]  ffffea00018382b0 0000000000000000 0000000000000034
0000000000000000
[  259.093829]  ffff880077521400 0000000000000000 ffff8800770e7cb8
ffffffff81104a57
[  259.093851]  ffffffff810050a3 ffffffff00000001 ffff880004307e48
ffff8800770e7ca8
[  259.093873] Call Trace:
[  259.093885]  [<ffffffff81104a57>] do_wp_page+0x241/0x53d
[  259.093899]  [<ffffffff810050a3>] ? xen_pte_val+0x6a/0x6c
[  259.093911]  [<ffffffff81004635>] ?
__raw_callee_save_xen_pte_val+0x11/0x1e
[  259.093926]  [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[  259.093941]  [<ffffffff81106491>] ? handle_mm_fault+0x6ea/0x7af
[  259.093954]  [<ffffffff811064e2>] handle_mm_fault+0x73b/0x7af
[  259.093969]  [<ffffffff81073597>] ? down_read_trylock+0x44/0x4e
[  259.093983]  [<ffffffff814b7aa4>] do_page_fault+0x363/0x385
[  259.093996]  [<ffffffff81006f59>] ? xen_force_evtchn_callback+0xd/0xf
[  259.094011]  [<ffffffff81007522>] ? check_events+0x12/0x20
[  259.094025]  [<ffffffff814b3912>] ? trace_hardirqs_off_thunk+0x3a/0x3c
[  259.094039]  [<ffffffff814b4ad5>] page_fault+0x25/0x30
[  259.094053]  [<ffffffff8125403d>] ? __put_user_4+0x1d/0x30
[  259.094066]  [<ffffffff8104bf66>] ? schedule_tail+0x61/0x65
[  259.094079]  [<ffffffff8100abf3>] ret_from_fork+0x13/0x80
[  259.094089] Code: 55 48 89 e5 41 55 41 54 53 48 83 ec 18 0f 1f 44 00 00 48 39
0a 48 89 fb 49 89 f4 0f 95 c0 45 85 c0 44 0f b6 e8 74 1c 84 c0 74 18 <48>
89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 bb
[  259.094149] RIP  [<ffffffff81037648>] ptep_set_access_flags+0x2b/0x51
[  259.094149]  RSP <ffff8800770e7bf8>
[  259.094149] CR2: ffff880077d352a8
[  259.094149] ---[ end trace 99b09fa378e85263 ]---

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.093472] Oops: 0003 [#2] SMP 

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.093481] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.093800] Stack:

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.093873] Call Trace:

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.094089] Code: 55 48 89 e5 41 55 41 54 53 48 83 ec 18 0f 1f 44 00
00 48 39 0a 48 89 fb 49 89 f4 0f 95 c0 45 85 c0 44 0f b6 e8 74 1c 84 c0 74 18
<48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 bb

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.094149] CR2: ffff880077d352a8


--- On Sun, 11/14/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Sunday, November 14, 2010, 4:35 PM

On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely
compatible with Xen.
I have my own suspicions regarding the more recent NFS clients. Post
10.04 Ubuntu variants do not tolerate large NFS transfers even without
Xen. Any more than a few 100 Megs and you start getting ''task blocked
for more than 120 sec..." messages along with stack traces showing
part of the NFS call stack.
Perhaps a parallel effort could be to test the 2.6.37-rc1 kernel with
something other than NFS for remote filesystems. I''ll see if I get the
same problems with glusterfs.

-Bruce
>
> I''m not using NFS (I still use file: based guests, and i use
glusterfs (fuse based userspace cluster fs) to share diskspace to
domU''s via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more
problems with disconnects.
>
> I haven''t seen any "unable to handle page request"
problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen
patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end
up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to
work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I''ve never tried
that before and I''m a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first
2 bisects all
>>> > panic at boot so I''m obviously doing something wrong.
>>> > I''ll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
>  Sander                            mailto:linux@eikelenboom.it
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-15 11:05 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Mount with stock kernel in PV DomU - No problems
---------------------------------------------------------------------
Started domain F14PV (id=4)
                           [    0.030070] PCI: Fatal: No config space access
function found
[    0.142207] drivers/rtc/hctosys.c: unable to open rtc device (rtc0)

Fedora release 14 (Laughlin)
Kernel 2.6.35.6-45.fc14.x86_64 on an x86_64 (/dev/hvc0)

fedora14 login: root
Password: 
Last login: Mon Nov 15 13:42:03 on hvc0
[root@fedora14 ~]# mount 192.168.1.9:/home/boris /mnt/nfs
[root@fedora14 ~]# 
----------------------------------------------------------------------
                        |
                        vs  
                        |
Mount with the most recent Michael Young kernel - Crashing kernel
----------------------------------------------------------------------
Fedora release 14 (Laughlin)
Kernel 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 on an x86_64 (/dev/hvc0)

fedora14 login: root
Password: [   25.825048] eth0: no IPv6 routers present

Last login: Mon Nov 15 13:48:31 on hvc0
[root@fedora14 ~]# mount 192.168.1.9:/home/boris /mnt/nfs
[   44.240979] FS-Cache: Loaded
[   44.275659] FS-Cache: Netfs ''nfs'' registered for caching
[root@fedora14 ~]# 
------------------------------------------------------------------------------
Boris.

--- On Mon, 11/15/10, Boris Derzhavets <bderzhavets@yahoo.com> wrote:

From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>, "Bruce
Edge" <bruce.edge@gmail.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Monday, November 15, 2010, 3:06 AM

Stack trace on f14 when working with NFS mount

[  218.984818] ------------[ cut here ]------------
[  218.984834] kernel BUG at mm/mmap.c:2399!
[  218.984844] invalid opcode: 0000 [#1] SMP 
[  218.984857] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[  218.984872] CPU 1 
[  218.984879] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia
cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64
twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd
aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4
xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro
xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel
tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs
 sunrpc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables
ipv6 uinput xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan]
[  218.985011] 
[  218.985011] Pid: 1566, comm: ls Not tainted
2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[  218.985011] RIP: e030:[<ffffffff8110ada1>]  [<ffffffff8110ada1>]
exit_mmap+0x10c/0x119
[  218.985011] RSP: e02b:ffff8800774a9e18  EFLAGS: 00010202
[  218.985011] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0020000000000000
[  218.985011] RDX: 0000000000100004 RSI: ffff8800770ea1b8 RDI: ffffea0001a00230
[  218.985011] RBP: ffff8800774a9e48 R08: ffff88007d045108 R09: 000000000000005a
[  218.985011] R10: ffffffff8100750f R11: ffffea000182b7b0 R12: ffff880077dc6300
[  218.985011] R13: ffff88007fa1b1e0 R14: ffff880077dc6368 R15: 0000000000000001
[  218.985011] FS: 
 00007f4a38dd17c0(0000) GS:ffff88007fa0d000(0000) knlGS:0000000000000000
[  218.985011] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  218.985011] CR2: 00007f4a380a1940 CR3: 0000000001a03000 CR4: 0000000000002660
[  218.985011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  218.985011] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  218.985011] Process ls (pid: 1566, threadinfo ffff8800774a8000, task
ffff880003ca47c0)
[  218.985011] Stack:
[  218.985011]  000000000000006b ffff88007fa1b1e0 ffff8800774a9e38
ffff880077dc6300
[  218.985011]  ffff880077dc6440 ffff880003ca4db0 ffff8800774a9e68
ffffffff810505fc
[  218.985011]  ffff880003ca47c0 ffff880077dc6300 ffff8800774a9eb8
ffffffff81056747
[  218.985011] Call Trace:
[  218.985011]  [<ffffffff810505fc>] mmput+0x65/0xd8
[ 
 218.985011]  [<ffffffff81056747>] exit_mm+0x13e/0x14b
[  218.985011]  [<ffffffff81056976>] do_exit+0x222/0x7c6
[  218.985011]  [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[  218.985011]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
[  218.985011]  [<ffffffff814b3949>] ? lockdep_sys_exit_thunk+0x35/0x67
[  218.985011]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
[  218.985011]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
[  218.985011]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
[  218.985011] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8
0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f>
0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
[  218.985011] RIP  [<ffffffff8110ada1>]
 exit_mmap+0x10c/0x119
[  218.985011]  RSP <ffff8800774a9e18>
[  218.985011] ---[ end trace 99b09fa378e85262 ]---
[  218.985011] Fixing recursive fault but reboot is needed!

Message from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.984818] ------------[ cut here ]------------

Message from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.984844] invalid opcode: 0000 [#1] SMP 

Message from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.984857] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map

Message from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.985011] Stack:

Message from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.985011] Call Trace:

Message from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.985011] Code: 8d 7d 18 e8 c3
 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75
f0 49 83 bc 24 98 01 00 00 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9
c3 55 48 89 e5 41 54 53 48
[  259.093423] BUG: unable to handle kernel paging request at ffff880077d352a8
[  259.093441] IP: [<ffffffff81037648>] ptep_set_access_flags+0x2b/0x51
[  259.093456] PGD 1a04067 PUD 59c9067 PMD 5b88067 PTE 8010000077d35065
[  259.093472] Oops: 0003 [#2] SMP 
[  259.093481] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[  259.093493] CPU 1 
[  259.093498] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia
cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64
twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd
aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4
xfrm4_mode_tunnel
 xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet
xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd
lockd nfs_acl auth_rpcgss exportfs sunrpc ip6t_REJECT nf_conntrack_ipv6
nf_defrag_ipv6 ip6table_filter ip6_tables ipv6 uinput xen_netfront microcode
xen_blkfront [last unloaded: scsi_wait_scan]
[  259.093652] 
[  259.093658] Pid: 1567, comm: abrtd Tainted: G      D    
2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[  259.093669] RIP: e030:[<ffffffff81037648>]  [<ffffffff81037648>]
ptep_set_access_flags+0x2b/0x51
[  259.093683] RSP: e02b:ffff8800770e7bf8  EFLAGS: 00010202
[  259.093690] RAX: 80000001bf75f101 RBX: ffff880077521400 RCX: 80000001bf75f167
[  259.093699] RDX: ffff880077d352a8 RSI: 00007fb9b9255ad0 RDI: ffff880077521400
[  259.093708] RBP: ffff8800770e7c28 R08:
 0000000000000001 R09: 1580000000000000
[  259.093717] R10: ffffffff8100750f R11: ffff880077dc5800 R12: 00007fb9b9255ad0
[  259.093726] R13: 0000000000000001 R14: ffff880003f2f9f8 R15: ffff880077d352a8
[  259.093737] FS:  00007fb9b9255800(0000) GS:ffff88007fa0d000(0000)
knlGS:0000000000000000
[  259.093747] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  259.093755] CR2: ffff880077d352a8 CR3: 00000000043c8000 CR4: 0000000000002660
[  259.093764] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  259.093773] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  259.093783] Process abrtd (pid: 1567, threadinfo ffff8800770e6000, task
ffff880003d2c7c0)
[  259.093800] Stack:
[  259.093807]  ffffea00018382b0 0000000000000000 0000000000000034
0000000000000000
[  259.093829]  ffff880077521400 0000000000000000
 ffff8800770e7cb8 ffffffff81104a57
[  259.093851]  ffffffff810050a3 ffffffff00000001 ffff880004307e48
ffff8800770e7ca8
[  259.093873] Call Trace:
[  259.093885]  [<ffffffff81104a57>] do_wp_page+0x241/0x53d
[  259.093899]  [<ffffffff810050a3>] ? xen_pte_val+0x6a/0x6c
[  259.093911]  [<ffffffff81004635>] ?
__raw_callee_save_xen_pte_val+0x11/0x1e
[  259.093926]  [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[  259.093941]  [<ffffffff81106491>] ? handle_mm_fault+0x6ea/0x7af
[  259.093954]  [<ffffffff811064e2>] handle_mm_fault+0x73b/0x7af
[  259.093969]  [<ffffffff81073597>] ? down_read_trylock+0x44/0x4e
[  259.093983]  [<ffffffff814b7aa4>] do_page_fault+0x363/0x385
[  259.093996]  [<ffffffff81006f59>] ? xen_force_evtchn_callback+0xd/0xf
[ 
 259.094011]  [<ffffffff81007522>] ? check_events+0x12/0x20
[  259.094025]  [<ffffffff814b3912>] ? trace_hardirqs_off_thunk+0x3a/0x3c
[  259.094039]  [<ffffffff814b4ad5>] page_fault+0x25/0x30
[  259.094053]  [<ffffffff8125403d>] ? __put_user_4+0x1d/0x30
[  259.094066]  [<ffffffff8104bf66>] ? schedule_tail+0x61/0x65
[  259.094079]  [<ffffffff8100abf3>] ret_from_fork+0x13/0x80
[  259.094089] Code: 55 48 89 e5 41 55 41 54 53 48 83 ec 18 0f 1f 44 00 00 48 39
0a 48 89 fb 49 89 f4 0f 95 c0 45 85 c0 44 0f b6 e8 74 1c 84 c0 74 18 <48>
89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 bb
[  259.094149] RIP  [<ffffffff81037648>] ptep_set_access_flags+0x2b/0x51
[  259.094149]  RSP <ffff8800770e7bf8>
[  259.094149] CR2: ffff880077d352a8
[  259.094149] ---[ end trace 99b09fa378e85263
 ]---

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.093472] Oops: 0003 [#2] SMP 

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.093481] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.093800] Stack:

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.093873] Call Trace:

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.094089] Code: 55 48 89 e5 41 55 41 54 53 48 83 ec 18 0f 1f 44 00
00 48 39 0a 48 89 fb 49 89 f4 0f 95 c0 45 85 c0 44 0f b6 e8 74 1c 84 c0 74 18
<48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 bb

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.094149] CR2: ffff880077d352a8


--- On Sun, 11/14/10, Bruce
 Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Sunday, November 14, 2010, 4:35 PM

On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely compatible with Xen.

I have my own suspicions regarding the more recent NFS clients. Post
10.04 Ubuntu variants do not tolerate large NFS transfers even without
Xen. Any more than a few 100 Megs and you start getting ''task blocked
for more than 120 sec..." messages along with stack traces showing
part of the NFS call stack.
Perhaps a parallel effort could be to test the 2.6.37-rc1 kernel with
something other than NFS for remote filesystems. I''ll see if I get the
same problems with glusterfs.

-Bruce
>
> I''m not using NFS (I still use file: based guests, and i use
glusterfs (fuse based userspace cluster fs) to share diskspace to
domU''s via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more
problems with disconnects.
>
> I haven''t seen any "unable to handle page request"
problems with my mix of guest kernels, which includes some 2.6.37-rc1
 kernels.>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
 <jeremy@goop.org>>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen
patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end
up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to
work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I''ve never tried
that before and I''m a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first
2 bisects all
>>> > panic at boot so I''m
 obviously doing something wrong.>>> > I''ll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
>  Sander                            mailto:linux@eikelenboom.it
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      
-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-15 14:48 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request, nfsd changes for 2.6.37

My skills are not good enough to decide might it affect Xen or not.

Please, view :-
http://www.spinics.net/lists/linux-nfs/msg16893.html
http://www.spinics.net/lists/linux-nfs/msg17071.html

Boris.
P.S. Finally,  i got F14 PV DomU crashing , when running NFS client, as well.
Just get crash on F14 takes longer then on Ubuntu 10.04.1. As soon as it
crashed once it started  crashing on regular basis ( looks like a kind of
locking
mechanism).


--- On Sun, 11/14/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, "Konrad
Rzeszutek Wilk" <konrad.wilk@oracle.com>,
xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>
Date: Sunday, November 14, 2010, 4:35 PM

On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely
compatible with Xen.
I have my own suspicions regarding the more recent NFS clients. Post
10.04 Ubuntu variants do not tolerate large NFS transfers even without
Xen. Any more than a few 100 Megs and you start getting ''task blocked
for more than 120 sec..." messages along with stack traces showing
part of the NFS call stack.
Perhaps a parallel effort could be to test the 2.6.37-rc1 kernel with
something other than NFS for remote filesystems. I''ll see if I get the
same problems with glusterfs.

-Bruce
>
> I''m not using NFS (I still use file: based guests, and i use
glusterfs (fuse based userspace cluster fs) to share diskspace to
domU''s via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more
problems with disconnects.
>
> I haven''t seen any "unable to handle page request"
problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen
patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end
up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to
work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I''ve never tried
that before and I''m a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first
2 bisects all
>>> > panic at boot so I''m obviously doing something wrong.
>>> > I''ll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
>  Sander                            mailto:linux@eikelenboom.it
>
>


      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-15 17:14 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request as Dom0 kernel

I''ve also tested the most recent kernel from Michael Young
http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434]
  kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64(rc1.git8 +
xen-next.patch) 
as Dom0 kernel under Xen 4.0.1 on top of F14.
Attempt to work with NFS remote folder results kernel crash again
- unable to handle paging request and stack trace.

Kernel works OK only as vanilla.

Boris.

--- On Sun, 11/14/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Sunday, November 14, 2010, 4:35 PM

On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely
compatible with Xen.
I have my own suspicions regarding the more recent NFS clients. Post
10.04 Ubuntu variants do not tolerate large NFS transfers even without
Xen. Any more than a few 100 Megs and you start getting ''task blocked
for more than 120 sec..." messages along with stack traces showing
part of the NFS call stack.
Perhaps a parallel effort could be to test the 2.6.37-rc1 kernel with
something other than NFS for remote filesystems. I''ll see if I get the
same problems with glusterfs.

-Bruce
>
> I''m not using NFS (I still use file: based guests, and i use
glusterfs (fuse based userspace cluster fs) to share diskspace to
domU''s via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more
problems with disconnects.
>
> I haven''t seen any "unable to handle page request"
problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen
patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end
up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to
work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I''ve never tried
that before and I''m a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first
2 bisects all
>>> > panic at boot so I''m obviously doing something wrong.
>>> > I''ll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
>  Sander                            mailto:linux@eikelenboom.it
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Nov-15 20:21 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely
compatible with Xen.
>
> I''m not using NFS (I still use file: based guests, and i use
glusterfs (fuse based userspace cluster fs) to share diskspace to
domU''s via ethernet).
Sander,
I took a quick look at glusterfs and it uses the same nfs client:
http://www.gluster.com/community/documentation/index.php/Gluster_3.1:_Manually_Mounting_Volumes_Using_NFS
I''m assuming that this would cause the same problem on the VM as
we''re
seeing with NFS, unless it''s really an Xen/dom0 NFS server problem
that triggering the domU nfs client crash.

Perhaps the context is different. I''m exporting filesystems to domU
from dom0 via NFS. Is that how you''re using glusterfs, or are you
using it to host your file backed VM''s storage? In the latter, that
may explain why you''re not seeing these problems because
you''re not
using the nfs client on domU.

> I tried NFS in the past, but had some troubles setting it up, and even more
problems with disconnects.
What kind of NFS problems? It was working very well for us until this
problem cropped up.

-Bruce
>
> I haven''t seen any "unable to handle page request"
problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen
patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end
up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to
work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I''ve never tried
that before and I''m a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first
2 bisects all
>>> > panic at boot so I''m obviously doing something wrong.
>>> > I''ll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
>  Sander                            mailto:linux@eikelenboom.it
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Sander Eikelenboom

2010-Nov-15 20:32 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Monday, November 15, 2010, 9:21:28 PM, you wrote:
> On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
> <linux@eikelenboom.it> wrote:
>> Hmmm have you tried do do a lot of I/O with something else as NFS ?
>> That would perhaps pinpoint it to NFS doing something not completely
compatible with Xen.
>>
>> I''m not using NFS (I still use file: based guests, and i use
glusterfs (fuse based userspace cluster fs) to share diskspace to
domU''s via ethernet).
> Sander,
> I took a quick look at glusterfs and it uses the same nfs client:
>
http://www.gluster.com/community/documentation/index.php/Gluster_3.1:_Manually_Mounting_Volumes_Using_NFS
> I''m assuming that this would cause the same problem on the VM as
we''re
> seeing with NFS, unless it''s really an Xen/dom0 NFS server problem
> that triggering the domU nfs client crash.
> Perhaps the context is different. I''m exporting filesystems to
domU
> from dom0 via NFS. Is that how you''re using glusterfs, or are you
> using it to host your file backed VM''s storage? In the latter,
that
> may explain why you''re not seeing these problems because
you''re not
> using the nfs client on domU.
Nope it CAN export volumes as NFS, but that''s the non native way.
I don''t know what distribution you use, but for debian there are
packages available
http://packages.debian.org/search?keywords=glusterfs&searchon=names&suite=all&section=all
in the backports repository.

And indeed i don''t use NFS anywhere. (I have tried, but had
configuration and other problems. So i looked for an alternative.)

--
Sander

>> I tried NFS in the past, but had some troubles setting it up, and even
more problems with disconnects.
> What kind of NFS problems? It was working very well for us until this
> problem cropped up.
> -Bruce
>>
>> I haven''t seen any "unable to handle page request"
problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>>
>> --
>>
>> Sander
>>
>>
>>
>>
>>
>> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>>
>>> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
>>
>>> Boris.
>>
>>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>>
>>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable
to handle kernel paging request
>>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>>> Date: Friday, November 12, 2010, 12:01 PM
>>
>>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>>> Hi Bruce,
>>>>
>>>> Perhaps handpick some kernels before and after the pulls of the
xen patches (pv-on-hvm etc) to begin with ?
>>>> When you let git choose, especially with rc-1 kernels, you will
end up with kernels in between patch series, resulting in panics.
>>
>>> Well, just the bare-bone boot of PV guests with nothing fancy ought
to work.
>>
>>> But that is the theory and ..
>>>> > The git bisecting is slow going. I''ve never tried
that before and I''m a git
>>>> > rookie.
>>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my
first 2 bisects all
>>>> > panic at boot so I''m obviously doing something
wrong.
>>>> > I''ll RTFM a bit more and keep at it.
>>
>>> .. as Bruce experiences this is not the case. Hmm..
>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>>
>>
>>
>>>
>>
>>
>>
>> --
>> Best regards,
>>  Sander                            mailto:linux@eikelenboom.it
>>
>>


-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-16 18:43 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Today i''ve built vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64
via Michael''s
http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
and uncommented xen.pcifront.fixes.patch in kernel.spec, i.e.

# Xen patches
ApplyPatch xen.next-2.6.37.patch
# ApplyPatch xen.upstream.core.patch
ApplyPatch xen.pcifront.fixes.patch
# ApplyPatch xen.pvhvm.fixes.patch

as a fesult a got kernel wich runs pretty stable NFS client at Xen 4.0.1 
F14 Dom0 (2.6.32.25-172.xendom0.fc14.x86_64).
  
  I was able several times copied from NFS folder F14''s ISO image (3.2
GB)
to DomU and scp''ed it back and didn''t get any kernel crashing
on DomU.

 On Ubuntu 10.04 this kernel may be built as 2.6.37-rc1&git8 patched via
xen.next-2.6.37.patch
xen.pcifront.fixes.patch
All required upstream patches may be taken (as well as 2 above)
from  http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
  I believe as soon as xen.pcifront.fixes.patch will be accepted by upstream
NFS client issue on F14 will be gone 

Boris


--- On Mon, 11/15/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, "Konrad
Rzeszutek Wilk" <konrad.wilk@oracle.com>,
xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>
Date: Monday, November 15, 2010, 3:21 PM

On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely
compatible with Xen.
>
> I''m not using NFS (I still use file: based guests, and i use
glusterfs (fuse based userspace cluster fs) to share diskspace to
domU''s via ethernet).
Sander,
I took a quick look at glusterfs and it uses the same nfs client:
http://www.gluster.com/community/documentation/index.php/Gluster_3.1:_Manually_Mounting_Volumes_Using_NFS
I''m assuming that this would cause the same problem on the VM as
we''re
seeing with NFS, unless it''s really an Xen/dom0 NFS server problem
that triggering the domU nfs client crash.

Perhaps the context is different. I''m exporting filesystems to domU
from dom0 via NFS. Is that how you''re using glusterfs, or are you
using it to host your file backed VM''s storage? In the latter, that
may explain why you''re not seeing these problems because
you''re not
using the nfs client on domU.

> I tried NFS in the past, but had some troubles setting it up, and even more
problems with disconnects.
What kind of NFS problems? It was working very well for us until this
problem cropped up.

-Bruce
>
> I haven''t seen any "unable to handle page request"
problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14
Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied
700 MB ISO images from NFS folder at Dom0 to DomU and scp''ed them back
to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel
crash as "unable to handle page request" was reported once by F14
DomU, but it didn''t actually crash DomU. Same excersises with
replacement F14 by Ubuntu 10.04 Server results DomU crash in about several
minutes. Dom0''s instances dual boot on same development box (
Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen
patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end
up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to
work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I''ve never tried
that before and I''m a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first
2 bisects all
>>> > panic at boot so I''m obviously doing something wrong.
>>> > I''ll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
>  Sander                            mailto:linux@eikelenboom.it
>
>


      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Nov-16 19:00 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Tue, Nov 16, 2010 at 10:43:49AM -0800, Boris Derzhavets
wrote:> Today i''ve built vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64
> via Michael''s
http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
> and uncommented xen.pcifront.fixes.patch in kernel.spec, i.e.
> 
> # Xen patches
> ApplyPatch xen.next-2.6.37.patch
> # ApplyPatch xen.upstream.core.patch
> ApplyPatch xen.pcifront.fixes.patch
> # ApplyPatch xen.pvhvm.fixes.patch
> 
> as a fesult a got kernel wich runs pretty stable NFS client at Xen 4.0.1 
> F14 Dom0 (2.6.32.25-172.xendom0.fc14.x86_64).
>   
>   I was able several times copied from NFS folder F14''s ISO image
(3.2 GB)
> to DomU and scp''ed it back and didn''t get any kernel
crashing on DomU.
Huh. I .. what? I am confused. I thought we established that the issue
was not related to Xen PCI front? You also seem to uncomment the
upstream.core.patches and the xen.pvhvm.patch - why?
> 
>  On Ubuntu 10.04 this kernel may be built as 2.6.37-rc1&git8 patched
via
> xen.next-2.6.37.patch
> xen.pcifront.fixes.patch
> All required upstream patches may be taken (as well as 2 above)
> from  http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
>   I believe as soon as xen.pcifront.fixes.patch will be accepted by
upstream
> NFS client issue on F14 will be gone 
Ok, they are.. v2.6.37-rc2 which came out today has the fixes.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-16 19:20 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Sorry, patches for 2.6.37-rc1-git8 are attached.
You also would need :-

patch-2.6.37-rc1.bz2
patch-2.6.37-rc1-git8.bz2

All of them should be in /root/rpmbuild/SOURCES folder after src.rpm install on 
any F14,F13,F12. 

Boris.

--- On Tue, 11/16/10, Boris Derzhavets <bderzhavets@yahoo.com> wrote:

From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Bruce Edge" <bruce.edge@gmail.com>, "Konrad Rzeszutek
Wilk" <konrad.wilk@oracle.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com
Date: Tuesday, November 16, 2010, 1:43 PM

Today i''ve built vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64
via Michael''s
http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
and uncommented xen.pcifront.fixes.patch in kernel.spec, i.e.

# Xen patches
ApplyPatch xen.next-2.6.37.patch
# ApplyPatch xen.upstream.core.patch
ApplyPatch xen.pcifront.fixes.patch
# ApplyPatch xen.pvhvm.fixes.patch

as a fesult a got kernel wich runs pretty stable NFS client at Xen 4.0.1 
F14 Dom0 (2.6.32.25-172.xendom0.fc14.x86_64).
  
  I was able several times copied from NFS folder F14''s ISO image (3.2
GB)
to DomU and scp''ed it back and didn''t get any kernel crashing
on DomU.

 On Ubuntu 10.04 this kernel may be built as 2.6.37-rc1&git8 patched via
xen.next-2.6.37.patch
xen.pcifront.fixes.patch
All required upstream patches may be taken (as
 well as 2 above)
from  http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
  I believe as soon as xen.pcifront.fixes.patch will be accepted by upstream
NFS client issue on F14 will be gone 

Boris


--- On Mon, 11/15/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, "Konrad
Rzeszutek Wilk" <konrad.wilk@oracle.com>,
xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>
Date: Monday, November 15, 2010, 3:21 PM

On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely
compatible with Xen.
>
> I''m not using NFS (I still use file: based guests, and i use
glusterfs (fuse based userspace cluster fs) to share diskspace to
domU''s via ethernet).
Sander,
I took a quick look at glusterfs and it uses the same nfs client:
http://www.gluster.com/community/documentation/index.php/Gluster_3.1:_Manually_Mounting_Volumes_Using_NFS
I''m assuming that this would cause the same problem on the VM as
we''re
seeing with NFS, unless it''s really an Xen/dom0 NFS server problem
that triggering the domU nfs client
 crash.

Perhaps the context is different. I''m exporting filesystems to domU
from dom0 via NFS. Is that how you''re using glusterfs, or are you
using it to host your file backed VM''s storage? In the latter, that
may explain why you''re not seeing these problems because
you''re not
using the nfs client on domU.

> I tried NFS in the past, but had some troubles setting it up, and even more
problems with disconnects.
What kind of NFS problems? It was working very well for us until this
problem cropped up.

-Bruce
>
> I haven''t seen any "unable to handle page request"
problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I''ve tested F14 DomU (kernel
vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS
server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and
scp''ed them back to Dom0. During about 30 - 40 min DomU ran pretty
stable , regardless kernel crash as "unable to handle page request"
was reported once by F14 DomU, but it didn''t actually crash DomU. Same
excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in
about several minutes. Dom0''s instances dual boot on same development
box ( Q9500,ASUS P5Q3,8GB)>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging
request>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi
 Bruce,>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen
patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end
up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to
work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I''ve never tried
that before and I''m a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first
2 bisects all
>>> > panic at boot so I''m obviously doing something wrong.
>>> > I''ll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing
 list>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
>  Sander                            mailto:linux@eikelenboom.it
>
>








      
-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-16 20:43 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

> Huh. I .. what? I am confused. I thought we established that the issue
> was not related to Xen PCI front? You also seem to uncomment the
> upstream.core.patches and the xen.pvhvm.patch - why?
I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
it gives failed HUNKs
> Ok, they are.. v2.6.37-rc2 which came out today has the fixes
I am pretty sure rc2 doesn''t contain everything from
xen.next-2.6.37.patch,
gntdev''s stuff for sure. I''ve built 2.6.37-rc2 kernel rpms and
loaded
kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1. 
Device /dev/xen/gntdev has not been created. I understand that it''s
unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i cannot
get 3.2 GB copied over to DomU from NFS share at Dom0.
The best results for 2.6.37-rc1 + xen.next-2.6.37.patch +
xen.pcifront.fixes.patch

Boris.



--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Bruce Edge" <bruce.edge@gmail.com>,
xen-devel@lists.xensource.com, "Jeremy Fitzhardinge"
<jeremy@goop.org>
Date: Tuesday, November 16, 2010, 2:00 PM

On Tue, Nov 16, 2010 at 10:43:49AM -0800, Boris Derzhavets
wrote:> Today i''ve built vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64
> via Michael''s
http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
> and uncommented xen.pcifront.fixes.patch in kernel.spec, i.e.
> 
> # Xen patches
> ApplyPatch xen.next-2.6.37.patch
> # ApplyPatch xen.upstream.core.patch
> ApplyPatch xen.pcifront.fixes.patch
> # ApplyPatch xen.pvhvm.fixes.patch
> 
> as a fesult a got kernel wich runs pretty stable NFS client at Xen 4.0.1 
> F14 Dom0 (2.6.32.25-172.xendom0.fc14.x86_64).
>   
>   I was able several times copied from NFS folder F14''s ISO image
(3.2 GB)
> to DomU and scp''ed it back and didn''t get any kernel
crashing on DomU.
Huh. I .. what? I am confused. I thought we established that the issue
was not related to Xen PCI front? You also seem to uncomment the
upstream.core.patches and the xen.pvhvm.patch - why?
> 
>  On Ubuntu 10.04 this kernel may be built as 2.6.37-rc1&git8 patched
via
> xen.next-2.6.37.patch
> xen.pcifront.fixes.patch
> All required upstream patches may be taken (as well as 2 above)
> from  http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
>   I believe as soon as xen.pcifront.fixes.patch will be accepted by
upstream
> NFS client issue on F14 will be gone 
Ok, they are.. v2.6.37-rc2 which came out today has the fixes.



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-16 20:50 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Sorry, typo. Correct :-

The best results for 2.6.37-rc1-git8 + xen.next-2.6.37.patch +
 xen.pcifront.fixes.patch

Boris.

--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>
Date: Tuesday, November 16, 2010, 2:00 PM

On Tue, Nov 16, 2010 at 10:43:49AM -0800, Boris Derzhavets
wrote:> Today i''ve built vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64
> via Michael''s
http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
> and uncommented xen.pcifront.fixes.patch in kernel.spec, i.e.
> 
> # Xen patches
> ApplyPatch xen.next-2.6.37.patch
> # ApplyPatch xen.upstream.core.patch
> ApplyPatch xen.pcifront.fixes.patch
> # ApplyPatch xen.pvhvm.fixes.patch
> 
> as a fesult a got kernel wich runs pretty stable NFS client at Xen 4.0.1 
> F14 Dom0 (2.6.32.25-172.xendom0.fc14.x86_64).
>   
>   I was able several times copied from NFS folder F14''s ISO image
(3.2 GB)
> to DomU and scp''ed it back and didn''t get any kernel
crashing on DomU.
Huh. I .. what? I am confused. I thought we established that the issue
was not related to Xen PCI front? You also seem to uncomment the
upstream.core.patches and the xen.pvhvm.patch - why?
> 
>  On Ubuntu 10.04 this kernel may be built as 2.6.37-rc1&git8 patched
via
> xen.next-2.6.37.patch
> xen.pcifront.fixes.patch
> All required upstream patches may be taken (as well as 2 above)
> from  http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
>   I believe as soon as xen.pcifront.fixes.patch will be accepted by
upstream
> NFS client issue on F14 will be gone 
Ok, they are.. v2.6.37-rc2 which came out today has the fixes.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk

2010-Nov-16 21:15 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets
wrote:> > Huh. I .. what? I am confused. I thought we established that the issue
> > was not related to Xen PCI front? You also seem to uncomment the
> > upstream.core.patches and the xen.pvhvm.patch - why?
> 
> I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> it gives failed HUNKs
Uhh.. I am even more confused.> 
> > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
> 
> I am pretty sure rc2 doesn''t contain everything from
xen.next-2.6.37.patch,
> gntdev''s stuff for sure. I''ve built 2.6.37-rc2 kernel
rpms and loaded
> kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1. 
> Device /dev/xen/gntdev has not been created. I understand that
it''s
> unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i
cannot
> get 3.2 GB copied over to DomU from NFS share at Dom0.
So what I think you are saying is that you keep on getting the bug in DomU?
Is the stack-trace the same as in rc1?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-16 21:42 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

> So what I think you are saying is that you keep on getting the bug in DomU?
> Is the stack-trace the same as in rc1?
Yes.
When i want to get 1-2 hr of stable work :-

# service network restart
# service nfs restart

at Dom0.

I also believe that presence of xen-pcifront.fix.patch is making things much
more stable
on F14.

Boris.

--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>
Date: Tuesday, November 16, 2010, 4:15 PM

On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets
wrote:> > Huh. I .. what? I am confused. I thought we established that the issue
> > was not related to Xen PCI front? You also seem to uncomment the
> > upstream.core.patches and the xen.pvhvm.patch - why?
> 
> I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> it gives failed HUNKs
Uhh.. I am even more confused.> 
> > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
> 
> I am pretty sure rc2 doesn''t contain everything from
xen.next-2.6.37.patch,
> gntdev''s stuff for sure. I''ve built 2.6.37-rc2 kernel
rpms and loaded
> kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1. 
> Device /dev/xen/gntdev has not been created. I understand that
it''s
> unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i
cannot
> get 3.2 GB copied over to DomU from NFS share at Dom0.
So what I think you are saying is that you keep on getting the bug in DomU?
Is the stack-trace the same as in rc1?




      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-16 21:49 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Yes, here we are

[  186.975228] ------------[ cut here ]------------
[  186.975245] kernel BUG at mm/mmap.c:2399!
[  186.975254] invalid opcode: 0000 [#1] SMP 
[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[  186.975284] CPU 0 
[  186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia
cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64
twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd
aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4
xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro
xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel
tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ipv6 uinput
xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan]
[  186.975507] 
[  186.975515] Pid: 1562, comm: ls Not tainted
2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[  186.975529] RIP: e030:[<ffffffff8110ada1>]  [<ffffffff8110ada1>]
exit_mmap+0x10c/0x119
[  186.975550] RSP: e02b:ffff8800781bde18  EFLAGS: 00010202
[  186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[  186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI: ffffea00000c0280
[  186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09: 0000000000000001
[  186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12: ffff880076c68b00
[  186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15: 0000000000000001
[  186.975625] FS:  00007f8e471d97c0(0000) GS:ffff88007f831000(0000)
knlGS:0000000000000000
[  186.975639] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4: 0000000000002660
[  186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task
ffff8800788223e0)
[  186.976012] Stack:
[  186.976012]  000000000000006b ffff88007f83f1e0 ffff8800781bde38
ffff880076c68b00
[  186.976012]  ffff880076c68c40 ffff8800788229d0 ffff8800781bde68
ffffffff810505fc
[  186.976012]  ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8
ffffffff81056747
[  186.976012] Call Trace:
[  186.976012]  [<ffffffff810505fc>] mmput+0x65/0xd8
[  186.976012]  [<ffffffff81056747>] exit_mm+0x13e/0x14b
[  186.976012]  [<ffffffff81056976>] do_exit+0x222/0x7c6
[  186.976012]  [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[  186.976012]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
[  186.976012]  [<ffffffff814b3949>] ? lockdep_sys_exit_thunk+0x35/0x67
[  186.976012]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
[  186.976012]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
[  186.976012]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8
0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f>
0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
[  186.976012] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[  186.976012]  RSP <ffff8800781bde18>
[  186.976012] ---[ end trace c0f4eff4054a67e4 ]---
[  186.976012] Fixing recursive fault but reboot is needed!

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975228] ------------[ cut here ]------------

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975254] invalid opcode: 0000 [#1] SMP 

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Stack:

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Call Trace:

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48
89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
<0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48

--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>
Date: Tuesday, November 16, 2010, 4:15 PM

On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets
wrote:> > Huh. I .. what? I am confused. I thought we established that the issue
> > was not related to Xen PCI front? You also seem to uncomment the
> > upstream.core.patches and the xen.pvhvm.patch - why?
> 
> I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> it gives failed HUNKs
Uhh.. I am even more confused.> 
> > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
> 
> I am pretty sure rc2 doesn''t contain everything from
xen.next-2.6.37.patch,
> gntdev''s stuff for sure. I''ve built 2.6.37-rc2 kernel
rpms and loaded
> kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1. 
> Device /dev/xen/gntdev has not been created. I understand that
it''s
> unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i
cannot
> get 3.2 GB copied over to DomU from NFS share at Dom0.
So what I think you are saying is that you keep on getting the bug in DomU?
Is the stack-trace the same as in rc1?




      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Nov-17 21:28 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Tue, Nov 16, 2010 at 1:49 PM, Boris Derzhavets
<bderzhavets@yahoo.com>wrote:
> Yes, here we are
>
> [  186.975228] ------------[ cut here ]------------
> [  186.975245] kernel BUG at mm/mmap.c:2399!
> [  186.975254] invalid opcode: 0000 [#1] SMP
> [  186.975269] last sysfs file:
> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
> [  186.975284] CPU 0
> [  186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr
> camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic
> twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic
> des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet
> xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport
> xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp
> ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl
> auth_rpcgss exportfs sunrpc ipv6 uinput xen_netfront microcode xen_blkfront
> [last unloaded: scsi_wait_scan]
> [  186.975507]
> [  186.975515] Pid: 1562, comm: ls Not tainted
> 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
> [  186.975529] RIP: e030:[<ffffffff8110ada1>] 
[<ffffffff8110ada1>]
> exit_mmap+0x10c/0x119
> [  186.975550] RSP: e02b:ffff8800781bde18  EFLAGS: 00010202
> [  186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
> 0000000000000000
> [  186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI:
> ffffea00000c0280
> [  186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09:
> 0000000000000001
> [  186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12:
> ffff880076c68b00
> [  186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15:
> 0000000000000001
> [  186.975625] FS:  00007f8e471d97c0(0000) GS:ffff88007f831000(0000)
> knlGS:0000000000000000
> [  186.975639] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4:
> 0000000000002660
> [  186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [  186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [  186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task
> ffff8800788223e0)
> [  186.976012] Stack:
> [  186.976012]  000000000000006b ffff88007f83f1e0 ffff8800781bde38
> ffff880076c68b00
> [  186.976012]  ffff880076c68c40 ffff8800788229d0 ffff8800781bde68
> ffffffff810505fc
> [  186.976012]  ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8
> ffffffff81056747
> [  186.976012] Call Trace:
> [  186.976012]  [<ffffffff810505fc>] mmput+0x65/0xd8
> [  186.976012]  [<ffffffff81056747>] exit_mm+0x13e/0x14b
> [  186.976012]  [<ffffffff81056976>] do_exit+0x222/0x7c6
> [  186.976012]  [<ffffffff8100750f>] ?
xen_restore_fl_direct_end+0x0/0x1
> [  186.976012]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
> [  186.976012]  [<ffffffff814b3949>] ?
lockdep_sys_exit_thunk+0x35/0x67
> [  186.976012]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
> [  186.976012]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
> [  186.976012]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
> [  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89
> df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
> <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
> [  186.976012] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
> [  186.976012]  RSP <ffff8800781bde18>
> [  186.976012] ---[ end trace c0f4eff4054a67e4 ]---
> [  186.976012] Fixing recursive fault but reboot is needed!
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975228] ------------[ cut here ]------------
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975254] invalid opcode: 0000 [#1] SMP
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975269] last sysfs file:
> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Stack:
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Call Trace:
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00
> 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00
> 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54
53 48
>
> --- On *Tue, 11/16/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com>*wrote:
>
>
> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com,
> "Bruce Edge" <bruce.edge@gmail.com>
> Date: Tuesday, November 16, 2010, 4:15 PM
>
> On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets wrote:
> > > Huh. I .. what? I am confused. I thought we established that the
issue
> > > was not related to Xen PCI front? You also seem to uncomment the
> > > upstream.core.patches and the xen.pvhvm.patch - why?
> >
> > I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> > it gives failed HUNKs
>
> Uhh.. I am even more confused.
> >
> > > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
> >
> > I am pretty sure rc2 doesn''t contain everything from
> xen.next-2.6.37.patch,
> > gntdev''s stuff for sure. I''ve built 2.6.37-rc2
kernel rpms and loaded
> > kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1.
> > Device /dev/xen/gntdev has not been created. I understand that
it''s
> > unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU
i
> cannot
> > get 3.2 GB copied over to DomU from NFS share at Dom0.
>
> So what I think you are saying is that you keep on getting the bug in DomU?
> Is the stack-trace the same as in rc1?
>
>
>I haven''t had much time to look into the broken/working version issues
here,
but I did confirm a couple of points:
1) The 2.6.37-rc2 has the same problem still
2) This problem goes away of one is not using NFS.

Not staggeringly helpful I know, but it''s one small data point.

-Bruce


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-18 10:34 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Could you apply two attached patches on top of 2.6.37-rc2 and see
whether it gives some improvement or no ( with active NFS client at DomU)

Boris  

--- On Wed, 11/17/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com
Date: Wednesday, November 17, 2010, 4:28 PM

On Tue, Nov 16, 2010 at 1:49 PM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:

Yes, here we are

[  186.975228] ------------[ cut here ]------------
[  186.975245] kernel BUG at mm/mmap.c:2399!

[  186.975254] invalid opcode: 0000 [#1] SMP 
[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[  186.975284] CPU 0 
[  186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia
cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64
twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd
aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4
xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro
xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel
tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ipv6 uinput
xen_netfront
 microcode xen_blkfront [last unloaded: scsi_wait_scan]
[  186.975507] 
[  186.975515] Pid: 1562, comm: ls Not tainted
2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[  186.975529] RIP: e030:[<ffffffff8110ada1>]  [<ffffffff8110ada1>]
exit_mmap+0x10c/0x119

[  186.975550] RSP: e02b:ffff8800781bde18  EFLAGS: 00010202
[  186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[  186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI: ffffea00000c0280

[  186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09: 0000000000000001
[  186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12: ffff880076c68b00
[  186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15: 0000000000000001

[  186.975625] FS:  00007f8e471d97c0(0000) GS:ffff88007f831000(0000)
knlGS:0000000000000000
[  186.975639] CS:  e033 DS: 0000 ES:
 0000 CR0: 000000008005003b
[  186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4: 0000000000002660
[  186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

[  186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task
ffff8800788223e0)
[  186.976012] Stack:
[  186.976012]  000000000000006b ffff88007f83f1e0 ffff8800781bde38
ffff880076c68b00
[  186.976012]  ffff880076c68c40 ffff8800788229d0 ffff8800781bde68
ffffffff810505fc

[  186.976012]  ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8
ffffffff81056747
[  186.976012] Call Trace:
[  186.976012]  [<ffffffff810505fc>] mmput+0x65/0xd8
[  186.976012]  [<ffffffff81056747>] exit_mm+0x13e/0x14b

[  186.976012]  [<ffffffff81056976>]
 do_exit+0x222/0x7c6
[  186.976012]  [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[  186.976012]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
[  186.976012]  [<ffffffff814b3949>] ? lockdep_sys_exit_thunk+0x35/0x67

[  186.976012]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
[  186.976012]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
[  186.976012]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8
0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f>
0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48

[  186.976012] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[  186.976012]  RSP <ffff8800781bde18>
[  186.976012] ---[ end trace c0f4eff4054a67e4
 ]---
[  186.976012] Fixing recursive fault but reboot is needed!

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975228] ------------[ cut here ]------------

Message from syslogd@fedora14 at Nov 17 00:47:40 ...

 kernel:[  186.975254] invalid opcode: 0000 [#1] SMP 

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map


Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Stack:

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Call Trace:

Message from syslogd@fedora14 at Nov 17 00:47:40 ...

 kernel:[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48
89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
<0f> 0b 48 83 c4
 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48 

--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>

Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>

Date: Tuesday, November 16, 2010, 4:15 PM

On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets
wrote:> > Huh. I .. what? I am confused. I thought we established that the issue
> > was not related to Xen PCI front? You also seem to uncomment the
> > upstream.core.patches and the xen.pvhvm.patch -
 why?> 
> I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> it gives failed HUNKs
Uhh.. I am even more confused.> 
> > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
> 
> I am pretty sure rc2 doesn''t contain everything from
xen.next-2.6.37.patch,
> gntdev''s stuff for sure. I''ve built 2.6.37-rc2 kernel
rpms and loaded
> kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1. 
> Device /dev/xen/gntdev has not been created. I understand that
it''s
> unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i
cannot
> get 3.2 GB copied over to DomU from NFS share at Dom0.
So what I think you are saying is that you keep on getting the bug in DomU?
Is the stack-trace the same as in rc1?




      
I haven''t had much time to look into the broken/working version issues
here, but I did confirm a couple of points:
1) The 2.6.37-rc2 has the same problem still
2) This problem goes away of one is not using NFS.

Not staggeringly helpful I know, but it''s one small data point.

-Bruce




      



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Nov-18 16:40 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Thu, Nov 18, 2010 at 2:34 AM, Boris Derzhavets
<bderzhavets@yahoo.com>wrote:
> Could you apply two attached patches on top of 2.6.37-rc2 and see
> whether it gives some improvement or no ( with active NFS client at DomU)
>
> Boris
>
Hi Boris,

Are you using the mainline kernel or a pvops branch with these patches?
Maybe I''m doing something wrong, but they don''t apply cleanly
with
2.6.37-rc2:

%> patch --dry-run <../patches.2.6.37/xen.next-2.6.37.patch

patching file pgtable.h
Hunk #1 FAILED at 399.
1 out of 1 hunk FAILED -- saving rejects to file pgtable.h.rej
patching file pgtable.c
Hunk #1 FAILED at 15.
1 out of 1 hunk FAILED -- saving rejects to file pgtable.c.rej
patching file ttm_bo_vm.c
Hunk #1 FAILED at 273.
Hunk #2 FAILED at 288.
2 out of 2 hunks FAILED -- saving rejects to file ttm_bo_vm.c.rej
......

 %> patch --dry-run <../patches.2.6.37/xen.pcifront.fixes.patch

patching file enlighten.c
Hunk #1 FAILED at 1090.
Hunk #2 FAILED at 1202.
2 out of 2 hunks FAILED -- saving rejects to file enlighten.c.rej
patching file setup.c
Hunk #1 FAILED at 337.
Hunk #2 FAILED at 356.
2 out of 2 hunks FAILED -- saving rejects to file setup.c.rej

Same result for linux-2.6.37-xen-next branch.

-Bruce

> --- On *Wed, 11/17/10, Bruce Edge <bruce.edge@gmail.com>* wrote:
>
>
> From: Bruce Edge <bruce.edge@gmail.com>
>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy
> Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com
> Date: Wednesday, November 17, 2010, 4:28 PM
>
>
> On Tue, Nov 16, 2010 at 1:49 PM, Boris Derzhavets
<bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> > wrote:
>
> Yes, here we are
>
> [  186.975228] ------------[ cut here ]------------
> [  186.975245] kernel BUG at mm/mmap.c:2399!
> [  186.975254] invalid opcode: 0000 [#1] SMP
> [  186.975269] last sysfs file:
> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
> [  186.975284] CPU 0
> [  186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr
> camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic
> twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic
> des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet
> xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport
> xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp
> ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl
> auth_rpcgss exportfs sunrpc ipv6 uinput xen_netfront microcode xen_blkfront
> [last unloaded: scsi_wait_scan]
> [  186.975507]
> [  186.975515] Pid: 1562, comm: ls Not tainted
> 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
> [  186.975529] RIP: e030:[<ffffffff8110ada1>] 
[<ffffffff8110ada1>]
> exit_mmap+0x10c/0x119
> [  186.975550] RSP: e02b:ffff8800781bde18  EFLAGS: 00010202
> [  186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
> 0000000000000000
> [  186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI:
> ffffea00000c0280
> [  186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09:
> 0000000000000001
> [  186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12:
> ffff880076c68b00
> [  186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15:
> 0000000000000001
> [  186.975625] FS:  00007f8e471d97c0(0000) GS:ffff88007f831000(0000)
> knlGS:0000000000000000
> [  186.975639] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4:
> 0000000000002660
> [  186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [  186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [  186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task
> ffff8800788223e0)
> [  186.976012] Stack:
> [  186.976012]  000000000000006b ffff88007f83f1e0 ffff8800781bde38
> ffff880076c68b00
> [  186.976012]  ffff880076c68c40 ffff8800788229d0 ffff8800781bde68
> ffffffff810505fc
> [  186.976012]  ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8
> ffffffff81056747
> [  186.976012] Call Trace:
> [  186.976012]  [<ffffffff810505fc>] mmput+0x65/0xd8
> [  186.976012]  [<ffffffff81056747>] exit_mm+0x13e/0x14b
> [  186.976012]  [<ffffffff81056976>] do_exit+0x222/0x7c6
> [  186.976012]  [<ffffffff8100750f>] ?
xen_restore_fl_direct_end+0x0/0x1
> [  186.976012]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
> [  186.976012]  [<ffffffff814b3949>] ?
lockdep_sys_exit_thunk+0x35/0x67
> [  186.976012]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
> [  186.976012]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
> [  186.976012]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
> [  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89
> df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
> <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
> [  186.976012] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
> [  186.976012]  RSP <ffff8800781bde18>
> [  186.976012] ---[ end trace c0f4eff4054a67e4 ]---
> [  186.976012] Fixing recursive fault but reboot is needed!
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975228] ------------[ cut here ]------------
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975254] invalid opcode: 0000 [#1] SMP
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975269] last sysfs file:
> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Stack:
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Call Trace:
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00
> 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00
> 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54
53 48
>
> --- On *Tue, 11/16/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com<http://mc/compose?to=konrad.wilk@oracle.com>
> >* wrote:
>
>
> From: Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com<http://mc/compose?to=konrad.wilk@oracle.com>
> >
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets"
<bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> >
> Cc: "Jeremy Fitzhardinge"
<jeremy@goop.org<http://mc/compose?to=jeremy@goop.org>>,
>
xen-devel@lists.xensource.com<http://mc/compose?to=xen-devel@lists.xensource.com>,
> "Bruce Edge"
<bruce.edge@gmail.com<http://mc/compose?to=bruce.edge@gmail.com>
> >
> Date: Tuesday, November 16, 2010, 4:15 PM
>
> On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets wrote:
> > > Huh. I .. what? I am confused. I thought we established that the
issue
> > > was not related to Xen PCI front? You also seem to uncomment the
> > > upstream.core.patches and the xen.pvhvm.patch - why?
> >
> > I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> > it gives failed HUNKs
>
> Uhh.. I am even more confused.
> >
> > > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
> >
> > I am pretty sure rc2 doesn''t contain everything from
> xen.next-2.6.37.patch,
> > gntdev''s stuff for sure. I''ve built 2.6.37-rc2
kernel rpms and loaded
> > kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1.
> > Device /dev/xen/gntdev has not been created. I understand that
it''s
> > unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU
i
> cannot
> > get 3.2 GB copied over to DomU from NFS share at Dom0.
>
> So what I think you are saying is that you keep on getting the bug in DomU?
> Is the stack-trace the same as in rc1?
>
>
>
> I haven''t had much time to look into the broken/working version
issues
> here, but I did confirm a couple of points:
> 1) The 2.6.37-rc2 has the same problem still
> 2) This problem goes away of one is not using NFS.
>
> Not staggeringly helpful I know, but it''s one small data point.
>
> -Bruce
>
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-18 17:05 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Bruce,
You should be able to apply patches to mainline 2.6.37-rc2 cleanly. This patches
are
taken out from MY''s kernel-2.6.37-rc2.git0.fc15.src.rpm.
I already applied them on Ubuntu 10.10 to uncompressed mainline rc2.

I have also to notice, that
# mount IP-Dom0:/home/user1 /mnt/nfs
# cd /mnt/nfs
# ls -l
crashes DomU immediately in text mode.
In graphics mode it doesn''t necessary happen every time.
DomU might survive this "hack" and crashed one
hour latter by another reason.

Boris.

--- On Thu, 11/18/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com
Date: Thursday, November 18, 2010, 11:40 AM



On Thu, Nov 18, 2010 at 2:34 AM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:

Could you apply two attached patches on top of 2.6.37-rc2 and see
whether it gives some improvement or no ( with active NFS client at DomU)

Boris  

Hi Boris,

Are you using the mainline kernel or a pvops branch with these patches?
Maybe I''m doing something wrong, but they don''t apply cleanly
with 2.6.37-rc2:

%> patch --dry-run <../patches.2.6.37/xen.next-2.6.37.patch 

patching file pgtable.h
Hunk #1 FAILED at 399.
1 out of 1 hunk FAILED -- saving rejects to file pgtable.h.rej
patching file pgtable.c

Hunk #1 FAILED at 15.
1 out of 1 hunk FAILED -- saving rejects to file pgtable.c.rej
patching file ttm_bo_vm.c
Hunk #1 FAILED at 273.
Hunk #2 FAILED at 288.
2 out of 2 hunks FAILED -- saving rejects to file ttm_bo_vm.c.rej

......

 %> patch --dry-run <../patches.2.6.37/xen.pcifront.fixes.patch

patching file enlighten.c
Hunk #1 FAILED at 1090.
Hunk #2 FAILED at 1202.
2 out of 2 hunks FAILED -- saving rejects to file enlighten.c.rej

patching file setup.c
Hunk #1 FAILED at 337.
Hunk #2 FAILED at 356.
2 out of 2 hunks FAILED -- saving rejects to file setup.c.rej

Same result for linux-2.6.37-xen-next branch.

-Bruce


--- On Wed, 11/17/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>

Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com

Date: Wednesday, November 17, 2010, 4:28 PM

On Tue, Nov 16, 2010 at 1:49 PM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:


Yes, here we are

[  186.975228] ------------[ cut here ]------------
[  186.975245] kernel BUG at mm/mmap.c:2399!


[  186.975254] invalid opcode: 0000 [#1] SMP 
[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[  186.975284] CPU 0 
[  186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia
cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64
twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd
aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4
xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro
xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel
tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ipv6 uinput
xen_netfront
 microcode xen_blkfront [last unloaded: scsi_wait_scan]
[  186.975507] 
[  186.975515] Pid: 1562, comm: ls Not tainted
2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[  186.975529] RIP: e030:[<ffffffff8110ada1>]  [<ffffffff8110ada1>]
exit_mmap+0x10c/0x119


[  186.975550] RSP: e02b:ffff8800781bde18  EFLAGS: 00010202
[  186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[  186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI: ffffea00000c0280


[  186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09: 0000000000000001
[  186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12: ffff880076c68b00
[  186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15: 0000000000000001


[  186.975625] FS:  00007f8e471d97c0(0000) GS:ffff88007f831000(0000)
knlGS:0000000000000000
[  186.975639] CS:  e033 DS: 0000 ES:
 0000 CR0: 000000008005003b
[  186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4: 0000000000002660
[  186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400


[  186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task
ffff8800788223e0)
[  186.976012] Stack:
[  186.976012]  000000000000006b ffff88007f83f1e0 ffff8800781bde38
ffff880076c68b00
[  186.976012]  ffff880076c68c40 ffff8800788229d0 ffff8800781bde68
ffffffff810505fc


[  186.976012]  ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8
ffffffff81056747
[  186.976012] Call Trace:
[  186.976012]  [<ffffffff810505fc>] mmput+0x65/0xd8
[  186.976012]  [<ffffffff81056747>] exit_mm+0x13e/0x14b


[  186.976012]  [<ffffffff81056976>]
 do_exit+0x222/0x7c6
[  186.976012]  [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[  186.976012]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
[  186.976012]  [<ffffffff814b3949>] ? lockdep_sys_exit_thunk+0x35/0x67


[  186.976012]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
[  186.976012]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
[  186.976012]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8
0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f>
0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48


[  186.976012] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[  186.976012]  RSP <ffff8800781bde18>
[  186.976012] ---[ end trace c0f4eff4054a67e4
 ]---
[  186.976012] Fixing recursive fault but reboot is needed!

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975228] ------------[ cut here ]------------

Message from syslogd@fedora14 at Nov 17 00:47:40 ...


 kernel:[  186.975254] invalid opcode: 0000 [#1] SMP 

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map



Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Stack:

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Call Trace:

Message from syslogd@fedora14 at Nov 17 00:47:40 ...


 kernel:[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48
89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
<0f> 0b 48 83 c4
 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48 

--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:


From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>


Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>


Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>


Date: Tuesday, November 16, 2010, 4:15 PM

On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets
wrote:> > Huh. I .. what? I am confused. I thought we established that the issue
> > was not related to Xen PCI front? You also seem to uncomment the
> > upstream.core.patches and the xen.pvhvm.patch -
 why?> 
> I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> it gives failed HUNKs
Uhh.. I am even more confused.> 
> > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
> 
> I am pretty sure rc2 doesn''t contain everything from
xen.next-2.6.37.patch,
> gntdev''s stuff for sure. I''ve built 2.6.37-rc2 kernel
rpms and loaded
> kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1. 
> Device /dev/xen/gntdev has not been created. I understand that
it''s
> unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i
cannot
> get 3.2 GB copied over to DomU from NFS share at Dom0.

So what I think you are saying is that you keep on getting the bug in DomU?
Is the stack-trace the same as in rc1?




      
I haven''t had much time to look into the broken/working version issues
here, but I did confirm a couple of points:
1) The 2.6.37-rc2 has the same problem still
2) This problem goes away of one is not using NFS.


Not staggeringly helpful I know, but it''s one small data point.

-Bruce



      




      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Nov-18 20:05 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Thu, Nov 18, 2010 at 9:05 AM, Boris Derzhavets
<bderzhavets@yahoo.com>wrote:
> Bruce,
> You should be able to apply patches to mainline 2.6.37-rc2 cleanly. This
> patches are
> taken out from MY''s kernel-2.6.37-rc2.git0.fc15.src.rpm.
> I already applied them on Ubuntu 10.10 to uncompressed mainline rc2.
>
I''m assuming fc=fedora core here?
Doesn''t the fc kernel have a number of other patches already installed?
I''m using a kernel.org kernel, which has none of the fc patches so
it''s not
surprising that they don''t apply.

-Bruce

>
> I have also to notice, that
> # mount IP-Dom0:/home/user1 /mnt/nfs
> # cd /mnt/nfs
> # ls -l
> crashes DomU immediately in text mode.
> In graphics mode it doesn''t necessary happen every time.
> DomU might survive this "hack" and crashed one
> hour latter by another reason.
>
> Boris.
>
>
> --- On *Thu, 11/18/10, Bruce Edge <bruce.edge@gmail.com>* wrote:
>
>
> From: Bruce Edge <bruce.edge@gmail.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy
> Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com
> Date: Thursday, November 18, 2010, 11:40 AM
>
>
>
>
> On Thu, Nov 18, 2010 at 2:34 AM, Boris Derzhavets
<bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> > wrote:
>
> Could you apply two attached patches on top of 2.6.37-rc2 and see
> whether it gives some improvement or no ( with active NFS client at DomU)
>
> Boris
>
>
> Hi Boris,
>
> Are you using the mainline kernel or a pvops branch with these patches?
> Maybe I''m doing something wrong, but they don''t apply
cleanly with
> 2.6.37-rc2:
>
> %> patch --dry-run <../patches.2.6.37/xen.next-2.6.37.patch
>
> patching file pgtable.h
> Hunk #1 FAILED at 399.
> 1 out of 1 hunk FAILED -- saving rejects to file pgtable.h.rej
> patching file pgtable.c
> Hunk #1 FAILED at 15.
> 1 out of 1 hunk FAILED -- saving rejects to file pgtable.c.rej
> patching file ttm_bo_vm.c
> Hunk #1 FAILED at 273.
> Hunk #2 FAILED at 288.
> 2 out of 2 hunks FAILED -- saving rejects to file ttm_bo_vm.c.rej
> ......
>
>  %> patch --dry-run <../patches.2.6.37/xen.pcifront.fixes.patch
>
> patching file enlighten.c
> Hunk #1 FAILED at 1090.
> Hunk #2 FAILED at 1202.
> 2 out of 2 hunks FAILED -- saving rejects to file enlighten.c.rej
> patching file setup.c
> Hunk #1 FAILED at 337.
> Hunk #2 FAILED at 356.
> 2 out of 2 hunks FAILED -- saving rejects to file setup.c.rej
>
> Same result for linux-2.6.37-xen-next branch.
>
> -Bruce
>
>
> --- On *Wed, 11/17/10, Bruce Edge
<bruce.edge@gmail.com<http://mc/compose?to=bruce.edge@gmail.com>
> >* wrote:
>
>
> From: Bruce Edge
<bruce.edge@gmail.com<http://mc/compose?to=bruce.edge@gmail.com>
> >
>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets"
<bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> >
> Cc: "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com<http://mc/compose?to=konrad.wilk@oracle.com>>,
> "Jeremy Fitzhardinge"
<jeremy@goop.org<http://mc/compose?to=jeremy@goop.org>>,
>
xen-devel@lists.xensource.com<http://mc/compose?to=xen-devel@lists.xensource.com>
> Date: Wednesday, November 17, 2010, 4:28 PM
>
>
> On Tue, Nov 16, 2010 at 1:49 PM, Boris Derzhavets
<bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> > wrote:
>
> Yes, here we are
>
> [  186.975228] ------------[ cut here ]------------
> [  186.975245] kernel BUG at mm/mmap.c:2399!
> [  186.975254] invalid opcode: 0000 [#1] SMP
> [  186.975269] last sysfs file:
> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
> [  186.975284] CPU 0
> [  186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr
> camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic
> twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic
> des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet
> xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport
> xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp
> ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl
> auth_rpcgss exportfs sunrpc ipv6 uinput xen_netfront microcode xen_blkfront
> [last unloaded: scsi_wait_scan]
> [  186.975507]
> [  186.975515] Pid: 1562, comm: ls Not tainted
> 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
> [  186.975529] RIP: e030:[<ffffffff8110ada1>] 
[<ffffffff8110ada1>]
> exit_mmap+0x10c/0x119
> [  186.975550] RSP: e02b:ffff8800781bde18  EFLAGS: 00010202
> [  186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
> 0000000000000000
> [  186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI:
> ffffea00000c0280
> [  186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09:
> 0000000000000001
> [  186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12:
> ffff880076c68b00
> [  186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15:
> 0000000000000001
> [  186.975625] FS:  00007f8e471d97c0(0000) GS:ffff88007f831000(0000)
> knlGS:0000000000000000
> [  186.975639] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4:
> 0000000000002660
> [  186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [  186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [  186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task
> ffff8800788223e0)
> [  186.976012] Stack:
> [  186.976012]  000000000000006b ffff88007f83f1e0 ffff8800781bde38
> ffff880076c68b00
> [  186.976012]  ffff880076c68c40 ffff8800788229d0 ffff8800781bde68
> ffffffff810505fc
> [  186.976012]  ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8
> ffffffff81056747
> [  186.976012] Call Trace:
> [  186.976012]  [<ffffffff810505fc>] mmput+0x65/0xd8
> [  186.976012]  [<ffffffff81056747>] exit_mm+0x13e/0x14b
> [  186.976012]  [<ffffffff81056976>] do_exit+0x222/0x7c6
> [  186.976012]  [<ffffffff8100750f>] ?
xen_restore_fl_direct_end+0x0/0x1
> [  186.976012]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
> [  186.976012]  [<ffffffff814b3949>] ?
lockdep_sys_exit_thunk+0x35/0x67
> [  186.976012]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
> [  186.976012]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
> [  186.976012]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
> [  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89
> df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
> <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
> [  186.976012] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
> [  186.976012]  RSP <ffff8800781bde18>
> [  186.976012] ---[ end trace c0f4eff4054a67e4 ]---
> [  186.976012] Fixing recursive fault but reboot is needed!
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975228] ------------[ cut here ]------------
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975254] invalid opcode: 0000 [#1] SMP
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975269] last sysfs file:
> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Stack:
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Call Trace:
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00
> 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00
> 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54
53 48
>
> --- On *Tue, 11/16/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com<http://mc/compose?to=konrad.wilk@oracle.com>
> >* wrote:
>
>
> From: Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com<http://mc/compose?to=konrad.wilk@oracle.com>
> >
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets"
<bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> >
> Cc: "Jeremy Fitzhardinge"
<jeremy@goop.org<http://mc/compose?to=jeremy@goop.org>>,
>
xen-devel@lists.xensource.com<http://mc/compose?to=xen-devel@lists.xensource.com>,
> "Bruce Edge"
<bruce.edge@gmail.com<http://mc/compose?to=bruce.edge@gmail.com>
> >
> Date: Tuesday, November 16, 2010, 4:15 PM
>
> On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets wrote:
> > > Huh. I .. what? I am confused. I thought we established that the
issue
> > > was not related to Xen PCI front? You also seem to uncomment the
> > > upstream.core.patches and the xen.pvhvm.patch - why?
> >
> > I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> > it gives failed HUNKs
>
> Uhh.. I am even more confused.
> >
> > > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
> >
> > I am pretty sure rc2 doesn''t contain everything from
> xen.next-2.6.37.patch,
> > gntdev''s stuff for sure. I''ve built 2.6.37-rc2
kernel rpms and loaded
> > kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1.
> > Device /dev/xen/gntdev has not been created. I understand that
it''s
> > unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU
i
> cannot
> > get 3.2 GB copied over to DomU from NFS share at Dom0.
>
> So what I think you are saying is that you keep on getting the bug in DomU?
> Is the stack-trace the same as in rc1?
>
>
>
> I haven''t had much time to look into the broken/working version
issues
> here, but I did confirm a couple of points:
> 1) The 2.6.37-rc2 has the same problem still
> 2) This problem goes away of one is not using NFS.
>
> Not staggeringly helpful I know, but it''s one small data point.
>
> -Bruce
>
>
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-19 07:12 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

I would guess, applying this patches to mainline 2.6.37-rc2 has nothing to do
with specific fedora patches. It was done on Ubuntu 10.10
1. Kernel source extracted.
2. Patches copied over LAN and applied.

root@boris-System-P5Q3:~# cd Downloads/*-rc2
root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# ls -l ../*.patch
-rw-r--r-- 1 root root 84442 2010-11-17 01:06 ../xen.next-2.6.37.patch
-rw-r--r-- 1 root root  4062 2010-11-17 01:06 ../xen.pcifront.fixes.patch
root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# patch -p1 -s -i
../xen.next-2.6.37.patch
root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# patch -p1 -s -i
../xen.pcifront.fixes.patch
root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# 

Boris.
P.S Directory /root/Downloads/linux-2.6.37-rc2 contains mainline''s
2.6.37-rc2

--- On Thu, 11/18/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Thursday, November 18, 2010, 3:05 PM



On Thu, Nov 18, 2010 at 9:05 AM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:

Bruce,
You should be able to apply patches to mainline 2.6.37-rc2 cleanly. This patches
are
taken out from MY''s kernel-2.6.37-rc2.git0.fc15.src.rpm.

I already applied them on Ubuntu 10.10 to uncompressed mainline rc2.

I''m assuming fc=fedora core here? 
Doesn''t the fc kernel have a number of other patches already installed?

I''m using a kernel.org kernel, which has none of the fc patches so
it''s not surprising that they don''t apply.

-Bruce
 
I have also to notice, that
# mount IP-Dom0:/home/user1 /mnt/nfs
# cd /mnt/nfs
# ls -l
crashes DomU immediately in text mode.

In graphics mode it doesn''t necessary happen every time.
DomU might survive this "hack" and crashed one
hour latter by another reason.

Boris.

--- On Thu, 11/18/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>

Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging
 request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com

Date: Thursday, November 18, 2010, 11:40 AM



On Thu, Nov 18, 2010 at 2:34 AM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:


Could you apply two attached patches on top of 2.6.37-rc2 and see
whether it gives some improvement or no ( with active NFS client at DomU)


Boris  

Hi Boris,

Are you using the mainline kernel or a pvops branch with these patches?
Maybe I''m doing something wrong, but they don''t apply cleanly
with 2.6.37-rc2:


%> patch --dry-run <../patches.2.6.37/xen.next-2.6.37.patch 

patching file pgtable.h
Hunk #1 FAILED at 399.
1 out of 1 hunk FAILED -- saving rejects to file pgtable.h.rej
patching file pgtable.c


Hunk #1 FAILED at 15.
1 out of 1 hunk FAILED -- saving rejects to file pgtable.c.rej
patching file ttm_bo_vm.c
Hunk #1 FAILED at 273.
Hunk #2 FAILED at 288.
2 out of 2 hunks FAILED -- saving rejects to file ttm_bo_vm.c.rej


......

 %> patch --dry-run <../patches.2.6.37/xen.pcifront.fixes.patch

patching file enlighten.c
Hunk #1 FAILED at 1090.
Hunk #2 FAILED at 1202.
2 out of 2 hunks FAILED -- saving rejects to file enlighten.c.rej


patching file setup.c
Hunk #1 FAILED at 337.
Hunk #2 FAILED at 356.
2 out of 2 hunks FAILED -- saving rejects to file setup.c.rej

Same result for linux-2.6.37-xen-next branch.

-Bruce



--- On Wed, 11/17/10, Bruce Edge <bruce.edge@gmail.com> wrote:


From: Bruce Edge <bruce.edge@gmail.com>

Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>


Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com


Date: Wednesday, November 17, 2010, 4:28 PM

On Tue, Nov 16, 2010 at 1:49 PM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:



Yes, here we are

[  186.975228] ------------[ cut here ]------------
[  186.975245] kernel BUG at mm/mmap.c:2399!



[  186.975254] invalid opcode: 0000 [#1] SMP 
[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[  186.975284] CPU 0 
[  186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia
cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64
twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd
aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4
xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro
xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel
tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ipv6 uinput
xen_netfront
 microcode xen_blkfront [last unloaded: scsi_wait_scan]
[  186.975507] 
[  186.975515] Pid: 1562, comm: ls Not tainted
2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[  186.975529] RIP: e030:[<ffffffff8110ada1>]  [<ffffffff8110ada1>]
exit_mmap+0x10c/0x119



[  186.975550] RSP: e02b:ffff8800781bde18  EFLAGS: 00010202
[  186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[  186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI: ffffea00000c0280



[  186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09: 0000000000000001
[  186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12: ffff880076c68b00
[  186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15: 0000000000000001



[  186.975625] FS:  00007f8e471d97c0(0000) GS:ffff88007f831000(0000)
knlGS:0000000000000000
[  186.975639] CS:  e033 DS: 0000 ES:
 0000 CR0: 000000008005003b
[  186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4: 0000000000002660
[  186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400



[  186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task
ffff8800788223e0)
[  186.976012] Stack:
[  186.976012]  000000000000006b ffff88007f83f1e0 ffff8800781bde38
ffff880076c68b00
[  186.976012]  ffff880076c68c40 ffff8800788229d0 ffff8800781bde68
ffffffff810505fc



[  186.976012]  ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8
ffffffff81056747
[  186.976012] Call Trace:
[  186.976012]  [<ffffffff810505fc>] mmput+0x65/0xd8
[  186.976012]  [<ffffffff81056747>] exit_mm+0x13e/0x14b



[  186.976012]  [<ffffffff81056976>]
 do_exit+0x222/0x7c6
[  186.976012]  [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[  186.976012]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
[  186.976012]  [<ffffffff814b3949>] ? lockdep_sys_exit_thunk+0x35/0x67



[  186.976012]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
[  186.976012]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
[  186.976012]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8
0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f>
0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48



[  186.976012] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[  186.976012]  RSP <ffff8800781bde18>
[  186.976012] ---[ end trace c0f4eff4054a67e4
 ]---
[  186.976012] Fixing recursive fault but reboot is needed!

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975228] ------------[ cut here ]------------

Message from syslogd@fedora14 at Nov 17 00:47:40 ...



 kernel:[  186.975254] invalid opcode: 0000 [#1] SMP 

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map




Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Stack:

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Call Trace:

Message from syslogd@fedora14 at Nov 17 00:47:40 ...



 kernel:[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48
89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
<0f> 0b 48 83 c4
 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48 

--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:



From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>



Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>



Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>



Date: Tuesday, November 16, 2010, 4:15 PM

On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets
wrote:> > Huh. I .. what? I am confused. I thought we established that the issue

> > was not related to Xen PCI front? You also seem to uncomment the
> > upstream.core.patches and the xen.pvhvm.patch -
 why?> 
> I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> it gives failed HUNKs
Uhh.. I am even more confused.> 
> > Ok, they are.. v2.6.37-rc2 which came out today has the fixes

> 
> I am pretty sure rc2 doesn''t contain everything from
xen.next-2.6.37.patch,
> gntdev''s stuff for sure. I''ve built 2.6.37-rc2 kernel
rpms and loaded
> kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1. 

> Device /dev/xen/gntdev has not been created. I understand that
it''s
> unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i
cannot
> get 3.2 GB copied over to DomU from NFS share at Dom0.


So what I think you are saying is that you keep on getting the bug in DomU?
Is the stack-trace the same as in rc1?




      
I haven''t had much time to look into the broken/working version issues
here, but I did confirm a couple of points:
1) The 2.6.37-rc2 has the same problem still
2) This problem goes away of one is not using NFS.



Not staggeringly helpful I know, but it''s one small data point.

-Bruce



      



      
_______________________________________________

Xen-devel mailing list

Xen-devel@lists.xensource.com

http://lists.xensource.com/xen-devel



-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-19 14:32 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

I''ve also noticed , that if i change file say under /mnt/nfs/fedora

1. ls runs fine
2. `ls -l` - generates page fault

But doesn''t crash DomU, regardless of stack trace printed in particular
terminal session.

I can close crashed terminal and open second one.
# cd /mnt/nfs/fedora
# ls -l
will succeed until i make some new changes to file descriptors,eg. edit some
file.
Then crashes second terminal session and third should be opened to be able
work with file descriptors ( `ls -l`).
When number of page faults reaches some critical value ( more then 5 , but in
general unpredictable) DomU crashes. So , i cannot any more open new terminal
session. This  is stable and obvious regression in 2.6.37-rc2 vs 2.6.36 as PV
DomU
kernel.

Boris

--- On Thu, 11/18/10, Boris Derzhavets <bderzhavets@yahoo.com> wrote:

From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Bruce Edge" <bruce.edge@gmail.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Thursday, November 18, 2010, 12:05 PM

Bruce,
You should be able to apply patches to mainline 2.6.37-rc2 cleanly. This patches
are
taken out from MY''s kernel-2.6.37-rc2.git0.fc15.src.rpm.
I already applied them on Ubuntu 10.10 to uncompressed mainline rc2.

I have also to notice, that
# mount IP-Dom0:/home/user1 /mnt/nfs
# cd /mnt/nfs
# ls -l
crashes DomU immediately in text mode.
In graphics mode it doesn''t necessary happen every time.
DomU might survive this "hack" and crashed one
hour latter by another reason.

Boris.

--- On Thu, 11/18/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging
 request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com
Date: Thursday, November 18, 2010, 11:40 AM



On Thu, Nov 18, 2010 at 2:34 AM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:

Could you apply two attached patches on top of 2.6.37-rc2 and see
whether it gives some improvement or no ( with active NFS client at DomU)

Boris  

Hi Boris,

Are you using the mainline kernel or a pvops branch with these patches?
Maybe I''m doing something wrong, but they don''t apply cleanly
with 2.6.37-rc2:

%> patch --dry-run <../patches.2.6.37/xen.next-2.6.37.patch 

patching file pgtable.h
Hunk #1 FAILED at 399.
1 out of 1 hunk FAILED -- saving rejects to file pgtable.h.rej
patching file pgtable.c

Hunk #1 FAILED at 15.
1 out of 1 hunk FAILED -- saving rejects to file pgtable.c.rej
patching file ttm_bo_vm.c
Hunk #1 FAILED at 273.
Hunk #2 FAILED at 288.
2 out of 2 hunks FAILED -- saving rejects to file ttm_bo_vm.c.rej

......

 %> patch --dry-run <../patches.2.6.37/xen.pcifront.fixes.patch

patching file enlighten.c
Hunk #1 FAILED at 1090.
Hunk #2 FAILED at 1202.
2 out of 2 hunks FAILED -- saving rejects to file enlighten.c.rej

patching file setup.c
Hunk #1 FAILED at 337.
Hunk #2 FAILED at 356.
2 out of 2 hunks FAILED -- saving rejects to file setup.c.rej

Same result for linux-2.6.37-xen-next branch.

-Bruce


--- On Wed, 11/17/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>

Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com

Date: Wednesday, November 17, 2010, 4:28 PM

On Tue, Nov 16, 2010 at 1:49 PM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:


Yes, here we are

[  186.975228] ------------[ cut here ]------------
[  186.975245] kernel BUG at mm/mmap.c:2399!


[  186.975254] invalid opcode: 0000 [#1] SMP 
[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[  186.975284] CPU 0 
[  186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia
cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64
twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd
aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4
xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro
xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel
tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ipv6 uinput
xen_netfront
 microcode xen_blkfront [last unloaded: scsi_wait_scan]
[  186.975507] 
[  186.975515] Pid: 1562, comm: ls Not tainted
2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[  186.975529] RIP: e030:[<ffffffff8110ada1>]  [<ffffffff8110ada1>]
exit_mmap+0x10c/0x119


[  186.975550] RSP: e02b:ffff8800781bde18  EFLAGS: 00010202
[  186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[  186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI: ffffea00000c0280


[  186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09: 0000000000000001
[  186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12: ffff880076c68b00
[  186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15: 0000000000000001


[  186.975625] FS:  00007f8e471d97c0(0000) GS:ffff88007f831000(0000)
knlGS:0000000000000000
[  186.975639] CS:  e033 DS: 0000 ES:
 0000 CR0: 000000008005003b
[  186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4: 0000000000002660
[  186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400


[  186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task
ffff8800788223e0)
[  186.976012] Stack:
[  186.976012]  000000000000006b ffff88007f83f1e0 ffff8800781bde38
ffff880076c68b00
[  186.976012]  ffff880076c68c40 ffff8800788229d0 ffff8800781bde68
ffffffff810505fc


[  186.976012]  ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8
ffffffff81056747
[  186.976012] Call Trace:
[  186.976012]  [<ffffffff810505fc>] mmput+0x65/0xd8
[  186.976012]  [<ffffffff81056747>] exit_mm+0x13e/0x14b


[  186.976012]  [<ffffffff81056976>]
 do_exit+0x222/0x7c6
[  186.976012]  [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[  186.976012]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
[  186.976012]  [<ffffffff814b3949>] ? lockdep_sys_exit_thunk+0x35/0x67


[  186.976012]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
[  186.976012]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
[  186.976012]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8
0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f>
0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48


[  186.976012] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[  186.976012]  RSP <ffff8800781bde18>
[  186.976012] ---[ end trace c0f4eff4054a67e4
 ]---
[  186.976012] Fixing recursive fault but reboot is needed!

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975228] ------------[ cut here ]------------

Message from syslogd@fedora14 at Nov 17 00:47:40 ...


 kernel:[  186.975254] invalid opcode: 0000 [#1] SMP 

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map



Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Stack:

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Call Trace:

Message from syslogd@fedora14 at Nov 17 00:47:40 ...


 kernel:[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48
89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
<0f> 0b 48 83 c4
 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48 

--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:


From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>


Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>


Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>


Date: Tuesday, November 16, 2010, 4:15 PM

On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets
wrote:> > Huh. I .. what? I am confused. I thought we established that the issue
> > was not related to Xen PCI front? You also seem to uncomment the
> > upstream.core.patches and the xen.pvhvm.patch -
 why?> 
> I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> it gives failed HUNKs
Uhh.. I am even more confused.> 
> > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
> 
> I am pretty sure rc2 doesn''t contain everything from
xen.next-2.6.37.patch,
> gntdev''s stuff for sure. I''ve built 2.6.37-rc2 kernel
rpms and loaded
> kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1. 
> Device /dev/xen/gntdev has not been created. I understand that
it''s
> unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i
cannot
> get 3.2 GB copied over to DomU from NFS share at Dom0.

So what I think you are saying is that you keep on getting the bug in DomU?
Is the stack-trace the same as in rc1?




      
I haven''t had much time to look into the broken/working version issues
here, but I did confirm a couple of points:
1) The 2.6.37-rc2 has the same problem still
2) This problem goes away of one is not using NFS.


Not staggeringly helpful I know, but it''s one small data point.

-Bruce



      



      
-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Nov-19 18:16 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Thu, Nov 18, 2010 at 11:12 PM, Boris Derzhavets
<bderzhavets@yahoo.com>wrote:
> I would guess, applying this patches to mainline 2.6.37-rc2 has nothing to
> do with specific fedora patches. It was done on Ubuntu 10.10
> 1. Kernel source extracted.
> 2. Patches copied over LAN and applied.
>
> root@boris-System-P5Q3:~# cd Downloads/*-rc2
> root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# ls -l ../*.patch
> -rw-r--r-- 1 root root 84442 2010-11-17 01:06 ../xen.next-2.6.37.patch
> -rw-r--r-- 1 root root  4062 2010-11-17 01:06 ../xen.pcifront.fixes.patch
> root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# patch -p1 -s -i
> ../xen.next-2.6.37.patch
> root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# patch -p1 -s -i
> ../xen.pcifront.fixes.patch
> root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2#
>
> Boris.
> P.S Directory /root/Downloads/linux-2.6.37-rc2 contains mainline''s
> 2.6.37-rc2

I feel like I must be doing something unbelievably stupid:

%> rm -rf linux-2.6.37
%> tar lxf linux-2.6.37-rc2.tar.bz2 (This is
http://www.kernel.org/pub/linux/kernel/v2.6/testing/linux-2.6.37-rc2.tar.bz2
)
%> cd linux-2.6.37-rc2
%> patch --dry-run -p1 -s -i  ../patches.2.6.37/xen.next-2.6.37.patch

1 out of 2 hunks FAILED -- saving rejects to file
drivers/block/xen-blkfront.c.rej
1 out of 1 hunk FAILED -- saving rejects to file
arch/x86/include/asm/microcode.h.rej
1 out of 1 hunk FAILED -- saving rejects to file
arch/x86/kernel/Makefile.rej
1 out of 1 hunk FAILED -- saving rejects to file arch/x86/xen/Kconfig.rej
1 out of 2 hunks FAILED -- saving rejects to file
arch/x86/include/asm/processor.h.rej
1 out of 1 hunk FAILED -- saving rejects to file drivers/xen/gntdev.c.rej

-Bruce

>
>
> --- On *Thu, 11/18/10, Bruce Edge <bruce.edge@gmail.com>* wrote:
>
>
> From: Bruce Edge <bruce.edge@gmail.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com,
> "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
> Date: Thursday, November 18, 2010, 3:05 PM
>
>
>
>
> On Thu, Nov 18, 2010 at 9:05 AM, Boris Derzhavets
<bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> > wrote:
>
> Bruce,
> You should be able to apply patches to mainline 2.6.37-rc2 cleanly. This
> patches are
> taken out from MY''s kernel-2.6.37-rc2.git0.fc15.src.rpm.
> I already applied them on Ubuntu 10.10 to uncompressed mainline rc2.
>
>
> I''m assuming fc=fedora core here?
> Doesn''t the fc kernel have a number of other patches already
installed?
> I''m using a kernel.org kernel, which has none of the fc patches so
it''s
> not surprising that they don''t apply.
>
> -Bruce
>
>
>
> I have also to notice, that
> # mount IP-Dom0:/home/user1 /mnt/nfs
> # cd /mnt/nfs
> # ls -l
> crashes DomU immediately in text mode.
> In graphics mode it doesn''t necessary happen every time.
> DomU might survive this "hack" and crashed one
> hour latter by another reason.
>
> Boris.
>
>
> --- On *Thu, 11/18/10, Bruce Edge
<bruce.edge@gmail.com<http://mc/compose?to=bruce.edge@gmail.com>
> >* wrote:
>
>
> From: Bruce Edge
<bruce.edge@gmail.com<http://mc/compose?to=bruce.edge@gmail.com>
> >
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets"
<bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> >
> Cc: "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com<http://mc/compose?to=konrad.wilk@oracle.com>>,
> "Jeremy Fitzhardinge"
<jeremy@goop.org<http://mc/compose?to=jeremy@goop.org>>,
>
xen-devel@lists.xensource.com<http://mc/compose?to=xen-devel@lists.xensource.com>
> Date: Thursday, November 18, 2010, 11:40 AM
>
>
>
>
> On Thu, Nov 18, 2010 at 2:34 AM, Boris Derzhavets
<bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> > wrote:
>
> Could you apply two attached patches on top of 2.6.37-rc2 and see
> whether it gives some improvement or no ( with active NFS client at DomU)
>
> Boris
>
>
> Hi Boris,
>
> Are you using the mainline kernel or a pvops branch with these patches?
> Maybe I''m doing something wrong, but they don''t apply
cleanly with
> 2.6.37-rc2:
>
> %> patch --dry-run <../patches.2.6.37/xen.next-2.6.37.patch
>
> patching file pgtable.h
> Hunk #1 FAILED at 399.
> 1 out of 1 hunk FAILED -- saving rejects to file pgtable.h.rej
> patching file pgtable.c
> Hunk #1 FAILED at 15.
> 1 out of 1 hunk FAILED -- saving rejects to file pgtable.c.rej
> patching file ttm_bo_vm.c
> Hunk #1 FAILED at 273.
> Hunk #2 FAILED at 288.
> 2 out of 2 hunks FAILED -- saving rejects to file ttm_bo_vm.c.rej
> ......
>
>  %> patch --dry-run <../patches.2.6.37/xen.pcifront.fixes.patch
>
> patching file enlighten.c
> Hunk #1 FAILED at 1090.
> Hunk #2 FAILED at 1202.
> 2 out of 2 hunks FAILED -- saving rejects to file enlighten.c.rej
> patching file setup.c
> Hunk #1 FAILED at 337.
> Hunk #2 FAILED at 356.
> 2 out of 2 hunks FAILED -- saving rejects to file setup.c.rej
>
> Same result for linux-2.6.37-xen-next branch.
>
> -Bruce
>
>
> --- On *Wed, 11/17/10, Bruce Edge
<bruce.edge@gmail.com<http://mc/compose?to=bruce.edge@gmail.com>
> >* wrote:
>
>
> From: Bruce Edge
<bruce.edge@gmail.com<http://mc/compose?to=bruce.edge@gmail.com>
> >
>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets"
<bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> >
> Cc: "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com<http://mc/compose?to=konrad.wilk@oracle.com>>,
> "Jeremy Fitzhardinge"
<jeremy@goop.org<http://mc/compose?to=jeremy@goop.org>>,
>
xen-devel@lists.xensource.com<http://mc/compose?to=xen-devel@lists.xensource.com>
> Date: Wednesday, November 17, 2010, 4:28 PM
>
>
> On Tue, Nov 16, 2010 at 1:49 PM, Boris Derzhavets
<bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> > wrote:
>
> Yes, here we are
>
> [  186.975228] ------------[ cut here ]------------
> [  186.975245] kernel BUG at mm/mmap.c:2399!
> [  186.975254] invalid opcode: 0000 [#1] SMP
> [  186.975269] last sysfs file:
> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
> [  186.975284] CPU 0
> [  186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr
> camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic
> twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic
> des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet
> xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport
> xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp
> ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl
> auth_rpcgss exportfs sunrpc ipv6 uinput xen_netfront microcode xen_blkfront
> [last unloaded: scsi_wait_scan]
> [  186.975507]
> [  186.975515] Pid: 1562, comm: ls Not tainted
> 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
> [  186.975529] RIP: e030:[<ffffffff8110ada1>] 
[<ffffffff8110ada1>]
> exit_mmap+0x10c/0x119
> [  186.975550] RSP: e02b:ffff8800781bde18  EFLAGS: 00010202
> [  186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
> 0000000000000000
> [  186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI:
> ffffea00000c0280
> [  186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09:
> 0000000000000001
> [  186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12:
> ffff880076c68b00
> [  186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15:
> 0000000000000001
> [  186.975625] FS:  00007f8e471d97c0(0000) GS:ffff88007f831000(0000)
> knlGS:0000000000000000
> [  186.975639] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4:
> 0000000000002660
> [  186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [  186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [  186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task
> ffff8800788223e0)
> [  186.976012] Stack:
> [  186.976012]  000000000000006b ffff88007f83f1e0 ffff8800781bde38
> ffff880076c68b00
> [  186.976012]  ffff880076c68c40 ffff8800788229d0 ffff8800781bde68
> ffffffff810505fc
> [  186.976012]  ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8
> ffffffff81056747
> [  186.976012] Call Trace:
> [  186.976012]  [<ffffffff810505fc>] mmput+0x65/0xd8
> [  186.976012]  [<ffffffff81056747>] exit_mm+0x13e/0x14b
> [  186.976012]  [<ffffffff81056976>] do_exit+0x222/0x7c6
> [  186.976012]  [<ffffffff8100750f>] ?
xen_restore_fl_direct_end+0x0/0x1
> [  186.976012]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
> [  186.976012]  [<ffffffff814b3949>] ?
lockdep_sys_exit_thunk+0x35/0x67
> [  186.976012]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
> [  186.976012]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
> [  186.976012]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
> [  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89
> df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
> <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
> [  186.976012] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
> [  186.976012]  RSP <ffff8800781bde18>
> [  186.976012] ---[ end trace c0f4eff4054a67e4 ]---
> [  186.976012] Fixing recursive fault but reboot is needed!
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975228] ------------[ cut here ]------------
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975254] invalid opcode: 0000 [#1] SMP
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975269] last sysfs file:
> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Stack:
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Call Trace:
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00
> 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00
> 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54
53 48
>
> --- On *Tue, 11/16/10, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com<http://mc/compose?to=konrad.wilk@oracle.com>
> >* wrote:
>
>
> From: Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com<http://mc/compose?to=konrad.wilk@oracle.com>
> >
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets"
<bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> >
> Cc: "Jeremy Fitzhardinge"
<jeremy@goop.org<http://mc/compose?to=jeremy@goop.org>>,
>
xen-devel@lists.xensource.com<http://mc/compose?to=xen-devel@lists.xensource.com>,
> "Bruce Edge"
<bruce.edge@gmail.com<http://mc/compose?to=bruce.edge@gmail.com>
> >
> Date: Tuesday, November 16, 2010, 4:15 PM
>
> On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets wrote:
> > > Huh. I .. what? I am confused. I thought we established that the
issue
> > > was not related to Xen PCI front? You also seem to uncomment the
> > > upstream.core.patches and the xen.pvhvm.patch - why?
> >
> > I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> > it gives failed HUNKs
>
> Uhh.. I am even more confused.
> >
> > > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
> >
> > I am pretty sure rc2 doesn''t contain everything from
> xen.next-2.6.37.patch,
> > gntdev''s stuff for sure. I''ve built 2.6.37-rc2
kernel rpms and loaded
> > kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1.
> > Device /dev/xen/gntdev has not been created. I understand that
it''s
> > unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU
i
> cannot
> > get 3.2 GB copied over to DomU from NFS share at Dom0.
>
> So what I think you are saying is that you keep on getting the bug in DomU?
> Is the stack-trace the same as in rc1?
>
>
>
> I haven''t had much time to look into the broken/working version
issues
> here, but I did confirm a couple of points:
> 1) The 2.6.37-rc2 has the same problem still
> 2) This problem goes away of one is not using NFS.
>
> Not staggeringly helpful I know, but it''s one small data point.
>
> -Bruce
>
>
>
>
>
> _______________________________________________
> Xen-devel mailing list
>
Xen-devel@lists.xensource.com<http://mc/compose?to=Xen-devel@lists.xensource.com>
> http://lists.xensource.com/xen-devel
>
>
>
> -----Inline Attachment Follows-----
>
>
> _______________________________________________
> Xen-devel mailing list
>
Xen-devel@lists.xensource.com<http://mc/compose?to=Xen-devel@lists.xensource.com>
> http://lists.xensource.com/xen-devel
>
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Nov-19 18:52 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Miracles happen ;)
My link is also :-

http://www.kernel.org/pub/linux/kernel/v2.6/testing/linux-2.6.37-rc2.tar.bz2

It''s not important . Maybe some kind of backport. I downloaded
linux-2.6.37-rc2.tar.bz2
several days ago. Some changes are already there. Right ? That''s a
reason of HUNK''s failure. Issue is still in place for the most recent
checkout xen/next-2.6.37.

Boris.

--- On Fri, 11/19/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Friday, November 19, 2010, 1:16 PM

On Thu, Nov 18, 2010 at 11:12 PM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:

I would guess, applying this patches to mainline 2.6.37-rc2 has nothing to do
with specific fedora patches. It was done on Ubuntu 10.10

1. Kernel source extracted.
2. Patches copied over LAN and applied.

root@boris-System-P5Q3:~# cd Downloads/*-rc2
root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# ls -l ../*.patch
-rw-r--r-- 1 root root 84442 2010-11-17 01:06 ../xen.next-2.6.37.patch

-rw-r--r-- 1 root root  4062 2010-11-17 01:06 ../xen.pcifront.fixes.patch
root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# patch -p1 -s -i
../xen.next-2.6.37.patch
root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# patch -p1 -s -i
../xen.pcifront.fixes.patch

root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# 

Boris.
P.S Directory /root/Downloads/linux-2.6.37-rc2 contains mainline''s
2.6.37-rc2
I feel like I must be doing something unbelievably stupid:

%> rm -rf linux-2.6.37
%> tar lxf linux-2.6.37-rc2.tar.bz2 (This is
http://www.kernel.org/pub/linux/kernel/v2.6/testing/linux-2.6.37-rc2.tar.bz2)

%> cd linux-2.6.37-rc2
%> patch --dry-run -p1 -s -i  ../patches.2.6.37/xen.next-2.6.37.patch

1 out of 2 hunks FAILED -- saving rejects to file
drivers/block/xen-blkfront.c.rej
1 out of 1 hunk FAILED -- saving rejects to file
arch/x86/include/asm/microcode.h.rej

1 out of 1 hunk FAILED -- saving rejects to file arch/x86/kernel/Makefile.rej
1 out of 1 hunk FAILED -- saving rejects to file arch/x86/xen/Kconfig.rej
1 out of 2 hunks FAILED -- saving rejects to file
arch/x86/include/asm/processor.h.rej

1 out of 1 hunk FAILED -- saving rejects to file drivers/xen/gntdev.c.rej

-Bruce
 

--- On Thu, 11/18/10,
 Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request

To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>

Date: Thursday, November 18, 2010, 3:05 PM



On Thu, Nov 18, 2010 at 9:05 AM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:


Bruce,
You should be able to apply patches to mainline 2.6.37-rc2 cleanly. This patches
are
taken out from MY''s kernel-2.6.37-rc2.git0.fc15.src.rpm.


I already applied them on Ubuntu 10.10 to uncompressed mainline rc2.

I''m assuming fc=fedora core here? 
Doesn''t the fc kernel have a number of other patches already installed?


I''m using a kernel.org kernel, which has none of the fc patches so
it''s not surprising that they don''t apply.

-Bruce
 

I have also to notice, that
# mount IP-Dom0:/home/user1 /mnt/nfs
# cd /mnt/nfs
# ls -l
crashes DomU immediately in text mode.


In graphics mode it doesn''t necessary happen every time.
DomU might survive this "hack" and crashed one
hour latter by another reason.

Boris.

--- On Thu, 11/18/10, Bruce Edge <bruce.edge@gmail.com> wrote:


From: Bruce Edge <bruce.edge@gmail.com>


Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging
 request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com


Date: Thursday, November 18, 2010, 11:40 AM



On Thu, Nov 18, 2010 at 2:34 AM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:



Could you apply two attached patches on top of 2.6.37-rc2 and see
whether it gives some improvement or no ( with active NFS client at DomU)



Boris  

Hi Boris,

Are you using the mainline kernel or a pvops branch with these patches?
Maybe I''m doing something wrong, but they don''t apply cleanly
with 2.6.37-rc2:



%> patch --dry-run <../patches.2.6.37/xen.next-2.6.37.patch 

patching file pgtable.h
Hunk #1 FAILED at 399.
1 out of 1 hunk FAILED -- saving rejects to file pgtable.h.rej
patching file pgtable.c



Hunk #1 FAILED at 15.
1 out of 1 hunk FAILED -- saving rejects to file pgtable.c.rej
patching file ttm_bo_vm.c
Hunk #1 FAILED at 273.
Hunk #2 FAILED at 288.
2 out of 2 hunks FAILED -- saving rejects to file ttm_bo_vm.c.rej



......

 %> patch --dry-run <../patches.2.6.37/xen.pcifront.fixes.patch

patching file enlighten.c
Hunk #1 FAILED at 1090.
Hunk #2 FAILED at 1202.
2 out of 2 hunks FAILED -- saving rejects to file enlighten.c.rej



patching file setup.c
Hunk #1 FAILED at 337.
Hunk #2 FAILED at 356.
2 out of 2 hunks FAILED -- saving rejects to file setup.c.rej

Same result for linux-2.6.37-xen-next branch.

-Bruce




--- On Wed, 11/17/10, Bruce Edge <bruce.edge@gmail.com> wrote:



From: Bruce Edge <bruce.edge@gmail.com>


Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>



Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com



Date: Wednesday, November 17, 2010, 4:28 PM

On Tue, Nov 16, 2010 at 1:49 PM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:




Yes, here we are

[  186.975228] ------------[ cut here ]------------
[  186.975245] kernel BUG at mm/mmap.c:2399!




[  186.975254] invalid opcode: 0000 [#1] SMP 
[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[  186.975284] CPU 0 
[  186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia
cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64
twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd
aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4
xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro
xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel
tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ipv6 uinput
xen_netfront
 microcode xen_blkfront [last unloaded: scsi_wait_scan]
[  186.975507] 
[  186.975515] Pid: 1562, comm: ls Not tainted
2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[  186.975529] RIP: e030:[<ffffffff8110ada1>]  [<ffffffff8110ada1>]
exit_mmap+0x10c/0x119




[  186.975550] RSP: e02b:ffff8800781bde18  EFLAGS: 00010202
[  186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[  186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI: ffffea00000c0280




[  186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09: 0000000000000001
[  186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12: ffff880076c68b00
[  186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15: 0000000000000001




[  186.975625] FS:  00007f8e471d97c0(0000) GS:ffff88007f831000(0000)
knlGS:0000000000000000
[  186.975639] CS:  e033 DS: 0000 ES:
 0000 CR0: 000000008005003b
[  186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4: 0000000000002660
[  186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400




[  186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task
ffff8800788223e0)
[  186.976012] Stack:
[  186.976012]  000000000000006b ffff88007f83f1e0 ffff8800781bde38
ffff880076c68b00
[  186.976012]  ffff880076c68c40 ffff8800788229d0 ffff8800781bde68
ffffffff810505fc




[  186.976012]  ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8
ffffffff81056747
[  186.976012] Call Trace:
[  186.976012]  [<ffffffff810505fc>] mmput+0x65/0xd8
[  186.976012]  [<ffffffff81056747>] exit_mm+0x13e/0x14b




[  186.976012]  [<ffffffff81056976>]
 do_exit+0x222/0x7c6
[  186.976012]  [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[  186.976012]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
[  186.976012]  [<ffffffff814b3949>] ? lockdep_sys_exit_thunk+0x35/0x67




[  186.976012]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
[  186.976012]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
[  186.976012]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8
0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f>
0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48




[  186.976012] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[  186.976012]  RSP <ffff8800781bde18>
[  186.976012] ---[ end trace c0f4eff4054a67e4
 ]---
[  186.976012] Fixing recursive fault but reboot is needed!

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975228] ------------[ cut here ]------------

Message from syslogd@fedora14 at Nov 17 00:47:40 ...




 kernel:[  186.975254] invalid opcode: 0000 [#1] SMP 

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map





Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Stack:

Message from syslogd@fedora14 at Nov 17 00:47:40 ...
 kernel:[  186.976012] Call Trace:

Message from syslogd@fedora14 at Nov 17 00:47:40 ...




 kernel:[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48
89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
<0f> 0b 48 83 c4
 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48 

--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:




From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>




Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>




Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>




Date: Tuesday, November 16, 2010, 4:15 PM

On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets
wrote:> > Huh. I .. what? I am confused. I thought we established that the issue


> > was not related to Xen PCI front? You also seem to uncomment the
> > upstream.core.patches and the xen.pvhvm.patch -
 why?> 
> I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> it gives failed HUNKs
Uhh.. I am even more confused.> 
> > Ok, they are.. v2.6.37-rc2 which came out today has the fixes


> 
> I am pretty sure rc2 doesn''t contain everything from
xen.next-2.6.37.patch,
> gntdev''s stuff for sure. I''ve built 2.6.37-rc2 kernel
rpms and loaded
> kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1. 


> Device /dev/xen/gntdev has not been created. I understand that
it''s
> unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i
cannot
> get 3.2 GB copied over to DomU from NFS share at Dom0.



So what I think you are saying is that you keep on getting the bug in DomU?
Is the stack-trace the same as in rc1?




      
I haven''t had much time to look into the broken/working version issues
here, but I did confirm a couple of points:
1) The 2.6.37-rc2 has the same problem still
2) This problem goes away of one is not using NFS.




Not staggeringly helpful I know, but it''s one small data point.

-Bruce



      



      
_______________________________________________

Xen-devel mailing list

Xen-devel@lists.xensource.com

http://lists.xensource.com/xen-devel



-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com

http://lists.xensource.com/xen-devel



      




      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Dec-01 21:32 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Fri, Nov 19, 2010 at 11:17 AM, Boris Derzhavets
<bderzhavets@yahoo.com> wrote:>
> You uncompress :-
>   tar lxf linux-2.6.37-rc2.tar.bz2
> Mine :-
>  tar jxvf linux-2.6.37-rc2.tar.bz2
>
> Boris
>
> --- On Fri, 11/19/10, Bruce Edge <bruce.edge@gmail.com> wrote:
>
> From: Bruce Edge <bruce.edge@gmail.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
> Date: Friday, November 19, 2010, 1:16 PM
>
> On Thu, Nov 18, 2010 at 11:12 PM, Boris Derzhavets
<bderzhavets@yahoo.com> wrote:
>
> I would guess, applying this patches to mainline 2.6.37-rc2 has nothing to
do with specific fedora patches. It was done on Ubuntu 10.10
> 1. Kernel source extracted.
> 2. Patches copied over LAN and applied.
>
> root@boris-System-P5Q3:~# cd Downloads/*-rc2
> root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# ls -l ../*.patch
> -rw-r--r-- 1 root root 84442 2010-11-17 01:06 ../xen.next-2.6.37.patch
> -rw-r--r-- 1 root root  4062 2010-11-17 01:06 ../xen.pcifront.fixes.patch
> root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# patch -p1 -s -i
../xen.next-2.6.37.patch
> root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# patch -p1 -s -i
../xen.pcifront.fixes.patch
> root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2#
>
> Boris.
> P.S Directory /root/Downloads/linux-2.6.37-rc2 contains mainline''s
2.6.37-rc2
>
> I feel like I must be doing something unbelievably stupid:
>
> %> rm -rf linux-2.6.37
> %> tar lxf linux-2.6.37-rc2.tar.bz2 (This is
http://www.kernel.org/pub/linux/kernel/v2.6/testing/linux-2.6.37-rc2.tar.bz2)
> %> cd linux-2.6.37-rc2
> %> patch --dry-run -p1 -s -i  ../patches.2.6.37/xen.next-2.6.37.patch
>
> 1 out of 2 hunks FAILED -- saving rejects to file
drivers/block/xen-blkfront.c.rej
> 1 out of 1 hunk FAILED -- saving rejects to file
arch/x86/include/asm/microcode.h.rej
> 1 out of 1 hunk FAILED -- saving rejects to file
arch/x86/kernel/Makefile.rej
> 1 out of 1 hunk FAILED -- saving rejects to file arch/x86/xen/Kconfig.rej
> 1 out of 2 hunks FAILED -- saving rejects to file
arch/x86/include/asm/processor.h.rej
> 1 out of 1 hunk FAILED -- saving rejects to file drivers/xen/gntdev.c.rej
>
> -Bruce
>
>
> --- On Thu, 11/18/10, Bruce Edge <bruce.edge@gmail.com> wrote:
>
> From: Bruce Edge <bruce.edge@gmail.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
> Date: Thursday, November 18, 2010, 3:05 PM
>
>
>
> On Thu, Nov 18, 2010 at 9:05 AM, Boris Derzhavets
<bderzhavets@yahoo.com> wrote:
>
> Bruce,
> You should be able to apply patches to mainline 2.6.37-rc2 cleanly. This
patches are
> taken out from MY''s kernel-2.6.37-rc2.git0.fc15.src.rpm.
> I already applied them on Ubuntu 10.10 to uncompressed mainline rc2.
>
> I''m assuming fc=fedora core here?
> Doesn''t the fc kernel have a number of other patches already
installed?
> I''m using a kernel.org kernel, which has none of the fc patches so
it''s not surprising that they don''t apply.
>
> -Bruce
>
>
> I have also to notice, that
> # mount IP-Dom0:/home/user1 /mnt/nfs
> # cd /mnt/nfs
> # ls -l
> crashes DomU immediately in text mode.
> In graphics mode it doesn''t necessary happen every time.
> DomU might survive this "hack" and crashed one
> hour latter by another reason.
>
> Boris.
>
> --- On Thu, 11/18/10, Bruce Edge <bruce.edge@gmail.com> wrote:
>
> From: Bruce Edge <bruce.edge@gmail.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com
> Date: Thursday, November 18, 2010, 11:40 AM
>
>
>
> On Thu, Nov 18, 2010 at 2:34 AM, Boris Derzhavets
<bderzhavets@yahoo.com> wrote:
>
> Could you apply two attached patches on top of 2.6.37-rc2 and see
> whether it gives some improvement or no ( with active NFS client at DomU)
>
> Boris
>
> Hi Boris,
>
> Are you using the mainline kernel or a pvops branch with these patches?
> Maybe I''m doing something wrong, but they don''t apply
cleanly with 2.6.37-rc2:
>
> %> patch --dry-run <../patches.2.6.37/xen.next-2.6.37.patch
>
> patching file pgtable.h
> Hunk #1 FAILED at 399.
> 1 out of 1 hunk FAILED -- saving rejects to file pgtable.h.rej
> patching file pgtable.c
> Hunk #1 FAILED at 15.
> 1 out of 1 hunk FAILED -- saving rejects to file pgtable.c.rej
> patching file ttm_bo_vm.c
> Hunk #1 FAILED at 273.
> Hunk #2 FAILED at 288.
> 2 out of 2 hunks FAILED -- saving rejects to file ttm_bo_vm.c.rej
> ......
>
>  %> patch --dry-run <../patches.2.6.37/xen.pcifront.fixes.patch
>
> patching file enlighten.c
> Hunk #1 FAILED at 1090.
> Hunk #2 FAILED at 1202.
> 2 out of 2 hunks FAILED -- saving rejects to file enlighten.c.rej
> patching file setup.c
> Hunk #1 FAILED at 337.
> Hunk #2 FAILED at 356.
> 2 out of 2 hunks FAILED -- saving rejects to file setup.c.rej
>
> Same result for linux-2.6.37-xen-next branch.
>
> -Bruce
>
>
> --- On Wed, 11/17/10, Bruce Edge <bruce.edge@gmail.com> wrote:
>
> From: Bruce Edge <bruce.edge@gmail.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
"Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com
> Date: Wednesday, November 17, 2010, 4:28 PM
>
> On Tue, Nov 16, 2010 at 1:49 PM, Boris Derzhavets
<bderzhavets@yahoo.com> wrote:
>
> Yes, here we are
>
> [  186.975228] ------------[ cut here ]------------
> [  186.975245] kernel BUG at mm/mmap.c:2399!
> [  186.975254] invalid opcode: 0000 [#1] SMP
> [  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
> [  186.975284] CPU 0
> [  186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr
camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic
twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic
des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet
xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport
xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp
xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ipv6
uinput xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan]
> [  186.975507]
> [  186.975515] Pid: 1562, comm: ls Not tainted
2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
> [  186.975529] RIP: e030:[<ffffffff8110ada1>] 
[<ffffffff8110ada1>] exit_mmap+0x10c/0x119
> [  186.975550] RSP: e02b:ffff8800781bde18  EFLAGS: 00010202
> [  186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
0000000000000000
> [  186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI:
ffffea00000c0280
> [  186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09:
0000000000000001
> [  186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12:
ffff880076c68b00
> [  186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15:
0000000000000001
> [  186.975625] FS:  00007f8e471d97c0(0000) GS:ffff88007f831000(0000)
knlGS:0000000000000000
> [  186.975639] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4:
0000000000002660
> [  186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
> [  186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
> [  186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task
ffff8800788223e0)
> [  186.976012] Stack:
> [  186.976012]  000000000000006b ffff88007f83f1e0 ffff8800781bde38
ffff880076c68b00
> [  186.976012]  ffff880076c68c40 ffff8800788229d0 ffff8800781bde68
ffffffff810505fc
> [  186.976012]  ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8
ffffffff81056747
> [  186.976012] Call Trace:
> [  186.976012]  [<ffffffff810505fc>] mmput+0x65/0xd8
> [  186.976012]  [<ffffffff81056747>] exit_mm+0x13e/0x14b
> [  186.976012]  [<ffffffff81056976>] do_exit+0x222/0x7c6
> [  186.976012]  [<ffffffff8100750f>] ?
xen_restore_fl_direct_end+0x0/0x1
> [  186.976012]  [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
> [  186.976012]  [<ffffffff814b3949>] ?
lockdep_sys_exit_thunk+0x35/0x67
> [  186.976012]  [<ffffffff810571b0>] do_group_exit+0x88/0xb6
> [  186.976012]  [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
> [  186.976012]  [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
> [  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89
df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
<0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
> [  186.976012] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
> [  186.976012]  RSP <ffff8800781bde18>
> [  186.976012] ---[ end trace c0f4eff4054a67e4 ]---
> [  186.976012] Fixing recursive fault but reboot is needed!
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975228] ------------[ cut here ]------------
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975254] invalid opcode: 0000 [#1] SMP
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.975269] last sysfs file:
/sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Stack:
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Call Trace:
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
>  kernel:[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00
00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74
02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
>
> --- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
wrote:
>
> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Bruce Edge"
<bruce.edge@gmail.com>
> Date: Tuesday, November 16, 2010, 4:15 PM
>
> On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets wrote:
> > > Huh. I .. what? I am confused. I thought we established that the
issue
> > > was not related to Xen PCI front? You also seem to uncomment the
> > > upstream.core.patches and the xen.pvhvm.patch - why?
> >
> > I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> > it gives failed HUNKs
>
> Uhh.. I am even more confused.
> >
> > > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
> >
> > I am pretty sure rc2 doesn''t contain everything from
xen.next-2.6.37.patch,
> > gntdev''s stuff for sure. I''ve built 2.6.37-rc2
kernel rpms and loaded
> > kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1.
> > Device /dev/xen/gntdev has not been created. I understand that
it''s
> > unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU
i cannot
> > get 3.2 GB copied over to DomU from NFS share at Dom0.
>
> So what I think you are saying is that you keep on getting the bug in DomU?
> Is the stack-trace the same as in rc1?
>
>
>
> I haven''t had much time to look into the broken/working version
issues here, but I did confirm a couple of points:
> 1) The 2.6.37-rc2 has the same problem still
> 2) This problem goes away of one is not using NFS.
>
> Not staggeringly helpful I know, but it''s one small data point.
>
> -Bruce
>
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>
>
> -----Inline Attachment Follows-----
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>
>
I just checked the recently released 2.6.73-rc4, and while the BUG
signature is different, it still fails under NFS accesses. This is
100% recreatable.

0 uss004-dpm  ~ #> [   60.303318] ------------[ cut here ]------------
[   60.303335] kernel BUG at arch/x86/xen/mmu.c:1831!
[   60.303345] invalid opcode: 0000 [#1] SMP
[   60.303362] last sysfs file: /sys/kernel/uevent_seqnum
[   60.303371] CPU 0
[   60.303377] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
xen_netfront xen_blkfront
[   60.303444]
[   60.303453] Pid: 3965, comm: cron Not tainted
2.6.37-rc4-kernel.org-domu-6.0.nopvhvm.bedge-debug #1 /
[   60.303468] RIP: e030:[<ffffffff810047dd>]  [<ffffffff810047dd>]
pin_pagetable_pfn+0x31/0x37
[   60.303491] RSP: e02b:ffff880077d37ce8  EFLAGS: 00010282
[   60.303501] RAX: 00000000ffffffea RBX: 0000000000077da4 RCX:
0000000000000001
[   60.303512] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
00000000deadbeef
[   60.303524] RBP: ffff880077d37d08 R08: 00003ffffffff000 R09:
ffff880000000000
[   60.303536] R10: 00000000deadbeef R11: dead000000200200 R12:
0000000000000003
[   60.303548] R13: ffff880077d01598 R14: ffff880075dbd7e8 R15:
ffff880076335070
[   60.303564] FS:  00007fd3d87aa7a0(0000) GS:ffff88007ff40000(0000)
knlGS:0000000000000000
[   60.303577] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[   60.303588] CR2: 00007fd3d663d398 CR3: 00000000776b1000 CR4:
0000000000002660
[   60.303600] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[   60.303612] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[   60.303624] Process cron (pid: 3965, threadinfo ffff880077d36000,
task ffff880076335070)
[   60.303636] Stack:
[   60.303643]  ffff880000000000 0000000000047c6a ffff880076297800
0000000000077da4
[   60.303671]  ffff880077d37d28 ffffffff8100665c 0000000000077da4
ffff880076297800
[   60.303699]  ffff880077d37d38 ffffffff8100668f ffff880077d37d78
ffffffff810e0653
[   60.303726] Call Trace:
[   60.303739]  [<ffffffff8100665c>] xen_alloc_ptpage+0x68/0x6d
[   60.303754]  [<ffffffff8100668f>] xen_alloc_pte+0xe/0x10
[   60.303769]  [<ffffffff810e0653>] __pte_alloc+0x72/0xd7
[   60.303783]  [<ffffffff810e28e2>] handle_mm_fault+0x131/0x8c1
[   60.303797]  [<ffffffff810e7006>] ? mmap_region+0x391/0x4bf
[   60.303812]  [<ffffffff8144740f>] do_page_fault+0x374/0x396
[   60.303827]  [<ffffffff8102b7be>] ?
pvclock_clocksource_read+0x4b/0xb4
[   60.303843]  [<ffffffff81007be9>] ? __spin_time_accum+0x21/0x37
[   60.303858]  [<ffffffff81007e65>] ? __xen_spin_lock+0xb7/0xcd
[   60.303873]  [<ffffffff81444615>] page_fault+0x25/0x30
[   60.303883] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 67 cb ff ff
85 c0 74 04 <0f>
0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 30 48
[   60.304076] RIP  [<ffffffff810047dd>] pin_pagetable_pfn+0x31/0x37
[   60.304076]  RSP <ffff880077d37ce8>
[   60.304076] ---[ end trace 8bd01a369ffae446 ]---
[   64.355381] kmemleak: 4 new suspected memory leaks (see
/sys/kernel/debug/kmemleak)

Another domU boot yielded a slightly different crash:

uss004-dpm login: [  164.897190] ------------[ cut here ]------------
[  164.897208] kernel BUG at arch/x86/xen/mmu.c:1831!
[  164.897221] invalid opcode: 0000 [#1] SMP
[  164.897238] last sysfs file: /sys/kernel/uevent_seqnum
[  164.897250] CPU 2
[  164.897257] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
xen_netfront xen_blkfront
[  164.897326]
[  164.897337] Pid: 3938, comm: cron Not tainted
2.6.37-rc4-kernel.org-domu-6.0.nopvhvm.bedge-debug #1 /
[  164.897352] RIP: e030:[<ffffffff810047dd>]  [<ffffffff810047dd>]
pin_pagetable_pfn+0x31/0x37
[  164.897375] RSP: e02b:ffff8800781dbce8  EFLAGS: 00010282
[  164.897386] RAX: 00000000ffffffea RBX: 0000000000077e3b RCX: 0000000000000001
[  164.897398] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
[  164.897410] RBP: ffff8800781dbd08 R08: 00003ffffffff000 R09: ffff880000000000
[  164.897421] R10: 00000000deadbeef R11: dead000000200200 R12: 0000000000000003
[  164.897433] R13: ffff88007646a0f0 R14: ffff880076309678 R15: ffff880075c08ea0
[  164.897451] FS:  00007fcd44d5e7a0(0000) GS:ffff88007ff78000(0000)
knlGS:0000000000000000
[  164.897464] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  164.897474] CR2: 00007fcd43d55850 CR3: 0000000075e6e000 CR4: 0000000000002660
[  164.897488] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  164.897500] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  164.897513] Process cron (pid: 3938, threadinfo ffff8800781da000,
task ffff880075c08ea0)
[  164.897528] Stack:
[  164.897536]  ffff880000000000 0000000000047bd3 ffff88007631c700
0000000000077e3b
[  164.897565]  ffff8800781dbd28 ffffffff8100665c 0000000000077e3b
ffff88007631c700
[  164.897595]  ffff8800781dbd38 ffffffff8100668f ffff8800781dbd78
ffffffff810e0653
[  164.897625] Call Trace:
[  164.897639]  [<ffffffff8100665c>] xen_alloc_ptpage+0x68/0x6d
[  164.897655]  [<ffffffff8100668f>] xen_alloc_pte+0xe/0x10
[  164.897670]  [<ffffffff810e0653>] __pte_alloc+0x72/0xd7
[  164.897684]  [<ffffffff810e28e2>] handle_mm_fault+0x131/0x8c1
[  164.897699]  [<ffffffff8100723f>] ? xen_restore_fl_direct_end+0x0/0x1
[  164.897714]  [<ffffffff8100429c>] ? xen_mc_flush+0x1cd/0x1f2
[  164.897730]  [<ffffffff8144740f>] do_page_fault+0x374/0x396
[  164.897744]  [<ffffffff81007252>] ? check_events+0x12/0x20
[  164.897758]  [<ffffffff81006bf1>] ? xen_force_evtchn_callback+0xd/0xf
[  164.897772]  [<ffffffff81007252>] ? check_events+0x12/0x20
[  164.897786]  [<ffffffff8100723f>] ? xen_restore_fl_direct_end+0x0/0x1
[  164.897804]  [<ffffffff81003d62>] ? T.1087+0x22/0x24
[  164.897816]  [<ffffffff81003e31>] ? xen_clts+0x57/0x59
[  164.897832]  [<ffffffff81444615>] page_fault+0x25/0x30
[  164.897842] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 67 cb ff ff
85 c0 74 04 <0f>
0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 30 48
[  164.898062] RIP  [<ffffffff810047dd>] pin_pagetable_pfn+0x31/0x37
[  164.898062]  RSP <ffff8800781dbce8>
[  164.898062] ---[ end trace 85c4388599f45d39 ]---



The dom0 printed this on the serial console around the same time the
domU crashed:

0 uss004  ~ #> (XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 !exp
1000000000000000) for mfn 47c6a (pfn 77da4)
(XEN) mm.c:2965:d5 Error while pinning mfn 47c6a

Then, a few min later started spewing these to the dom0 console:

(XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp
3000000000000000) for mfn 49c0a (pfn 75e04)
(XEN) mm.c:964:d5 Attempt to create linear p.t. with write perms
(XEN) mm.c:1505:d5 Failure in alloc_l4_table: entry 0
(XEN) mm.c:2142:d5 Error while validating mfn 484d4 (pfn 7753a) for
type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2965:d5 Error while pinning mfn 484d4
(XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp
3000000000000000) for mfn 49c0a (pfn 75e04)
(XEN) mm.c:964:d5 Attempt to create linear p.t. with write perms
(XEN) mm.c:1505:d5 Failure in alloc_l4_table: entry 0
(XEN) mm.c:2142:d5 Error while validating mfn 49b69 (pfn 75ea5) for
type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2965:d5 Error while pinning mfn 49b69
(XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp
3000000000000000) for mfn 49c0a (pfn 75e04)
(XEN) mm.c:964:d5 Attempt to create linear p.t. with write perms
(XEN) mm.c:1505:d5 Failure in alloc_l4_table: entry 0
(XEN) mm.c:2142:d5 Error while validating mfn 484d4 (pfn 7753a) for
type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2732:d5 Error while installing new baseptr 484d4
(XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp
3000000000000000) for mfn 49c0a (pfn 75e04)
(XEN) mm.c:964:d5 Attempt to create linear p.t. with write perms
(XEN) mm.c:1505:d5 Failure in alloc_l4_table: entry 0
(XEN) mm.c:2142:d5 Error while validating mfn 49b69 (pfn 75ea5) for
type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:3057:d5 Error while installing new mfn 49b69
(XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp
3000000000000000) for mfn 49c0a (pfn 75e04)
(XEN) mm.c:964:d5 Attempt to create linear p.t. with write perms
(XEN) mm.c:1505:d5 Failure in alloc_l4_table: entry 0
(XEN) mm.c:2142:d5 Error while validating mfn 484d4 (pfn 7753a) for
type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2732:d5 Error while installing new baseptr 484d4
(XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp
3000000000000000) for mfn 49c0a (pfn 75e04)
(XEN) mm.c:964:d5 Attempt to create linear p.t. with write perms
(XEN) mm.c:1505:d5 Failure in alloc_l4_table: entry 0
(XEN) mm.c:2142:d5 Error while validating mfn 49b69 (pfn 75ea5) for
type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:3057:d5 Error while installing new mfn 49b69
..........
These messages continued until I killed the domU.

The hypervisor is xen-unstable 22241

Here''s the complete domU boot log:

Started domain dpm (id=5)
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version
2.6.37-rc4-kernel.org-domu-6.0.nopvhvm.bedge-debug (root@opal) (gcc
version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Mon Nov 29 22:22:40 PST
2010
[    0.000000] Command line:
root=UUID=e971c31a-7353-4ac7-90c4-d6eb7fc33735 ro   iommu=soft
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] released 0 pages of unused memory
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000080800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[    0.000000] Scanning 0 areas for low memory corruption
[    0.000000] init_memory_mapping: 0000000000000000-0000000080800000
[    0.000000] RAMDISK: 01d95000 - 02faf000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-0000000080800000
[    0.000000] Initmem setup node 0 0000000000000000-0000000080800000
[    0.000000]   NODE_DATA [000000007fffb000 - 000000007fffffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x000000a0
[    0.000000]     0: 0x00000100 -> 0x00080800
[    0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
[    0.000000] No local APIC present
[    0.000000] APIC: disable apic facility
[    0.000000] APIC: switched to apic NOOP
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
[    0.000000] Allocating PCI resources starting at 80800000 (gap:
80800000:7f800000)
[    0.000000] Booting paravirtualized kernel on Xen
[    0.000000] Xen version: 4.1-unstable (preserve-AD)
[    0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
nr_cpu_ids:6 nr_node_ids:1
[    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
r8192 d23744 u114688
[    0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.
Total pages: 519022
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:
root=UUID=e971c31a-7353-4ac7-90c4-d6eb7fc33735 ro   iommu=soft
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Placing 64MB software IO TLB between ffff880079400000 -
ffff88007d400000
[    0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
[    0.000000] Memory: 1959872k/2105344k available (4402k kernel code,
448k absent, 145024k reserved, 6323k data, 756k init)
[    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=6, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU-based detection of stalled CPUs is disabled.
[    0.000000] NR_IRQS:4352 nr_irqs:320 16
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [hvc0] enabled
[    0.000000] installing Xen timer for CPU 0
[    0.000000] Detected 2660.088 MHz processor.
[    0.000999] Calibrating delay loop (skipped), value calculated
using timer frequency.. 5320.17 BogoMIPS (lpj=2660088)
[    0.000999] pid_max: default: 32768 minimum: 301
[    0.000999] Security Framework initialized
[    0.000999] SELinux:  Initializing.
[    0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.001403] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.001616] Mount-cache hash table entries: 256
[    0.002223] Initializing cgroup subsys ns
[    0.002235] ns_cgroup deprecated: consider using the
''clone_children'' flag without the ns_cgroup.
[    0.002302] Initializing cgroup subsys cpuacct
[    0.002321] Initializing cgroup subsys freezer
[    0.002450] CPU: Unsupported number of siblings 16
[    0.002456] Performance Events: unsupported p6 CPU model 26 no PMU
driver, software events only.
[    0.002543] SMP alternatives: switching to UP code
[    0.003119] cpu 0 spinlock event irq 318
[    0.003764] NMI watchdog failed to create perf event on cpu0:
fffffffffffffffe
[    0.004040] installing Xen timer for CPU 1
[    0.004147] cpu 1 spinlock event irq 312
[    0.004184] SMP alternatives: switching to SMP code
[    0.000999] CPU: Unsupported number of siblings 16
[    0.005154] NMI watchdog failed to create perf event on cpu1:
fffffffffffffffe
[    0.005466] installing Xen timer for CPU 2
[    0.005570] cpu 2 spinlock event irq 306
[    0.000999] CPU: Unsupported number of siblings 16
[    0.005856] NMI watchdog failed to create perf event on cpu2:
fffffffffffffffe
[    0.006027] installing Xen timer for CPU 3
[    0.006068] cpu 3 spinlock event irq 300
[    0.000999] CPU: Unsupported number of siblings 16
[    0.006374] NMI watchdog failed to create perf event on cpu3:
fffffffffffffffe
[    0.006608] installing Xen timer for CPU 4
[    0.006716] cpu 4 spinlock event irq 294
[    0.000999] CPU: Unsupported number of siblings 16
[    0.006999] NMI watchdog failed to create perf event on cpu4:
fffffffffffffffe
[    0.007213] installing Xen timer for CPU 5
[    0.007316] cpu 5 spinlock event irq 288
[    0.000999] CPU: Unsupported number of siblings 16
[    0.007594] NMI watchdog failed to create perf event on cpu5:
fffffffffffffffe
[    0.007605] Brought up 6 CPUs
[    0.008290] kworker/u:0 used greatest stack depth: 5696 bytes left
[    0.011150] Grant table initialized
[    0.030141] Time: 165:165:165  Date: 165/165/65
[    0.030386] NET: Registered protocol family 16
[    0.032157] kworker/u:0 used greatest stack depth: 5528 bytes left
[    0.035695] PCI: setting up Xen PCI frontend stub
[    0.063222] bio: create slab <bio-0> at 0
[    0.064255] ACPI: Interpreter disabled.
[    0.065099] xen_balloon: Initialising balloon driver.
[    0.065149] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[    0.066259] vgaarb: loaded
[    0.067215] usbcore: registered new interface driver usbfs
[    0.068051] usbcore: registered new interface driver hub
[    0.068163] usbcore: registered new device driver usb
[    0.070102] PCI: System does not support PCI
[    0.070147] PCI: System does not support PCI
[    0.071161] NetLabel: Initializing
[    0.071161] NetLabel:  domain hash size = 128
[    0.071161] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.071276] NetLabel:  unlabeled traffic allowed by default
[    0.072077] Switching to clocksource xen
[    0.086791] pnp: PnP ACPI: disabled
[    0.121206] NET: Registered protocol family 2
[    0.121669] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.124289] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[    0.125702] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.125921] TCP: Hash tables configured (established 262144 bind 65536)
[    0.125934] TCP reno registered
[    0.125984] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[    0.126043] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[    0.126722] NET: Registered protocol family 1
[    0.127683] RPC: Registered udp transport module.
[    0.128057] RPC: Registered tcp transport module.
[    0.128057] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.140312] Trying to unpack rootfs image as initramfs...
[    0.181030] Freeing initrd memory: 18536k freed
[    0.270706] DMA-API: preallocated 32768 debug entries
[    0.270720] DMA-API: debugging enabled by kernel config
[    0.272362] platform rtc_cmos: registered platform RTC device (no
PNP device found)
[    0.282120] Machine check injector initialized
[    0.288555] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
[    0.288614] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
[    0.288697] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
[    0.288738] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
[    0.288806] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
[    0.288849] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
[    0.289277] microcode: Microcode Update Driver: v2.00
<tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    0.289295] Scanning for low memory corruption every 60 seconds
[    0.290872] audit: initializing netlink socket (disabled)
[    0.290949] type=2000 audit(1291237981.045:1): initialized
[    0.307918] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.346385] VFS: Disk quotas dquot_6.5.2
[    0.347173] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.352395] NTFS driver 2.1.29 [Flags: R/W].
[    0.354350] msgmni has been set to 3864
[    0.359310] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[    0.359327] io scheduler noop registered
[    0.359336] io scheduler deadline registered
[    0.359975] io scheduler cfq registered (default)
[    0.361893] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.376639] pcifront pci-0: Installing PCI frontend
[    0.377039] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[    0.387317] pcifront pci-0: claiming resource 0000:00:00.0/0
[    0.387317] pcifront pci-0: claiming resource 0000:00:00.0/2
[    0.387317] pcifront pci-0: claiming resource 0000:00:00.0/3
[    0.387317] pcifront pci-0: claiming resource 0000:00:00.0/4
[    0.387317] pcifront pci-0: claiming resource 0000:00:01.0/0
[    0.387317] pcifront pci-0: claiming resource 0000:00:01.0/2
[    0.387317] pcifront pci-0: claiming resource 0000:00:01.0/3
[    0.387317] pcifront pci-0: claiming resource 0000:00:01.0/4
[    0.387317] pcifront pci-0: claiming resource 0000:00:02.0/0
[    0.387317] pcifront pci-0: claiming resource 0000:00:02.0/2
[    0.387317] pcifront pci-0: claiming resource 0000:00:02.0/3
[    0.387317] pcifront pci-0: claiming resource 0000:00:02.0/4
[    0.387317] pcifront pci-0: claiming resource 0000:00:03.0/0
[    0.387317] pcifront pci-0: claiming resource 0000:00:03.0/2
[    0.387317] pcifront pci-0: claiming resource 0000:00:03.0/3
[    0.387317] pcifront pci-0: claiming resource 0000:00:03.0/4
[    0.683090] Non-volatile memory driver v1.3
[    0.683109] Linux agpgart interface v0.103
[    0.684495] [drm] Initialized drm 1.1.0 20060810
[    0.684516] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.782067] brd: module loaded
[    0.795822] loop: module loaded
[    0.796963] Fixed MDIO Bus: probed
[    0.799041] ehci_hcd: USB 2.0 ''Enhanced'' Host Controller
(EHCI) Driver
[    0.799696] ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI)
Driver
[    0.800288] uhci_hcd: USB Universal Host Controller Interface driver
[    0.802024] usbcore: registered new interface driver usblp
[    0.802618] usbcore: registered new interface driver libusual
[    0.804297] PNP: No PS/2 controller found. Probing ports directly.
[    0.805147] i8042.c: No controller found.
[    0.805764] mice: PS/2 mouse device common for all mice
[    0.808307] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[    0.808937] rtc_cmos: probe of rtc_cmos failed with error -38
[    0.810975] cpuidle: using governor ladder
[    0.810988] cpuidle: using governor menu
[    0.811099] Netfilter messages via NETLINK v0.30.
[    0.811187] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    0.812949] ctnetlink v0.93: registering with nfnetlink.
[    0.815366] ip_tables: (C) 2000-2006 Netfilter Core Team
[    0.815763] TCP cubic registered
[    0.815773] Initializing XFRM netlink socket
[    0.818404] NET: Registered protocol family 10
[    0.822365] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    0.822752] IPv6 over IPv4 tunneling driver
[    0.826707] NET: Registered protocol family 17
[    0.826938] Registering the dns_resolver key type
[    0.829092] registered taskstats version 1
[    0.829381] kmemleak: Kernel memory leak detector initialized
[    0.829400] kmemleak: Automatic memory scanning thread started
[    0.830361] XENBUS: Device with no driver: device/vbd/51712
[    0.830361] XENBUS: Device with no driver: device/vif/0
[    0.830361] XENBUS: Device with no driver: device/console/0
[    0.830361]   Magic number: 1:252:3141
[   13.345029] Freeing unused kernel memory: 756k freed
[   13.345264] Write protecting the kernel read-only data: 10240k
[   13.355174] Freeing unused kernel memory: 1720k freed
[   13.357141] Freeing unused kernel memory: 1988k freed
[   13.413416] exe used greatest stack depth: 5360 bytes left
[   13.454058] exe used greatest stack depth: 5296 bytes left
[   13.480525] udevd (1111): /proc/1111/oom_adj is deprecated, please
use /proc/1111/oom_score_adj instead.
Begin: Loading essential drivers... ...
[   13.503292] blkid used greatest stack depth: 5088 bytes left
[   13.522790] blkfront: xvda: barriers enabled
[   13.534570]  xvda: xvda1 xvda2 < xvda5 >
[   13.606023] blkid used greatest stack depth: 4752 bytes left
[   13.631992] Initialising Xen virtual ethernet driver.
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
[   13.885288] EXT4-fs (xvda1): mounted filesystem with ordered data
mode. Opts: (null)
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
[   14.704681] hwclock used greatest stack depth: 4592 bytes left
init: ureadahead main process (1211) terminated with status 5
[   14.812832] plymouthd used greatest stack depth: 4352 bytes left
[   15.402077] cp used greatest stack depth: 4304 bytes left
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
/dev/xvda1: clean, 56385/288576 files, 309277/1153280 blocks
 * Enable login on console                                               [ OK ]
 * Check for hostname changes...                                         [ OK ]
   ...done.
 * Setting sensors limits
         No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
                                                                         [ OK ]
 * Initializing random number generator...                               [ OK ]
 * Setting up X server socket directory /tmp/.X11-unix...                [ OK ]
 * Setting up ICE socket directory /tmp/.ICE-unix...                     [ OK ]
 * Not starting fancontrol; run pwmconfig first.
 * Starting internet superserver inetd                                   [ OK ]
Starting openntpd: ntpd.
 * Starting Tomcat servlet engine tomcat6
         Using CATALINA_BASE:   /var/lib/tomcat6
Using CATALINA_HOME:   /usr/share/tomcat6
Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
Using JRE_HOME:        /usr/lib/jvm/java-6-openjdk
Using CLASSPATH:       /usr/share/tomcat6/bin/bootstrap.jar
                                                                         [ OK ]
 * Recording successful boot for GRUB
   ...done.
 * Running local boot scripts (/etc/rc.local)                            [ OK ]
mountall: Plymouth command failed
mountall: Disconnected from Plymouth

uss004-dpm login: root
Password:
Linux uss004-dpm 2.6.37-rc4-kernel.org-domu-6.0.nopvhvm.bedge-debug #1
SMP Mon Nov 29 22:22:40 PST 2010 x86_64 GNU/Linux
DPM-8800 6.0.16-01

Welcome to Ubuntu!
 * Documentation:  https://help.ubuntu.com/
uss004-dpm:~# zsh
0 uss004-dpm  ~ #> mount
/dev/xvda1 on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/kernel/debug type debugfs (rw)
none on /dev type tmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
none on /dev/shm type tmpfs (rw,nosuid,nodev)
none on /var/run type tmpfs (rw,nosuid,mode=0755)
none on /var/lock type tmpfs (rw,noexec,nosuid,nodev)
none on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
sys:/common/var/dpm/log on /var/log type nfs
(rw,nolock,rsize=32768,wsize=32768,sloppy,addr=135.149.78.230)
sys:/common/var/dpm/core on /var/core type nfs
(rw,nolock,rsize=32768,wsize=32768,sloppy,addr=135.149.78.230)


DomU config is attached


-Bruce


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2010-Dec-02 06:33 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On 12/01/2010 01:32 PM, Bruce Edge wrote:> I just checked the recently released 2.6.73-rc4,
Do you mean mainline 2.6.37-rc4, or the one in xen/next-2.6.37?
>  and while the BUG
> signature is different, it still fails under NFS accesses. This is
> 100% recreatable.
Please try this patch which is queued up for mainline.  It''s already in
xen/next-2.6.37.

    J


diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
index 21ed8d7..0e4ecac 100644
--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -2358,8 +2358,6 @@ void __init xen_init_mmu_ops(void)
 	x86_init.paging.pagetable_setup_done = xen_pagetable_setup_done;
 	pv_mmu_ops = xen_mmu_ops;
 
-	vmap_lazy_unmap = false;
-
 	memset(dummy_mapping, 0xff, PAGE_SIZE);
 }
 
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index a03dcf6..44b54f6 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -7,8 +7,6 @@
 
 struct vm_area_struct;		/* vma defining user mapping in mm_types.h */
 
-extern bool vmap_lazy_unmap;
-
 /* bits in flags of vmalloc''s vm_struct below */
 #define VM_IOREMAP	0x00000001	/* ioremap() and friends */
 #define VM_ALLOC	0x00000002	/* vmalloc() */
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index a3d66b3..eb5cc7d 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -31,8 +31,6 @@
 #include <asm/tlbflush.h>
 #include <asm/shmparam.h>
 
-bool vmap_lazy_unmap __read_mostly = true;
-
 /*** Page table manipulation functions ***/
 
 static void vunmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end)
@@ -503,9 +501,6 @@ static unsigned long lazy_max_pages(void)
 {
 	unsigned int log;
 
-	if (!vmap_lazy_unmap)
-		return 0;
-
 	log = fls(num_online_cpus());
 
 	return log * (32UL * 1024 * 1024 / PAGE_SIZE);
@@ -566,7 +561,6 @@ static void __purge_vmap_area_lazy(unsigned long *start,
unsigned long *end,
 			if (va->va_end > *end)
 				*end = va->va_end;
 			nr += (va->va_end - va->va_start) >> PAGE_SHIFT;
-			unmap_vmap_area(va);
 			list_add_tail(&va->purge_list, &valist);
 			va->flags |= VM_LAZY_FREEING;
 			va->flags &= ~VM_LAZY_FREE;
@@ -611,10 +605,11 @@ static void purge_vmap_area_lazy(void)
 }
 
 /*
- * Free and unmap a vmap area, caller ensuring flush_cache_vunmap had been
- * called for the correct range previously.
+ * Free a vmap area, caller ensuring that the area has been unmapped
+ * and flush_cache_vunmap had been called for the correct range
+ * previously.
  */
-static void free_unmap_vmap_area_noflush(struct vmap_area *va)
+static void free_vmap_area_noflush(struct vmap_area *va)
 {
 	va->flags |= VM_LAZY_FREE;
 	atomic_add((va->va_end - va->va_start) >> PAGE_SHIFT,
&vmap_lazy_nr);
@@ -623,6 +618,16 @@ static void free_unmap_vmap_area_noflush(struct vmap_area
*va)
 }
 
 /*
+ * Free and unmap a vmap area, caller ensuring flush_cache_vunmap had been
+ * called for the correct range previously.
+ */
+static void free_unmap_vmap_area_noflush(struct vmap_area *va)
+{
+	unmap_vmap_area(va);
+	free_vmap_area_noflush(va);
+}
+
+/*
  * Free and unmap a vmap area
  */
 static void free_unmap_vmap_area(struct vmap_area *va)
@@ -798,7 +803,7 @@ static void free_vmap_block(struct vmap_block *vb)
 	spin_unlock(&vmap_block_tree_lock);
 	BUG_ON(tmp != vb);
 
-	free_unmap_vmap_area_noflush(vb->va);
+	free_vmap_area_noflush(vb->va);
 	call_rcu(&vb->rcu_head, rcu_free_vb);
 }
 
@@ -936,6 +941,8 @@ static void vb_free(const void *addr, unsigned long size)
 	rcu_read_unlock();
 	BUG_ON(!vb);
 
+	vunmap_page_range((unsigned long)addr, (unsigned long)addr + size);
+
 	spin_lock(&vb->lock);
 	BUG_ON(bitmap_allocate_region(vb->dirty_map, offset >> PAGE_SHIFT,
order));
 
@@ -988,7 +995,6 @@ void vm_unmap_aliases(void)
 
 				s = vb->va->va_start + (i << PAGE_SHIFT);
 				e = vb->va->va_start + (j << PAGE_SHIFT);
-				vunmap_page_range(s, e);
 				flush = 1;
 
 				if (s < start)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2010-Dec-02 08:33 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Could you , please, resend the patch as raw attachment.

Boris.

--- On Thu, 12/2/10, Jeremy Fitzhardinge <jeremy@goop.org> wrote:

From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle
kernel paging request
To: "Bruce Edge" <bruce.edge@gmail.com>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>,
xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk"
<konrad.wilk@oracle.com>
Date: Thursday, December 2, 2010, 1:33 AM

On 12/01/2010 01:32 PM, Bruce Edge wrote:> I just checked the recently released 2.6.73-rc4,
Do you mean mainline 2.6.37-rc4, or the one in xen/next-2.6.37?
>  and while the BUG
> signature is different, it still fails under NFS accesses. This is
> 100% recreatable.
Please try this patch which is queued up for mainline.  It''s already in
xen/next-2.6.37.

    J


diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
index 21ed8d7..0e4ecac 100644
--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -2358,8 +2358,6 @@ void __init xen_init_mmu_ops(void)
     x86_init.paging.pagetable_setup_done = xen_pagetable_setup_done;
     pv_mmu_ops = xen_mmu_ops;
 
-    vmap_lazy_unmap = false;
-
     memset(dummy_mapping, 0xff, PAGE_SIZE);
 }
 
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index a03dcf6..44b54f6 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -7,8 +7,6 @@
 
 struct vm_area_struct;        /* vma defining user mapping in mm_types.h */
 
-extern bool vmap_lazy_unmap;
-
 /* bits in flags of vmalloc''s vm_struct below */
 #define VM_IOREMAP    0x00000001    /* ioremap() and friends */
 #define VM_ALLOC    0x00000002    /* vmalloc() */
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index a3d66b3..eb5cc7d 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -31,8 +31,6 @@
 #include <asm/tlbflush.h>
 #include <asm/shmparam.h>
 
-bool vmap_lazy_unmap __read_mostly = true;
-
 /*** Page table manipulation functions ***/
 
 static void vunmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end)
@@ -503,9 +501,6 @@ static unsigned long lazy_max_pages(void)
 {
     unsigned int log;
 
-    if (!vmap_lazy_unmap)
-        return 0;
-
     log = fls(num_online_cpus());
 
     return log * (32UL * 1024 * 1024 / PAGE_SIZE);
@@ -566,7 +561,6 @@ static void __purge_vmap_area_lazy(unsigned long *start,
unsigned long *end,
             if (va->va_end > *end)
                 *end = va->va_end;
             nr += (va->va_end - va->va_start) >> PAGE_SHIFT;
-            unmap_vmap_area(va);
             list_add_tail(&va->purge_list, &valist);
             va->flags |= VM_LAZY_FREEING;
             va->flags &= ~VM_LAZY_FREE;
@@ -611,10 +605,11 @@ static void purge_vmap_area_lazy(void)
 }
 
 /*
- * Free and unmap a vmap area, caller ensuring flush_cache_vunmap had been
- * called for the correct range previously.
+ * Free a vmap area, caller ensuring that the area has been unmapped
+ * and flush_cache_vunmap had been called for the correct range
+ * previously.
  */
-static void free_unmap_vmap_area_noflush(struct vmap_area *va)
+static void free_vmap_area_noflush(struct vmap_area *va)
 {
     va->flags |= VM_LAZY_FREE;
     atomic_add((va->va_end - va->va_start) >> PAGE_SHIFT,
&vmap_lazy_nr);
@@ -623,6 +618,16 @@ static void free_unmap_vmap_area_noflush(struct vmap_area
*va)
 }
 
 /*
+ * Free and unmap a vmap area, caller ensuring flush_cache_vunmap had been
+ * called for the correct range previously.
+ */
+static void free_unmap_vmap_area_noflush(struct vmap_area *va)
+{
+    unmap_vmap_area(va);
+    free_vmap_area_noflush(va);
+}
+
+/*
  * Free and unmap a vmap area
  */
 static void free_unmap_vmap_area(struct vmap_area *va)
@@ -798,7 +803,7 @@ static void free_vmap_block(struct vmap_block *vb)
     spin_unlock(&vmap_block_tree_lock);
     BUG_ON(tmp != vb);
 
-    free_unmap_vmap_area_noflush(vb->va);
+    free_vmap_area_noflush(vb->va);
     call_rcu(&vb->rcu_head, rcu_free_vb);
 }
 
@@ -936,6 +941,8 @@ static void vb_free(const void *addr, unsigned long size)
     rcu_read_unlock();
     BUG_ON(!vb);
 
+    vunmap_page_range((unsigned long)addr, (unsigned long)addr + size);
+
     spin_lock(&vb->lock);
     BUG_ON(bitmap_allocate_region(vb->dirty_map, offset >> PAGE_SHIFT,
order));
 
@@ -988,7 +995,6 @@ void vm_unmap_aliases(void)
 
                 s = vb->va->va_start + (i << PAGE_SHIFT);
                 e = vb->va->va_start + (j << PAGE_SHIFT);
-                vunmap_page_range(s, e);
                 flush = 1;
 
                 if (s < start)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Dec-02 14:41 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Wed, Dec 1, 2010 at 10:33 PM, Jeremy Fitzhardinge
<jeremy@goop.org>wrote:
> On 12/01/2010 01:32 PM, Bruce Edge wrote:
> > I just checked the recently released 2.6.73-rc4,
>
> Do you mean mainline 2.6.37-rc4, or the one in xen/next-2.6.37?
>
mainline

>
> >  and while the BUG
> > signature is different, it still fails under NFS accesses. This is
> > 100% recreatable.
>
> Please try this patch which is queued up for mainline.  It''s
already in
> xen/next-2.6.37.
>
After testing the mainline .37 I also tried xen/next-2.6.37 with the same
result. At least, I still hit a BUG after accessing an NFS mount. I
didn''t
compare the stack, but the BUG condition looked the same.
I''ll reconfirm the location and traceback of the xen/next kernel.

-Bruce


>
>     J
>
>
> diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
> index 21ed8d7..0e4ecac 100644
> --- a/arch/x86/xen/mmu.c
> +++ b/arch/x86/xen/mmu.c
> @@ -2358,8 +2358,6 @@ void __init xen_init_mmu_ops(void)
>        x86_init.paging.pagetable_setup_done = xen_pagetable_setup_done;
>        pv_mmu_ops = xen_mmu_ops;
>
> -       vmap_lazy_unmap = false;
> -
>        memset(dummy_mapping, 0xff, PAGE_SIZE);
>  }
>
> diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
> index a03dcf6..44b54f6 100644
> --- a/include/linux/vmalloc.h
> +++ b/include/linux/vmalloc.h
> @@ -7,8 +7,6 @@
>
>  struct vm_area_struct;         /* vma defining user mapping in mm_types.h
> */
>
> -extern bool vmap_lazy_unmap;
> -
>  /* bits in flags of vmalloc''s vm_struct below */
>  #define VM_IOREMAP     0x00000001      /* ioremap() and friends */
>  #define VM_ALLOC       0x00000002      /* vmalloc() */
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index a3d66b3..eb5cc7d 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -31,8 +31,6 @@
>  #include <asm/tlbflush.h>
>  #include <asm/shmparam.h>
>
> -bool vmap_lazy_unmap __read_mostly = true;
> -
>  /*** Page table manipulation functions ***/
>
>  static void vunmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long
> end)
> @@ -503,9 +501,6 @@ static unsigned long lazy_max_pages(void)
>  {
>        unsigned int log;
>
> -       if (!vmap_lazy_unmap)
> -               return 0;
> -
>        log = fls(num_online_cpus());
>
>        return log * (32UL * 1024 * 1024 / PAGE_SIZE);
> @@ -566,7 +561,6 @@ static void __purge_vmap_area_lazy(unsigned long
> *start, unsigned long *end,
>                        if (va->va_end > *end)
>                                *end = va->va_end;
>                        nr += (va->va_end - va->va_start) >>
PAGE_SHIFT;
> -                       unmap_vmap_area(va);
>                        list_add_tail(&va->purge_list, &valist);
>                        va->flags |= VM_LAZY_FREEING;
>                        va->flags &= ~VM_LAZY_FREE;
> @@ -611,10 +605,11 @@ static void purge_vmap_area_lazy(void)
>  }
>
>  /*
> - * Free and unmap a vmap area, caller ensuring flush_cache_vunmap had been
> - * called for the correct range previously.
> + * Free a vmap area, caller ensuring that the area has been unmapped
> + * and flush_cache_vunmap had been called for the correct range
> + * previously.
>  */
> -static void free_unmap_vmap_area_noflush(struct vmap_area *va)
> +static void free_vmap_area_noflush(struct vmap_area *va)
>  {
>        va->flags |= VM_LAZY_FREE;
>        atomic_add((va->va_end - va->va_start) >> PAGE_SHIFT,
> &vmap_lazy_nr);
> @@ -623,6 +618,16 @@ static void free_unmap_vmap_area_noflush(struct
> vmap_area *va)
>  }
>
>  /*
> + * Free and unmap a vmap area, caller ensuring flush_cache_vunmap had been
> + * called for the correct range previously.
> + */
> +static void free_unmap_vmap_area_noflush(struct vmap_area *va)
> +{
> +       unmap_vmap_area(va);
> +       free_vmap_area_noflush(va);
> +}
> +
> +/*
>  * Free and unmap a vmap area
>  */
>  static void free_unmap_vmap_area(struct vmap_area *va)
> @@ -798,7 +803,7 @@ static void free_vmap_block(struct vmap_block *vb)
>        spin_unlock(&vmap_block_tree_lock);
>        BUG_ON(tmp != vb);
>
> -       free_unmap_vmap_area_noflush(vb->va);
> +       free_vmap_area_noflush(vb->va);
>        call_rcu(&vb->rcu_head, rcu_free_vb);
>  }
>
> @@ -936,6 +941,8 @@ static void vb_free(const void *addr, unsigned long
> size)
>        rcu_read_unlock();
>        BUG_ON(!vb);
>
> +       vunmap_page_range((unsigned long)addr, (unsigned long)addr + size);
> +
>        spin_lock(&vb->lock);
>        BUG_ON(bitmap_allocate_region(vb->dirty_map, offset >>
PAGE_SHIFT,
> order));
>
> @@ -988,7 +995,6 @@ void vm_unmap_aliases(void)
>
>                                s = vb->va->va_start + (i <<
PAGE_SHIFT);
>                                e = vb->va->va_start + (j <<
PAGE_SHIFT);
> -                               vunmap_page_range(s, e);
>                                flush = 1;
>
>                                if (s < start)
>
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Dec-02 18:05 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Thu, Dec 2, 2010 at 6:41 AM, Bruce Edge <bruce.edge@gmail.com>
wrote:>
>
> On Wed, Dec 1, 2010 at 10:33 PM, Jeremy Fitzhardinge
<jeremy@goop.org>
> wrote:
>>
>> On 12/01/2010 01:32 PM, Bruce Edge wrote:
>> > I just checked the recently released 2.6.73-rc4,
>>
>> Do you mean mainline 2.6.37-rc4, or the one in xen/next-2.6.37?
>
> mainline
>
>>
>> >  and while the BUG
>> > signature is different, it still fails under NFS accesses. This is
>> > 100% recreatable.
>>
>> Please try this patch which is queued up for mainline.  It''s
already in
>> xen/next-2.6.37.
>
> After testing the mainline .37 I also tried xen/next-2.6.37 with the same
> result. At least, I still hit a BUG after accessing an NFS mount. I
didn''t
> compare the stack, but the BUG condition looked the same.
> I''ll reconfirm the location and traceback of the xen/next kernel.
>
> -Bruce
>
>
I re-ran the test with xen/next-2.6.37. The fault was a bit different this time.
The dom0 started spewing these as the domU appeared to hang:

(XEN) mm.c:2732:d8 Error while installing new baseptr 47aa8
(XEN) mm.c:2389:d8 Bad type (saw 7400000000000001 != exp
1000000000000000) for mfn 47b1b (pfn 77ef3)
(XEN) mm.c:897:d8 Attempt to create linear p.t. with write perms
(XEN) mm.c:1348:d8 Failure in alloc_l2_table: entry 18
(XEN) mm.c:2142:d8 Error while validating mfn 48359 (pfn 776b5) for
type 2000000000000000: caf=8000000000000003 taf=2000000000000001
(XEN) mm.c:939:d8 Attempt to create linear p.t. with write perms
(XEN) mm.c:1458:d8 Failure in alloc_l3_table: entry 0
(XEN) mm.c:2142:d8 Error while validating mfn 49df4 (pfn 75c1a) for
type 3000000000000000: caf=8000000000000003 taf=3000000000000001
(XEN) mm.c:964:d8 Attempt to create linear p.t. with write perms
(XEN) mm.c:1505:d8 Failure in alloc_l4_table: entry 0
(XEN) mm.c:2142:d8 Error while validating mfn 47b1a (pfn 77ef4) for
type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:3057:d8 Error while installing new mfn 47b1a
(XEN) mm.c:2389:d8 Bad type (saw 7400000000000001 != exp
1000000000000000) for mfn 47b1b (pfn 77ef3)
(XEN) mm.c:897:d8 Attempt to create linear p.t. with write perms
(XEN) mm.c:1348:d8 Failure in alloc_l2_table: entry 18
(XEN) mm.c:2142:d8 Error while validating mfn 48359 (pfn 776b5) for
type 2000000000000000: caf=8000000000000003 taf=2000000000000001
(XEN) mm.c:939:d8 Attempt to create linear p.t. with write perms
(XEN) mm.c:1458:d8 Failure in alloc_l3_table: entry 0
(XEN) mm.c:2142:d8 Error while validating mfn 49df4 (pfn 75c1a) for
type 3000000000000000: caf=8000000000000003 taf=3000000000000001
(XEN) mm.c:964:d8 Attempt to create linear p.t. with write perms
(XEN) mm.c:1505:d8 Failure in alloc_l4_table: entry 0
(XEN) mm.c:2142:d8 Error while validating mfn 47aa8 (pfn 77f66) for
type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2732:d8 Error while installing new baseptr 47aa8
(XEN) mm.c:2389:d8 Bad type (saw 7400000000000001 != exp
1000000000000000) for mfn 47b1b (pfn 77ef3)
(XEN) mm.c:897:d8 Attempt to create linear p.t. with write perms
(XEN) mm.c:1348:d8 Failure in alloc_l2_table: entry 18
(XEN) mm.c:2142:d8 Error while validating mfn 48359 (pfn 776b5) for
type 2000000000000000: caf=8000000000000003 taf=2000000000000001

Then a few seconds later there was a double fault on the domU:

[  117.035219] ------------[ cut here ]------------
[  117.035232] kernel BUG at arch/x86/xen/mmu.c:1831!
[  117.035238] invalid opcode: 0000 [#1] SMP
[  117.035248] last sysfs file: /sys/kernel/uevent_seqnum
[  117.035254] CPU 1
[  117.035260] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
xen_netfront xen_blkfront
[  117.035307]
[  117.035313] Pid: 3964, comm: cron Not tainted
2.6.37-rc4-pvops-2.6.37-next-domu-6.0.bedge-debug #1 /
[  117.035322] RIP: e030:[<ffffffff8100482d>]  [<ffffffff8100482d>]
pin_pagetable_pfn+0x31/0x37
[  117.035342] RSP: e02b:ffff8800782e9cf8  EFLAGS: 00010282
[  117.035349] RAX: 00000000ffffffea RBX: 0000000000077539 RCX: 0000000000000001
[  117.035359] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
[  117.035369] RBP: ffff8800782e9d18 R08: 00003ffffffff000 R09: ffff880000000000
[  117.035376] R10: 00000000deadbeef R11: dead000000200200 R12: 0000000000000003
[  117.035383] R13: ffff880076293380 R14: ffff88007828d398 R15: ffff88007630b330
[  117.035396] FS:  00007fd61166f7a0(0000) GS:ffff88007ff5c000(0000)
knlGS:0000000000000000
[  117.035406] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  117.035415] CR2: 00007fd60e0272f0 CR3: 0000000076350000 CR4: 0000000000002660
[  117.035424] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  117.035433] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  117.035444] Process cron (pid: 3964, threadinfo ffff8800782e8000,
task ffff88007630b330)
[  117.035453] Stack:
[  117.035459]  ffff880000000000 00000000000484d5 ffff880076280a80
0000000000077539
[  117.035480]  ffff8800782e9d38 ffffffff810066ac 0000000000077539
ffff880076280a80
[  117.035502]  ffff8800782e9d48 ffffffff810066df ffff8800782e9d88
ffffffff810e0a89
[  117.035523] Call Trace:
[  117.035534]  [<ffffffff810066ac>] xen_alloc_ptpage+0x68/0x6d
[  117.035545]  [<ffffffff810066df>] xen_alloc_pte+0xe/0x10
[  117.035559]  [<ffffffff810e0a89>] __pte_alloc+0x72/0xd7
[  117.035571]  [<ffffffff810e2dd1>] handle_mm_fault+0x12e/0x8d3
[  117.035584]  [<ffffffff810e76a9>] ? mmap_region+0x385/0x4b3
[  117.035597]  [<ffffffff8144825e>] do_page_fault+0x374/0x396
[  117.035607]  [<ffffffff8102b7be>] ? pvclock_clocksource_read+0x4b/0xb4
[  117.035617]  [<ffffffff81007c49>] ? __spin_time_accum+0x21/0x37
[  117.035626]  [<ffffffff81007ec5>] ? __xen_spin_lock+0xb7/0xcd
[  117.035637]  [<ffffffff81445415>] page_fault+0x25/0x30
[  117.035646] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 17 cb ff ff
85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74
30 48
[  117.035857] RIP  [<ffffffff8100482d>] pin_pagetable_pfn+0x31/0x37
[  117.035871]  RSP <ffff8800782e9cf8>
[  117.035879] ---[ end trace 8a5ce27e38cb6b86 ]---
[  117.039036] ------------[ cut here ]------------
[  117.039049] kernel BUG at arch/x86/xen/mmu.c:1831!
[  117.039055] invalid opcode: 0000 [#2] SMP
[  117.039055] last sysfs file: /sys/kernel/uevent_seqnum
[  117.039055] CPU 2
[  117.039055] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
xen_netfront xen_blkfront
[  117.039055]
[  117.039055] Pid: 3965, comm: cron Tainted: G      D
2.6.37-rc4-pvops-2.6.37-next-domu-6.0.bedge-debug #1 /
[  117.039055] RIP: e030:[<ffffffff8100482d>]  [<ffffffff8100482d>]
pin_pagetable_pfn+0x31/0x37
[  117.039055] RSP: e02b:ffff8800770fdcf8  EFLAGS: 00010282
[  117.039055] RAX: 00000000ffffffea RBX: 0000000000077c49 RCX: 0000000000000001
[  117.039055] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
[  117.039055] RBP: ffff8800770fdd18 R08: 00003ffffffff000 R09: ffff880000000000
[  117.039055] R10: 00000000deadbeef R11: dead000000200200 R12: 0000000000000003
[  117.039055] R13: ffff880075e9c378 R14: ffff880075ccef18 R15: ffff88007630d7c0
[  117.039055] FS:  00007fd61166f7a0(0000) GS:ffff88007ff78000(0000)
knlGS:0000000000000000
[  117.039055] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  117.039055] CR2: 00007fd60de21280 CR3: 0000000077e5c000 CR4: 0000000000002660
[  117.039055] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  117.039055] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  117.039055] Process cron (pid: 3965, threadinfo ffff8800770fc000,
task ffff88007630d7c0)
[  117.039055] Stack:
[  117.039055]  ffff880000000000 0000000000047dc5 ffff880076283b80
0000000000077c49
[  117.039055]  ffff8800770fdd38 ffffffff810066ac 0000000000077c49
ffff880076283b80
[  117.039055]  ffff8800770fdd48 ffffffff810066df ffff8800770fdd88
ffffffff810e0a89
[  117.039055] Call Trace:
[  117.039055]  [<ffffffff810066ac>] xen_alloc_ptpage+0x68/0x6d
[  117.039055]  [<ffffffff810066df>] xen_alloc_pte+0xe/0x10
[  117.039055]  [<ffffffff810e0a89>] __pte_alloc+0x72/0xd7
[  117.039055]  [<ffffffff810e2dd1>] handle_mm_fault+0x12e/0x8d3
[  117.039055]  [<ffffffff810e76a9>] ? mmap_region+0x385/0x4b3
[  117.039055]  [<ffffffff8144825e>] do_page_fault+0x374/0x396
[  117.039055]  [<ffffffff8102b7be>] ? pvclock_clocksource_read+0x4b/0xb4
[  117.039055]  [<ffffffff81007c49>] ? __spin_time_accum+0x21/0x37
[  117.039055]  [<ffffffff81007ec5>] ? __xen_spin_lock+0xb7/0xcd
[  117.039055]  [<ffffffff81445415>] page_fault+0x25/0x30
[  117.039055] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 17 cb ff ff
85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74
30 48
[  117.039055] RIP  [<ffffffff8100482d>] pin_pagetable_pfn+0x31/0x37
[  117.039055]  RSP <ffff8800770fdcf8>
[  117.039055] ---[ end trace 8a5ce27e38cb6b87 ]---


Here''s the complete boot log for the xen/next-2.6.37 domU:

[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version
2.6.37-rc4-pvops-2.6.37-next-domu-6.0.bedge-debug (root@opal) (gcc
version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Wed Dec 1 16:40:45 PST
2010
[    0.000000] Command line:
root=UUID=e971c31a-7353-4ac7-90c4-d6eb7fc33735 ro   iommu=soft
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] released 0 pages of unused memory
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000080800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[    0.000000] Scanning 0 areas for low memory corruption
[    0.000000] init_memory_mapping: 0000000000000000-0000000080800000
[    0.000000] RAMDISK: 01d96000 - 02fb0000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-0000000080800000
[    0.000000] Initmem setup node 0 0000000000000000-0000000080800000
[    0.000000]   NODE_DATA [000000007fffb000 - 000000007fffffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x000000a0
[    0.000000]     0: 0x00000100 -> 0x00080800
[    0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
[    0.000000] No local APIC present
[    0.000000] APIC: disable apic facility
[    0.000000] APIC: switched to apic NOOP
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
[    0.000000] Allocating PCI resources starting at 80800000 (gap:
80800000:7f800000)
[    0.000000] Booting paravirtualized kernel on Xen
[    0.000000] Xen version: 4.1-unstable (preserve-AD)
[    0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
nr_cpu_ids:6 nr_node_ids:1
[    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
r8192 d23744 u114688
[    0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.
Total pages: 519022
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:
root=UUID=e971c31a-7353-4ac7-90c4-d6eb7fc33735 ro   iommu=soft
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Placing 64MB software IO TLB between ffff880079400000 -
ffff88007d400000
[    0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
[    0.000000] Memory: 1959868k/2105344k available (4406k kernel code,
448k absent, 145028k reserved, 6320k data, 760k init)
[    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=6, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU-based detection of stalled CPUs is disabled.
[    0.000000] NR_IRQS:4352 nr_irqs:320 16
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [hvc0] enabled
[    0.000000] installing Xen timer for CPU 0
[    0.000000] Detected 2660.088 MHz processor.
[    0.000999] Calibrating delay loop (skipped), value calculated
using timer frequency.. 5320.17 BogoMIPS (lpj=2660088)
[    0.000999] pid_max: default: 32768 minimum: 301
[    0.000999] Security Framework initialized
[    0.000999] SELinux:  Initializing.
[    0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.001433] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.001688] Mount-cache hash table entries: 256
[    0.002227] Initializing cgroup subsys ns
[    0.002240] ns_cgroup deprecated: consider using the
''clone_children'' flag without the ns_cgroup.
[    0.002302] Initializing cgroup subsys cpuacct
[    0.002321] Initializing cgroup subsys freezer
[    0.002453] CPU: Unsupported number of siblings 16
[    0.002460] Performance Events: unsupported p6 CPU model 26 no PMU
driver, software events only.
[    0.002545] SMP alternatives: switching to UP code
[    0.003120] cpu 0 spinlock event irq 318
[    0.003689] NMI watchdog failed to create perf event on cpu0:
fffffffffffffffe
[    0.003999] installing Xen timer for CPU 1
[    0.004055] cpu 1 spinlock event irq 312
[    0.004092] SMP alternatives: switching to SMP code
[    0.000999] CPU: Unsupported number of siblings 16
[    0.005155] NMI watchdog failed to create perf event on cpu1:
fffffffffffffffe
[    0.005467] installing Xen timer for CPU 2
[    0.005570] cpu 2 spinlock event irq 306
[    0.000999] CPU: Unsupported number of siblings 16
[    0.005811] NMI watchdog failed to create perf event on cpu2:
fffffffffffffffe
[    0.005999] installing Xen timer for CPU 3
[    0.006037] cpu 3 spinlock event irq 300
[    0.000999] CPU: Unsupported number of siblings 16
[    0.006375] NMI watchdog failed to create perf event on cpu3:
fffffffffffffffe
[    0.006606] installing Xen timer for CPU 4
[    0.006707] cpu 4 spinlock event irq 294
[    0.000999] CPU: Unsupported number of siblings 16
[    0.006940] NMI watchdog failed to create perf event on cpu4:
fffffffffffffffe
[    0.007109] installing Xen timer for CPU 5
[    0.007155] cpu 5 spinlock event irq 288
[    0.000999] CPU: Unsupported number of siblings 16
[    0.007469] NMI watchdog failed to create perf event on cpu5:
fffffffffffffffe
[    0.007480] Brought up 6 CPUs
[    0.008105] kworker/u:0 used greatest stack depth: 5696 bytes left
[    0.010420] Grant table initialized
[    0.029827] Time: 165:165:165  Date: 165/165/65
[    0.030088] NET: Registered protocol family 16
[    0.034636] PCI: setting up Xen PCI frontend stub
[    0.042049] kworker/u:0 used greatest stack depth: 5528 bytes left
[    0.067412] bio: create slab <bio-0> at 0
[    0.069097] ACPI: Interpreter disabled.
[    0.070036] xen_balloon: Initialising balloon driver.
[    0.070208] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[    0.072045] vgaarb: loaded
[    0.073389] usbcore: registered new interface driver usbfs
[    0.074077] usbcore: registered new interface driver hub
[    0.074254] usbcore: registered new device driver usb
[    0.076219] PCI: System does not support PCI
[    0.076219] PCI: System does not support PCI
[    0.078257] NetLabel: Initializing
[    0.078257] NetLabel:  domain hash size = 128
[    0.078257] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.078342] NetLabel:  unlabeled traffic allowed by default
[    0.079851] Switching to clocksource xen
[    0.105131] pnp: PnP ACPI: disabled
[    0.144415] NET: Registered protocol family 2
[    0.144886] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.147612] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[    0.149020] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.149263] TCP: Hash tables configured (established 262144 bind 65536)
[    0.149279] TCP reno registered
[    0.149331] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[    0.149387] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[    0.150021] NET: Registered protocol family 1
[    0.151039] RPC: Registered udp transport module.
[    0.151068] RPC: Registered tcp transport module.
[    0.151080] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.151537] Trying to unpack rootfs image as initramfs...
[    0.195821] Freeing initrd memory: 18536k freed
[    0.286598] DMA-API: preallocated 32768 debug entries
[    0.286610] DMA-API: debugging enabled by kernel config
[    0.288249] platform rtc_cmos: registered platform RTC device (no
PNP device found)
[    0.300038] Machine check injector initialized
[    0.308999] microcode: no support for this CPU vendor
[    0.309013] Scanning for low memory corruption every 60 seconds
[    0.312235] audit: initializing netlink socket (disabled)
[    0.312318] type=2000 audit(1291310104.544:1): initialized
[    0.351935] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.394898] VFS: Disk quotas dquot_6.5.2
[    0.395664] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.400904] NTFS driver 2.1.29 [Flags: R/W].
[    0.402874] msgmni has been set to 3864
[    0.408008] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[    0.408026] io scheduler noop registered
[    0.408035] io scheduler deadline registered
[    0.408728] io scheduler cfq registered (default)
[    0.410538] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.425531] pcifront pci-0: Installing PCI frontend
[    0.425977] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[    0.443708] pcifront pci-0: claiming resource 0000:00:00.0/0
[    0.443708] pcifront pci-0: claiming resource 0000:00:00.0/2
[    0.443708] pcifront pci-0: claiming resource 0000:00:00.0/3
[    0.443708] pcifront pci-0: claiming resource 0000:00:00.0/4
[    0.443708] pcifront pci-0: claiming resource 0000:00:01.0/0
[    0.443708] pcifront pci-0: claiming resource 0000:00:01.0/2
[    0.443708] pcifront pci-0: claiming resource 0000:00:01.0/3
[    0.443708] pcifront pci-0: claiming resource 0000:00:01.0/4
[    0.443708] pcifront pci-0: claiming resource 0000:00:02.0/0
[    0.443708] pcifront pci-0: claiming resource 0000:00:02.0/2
[    0.443708] pcifront pci-0: claiming resource 0000:00:02.0/3
[    0.443708] pcifront pci-0: claiming resource 0000:00:02.0/4
[    0.443708] pcifront pci-0: claiming resource 0000:00:03.0/0
[    0.443708] pcifront pci-0: claiming resource 0000:00:03.0/2
[    0.443708] pcifront pci-0: claiming resource 0000:00:03.0/3
[    0.443708] pcifront pci-0: claiming resource 0000:00:03.0/4
[    0.728914] Non-volatile memory driver v1.3
[    0.728929] Linux agpgart interface v0.103
[    0.730416] [drm] Initialized drm 1.1.0 20060810
[    0.730437] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.833097] brd: module loaded
[    0.845967] loop: module loaded
[    0.847158] Fixed MDIO Bus: probed
[    0.849136] ehci_hcd: USB 2.0 ''Enhanced'' Host Controller
(EHCI) Driver
[    0.849688] ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI)
Driver
[    0.850172] uhci_hcd: USB Universal Host Controller Interface driver
[    0.851870] usbcore: registered new interface driver usblp
[    0.852383] usbcore: registered new interface driver libusual
[    0.854370] PNP: No PS/2 controller found. Probing ports directly.
[    0.855219] i8042.c: No controller found.
[    0.855790] mice: PS/2 mouse device common for all mice
[    0.858311] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[    0.858934] rtc_cmos: probe of rtc_cmos failed with error -38
[    0.860890] cpuidle: using governor ladder
[    0.860920] cpuidle: using governor menu
[    0.861007] Netfilter messages via NETLINK v0.30.
[    0.861116] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    0.862960] ctnetlink v0.93: registering with nfnetlink.
[    0.865256] ip_tables: (C) 2000-2006 Netfilter Core Team
[    0.865645] TCP cubic registered
[    0.865655] Initializing XFRM netlink socket
[    0.868322] NET: Registered protocol family 10
[    0.872388] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    0.872760] IPv6 over IPv4 tunneling driver
[    0.876502] NET: Registered protocol family 17
[    0.876723] Registering the dns_resolver key type
[    0.878973] registered taskstats version 1
[    0.879259] kmemleak: Kernel memory leak detector initialized
[    0.879313] kmemleak: Automatic memory scanning thread started
[    0.880238] XENBUS: Device with no driver: device/console/0
[    0.880238] XENBUS: Device with no driver: device/vbd/51712
[    0.880238] XENBUS: Device with no driver: device/vif/0
[    0.880238]   Magic number: 1:252:3141
[   13.395048] Freeing unused kernel memory: 760k freed
[   13.395116] Write protecting the kernel read-only data: 10240k
[   13.404961] Freeing unused kernel memory: 1716k freed
[   13.406216] Freeing unused kernel memory: 1988k freed
[   13.441682] exe used greatest stack depth: 5488 bytes left
Loading, please wait...
mount: mounting none on /dev failed: No such device
[   13.528124] udevd (1109): /proc/1109/oom_adj is deprecated, please
use /proc/1109/oom_score_adj instead.
Begin: Loading essential drivers... ...
[   13.559533] blkid used greatest stack depth: 5104 bytes left
[   13.588027] blkfront: xvda: barriers enabled
[   13.600991]  xvda: xvda1 xvda2 < xvda5 >
[   13.664386] Initialising Xen virtual ethernet driver.
[   13.671581] blkid used greatest stack depth: 4752 bytes left
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
[   13.931586] EXT4-fs (xvda1): mounted filesystem with ordered data
mode. Opts: (null)
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
[   14.754489] hwclock used greatest stack depth: 4608 bytes left
init: ureadahead main process (1209) terminated with status 5
[   14.854024] plymouthd used greatest stack depth: 4320 bytes left
[   16.117059] chown used greatest stack depth: 3648 bytes left
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
/dev/xvda1: clean, 57230/288576 files, 328709/1153280 blocks
 * Enable login on console                                               [ OK ]
 * Setting sensors limits
         No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
                                                                         [ OK ]
 * Initializing random number generator...                               [ OK ]
 * Setting up X server socket directory /tmp/.X11-unix...                [ OK ]
 * Setting up ICE socket directory /tmp/.ICE-unix...                     [ OK ]
 * Not starting fancontrol; run pwmconfig first.
 * Starting internet superserver inetd                                   [ OK ]
 * Starting the system activity data collector sadc
 * Restoring resolver state...                                           [ OK ]
 * Starting Tomcat servlet engine tomcat6
         Using CATALINA_BASE:   /var/lib/tomcat6
Using CATALINA_HOME:   /usr/share/tomcat6
Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
Using JRE_HOME:        /usr/lib/jvm/java-6-openjdk
Using CLASSPATH:       /usr/share/tomcat6/bin/bootstrap.jar
                                                                         [ OK ]
 * Recording successful boot for GRUB
   ...done.
 * Running local boot scripts (/etc/rc.local)                            [ OK ]
mountall: Disconnected from Plymouth

6.0.16-01
uss004-dpm login:


I am going to try migrate away from NFS for dom0 <-> domU file sharing
because this is preventing us form moving to a mainline kernel.
There was one recommendation for glusterfs, is that the preferred Xen
VM file sharing option or is there something else?

Note that mounting the same dir over sshfs does not have this problem
with either .37 kernel.

-Bruce

>>
>>    J
>>
>>
>> diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
>> index 21ed8d7..0e4ecac 100644
>> --- a/arch/x86/xen/mmu.c
>> +++ b/arch/x86/xen/mmu.c
>> @@ -2358,8 +2358,6 @@ void __init xen_init_mmu_ops(void)
>>        x86_init.paging.pagetable_setup_done = xen_pagetable_setup_done;
>>        pv_mmu_ops = xen_mmu_ops;
>>
>> -       vmap_lazy_unmap = false;
>> -
>>        memset(dummy_mapping, 0xff, PAGE_SIZE);
>>  }
>>
>> diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
>> index a03dcf6..44b54f6 100644
>> --- a/include/linux/vmalloc.h
>> +++ b/include/linux/vmalloc.h
>> @@ -7,8 +7,6 @@
>>
>>  struct vm_area_struct;         /* vma defining user mapping in
mm_types.h
>> */
>>
>> -extern bool vmap_lazy_unmap;
>> -
>>  /* bits in flags of vmalloc''s vm_struct below */
>>  #define VM_IOREMAP     0x00000001      /* ioremap() and friends */
>>  #define VM_ALLOC       0x00000002      /* vmalloc() */
>> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
>> index a3d66b3..eb5cc7d 100644
>> --- a/mm/vmalloc.c
>> +++ b/mm/vmalloc.c
>> @@ -31,8 +31,6 @@
>>  #include <asm/tlbflush.h>
>>  #include <asm/shmparam.h>
>>
>> -bool vmap_lazy_unmap __read_mostly = true;
>> -
>>  /*** Page table manipulation functions ***/
>>
>>  static void vunmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned
>> long end)
>> @@ -503,9 +501,6 @@ static unsigned long lazy_max_pages(void)
>>  {
>>        unsigned int log;
>>
>> -       if (!vmap_lazy_unmap)
>> -               return 0;
>> -
>>        log = fls(num_online_cpus());
>>
>>        return log * (32UL * 1024 * 1024 / PAGE_SIZE);
>> @@ -566,7 +561,6 @@ static void __purge_vmap_area_lazy(unsigned long
>> *start, unsigned long *end,
>>                        if (va->va_end > *end)
>>                                *end = va->va_end;
>>                        nr += (va->va_end - va->va_start) >>
PAGE_SHIFT;
>> -                       unmap_vmap_area(va);
>>                        list_add_tail(&va->purge_list,
&valist);
>>                        va->flags |= VM_LAZY_FREEING;
>>                        va->flags &= ~VM_LAZY_FREE;
>> @@ -611,10 +605,11 @@ static void purge_vmap_area_lazy(void)
>>  }
>>
>>  /*
>> - * Free and unmap a vmap area, caller ensuring flush_cache_vunmap had
>> been
>> - * called for the correct range previously.
>> + * Free a vmap area, caller ensuring that the area has been unmapped
>> + * and flush_cache_vunmap had been called for the correct range
>> + * previously.
>>  */
>> -static void free_unmap_vmap_area_noflush(struct vmap_area *va)
>> +static void free_vmap_area_noflush(struct vmap_area *va)
>>  {
>>        va->flags |= VM_LAZY_FREE;
>>        atomic_add((va->va_end - va->va_start) >>
PAGE_SHIFT,
>> &vmap_lazy_nr);
>> @@ -623,6 +618,16 @@ static void free_unmap_vmap_area_noflush(struct
>> vmap_area *va)
>>  }
>>
>>  /*
>> + * Free and unmap a vmap area, caller ensuring flush_cache_vunmap had
>> been
>> + * called for the correct range previously.
>> + */
>> +static void free_unmap_vmap_area_noflush(struct vmap_area *va)
>> +{
>> +       unmap_vmap_area(va);
>> +       free_vmap_area_noflush(va);
>> +}
>> +
>> +/*
>>  * Free and unmap a vmap area
>>  */
>>  static void free_unmap_vmap_area(struct vmap_area *va)
>> @@ -798,7 +803,7 @@ static void free_vmap_block(struct vmap_block *vb)
>>        spin_unlock(&vmap_block_tree_lock);
>>        BUG_ON(tmp != vb);
>>
>> -       free_unmap_vmap_area_noflush(vb->va);
>> +       free_vmap_area_noflush(vb->va);
>>        call_rcu(&vb->rcu_head, rcu_free_vb);
>>  }
>>
>> @@ -936,6 +941,8 @@ static void vb_free(const void *addr, unsigned long
>> size)
>>        rcu_read_unlock();
>>        BUG_ON(!vb);
>>
>> +       vunmap_page_range((unsigned long)addr, (unsigned long)addr +
>> size);
>> +
>>        spin_lock(&vb->lock);
>>        BUG_ON(bitmap_allocate_region(vb->dirty_map, offset >>
PAGE_SHIFT,
>> order));
>>
>> @@ -988,7 +995,6 @@ void vm_unmap_aliases(void)
>>
>>                                s = vb->va->va_start + (i <<
PAGE_SHIFT);
>>                                e = vb->va->va_start + (j <<
PAGE_SHIFT);
>> -                               vunmap_page_range(s, e);
>>                                flush = 1;
>>
>>                                if (s < start)
>>
>>
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2010-Dec-02 18:28 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On 12/02/2010 10:05 AM, Bruce Edge wrote:> On Thu, Dec 2, 2010 at 6:41 AM, Bruce Edge <bruce.edge@gmail.com>
wrote:
>>
>> On Wed, Dec 1, 2010 at 10:33 PM, Jeremy Fitzhardinge
<jeremy@goop.org>
>> wrote:
>>> On 12/01/2010 01:32 PM, Bruce Edge wrote:
>>>> I just checked the recently released 2.6.73-rc4,
>>> Do you mean mainline 2.6.37-rc4, or the one in xen/next-2.6.37?
>> mainline
>>
>>>>  and while the BUG
>>>> signature is different, it still fails under NFS accesses. This
is
>>>> 100% recreatable.
>>> Please try this patch which is queued up for mainline. 
It''s already in
>>> xen/next-2.6.37.
>> After testing the mainline .37 I also tried xen/next-2.6.37 with the
same
>> result. At least, I still hit a BUG after accessing an NFS mount. I
didn''t
>> compare the stack, but the BUG condition looked the same.
>> I''ll reconfirm the location and traceback of the xen/next
kernel.
>>
>> -Bruce
>>
>>
> I re-ran the test with xen/next-2.6.37. The fault was a bit different this
time.
> The dom0 started spewing these as the domU appeared to hang:
OK, these look like exactly the sort of symptoms which should be fixed
by that patch, so assuming you''re actually running the right kernel
(worth checking!  I get bitten by this occasionally), there''s something
else going on here.

What NFS version are you using, and other parameters?  What is the NFS
server?  What''s the file access pattern going on?

Thanks,
    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Dec-02 18:38 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Thu, Dec 2, 2010 at 10:28 AM, Jeremy Fitzhardinge <jeremy@goop.org>
wrote:> On 12/02/2010 10:05 AM, Bruce Edge wrote:
>> On Thu, Dec 2, 2010 at 6:41 AM, Bruce Edge <bruce.edge@gmail.com>
wrote:
>>>
>>> On Wed, Dec 1, 2010 at 10:33 PM, Jeremy Fitzhardinge
<jeremy@goop.org>
>>> wrote:
>>>> On 12/01/2010 01:32 PM, Bruce Edge wrote:
>>>>> I just checked the recently released 2.6.73-rc4,
>>>> Do you mean mainline 2.6.37-rc4, or the one in xen/next-2.6.37?
>>> mainline
>>>
>>>>>  and while the BUG
>>>>> signature is different, it still fails under NFS accesses.
This is
>>>>> 100% recreatable.
>>>> Please try this patch which is queued up for mainline.
 It''s already in
>>>> xen/next-2.6.37.
>>> After testing the mainline .37 I also tried xen/next-2.6.37 with
the same
>>> result. At least, I still hit a BUG after accessing an NFS mount. I
didn''t
>>> compare the stack, but the BUG condition looked the same.
>>> I''ll reconfirm the location and traceback of the xen/next
kernel.
>>>
>>> -Bruce
>>>
>>>
>> I re-ran the test with xen/next-2.6.37. The fault was a bit different
this time.
>> The dom0 started spewing these as the domU appeared to hang:
>
> OK, these look like exactly the sort of symptoms which should be fixed
> by that patch, so assuming you''re actually running the right
kernel
> (worth checking!  I get bitten by this occasionally), there''s
something
> else going on here.
You''re right, my branch was a day out of date. I didn''t have
the last
changes to:

 arch/x86/xen/grant-table.c |   28 +++++---
 arch/x86/xen/mmu.c         |  124 ++++++++++--------------------
 include/linux/mm.h         |    6 ++
 include/linux/vmalloc.h    |    2 -
 mm/memory.c                |   56 ++++++++++----
 mm/vmalloc.c               |  182 ++++++++++++++------------------------------

rebuilding with great expectations now :-)

-Bruce
>
> What NFS version are you using, and other parameters?  What is the NFS
> server?  What''s the file access pattern going on?
>
> Thanks,
>    J
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2010-Dec-02 18:50 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On 12/02/2010 10:38 AM, Bruce Edge wrote:>> OK, these look like exactly the sort of symptoms which should be fixed
>> by that patch, so assuming you''re actually running the right
kernel
>> (worth checking!  I get bitten by this occasionally), there''s
something
>> else going on here.
> You''re right, my branch was a day out of date. I didn''t
have the last
> changes to:
>
>  arch/x86/xen/grant-table.c |   28 +++++---
>  arch/x86/xen/mmu.c         |  124 ++++++++++--------------------
>  include/linux/mm.h         |    6 ++
>  include/linux/vmalloc.h    |    2 -
>  mm/memory.c                |   56 ++++++++++----
>  mm/vmalloc.c               |  182
++++++++++++++------------------------------
Those are the ones.
> rebuilding with great expectations now :-)
I''m doing a kernel build over NFS at the moment.  (Well, still copying
sources.)

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bruce Edge

2010-Dec-02 19:34 UTC

head link

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

On Thu, Dec 2, 2010 at 10:50 AM, Jeremy Fitzhardinge <jeremy@goop.org>
wrote:> On 12/02/2010 10:38 AM, Bruce Edge wrote:
>>> OK, these look like exactly the sort of symptoms which should be
fixed
>>> by that patch, so assuming you''re actually running the
right kernel
>>> (worth checking!  I get bitten by this occasionally),
there''s something
>>> else going on here.
>> You''re right, my branch was a day out of date. I
didn''t have the last
>> changes to:
>>
>>  arch/x86/xen/grant-table.c |   28 +++++---
>>  arch/x86/xen/mmu.c         |  124 ++++++++++--------------------
>>  include/linux/mm.h         |    6 ++
>>  include/linux/vmalloc.h    |    2 -
>>  mm/memory.c                |   56 ++++++++++----
>>  mm/vmalloc.c               |  182
++++++++++++++------------------------------
>
> Those are the ones.
>
>> rebuilding with great expectations now :-)
>
> I''m doing a kernel build over NFS at the moment.  (Well, still
copying
> sources.)
Fixed!  Nice work.

This unblocks a large chunk of my work. Thank you.

This thread can happily die now.

-Bruce
>
>    J
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Nov 2010 - 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

[Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

[Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

[Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

[Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

[Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

[Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request, nfsd changes for 2.6.37

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request as Dom0 kernel

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request

Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request