Is there any common factor across all the boxes you''ve reproduced this
issue
on? You''ve reproduced it across a lot of versions of Xen, run and
tested by
a lot of other people who have never once reported similar symptoms. This
would indicate that there is some other factor -- a particular I/O device,
or the dom0 kernel that you run, for example -- that is causing you alone to
experience this issue. The crash here could be dug into a bit but it''s
pretty likely that something has dumped in one of this guest''s
pagetable
pages. Hence when we try to find the page_info structure corresponding to
the page mapped by that pagetable entry, we index off the end of Xen''s
address space and crash. This value can''t have come from the usual
pagetable
write interfaces (because we apply checks there). I assume the value has
come from either shadow code (just because I know it less well and it''s
much
more complicated than paravirtual guest pagetable management) or from
misdirected device DMA (I would vote this as most likely, and woul dbe due
to buggy hardware, buggy firmware, or buggy dom0 kernel device driver).
-- Keir
On 27/8/08 15:42, "Christopher S. Aker" <caker@theshore.net>
wrote:
> I was hoping changes made in 3.3 inadvertently fixed our
> put_page_type/domain_relinquish_resources hypervisor crashes, but we
> just had a box crash with our old friend.
>
> This bug is getting really annoying. What can we do to help debug this
> problem?
>
> (XEN) Pagetable walk from 00000000dfc27ffc:
> (XEN) L4[0x000] = 00000005ec8a6027 0000000000020839
> (XEN) L3[0x003] = 00000005eca56027 0000000000020689
> (XEN) L2[0x0fe] = 00000006168e5067 00000000000000fa
> (XEN) L1[0x027] = 80000005ed4b8061 000000000001fc27
> (XEN) domain_crash_sync called from entry.S
> (XEN) Domain 4 (vcpu#0) crashed on cpu#0:
> (XEN) ----[ Xen-3.3.0-rc4-pre x86_64 debug=n Not tainted ]----
> (XEN) CPU: 0
> (XEN) RIP: 0061:[<00000000c048e84c>]
> (XEN) RFLAGS: 0000000000210246 EM: 1 CONTEXT: pv guest
> (XEN) rax: 0000000000000000 rbx: 00000000dfc2fb6c rcx: 0000000000000001
> (XEN) rdx: 0000000000000000 rsi: 0000000000007ff0 rdi: 00000000e17ffc80
> (XEN) rbp: 00000000dfc2fb9c rsp: 00000000dfc2800c r8: 0000000000000000
> (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000
> (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000
> (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026b0
> (XEN) cr3: 00000006091f1000 cr2: 00000000c048e84c
> (XEN) ds: 007b es: 007b fs: 0000 gs: 0000 ss: 0069 cs: 0061
> (XEN) Guest stack trace from esp=dfc2800c:
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) 00000010 c048e84c 00010061 00210046 00000010 c048e84c 00010061
> 00210046
> (XEN) ----[ Xen-3.3.0-rc4-pre x86_64 debug=n Not tainted ]----
> (XEN) CPU: 7
> (XEN) RIP: e008:[<ffff828c801417b5>] put_page_type+0x15/0xf0
> (XEN) RFLAGS: 0000000000210296 CONTEXT: hypervisor
> (XEN) rax: 00001ae8e0eae4e0 rbx: ffffa42719259e18 rcx: 0000000080000000
> (XEN) rdx: 000000d747075727 rsi: 00000000005eca56 rdi: ffffa42719259e18
> (XEN) rbp: 0000000000000001 rsp: ffff8300cee1fc28 r8: 0000000000000000
> (XEN) r9: 0000000000305000 r10: 0000000000000000 r11: 0000000000000000
> (XEN) r12: ffffa42719259e18 r13: 00000000005eca56 r14: ffff82840ecf9d70
> (XEN) r15: ffff8300ceffc098 cr0: 000000008005003b cr4: 00000000000026b0
> (XEN) cr3: 000000062fdea000 cr2: ffffa42719259e30
> (XEN) ds: 007b es: 007b fs: 0000 gs: 0033 ss: 0000 cs: e008
> (XEN) Xen stack trace from rsp=ffff8300cee1fc28:
> (XEN) ffffa42719259e18 0000000000000001 ffff8300ceffc080
> ffff828c80141341
> (XEN) ffff8305eca56008 ffff828c801415cb 0000000044000001
> 0000000044000000
> (XEN) ffff82840ecf9d70 ffff8300ceffc080 ffff82840ecf59f0
> ffff828c80141817
> (XEN) ffff82840ecf9d70 00000000000001fd 00000000005ec8a6
> ffff828c80141b31
> (XEN) ffff8305ec8a6018 ffff828c8014165e 0000000060000001
> 0000000060000000
> (XEN) ffff82840ecf59f0 ffff8300ceffc080 0000000080000000
> ffff828c80141817
> (XEN) ffff82840ecf59f0 ffff82840ecf59f0 ffff8300ceffd930
> ffff828c80135d64
> (XEN) ffff82840f3b4a98 ffff8300ceffc098 0000000000000000
> ffff8300ceffc080
> (XEN) ffff8300cee1fda8 0000000000000000 0000000000000000
> ffff828c80136226
> (XEN) ffff8300ceffc080 0000000000000000 0000000000000000
> ffff828c80105acd
> (XEN) ffff8300cf026080 ffff8300cfdec080 ffff828c801229de
> fffffffffffffff3
> (XEN) 00000000b4cfc508 ffff828c801045aa 0000000000000007
> ffff8300cee1fde8
> (XEN) 0000000500000002 083ae6d4b7b10004 0837dfc401cfc538
> b7dd4c50b7b17604
> (XEN) b7a62d7400000001 00000001b7a69d84 080facafb4cfc568
> 081361e0081cf598
> (XEN) 080797e7b5e8bb54 00000000b5e8bb70 b7dcf02c00000060
> b5e9716c00000000
> (XEN) 0805946cb4cfc588 b7dcf02cb5e9716c b7dd26a000000000
> 00000002b3cbdd9c
> (XEN) a5dba1eea5dba1ee 0000001f00000000 ffff828c8020d100
> ffff828c8011838b
> (XEN) 0000000000000000 0000000000000007 ffff8300cf0260b0
> ffff828c801ff200
> (XEN) 00043c56ecbcaa88 0000000000200296 0000000000000000
> 0000000000200296
> (XEN) ffff8300cf0260b0 ffff8300cfd92080 ffff828c8020d200
> ffff828c8012f0d2
> (XEN) Xen call trace:
> (XEN) [<ffff828c801417b5>] put_page_type+0x15/0xf0
> (XEN) [<ffff828c80141341>] put_page_from_l2e+0x31/0x50
> (XEN) [<ffff828c801415cb>] free_page_type+0x26b/0x440
> (XEN) [<ffff828c80141817>] put_page_type+0x77/0xf0
> (XEN) [<ffff828c80141b31>] put_page_from_l3e+0x31/0x50
> (XEN) [<ffff828c8014165e>] free_page_type+0x2fe/0x440
> (XEN) [<ffff828c80141817>] put_page_type+0x77/0xf0
> (XEN) [<ffff828c80135d64>] relinquish_memory+0xb4/0x2a0
> (XEN) [<ffff828c80136226>] domain_relinquish_resources+0x116/0x1f0
> (XEN) [<ffff828c80105acd>] domain_kill+0x7d/0x160
> (XEN) [<ffff828c801229de>] ns16550_tx_empty+0xe/0x20
> (XEN) [<ffff828c801045aa>] do_domctl+0x60a/0xc90
> (XEN) [<ffff828c8011838b>] add_entry+0xdb/0x110
> (XEN) [<ffff828c8012f0d2>] reprogram_timer+0x22/0x90
> (XEN) [<ffff828c801184b5>] timer_softirq_action+0xf5/0x130
> (XEN) [<ffff828c801b5363>] compat_hypercall+0x83/0x90
> (XEN)
> (XEN) Pagetable walk from ffffa42719259e30:
> (XEN) L4[0x148] = 0000000000000000 ffffffffffffffff
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 7:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=0000]
> (XEN) Faulting linear address: ffffa42719259e30
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
>
> -Chris
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel