When I ran "xentrace -D -S 256 -e all /tmp/test.trace" from the xenanalyze documentation, Xen immediately crashed with: (XEN) tbuf_size 256 (XEN) p0 mfn 106a00 offset 64 (XEN) p1 mfn 115700 offset 320 (XEN) p2 mfn 113f00 offset 576 (XEN) p3 mfn 113e00 offset 832 (XEN) Xen trace buffers: initialized (XEN) ----[ Xen-4.1-unstable x86_64 debug=y Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: e008:[<ffff82c4801215b3>] check_lock+0x1b/0x45 (XEN) RFLAGS: 0000000000010246 CONTEXT: hypervisor (XEN) rax: 0000000000000001 rbx: 0000000000000028 rcx: 0000000000000000 (XEN) rdx: 0000000000000000 rsi: 00000000c3fc8050 rdi: 000000000000002c (XEN) rbp: ffff83013ff2fc60 rsp: ffff83013ff2fc60 r8: ffff8300bf78a000 (XEN) r9: ffff83013ff60000 r10: 0080000000000001 r11: ffff82f60164b930 (XEN) r12: 00000000c3fc8050 r13: 00000000c3fc8050 r14: 0000000000000028 (XEN) r15: 0000000000800627 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 00000000b2fa0000 cr2: 000000000000002c (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 (XEN) Xen stack trace from rsp=ffff83013ff2fc60: (XEN) ffff83013ff2fc78 ffff82c480121993 0000000000000000 ffff83013ff2fca8 (XEN) ffff82c480117fe0 ffff83013fee0000 80000c3fc8050627 00000000c3fc8050 (XEN) ffff83013ff60000 ffff83013ff2fd28 ffff82c48016538c ffff8800aec97b08 (XEN) ffff83013ff2fcf0 00000000000b25c9 0000000000000100 ffff8300bf78a000 (XEN) 0000000000000206 0000000000000000 ffff830e7f900a00 ffff83013ff2fd28 (XEN) ffff8300b25c9228 ffff8300bf78a000 0000000000800627 0000000000000000 (XEN) 0000000000000000 ffff83013ff2fdb8 ffff82c48016a9f7 ffff83013fee0018 (XEN) 00007ff03ff2ff28 0000000000000000 00000000000b25c9 ffff83013fee0000 (XEN) 80000c3fc8050627 80000c3fc8050627 ffff83013ff60000 ffff83013ff2fdb8 (XEN) 0000000180162c10 ffff83013ff2fdd8 0000000000000000 0000000000000000 (XEN) ffff8800aec97bb8 ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff08 (XEN) ffff82c48016b2a9 0000000000000000 0000000000000000 0000000000000000 (XEN) 00000000bf78a000 0000000000000006 00000000b2e55067 ffff83013ff2ff28 (XEN) ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 (XEN) ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 0000000000000000 (XEN) 00007ff200000082 ffff83013ff2fe68 80000c3fc8050627 ffff8300bf78a000 (XEN) ffff8300b25c9228 ffff83013ff2fee8 ffff82f60164b920 ffff83013fee0000 (XEN) 000000000003e6b8 00000000000b25c9 00000001032cc060 ffff83013ff60000 (XEN) 0000000000000000 aaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaa 00000001aaaaaaaa (XEN) 0000000000000001 00000000b25c9228 80000c3fc8050627 ffff8800aec97bb8 (XEN) Xen call trace: (XEN) [<ffff82c4801215b3>] check_lock+0x1b/0x45 (XEN) [<ffff82c480121993>] _spin_lock+0x11/0x3f (XEN) [<ffff82c480117fe0>] rangeset_contains_range+0x44/0x82 (XEN) [<ffff82c48016538c>] get_page_from_l1e+0x24c/0x47f (XEN) [<ffff82c48016a9f7>] mod_l1_entry+0x47f/0x64e (XEN) [<ffff82c48016b2a9>] do_mmu_update+0x6e3/0x1962 (XEN) [<ffff82c4801f71bf>] syscall_enter+0xef/0x149 (XEN) (XEN) Pagetable walk from 000000000000002c: (XEN) L4[0x000] = 00000000b2c3a067 00000000000aecc5 (XEN) L3[0x000] = 00000000b2c70067 00000000000aec8f (XEN) L2[0x000] = 0000000000000000 ffffffffffffffff (XEN) (XEN) **************************************** (XEN) Panic on CPU 1: (XEN) FATAL PAGE FAULT (XEN) [error_code=0000] (XEN) Faulting linear address: 000000000000002c (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
New xentrace patches have gone in without being tested with a debug=y build, I expect. This is probably an irq-safe lock being taken with irqs enabled, or vice versa. -- Keir On 30/04/2010 14:34, "Jeremy Fitzhardinge" <jeremy@goop.org> wrote:> When I ran "xentrace -D -S 256 -e all /tmp/test.trace" from the > xenanalyze documentation, Xen immediately crashed with: > > (XEN) tbuf_size 256 > (XEN) p0 mfn 106a00 offset 64 > (XEN) p1 mfn 115700 offset 320 > (XEN) p2 mfn 113f00 offset 576 > (XEN) p3 mfn 113e00 offset 832 > (XEN) Xen trace buffers: initialized > (XEN) ----[ Xen-4.1-unstable x86_64 debug=y Not tainted ]---- > (XEN) CPU: 1 > (XEN) RIP: e008:[<ffff82c4801215b3>] check_lock+0x1b/0x45 > (XEN) RFLAGS: 0000000000010246 CONTEXT: hypervisor > (XEN) rax: 0000000000000001 rbx: 0000000000000028 rcx: 0000000000000000 > (XEN) rdx: 0000000000000000 rsi: 00000000c3fc8050 rdi: 000000000000002c > (XEN) rbp: ffff83013ff2fc60 rsp: ffff83013ff2fc60 r8: ffff8300bf78a000 > (XEN) r9: ffff83013ff60000 r10: 0080000000000001 r11: ffff82f60164b930 > (XEN) r12: 00000000c3fc8050 r13: 00000000c3fc8050 r14: 0000000000000028 > (XEN) r15: 0000000000800627 cr0: 000000008005003b cr4: 00000000000026f0 > (XEN) cr3: 00000000b2fa0000 cr2: 000000000000002c > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 > (XEN) Xen stack trace from rsp=ffff83013ff2fc60: > (XEN) ffff83013ff2fc78 ffff82c480121993 0000000000000000 ffff83013ff2fca8 > (XEN) ffff82c480117fe0 ffff83013fee0000 80000c3fc8050627 00000000c3fc8050 > (XEN) ffff83013ff60000 ffff83013ff2fd28 ffff82c48016538c ffff8800aec97b08 > (XEN) ffff83013ff2fcf0 00000000000b25c9 0000000000000100 ffff8300bf78a000 > (XEN) 0000000000000206 0000000000000000 ffff830e7f900a00 ffff83013ff2fd28 > (XEN) ffff8300b25c9228 ffff8300bf78a000 0000000000800627 0000000000000000 > (XEN) 0000000000000000 ffff83013ff2fdb8 ffff82c48016a9f7 ffff83013fee0018 > (XEN) 00007ff03ff2ff28 0000000000000000 00000000000b25c9 ffff83013fee0000 > (XEN) 80000c3fc8050627 80000c3fc8050627 ffff83013ff60000 ffff83013ff2fdb8 > (XEN) 0000000180162c10 ffff83013ff2fdd8 0000000000000000 0000000000000000 > (XEN) ffff8800aec97bb8 ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff08 > (XEN) ffff82c48016b2a9 0000000000000000 0000000000000000 0000000000000000 > (XEN) 00000000bf78a000 0000000000000006 00000000b2e55067 ffff83013ff2ff28 > (XEN) ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 > (XEN) ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 0000000000000000 > (XEN) 00007ff200000082 ffff83013ff2fe68 80000c3fc8050627 ffff8300bf78a000 > (XEN) ffff8300b25c9228 ffff83013ff2fee8 ffff82f60164b920 ffff83013fee0000 > (XEN) 000000000003e6b8 00000000000b25c9 00000001032cc060 ffff83013ff60000 > (XEN) 0000000000000000 aaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaa 00000001aaaaaaaa > (XEN) 0000000000000001 00000000b25c9228 80000c3fc8050627 ffff8800aec97bb8 > (XEN) Xen call trace: > (XEN) [<ffff82c4801215b3>] check_lock+0x1b/0x45 > (XEN) [<ffff82c480121993>] _spin_lock+0x11/0x3f > (XEN) [<ffff82c480117fe0>] rangeset_contains_range+0x44/0x82 > (XEN) [<ffff82c48016538c>] get_page_from_l1e+0x24c/0x47f > (XEN) [<ffff82c48016a9f7>] mod_l1_entry+0x47f/0x64e > (XEN) [<ffff82c48016b2a9>] do_mmu_update+0x6e3/0x1962 > (XEN) [<ffff82c4801f71bf>] syscall_enter+0xef/0x149 > (XEN) > (XEN) Pagetable walk from 000000000000002c: > (XEN) L4[0x000] = 00000000b2c3a067 00000000000aecc5 > (XEN) L3[0x000] = 00000000b2c70067 00000000000aec8f > (XEN) L2[0x000] = 0000000000000000 ffffffffffffffff > (XEN) > (XEN) **************************************** > (XEN) Panic on CPU 1: > (XEN) FATAL PAGE FAULT > (XEN) [error_code=0000] > (XEN) Faulting linear address: 000000000000002c > (XEN) **************************************** > (XEN) > (XEN) Reboot in five seconds... > > J >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
FYI, this is on my radar, but it may be a few days before I get to it. -George On Fri, Apr 30, 2010 at 2:43 PM, Keir Fraser <keir.fraser@eu.citrix.com> wrote:> New xentrace patches have gone in without being tested with a debug=y build, > I expect. This is probably an irq-safe lock being taken with irqs enabled, > or vice versa. > > -- Keir > > On 30/04/2010 14:34, "Jeremy Fitzhardinge" <jeremy@goop.org> wrote: > >> When I ran "xentrace -D -S 256 -e all /tmp/test.trace" from the >> xenanalyze documentation, Xen immediately crashed with: >> >> (XEN) tbuf_size 256 >> (XEN) p0 mfn 106a00 offset 64 >> (XEN) p1 mfn 115700 offset 320 >> (XEN) p2 mfn 113f00 offset 576 >> (XEN) p3 mfn 113e00 offset 832 >> (XEN) Xen trace buffers: initialized >> (XEN) ----[ Xen-4.1-unstable x86_64 debug=y Not tainted ]---- >> (XEN) CPU: 1 >> (XEN) RIP: e008:[<ffff82c4801215b3>] check_lock+0x1b/0x45 >> (XEN) RFLAGS: 0000000000010246 CONTEXT: hypervisor >> (XEN) rax: 0000000000000001 rbx: 0000000000000028 rcx: 0000000000000000 >> (XEN) rdx: 0000000000000000 rsi: 00000000c3fc8050 rdi: 000000000000002c >> (XEN) rbp: ffff83013ff2fc60 rsp: ffff83013ff2fc60 r8: ffff8300bf78a000 >> (XEN) r9: ffff83013ff60000 r10: 0080000000000001 r11: ffff82f60164b930 >> (XEN) r12: 00000000c3fc8050 r13: 00000000c3fc8050 r14: 0000000000000028 >> (XEN) r15: 0000000000800627 cr0: 000000008005003b cr4: 00000000000026f0 >> (XEN) cr3: 00000000b2fa0000 cr2: 000000000000002c >> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 >> (XEN) Xen stack trace from rsp=ffff83013ff2fc60: >> (XEN) ffff83013ff2fc78 ffff82c480121993 0000000000000000 ffff83013ff2fca8 >> (XEN) ffff82c480117fe0 ffff83013fee0000 80000c3fc8050627 00000000c3fc8050 >> (XEN) ffff83013ff60000 ffff83013ff2fd28 ffff82c48016538c ffff8800aec97b08 >> (XEN) ffff83013ff2fcf0 00000000000b25c9 0000000000000100 ffff8300bf78a000 >> (XEN) 0000000000000206 0000000000000000 ffff830e7f900a00 ffff83013ff2fd28 >> (XEN) ffff8300b25c9228 ffff8300bf78a000 0000000000800627 0000000000000000 >> (XEN) 0000000000000000 ffff83013ff2fdb8 ffff82c48016a9f7 ffff83013fee0018 >> (XEN) 00007ff03ff2ff28 0000000000000000 00000000000b25c9 ffff83013fee0000 >> (XEN) 80000c3fc8050627 80000c3fc8050627 ffff83013ff60000 ffff83013ff2fdb8 >> (XEN) 0000000180162c10 ffff83013ff2fdd8 0000000000000000 0000000000000000 >> (XEN) ffff8800aec97bb8 ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff08 >> (XEN) ffff82c48016b2a9 0000000000000000 0000000000000000 0000000000000000 >> (XEN) 00000000bf78a000 0000000000000006 00000000b2e55067 ffff83013ff2ff28 >> (XEN) ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 >> (XEN) ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 0000000000000000 >> (XEN) 00007ff200000082 ffff83013ff2fe68 80000c3fc8050627 ffff8300bf78a000 >> (XEN) ffff8300b25c9228 ffff83013ff2fee8 ffff82f60164b920 ffff83013fee0000 >> (XEN) 000000000003e6b8 00000000000b25c9 00000001032cc060 ffff83013ff60000 >> (XEN) 0000000000000000 aaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaa 00000001aaaaaaaa >> (XEN) 0000000000000001 00000000b25c9228 80000c3fc8050627 ffff8800aec97bb8 >> (XEN) Xen call trace: >> (XEN) [<ffff82c4801215b3>] check_lock+0x1b/0x45 >> (XEN) [<ffff82c480121993>] _spin_lock+0x11/0x3f >> (XEN) [<ffff82c480117fe0>] rangeset_contains_range+0x44/0x82 >> (XEN) [<ffff82c48016538c>] get_page_from_l1e+0x24c/0x47f >> (XEN) [<ffff82c48016a9f7>] mod_l1_entry+0x47f/0x64e >> (XEN) [<ffff82c48016b2a9>] do_mmu_update+0x6e3/0x1962 >> (XEN) [<ffff82c4801f71bf>] syscall_enter+0xef/0x149 >> (XEN) >> (XEN) Pagetable walk from 000000000000002c: >> (XEN) L4[0x000] = 00000000b2c3a067 00000000000aecc5 >> (XEN) L3[0x000] = 00000000b2c70067 00000000000aec8f >> (XEN) L2[0x000] = 0000000000000000 ffffffffffffffff >> (XEN) >> (XEN) **************************************** >> (XEN) Panic on CPU 1: >> (XEN) FATAL PAGE FAULT >> (XEN) [error_code=0000] >> (XEN) Faulting linear address: 000000000000002c >> (XEN) **************************************** >> (XEN) >> (XEN) Reboot in five seconds... >> >> J >> > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hmm, -S 256 is an obsolete size; it should be something more like -S 32. However, I added a patch that was supposed to do appropriate bounds-checking and return an error if the number was too high. Looks like it''s not working properly for some reason... I''ll take a look. -George On Fri, Apr 30, 2010 at 4:34 PM, Jeremy Fitzhardinge <jeremy@goop.org> wrote:> When I ran "xentrace -D -S 256 -e all /tmp/test.trace" from the > xenanalyze documentation, Xen immediately crashed with: > > (XEN) tbuf_size 256 > (XEN) p0 mfn 106a00 offset 64 > (XEN) p1 mfn 115700 offset 320 > (XEN) p2 mfn 113f00 offset 576 > (XEN) p3 mfn 113e00 offset 832 > (XEN) Xen trace buffers: initialized > (XEN) ----[ Xen-4.1-unstable x86_64 debug=y Not tainted ]---- > (XEN) CPU: 1 > (XEN) RIP: e008:[<ffff82c4801215b3>] check_lock+0x1b/0x45 > (XEN) RFLAGS: 0000000000010246 CONTEXT: hypervisor > (XEN) rax: 0000000000000001 rbx: 0000000000000028 rcx: 0000000000000000 > (XEN) rdx: 0000000000000000 rsi: 00000000c3fc8050 rdi: 000000000000002c > (XEN) rbp: ffff83013ff2fc60 rsp: ffff83013ff2fc60 r8: ffff8300bf78a000 > (XEN) r9: ffff83013ff60000 r10: 0080000000000001 r11: ffff82f60164b930 > (XEN) r12: 00000000c3fc8050 r13: 00000000c3fc8050 r14: 0000000000000028 > (XEN) r15: 0000000000800627 cr0: 000000008005003b cr4: 00000000000026f0 > (XEN) cr3: 00000000b2fa0000 cr2: 000000000000002c > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 > (XEN) Xen stack trace from rsp=ffff83013ff2fc60: > (XEN) ffff83013ff2fc78 ffff82c480121993 0000000000000000 ffff83013ff2fca8 > (XEN) ffff82c480117fe0 ffff83013fee0000 80000c3fc8050627 00000000c3fc8050 > (XEN) ffff83013ff60000 ffff83013ff2fd28 ffff82c48016538c ffff8800aec97b08 > (XEN) ffff83013ff2fcf0 00000000000b25c9 0000000000000100 ffff8300bf78a000 > (XEN) 0000000000000206 0000000000000000 ffff830e7f900a00 ffff83013ff2fd28 > (XEN) ffff8300b25c9228 ffff8300bf78a000 0000000000800627 0000000000000000 > (XEN) 0000000000000000 ffff83013ff2fdb8 ffff82c48016a9f7 ffff83013fee0018 > (XEN) 00007ff03ff2ff28 0000000000000000 00000000000b25c9 ffff83013fee0000 > (XEN) 80000c3fc8050627 80000c3fc8050627 ffff83013ff60000 ffff83013ff2fdb8 > (XEN) 0000000180162c10 ffff83013ff2fdd8 0000000000000000 0000000000000000 > (XEN) ffff8800aec97bb8 ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff08 > (XEN) ffff82c48016b2a9 0000000000000000 0000000000000000 0000000000000000 > (XEN) 00000000bf78a000 0000000000000006 00000000b2e55067 ffff83013ff2ff28 > (XEN) ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 > (XEN) ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 0000000000000000 > (XEN) 00007ff200000082 ffff83013ff2fe68 80000c3fc8050627 ffff8300bf78a000 > (XEN) ffff8300b25c9228 ffff83013ff2fee8 ffff82f60164b920 ffff83013fee0000 > (XEN) 000000000003e6b8 00000000000b25c9 00000001032cc060 ffff83013ff60000 > (XEN) 0000000000000000 aaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaa 00000001aaaaaaaa > (XEN) 0000000000000001 00000000b25c9228 80000c3fc8050627 ffff8800aec97bb8 > (XEN) Xen call trace: > (XEN) [<ffff82c4801215b3>] check_lock+0x1b/0x45 > (XEN) [<ffff82c480121993>] _spin_lock+0x11/0x3f > (XEN) [<ffff82c480117fe0>] rangeset_contains_range+0x44/0x82 > (XEN) [<ffff82c48016538c>] get_page_from_l1e+0x24c/0x47f > (XEN) [<ffff82c48016a9f7>] mod_l1_entry+0x47f/0x64e > (XEN) [<ffff82c48016b2a9>] do_mmu_update+0x6e3/0x1962 > (XEN) [<ffff82c4801f71bf>] syscall_enter+0xef/0x149 > (XEN) > (XEN) Pagetable walk from 000000000000002c: > (XEN) L4[0x000] = 00000000b2c3a067 00000000000aecc5 > (XEN) L3[0x000] = 00000000b2c70067 00000000000aec8f > (XEN) L2[0x000] = 0000000000000000 ffffffffffffffff > (XEN) > (XEN) **************************************** > (XEN) Panic on CPU 1: > (XEN) FATAL PAGE FAULT > (XEN) [error_code=0000] > (XEN) Faulting linear address: 000000000000002c > (XEN) **************************************** > (XEN) > (XEN) Reboot in five seconds... > > J > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 05/07/2010 01:48 PM, George Dunlap wrote:> Hmm, -S 256 is an obsolete size; it should be something more like -S 32. > > However, I added a patch that was supposed to do appropriate > bounds-checking and return an error if the number was too high. Looks > like it''s not working properly for some reason... I''ll take a look. > > -George > > On Fri, Apr 30, 2010 at 4:34 PM, Jeremy Fitzhardinge <jeremy@goop.org> wrote: > >> When I ran "xentrace -D -S 256 -e all /tmp/test.trace" from the >> xenanalyze documentation, Xen immediately crashed with: >> >> (XEN) tbuf_size 256 >> (XEN) p0 mfn 106a00 offset 64 >> (XEN) p1 mfn 115700 offset 320 >> (XEN) p2 mfn 113f00 offset 576 >> (XEN) p3 mfn 113e00 offset 832 >> (XEN) Xen trace buffers: initialized >> (XEN) ----[ Xen-4.1-unstable x86_64 debug=y Not tainted ]---- >> (XEN) CPU: 1 >> (XEN) RIP: e008:[<ffff82c4801215b3>] check_lock+0x1b/0x45 >>This suggests the problem is with misusing a lock in the wrong interrupt context, rather than anything to do with sizes. J>> (XEN) RFLAGS: 0000000000010246 CONTEXT: hypervisor >> (XEN) rax: 0000000000000001 rbx: 0000000000000028 rcx: 0000000000000000 >> (XEN) rdx: 0000000000000000 rsi: 00000000c3fc8050 rdi: 000000000000002c >> (XEN) rbp: ffff83013ff2fc60 rsp: ffff83013ff2fc60 r8: ffff8300bf78a000 >> (XEN) r9: ffff83013ff60000 r10: 0080000000000001 r11: ffff82f60164b930 >> (XEN) r12: 00000000c3fc8050 r13: 00000000c3fc8050 r14: 0000000000000028 >> (XEN) r15: 0000000000800627 cr0: 000000008005003b cr4: 00000000000026f0 >> (XEN) cr3: 00000000b2fa0000 cr2: 000000000000002c >> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 >> (XEN) Xen stack trace from rsp=ffff83013ff2fc60: >> (XEN) ffff83013ff2fc78 ffff82c480121993 0000000000000000 ffff83013ff2fca8 >> (XEN) ffff82c480117fe0 ffff83013fee0000 80000c3fc8050627 00000000c3fc8050 >> (XEN) ffff83013ff60000 ffff83013ff2fd28 ffff82c48016538c ffff8800aec97b08 >> (XEN) ffff83013ff2fcf0 00000000000b25c9 0000000000000100 ffff8300bf78a000 >> (XEN) 0000000000000206 0000000000000000 ffff830e7f900a00 ffff83013ff2fd28 >> (XEN) ffff8300b25c9228 ffff8300bf78a000 0000000000800627 0000000000000000 >> (XEN) 0000000000000000 ffff83013ff2fdb8 ffff82c48016a9f7 ffff83013fee0018 >> (XEN) 00007ff03ff2ff28 0000000000000000 00000000000b25c9 ffff83013fee0000 >> (XEN) 80000c3fc8050627 80000c3fc8050627 ffff83013ff60000 ffff83013ff2fdb8 >> (XEN) 0000000180162c10 ffff83013ff2fdd8 0000000000000000 0000000000000000 >> (XEN) ffff8800aec97bb8 ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff08 >> (XEN) ffff82c48016b2a9 0000000000000000 0000000000000000 0000000000000000 >> (XEN) 00000000bf78a000 0000000000000006 00000000b2e55067 ffff83013ff2ff28 >> (XEN) ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 >> (XEN) ffff83013ff2ff28 ffff83013ff2ff28 ffff83013ff2ff28 0000000000000000 >> (XEN) 00007ff200000082 ffff83013ff2fe68 80000c3fc8050627 ffff8300bf78a000 >> (XEN) ffff8300b25c9228 ffff83013ff2fee8 ffff82f60164b920 ffff83013fee0000 >> (XEN) 000000000003e6b8 00000000000b25c9 00000001032cc060 ffff83013ff60000 >> (XEN) 0000000000000000 aaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaa 00000001aaaaaaaa >> (XEN) 0000000000000001 00000000b25c9228 80000c3fc8050627 ffff8800aec97bb8 >> (XEN) Xen call trace: >> (XEN) [<ffff82c4801215b3>] check_lock+0x1b/0x45 >> (XEN) [<ffff82c480121993>] _spin_lock+0x11/0x3f >> (XEN) [<ffff82c480117fe0>] rangeset_contains_range+0x44/0x82 >> (XEN) [<ffff82c48016538c>] get_page_from_l1e+0x24c/0x47f >> (XEN) [<ffff82c48016a9f7>] mod_l1_entry+0x47f/0x64e >> (XEN) [<ffff82c48016b2a9>] do_mmu_update+0x6e3/0x1962 >> (XEN) [<ffff82c4801f71bf>] syscall_enter+0xef/0x149 >> (XEN) >> (XEN) Pagetable walk from 000000000000002c: >> (XEN) L4[0x000] = 00000000b2c3a067 00000000000aecc5 >> (XEN) L3[0x000] = 00000000b2c70067 00000000000aec8f >> (XEN) L2[0x000] = 0000000000000000 ffffffffffffffff >> (XEN) >> (XEN) **************************************** >> (XEN) Panic on CPU 1: >> (XEN) FATAL PAGE FAULT >> (XEN) [error_code=0000] >> (XEN) Faulting linear address: 000000000000002c >> (XEN) **************************************** >> (XEN) >> (XEN) Reboot in five seconds... >> >> J >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge wrote:>>> (XEN) ----[ Xen-4.1-unstable x86_64 debug=y Not tainted ]---- >>> (XEN) CPU: 1 >>> (XEN) RIP: e008:[<ffff82c4801215b3>] check_lock+0x1b/0x45 >>> >>> > > This suggests the problem is with misusing a lock in the wrong interrupt > context, rather than anything to do with sizes. >Except that, it works for me if I use -S 32, and doesn''t if I use -S 512 (on my 2-core box, equivalent # of pages to -S 256 on your 4-core box). :-) Try it, I suspect it will work. Also: * It''s a page fault with a null pointer, not a bugcheck. In a non-debug build, it will crash in spin_lock instead of check_lock. * The fault is in the MMU update hypercall; I believe done when xentrace tries to map garbage pages or invalid MFNs. * This is the exact bug we were getting in product, and the bounds-checking fixed it. Hmm... the bounds checking should be working. The maximum index is meant to be 2048 (2 pages = 8k, / sizeof(uint32_t) = 2048), and the maximum index for you is 1088, well within the t_info size. Hmm... -George _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 05/07/2010 02:16 PM, George Dunlap wrote:> Jeremy Fitzhardinge wrote: >>>> (XEN) ----[ Xen-4.1-unstable x86_64 debug=y Not tainted ]---- >>>> (XEN) CPU: 1 >>>> (XEN) RIP: e008:[<ffff82c4801215b3>] check_lock+0x1b/0x45 >>>> >> >> This suggests the problem is with misusing a lock in the wrong interrupt >> context, rather than anything to do with sizes. >> > Except that, it works for me if I use -S 32, and doesn''t if I use -S > 512 (on my 2-core box, equivalent # of pages to -S 256 on your 4-core > box). :-) Try it, I suspect it will work.Yes, it does. But I''m seeing some pretty odd things while xentrace is running: first time all my SATA drives stopped responding, and the second time my ethernet device started getting tx watchdog timeouts. Perhaps the large amount of IO caused by xentrace is causing other bugs to become apparent, but this has been an otherwise very stable system... J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge wrote:> Yes, it does. But I''m seeing some pretty odd things while xentrace is > running: first time all my SATA drives stopped responding, and the > second time my ethernet device started getting tx watchdog timeouts. > Perhaps the large amount of IO caused by xentrace is causing other bugs > to become apparent, but this has been an otherwise very stable system... >That is a bit weird. None of my test machines have had that problem (probably 3-4 different varieties), and our automated testing infrastructure for the upcoming release hasn''t complained so far. Let me know if you find out anything, and I''ll keep an eye out for any weirdness on my systems. -George _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel