I''m getting the following panic in Xen with tip when building with debug=y. It seems to be something with the spinlock/irq sanity-checking code; but it doesn''t make any sense, because the lock in question is defined statically in xen/arch/x86/pci.c, and only used two places in that file, with irqsave/irqrestore both times. Any ideas? -George (XEN) Freed 140kB init memory. (XEN) Xen BUG at spinlock.c:24 (XEN) ----[ Xen-3.5-unstable x86_64 debug=y Tainted: C ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8011b7b7>] check_lock+0x3d/0x49 (XEN) RFLAGS: 0000000000010046 CONTEXT: hypervisor (XEN) rax: 0000000000000000 rbx: ffff828c8020fa68 rcx: 0000000000000001 (XEN) rdx: 0000000000000000 rsi: 0000000000000001 rdi: ffff828c8020fa6c (XEN) rbp: ffff828c8028fd38 rsp: ffff828c8028fd38 r8: 0000000000000000 (XEN) r9: ffff828c8028fee4 r10: 0000000000cfbb00 r11: 000000000000ffff (XEN) r12: 0000000000000286 r13: 0000000000000004 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000012ffae000 cr2: 00000000c1013fe8 (XEN) ds: e021 es: e021 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8028fd38: (XEN) ffff828c8028fd58 ffff828c8011ba2f 0000000000000004 0000000080000008 (XEN) 0000000000000000 ffff828c80161187 0000000000000004 ffff828c8028fdb8 (XEN) 0000000000000cfc ffff828c80167ba2 ffff8300cf2fa000 0000000000000000 (XEN) 0000000000000cfc ffff828c8028fe88 ffff828c8028ff28 0000000000000000 (XEN) ffff828c8028ff18 ffff828c8016b11c ffff83010000f800 ffff8300cf2fa000 (XEN) ffff828c8028fe78 ffff828c8015e527 ffff828c8028fe28 0000000000000282 (XEN) 0000000000000282 0000000000000286 000000008028fe58 ffff83012ffc6308 (XEN) ffff83012feb9a80 0000000000000006 00000000ff750000 ffff8300cf2fa000 (XEN) ffff828c8028fe78 ffff828c80162f66 000000000000e021 ffff828402000020 (XEN) ffff83012feb9a80 0000000000000006 00000004025fd720 0000000400000004 (XEN) 00000000c0603d87 ffff828c8015ebf6 828c801db970b948 c390ed90d1ffffff (XEN) 00000000ff750000 000000008011d687 000000012feb9067 ffff83012ffcc0e0 (XEN) ffff828c8028ffed 00000000ffffffff 0000000000000000 ffff8300cf2fa000 (XEN) 0000000000000000 00c0bb0000000000 0000000000000000 ffff8300cf2fa000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 00007d737fd700b7 ffff828c801db6db 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 00000000c05f3f34 0000000000000cfc (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000cfc 0000000000000000 (XEN) 0000000000000000 0000000d00000000 00000000c0603d86 000000000000e019 (XEN) Xen call trace: (XEN) [<ffff828c8011b7b7>] check_lock+0x3d/0x49 (XEN) [<ffff828c8011ba2f>] _spin_lock_irqsave+0x21/0x53 (XEN) [<ffff828c80161187>] pci_conf_read+0x37/0xd0 (XEN) (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) Xen BUG at spinlock.c:24 (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hm, I added some code to the check_lock() to print whenever it was initializing, and found that the lock in pci.c was initialized to zero. So I added some code to double-check the lock, modifying the layout of pci.c slightly, and the problem went away. Which likely means that there''s a buffer overflow somewhere clobbering the spinlock debug value. Fun, fun, fun... -George On Tue, Jun 2, 2009 at 4:09 PM, George Dunlap <George.Dunlap@eu.citrix.com> wrote:> I''m getting the following panic in Xen with tip when building with > debug=y. It seems to be something with the spinlock/irq > sanity-checking code; but it doesn''t make any sense, because the lock > in question is defined statically in xen/arch/x86/pci.c, and only used > two places in that file, with irqsave/irqrestore both times. Any > ideas? > > -George > > (XEN) Freed 140kB init memory. > (XEN) Xen BUG at spinlock.c:24 > (XEN) ----[ Xen-3.5-unstable x86_64 debug=y Tainted: C ]---- > (XEN) CPU: 0 > (XEN) RIP: e008:[<ffff828c8011b7b7>] check_lock+0x3d/0x49 > (XEN) RFLAGS: 0000000000010046 CONTEXT: hypervisor > (XEN) rax: 0000000000000000 rbx: ffff828c8020fa68 rcx: 0000000000000001 > (XEN) rdx: 0000000000000000 rsi: 0000000000000001 rdi: ffff828c8020fa6c > (XEN) rbp: ffff828c8028fd38 rsp: ffff828c8028fd38 r8: 0000000000000000 > (XEN) r9: ffff828c8028fee4 r10: 0000000000cfbb00 r11: 000000000000ffff > (XEN) r12: 0000000000000286 r13: 0000000000000004 r14: 0000000000000000 > (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026f0 > (XEN) cr3: 000000012ffae000 cr2: 00000000c1013fe8 > (XEN) ds: e021 es: e021 fs: 0000 gs: 0000 ss: 0000 cs: e008 > (XEN) Xen stack trace from rsp=ffff828c8028fd38: > (XEN) ffff828c8028fd58 ffff828c8011ba2f 0000000000000004 0000000080000008 > (XEN) 0000000000000000 ffff828c80161187 0000000000000004 ffff828c8028fdb8 > (XEN) 0000000000000cfc ffff828c80167ba2 ffff8300cf2fa000 0000000000000000 > (XEN) 0000000000000cfc ffff828c8028fe88 ffff828c8028ff28 0000000000000000 > (XEN) ffff828c8028ff18 ffff828c8016b11c ffff83010000f800 ffff8300cf2fa000 > (XEN) ffff828c8028fe78 ffff828c8015e527 ffff828c8028fe28 0000000000000282 > (XEN) 0000000000000282 0000000000000286 000000008028fe58 ffff83012ffc6308 > (XEN) ffff83012feb9a80 0000000000000006 00000000ff750000 ffff8300cf2fa000 > (XEN) ffff828c8028fe78 ffff828c80162f66 000000000000e021 ffff828402000020 > (XEN) ffff83012feb9a80 0000000000000006 00000004025fd720 0000000400000004 > (XEN) 00000000c0603d87 ffff828c8015ebf6 828c801db970b948 c390ed90d1ffffff > (XEN) 00000000ff750000 000000008011d687 000000012feb9067 ffff83012ffcc0e0 > (XEN) ffff828c8028ffed 00000000ffffffff 0000000000000000 ffff8300cf2fa000 > (XEN) 0000000000000000 00c0bb0000000000 0000000000000000 ffff8300cf2fa000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 00007d737fd700b7 ffff828c801db6db 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 00000000c05f3f34 0000000000000cfc > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000cfc 0000000000000000 > (XEN) 0000000000000000 0000000d00000000 00000000c0603d86 000000000000e019 > (XEN) Xen call trace: > (XEN) [<ffff828c8011b7b7>] check_lock+0x3d/0x49 > (XEN) [<ffff828c8011ba2f>] _spin_lock_irqsave+0x21/0x53 > (XEN) [<ffff828c80161187>] pci_conf_read+0x37/0xd0 > (XEN) > (XEN) > (XEN) **************************************** > (XEN) Panic on CPU 0: > (XEN) Xen BUG at spinlock.c:24 > (XEN) **************************************** > (XEN) > (XEN) Reboot in five seconds... >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
There is a known debug=y problem caused by cs# 19602. We will be submitting a patch shortly for it. Are you using a NHM system? Allen -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of George Dunlap Sent: Tuesday, June 02, 2009 10:20 AM To: xen-devel@lists.xensource.com Subject: [Xen-devel] Re: Xen crash during boot with debug=y Hm, I added some code to the check_lock() to print whenever it was initializing, and found that the lock in pci.c was initialized to zero. So I added some code to double-check the lock, modifying the layout of pci.c slightly, and the problem went away. Which likely means that there''s a buffer overflow somewhere clobbering the spinlock debug value. Fun, fun, fun... -George On Tue, Jun 2, 2009 at 4:09 PM, George Dunlap <George.Dunlap@eu.citrix.com> wrote:> I''m getting the following panic in Xen with tip when building with > debug=y. It seems to be something with the spinlock/irq > sanity-checking code; but it doesn''t make any sense, because the lock > in question is defined statically in xen/arch/x86/pci.c, and only used > two places in that file, with irqsave/irqrestore both times. Any > ideas? > > -George > > (XEN) Freed 140kB init memory. > (XEN) Xen BUG at spinlock.c:24 > (XEN) ----[ Xen-3.5-unstable x86_64 debug=y Tainted: C ]---- > (XEN) CPU: 0 > (XEN) RIP: e008:[<ffff828c8011b7b7>] check_lock+0x3d/0x49 > (XEN) RFLAGS: 0000000000010046 CONTEXT: hypervisor > (XEN) rax: 0000000000000000 rbx: ffff828c8020fa68 rcx: 0000000000000001 > (XEN) rdx: 0000000000000000 rsi: 0000000000000001 rdi: ffff828c8020fa6c > (XEN) rbp: ffff828c8028fd38 rsp: ffff828c8028fd38 r8: 0000000000000000 > (XEN) r9: ffff828c8028fee4 r10: 0000000000cfbb00 r11: 000000000000ffff > (XEN) r12: 0000000000000286 r13: 0000000000000004 r14: 0000000000000000 > (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026f0 > (XEN) cr3: 000000012ffae000 cr2: 00000000c1013fe8 > (XEN) ds: e021 es: e021 fs: 0000 gs: 0000 ss: 0000 cs: e008 > (XEN) Xen stack trace from rsp=ffff828c8028fd38: > (XEN) ffff828c8028fd58 ffff828c8011ba2f 0000000000000004 0000000080000008 > (XEN) 0000000000000000 ffff828c80161187 0000000000000004 ffff828c8028fdb8 > (XEN) 0000000000000cfc ffff828c80167ba2 ffff8300cf2fa000 0000000000000000 > (XEN) 0000000000000cfc ffff828c8028fe88 ffff828c8028ff28 0000000000000000 > (XEN) ffff828c8028ff18 ffff828c8016b11c ffff83010000f800 ffff8300cf2fa000 > (XEN) ffff828c8028fe78 ffff828c8015e527 ffff828c8028fe28 0000000000000282 > (XEN) 0000000000000282 0000000000000286 000000008028fe58 ffff83012ffc6308 > (XEN) ffff83012feb9a80 0000000000000006 00000000ff750000 ffff8300cf2fa000 > (XEN) ffff828c8028fe78 ffff828c80162f66 000000000000e021 ffff828402000020 > (XEN) ffff83012feb9a80 0000000000000006 00000004025fd720 0000000400000004 > (XEN) 00000000c0603d87 ffff828c8015ebf6 828c801db970b948 c390ed90d1ffffff > (XEN) 00000000ff750000 000000008011d687 000000012feb9067 ffff83012ffcc0e0 > (XEN) ffff828c8028ffed 00000000ffffffff 0000000000000000 ffff8300cf2fa000 > (XEN) 0000000000000000 00c0bb0000000000 0000000000000000 ffff8300cf2fa000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 00007d737fd700b7 ffff828c801db6db 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 00000000c05f3f34 0000000000000cfc > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000cfc 0000000000000000 > (XEN) 0000000000000000 0000000d00000000 00000000c0603d86 000000000000e019 > (XEN) Xen call trace: > (XEN) [<ffff828c8011b7b7>] check_lock+0x3d/0x49 > (XEN) [<ffff828c8011ba2f>] _spin_lock_irqsave+0x21/0x53 > (XEN) [<ffff828c80161187>] pci_conf_read+0x37/0xd0 > (XEN) > (XEN) > (XEN) **************************************** > (XEN) Panic on CPU 0: > (XEN) Xen BUG at spinlock.c:24 > (XEN) **************************************** > (XEN) > (XEN) Reboot in five seconds... >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Kay, Allen M wrote:> There is a known debug=y problem caused by cs# 19602. We will be > submitting a patch shortly for it. > > Are you using a NHM system?Allen, George has solved the his issue. It''s not related to cs # 19602. Pls search his mail in mailing list. cs# 19602 with debug=y causes the spinlock panic only when you enable VT-d with interrupt remapping, because it calls xmalloc with interrupt disabled. I will fix it soon. Regards, Weidong> > Allen > > -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of George > Dunlap Sent: Tuesday, June 02, 2009 10:20 AM > To: xen-devel@lists.xensource.com > Subject: [Xen-devel] Re: Xen crash during boot with debug=y > > Hm, I added some code to the check_lock() to print whenever it was > initializing, and found that the lock in pci.c was initialized to > zero. > > So I added some code to double-check the lock, modifying the layout of > pci.c slightly, and the problem went away. > > Which likely means that there''s a buffer overflow somewhere clobbering > the spinlock debug value. Fun, fun, fun... > > -George > > On Tue, Jun 2, 2009 at 4:09 PM, George Dunlap > <George.Dunlap@eu.citrix.com> wrote: >> I''m getting the following panic in Xen with tip when building with >> debug=y. It seems to be something with the spinlock/irq >> sanity-checking code; but it doesn''t make any sense, because the lock >> in question is defined statically in xen/arch/x86/pci.c, and only >> used >> two places in that file, with irqsave/irqrestore both times. Any >> ideas? >> >> -George >> >> (XEN) Freed 140kB init memory. >> (XEN) Xen BUG at spinlock.c:24 >> (XEN) ----[ Xen-3.5-unstable x86_64 debug=y Tainted: C ]---- >> (XEN) CPU: 0 (XEN) RIP: e008:[<ffff828c8011b7b7>] >> check_lock+0x3d/0x49 (XEN) RFLAGS: 0000000000010046 CONTEXT: >> hypervisor (XEN) rax: 0000000000000000 rbx: ffff828c8020fa68 >> rcx: 0000000000000001 (XEN) rdx: 0000000000000000 rsi: >> 0000000000000001 rdi: ffff828c8020fa6c (XEN) rbp: ffff828c8028fd38 >> rsp: ffff828c8028fd38 r8: 0000000000000000 (XEN) r9: >> ffff828c8028fee4 r10: 0000000000cfbb00 r11: 000000000000ffff >> (XEN) r12: 0000000000000286 r13: 0000000000000004 r14: >> 0000000000000000 (XEN) r15: 0000000000000000 cr0: 000000008005003b >> cr4: 00000000000026f0 (XEN) cr3: 000000012ffae000 cr2: >> 00000000c1013fe8 (XEN) ds: e021 es: e021 fs: 0000 gs: 0000 >> ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff828c8028fd38: >> (XEN) ffff828c8028fd58 ffff828c8011ba2f 0000000000000004 >> 0000000080000008 (XEN) 0000000000000000 ffff828c80161187 >> 0000000000000004 ffff828c8028fdb8 (XEN) 0000000000000cfc >> ffff828c80167ba2 ffff8300cf2fa000 0000000000000000 (XEN) >> 0000000000000cfc ffff828c8028fe88 ffff828c8028ff28 0000000000000000 >> (XEN) ffff828c8028ff18 ffff828c8016b11c ffff83010000f800 >> ffff8300cf2fa000 (XEN) ffff828c8028fe78 ffff828c8015e527 >> ffff828c8028fe28 0000000000000282 (XEN) 0000000000000282 >> 0000000000000286 000000008028fe58 ffff83012ffc6308 (XEN) >> ffff83012feb9a80 0000000000000006 00000000ff750000 ffff8300cf2fa000 >> (XEN) ffff828c8028fe78 ffff828c80162f66 000000000000e021 >> ffff828402000020 (XEN) ffff83012feb9a80 0000000000000006 >> 00000004025fd720 0000000400000004 (XEN) 00000000c0603d87 >> ffff828c8015ebf6 828c801db970b948 c390ed90d1ffffff (XEN) >> 00000000ff750000 000000008011d687 000000012feb9067 ffff83012ffcc0e0 >> (XEN) ffff828c8028ffed 00000000ffffffff 0000000000000000 >> ffff8300cf2fa000 (XEN) 0000000000000000 00c0bb0000000000 >> 0000000000000000 ffff8300cf2fa000 (XEN) 0000000000000000 >> 0000000000000000 0000000000000000 0000000000000000 (XEN) >> 00007d737fd700b7 ffff828c801db6db 0000000000000000 0000000000000000 >> (XEN) 0000000000000000 0000000000000000 00000000c05f3f34 >> 0000000000000cfc (XEN) 0000000000000000 0000000000000000 >> 0000000000000000 0000000000000000 (XEN) 0000000000000000 >> 0000000000000000 0000000000000cfc 0000000000000000 (XEN) >> 0000000000000000 0000000d00000000 00000000c0603d86 000000000000e019 >> (XEN) Xen call trace: (XEN) [<ffff828c8011b7b7>] >> check_lock+0x3d/0x49 (XEN) [<ffff828c8011ba2f>] >> _spin_lock_irqsave+0x21/0x53 (XEN) [<ffff828c80161187>] >> pci_conf_read+0x37/0xd0 (XEN) (XEN) (XEN) >> **************************************** (XEN) Panic on CPU 0: (XEN) >> Xen BUG at spinlock.c:24 (XEN) >> **************************************** (XEN) (XEN) Reboot in five >> seconds... >> > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel