thr3ads.net - Xen devel - [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7 [Dec 2009]

If this information is useful, please help other people find it:
Share via:

John Anderson

2009-Dec-31 21:39 UTC

[Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

Greetings Xen Team,

I am trying to help the PaX Team (http://pax.grsecurity.net/)  integrate their
PaX product into the Kernel.org''s domU kernel for 2.6.31.7.   It seems
however, that we''ve run into a wall in the process.      The
GRSecurity/PaX patch applies and compiles cleanly, but at early boot we get the
page fault below.  The PaX Team has narrowed down the cause of the error to
xen_setup_kernel_pagetable while establishing the new pgd.   It seems as if
during the initial page table setup the pgd had become completely empty and on
return from the hypervisor everything triggers various page faults and kills the
guest kernel.    Can anyone describe what happens to the pgd during this phase? 
Also, does anyone know how to get printk or print any information from the guest
kernel at this early stage?

Thanks in advance for any help you can offer.

John A.

Page Fault Follows:


(XEN) Unhandled page fault in domain 26 on VCPU 0 (ec=0000)
(XEN) Pagetable walk from 0000000000000028:
(XEN)  L4[0x000] = 0000000000000000 ffffffffffffffff
(XEN) domain_crash_sync called from entry.S
(XEN) Domain 26 (vcpu#0) crashed on cpu#4:
(XEN) ----[ Xen-3.1.3  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    4
(XEN) RIP:    e033:[<ffffffff81018496>]
(XEN) RFLAGS: 0000000000000282   CONTEXT: guest
(XEN) rax: 0000000000521109   rbx: 0000000000000000   rcx: 0000000000000020
(XEN) rdx: ffffffff82ba6000   rsi: 00000000deadbeef   rdi: 0000000000000000
(XEN) rbp: 0000000000000000   rsp: ffffffff81601f50   r8:  0000000000000000
(XEN) r9:  ffffffff81817283   r10: ffffffff8102f528   r11: ffffffff81004280
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000006b0
(XEN) cr3: 0000000503189000   cr2: 0000000000000028
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
(XEN) Guest stack trace from rsp=ffffffff81601f50:
(XEN)    0000000000000020 ffffffff81004280 0000000000000000 ffffffff81018496
(XEN)    000000010000e030 0000000000010082 ffffffff81601f98 000000000000e02b
(XEN)    0000000000000007 ffffffff81004890 ffffffff8181719e 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    ffffffff81816c47 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2010-Jan-04 13:26 UTC

head link

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

Looks like NULL page is being accessed, which will not have a kernel
mapping. Then Xen complains the fault is not handled, probably because this
is too early in guest kernel boot for fault handlers to have been registered
with Xen. The pgd is clearly not empty, as Xen is able to dump the guest
kernel stack contents, which accesses guest kernel address space.

Dumping stuff early may be a bit tricky. If you do a debug build of Xen
itself then even as domU guest you can print stuff to Xen''s own console
via
the HYPERVISOR_console_io() hypercall (which is what usually only dom0 uses
for kernel console writes). This might work in concert with earlyprintk?
I''m
sure Jeremy will be able to say if I''m wrong, although he might be off
email
for the next week or so.

 -- Keir

On 31/12/2009 21:39, "John Anderson" <johna@ccbill.com> wrote:
> Greetings Xen Team,
>  
> I am trying to help the PaX Team (http://pax.grsecurity.net/)  integrate
their
> PaX product into the Kernel.org¹s domU kernel for 2.6.31.7.   It seems
> however, that we¹ve run into a wall in the process.      The GRSecurity/PaX
> patch applies and compiles cleanly, but at early boot we get the page fault
> below.  The PaX Team has narrowed down the cause of the error to
> xen_setup_kernel_pagetable while establishing the new pgd.   It seems as if
> during the initial page table setup the pgd had become completely empty and
on
> return from the hypervisor everything triggers various page faults and
kills
> the guest kernel.    Can anyone describe what happens to the pgd during
this
> phase?  Also, does anyone know how to get printk or print any information
from
> the guest kernel at this early stage?
>  
> Thanks in advance for any help you can offer.
>  
> John A.
>  
> Page Fault Follows:
>  
>  
> (XEN) Unhandled page fault in domain 26 on VCPU 0 (ec=0000)
> (XEN) Pagetable walk from 0000000000000028:
> (XEN)  L4[0x000] = 0000000000000000 ffffffffffffffff
> (XEN) domain_crash_sync called from entry.S
> (XEN) Domain 26 (vcpu#0) crashed on cpu#4:
> (XEN) ----[ Xen-3.1.3  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    4
> (XEN) RIP:    e033:[<ffffffff81018496>]
> (XEN) RFLAGS: 0000000000000282   CONTEXT: guest
> (XEN) rax: 0000000000521109   rbx: 0000000000000000  rcx: 0000000000000020
> (XEN) rdx: ffffffff82ba6000   rsi: 00000000deadbeef  rdi: 0000000000000000
> (XEN) rbp: 0000000000000000   rsp: ffffffff81601f50  r8:  0000000000000000
> (XEN) r9:  ffffffff81817283   r10: ffffffff8102f528  r11: ffffffff81004280
> (XEN) r12: 0000000000000000   r13: 0000000000000000  r14: 0000000000000000
> (XEN) r15: 0000000000000000   cr0: 000000008005003b  cr4: 00000000000006b0
> (XEN) cr3: 0000000503189000   cr2: 0000000000000028
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> (XEN) Guest stack trace from rsp=ffffffff81601f50:
> (XEN)    0000000000000020 ffffffff81004280 0000000000000000
ffffffff81018496
> (XEN)    000000010000e030 0000000000010082 ffffffff81601f98
000000000000e02b
> (XEN)    0000000000000007 ffffffff81004890 ffffffff8181719e
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    ffffffff81816c47 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Campbell

2010-Jan-04 13:35 UTC

head link

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

On Mon, 2010-01-04 at 13:26 +0000, Keir Fraser wrote:> Looks like NULL page is being accessed, which will not have a kernel
> mapping. Then Xen complains the fault is not handled, probably because this
> is too early in guest kernel boot for fault handlers to have been
registered
> with Xen. The pgd is clearly not empty, as Xen is able to dump the guest
> kernel stack contents, which accesses guest kernel address space.
> 
> Dumping stuff early may be a bit tricky. If you do a debug build of Xen
> itself then even as domU guest you can print stuff to Xen''s own
console via
> the HYPERVISOR_console_io() hypercall (which is what usually only dom0 uses
> for kernel console writes). This might work in concert with earlyprintk?
I''m
> sure Jeremy will be able to say if I''m wrong, although he might be
off email
> for the next week or so.
earlyprintk=xen works but sometimes even that isn''t available early
enough (IIRC earlyprintk accesses spinlock which isn''t yet initialised
during very early boot, or something like that) in which case you can
use xen_raw_printk (defined in <xen/hvc-console.h>). This also needs a
debug=y hypervisor.

Ian.
> 
>  -- Keir
> 
> On 31/12/2009 21:39, "John Anderson" <johna@ccbill.com>
wrote:
> 
> > Greetings Xen Team,
> >  
> > I am trying to help the PaX Team (http://pax.grsecurity.net/) 
integrate their
> > PaX product into the Kernel.org¹s domU kernel for 2.6.31.7.   It seems
> > however, that we¹ve run into a wall in the process.      The
GRSecurity/PaX
> > patch applies and compiles cleanly, but at early boot we get the page
fault
> > below.  The PaX Team has narrowed down the cause of the error to
> > xen_setup_kernel_pagetable while establishing the new pgd.   It seems
as if
> > during the initial page table setup the pgd had become completely
empty and on
> > return from the hypervisor everything triggers various page faults and
kills
> > the guest kernel.    Can anyone describe what happens to the pgd
during this
> > phase?  Also, does anyone know how to get printk or print any
information from
> > the guest kernel at this early stage?
> >  
> > Thanks in advance for any help you can offer.
> >  
> > John A.
> >  
> > Page Fault Follows:
> >  
> >  
> > (XEN) Unhandled page fault in domain 26 on VCPU 0 (ec=0000)
> > (XEN) Pagetable walk from 0000000000000028:
> > (XEN)  L4[0x000] = 0000000000000000 ffffffffffffffff
> > (XEN) domain_crash_sync called from entry.S
> > (XEN) Domain 26 (vcpu#0) crashed on cpu#4:
> > (XEN) ----[ Xen-3.1.3  x86_64  debug=y  Not tainted ]----
> > (XEN) CPU:    4
> > (XEN) RIP:    e033:[<ffffffff81018496>]
> > (XEN) RFLAGS: 0000000000000282   CONTEXT: guest
> > (XEN) rax: 0000000000521109   rbx: 0000000000000000  rcx:
0000000000000020
> > (XEN) rdx: ffffffff82ba6000   rsi: 00000000deadbeef  rdi:
0000000000000000
> > (XEN) rbp: 0000000000000000   rsp: ffffffff81601f50  r8: 
0000000000000000
> > (XEN) r9:  ffffffff81817283   r10: ffffffff8102f528  r11:
ffffffff81004280
> > (XEN) r12: 0000000000000000   r13: 0000000000000000  r14:
0000000000000000
> > (XEN) r15: 0000000000000000   cr0: 000000008005003b  cr4:
00000000000006b0
> > (XEN) cr3: 0000000503189000   cr2: 0000000000000028
> > (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> > (XEN) Guest stack trace from rsp=ffffffff81601f50:
> > (XEN)    0000000000000020 ffffffff81004280 0000000000000000
ffffffff81018496
> > (XEN)    000000010000e030 0000000000010082 ffffffff81601f98
000000000000e02b
> > (XEN)    0000000000000007 ffffffff81004890 ffffffff8181719e
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    ffffffff81816c47 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> > 
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Andrew Lyon

2010-Jan-04 19:13 UTC

head link

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

On Mon, Jan 4, 2010 at 1:35 PM, Ian Campbell <Ian.Campbell@citrix.com>
wrote:> On Mon, 2010-01-04 at 13:26 +0000, Keir Fraser wrote:
>> Looks like NULL page is being accessed, which will not have a kernel
>> mapping. Then Xen complains the fault is not handled, probably because
this
>> is too early in guest kernel boot for fault handlers to have been
registered
>> with Xen. The pgd is clearly not empty, as Xen is able to dump the
guest
>> kernel stack contents, which accesses guest kernel address space.
>>
>> Dumping stuff early may be a bit tricky. If you do a debug build of Xen
>> itself then even as domU guest you can print stuff to Xen''s
own console via
>> the HYPERVISOR_console_io() hypercall (which is what usually only dom0
uses
>> for kernel console writes). This might work in concert with
earlyprintk? I''m
>> sure Jeremy will be able to say if I''m wrong, although he
might be off email
>> for the next week or so.
>
> earlyprintk=xen works but sometimes even that isn''t available
early
> enough (IIRC earlyprintk accesses spinlock which isn''t yet
initialised
> during very early boot, or something like that) in which case you can
> use xen_raw_printk (defined in <xen/hvc-console.h>). This also needs
a
> debug=y hypervisor.
>
> Ian.
Daft question, how can I check if the Xen I am running was build with
debug=y or not?
>
>>
>>  -- Keir
>>
>> On 31/12/2009 21:39, "John Anderson" <johna@ccbill.com>
wrote:
>>
>> > Greetings Xen Team,
>> >
>> > I am trying to help the PaX Team (http://pax.grsecurity.net/)
 integrate their
>> > PaX product into the Kernel.org¹s domU kernel for 2.6.31.7.   It
seems
>> > however, that we¹ve run into a wall in the process.      The
GRSecurity/PaX
>> > patch applies and compiles cleanly, but at early boot we get the
page fault
>> > below.  The PaX Team has narrowed down the cause of the error to
>> > xen_setup_kernel_pagetable while establishing the new pgd.   It
seems as if
>> > during the initial page table setup the pgd had become completely
empty and on
>> > return from the hypervisor everything triggers various page faults
and kills
>> > the guest kernel.    Can anyone describe what happens to the pgd
during this
>> > phase?  Also, does anyone know how to get printk or print any
information from
>> > the guest kernel at this early stage?
>> >
>> > Thanks in advance for any help you can offer.
>> >
>> > John A.
>> >
>> > Page Fault Follows:
>> >
>> >
>> > (XEN) Unhandled page fault in domain 26 on VCPU 0 (ec=0000)
>> > (XEN) Pagetable walk from 0000000000000028:
>> > (XEN)  L4[0x000] = 0000000000000000 ffffffffffffffff
>> > (XEN) domain_crash_sync called from entry.S
>> > (XEN) Domain 26 (vcpu#0) crashed on cpu#4:
>> > (XEN) ----[ Xen-3.1.3  x86_64  debug=y  Not tainted ]----
>> > (XEN) CPU:    4
>> > (XEN) RIP:    e033:[<ffffffff81018496>]
>> > (XEN) RFLAGS: 0000000000000282   CONTEXT: guest
>> > (XEN) rax: 0000000000521109   rbx: 0000000000000000  rcx:
0000000000000020
>> > (XEN) rdx: ffffffff82ba6000   rsi: 00000000deadbeef  rdi:
0000000000000000
>> > (XEN) rbp: 0000000000000000   rsp: ffffffff81601f50  r8:
 0000000000000000
>> > (XEN) r9:  ffffffff81817283   r10: ffffffff8102f528  r11:
ffffffff81004280
>> > (XEN) r12: 0000000000000000   r13: 0000000000000000  r14:
0000000000000000
>> > (XEN) r15: 0000000000000000   cr0: 000000008005003b  cr4:
00000000000006b0
>> > (XEN) cr3: 0000000503189000   cr2: 0000000000000028
>> > (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs:
e033
>> > (XEN) Guest stack trace from rsp=ffffffff81601f50:
>> > (XEN)    0000000000000020 ffffffff81004280 0000000000000000
ffffffff81018496
>> > (XEN)    000000010000e030 0000000000010082 ffffffff81601f98
000000000000e02b
>> > (XEN)    0000000000000007 ffffffff81004890 ffffffff8181719e
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    ffffffff81816c47 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> > (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> >
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Campbell

2010-Jan-04 19:47 UTC

head link

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

On Mon, 2010-01-04 at 19:13 +0000, Andrew Lyon wrote:> Daft question, how can I check if the Xen I am running was build with
> debug=y or not? 
Not daft at all, I''ve got no idea... I think it is printed as part of
Xen''s oops messages but that''s not terribly useful if you
haven''t got
one.

Aha, Xen''s "d" debug key includes debug=<y|n> in its
output. Not
terribly elegant but it works...

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Andrew Lyon

2010-Jan-04 20:18 UTC

head link

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

On Mon, Jan 4, 2010 at 7:47 PM, Ian Campbell <Ian.Campbell@citrix.com>
wrote:> On Mon, 2010-01-04 at 19:13 +0000, Andrew Lyon wrote:
>> Daft question, how can I check if the Xen I am running was build with
>> debug=y or not?
>
> Not daft at all, I''ve got no idea... I think it is printed as part
of
> Xen''s oops messages but that''s not terribly useful if you
haven''t got
> one.
>
> Aha, Xen''s "d" debug key includes debug=<y|n> in
its output. Not
> terribly elegant but it works...
>
> Ian.
>
>
Thanks Ian, indeed it does show if Xen was built with debug,
unfortunately it also crashed the entire system, and although there
was a message "Reboot in 5 seconds" it did not reboot :(, I will see
if I can replicate this but I believe I tested all of the Xen debug
keys recently as I discovered that dumping dom0 registers triggered
the watchdog and submitted a patch to fix that.

Running Xen 3.4.2, here is the log
d
(XEN) ''d'' pressed -> dumping registers
(XEN)
(XEN) *** Dumping CPU5 host state: ***
(XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    5
(XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
(XEN) RFLAGS: 0000000000010292   CONTEXT: hypervisor
(XEN) rax: 0000000000000000   rbx: 0000000000000064   rcx: 0000000000000000
(XEN) rdx: 0000000000000000   rsi: 000000000000000a   rdi: 0000000000000000
(XEN) rbp: ffff83043ff97c48   rsp: ffff83043ff97c38   r8:  0000000000000004
(XEN) r9:  0000000000000004   r10: 00000000fffffffc   r11: 0000000000000001
(XEN) r12: ffff83043ff97dd8   r13: ffff828c80203b78   r14: 0000000000000282
(XEN) r15: ffff83043ff97dd8   cr0: 000000008005003b   cr4: 00000000000026f0
(XEN) cr3: 00000000bfc6c000   cr2: 00007f5cf492d8d0
(XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff83043ff97c38:
(XEN)    ffff83043ff97dd8 0000000000000064 ffff83043ff97c88 ffff828c8010e1bc
(XEN)    ffff83043ff97c78 0000000000000064 ffff83043ff97dd8 ffff828c80203b78
(XEN)    0000000000000282 ffff83043ff97dd8 ffff83043ff97ca8 ffff828c8010e34b
(XEN)    ffff83043ff97dd8 0000000000000064 ffff83043ff97cb8 ffff828c80127d98
(XEN)    ffff83043ff97cd8 ffff828c80127e65 ffff828c80203b00 ffff828c80127ddc
(XEN)    ffff83043ff97d28 ffff828c8012924e ffff83043ff97cf8 ffff828c8012859d
(XEN)    64ff83043ff97d28 0000000000000061 ffff828c80284340 ffff828c80203b00
(XEN)    ffff83043ff97dd8 ffff828c80284368 ffff83043ff97d58 ffff828c8012885c
(XEN)    ffff828c80294f80 ffff83043ff97dd8 ffff83043ff97f28 ffff828c8027e900
(XEN)    ffff83043ff97dc8 ffff828c80148d99 00000000000000f1 0000000000000005
(XEN)    0000000000000006 ffff828c80294fa4 000000f18011cb8e 0000000000000005
(XEN)    0000000000000000 000000000072f524 000000000072ef9d ffff83042e1d35f0
(XEN)    ffff83042e1d3590 ffff828c8021a760 00007cfbc0068207 ffff828c801423b6
(XEN)    ffff828c8021a760 ffff83042e1d3590 ffff83042e1d35f0 000000000072ef9d
(XEN)    ffff83043ff97ed0 000000000072f524 000052b9b64e72e4 ffff830093724060
(XEN)    0000000000000006 0000000000000004 000000000072f524 0000000000000001
(XEN)    0000000000001008 0000000000000001 0000000000000000 000000f100000000
(XEN)    ffff828c801791df 000000000000e008 0000000000000283 ffff83043ff97e80
(XEN)    000000000000e010 ffff828c8027e900 ffff828c8027f900 ffff83043ff97ed0
(XEN)    000000008011af48 ffffffffffffffff ffff83043ff97f28 ffff83043ff97f28
(XEN) Xen call trace:
(XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
(XEN)    [<ffff828c8010e1bc>] dump_registers+0x5d/0x135
(XEN)    [<ffff828c8010e34b>] handle_keypress+0x76/0x9b
(XEN)    [<ffff828c80127d98>] __serial_rx+0x16/0x5a
(XEN)    [<ffff828c80127e65>] serial_rx+0x89/0x8e
(XEN)    [<ffff828c8012924e>] serial_rx_interrupt+0xa9/0xbf
(XEN)    [<ffff828c8012885c>] ns16550_interrupt+0x47/0x61
(XEN)    [<ffff828c80148d99>] do_IRQ+0x3e7/0x42e
(XEN)    [<ffff828c801423b6>] common_interrupt+0x26/0x30
(XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
(XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
(XEN)
(XEN) *** Dumping CPU5 guest state: ***
(XEN) No guest context (CPU is idle).
(XEN)
(XEN) *** Dumping CPU0 host state: ***
(XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
(XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
(XEN) rax: ffff83043ff97bf8   rbx: 0000000000a13d97   rcx: 0000000000000001
(XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi: 0000000000000000
(XEN) rbp: ffff828c80267db8   rsp: ffff828c80267da8   r8:  ffff828c8010e0e7
(XEN) r9:  0000000000000006   r10: ffff828c8023c560   r11: 000052b9d8788566
(XEN) r12: 0000000000a13779   r13: ffff83042e1d8210   r14: ffff83042e1d81b0
(XEN) r15: ffff828c8021a760   cr0: 000000008005003b   cr4: 00000000000026f0
(XEN) cr3: 0000000080804000   cr2: 00007f9695f32315
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff828c80267da8:
(XEN)    00000000000000ff 0000000000a13d97 ffff828c80267dc8 ffff828c8015942c
(XEN)    00007d737fd98207 ffff828c80143470 ffff828c8021a760 ffff83042e1d81b0
(XEN)    ffff83042e1d8210 0000000000a13779 ffff828c80267ed0 0000000000a13d97
(XEN)    000052b9d8788566 ffff828c8023c560 0000000000000006 0000000000000004
(XEN)    0000000000a13d97 0000000000000001 0000000000001008 0000000000000001
(XEN)    0000000000000000 000000fb00000000 ffff828c801791df 000000000000e008
(XEN)    0000000000000287 ffff828c80267e80 000000000000e010 ffff828c8027e900
(XEN)    ffff828c8027f900 ffff828c80267ed0 000000008011af48 ffffffffffffffff
(XEN)    ffff828c80267f28 ffff828c80267f28 ffff828c8027ad80 ffff828c8027e900
(XEN)    ffff828c8021a760 ffff828c80267f20 ffff828c8013e7db ffff828c8011af79
(XEN)    ffff828c80267f28 ffff828c8023c5b0 ffff8300bfafc000 0000000000000000
(XEN)    ffff8300beb62000 000052b9d6ab2f39 ffff828c8023c100 ffff828c80267dc8
(XEN)    0000000000000000 ffffffff808500d0 ffffffff8084f6d0 ffffffffffffffff
(XEN)    0000000000000000 0000000000000000 0000000000000246 ffff88005ce1d938
(XEN)    ffff88007f5bc108 0000000000000000 0000000000000000 ffffffff802063aa
(XEN)    00000000deadbeef 00000000deadbeef 00000000deadbeef 0000010000000000
(XEN)    ffffffff802063aa 000000000000e033 0000000000000246 ffffffff80781f30
(XEN)    000000000000e02b 000000000000beef 000000000000beef 000000000000beef
(XEN)    000000000000beef 0000000000000000 ffff8300bfafc000
(XEN) Xen call trace:
(XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
(XEN)    [<ffff828c8015942c>] smp_call_function_interrupt+0x99/0xe8
(XEN)    [<ffff828c80143470>] call_function_interrupt+0x30/0x40
(XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
(XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
(XEN)
(XEN) *** Dumping CPU0 guest state: ***
(XEN) No guest context (CPU is idle).
(XEN)
(XEN) *** Dumping CPU1 host state: ***
(XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    1
(XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
(XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
(XEN) rax: ffff83043ff97bf8   rbx: 0000000000c881c7   rcx: 0000000000000001
(XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi: 0000000000000000
(XEN) rbp: ffff83043ffd7db8   rsp: ffff83043ffd7da8   r8:  ffff828c8010e0e7
(XEN) r9:  0000000000000003   r10: ffff83040ef14a88   r11: 000052ba2e8e2eda
(XEN) r12: 0000000000c843b5   r13: ffff83042e1d8660   r14: ffff83042e1d8600
(XEN) r15: ffff828c8021a760   cr0: 000000008005003b   cr4: 00000000000026f0
(XEN) cr3: 000000022394c000   cr2: 00007f9695f32315
(XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff83043ffd7da8:
(XEN)    0000000000c8437d 0000000000c881c7 ffff83043ffd7dc8 ffff828c8015942c
(XEN)    00007cfbc0028207 ffff828c80143470 ffff828c8021a760 ffff83042e1d8600
(XEN)    ffff83042e1d8660 0000000000c843b5 ffff83043ffd7ed0 0000000000c881c7
(XEN)    000052ba2e8e2eda ffff83040ef14a88 0000000000000003 0000000000000004
(XEN)    0000000000c881c7 0000000000000001 0000000000001008 0000000000000001
(XEN)    0000000000000000 000000fb00000000 ffff828c801791df 000000000000e008
(XEN)    0000000000000297 ffff83043ffd7e80 000000000000e010 ffff828c8027e900
(XEN)    ffff828c8027f900 ffff83043ffd7ed0 000000018011af48 ffffffffffffffff
(XEN)    ffff83043ffd7f28 ffff83043ffd7f28 ffff828c8027ad80 ffff828c8027e900
(XEN)    ffff828c8021a760 ffff83043ffd7f20 ffff828c8013e7db ffff828c8011af79
(XEN)    ffff83043ffd7f28 ffff828c8023c5b0 ffff8300bff40000 0000000000000001
(XEN)    ffff8300beb60000 000052b9fb24146d ffff828c8023d100 ffff83043ffd7dc8
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000001 0000000000000246 0000979700000000
(XEN)    0000000000000000 ffff8803e98c9ea0 0000000000000000 ffffffff802063aa
(XEN)    00000000deadbeef 00000000deadbeef 00000000deadbeef 0000010000000000
(XEN)    ffffffff802063aa 000000000000e033 0000000000000246 ffff8803e98c9ed8
(XEN)    000000000000e02b 000000000000beef 000000000000beef 000000000000beef
(XEN)    000000000000beef 0000000000000001 ffff8300bff40000
(XEN) Xen call trace:
(XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
(XEN)    [<ffff828c8015942c>] smp_call_function_interrupt+0x99/0xe8
(XEN)    [<ffff828c80143470>] call_function_interrupt+0x30/0x40
(XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
(XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
(XEN)
(XEN) *** Dumping CPU1 guest state: ***
(XEN) No guest context (CPU is idle).
(XEN)
(XEN) *** Dumping CPU2 host state: ***
(XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    2
(XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
(XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
(XEN) rax: ffff83043ff97bf8   rbx: 0000000000efc5ce   rcx: 0000000000000001
(XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi: 0000000000000000
(XEN) rbp: ffff83043ffc7db8   rsp: ffff83043ffc7da8   r8:  ffff828c8010e0e7
(XEN) r9:  0000000000000004   r10: ffff83041a9bc718   r11: 000052ba2cce99f9
(XEN) r12: 0000000000efa19d   r13: ffff83042e1d8ab0   r14: ffff83042e1d8a50
(XEN) r15: ffff828c8021a760   cr0: 000000008005003b   cr4: 00000000000026f0
(XEN) cr3: 0000000080804000   cr2: 00000000119e3000
(XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff83043ffc7da8:
(XEN)    0000000000000282 0000000000efc5ce ffff83043ffc7dc8 ffff828c8015942c
(XEN)    00007cfbc0038207 ffff828c80143470 ffff828c8021a760 ffff83042e1d8a50
(XEN)    ffff83042e1d8ab0 0000000000efa19d ffff83043ffc7ed0 0000000000efc5ce
(XEN)    000052ba2cce99f9 ffff83041a9bc718 0000000000000004 0000000000000004
(XEN)    0000000000efc5ce 0000000000000001 0000000000001008 0000000000000001
(XEN)    0000000000000000 000000fb00000000 ffff828c801791df 000000000000e008
(XEN)    0000000000000297 ffff83043ffc7e80 000000000000e010 ffffffffffffffff
(XEN)    ffff83043ffc7ed8 ffff828c8011d137 00000001bfafa000 ffffffffffffffff
(XEN)    ffff83043ffc7f28 ffff83043ffc7f28 ffff828c8027ad80 ffff828c8027e900
(XEN)    ffff828c8021a760 ffff83043ffc7f20 ffff828c8013e7db ffff828c8011af79
(XEN)    ffff83043ffc7f28 ffff828c8023c5b0 ffff8300bfafa000 0000000000000002
(XEN)    ffff8300bfaf6000 000052ba2c40dc61 ffff828c8023e100 ffff83043ffc7dc8
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000002 0000000000000246 ffff8803e99afcf0
(XEN)    ffff8803e99ad988 0000000000000000 0000000000000000 ffffffff802063aa
(XEN)    00000000deadbeef 00000000deadbeef 00000000deadbeef 0000010000000000
(XEN)    ffffffff802063aa 000000000000e033 0000000000000246 ffff8803e98cbed8
(XEN)    000000000000e02b 000000000000beef 000000000000beef 000000000000beef
(XEN)    000000000000beef 0000000000000002 ffff8300bfafa000
(XEN) Xen call trace:
(XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
(XEN)    [<ffff828c8015942c>] smp_call_function_interrupt+0x99/0xe8
(XEN)    [<ffff828c80143470>] call_function_interrupt+0x30/0x40
(XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
(XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
(XEN)
(XEN) *** Dumping CPU2 guest state: ***
(XEN) No guest context (CPU is idle).
(XEN)
(XEN) *** Dumping CPU3 host state: ***
(XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    3
(XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
(XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
(XEN) rax: ffff83043ff97bf8   rbx: 0000000000000003   rcx: 0000000000000001
(XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi: 0000000000000000
(XEN) rbp: ffff83043ffb7dd8   rsp: ffff83043ffb7dc8   r8:  ffff828c8010e0e7
(XEN) r9:  0000000000000004   r10: ffff8300beb456d8   r11: 000052ba60111156
(XEN) r12: ffff83043ffb7f28   r13: ffff83043ffb7f28   r14: ffff828c8027e900
(XEN) r15: ffff828c8027f900   cr0: 000000008005003b   cr4: 00000000000026f0
(XEN) cr3: 000000008b7db000   cr2: 000000000333cd38
(XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff83043ffb7dc8:
(XEN)    ffff828c801075a7 0000000000000003 ffff83043ffb7de8 ffff828c8015942c
(XEN)    00007cfbc00481e7 ffff828c80143470 ffff828c8027f900 ffff828c8027e900
(XEN)    ffff83043ffb7f28 ffff83043ffb7f28 ffff83043ffb7ed0 0000000000000003
(XEN)    000052ba60111156 ffff8300beb456d8 0000000000000004 0000000000000002
(XEN)    ffff828c8027ea80 0000000000000001 0000000000000001 ffff828c8023f5c0
(XEN)    0000000000003fff828c8011b157>] _spin_unlock_irqrestore+0xc/0x24
(XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
(XEN) rax: 0000000000000001   rbx: 0000000000000000   rcx: 0000000000000020
(XEN) rdx: 00000000000003f8   rsi: 0000000000000082   rdi: ffff828c80203b80
(XEN) rbp: ffff83043ff90b48   rsp: ffff83043ff90b48   r8:  0000000000000004
(XEN) r9:  0000000000000004   r10: 00000000fffffffc   r11: 0000000000000001
(XEN) r12: ffff828c801ec9cb   r13: ffff828c80203b00   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000026f0
(XEN) cr3: 00000000bfc6c000   cr2: 00007f5cf492d8d0
(XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff83043ff90b48:
(XEN)    ffff83043ff90b98 ffff828c801290c4 000000083ff90b88 ffff828c80203b80
(XEN)    0000000000000082 ffff828c801ec9c5 ffff83043ff90c70 0000000000000082
(XEN)    ffff83043ff90f58 0000000000000001 ffff83043ff90ba8 ffff828c801276ea
(XEN)    ffff83043ff90bc8 ffff828c8012771f ffff83043ff90bd8 0000000000000000
(XEN)    ffff83043ff90c58 ffff828c80127776 ffff83043ff90c28 ffff828c801290c4
(XEN)    000000083ff90c58 ffff828c80203b80 0000000000000082 ffff828c801f3523
(XEN)    ffff83043ff90c70 0000000000000082 ffff83043ff90f58 0000000000000001
(XEN)    ffff83043ff90c38 ffff828c801276ea ffff83043ff90c58 ffff828c80127728
(XEN)    ffff83043ff90f58 0000000000000000 ffff83043ff90d68 ffff828c80127cae
(XEN)    0000000000000079 ffff828c8027fe8a 0000003000000010 ffff83043ff90d78
(XEN)    ffff83043ff90c98 0000000000000000 0000000000000000 0000000000000005
(XEN)    0000000000000000 0000000000000000 0000000000000004 0000000000000004
(XEN)    ffff83043ff90ce8 0000000000000020 ffff828c801ec9cb 0000000000000086
(XEN)    ffff83043ff90d38 ffff828c801290c4 ffff83043ff90d18 0000000000000004
(XEN)    ffff83043ff90d28 ffff828c801e92a9 ffff83043ff90d68 ffff828c801d55e5
(XEN)    6e69617420746f4e 0000000000646574 ffff83043ff90d48 ffff828c801285bb
(XEN)    ffff83043ff90d78 ffff83043ff97f28 0000000000000001290c4
(XEN)    000000008005003b ffff828c80203b80 00007f5cf492d8d0 00000000bfc6c000
(XEN)    00000000000026f0 0000000000000082 ffff828c8010e0e7 0000000000000001
(XEN) Xen call trace:
(XEN)    [<ffff828c8011b157>] _spin_unlock_irqrestore+0xc/0x24
(XEN)    [<ffff828c801290c4>] serial_puts+0xea/0xf9
(XEN)    [<ffff828c801276ea>] sercon_puts+0x25/0x27
(XEN)    [<ffff828c8012771f>] __putstr+0x33/0x75
(XEN)    [<ffff828c80127776>] printk_start_of_line+0x15/0x81
(XEN)    [<ffff828c80127cae>] printk+0x19c/0x1cf
(XEN)    [<ffff828c801d59ec>] show_registers+0x29e/0x374
(XEN)    [<ffff828c8015f4d4>] show_execution_state+0x11/0x20
(XEN)    [<ffff828c8015fdeb>] fatal_trap+0x5d/0xd6
(XEN)    [<ffff828c80156d74>] nmi_watchdog_tick+0xc4/0x1b4
(XEN)    [<ffff828c8015ffa9>] do_nmi+0x5d/0xf9
(XEN)    [<ffff828c801d2943>] handle_ist_exception+0x58/0x65
(XEN)    [<ffff828c801595e6>] on_selected_cpus+0x10a/0x12f
(XEN)    [<ffff828c8010e24a>] dump_registers+0xeb/0x135
(XEN)    [<ffff828c8010e34b>] handle_keypress+0x76/0x9b
(XEN)    [<ffff828c80127d98>] __serial_rx+0x16/0x5a
(XEN)    [<ffff828c80127e65>] serial_rx+0x89/0x8e
(XEN)    [<ffff828c8012924e>] serial_rx_interrupt+0xa9/0xbf
(XEN)    [<ffff828c8012885c>] ns16550_interrupt+0x47/0x61
(XEN)    [<ffff828c80148d99>] do_IRQ+0x3e7/0x42e
(XEN)    [<ffff828c801423b6>] common_interrupt+0x26/0x30
(XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
(XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 5:
(XEN) Assertion ''((lock)->lock <= 0)'' failed at
/var/tmp/portage/app-emulation/xen-3.4.2/work/xen-:18
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...


Andy

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2010-Jan-04 21:44 UTC

head link

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

On 04/01/2010 19:13, "Andrew Lyon" <andrew.lyon@gmail.com>
wrote:
> Daft question, how can I check if the Xen I am running was build with
> debug=y or not?
>>>> (XEN) Unhandled page fault in domain 26 on VCPU 0 (ec=0000)
>>>> (XEN) Pagetable walk from 0000000000000028:
>>>> (XEN)  L4[0x000] = 0000000000000000 ffffffffffffffff
>>>> (XEN) domain_crash_sync called from entry.S
>>>> (XEN) Domain 26 (vcpu#0) crashed on cpu#4:
>>>> (XEN) ----[ Xen-3.1.3  x86_64  debug=y  Not tainted ]----                                    ^^^^^^^

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Andrew Lyon

2010-Jan-05 09:15 UTC

head link

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

On Mon, Jan 4, 2010 at 12:18 PM, Andrew Lyon <andrew.lyon@gmail.com>
wrote:> On Mon, Jan 4, 2010 at 7:47 PM, Ian Campbell
<Ian.Campbell@citrix.com> wrote:
>> On Mon, 2010-01-04 at 19:13 +0000, Andrew Lyon wrote:
>>> Daft question, how can I check if the Xen I am running was build
with
>>> debug=y or not?
>>
>> Not daft at all, I''ve got no idea... I think it is printed as
part of
>> Xen''s oops messages but that''s not terribly useful if
you haven''t got
>> one.
>>
>> Aha, Xen''s "d" debug key includes debug=<y|n>
in its output. Not
>> terribly elegant but it works...
>>
>> Ian.
>>
>>
>
> Thanks Ian, indeed it does show if Xen was built with debug,
> unfortunately it also crashed the entire system, and although there
> was a message "Reboot in 5 seconds" it did not reboot :(, I will
see
> if I can replicate this but I believe I tested all of the Xen debug
> keys recently as I discovered that dumping dom0 registers triggered
> the watchdog and submitted a patch to fix that.
>
> Running Xen 3.4.2, here is the log
> d
> (XEN) ''d'' pressed -> dumping registers
> (XEN)
> (XEN) *** Dumping CPU5 host state: ***
> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    5
> (XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
> (XEN) RFLAGS: 0000000000010292   CONTEXT: hypervisor
> (XEN) rax: 0000000000000000   rbx: 0000000000000064   rcx: 0000000000000000
> (XEN) rdx: 0000000000000000   rsi: 000000000000000a   rdi: 0000000000000000
> (XEN) rbp: ffff83043ff97c48   rsp: ffff83043ff97c38   r8:  0000000000000004
> (XEN) r9:  0000000000000004   r10: 00000000fffffffc   r11: 0000000000000001
> (XEN) r12: ffff83043ff97dd8   r13: ffff828c80203b78   r14: 0000000000000282
> (XEN) r15: ffff83043ff97dd8   cr0: 000000008005003b   cr4: 00000000000026f0
> (XEN) cr3: 00000000bfc6c000   cr2: 00007f5cf492d8d0
> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen stack trace from rsp=ffff83043ff97c38:
> (XEN)    ffff83043ff97dd8 0000000000000064 ffff83043ff97c88
ffff828c8010e1bc
> (XEN)    ffff83043ff97c78 0000000000000064 ffff83043ff97dd8
ffff828c80203b78
> (XEN)    0000000000000282 ffff83043ff97dd8 ffff83043ff97ca8
ffff828c8010e34b
> (XEN)    ffff83043ff97dd8 0000000000000064 ffff83043ff97cb8
ffff828c80127d98
> (XEN)    ffff83043ff97cd8 ffff828c80127e65 ffff828c80203b00
ffff828c80127ddc
> (XEN)    ffff83043ff97d28 ffff828c8012924e ffff83043ff97cf8
ffff828c8012859d
> (XEN)    64ff83043ff97d28 0000000000000061 ffff828c80284340
ffff828c80203b00
> (XEN)    ffff83043ff97dd8 ffff828c80284368 ffff83043ff97d58
ffff828c8012885c
> (XEN)    ffff828c80294f80 ffff83043ff97dd8 ffff83043ff97f28
ffff828c8027e900
> (XEN)    ffff83043ff97dc8 ffff828c80148d99 00000000000000f1
0000000000000005
> (XEN)    0000000000000006 ffff828c80294fa4 000000f18011cb8e
0000000000000005
> (XEN)    0000000000000000 000000000072f524 000000000072ef9d
ffff83042e1d35f0
> (XEN)    ffff83042e1d3590 ffff828c8021a760 00007cfbc0068207
ffff828c801423b6
> (XEN)    ffff828c8021a760 ffff83042e1d3590 ffff83042e1d35f0
000000000072ef9d
> (XEN)    ffff83043ff97ed0 000000000072f524 000052b9b64e72e4
ffff830093724060
> (XEN)    0000000000000006 0000000000000004 000000000072f524
0000000000000001
> (XEN)    0000000000001008 0000000000000001 0000000000000000
000000f100000000
> (XEN)    ffff828c801791df 000000000000e008 0000000000000283
ffff83043ff97e80
> (XEN)    000000000000e010 ffff828c8027e900 ffff828c8027f900
ffff83043ff97ed0
> (XEN)    000000008011af48 ffffffffffffffff ffff83043ff97f28
ffff83043ff97f28
> (XEN) Xen call trace:
> (XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
> (XEN)    [<ffff828c8010e1bc>] dump_registers+0x5d/0x135
> (XEN)    [<ffff828c8010e34b>] handle_keypress+0x76/0x9b
> (XEN)    [<ffff828c80127d98>] __serial_rx+0x16/0x5a
> (XEN)    [<ffff828c80127e65>] serial_rx+0x89/0x8e
> (XEN)    [<ffff828c8012924e>] serial_rx_interrupt+0xa9/0xbf
> (XEN)    [<ffff828c8012885c>] ns16550_interrupt+0x47/0x61
> (XEN)    [<ffff828c80148d99>] do_IRQ+0x3e7/0x42e
> (XEN)    [<ffff828c801423b6>] common_interrupt+0x26/0x30
> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
> (XEN)
> (XEN) *** Dumping CPU5 guest state: ***
> (XEN) No guest context (CPU is idle).
> (XEN)
> (XEN) *** Dumping CPU0 host state: ***
> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    0
> (XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
> (XEN) rax: ffff83043ff97bf8   rbx: 0000000000a13d97   rcx: 0000000000000001
> (XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi: 0000000000000000
> (XEN) rbp: ffff828c80267db8   rsp: ffff828c80267da8   r8:  ffff828c8010e0e7
> (XEN) r9:  0000000000000006   r10: ffff828c8023c560   r11: 000052b9d8788566
> (XEN) r12: 0000000000a13779   r13: ffff83042e1d8210   r14: ffff83042e1d81b0
> (XEN) r15: ffff828c8021a760   cr0: 000000008005003b   cr4: 00000000000026f0
> (XEN) cr3: 0000000080804000   cr2: 00007f9695f32315
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen stack trace from rsp=ffff828c80267da8:
> (XEN)    00000000000000ff 0000000000a13d97 ffff828c80267dc8
ffff828c8015942c
> (XEN)    00007d737fd98207 ffff828c80143470 ffff828c8021a760
ffff83042e1d81b0
> (XEN)    ffff83042e1d8210 0000000000a13779 ffff828c80267ed0
0000000000a13d97
> (XEN)    000052b9d8788566 ffff828c8023c560 0000000000000006
0000000000000004
> (XEN)    0000000000a13d97 0000000000000001 0000000000001008
0000000000000001
> (XEN)    0000000000000000 000000fb00000000 ffff828c801791df
000000000000e008
> (XEN)    0000000000000287 ffff828c80267e80 000000000000e010
ffff828c8027e900
> (XEN)    ffff828c8027f900 ffff828c80267ed0 000000008011af48
ffffffffffffffff
> (XEN)    ffff828c80267f28 ffff828c80267f28 ffff828c8027ad80
ffff828c8027e900
> (XEN)    ffff828c8021a760 ffff828c80267f20 ffff828c8013e7db
ffff828c8011af79
> (XEN)    ffff828c80267f28 ffff828c8023c5b0 ffff8300bfafc000
0000000000000000
> (XEN)    ffff8300beb62000 000052b9d6ab2f39 ffff828c8023c100
ffff828c80267dc8
> (XEN)    0000000000000000 ffffffff808500d0 ffffffff8084f6d0
ffffffffffffffff
> (XEN)    0000000000000000 0000000000000000 0000000000000246
ffff88005ce1d938
> (XEN)    ffff88007f5bc108 0000000000000000 0000000000000000
ffffffff802063aa
> (XEN)    00000000deadbeef 00000000deadbeef 00000000deadbeef
0000010000000000
> (XEN)    ffffffff802063aa 000000000000e033 0000000000000246
ffffffff80781f30
> (XEN)    000000000000e02b 000000000000beef 000000000000beef
000000000000beef
> (XEN)    000000000000beef 0000000000000000 ffff8300bfafc000
> (XEN) Xen call trace:
> (XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
> (XEN)    [<ffff828c8015942c>] smp_call_function_interrupt+0x99/0xe8
> (XEN)    [<ffff828c80143470>] call_function_interrupt+0x30/0x40
> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
> (XEN)
> (XEN) *** Dumping CPU0 guest state: ***
> (XEN) No guest context (CPU is idle).
> (XEN)
> (XEN) *** Dumping CPU1 host state: ***
> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    1
> (XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
> (XEN) rax: ffff83043ff97bf8   rbx: 0000000000c881c7   rcx: 0000000000000001
> (XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi: 0000000000000000
> (XEN) rbp: ffff83043ffd7db8   rsp: ffff83043ffd7da8   r8:  ffff828c8010e0e7
> (XEN) r9:  0000000000000003   r10: ffff83040ef14a88   r11: 000052ba2e8e2eda
> (XEN) r12: 0000000000c843b5   r13: ffff83042e1d8660   r14: ffff83042e1d8600
> (XEN) r15: ffff828c8021a760   cr0: 000000008005003b   cr4: 00000000000026f0
> (XEN) cr3: 000000022394c000   cr2: 00007f9695f32315
> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen stack trace from rsp=ffff83043ffd7da8:
> (XEN)    0000000000c8437d 0000000000c881c7 ffff83043ffd7dc8
ffff828c8015942c
> (XEN)    00007cfbc0028207 ffff828c80143470 ffff828c8021a760
ffff83042e1d8600
> (XEN)    ffff83042e1d8660 0000000000c843b5 ffff83043ffd7ed0
0000000000c881c7
> (XEN)    000052ba2e8e2eda ffff83040ef14a88 0000000000000003
0000000000000004
> (XEN)    0000000000c881c7 0000000000000001 0000000000001008
0000000000000001
> (XEN)    0000000000000000 000000fb00000000 ffff828c801791df
000000000000e008
> (XEN)    0000000000000297 ffff83043ffd7e80 000000000000e010
ffff828c8027e900
> (XEN)    ffff828c8027f900 ffff83043ffd7ed0 000000018011af48
ffffffffffffffff
> (XEN)    ffff83043ffd7f28 ffff83043ffd7f28 ffff828c8027ad80
ffff828c8027e900
> (XEN)    ffff828c8021a760 ffff83043ffd7f20 ffff828c8013e7db
ffff828c8011af79
> (XEN)    ffff83043ffd7f28 ffff828c8023c5b0 ffff8300bff40000
0000000000000001
> (XEN)    ffff8300beb60000 000052b9fb24146d ffff828c8023d100
ffff83043ffd7dc8
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000001 0000000000000246
0000979700000000
> (XEN)    0000000000000000 ffff8803e98c9ea0 0000000000000000
ffffffff802063aa
> (XEN)    00000000deadbeef 00000000deadbeef 00000000deadbeef
0000010000000000
> (XEN)    ffffffff802063aa 000000000000e033 0000000000000246
ffff8803e98c9ed8
> (XEN)    000000000000e02b 000000000000beef 000000000000beef
000000000000beef
> (XEN)    000000000000beef 0000000000000001 ffff8300bff40000
> (XEN) Xen call trace:
> (XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
> (XEN)    [<ffff828c8015942c>] smp_call_function_interrupt+0x99/0xe8
> (XEN)    [<ffff828c80143470>] call_function_interrupt+0x30/0x40
> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
> (XEN)
> (XEN) *** Dumping CPU1 guest state: ***
> (XEN) No guest context (CPU is idle).
> (XEN)
> (XEN) *** Dumping CPU2 host state: ***
> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    2
> (XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
> (XEN) rax: ffff83043ff97bf8   rbx: 0000000000efc5ce   rcx: 0000000000000001
> (XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi: 0000000000000000
> (XEN) rbp: ffff83043ffc7db8   rsp: ffff83043ffc7da8   r8:  ffff828c8010e0e7
> (XEN) r9:  0000000000000004   r10: ffff83041a9bc718   r11: 000052ba2cce99f9
> (XEN) r12: 0000000000efa19d   r13: ffff83042e1d8ab0   r14: ffff83042e1d8a50
> (XEN) r15: ffff828c8021a760   cr0: 000000008005003b   cr4: 00000000000026f0
> (XEN) cr3: 0000000080804000   cr2: 00000000119e3000
> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen stack trace from rsp=ffff83043ffc7da8:
> (XEN)    0000000000000282 0000000000efc5ce ffff83043ffc7dc8
ffff828c8015942c
> (XEN)    00007cfbc0038207 ffff828c80143470 ffff828c8021a760
ffff83042e1d8a50
> (XEN)    ffff83042e1d8ab0 0000000000efa19d ffff83043ffc7ed0
0000000000efc5ce
> (XEN)    000052ba2cce99f9 ffff83041a9bc718 0000000000000004
0000000000000004
> (XEN)    0000000000efc5ce 0000000000000001 0000000000001008
0000000000000001
> (XEN)    0000000000000000 000000fb00000000 ffff828c801791df
000000000000e008
> (XEN)    0000000000000297 ffff83043ffc7e80 000000000000e010
ffffffffffffffff
> (XEN)    ffff83043ffc7ed8 ffff828c8011d137 00000001bfafa000
ffffffffffffffff
> (XEN)    ffff83043ffc7f28 ffff83043ffc7f28 ffff828c8027ad80
ffff828c8027e900
> (XEN)    ffff828c8021a760 ffff83043ffc7f20 ffff828c8013e7db
ffff828c8011af79
> (XEN)    ffff83043ffc7f28 ffff828c8023c5b0 ffff8300bfafa000
0000000000000002
> (XEN)    ffff8300bfaf6000 000052ba2c40dc61 ffff828c8023e100
ffff83043ffc7dc8
> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
> (XEN)    0000000000000000 0000000000000002 0000000000000246
ffff8803e99afcf0
> (XEN)    ffff8803e99ad988 0000000000000000 0000000000000000
ffffffff802063aa
> (XEN)    00000000deadbeef 00000000deadbeef 00000000deadbeef
0000010000000000
> (XEN)    ffffffff802063aa 000000000000e033 0000000000000246
ffff8803e98cbed8
> (XEN)    000000000000e02b 000000000000beef 000000000000beef
000000000000beef
> (XEN)    000000000000beef 0000000000000002 ffff8300bfafa000
> (XEN) Xen call trace:
> (XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
> (XEN)    [<ffff828c8015942c>] smp_call_function_interrupt+0x99/0xe8
> (XEN)    [<ffff828c80143470>] call_function_interrupt+0x30/0x40
> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
> (XEN)
> (XEN) *** Dumping CPU2 guest state: ***
> (XEN) No guest context (CPU is idle).
> (XEN)
> (XEN) *** Dumping CPU3 host state: ***
> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    3
> (XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
> (XEN) rax: ffff83043ff97bf8   rbx: 0000000000000003   rcx: 0000000000000001
> (XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi: 0000000000000000
> (XEN) rbp: ffff83043ffb7dd8   rsp: ffff83043ffb7dc8   r8:  ffff828c8010e0e7
> (XEN) r9:  0000000000000004   r10: ffff8300beb456d8   r11: 000052ba60111156
> (XEN) r12: ffff83043ffb7f28   r13: ffff83043ffb7f28   r14: ffff828c8027e900
> (XEN) r15: ffff828c8027f900   cr0: 000000008005003b   cr4: 00000000000026f0
> (XEN) cr3: 000000008b7db000   cr2: 000000000333cd38
> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen stack trace from rsp=ffff83043ffb7dc8:
> (XEN)    ffff828c801075a7 0000000000000003 ffff83043ffb7de8
ffff828c8015942c
> (XEN)    00007cfbc00481e7 ffff828c80143470 ffff828c8027f900
ffff828c8027e900
> (XEN)    ffff83043ffb7f28 ffff83043ffb7f28 ffff83043ffb7ed0
0000000000000003
> (XEN)    000052ba60111156 ffff8300beb456d8 0000000000000004
0000000000000002
> (XEN)    ffff828c8027ea80 0000000000000001 0000000000000001
ffff828c8023f5c0
> (XEN)    0000000000003fff828c8011b157>] _spin_unlock_irqrestore+0xc/0x24
> (XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
> (XEN) rax: 0000000000000001   rbx: 0000000000000000   rcx: 0000000000000020
> (XEN) rdx: 00000000000003f8   rsi: 0000000000000082   rdi: ffff828c80203b80
> (XEN) rbp: ffff83043ff90b48   rsp: ffff83043ff90b48   r8:  0000000000000004
> (XEN) r9:  0000000000000004   r10: 00000000fffffffc   r11: 0000000000000001
> (XEN) r12: ffff828c801ec9cb   r13: ffff828c80203b00   r14: 0000000000000000
> (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000026f0
> (XEN) cr3: 00000000bfc6c000   cr2: 00007f5cf492d8d0
> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen stack trace from rsp=ffff83043ff90b48:
> (XEN)    ffff83043ff90b98 ffff828c801290c4 000000083ff90b88
ffff828c80203b80
> (XEN)    0000000000000082 ffff828c801ec9c5 ffff83043ff90c70
0000000000000082
> (XEN)    ffff83043ff90f58 0000000000000001 ffff83043ff90ba8
ffff828c801276ea
> (XEN)    ffff83043ff90bc8 ffff828c8012771f ffff83043ff90bd8
0000000000000000
> (XEN)    ffff83043ff90c58 ffff828c80127776 ffff83043ff90c28
ffff828c801290c4
> (XEN)    000000083ff90c58 ffff828c80203b80 0000000000000082
ffff828c801f3523
> (XEN)    ffff83043ff90c70 0000000000000082 ffff83043ff90f58
0000000000000001
> (XEN)    ffff83043ff90c38 ffff828c801276ea ffff83043ff90c58
ffff828c80127728
> (XEN)    ffff83043ff90f58 0000000000000000 ffff83043ff90d68
ffff828c80127cae
> (XEN)    0000000000000079 ffff828c8027fe8a 0000003000000010
ffff83043ff90d78
> (XEN)    ffff83043ff90c98 0000000000000000 0000000000000000
0000000000000005
> (XEN)    0000000000000000 0000000000000000 0000000000000004
0000000000000004
> (XEN)    ffff83043ff90ce8 0000000000000020 ffff828c801ec9cb
0000000000000086
> (XEN)    ffff83043ff90d38 ffff828c801290c4 ffff83043ff90d18
0000000000000004
> (XEN)    ffff83043ff90d28 ffff828c801e92a9 ffff83043ff90d68
ffff828c801d55e5
> (XEN)    6e69617420746f4e 0000000000646574 ffff83043ff90d48
ffff828c801285bb
> (XEN)    ffff83043ff90d78 ffff83043ff97f28 0000000000000001290c4
> (XEN)    000000008005003b ffff828c80203b80 00007f5cf492d8d0
00000000bfc6c000
> (XEN)    00000000000026f0 0000000000000082 ffff828c8010e0e7
0000000000000001
> (XEN) Xen call trace:
> (XEN)    [<ffff828c8011b157>] _spin_unlock_irqrestore+0xc/0x24
> (XEN)    [<ffff828c801290c4>] serial_puts+0xea/0xf9
> (XEN)    [<ffff828c801276ea>] sercon_puts+0x25/0x27
> (XEN)    [<ffff828c8012771f>] __putstr+0x33/0x75
> (XEN)    [<ffff828c80127776>] printk_start_of_line+0x15/0x81
> (XEN)    [<ffff828c80127cae>] printk+0x19c/0x1cf
> (XEN)    [<ffff828c801d59ec>] show_registers+0x29e/0x374
> (XEN)    [<ffff828c8015f4d4>] show_execution_state+0x11/0x20
> (XEN)    [<ffff828c8015fdeb>] fatal_trap+0x5d/0xd6
> (XEN)    [<ffff828c80156d74>] nmi_watchdog_tick+0xc4/0x1b4
> (XEN)    [<ffff828c8015ffa9>] do_nmi+0x5d/0xf9
> (XEN)    [<ffff828c801d2943>] handle_ist_exception+0x58/0x65
> (XEN)    [<ffff828c801595e6>] on_selected_cpus+0x10a/0x12f
> (XEN)    [<ffff828c8010e24a>] dump_registers+0xeb/0x135
> (XEN)    [<ffff828c8010e34b>] handle_keypress+0x76/0x9b
> (XEN)    [<ffff828c80127d98>] __serial_rx+0x16/0x5a
> (XEN)    [<ffff828c80127e65>] serial_rx+0x89/0x8e
> (XEN)    [<ffff828c8012924e>] serial_rx_interrupt+0xa9/0xbf
> (XEN)    [<ffff828c8012885c>] ns16550_interrupt+0x47/0x61
> (XEN)    [<ffff828c80148d99>] do_IRQ+0x3e7/0x42e
> (XEN)    [<ffff828c801423b6>] common_interrupt+0x26/0x30
> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
> (XEN)
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 5:
> (XEN) Assertion ''((lock)->lock <= 0)'' failed at
> /var/tmp/portage/app-emulation/xen-3.4.2/work/xen-:18
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
>
>
> Andy
>
I tried ''d'' again on the same supermicro system and triggered
a
similar crash and lockup, I then tried it on my optiplex test system
which had a debug=n Xen and it did not crash, however after
recompiling Xen on the Optiplex with debug=y ''d'' triggers the
watchdog, so perhaps the problem is similar to the one I found
recently where dumping dom0 registers trigged the watchdog.

Both systems were dead locked after the crash, they did not reboot
automatically.

The supermicro has a 38400 baud physical serial port, the Optiplex has
a intel amt virtual serial port.

I will try adding a watchdog disable/enable around the dump registers
and test again on both systems, I will also test if the supermicro
system crashes when running a non debug Xen, perhaps this only happens
with debug builds.

Logs attached.

Andy



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Andrew Lyon

2010-Jan-05 09:24 UTC

head link

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

On Tue, Jan 5, 2010 at 1:15 AM, Andrew Lyon <andrew.lyon@gmail.com>
wrote:> On Mon, Jan 4, 2010 at 12:18 PM, Andrew Lyon <andrew.lyon@gmail.com>
wrote:
>> On Mon, Jan 4, 2010 at 7:47 PM, Ian Campbell
<Ian.Campbell@citrix.com> wrote:
>>> On Mon, 2010-01-04 at 19:13 +0000, Andrew Lyon wrote:
>>>> Daft question, how can I check if the Xen I am running was
build with
>>>> debug=y or not?
>>>
>>> Not daft at all, I''ve got no idea... I think it is printed
as part of
>>> Xen''s oops messages but that''s not terribly
useful if you haven''t got
>>> one.
>>>
>>> Aha, Xen''s "d" debug key includes
debug=<y|n> in its output. Not
>>> terribly elegant but it works...
>>>
>>> Ian.
>>>
>>>
>>
>> Thanks Ian, indeed it does show if Xen was built with debug,
>> unfortunately it also crashed the entire system, and although there
>> was a message "Reboot in 5 seconds" it did not reboot :(, I
will see
>> if I can replicate this but I believe I tested all of the Xen debug
>> keys recently as I discovered that dumping dom0 registers triggered
>> the watchdog and submitted a patch to fix that.
>>
>> Running Xen 3.4.2, here is the log
>> d
>> (XEN) ''d'' pressed -> dumping registers
>> (XEN)
>> (XEN) *** Dumping CPU5 host state: ***
>> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
>> (XEN) CPU:    5
>> (XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>> (XEN) RFLAGS: 0000000000010292   CONTEXT: hypervisor
>> (XEN) rax: 0000000000000000   rbx: 0000000000000064   rcx:
0000000000000000
>> (XEN) rdx: 0000000000000000   rsi: 000000000000000a   rdi:
0000000000000000
>> (XEN) rbp: ffff83043ff97c48   rsp: ffff83043ff97c38   r8:
 0000000000000004
>> (XEN) r9:  0000000000000004   r10: 00000000fffffffc   r11:
0000000000000001
>> (XEN) r12: ffff83043ff97dd8   r13: ffff828c80203b78   r14:
0000000000000282
>> (XEN) r15: ffff83043ff97dd8   cr0: 000000008005003b   cr4:
00000000000026f0
>> (XEN) cr3: 00000000bfc6c000   cr2: 00007f5cf492d8d0
>> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
>> (XEN) Xen stack trace from rsp=ffff83043ff97c38:
>> (XEN)    ffff83043ff97dd8 0000000000000064 ffff83043ff97c88
ffff828c8010e1bc
>> (XEN)    ffff83043ff97c78 0000000000000064 ffff83043ff97dd8
ffff828c80203b78
>> (XEN)    0000000000000282 ffff83043ff97dd8 ffff83043ff97ca8
ffff828c8010e34b
>> (XEN)    ffff83043ff97dd8 0000000000000064 ffff83043ff97cb8
ffff828c80127d98
>> (XEN)    ffff83043ff97cd8 ffff828c80127e65 ffff828c80203b00
ffff828c80127ddc
>> (XEN)    ffff83043ff97d28 ffff828c8012924e ffff83043ff97cf8
ffff828c8012859d
>> (XEN)    64ff83043ff97d28 0000000000000061 ffff828c80284340
ffff828c80203b00
>> (XEN)    ffff83043ff97dd8 ffff828c80284368 ffff83043ff97d58
ffff828c8012885c
>> (XEN)    ffff828c80294f80 ffff83043ff97dd8 ffff83043ff97f28
ffff828c8027e900
>> (XEN)    ffff83043ff97dc8 ffff828c80148d99 00000000000000f1
0000000000000005
>> (XEN)    0000000000000006 ffff828c80294fa4 000000f18011cb8e
0000000000000005
>> (XEN)    0000000000000000 000000000072f524 000000000072ef9d
ffff83042e1d35f0
>> (XEN)    ffff83042e1d3590 ffff828c8021a760 00007cfbc0068207
ffff828c801423b6
>> (XEN)    ffff828c8021a760 ffff83042e1d3590 ffff83042e1d35f0
000000000072ef9d
>> (XEN)    ffff83043ff97ed0 000000000072f524 000052b9b64e72e4
ffff830093724060
>> (XEN)    0000000000000006 0000000000000004 000000000072f524
0000000000000001
>> (XEN)    0000000000001008 0000000000000001 0000000000000000
000000f100000000
>> (XEN)    ffff828c801791df 000000000000e008 0000000000000283
ffff83043ff97e80
>> (XEN)    000000000000e010 ffff828c8027e900 ffff828c8027f900
ffff83043ff97ed0
>> (XEN)    000000008011af48 ffffffffffffffff ffff83043ff97f28
ffff83043ff97f28
>> (XEN) Xen call trace:
>> (XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>> (XEN)    [<ffff828c8010e1bc>] dump_registers+0x5d/0x135
>> (XEN)    [<ffff828c8010e34b>] handle_keypress+0x76/0x9b
>> (XEN)    [<ffff828c80127d98>] __serial_rx+0x16/0x5a
>> (XEN)    [<ffff828c80127e65>] serial_rx+0x89/0x8e
>> (XEN)    [<ffff828c8012924e>] serial_rx_interrupt+0xa9/0xbf
>> (XEN)    [<ffff828c8012885c>] ns16550_interrupt+0x47/0x61
>> (XEN)    [<ffff828c80148d99>] do_IRQ+0x3e7/0x42e
>> (XEN)    [<ffff828c801423b6>] common_interrupt+0x26/0x30
>> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
>> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
>> (XEN)
>> (XEN) *** Dumping CPU5 guest state: ***
>> (XEN) No guest context (CPU is idle).
>> (XEN)
>> (XEN) *** Dumping CPU0 host state: ***
>> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
>> (XEN) CPU:    0
>> (XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>> (XEN) rax: ffff83043ff97bf8   rbx: 0000000000a13d97   rcx:
0000000000000001
>> (XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi:
0000000000000000
>> (XEN) rbp: ffff828c80267db8   rsp: ffff828c80267da8   r8:
 ffff828c8010e0e7
>> (XEN) r9:  0000000000000006   r10: ffff828c8023c560   r11:
000052b9d8788566
>> (XEN) r12: 0000000000a13779   r13: ffff83042e1d8210   r14:
ffff83042e1d81b0
>> (XEN) r15: ffff828c8021a760   cr0: 000000008005003b   cr4:
00000000000026f0
>> (XEN) cr3: 0000000080804000   cr2: 00007f9695f32315
>> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
>> (XEN) Xen stack trace from rsp=ffff828c80267da8:
>> (XEN)    00000000000000ff 0000000000a13d97 ffff828c80267dc8
ffff828c8015942c
>> (XEN)    00007d737fd98207 ffff828c80143470 ffff828c8021a760
ffff83042e1d81b0
>> (XEN)    ffff83042e1d8210 0000000000a13779 ffff828c80267ed0
0000000000a13d97
>> (XEN)    000052b9d8788566 ffff828c8023c560 0000000000000006
0000000000000004
>> (XEN)    0000000000a13d97 0000000000000001 0000000000001008
0000000000000001
>> (XEN)    0000000000000000 000000fb00000000 ffff828c801791df
000000000000e008
>> (XEN)    0000000000000287 ffff828c80267e80 000000000000e010
ffff828c8027e900
>> (XEN)    ffff828c8027f900 ffff828c80267ed0 000000008011af48
ffffffffffffffff
>> (XEN)    ffff828c80267f28 ffff828c80267f28 ffff828c8027ad80
ffff828c8027e900
>> (XEN)    ffff828c8021a760 ffff828c80267f20 ffff828c8013e7db
ffff828c8011af79
>> (XEN)    ffff828c80267f28 ffff828c8023c5b0 ffff8300bfafc000
0000000000000000
>> (XEN)    ffff8300beb62000 000052b9d6ab2f39 ffff828c8023c100
ffff828c80267dc8
>> (XEN)    0000000000000000 ffffffff808500d0 ffffffff8084f6d0
ffffffffffffffff
>> (XEN)    0000000000000000 0000000000000000 0000000000000246
ffff88005ce1d938
>> (XEN)    ffff88007f5bc108 0000000000000000 0000000000000000
ffffffff802063aa
>> (XEN)    00000000deadbeef 00000000deadbeef 00000000deadbeef
0000010000000000
>> (XEN)    ffffffff802063aa 000000000000e033 0000000000000246
ffffffff80781f30
>> (XEN)    000000000000e02b 000000000000beef 000000000000beef
000000000000beef
>> (XEN)    000000000000beef 0000000000000000 ffff8300bfafc000
>> (XEN) Xen call trace:
>> (XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>> (XEN)    [<ffff828c8015942c>]
smp_call_function_interrupt+0x99/0xe8
>> (XEN)    [<ffff828c80143470>] call_function_interrupt+0x30/0x40
>> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
>> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
>> (XEN)
>> (XEN) *** Dumping CPU0 guest state: ***
>> (XEN) No guest context (CPU is idle).
>> (XEN)
>> (XEN) *** Dumping CPU1 host state: ***
>> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
>> (XEN) CPU:    1
>> (XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>> (XEN) rax: ffff83043ff97bf8   rbx: 0000000000c881c7   rcx:
0000000000000001
>> (XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi:
0000000000000000
>> (XEN) rbp: ffff83043ffd7db8   rsp: ffff83043ffd7da8   r8:
 ffff828c8010e0e7
>> (XEN) r9:  0000000000000003   r10: ffff83040ef14a88   r11:
000052ba2e8e2eda
>> (XEN) r12: 0000000000c843b5   r13: ffff83042e1d8660   r14:
ffff83042e1d8600
>> (XEN) r15: ffff828c8021a760   cr0: 000000008005003b   cr4:
00000000000026f0
>> (XEN) cr3: 000000022394c000   cr2: 00007f9695f32315
>> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
>> (XEN) Xen stack trace from rsp=ffff83043ffd7da8:
>> (XEN)    0000000000c8437d 0000000000c881c7 ffff83043ffd7dc8
ffff828c8015942c
>> (XEN)    00007cfbc0028207 ffff828c80143470 ffff828c8021a760
ffff83042e1d8600
>> (XEN)    ffff83042e1d8660 0000000000c843b5 ffff83043ffd7ed0
0000000000c881c7
>> (XEN)    000052ba2e8e2eda ffff83040ef14a88 0000000000000003
0000000000000004
>> (XEN)    0000000000c881c7 0000000000000001 0000000000001008
0000000000000001
>> (XEN)    0000000000000000 000000fb00000000 ffff828c801791df
000000000000e008
>> (XEN)    0000000000000297 ffff83043ffd7e80 000000000000e010
ffff828c8027e900
>> (XEN)    ffff828c8027f900 ffff83043ffd7ed0 000000018011af48
ffffffffffffffff
>> (XEN)    ffff83043ffd7f28 ffff83043ffd7f28 ffff828c8027ad80
ffff828c8027e900
>> (XEN)    ffff828c8021a760 ffff83043ffd7f20 ffff828c8013e7db
ffff828c8011af79
>> (XEN)    ffff83043ffd7f28 ffff828c8023c5b0 ffff8300bff40000
0000000000000001
>> (XEN)    ffff8300beb60000 000052b9fb24146d ffff828c8023d100
ffff83043ffd7dc8
>> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> (XEN)    0000000000000000 0000000000000001 0000000000000246
0000979700000000
>> (XEN)    0000000000000000 ffff8803e98c9ea0 0000000000000000
ffffffff802063aa
>> (XEN)    00000000deadbeef 00000000deadbeef 00000000deadbeef
0000010000000000
>> (XEN)    ffffffff802063aa 000000000000e033 0000000000000246
ffff8803e98c9ed8
>> (XEN)    000000000000e02b 000000000000beef 000000000000beef
000000000000beef
>> (XEN)    000000000000beef 0000000000000001 ffff8300bff40000
>> (XEN) Xen call trace:
>> (XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>> (XEN)    [<ffff828c8015942c>]
smp_call_function_interrupt+0x99/0xe8
>> (XEN)    [<ffff828c80143470>] call_function_interrupt+0x30/0x40
>> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
>> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
>> (XEN)
>> (XEN) *** Dumping CPU1 guest state: ***
>> (XEN) No guest context (CPU is idle).
>> (XEN)
>> (XEN) *** Dumping CPU2 host state: ***
>> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
>> (XEN) CPU:    2
>> (XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>> (XEN) rax: ffff83043ff97bf8   rbx: 0000000000efc5ce   rcx:
0000000000000001
>> (XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi:
0000000000000000
>> (XEN) rbp: ffff83043ffc7db8   rsp: ffff83043ffc7da8   r8:
 ffff828c8010e0e7
>> (XEN) r9:  0000000000000004   r10: ffff83041a9bc718   r11:
000052ba2cce99f9
>> (XEN) r12: 0000000000efa19d   r13: ffff83042e1d8ab0   r14:
ffff83042e1d8a50
>> (XEN) r15: ffff828c8021a760   cr0: 000000008005003b   cr4:
00000000000026f0
>> (XEN) cr3: 0000000080804000   cr2: 00000000119e3000
>> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
>> (XEN) Xen stack trace from rsp=ffff83043ffc7da8:
>> (XEN)    0000000000000282 0000000000efc5ce ffff83043ffc7dc8
ffff828c8015942c
>> (XEN)    00007cfbc0038207 ffff828c80143470 ffff828c8021a760
ffff83042e1d8a50
>> (XEN)    ffff83042e1d8ab0 0000000000efa19d ffff83043ffc7ed0
0000000000efc5ce
>> (XEN)    000052ba2cce99f9 ffff83041a9bc718 0000000000000004
0000000000000004
>> (XEN)    0000000000efc5ce 0000000000000001 0000000000001008
0000000000000001
>> (XEN)    0000000000000000 000000fb00000000 ffff828c801791df
000000000000e008
>> (XEN)    0000000000000297 ffff83043ffc7e80 000000000000e010
ffffffffffffffff
>> (XEN)    ffff83043ffc7ed8 ffff828c8011d137 00000001bfafa000
ffffffffffffffff
>> (XEN)    ffff83043ffc7f28 ffff83043ffc7f28 ffff828c8027ad80
ffff828c8027e900
>> (XEN)    ffff828c8021a760 ffff83043ffc7f20 ffff828c8013e7db
ffff828c8011af79
>> (XEN)    ffff83043ffc7f28 ffff828c8023c5b0 ffff8300bfafa000
0000000000000002
>> (XEN)    ffff8300bfaf6000 000052ba2c40dc61 ffff828c8023e100
ffff83043ffc7dc8
>> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>> (XEN)    0000000000000000 0000000000000002 0000000000000246
ffff8803e99afcf0
>> (XEN)    ffff8803e99ad988 0000000000000000 0000000000000000
ffffffff802063aa
>> (XEN)    00000000deadbeef 00000000deadbeef 00000000deadbeef
0000010000000000
>> (XEN)    ffffffff802063aa 000000000000e033 0000000000000246
ffff8803e98cbed8
>> (XEN)    000000000000e02b 000000000000beef 000000000000beef
000000000000beef
>> (XEN)    000000000000beef 0000000000000002 ffff8300bfafa000
>> (XEN) Xen call trace:
>> (XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>> (XEN)    [<ffff828c8015942c>]
smp_call_function_interrupt+0x99/0xe8
>> (XEN)    [<ffff828c80143470>] call_function_interrupt+0x30/0x40
>> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
>> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
>> (XEN)
>> (XEN) *** Dumping CPU2 guest state: ***
>> (XEN) No guest context (CPU is idle).
>> (XEN)
>> (XEN) *** Dumping CPU3 host state: ***
>> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
>> (XEN) CPU:    3
>> (XEN) RIP:    e008:[<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>> (XEN) rax: ffff83043ff97bf8   rbx: 0000000000000003   rcx:
0000000000000001
>> (XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi:
0000000000000000
>> (XEN) rbp: ffff83043ffb7dd8   rsp: ffff83043ffb7dc8   r8:
 ffff828c8010e0e7
>> (XEN) r9:  0000000000000004   r10: ffff8300beb456d8   r11:
000052ba60111156
>> (XEN) r12: ffff83043ffb7f28   r13: ffff83043ffb7f28   r14:
ffff828c8027e900
>> (XEN) r15: ffff828c8027f900   cr0: 000000008005003b   cr4:
00000000000026f0
>> (XEN) cr3: 000000008b7db000   cr2: 000000000333cd38
>> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
>> (XEN) Xen stack trace from rsp=ffff83043ffb7dc8:
>> (XEN)    ffff828c801075a7 0000000000000003 ffff83043ffb7de8
ffff828c8015942c
>> (XEN)    00007cfbc00481e7 ffff828c80143470 ffff828c8027f900
ffff828c8027e900
>> (XEN)    ffff83043ffb7f28 ffff83043ffb7f28 ffff83043ffb7ed0
0000000000000003
>> (XEN)    000052ba60111156 ffff8300beb456d8 0000000000000004
0000000000000002
>> (XEN)    ffff828c8027ea80 0000000000000001 0000000000000001
ffff828c8023f5c0
>> (XEN)    0000000000003fff828c8011b157>]
_spin_unlock_irqrestore+0xc/0x24
>> (XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
>> (XEN) rax: 0000000000000001   rbx: 0000000000000000   rcx:
0000000000000020
>> (XEN) rdx: 00000000000003f8   rsi: 0000000000000082   rdi:
ffff828c80203b80
>> (XEN) rbp: ffff83043ff90b48   rsp: ffff83043ff90b48   r8:
 0000000000000004
>> (XEN) r9:  0000000000000004   r10: 00000000fffffffc   r11:
0000000000000001
>> (XEN) r12: ffff828c801ec9cb   r13: ffff828c80203b00   r14:
0000000000000000
>> (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
00000000000026f0
>> (XEN) cr3: 00000000bfc6c000   cr2: 00007f5cf492d8d0
>> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
>> (XEN) Xen stack trace from rsp=ffff83043ff90b48:
>> (XEN)    ffff83043ff90b98 ffff828c801290c4 000000083ff90b88
ffff828c80203b80
>> (XEN)    0000000000000082 ffff828c801ec9c5 ffff83043ff90c70
0000000000000082
>> (XEN)    ffff83043ff90f58 0000000000000001 ffff83043ff90ba8
ffff828c801276ea
>> (XEN)    ffff83043ff90bc8 ffff828c8012771f ffff83043ff90bd8
0000000000000000
>> (XEN)    ffff83043ff90c58 ffff828c80127776 ffff83043ff90c28
ffff828c801290c4
>> (XEN)    000000083ff90c58 ffff828c80203b80 0000000000000082
ffff828c801f3523
>> (XEN)    ffff83043ff90c70 0000000000000082 ffff83043ff90f58
0000000000000001
>> (XEN)    ffff83043ff90c38 ffff828c801276ea ffff83043ff90c58
ffff828c80127728
>> (XEN)    ffff83043ff90f58 0000000000000000 ffff83043ff90d68
ffff828c80127cae
>> (XEN)    0000000000000079 ffff828c8027fe8a 0000003000000010
ffff83043ff90d78
>> (XEN)    ffff83043ff90c98 0000000000000000 0000000000000000
0000000000000005
>> (XEN)    0000000000000000 0000000000000000 0000000000000004
0000000000000004
>> (XEN)    ffff83043ff90ce8 0000000000000020 ffff828c801ec9cb
0000000000000086
>> (XEN)    ffff83043ff90d38 ffff828c801290c4 ffff83043ff90d18
0000000000000004
>> (XEN)    ffff83043ff90d28 ffff828c801e92a9 ffff83043ff90d68
ffff828c801d55e5
>> (XEN)    6e69617420746f4e 0000000000646574 ffff83043ff90d48
ffff828c801285bb
>> (XEN)    ffff83043ff90d78 ffff83043ff97f28 0000000000000001290c4
>> (XEN)    000000008005003b ffff828c80203b80 00007f5cf492d8d0
00000000bfc6c000
>> (XEN)    00000000000026f0 0000000000000082 ffff828c8010e0e7
0000000000000001
>> (XEN) Xen call trace:
>> (XEN)    [<ffff828c8011b157>] _spin_unlock_irqrestore+0xc/0x24
>> (XEN)    [<ffff828c801290c4>] serial_puts+0xea/0xf9
>> (XEN)    [<ffff828c801276ea>] sercon_puts+0x25/0x27
>> (XEN)    [<ffff828c8012771f>] __putstr+0x33/0x75
>> (XEN)    [<ffff828c80127776>] printk_start_of_line+0x15/0x81
>> (XEN)    [<ffff828c80127cae>] printk+0x19c/0x1cf
>> (XEN)    [<ffff828c801d59ec>] show_registers+0x29e/0x374
>> (XEN)    [<ffff828c8015f4d4>] show_execution_state+0x11/0x20
>> (XEN)    [<ffff828c8015fdeb>] fatal_trap+0x5d/0xd6
>> (XEN)    [<ffff828c80156d74>] nmi_watchdog_tick+0xc4/0x1b4
>> (XEN)    [<ffff828c8015ffa9>] do_nmi+0x5d/0xf9
>> (XEN)    [<ffff828c801d2943>] handle_ist_exception+0x58/0x65
>> (XEN)    [<ffff828c801595e6>] on_selected_cpus+0x10a/0x12f
>> (XEN)    [<ffff828c8010e24a>] dump_registers+0xeb/0x135
>> (XEN)    [<ffff828c8010e34b>] handle_keypress+0x76/0x9b
>> (XEN)    [<ffff828c80127d98>] __serial_rx+0x16/0x5a
>> (XEN)    [<ffff828c80127e65>] serial_rx+0x89/0x8e
>> (XEN)    [<ffff828c8012924e>] serial_rx_interrupt+0xa9/0xbf
>> (XEN)    [<ffff828c8012885c>] ns16550_interrupt+0x47/0x61
>> (XEN)    [<ffff828c80148d99>] do_IRQ+0x3e7/0x42e
>> (XEN)    [<ffff828c801423b6>] common_interrupt+0x26/0x30
>> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
>> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
>> (XEN)
>> (XEN)
>> (XEN) ****************************************
>> (XEN) Panic on CPU 5:
>> (XEN) Assertion ''((lock)->lock <= 0)'' failed at
>> /var/tmp/portage/app-emulation/xen-3.4.2/work/xen-:18
>> (XEN) ****************************************
>> (XEN)
>> (XEN) Reboot in five seconds...
>>
>>
>> Andy
>>
>
> I tried ''d'' again on the same supermicro system and
triggered a
> similar crash and lockup, I then tried it on my optiplex test system
> which had a debug=n Xen and it did not crash, however after
> recompiling Xen on the Optiplex with debug=y ''d'' triggers
the
> watchdog, so perhaps the problem is similar to the one I found
> recently where dumping dom0 registers trigged the watchdog.
>
> Both systems were dead locked after the crash, they did not reboot
> automatically.
>
> The supermicro has a 38400 baud physical serial port, the Optiplex has
> a intel amt virtual serial port.
>
> I will try adding a watchdog disable/enable around the dump registers
> and test again on both systems, I will also test if the supermicro
> system crashes when running a non debug Xen, perhaps this only happens
> with debug builds.
>
> Logs attached.
>
> Andy
>
Ian,

Sorry, I''ve got a head full of cold and got completely confused
between dump_registers and dump_dom0_registers, I''ve already submited
a patch to fix the ''d'' problem and it was merged!

Andy
,

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2010-Jan-05 09:26 UTC

head link

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

On 05/01/2010 09:15, "Andrew Lyon" <andrew.lyon@gmail.com>
wrote:
> I will try adding a watchdog disable/enable around the dump registers
> and test again on both systems, I will also test if the supermicro
> system crashes when running a non debug Xen, perhaps this only happens
> with debug builds.
The crash is due to an assertion, and assertions are only compiled in for
debug=y builds. I''ll see if I can repro.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Andrew Lyon

2010-Jan-05 09:48 UTC

head link

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

On Tue, Jan 5, 2010 at 1:24 AM, Andrew Lyon <andrew.lyon@gmail.com>
wrote:> On Tue, Jan 5, 2010 at 1:15 AM, Andrew Lyon <andrew.lyon@gmail.com>
wrote:
>> On Mon, Jan 4, 2010 at 12:18 PM, Andrew Lyon
<andrew.lyon@gmail.com> wrote:
>>> On Mon, Jan 4, 2010 at 7:47 PM, Ian Campbell
<Ian.Campbell@citrix.com> wrote:
>>>> On Mon, 2010-01-04 at 19:13 +0000, Andrew Lyon wrote:
>>>>> Daft question, how can I check if the Xen I am running was
build with
>>>>> debug=y or not?
>>>>
>>>> Not daft at all, I''ve got no idea... I think it is
printed as part of
>>>> Xen''s oops messages but that''s not terribly
useful if you haven''t got
>>>> one.
>>>>
>>>> Aha, Xen''s "d" debug key includes
debug=<y|n> in its output. Not
>>>> terribly elegant but it works...
>>>>
>>>> Ian.
>>>>
>>>>
>>>
>>> Thanks Ian, indeed it does show if Xen was built with debug,
>>> unfortunately it also crashed the entire system, and although there
>>> was a message "Reboot in 5 seconds" it did not reboot :(,
I will see
>>> if I can replicate this but I believe I tested all of the Xen debug
>>> keys recently as I discovered that dumping dom0 registers triggered
>>> the watchdog and submitted a patch to fix that.
>>>
>>> Running Xen 3.4.2, here is the log
>>> d
>>> (XEN) ''d'' pressed -> dumping registers
>>> (XEN)
>>> (XEN) *** Dumping CPU5 host state: ***
>>> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
>>> (XEN) CPU:    5
>>> (XEN) RIP:    e008:[<ffff828c8010e0f0>]
__dump_execstate+0x9/0x78
>>> (XEN) RFLAGS: 0000000000010292   CONTEXT: hypervisor
>>> (XEN) rax: 0000000000000000   rbx: 0000000000000064   rcx:
0000000000000000
>>> (XEN) rdx: 0000000000000000   rsi: 000000000000000a   rdi:
0000000000000000
>>> (XEN) rbp: ffff83043ff97c48   rsp: ffff83043ff97c38   r8:
 0000000000000004
>>> (XEN) r9:  0000000000000004   r10: 00000000fffffffc   r11:
0000000000000001
>>> (XEN) r12: ffff83043ff97dd8   r13: ffff828c80203b78   r14:
0000000000000282
>>> (XEN) r15: ffff83043ff97dd8   cr0: 000000008005003b   cr4:
00000000000026f0
>>> (XEN) cr3: 00000000bfc6c000   cr2: 00007f5cf492d8d0
>>> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs:
e008
>>> (XEN) Xen stack trace from rsp=ffff83043ff97c38:
>>> (XEN)    ffff83043ff97dd8 0000000000000064 ffff83043ff97c88
ffff828c8010e1bc
>>> (XEN)    ffff83043ff97c78 0000000000000064 ffff83043ff97dd8
ffff828c80203b78
>>> (XEN)    0000000000000282 ffff83043ff97dd8 ffff83043ff97ca8
ffff828c8010e34b
>>> (XEN)    ffff83043ff97dd8 0000000000000064 ffff83043ff97cb8
ffff828c80127d98
>>> (XEN)    ffff83043ff97cd8 ffff828c80127e65 ffff828c80203b00
ffff828c80127ddc
>>> (XEN)    ffff83043ff97d28 ffff828c8012924e ffff83043ff97cf8
ffff828c8012859d
>>> (XEN)    64ff83043ff97d28 0000000000000061 ffff828c80284340
ffff828c80203b00
>>> (XEN)    ffff83043ff97dd8 ffff828c80284368 ffff83043ff97d58
ffff828c8012885c
>>> (XEN)    ffff828c80294f80 ffff83043ff97dd8 ffff83043ff97f28
ffff828c8027e900
>>> (XEN)    ffff83043ff97dc8 ffff828c80148d99 00000000000000f1
0000000000000005
>>> (XEN)    0000000000000006 ffff828c80294fa4 000000f18011cb8e
0000000000000005
>>> (XEN)    0000000000000000 000000000072f524 000000000072ef9d
ffff83042e1d35f0
>>> (XEN)    ffff83042e1d3590 ffff828c8021a760 00007cfbc0068207
ffff828c801423b6
>>> (XEN)    ffff828c8021a760 ffff83042e1d3590 ffff83042e1d35f0
000000000072ef9d
>>> (XEN)    ffff83043ff97ed0 000000000072f524 000052b9b64e72e4
ffff830093724060
>>> (XEN)    0000000000000006 0000000000000004 000000000072f524
0000000000000001
>>> (XEN)    0000000000001008 0000000000000001 0000000000000000
000000f100000000
>>> (XEN)    ffff828c801791df 000000000000e008 0000000000000283
ffff83043ff97e80
>>> (XEN)    000000000000e010 ffff828c8027e900 ffff828c8027f900
ffff83043ff97ed0
>>> (XEN)    000000008011af48 ffffffffffffffff ffff83043ff97f28
ffff83043ff97f28
>>> (XEN) Xen call trace:
>>> (XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>>> (XEN)    [<ffff828c8010e1bc>] dump_registers+0x5d/0x135
>>> (XEN)    [<ffff828c8010e34b>] handle_keypress+0x76/0x9b
>>> (XEN)    [<ffff828c80127d98>] __serial_rx+0x16/0x5a
>>> (XEN)    [<ffff828c80127e65>] serial_rx+0x89/0x8e
>>> (XEN)    [<ffff828c8012924e>] serial_rx_interrupt+0xa9/0xbf
>>> (XEN)    [<ffff828c8012885c>] ns16550_interrupt+0x47/0x61
>>> (XEN)    [<ffff828c80148d99>] do_IRQ+0x3e7/0x42e
>>> (XEN)    [<ffff828c801423b6>] common_interrupt+0x26/0x30
>>> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
>>> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
>>> (XEN)
>>> (XEN) *** Dumping CPU5 guest state: ***
>>> (XEN) No guest context (CPU is idle).
>>> (XEN)
>>> (XEN) *** Dumping CPU0 host state: ***
>>> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
>>> (XEN) CPU:    0
>>> (XEN) RIP:    e008:[<ffff828c8010e0f0>]
__dump_execstate+0x9/0x78
>>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>>> (XEN) rax: ffff83043ff97bf8   rbx: 0000000000a13d97   rcx:
0000000000000001
>>> (XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi:
0000000000000000
>>> (XEN) rbp: ffff828c80267db8   rsp: ffff828c80267da8   r8:
 ffff828c8010e0e7
>>> (XEN) r9:  0000000000000006   r10: ffff828c8023c560   r11:
000052b9d8788566
>>> (XEN) r12: 0000000000a13779   r13: ffff83042e1d8210   r14:
ffff83042e1d81b0
>>> (XEN) r15: ffff828c8021a760   cr0: 000000008005003b   cr4:
00000000000026f0
>>> (XEN) cr3: 0000000080804000   cr2: 00007f9695f32315
>>> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs:
e008
>>> (XEN) Xen stack trace from rsp=ffff828c80267da8:
>>> (XEN)    00000000000000ff 0000000000a13d97 ffff828c80267dc8
ffff828c8015942c
>>> (XEN)    00007d737fd98207 ffff828c80143470 ffff828c8021a760
ffff83042e1d81b0
>>> (XEN)    ffff83042e1d8210 0000000000a13779 ffff828c80267ed0
0000000000a13d97
>>> (XEN)    000052b9d8788566 ffff828c8023c560 0000000000000006
0000000000000004
>>> (XEN)    0000000000a13d97 0000000000000001 0000000000001008
0000000000000001
>>> (XEN)    0000000000000000 000000fb00000000 ffff828c801791df
000000000000e008
>>> (XEN)    0000000000000287 ffff828c80267e80 000000000000e010
ffff828c8027e900
>>> (XEN)    ffff828c8027f900 ffff828c80267ed0 000000008011af48
ffffffffffffffff
>>> (XEN)    ffff828c80267f28 ffff828c80267f28 ffff828c8027ad80
ffff828c8027e900
>>> (XEN)    ffff828c8021a760 ffff828c80267f20 ffff828c8013e7db
ffff828c8011af79
>>> (XEN)    ffff828c80267f28 ffff828c8023c5b0 ffff8300bfafc000
0000000000000000
>>> (XEN)    ffff8300beb62000 000052b9d6ab2f39 ffff828c8023c100
ffff828c80267dc8
>>> (XEN)    0000000000000000 ffffffff808500d0 ffffffff8084f6d0
ffffffffffffffff
>>> (XEN)    0000000000000000 0000000000000000 0000000000000246
ffff88005ce1d938
>>> (XEN)    ffff88007f5bc108 0000000000000000 0000000000000000
ffffffff802063aa
>>> (XEN)    00000000deadbeef 00000000deadbeef 00000000deadbeef
0000010000000000
>>> (XEN)    ffffffff802063aa 000000000000e033 0000000000000246
ffffffff80781f30
>>> (XEN)    000000000000e02b 000000000000beef 000000000000beef
000000000000beef
>>> (XEN)    000000000000beef 0000000000000000 ffff8300bfafc000
>>> (XEN) Xen call trace:
>>> (XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>>> (XEN)    [<ffff828c8015942c>]
smp_call_function_interrupt+0x99/0xe8
>>> (XEN)    [<ffff828c80143470>]
call_function_interrupt+0x30/0x40
>>> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
>>> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
>>> (XEN)
>>> (XEN) *** Dumping CPU0 guest state: ***
>>> (XEN) No guest context (CPU is idle).
>>> (XEN)
>>> (XEN) *** Dumping CPU1 host state: ***
>>> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
>>> (XEN) CPU:    1
>>> (XEN) RIP:    e008:[<ffff828c8010e0f0>]
__dump_execstate+0x9/0x78
>>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>>> (XEN) rax: ffff83043ff97bf8   rbx: 0000000000c881c7   rcx:
0000000000000001
>>> (XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi:
0000000000000000
>>> (XEN) rbp: ffff83043ffd7db8   rsp: ffff83043ffd7da8   r8:
 ffff828c8010e0e7
>>> (XEN) r9:  0000000000000003   r10: ffff83040ef14a88   r11:
000052ba2e8e2eda
>>> (XEN) r12: 0000000000c843b5   r13: ffff83042e1d8660   r14:
ffff83042e1d8600
>>> (XEN) r15: ffff828c8021a760   cr0: 000000008005003b   cr4:
00000000000026f0
>>> (XEN) cr3: 000000022394c000   cr2: 00007f9695f32315
>>> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs:
e008
>>> (XEN) Xen stack trace from rsp=ffff83043ffd7da8:
>>> (XEN)    0000000000c8437d 0000000000c881c7 ffff83043ffd7dc8
ffff828c8015942c
>>> (XEN)    00007cfbc0028207 ffff828c80143470 ffff828c8021a760
ffff83042e1d8600
>>> (XEN)    ffff83042e1d8660 0000000000c843b5 ffff83043ffd7ed0
0000000000c881c7
>>> (XEN)    000052ba2e8e2eda ffff83040ef14a88 0000000000000003
0000000000000004
>>> (XEN)    0000000000c881c7 0000000000000001 0000000000001008
0000000000000001
>>> (XEN)    0000000000000000 000000fb00000000 ffff828c801791df
000000000000e008
>>> (XEN)    0000000000000297 ffff83043ffd7e80 000000000000e010
ffff828c8027e900
>>> (XEN)    ffff828c8027f900 ffff83043ffd7ed0 000000018011af48
ffffffffffffffff
>>> (XEN)    ffff83043ffd7f28 ffff83043ffd7f28 ffff828c8027ad80
ffff828c8027e900
>>> (XEN)    ffff828c8021a760 ffff83043ffd7f20 ffff828c8013e7db
ffff828c8011af79
>>> (XEN)    ffff83043ffd7f28 ffff828c8023c5b0 ffff8300bff40000
0000000000000001
>>> (XEN)    ffff8300beb60000 000052b9fb24146d ffff828c8023d100
ffff83043ffd7dc8
>>> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>>> (XEN)    0000000000000000 0000000000000001 0000000000000246
0000979700000000
>>> (XEN)    0000000000000000 ffff8803e98c9ea0 0000000000000000
ffffffff802063aa
>>> (XEN)    00000000deadbeef 00000000deadbeef 00000000deadbeef
0000010000000000
>>> (XEN)    ffffffff802063aa 000000000000e033 0000000000000246
ffff8803e98c9ed8
>>> (XEN)    000000000000e02b 000000000000beef 000000000000beef
000000000000beef
>>> (XEN)    000000000000beef 0000000000000001 ffff8300bff40000
>>> (XEN) Xen call trace:
>>> (XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>>> (XEN)    [<ffff828c8015942c>]
smp_call_function_interrupt+0x99/0xe8
>>> (XEN)    [<ffff828c80143470>]
call_function_interrupt+0x30/0x40
>>> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
>>> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
>>> (XEN)
>>> (XEN) *** Dumping CPU1 guest state: ***
>>> (XEN) No guest context (CPU is idle).
>>> (XEN)
>>> (XEN) *** Dumping CPU2 host state: ***
>>> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
>>> (XEN) CPU:    2
>>> (XEN) RIP:    e008:[<ffff828c8010e0f0>]
__dump_execstate+0x9/0x78
>>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>>> (XEN) rax: ffff83043ff97bf8   rbx: 0000000000efc5ce   rcx:
0000000000000001
>>> (XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi:
0000000000000000
>>> (XEN) rbp: ffff83043ffc7db8   rsp: ffff83043ffc7da8   r8:
 ffff828c8010e0e7
>>> (XEN) r9:  0000000000000004   r10: ffff83041a9bc718   r11:
000052ba2cce99f9
>>> (XEN) r12: 0000000000efa19d   r13: ffff83042e1d8ab0   r14:
ffff83042e1d8a50
>>> (XEN) r15: ffff828c8021a760   cr0: 000000008005003b   cr4:
00000000000026f0
>>> (XEN) cr3: 0000000080804000   cr2: 00000000119e3000
>>> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs:
e008
>>> (XEN) Xen stack trace from rsp=ffff83043ffc7da8:
>>> (XEN)    0000000000000282 0000000000efc5ce ffff83043ffc7dc8
ffff828c8015942c
>>> (XEN)    00007cfbc0038207 ffff828c80143470 ffff828c8021a760
ffff83042e1d8a50
>>> (XEN)    ffff83042e1d8ab0 0000000000efa19d ffff83043ffc7ed0
0000000000efc5ce
>>> (XEN)    000052ba2cce99f9 ffff83041a9bc718 0000000000000004
0000000000000004
>>> (XEN)    0000000000efc5ce 0000000000000001 0000000000001008
0000000000000001
>>> (XEN)    0000000000000000 000000fb00000000 ffff828c801791df
000000000000e008
>>> (XEN)    0000000000000297 ffff83043ffc7e80 000000000000e010
ffffffffffffffff
>>> (XEN)    ffff83043ffc7ed8 ffff828c8011d137 00000001bfafa000
ffffffffffffffff
>>> (XEN)    ffff83043ffc7f28 ffff83043ffc7f28 ffff828c8027ad80
ffff828c8027e900
>>> (XEN)    ffff828c8021a760 ffff83043ffc7f20 ffff828c8013e7db
ffff828c8011af79
>>> (XEN)    ffff83043ffc7f28 ffff828c8023c5b0 ffff8300bfafa000
0000000000000002
>>> (XEN)    ffff8300bfaf6000 000052ba2c40dc61 ffff828c8023e100
ffff83043ffc7dc8
>>> (XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
>>> (XEN)    0000000000000000 0000000000000002 0000000000000246
ffff8803e99afcf0
>>> (XEN)    ffff8803e99ad988 0000000000000000 0000000000000000
ffffffff802063aa
>>> (XEN)    00000000deadbeef 00000000deadbeef 00000000deadbeef
0000010000000000
>>> (XEN)    ffffffff802063aa 000000000000e033 0000000000000246
ffff8803e98cbed8
>>> (XEN)    000000000000e02b 000000000000beef 000000000000beef
000000000000beef
>>> (XEN)    000000000000beef 0000000000000002 ffff8300bfafa000
>>> (XEN) Xen call trace:
>>> (XEN)    [<ffff828c8010e0f0>] __dump_execstate+0x9/0x78
>>> (XEN)    [<ffff828c8015942c>]
smp_call_function_interrupt+0x99/0xe8
>>> (XEN)    [<ffff828c80143470>]
call_function_interrupt+0x30/0x40
>>> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
>>> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
>>> (XEN)
>>> (XEN) *** Dumping CPU2 guest state: ***
>>> (XEN) No guest context (CPU is idle).
>>> (XEN)
>>> (XEN) *** Dumping CPU3 host state: ***
>>> (XEN) ----[ Xen-3.4.2  x86_64  debug=y  Not tainted ]----
>>> (XEN) CPU:    3
>>> (XEN) RIP:    e008:[<ffff828c8010e0f0>]
__dump_execstate+0x9/0x78
>>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>>> (XEN) rax: ffff83043ff97bf8   rbx: 0000000000000003   rcx:
0000000000000001
>>> (XEN) rdx: ffff828c8027e900   rsi: ffff83043ff97bf8   rdi:
0000000000000000
>>> (XEN) rbp: ffff83043ffb7dd8   rsp: ffff83043ffb7dc8   r8:
 ffff828c8010e0e7
>>> (XEN) r9:  0000000000000004   r10: ffff8300beb456d8   r11:
000052ba60111156
>>> (XEN) r12: ffff83043ffb7f28   r13: ffff83043ffb7f28   r14:
ffff828c8027e900
>>> (XEN) r15: ffff828c8027f900   cr0: 000000008005003b   cr4:
00000000000026f0
>>> (XEN) cr3: 000000008b7db000   cr2: 000000000333cd38
>>> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs:
e008
>>> (XEN) Xen stack trace from rsp=ffff83043ffb7dc8:
>>> (XEN)    ffff828c801075a7 0000000000000003 ffff83043ffb7de8
ffff828c8015942c
>>> (XEN)    00007cfbc00481e7 ffff828c80143470 ffff828c8027f900
ffff828c8027e900
>>> (XEN)    ffff83043ffb7f28 ffff83043ffb7f28 ffff83043ffb7ed0
0000000000000003
>>> (XEN)    000052ba60111156 ffff8300beb456d8 0000000000000004
0000000000000002
>>> (XEN)    ffff828c8027ea80 0000000000000001 0000000000000001
ffff828c8023f5c0
>>> (XEN)    0000000000003fff828c8011b157>]
_spin_unlock_irqrestore+0xc/0x24
>>> (XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
>>> (XEN) rax: 0000000000000001   rbx: 0000000000000000   rcx:
0000000000000020
>>> (XEN) rdx: 00000000000003f8   rsi: 0000000000000082   rdi:
ffff828c80203b80
>>> (XEN) rbp: ffff83043ff90b48   rsp: ffff83043ff90b48   r8:
 0000000000000004
>>> (XEN) r9:  0000000000000004   r10: 00000000fffffffc   r11:
0000000000000001
>>> (XEN) r12: ffff828c801ec9cb   r13: ffff828c80203b00   r14:
0000000000000000
>>> (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
00000000000026f0
>>> (XEN) cr3: 00000000bfc6c000   cr2: 00007f5cf492d8d0
>>> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs:
e008
>>> (XEN) Xen stack trace from rsp=ffff83043ff90b48:
>>> (XEN)    ffff83043ff90b98 ffff828c801290c4 000000083ff90b88
ffff828c80203b80
>>> (XEN)    0000000000000082 ffff828c801ec9c5 ffff83043ff90c70
0000000000000082
>>> (XEN)    ffff83043ff90f58 0000000000000001 ffff83043ff90ba8
ffff828c801276ea
>>> (XEN)    ffff83043ff90bc8 ffff828c8012771f ffff83043ff90bd8
0000000000000000
>>> (XEN)    ffff83043ff90c58 ffff828c80127776 ffff83043ff90c28
ffff828c801290c4
>>> (XEN)    000000083ff90c58 ffff828c80203b80 0000000000000082
ffff828c801f3523
>>> (XEN)    ffff83043ff90c70 0000000000000082 ffff83043ff90f58
0000000000000001
>>> (XEN)    ffff83043ff90c38 ffff828c801276ea ffff83043ff90c58
ffff828c80127728
>>> (XEN)    ffff83043ff90f58 0000000000000000 ffff83043ff90d68
ffff828c80127cae
>>> (XEN)    0000000000000079 ffff828c8027fe8a 0000003000000010
ffff83043ff90d78
>>> (XEN)    ffff83043ff90c98 0000000000000000 0000000000000000
0000000000000005
>>> (XEN)    0000000000000000 0000000000000000 0000000000000004
0000000000000004
>>> (XEN)    ffff83043ff90ce8 0000000000000020 ffff828c801ec9cb
0000000000000086
>>> (XEN)    ffff83043ff90d38 ffff828c801290c4 ffff83043ff90d18
0000000000000004
>>> (XEN)    ffff83043ff90d28 ffff828c801e92a9 ffff83043ff90d68
ffff828c801d55e5
>>> (XEN)    6e69617420746f4e 0000000000646574 ffff83043ff90d48
ffff828c801285bb
>>> (XEN)    ffff83043ff90d78 ffff83043ff97f28 0000000000000001290c4
>>> (XEN)    000000008005003b ffff828c80203b80 00007f5cf492d8d0
00000000bfc6c000
>>> (XEN)    00000000000026f0 0000000000000082 ffff828c8010e0e7
0000000000000001
>>> (XEN) Xen call trace:
>>> (XEN)    [<ffff828c8011b157>]
_spin_unlock_irqrestore+0xc/0x24
>>> (XEN)    [<ffff828c801290c4>] serial_puts+0xea/0xf9
>>> (XEN)    [<ffff828c801276ea>] sercon_puts+0x25/0x27
>>> (XEN)    [<ffff828c8012771f>] __putstr+0x33/0x75
>>> (XEN)    [<ffff828c80127776>] printk_start_of_line+0x15/0x81
>>> (XEN)    [<ffff828c80127cae>] printk+0x19c/0x1cf
>>> (XEN)    [<ffff828c801d59ec>] show_registers+0x29e/0x374
>>> (XEN)    [<ffff828c8015f4d4>] show_execution_state+0x11/0x20
>>> (XEN)    [<ffff828c8015fdeb>] fatal_trap+0x5d/0xd6
>>> (XEN)    [<ffff828c80156d74>] nmi_watchdog_tick+0xc4/0x1b4
>>> (XEN)    [<ffff828c8015ffa9>] do_nmi+0x5d/0xf9
>>> (XEN)    [<ffff828c801d2943>] handle_ist_exception+0x58/0x65
>>> (XEN)    [<ffff828c801595e6>] on_selected_cpus+0x10a/0x12f
>>> (XEN)    [<ffff828c8010e24a>] dump_registers+0xeb/0x135
>>> (XEN)    [<ffff828c8010e34b>] handle_keypress+0x76/0x9b
>>> (XEN)    [<ffff828c80127d98>] __serial_rx+0x16/0x5a
>>> (XEN)    [<ffff828c80127e65>] serial_rx+0x89/0x8e
>>> (XEN)    [<ffff828c8012924e>] serial_rx_interrupt+0xa9/0xbf
>>> (XEN)    [<ffff828c8012885c>] ns16550_interrupt+0x47/0x61
>>> (XEN)    [<ffff828c80148d99>] do_IRQ+0x3e7/0x42e
>>> (XEN)    [<ffff828c801423b6>] common_interrupt+0x26/0x30
>>> (XEN)    [<ffff828c801791df>] acpi_processor_idle+0x1fe/0x42f
>>> (XEN)    [<ffff828c8013e7db>] idle_loop+0xbe/0xc5
>>> (XEN)
>>> (XEN)
>>> (XEN) ****************************************
>>> (XEN) Panic on CPU 5:
>>> (XEN) Assertion ''((lock)->lock <= 0)''
failed at
>>> /var/tmp/portage/app-emulation/xen-3.4.2/work/xen-:18
>>> (XEN) ****************************************
>>> (XEN)
>>> (XEN) Reboot in five seconds...
>>>
>>>
>>> Andy
>>>
>>
>> I tried ''d'' again on the same supermicro system and
triggered a
>> similar crash and lockup, I then tried it on my optiplex test system
>> which had a debug=n Xen and it did not crash, however after
>> recompiling Xen on the Optiplex with debug=y ''d''
triggers the
>> watchdog, so perhaps the problem is similar to the one I found
>> recently where dumping dom0 registers trigged the watchdog.
>>
>> Both systems were dead locked after the crash, they did not reboot
>> automatically.
>>
>> The supermicro has a 38400 baud physical serial port, the Optiplex has
>> a intel amt virtual serial port.
>>
>> I will try adding a watchdog disable/enable around the dump registers
>> and test again on both systems, I will also test if the supermicro
>> system crashes when running a non debug Xen, perhaps this only happens
>> with debug builds.
>>
>> Logs attached.
>>
>> Andy
>>
>
> Ian,
>
> Sorry, I''ve got a head full of cold and got completely confused
> between dump_registers and dump_dom0_registers, I''ve already
submited
> a patch to fix the ''d'' problem and it was merged!
>
> Andy
> ,
>
The patch I submitted recently to disable watchdog around
dump_registers fixes the crash on both systems, I believe the Optiplex
was only affected when running a debug kernel because that increased
the amount of output from ''d'', where the supermicro had so
many cpu''s
that regardless of debug build or not the serial console could not
keep up and the watchdog was triggered.

I''ve created my own ebuilds for 3.4.2 which include this fix and will
try to get the Gentoo maintainer to include them too, I ended up
running without my fix applied because I had upgraded gcc and
recompiled Xen using the in-portage build.

Andy

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

John Anderson

2010-Jan-06 01:06 UTC

head link

RE: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

I''ve put some xen_raw_printk statements in xen_setup_kernel_pagetable,
and I''m not seeing the output on the console, even when I use on_crash
= ''preserve''.  So the crash may be happening sooner than that.
I''ve also enabled domU crash dumps in xend, but when this crash occurs,
the core file is 0 bytes and labeled ''-incomplete''.  Could the
crash be happening too early to get a core dump?  Also, is there anything like
ksymoops which I can use to interpret the stack trace and find the function
where the crash is occurring?

Thanks, 

John A.

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Keir Fraser
Sent: Monday, January 04, 2010 2:45 PM
To: Andrew Lyon; Ian Campbell; Xen-devel
Subject: Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

On 04/01/2010 19:13, "Andrew Lyon" <andrew.lyon@gmail.com>
wrote:
> Daft question, how can I check if the Xen I am running was build with
> debug=y or not?
>>>> (XEN) Unhandled page fault in domain 26 on VCPU 0 (ec=0000)
>>>> (XEN) Pagetable walk from 0000000000000028:
>>>> (XEN)  L4[0x000] = 0000000000000000 ffffffffffffffff
>>>> (XEN) domain_crash_sync called from entry.S
>>>> (XEN) Domain 26 (vcpu#0) crashed on cpu#4:
>>>> (XEN) ----[ Xen-3.1.3  x86_64  debug=y  Not tainted ]----                                    ^^^^^^^

 -- Keir

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Dec 2009 - PaX Security w/ Kernel.org DomU 2.6.31.7

[Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

Re: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7

RE: [Xen-devel] PaX Security w/ Kernel.org DomU 2.6.31.7