thr3ads.net - Xen devel - [Xen-devel] Linux 2.6.33 crashes on boot as Xen PV domU [Feb 2010]

If this information is useful, please help other people find it:
Share via:

Pasi Kärkkäinen

2010-Feb-25 19:04 UTC

[Xen-devel] Linux 2.6.33 crashes on boot as Xen PV domU

Hello,

I just built and tried to boot upstream kernel.org Linux 2.6.33 kernel
as Xen PV domU, but that doesn''t get very far:

http://pasik.reaktio.net/xen/debug/bootlog-linux-2.6.33-xen-pv-domu-x86_64-crash.txt


Freeing unused kernel memory: 1544k freed
Write protecting the kernel read-only data: 10240k
Freeing unused kernel memory: 1764k freed
BUG: unable to handle kernel paging request at ffff880001447000
IP: [<ffffffff8102e9f2>] free_init_pages+0xb2/0xdb
PGD 1a3c067 PUD 1a40067 PMD 138d5067 PTE 10000001447025
Oops: 0003 [#1] SMP 
last sysfs file: 
CPU 3 
Pid: 1, comm: swapper Not tainted 2.6.33 #1 /
RIP: e030:[<ffffffff8102e9f2>]  [<ffffffff8102e9f2>]
free_init_pages+0xb2/0xdb
RSP: e02b:ffff88007dfdbe60  EFLAGS: 00010286
RAX: 00000000cccccccc RBX: ffff880001600000 RCX: 0000000000000400
RDX: ffff880001447000 RSI: 0000000000000000 RDI: ffff880001447000
RBP: ffff88007dfdbe90 R08: 0000000000000000 R09: ffff88007fc04000
R10: ffff88007fc04000 R11: 0000000000100000 R12: ffff880001447000
R13: 0000000000000400 R14: ffffea0000000000 R15: 00000000cccccccc
FS:  0000000000000000(0000) GS:ffff8800139d6000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff880001447000 CR3: 0000000001a3b000 CR4: 0000000000002620
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000000
Process swapper (pid: 1, threadinfo ffff88007dfda000, task ffff88007dfe0000)
Stack:
 0000000000000000 ffff880000000000 6db6db6db6db6db7 ffffffff81a00000
<0> 0000000000a00000 0000000000000000 ffff88007dfdbec0 ffffffff8102ed73
<0> ffffffff81c6aa38 ffffffff81aefdf0 0000000000000100 0000000000000100
Call Trace:
 [<ffffffff8102ed73>] mark_rodata_ro+0xea/0x151
 [<ffffffff810021b9>] init_post+0x30/0x113
 [<ffffffff81b0f715>] kernel_init+0x1c3/0x1d2
 [<ffffffff8100aa64>] kernel_thread_helper+0x4/0x10
 [<ffffffff81009e91>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff8143ae1d>] ? retint_restore_args+0x5/0x6
 [<ffffffff8100aa60>] ? kernel_thread_helper+0x0/0x10
Code: cd 47 00 00 48 c1 e8 0c 4c 89 e2 4c 89 e9 48 6b c0 38 48 81 e2 00 f0 ff ff
31 f6 48 89 d7 4c 01 f0 c7 40 08 01 00 00 00 44 89 f8 <f3> ab 4c 89 e7 49
81 c4 00 10 00 00 e8 bc ca 09 00 48 ff 05 16
RIP  [<ffffffff8102e9f2>] free_init_pages+0xb2/0xdb
 RSP <ffff88007dfdbe60>
CR2: ffff880001447000
---[ end trace 6e676731d52211fa ]---
Kernel panic - not syncing: Attempted to kill init!
Pid: 1, comm: swapper Tainted: G      D    2.6.33 #1
Call Trace:
 [<ffffffff81438663>] panic+0x7a/0x13d
 [<ffffffff81057609>] ? exit_ptrace+0xa1/0x121
 [<ffffffff8105074d>] do_exit+0x7a/0x6f3
 [<ffffffff8104d15d>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff8104dd76>] ? kmsg_dump+0x12b/0x145
 [<ffffffff8143bc31>] oops_end+0xbf/0xc7
 [<ffffffff8102f901>] no_context+0x1fc/0x20b
 [<ffffffff8102fa94>] __bad_area_nosemaphore+0x184/0x1a7
 [<ffffffff81004399>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
 [<ffffffff8102faca>] bad_area_nosemaphore+0x13/0x15
 [<ffffffff8143d663>] do_page_fault+0x14f/0x2a0
 [<ffffffff8143b0b5>] page_fault+0x25/0x30
 [<ffffffff8102e9f2>] ? free_init_pages+0xb2/0xdb
 [<ffffffff8102ed73>] mark_rodata_ro+0xea/0x151
 [<ffffffff810021b9>] init_post+0x30/0x113
 [<ffffffff81b0f715>] kernel_init+0x1c3/0x1d2
 [<ffffffff8100aa64>] kernel_thread_helper+0x4/0x10
 [<ffffffff81009e91>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff8143ae1d>] ? retint_restore_args+0x5/0x6
 [<ffffffff8100aa60>] ? kernel_thread_helper+0x0/0x10

-- Pasi


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2010-Feb-25 19:10 UTC

head link

Re: [Xen-devel] Linux 2.6.33 crashes on boot as Xen PV domU

On 02/25/2010 11:04 AM, Pasi Kärkkäinen wrote:> Hello,
>
> I just built and tried to boot upstream kernel.org Linux 2.6.33 kernel
> as Xen PV domU, but that doesn''t get very far:
>
>
http://pasik.reaktio.net/xen/debug/bootlog-linux-2.6.33-xen-pv-domu-x86_64-crash.txt
>    
Try the attached patch.

     J
>
> Freeing unused kernel memory: 1544k freed
> Write protecting the kernel read-only data: 10240k
> Freeing unused kernel memory: 1764k freed
> BUG: unable to handle kernel paging request at ffff880001447000
> IP: [<ffffffff8102e9f2>] free_init_pages+0xb2/0xdb
> PGD 1a3c067 PUD 1a40067 PMD 138d5067 PTE 10000001447025
> Oops: 0003 [#1] SMP
> last sysfs file:
> CPU 3
> Pid: 1, comm: swapper Not tainted 2.6.33 #1 /
> RIP: e030:[<ffffffff8102e9f2>]  [<ffffffff8102e9f2>]
free_init_pages+0xb2/0xdb
> RSP: e02b:ffff88007dfdbe60  EFLAGS: 00010286
> RAX: 00000000cccccccc RBX: ffff880001600000 RCX: 0000000000000400
> RDX: ffff880001447000 RSI: 0000000000000000 RDI: ffff880001447000
> RBP: ffff88007dfdbe90 R08: 0000000000000000 R09: ffff88007fc04000
> R10: ffff88007fc04000 R11: 0000000000100000 R12: ffff880001447000
> R13: 0000000000000400 R14: ffffea0000000000 R15: 00000000cccccccc
> FS:  0000000000000000(0000) GS:ffff8800139d6000(0000)
knlGS:0000000000000000
> CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: ffff880001447000 CR3: 0000000001a3b000 CR4: 0000000000002620
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000000
> Process swapper (pid: 1, threadinfo ffff88007dfda000, task
ffff88007dfe0000)
> Stack:
>   0000000000000000 ffff880000000000 6db6db6db6db6db7 ffffffff81a00000
> <0>  0000000000a00000 0000000000000000 ffff88007dfdbec0
ffffffff8102ed73
> <0>  ffffffff81c6aa38 ffffffff81aefdf0 0000000000000100
0000000000000100
> Call Trace:
>   [<ffffffff8102ed73>] mark_rodata_ro+0xea/0x151
>   [<ffffffff810021b9>] init_post+0x30/0x113
>   [<ffffffff81b0f715>] kernel_init+0x1c3/0x1d2
>   [<ffffffff8100aa64>] kernel_thread_helper+0x4/0x10
>   [<ffffffff81009e91>] ? int_ret_from_sys_call+0x7/0x1b
>   [<ffffffff8143ae1d>] ? retint_restore_args+0x5/0x6
>   [<ffffffff8100aa60>] ? kernel_thread_helper+0x0/0x10
> Code: cd 47 00 00 48 c1 e8 0c 4c 89 e2 4c 89 e9 48 6b c0 38 48 81 e2 00 f0
ff ff 31 f6 48 89 d7 4c 01 f0 c7 40 08 01 00 00 00 44 89 f8<f3>  ab 4c 89
e7 49 81 c4 00 10 00 00 e8 bc ca 09 00 48 ff 05 16
> RIP  [<ffffffff8102e9f2>] free_init_pages+0xb2/0xdb
>   RSP<ffff88007dfdbe60>
> CR2: ffff880001447000
> ---[ end trace 6e676731d52211fa ]---
> Kernel panic - not syncing: Attempted to kill init!
> Pid: 1, comm: swapper Tainted: G      D    2.6.33 #1
> Call Trace:
>   [<ffffffff81438663>] panic+0x7a/0x13d
>   [<ffffffff81057609>] ? exit_ptrace+0xa1/0x121
>   [<ffffffff8105074d>] do_exit+0x7a/0x6f3
>   [<ffffffff8104d15d>] ? spin_unlock_irqrestore+0xe/0x10
>   [<ffffffff8104dd76>] ? kmsg_dump+0x12b/0x145
>   [<ffffffff8143bc31>] oops_end+0xbf/0xc7
>   [<ffffffff8102f901>] no_context+0x1fc/0x20b
>   [<ffffffff8102fa94>] __bad_area_nosemaphore+0x184/0x1a7
>   [<ffffffff81004399>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
>   [<ffffffff8102faca>] bad_area_nosemaphore+0x13/0x15
>   [<ffffffff8143d663>] do_page_fault+0x14f/0x2a0
>   [<ffffffff8143b0b5>] page_fault+0x25/0x30
>   [<ffffffff8102e9f2>] ? free_init_pages+0xb2/0xdb
>   [<ffffffff8102ed73>] mark_rodata_ro+0xea/0x151
>   [<ffffffff810021b9>] init_post+0x30/0x113
>   [<ffffffff81b0f715>] kernel_init+0x1c3/0x1d2
>   [<ffffffff8100aa64>] kernel_thread_helper+0x4/0x10
>   [<ffffffff81009e91>] ? int_ret_from_sys_call+0x7/0x1b
>   [<ffffffff8143ae1d>] ? retint_restore_args+0x5/0x6
>   [<ffffffff8100aa60>] ? kernel_thread_helper+0x0/0x10
>
> -- Pasi
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>    


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Pasi Kärkkäinen

2010-Feb-25 19:35 UTC

head link

Re: [Xen-devel] Linux 2.6.33 crashes on boot as Xen PV domU

On Thu, Feb 25, 2010 at 11:10:45AM -0800, Jeremy Fitzhardinge
wrote:> On 02/25/2010 11:04 AM, Pasi Kärkkäinen wrote:
>> Hello,
>>
>> I just built and tried to boot upstream kernel.org Linux 2.6.33 kernel
>> as Xen PV domU, but that doesn''t get very far:
>>
>>
http://pasik.reaktio.net/xen/debug/bootlog-linux-2.6.33-xen-pv-domu-x86_64-crash.txt
>>    
>
> Try the attached patch.
>
Yep, this patch fixes the problem, boots OK now.

Thanks! Now some save/restore testing..

-- Pasi
>     J
>
>>
>> Freeing unused kernel memory: 1544k freed
>> Write protecting the kernel read-only data: 10240k
>> Freeing unused kernel memory: 1764k freed
>> BUG: unable to handle kernel paging request at ffff880001447000
>> IP: [<ffffffff8102e9f2>] free_init_pages+0xb2/0xdb
>> PGD 1a3c067 PUD 1a40067 PMD 138d5067 PTE 10000001447025
>> Oops: 0003 [#1] SMP
>> last sysfs file:
>> CPU 3
>> Pid: 1, comm: swapper Not tainted 2.6.33 #1 /
>> RIP: e030:[<ffffffff8102e9f2>]  [<ffffffff8102e9f2>]
free_init_pages+0xb2/0xdb
>> RSP: e02b:ffff88007dfdbe60  EFLAGS: 00010286
>> RAX: 00000000cccccccc RBX: ffff880001600000 RCX: 0000000000000400
>> RDX: ffff880001447000 RSI: 0000000000000000 RDI: ffff880001447000
>> RBP: ffff88007dfdbe90 R08: 0000000000000000 R09: ffff88007fc04000
>> R10: ffff88007fc04000 R11: 0000000000100000 R12: ffff880001447000
>> R13: 0000000000000400 R14: ffffea0000000000 R15: 00000000cccccccc
>> FS:  0000000000000000(0000) GS:ffff8800139d6000(0000)
knlGS:0000000000000000
>> CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
>> CR2: ffff880001447000 CR3: 0000000001a3b000 CR4: 0000000000002620
>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000000
>> Process swapper (pid: 1, threadinfo ffff88007dfda000, task
ffff88007dfe0000)
>> Stack:
>>   0000000000000000 ffff880000000000 6db6db6db6db6db7 ffffffff81a00000
>> <0>  0000000000a00000 0000000000000000 ffff88007dfdbec0
ffffffff8102ed73
>> <0>  ffffffff81c6aa38 ffffffff81aefdf0 0000000000000100
0000000000000100
>> Call Trace:
>>   [<ffffffff8102ed73>] mark_rodata_ro+0xea/0x151
>>   [<ffffffff810021b9>] init_post+0x30/0x113
>>   [<ffffffff81b0f715>] kernel_init+0x1c3/0x1d2
>>   [<ffffffff8100aa64>] kernel_thread_helper+0x4/0x10
>>   [<ffffffff81009e91>] ? int_ret_from_sys_call+0x7/0x1b
>>   [<ffffffff8143ae1d>] ? retint_restore_args+0x5/0x6
>>   [<ffffffff8100aa60>] ? kernel_thread_helper+0x0/0x10
>> Code: cd 47 00 00 48 c1 e8 0c 4c 89 e2 4c 89 e9 48 6b c0 38 48 81 e2 00
f0 ff ff 31 f6 48 89 d7 4c 01 f0 c7 40 08 01 00 00 00 44 89 f8<f3>  ab 4c
89 e7 49 81 c4 00 10 00 00 e8 bc ca 09 00 48 ff 05 16
>> RIP  [<ffffffff8102e9f2>] free_init_pages+0xb2/0xdb
>>   RSP<ffff88007dfdbe60>
>> CR2: ffff880001447000
>> ---[ end trace 6e676731d52211fa ]---
>> Kernel panic - not syncing: Attempted to kill init!
>> Pid: 1, comm: swapper Tainted: G      D    2.6.33 #1
>> Call Trace:
>>   [<ffffffff81438663>] panic+0x7a/0x13d
>>   [<ffffffff81057609>] ? exit_ptrace+0xa1/0x121
>>   [<ffffffff8105074d>] do_exit+0x7a/0x6f3
>>   [<ffffffff8104d15d>] ? spin_unlock_irqrestore+0xe/0x10
>>   [<ffffffff8104dd76>] ? kmsg_dump+0x12b/0x145
>>   [<ffffffff8143bc31>] oops_end+0xbf/0xc7
>>   [<ffffffff8102f901>] no_context+0x1fc/0x20b
>>   [<ffffffff8102fa94>] __bad_area_nosemaphore+0x184/0x1a7
>>   [<ffffffff81004399>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
>>   [<ffffffff8102faca>] bad_area_nosemaphore+0x13/0x15
>>   [<ffffffff8143d663>] do_page_fault+0x14f/0x2a0
>>   [<ffffffff8143b0b5>] page_fault+0x25/0x30
>>   [<ffffffff8102e9f2>] ? free_init_pages+0xb2/0xdb
>>   [<ffffffff8102ed73>] mark_rodata_ro+0xea/0x151
>>   [<ffffffff810021b9>] init_post+0x30/0x113
>>   [<ffffffff81b0f715>] kernel_init+0x1c3/0x1d2
>>   [<ffffffff8100aa64>] kernel_thread_helper+0x4/0x10
>>   [<ffffffff81009e91>] ? int_ret_from_sys_call+0x7/0x1b
>>   [<ffffffff8143ae1d>] ? retint_restore_args+0x5/0x6
>>   [<ffffffff8100aa60>] ? kernel_thread_helper+0x0/0x10
>>
>> -- Pasi
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>    
>
> From: Suresh Siddha <suresh.b.siddha@intel.com>
> To: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: "linux-kernel@vger.kernel.org"
<linux-kernel@vger.kernel.org>,
> 	"hpa@zytor.com" <hpa@zytor.com>,
> 	"rostedt@goodmis.org" <rostedt@goodmis.org>,
> 	"jeremy@goop.org" <jeremy@goop.org>, Ingo Molnar
<mingo@redhat.com>,
> 	Thomas Gleixner <tglx@linutronix.de>
> Date: Thu, 18 Feb 2010 11:51:40 -0800
> Subject: Re: [LKML] Re: [PATCH] x86_64: allow sections that are recycled to
> 	set _PAGE_RW
> 
> On Tue, 2010-02-16 at 14:13 -0800, Konrad Rzeszutek Wilk wrote:
> > On Sat, Feb 13, 2010 at 12:08:17PM -0800, Suresh Siddha wrote:
> > > The checks in static_protections() for kernel text mapping ensure
that
> > > we don''t break the 2MB kernel text pages unnecessarily
on 64bit kernels
> > > (as it has performance implications). We should be fine as long
as the
> > > kernel identity mappings reflect the correct RW permissions.
> > > 
> > > But somehow this is working fine on native kernels but not on Xen
pv
> > > guest. Your patch will cause the performance issues that we are
> > 
> > That would not be good.
> > 
> > > addressing using the static protections checks. I will look at
this more
> > > detailed on tuesday.
> > 
> > Great. Thank you for doing that. If you find yourself in a bind, here
are
> > some steps on how to build the Xen pv-ops kernel and such:
> > http://wiki.xensource.com/xenwiki/XenParavirtOps
> > 
> > It goes without saying that I would be happy to test your patch when
> > you have one ready.
> 
> x86 folks, can you please queue the appended patch? If you think it is
> too late for 2.6.33, I added a "cc: stable", so that they can
pick this
> up for both .32 and .33. Thanks.
> ---
> 
> From: Suresh Siddha <suresh.b.siddha@intel.com>
> Subject: x86_64, cpa: don''t work hard in preserving kernel text 2M
mapping when using 4K already
> 
> We currently enforce the !RW mapping for the kernel mapping that maps
> holes between different text, rodata and data sections. However, kernel
> identity mappings will have different RWX permissions to the pages mapping
to
> text and to the pages padding (which are freed) the text, rodata sections.
> Hence kernel identity mappings will be broken to smaller pages. For 64-bit,
> kernel text and kernel identity mappings are different, so we can enable
> protection checks that come with CONFIG_DEBUG_RODATA, as well as retain 2MB
> large page mappings for kernel text.
> 
> Konrad reported a boot failure with the Linux Xen paravirt guest because of
> this. In this paravirt guest case, the kernel text mapping and the kernel
> identity mapping share the same page-table pages. Thus forcing the !RW
mapping
> for some of the kernel mappings also cause the kernel identity mappings to
be
> read-only resulting in the boot failure. Linux Xen paravirt guest also
> uses 4k mappings and don''t use 2M mapping.
> 
> Fix this issue and retain large page performance advantage for native
kernels
> by not working hard and not enforcing !RW for the kernel text mapping,
> if the current mapping is already using small page mapping.
> 
> Reported-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
> Tested-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: stable@kernel.org	[2.6.32, 2.6.33]
> ---
> 
> index 1d4eb93..cf07c26 100644
> --- a/arch/x86/mm/pageattr.c
> +++ b/arch/x86/mm/pageattr.c
> @@ -291,8 +291,29 @@ static inline pgprot_t static_protections(pgprot_t
prot, unsigned long address,
>  	 */
>  	if (kernel_set_to_readonly &&
>  	    within(address, (unsigned long)_text,
> -		   (unsigned long)__end_rodata_hpage_align))
> -		pgprot_val(forbidden) |= _PAGE_RW;
> +		   (unsigned long)__end_rodata_hpage_align)) {
> +		unsigned int level;
> +
> +		/*
> +		 * Don''t enforce the !RW mapping for the kernel text mapping,
> +		 * if the current mapping is already using small page mapping.
> +		 * No need to work hard to preserve large page mappings in this
> +		 * case.
> +		 *
> +		 * This also fixes the Linux Xen paravirt guest boot failure
> +		 * (because of unexpected read-only mappings for kernel identity
> +		 * mappings). In this paravirt guest case, the kernel text
> +		 * mapping and the kernel identity mapping share the same
> +		 * page-table pages. Thus we can''t really use different
> +		 * protections for the kernel text and identity mappings. Also,
> +		 * these shared mappings are made of small page mappings.
> +		 * Thus this don''t enforce !RW mapping for small page kernel
> +		 * text mapping logic will help Linux Xen parvirt guest boot
> +		 * aswell.
> +		 */
> +		if (lookup_address(address, &level) && (level !=
PG_LEVEL_4K))
> +			pgprot_val(forbidden) |= _PAGE_RW;
> +	}
>  #endif
>  
>  	prot = __pgprot(pgprot_val(prot) & ~pgprot_val(forbidden));
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Feb 2010 - Linux 2.6.33 crashes on boot as Xen PV domU

[Xen-devel] Linux 2.6.33 crashes on boot as Xen PV domU

Re: [Xen-devel] Linux 2.6.33 crashes on boot as Xen PV domU

Re: [Xen-devel] Linux 2.6.33 crashes on boot as Xen PV domU