Xen devel - Sep 2010 - pvops0 kernel 2.6.32.21 does not boot

Hello,

current git stable 2.6.32.x (2968b25) crashes on boot with the attached 
debug output. The same config file works perfectly on 2.6.32.18 (e6b9b2c).

I manually created a traceback from the stack dump:

   get_phys_to_machine
   pte_pfn_to_mfn
   xen_make_pgd
   __raw_callee_save_xen_make_pgd
   vmemmap_pgd_populate
   vmemmap_populate
   sparse_mem_map_populate
   sparse_init

Andreas



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 09/13/2010 06:44 AM, Andreas Kinzler wrote:
> Hello,
>
> current git stable 2.6.32.x (2968b25) crashes on boot with the
> attached debug output. The same config file works perfectly on
> 2.6.32.18 (e6b9b2c).
Thanks for the report.  I''ve pushed an update to xen/stable-2.6.32.x
which should fix this.

    J
>
> I manually created a traceback from the stack dump:
>
>   get_phys_to_machine
>   pte_pfn_to_mfn
>   xen_make_pgd
>   __raw_callee_save_xen_make_pgd
>   vmemmap_pgd_populate
>   vmemmap_populate
>   sparse_mem_map_populate
>   sparse_init
>
> Andreas
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 14.09.2010 00:05, Jeremy Fitzhardinge wrote:
>> current git stable 2.6.32.x (2968b25) crashes on boot with the
>> attached debug output. The same config file works perfectly on
>> 2.6.32.18 (e6b9b2c).
> Thanks for the report.  I''ve pushed an update to
xen/stable-2.6.32.x
> which should fix this.
Sorry, still does not boot. I attached the serial console log and 
manually created a traceback from the stack hex dump.

get_phys_to_machine
pte_pfn_to_mfn
xen_make_pte
__raw_callee_save_xen_make_pte
__pte
pfn_pte
__early_set_fixmap
early_set_fixmap
early_memremap
alloc_low_page
phys_pud_init
kernel_physical_mapping_init

Andreas


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 09/15/2010 03:41 AM, Andreas Kinzler wrote:
> On 14.09.2010 00:05, Jeremy Fitzhardinge wrote:
>>> current git stable 2.6.32.x (2968b25) crashes on boot with the
>>> attached debug output. The same config file works perfectly on
>>> 2.6.32.18 (e6b9b2c).
>> Thanks for the report.  I''ve pushed an update to
xen/stable-2.6.32.x
>> which should fix this.
>
> Sorry, still does not boot. I attached the serial console log and
> manually created a traceback from the stack hex dump.
Does adding "mem=1g" to the kernel command line help?

    J
>
> get_phys_to_machine
> pte_pfn_to_mfn
> xen_make_pte
> __raw_callee_save_xen_make_pte
> __pte
> pfn_pte
> __early_set_fixmap
> early_set_fixmap
> early_memremap
> alloc_low_page
> phys_pud_init
> kernel_physical_mapping_init
>
> Andreas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 15.09.2010 19:34, Jeremy Fitzhardinge wrote:
>>>> current git stable 2.6.32.x (2968b25) crashes on boot with the
>>>> attached debug output. The same config file works perfectly on
>>>> 2.6.32.18 (e6b9b2c).
>>> Thanks for the report.  I''ve pushed an update to
xen/stable-2.6.32.x
>>> which should fix this.
>> Sorry, still does not boot. I attached the serial console log and
>> manually created a traceback from the stack hex dump.
> Does adding "mem=1g" to the kernel command line help?
No (log attached). But the stacktrace changed back to the first
"version":

   get_phys_to_machine
   pte_pfn_to_mfn
   xen_make_pgd
   __raw_callee_save_xen_make_pgd
   vmemmap_pgd_populate
   vmemmap_populate
   sparse_mem_map_populate
   sparse_init

Regards Andreas


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 09/15/2010 11:39 AM, Andreas Kinzler wrote:
> On 15.09.2010 19:34, Jeremy Fitzhardinge wrote:
>>>>> current git stable 2.6.32.x (2968b25) crashes on boot with
the
>>>>> attached debug output. The same config file works perfectly
on
>>>>> 2.6.32.18 (e6b9b2c).
>>>> Thanks for the report.  I''ve pushed an update to
xen/stable-2.6.32.x
>>>> which should fix this.
>>> Sorry, still does not boot. I attached the serial console log and
>>> manually created a traceback from the stack hex dump.
>> Does adding "mem=1g" to the kernel command line help?
>
> No (log attached). But the stacktrace changed back to the first
> "version":
>
>   get_phys_to_machine
>   pte_pfn_to_mfn
>   xen_make_pgd
>   __raw_callee_save_xen_make_pgd
>   vmemmap_pgd_populate
>   vmemmap_populate
>   sparse_mem_map_populate
>   sparse_init
What''s your kernel config?

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 15.09.2010 22:31, Jeremy Fitzhardinge wrote:
>   On 09/15/2010 11:39 AM, Andreas Kinzler wrote:
>> On 15.09.2010 19:34, Jeremy Fitzhardinge wrote:
>>>>>> current git stable 2.6.32.x (2968b25) crashes on boot
with the
>>>>>> attached debug output. The same config file works
perfectly on
>>>>>> 2.6.32.18 (e6b9b2c).
>>>>> Thanks for the report.  I''ve pushed an update to
xen/stable-2.6.32.x
>>>>> which should fix this.
>>>> Sorry, still does not boot. I attached the serial console log
and
>>>> manually created a traceback from the stack hex dump.
>>> Does adding "mem=1g" to the kernel command line help?
>>
>> No (log attached). But the stacktrace changed back to the first
>> "version":
>>
>>    get_phys_to_machine
>>    pte_pfn_to_mfn
>>    xen_make_pgd
>>    __raw_callee_save_xen_make_pgd
>>    vmemmap_pgd_populate
>>    vmemmap_populate
>>    sparse_mem_map_populate
>>    sparse_init
> What''s your kernel config?
Still the one from the original post 
(http://lists.xensource.com/archives/html/xen-devel/2010-09/msg00758.html).

Regards Andreas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hello Jeremy,
> On 15.09.2010 22:31, Jeremy Fitzhardinge wrote:
>> On 09/15/2010 11:39 AM, Andreas Kinzler wrote:
>>> On 15.09.2010 19:34, Jeremy Fitzhardinge wrote:
>>>>>>> current git stable 2.6.32.x (2968b25) crashes on
boot with the
>>>>>>> attached debug output. The same config file works
perfectly on
>>>>>>> 2.6.32.18 (e6b9b2c).
>>>>>> Thanks for the report. I''ve pushed an update
to xen/stable-2.6.32.x
>>>>>> which should fix this.
>>>>> Sorry, still does not boot. I attached the serial console
log and
>>>>> manually created a traceback from the stack hex dump.
>>>> Does adding "mem=1g" to the kernel command line help?
>>> No (log attached). But the stacktrace changed back to the first
>>> "version":
>>>
>>> get_phys_to_machine
>>> pte_pfn_to_mfn
>>> xen_make_pgd
>>> __raw_callee_save_xen_make_pgd
>>> vmemmap_pgd_populate
>>> vmemmap_populate
>>> sparse_mem_map_populate
>>> sparse_init
>> What''s your kernel config?
> Still the one from the original post
> (http://lists.xensource.com/archives/html/xen-devel/2010-09/msg00758.html).
Are you still checking this? Any news?

Regards Andreas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 09/22/2010 09:17 AM, Andreas Kinzler wrote:
> Hello Jeremy,
>
>> On 15.09.2010 22:31, Jeremy Fitzhardinge wrote:
>>> On 09/15/2010 11:39 AM, Andreas Kinzler wrote:
>>>> On 15.09.2010 19:34, Jeremy Fitzhardinge wrote:
>>>>>>>> current git stable 2.6.32.x (2968b25) crashes
on boot with the
>>>>>>>> attached debug output. The same config file
works perfectly on
>>>>>>>> 2.6.32.18 (e6b9b2c).
>>>>>>> Thanks for the report. I''ve pushed an
update to xen/stable-2.6.32.x
>>>>>>> which should fix this.
>>>>>> Sorry, still does not boot. I attached the serial
console log and
>>>>>> manually created a traceback from the stack hex dump.
>>>>> Does adding "mem=1g" to the kernel command line
help?
>>>> No (log attached). But the stacktrace changed back to the first
>>>> "version":
>>>>
>>>> get_phys_to_machine
>>>> pte_pfn_to_mfn
>>>> xen_make_pgd
>>>> __raw_callee_save_xen_make_pgd
>>>> vmemmap_pgd_populate
>>>> vmemmap_populate
>>>> sparse_mem_map_populate
>>>> sparse_init
>>> What''s your kernel config?
>> Still the one from the original post
>>
(http://lists.xensource.com/archives/html/xen-devel/2010-09/msg00758.html).
>>
>
> Are you still checking this? Any news?
>
Haven''t forgotten about it, but I was waiting for some inspiration to
strike.  Could you try some earlier versions and try to isolate when the
problem appeared?

Thanks,
    J


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 22.09.2010 19:23, Jeremy Fitzhardinge wrote:
>>>>>>>>> current git stable 2.6.32.x (2968b25)
crashes on boot with the
>>>>>>>>> attached debug output. The same config file
works perfectly on
>>>>>>>>> 2.6.32.18 (e6b9b2c).
>>>>>>>> Thanks for the report. I''ve pushed an
update to xen/stable-2.6.32.x
>>>>>>>> which should fix this.
>>>>>>> Sorry, still does not boot. I attached the serial
console log and
>>>>>>> manually created a traceback from the stack hex
dump.
>>>>>> Does adding "mem=1g" to the kernel command
line help?
>>>>> No (log attached). But the stacktrace changed back to the
first
>>>>> "version":
>> Are you still checking this? Any news?
> Haven''t forgotten about it, but I was waiting for some inspiration
to
> strike.  Could you try some earlier versions and try to isolate when the
> problem appeared?
Actually I found something very strange. Even the vanilla kernels from 
kernel.org stopped working as a domU with the transition of 2.6.32.21 to 
2.6.32.22. No output, no log, no nothing, just eating CPU time. Are 
there any debug options how to debug domUs that don''t output anything?

I am actually starting to wonder if the problems are related to kernels 
at all. I tried Xen 3.4.4 with the same result. Perhaps the whole 
problem is a binutils/gcc problem. What is your version of binutils / 
gcc / glibc?

Regards Andreas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 10/01/2010 03:53 AM, Andreas Kinzler wrote:
> On 22.09.2010 19:23, Jeremy Fitzhardinge wrote:
>>>>>>>>>> current git stable 2.6.32.x (2968b25)
crashes on boot with the
>>>>>>>>>> attached debug output. The same config
file works perfectly on
>>>>>>>>>> 2.6.32.18 (e6b9b2c).
>>>>>>>>> Thanks for the report. I''ve pushed
an update to
>>>>>>>>> xen/stable-2.6.32.x
>>>>>>>>> which should fix this.
>>>>>>>> Sorry, still does not boot. I attached the
serial console log and
>>>>>>>> manually created a traceback from the stack hex
dump.
>>>>>>> Does adding "mem=1g" to the kernel
command line help?
>>>>>> No (log attached). But the stacktrace changed back to
the first
>>>>>> "version":
>>> Are you still checking this? Any news?
>> Haven''t forgotten about it, but I was waiting for some
inspiration to
>> strike.  Could you try some earlier versions and try to isolate when
the
>> problem appeared?
>
> Actually I found something very strange. Even the vanilla kernels from
> kernel.org stopped working as a domU with the transition of 2.6.32.21
> to 2.6.32.22. No output, no log, no nothing, just eating CPU time. Are
> there any debug options how to debug domUs that don''t output
anything?
>
> I am actually starting to wonder if the problems are related to
> kernels at all. I tried Xen 3.4.4 with the same result. Perhaps the
> whole problem is a binutils/gcc problem. What is your version of
> binutils / gcc / glibc?
Vanilla upstream 2.6.32.22 and 23 are broken due to a typo that crept
into a Xen patch between mainline and stable-2.6.32; I''m still waiting
for that fix to hit (stable-2.6.35 was also affected, but it has since
been fixed).

Change b0dd220aefe1a is the fix, included below.

And on the subject of the original bug, I committed some fixes which may
fix it for you (certainly fixed *some* memory init stuff), so if you
could test out xen/next-2.6.32 I''d appreciate it.

Thanks,
    J

From: James Dingwall <james.dingwall@amdocs.com>
Date: Mon, 27 Sep 2010 09:37:17 +0100
Subject: [PATCH] Xen: fix typo in previous patch

Correctly name the irq_chip structure to fix an immediate failure when booting
as a xen pv_ops guest with a NULL pointer exception. The missing
''x'' was
introduced in commit [fb412a178502dc498430723b082a932f797e4763] applied to
2.6.3[25]-stable trees.  The commit to mainline was
[aaca49642b92c8a57d3ca5029a5a94019c7af69f] which did not have the problem.

Signed-off-by: James Dingwall <james@dingwall.me.uk>
Reported-by: Pawel Zuzelski <pawelz@pld-linux.org>
Tested-by: Pawel Zuzelski <pawelz@pld-linux.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 14ed358..6830a4d 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -935,7 +935,7 @@ static struct irq_chip xen_dynamic_chip __read_mostly = {
 	.retrigger	= retrigger_dynirq,
 };
 
-static struct irq_chip en_percpu_chip __read_mostly = {
+static struct irq_chip xen_percpu_chip __read_mostly = {
 	.name		= "xen-percpu",
 
 	.disable	= disable_dynirq,




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel