Hello,
I tried to get hypervisor built without debug=y option.
Unfortunately built binary doesn''t boot.
I''ve localized a problem and get a workaround for it:
diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
index d1290cd..aa8e057 100644
--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -332,7 +332,7 @@ void __cpuinit setup_virt_paging(void)
/* Boot-time pagetable setup.
* Changes here may need matching changes in head.S */
-void __init setup_pagetables(unsigned long boot_phys_offset, paddr_t
xen_paddr)
+void __attribute__((optimize("-fno-omit-frame-pointer"))) __init
setup_pagetables(unsigned long boot_phys_offset, paddr_t xen_paddr)
{
unsigned long dest_va;
lpae_t pte, *p;
Is it a known issue? Is there a better solution for it?
I work with 4.3 release, build with gcc version 4.6.3 (Ubuntu/Linaro
4.6.3-1ubuntu5)
Sincerely,
Andrii Anisov.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
On Mon, 2013-08-19 at 14:44 +0300, Andrii Anisov wrote:> Hello, > > > I tried to get hypervisor built without debug=y option. > Unfortunately built binary doesn''t boot. > I''ve localized a problem and get a workaround for it:Thanks.> diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c > index d1290cd..aa8e057 100644 > --- a/xen/arch/arm/mm.c > +++ b/xen/arch/arm/mm.c > @@ -332,7 +332,7 @@ void __cpuinit setup_virt_paging(void) > > /* Boot-time pagetable setup. > * Changes here may need matching changes in head.S */ > -void __init setup_pagetables(unsigned long boot_phys_offset, paddr_t xen_paddr) > +void __attribute__((optimize("-fno-omit-frame-pointer"))) __init setup_pagetables(unsigned long boot_phys_offset, paddr_t xen_paddr) > { > unsigned long dest_va; > lpae_t pte, *p; > > > Is it a known issue?I suspect none of us has ever really used a non-debug build!> Is there a better solution for it?So the confusing double negative -fno-omit-frame-pointer is causing the frame pointer to be omitted for this function? I wonder why the frame pointer is a problem here. This function is a bit sensitive since it relocates Xen in physical RAM but it doesn''t change the virtual address space so I would expect fp, sp etc to remain valid. The danger is if something is written to the stack between the initial copy and the switch over, but I''d have thought that the frame pointer would be written to the stack near the start and popped at the end and mostly not touched in the middle. I guess what I''m trying to say is I''m not confident that adding that flag has "fixed" the issue, rather than just causing the compiler to behave a bit differently and avoid it. Are you able to post any details about the boot failure? Is it a crash or something more subtle? Are you able to localise it to a particular instruction or section of code?> I work with 4.3 release, build with gcc version 4.6.3 (Ubuntu/Linaro > 4.6.3-1ubuntu5)For ARM stuff you are probably better off tracking mainline (unstable) xen. Depending on what you are doing of course. Ian.
> > So the confusing double negative -fno-omit-frame-pointer is causing the > frame pointer to be omitted for this function?Not exactly, debug=y forces -fno-omit-frame-pointer into CFLAG. Without debug, it is built without this option. I just forced -fno-omit-frame-pointer for the particular function.> I wonder why the frame pointer is a problem here. This function is a bit > sensitive since it relocates Xen in physical RAM but it doesn''t change > the virtual address space so I would expect fp, sp etc to remain valid. > > The danger is if something is written to the stack between the initial > copy and the switch over, but I''d have thought that the frame pointer > would be written to the stack near the start and popped at the end and > mostly not touched in the middle. >Well, I roughly compared the function dumps for both cases (with frame pointer and without), and noticed more aggressive pc-related data manipulation if built without fp. This could be a cause, f.e. if data modified in the old place, while than read from new location. I guess what I''m trying to say is I''m not confident that adding that> flag has "fixed" the issue, rather than just causing the compiler to > behave a bit differently and avoid it.I haven''t root caused the issue so I do not affirm it''s a fix. It''s just a WA works for me.> Are you able to post any details about the boot failure? Is it a crash > or something more subtle? Are you able to localise it to a particular > instruction or section of code?Unfortunately I can''t localize particular instruction. JTAG I have does not allow stepping through and use HW BP in HYP mode. With SW BP I just localized an issue in this function, but it isn''t right tool to debug that relocation stuff. Sincerely, Andrii Anisov. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
BTW,> Is it a known issue? > > I suspect none of us has ever really used a non-debug build! >Wiki says: Once you have a suitable cross compiler you can compile Xen with:> $ make dist-xen XEN_TARGET_ARCH=arm32 > CROSS_COMPILE=arm-unknown-linux-gnueabi- > or: > $ make dist-xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-Without debug=y flag. Sincerely, Andrii Anisov. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On 08/20/2013 05:03 PM, Andrii Anisov wrote:> BTW, > > > Is it a known issue? > > I suspect none of us has ever really used a non-debug build! > > > Wiki says: > > Once you have a suitable cross compiler you can compile Xen with: > $ make dist-xen XEN_TARGET_ARCH=arm32 > CROSS_COMPILE=arm-unknown-linux-gnueabi- > or: > $ make dist-xen XEN_TARGET_ARCH=arm64 > CROSS_COMPILE=aarch64-linux-gnu- > > > Without debug=y flag.On unstable branch by default debug=y. On release branches have debug=n by default. I have tried to boot a non-debug Xen on the Versatile Express: gcc 4.8.1: Xen boots gcc 4.6.3: Xen hangs with no output log (even early printk) Cheers, -- Julien Grall
On 08/20/2013 05:59 PM, Julien Grall wrote:> On 08/20/2013 05:03 PM, Andrii Anisov wrote: >> BTW, >> >> > Is it a known issue? >> >> I suspect none of us has ever really used a non-debug build! >> >> >> Wiki says: >> >> Once you have a suitable cross compiler you can compile Xen with: >> $ make dist-xen XEN_TARGET_ARCH=arm32 >> CROSS_COMPILE=arm-unknown-linux-gnueabi- >> or: >> $ make dist-xen XEN_TARGET_ARCH=arm64 >> CROSS_COMPILE=aarch64-linux-gnu- >> >> >> Without debug=y flag. > > On unstable branch by default debug=y. On release branches have debug=n > by default. > > I have tried to boot a non-debug Xen on the Versatile Express: > gcc 4.8.1: Xen boots > gcc 4.6.3: Xen hangs with no output log (even early printk)I was wrong, when debug=n it''s not possible to have early printk. So Xen only hangs. But, if I enable early printk (by hacking xen/arch/arm/Rules.mk) Xen is able to boot... -- Julien Grall
> > But, if I enable early printk (by hacking xen/arch/arm/Rules.mk) Xen is > able to boot... >Yep, I''ve seen this as well. But we are experimenting with boot time so early printk is not an option for us. Disabling debug gave us -30ms from 600 ms. Sincerely, Andrii Anisov. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel