Hi, I was wondering if there is any documentation on how to write a bare metal application for Xen. I don''t need to parse the device tree and such yet, a simple booting "Hello World" would be fine :-) We wrote one and when trying to boot we get ( this was an uncompressed binary, no image): libxl: notice: libxl_numa.c:451:libxl__get_numa_candidate: NUMA placement failed, performance might be affected xc: error: panic: xc_dom_core.c:587: xc_dom_find_loader: no loader found: Invalid kernel libxl: error: libxl_dom.c:380:libxl__build_pv: xc_dom_parse_image failed: No such file or directory libxl: error: libxl_create.c:908:domcreate_rebuild_done: cannot (re-)build domain: -3 I attached the code ( it''s very small). Thanks _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org lists.xen.org/xen-devel
On Wed, 2013-04-24 at 10:39 +0100, Sander Bogaert wrote:> I was wondering if there is any documentation on how to write a bare > metal application for Xen. I don''t need to parse the device tree and > such yet, a simple booting "Hello World" would be fine :-)> We wrote one and when trying to boot we get ( this was an uncompressed > binary, no image):> libxl: notice: libxl_numa.c:451:libxl__get_numa_candidate: NUMA placement failed, performance might be affected > xc: error: panic: xc_dom_core.c:587: xc_dom_find_loader: no loader found: Invalid kernel > libxl: error: libxl_dom.c:380:libxl__build_pv: xc_dom_parse_image failed: No such file or directory > libxl: error: libxl_create.c:908:domcreate_rebuild_done: cannot (re-)build domain: -3Are you trying to load test.elf or test.bin? A test.bin would need to have a Linux zImage header on the front to be loaded successfully. simtec.co.uk/products/SWLINUX/files/booting_article.html#d0e309 or it may be easier to look at either xen/arch/arm/arm32/head.S or linux/arch/arm/kernel/head.S I honestly don''t recall if the userspace domain build supports ELF files -- it should be obvious from the "xl -vvv create" logging if it is at least trying (would be useful to see that more verbose logging BTW) Ian.
On Wed, 2013-04-24 at 10:39 +0100, Sander Bogaert wrote:> I attached the code ( it''s very small).This:> .global hvcConsoleIO > hvcConsoleIO: > PUSH {r12} > MOV r12, #3 > HVC #0xEA1 > POP {r12} > BX lrappears to be calling __HYPERVISOR_stack_switch which isn''t a supported hypercall on ARM. Perhaps you wanted: #define __HYPERVISOR_console_io 18 ? Also r12 is defined to be clobbered by any function call, so you don''t really need to save it I don''t think. The only thing you should need to save when making a hypercall is r4 and only if you are making a 5 argument hypercall. Ian.
On 25 April 2013 10:43, Ian Campbell <Ian.Campbell@citrix.com> wrote:> > On Wed, 2013-04-24 at 10:39 +0100, Sander Bogaert wrote: > > > > I attached the code ( it''s very small). > > This: > > .global hvcConsoleIO > > hvcConsoleIO: > > PUSH {r12} > > MOV r12, #3 > > HVC #0xEA1 > > POP {r12} > > BX lr > > appears to be calling __HYPERVISOR_stack_switch which isn''t a supported > hypercall on ARM. > > Perhaps you wanted: > #define __HYPERVISOR_console_io 18 > ?I was getting the number from xen/arch/arm/traps.c:706 ''''call arm_hypercall_table[regs->r12].fn;" and the arm_hypercall_table in the same file. It should have been #4 by that reasoning I know.> > Also r12 is defined to be clobbered by any function call, so you don''t > really need to save it I don''t think. > > The only thing you should need to save when making a hypercall is r4 and > only if you are making a 5 argument hypercall.Thanks, I''ll change this.> > > Ian. > > >Also, I was trying an ELF file. I will try again tomorrow with a zImage header and report back. Thanks, Sander
On Thu, 2013-04-25 at 12:02 +0100, Sander Bogaert wrote:> On 25 April 2013 10:43, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > > > On Wed, 2013-04-24 at 10:39 +0100, Sander Bogaert wrote: > > > > > > > I attached the code ( it''s very small). > > > > This: > > > .global hvcConsoleIO > > > hvcConsoleIO: > > > PUSH {r12} > > > MOV r12, #3 > > > HVC #0xEA1 > > > POP {r12} > > > BX lr > > > > appears to be calling __HYPERVISOR_stack_switch which isn''t a supported > > hypercall on ARM. > > > > Perhaps you wanted: > > #define __HYPERVISOR_console_io 18 > > ? > > I was getting the number from xen/arch/arm/traps.c:706 ''''call > arm_hypercall_table[regs->r12].fn;" and the arm_hypercall_table in the > same file. It should have been #4 by that reasoning I know.That macro is such that the fact that something appears at position N In the list doesn''t mean it is position N in the resulting array, it expands into the syntax array[] = { [1] = a_thing [4] = another_thing } which initialises slots 1 and 4.> > > > > Also r12 is defined to be clobbered by any function call, so you don''t > > really need to save it I don''t think. > > > > The only thing you should need to save when making a hypercall is r4 and > > only if you are making a 5 argument hypercall. > > Thanks, I''ll change this. > > > > > > > Ian. > > > > > > > > Also, I was trying an ELF file. I will try again tomorrow with a > zImage header and report back. > > Thanks, > Sander
>> I was getting the number from xen/arch/arm/traps.c:706 ''''call >> arm_hypercall_table[regs->r12].fn;" and the arm_hypercall_table in the >> same file. It should have been #4 by that reasoning I know. > > That macro is such that the fact that something appears at position N In > the list doesn''t mean it is position N in the resulting array, it > expands into the syntax > array[] = { > [1] = a_thing > [4] = another_thing > } > which initialises slots 1 and 4.Thanks, I overlooked that.>> >> Also, I was trying an ELF file. I will try again tomorrow with a >> zImage header and report back. >> >> Thanks, >> Sander > >ELF does seem to be supported but then it gets stuck further down in domain builder code ( I also attached the entire output) : domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-unknown: 0x80000000 -> 0x80010010 xc: error: panic: xc_dom_core.c:608: xc_dom_find_arch_hooks: not found (type xen-3.0-unknown): Invalid kernel xc: error: panic: xc_dom_core.c:763: xc_dom_mem_init: arch hooks not set: Internal error libxl: error: libxl_dom.c:384:libxl__build_pv: xc_dom_mem_init failed: No such file or directory The vmlinux binary ( elf ) doesn''t boot neither and has the same problem. Could it have something to do with: "xc: detail: elf_xen_note_check: Not bothering with notes on ARM" Missing metadata? I''ll focus on getting an zImage working. Thanks! _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org lists.xen.org/xen-devel
On Fri, 2013-04-26 at 09:58 +0100, Sander Bogaert wrote:> >> I was getting the number from xen/arch/arm/traps.c:706 ''''call > >> arm_hypercall_table[regs->r12].fn;" and the arm_hypercall_table in the > >> same file. It should have been #4 by that reasoning I know. > > > > That macro is such that the fact that something appears at position N In > > the list doesn''t mean it is position N in the resulting array, it > > expands into the syntax > > array[] = { > > [1] = a_thing > > [4] = another_thing > > } > > which initialises slots 1 and 4. > > Thanks, I overlooked that. > > >> > >> Also, I was trying an ELF file. I will try again tomorrow with a > >> zImage header and report back. > >> > >> Thanks, > >> Sander > > > > > > ELF does seem to be supported but then it gets stuck further down in > domain builder code ( I also attached the entire output) : > > domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-unknown: > 0x80000000 -> 0x80010010 > xc: error: panic: xc_dom_core.c:608: xc_dom_find_arch_hooks: not found > (type xen-3.0-unknown): Invalid kernel > xc: error: panic: xc_dom_core.c:763: xc_dom_mem_init: arch hooks not > set: Internal error > libxl: error: libxl_dom.c:384:libxl__build_pv: xc_dom_mem_init failed: > No such file or directory > > The vmlinux binary ( elf ) doesn''t boot neither and has the same > problem. Could it have something to do with: > "xc: detail: elf_xen_note_check: Not bothering with notes on ARM" > Missing metadata?Could be something like that, it''s likely not needed on ARM but the code may not know that. If/when we want to support ELF (which we will at some point, e.g. for mini-os) someone probably needs to dig into that stuff again.> I''ll focus on getting an zImage working.That''s likely the path of least resistance right now. Ian.
On Fri, 26 Apr 2013, Sander Bogaert wrote:> >> I was getting the number from xen/arch/arm/traps.c:706 ''''call > >> arm_hypercall_table[regs->r12].fn;" and the arm_hypercall_table in the > >> same file. It should have been #4 by that reasoning I know. > > > > That macro is such that the fact that something appears at position N In > > the list doesn''t mean it is position N in the resulting array, it > > expands into the syntax > > array[] = { > > [1] = a_thing > > [4] = another_thing > > } > > which initialises slots 1 and 4. > > Thanks, I overlooked that. > > >> > >> Also, I was trying an ELF file. I will try again tomorrow with a > >> zImage header and report back. > >> > >> Thanks, > >> Sander > > > > > > ELF does seem to be supported but then it gets stuck further down in > domain builder code ( I also attached the entire output) : > > domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-unknown: > 0x80000000 -> 0x80010010 > xc: error: panic: xc_dom_core.c:608: xc_dom_find_arch_hooks: not found > (type xen-3.0-unknown): Invalid kernel > xc: error: panic: xc_dom_core.c:763: xc_dom_mem_init: arch hooks not > set: Internal error > libxl: error: libxl_dom.c:384:libxl__build_pv: xc_dom_mem_init failed: > No such file or directory > > The vmlinux binary ( elf ) doesn''t boot neither and has the same > problem. Could it have something to do with: > "xc: detail: elf_xen_note_check: Not bothering with notes on ARM" > Missing metadata? > > I''ll focus on getting an zImage working.The problem seems to be that dom->guest_type is set to "xen-3.0-unknown" by xc_dom_guest_type at the end of xc_dom_parse_elf_kernel. In order to match the ARMv7 supported guest_type it should have been "xen-3.0-armv7l", as you can see from xc_dom_armzimageloader.c:xc_dom_parse_zimage_kernel. Maybe we just need to add the appropriate case in the switch in xc_dom_guest_type (see below)? diff --git a/tools/libxc/xc_dom_elfloader.c b/tools/libxc/xc_dom_elfloader.c index 6583859..911f316 100644 --- a/tools/libxc/xc_dom_elfloader.c +++ b/tools/libxc/xc_dom_elfloader.c @@ -74,6 +74,8 @@ static char *xc_dom_guest_type(struct xc_dom_image *dom, } case EM_X86_64: return "xen-3.0-x86_64"; + case EM_ARM: + return "xen-3.0-armv7l"; default: return "xen-3.0-unknown"; }
On Sun, 2013-04-28 at 15:04 +0100, Stefano Stabellini wrote:> On Fri, 26 Apr 2013, Sander Bogaert wrote: > > >> I was getting the number from xen/arch/arm/traps.c:706 ''''call > > >> arm_hypercall_table[regs->r12].fn;" and the arm_hypercall_table in the > > >> same file. It should have been #4 by that reasoning I know. > > > > > > That macro is such that the fact that something appears at position N In > > > the list doesn''t mean it is position N in the resulting array, it > > > expands into the syntax > > > array[] = { > > > [1] = a_thing > > > [4] = another_thing > > > } > > > which initialises slots 1 and 4. > > > > Thanks, I overlooked that. > > > > >> > > >> Also, I was trying an ELF file. I will try again tomorrow with a > > >> zImage header and report back. > > >> > > >> Thanks, > > >> Sander > > > > > > > > > > ELF does seem to be supported but then it gets stuck further down in > > domain builder code ( I also attached the entire output) : > > > > domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-unknown: > > 0x80000000 -> 0x80010010 > > xc: error: panic: xc_dom_core.c:608: xc_dom_find_arch_hooks: not found > > (type xen-3.0-unknown): Invalid kernel > > xc: error: panic: xc_dom_core.c:763: xc_dom_mem_init: arch hooks not > > set: Internal error > > libxl: error: libxl_dom.c:384:libxl__build_pv: xc_dom_mem_init failed: > > No such file or directory > > > > The vmlinux binary ( elf ) doesn''t boot neither and has the same > > problem. Could it have something to do with: > > "xc: detail: elf_xen_note_check: Not bothering with notes on ARM" > > Missing metadata? > > > > I''ll focus on getting an zImage working. > > The problem seems to be that dom->guest_type is set to > "xen-3.0-unknown" by xc_dom_guest_type at the end of > xc_dom_parse_elf_kernel. > In order to match the ARMv7 supported guest_type it should have been > "xen-3.0-armv7l", as you can see from > xc_dom_armzimageloader.c:xc_dom_parse_zimage_kernel. > > Maybe we just need to add the appropriate case in the switch in > xc_dom_guest_type (see below)?We certainly need to add something there, and your string seems as plausible as the next one ;-) I suppose we need to handle EM_AARCH64 at some point too... I reckon a warning print under the default case would have us then and other porters in the future...> > diff --git a/tools/libxc/xc_dom_elfloader.c b/tools/libxc/xc_dom_elfloader.c > index 6583859..911f316 100644 > --- a/tools/libxc/xc_dom_elfloader.c > +++ b/tools/libxc/xc_dom_elfloader.c > @@ -74,6 +74,8 @@ static char *xc_dom_guest_type(struct xc_dom_image *dom, > } > case EM_X86_64: > return "xen-3.0-x86_64"; > + case EM_ARM: > + return "xen-3.0-armv7l"; > default: > return "xen-3.0-unknown"; > }
On Mon, 29 Apr 2013, Ian Campbell wrote:> On Sun, 2013-04-28 at 15:04 +0100, Stefano Stabellini wrote: > > On Fri, 26 Apr 2013, Sander Bogaert wrote: > > > >> I was getting the number from xen/arch/arm/traps.c:706 ''''call > > > >> arm_hypercall_table[regs->r12].fn;" and the arm_hypercall_table in the > > > >> same file. It should have been #4 by that reasoning I know. > > > > > > > > That macro is such that the fact that something appears at position N In > > > > the list doesn''t mean it is position N in the resulting array, it > > > > expands into the syntax > > > > array[] = { > > > > [1] = a_thing > > > > [4] = another_thing > > > > } > > > > which initialises slots 1 and 4. > > > > > > Thanks, I overlooked that. > > > > > > >> > > > >> Also, I was trying an ELF file. I will try again tomorrow with a > > > >> zImage header and report back. > > > >> > > > >> Thanks, > > > >> Sander > > > > > > > > > > > > > > ELF does seem to be supported but then it gets stuck further down in > > > domain builder code ( I also attached the entire output) : > > > > > > domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-unknown: > > > 0x80000000 -> 0x80010010 > > > xc: error: panic: xc_dom_core.c:608: xc_dom_find_arch_hooks: not found > > > (type xen-3.0-unknown): Invalid kernel > > > xc: error: panic: xc_dom_core.c:763: xc_dom_mem_init: arch hooks not > > > set: Internal error > > > libxl: error: libxl_dom.c:384:libxl__build_pv: xc_dom_mem_init failed: > > > No such file or directory > > > > > > The vmlinux binary ( elf ) doesn''t boot neither and has the same > > > problem. Could it have something to do with: > > > "xc: detail: elf_xen_note_check: Not bothering with notes on ARM" > > > Missing metadata? > > > > > > I''ll focus on getting an zImage working. > > > > The problem seems to be that dom->guest_type is set to > > "xen-3.0-unknown" by xc_dom_guest_type at the end of > > xc_dom_parse_elf_kernel. > > In order to match the ARMv7 supported guest_type it should have been > > "xen-3.0-armv7l", as you can see from > > xc_dom_armzimageloader.c:xc_dom_parse_zimage_kernel. > > > > Maybe we just need to add the appropriate case in the switch in > > xc_dom_guest_type (see below)? > > We certainly need to add something there, and your string seems as > plausible as the next one ;-) > > I suppose we need to handle EM_AARCH64 at some point too... > > I reckon a warning print under the default case would have us then and > other porters in the future... > > > > > diff --git a/tools/libxc/xc_dom_elfloader.c b/tools/libxc/xc_dom_elfloader.c > > index 6583859..911f316 100644 > > --- a/tools/libxc/xc_dom_elfloader.c > > +++ b/tools/libxc/xc_dom_elfloader.c > > @@ -74,6 +74,8 @@ static char *xc_dom_guest_type(struct xc_dom_image *dom, > > } > > case EM_X86_64: > > return "xen-3.0-x86_64"; > > + case EM_ARM: > > + return "xen-3.0-armv7l"; > > default: > > return "xen-3.0-unknown"; > > }Sander, if you confirm that this change solved your problem, I''ll resend as a proper patch.
On 29 April 2013 14:56, Stefano Stabellini <stefano.stabellini@eu.citrix.com> wrote:> On Mon, 29 Apr 2013, Ian Campbell wrote: >> On Sun, 2013-04-28 at 15:04 +0100, Stefano Stabellini wrote: >> > On Fri, 26 Apr 2013, Sander Bogaert wrote: >> > > >> I was getting the number from xen/arch/arm/traps.c:706 ''''call >> > > >> arm_hypercall_table[regs->r12].fn;" and the arm_hypercall_table in the >> > > >> same file. It should have been #4 by that reasoning I know. >> > > > >> > > > That macro is such that the fact that something appears at position N In >> > > > the list doesn''t mean it is position N in the resulting array, it >> > > > expands into the syntax >> > > > array[] = { >> > > > [1] = a_thing >> > > > [4] = another_thing >> > > > } >> > > > which initialises slots 1 and 4. >> > > >> > > Thanks, I overlooked that. >> > > >> > > >> >> > > >> Also, I was trying an ELF file. I will try again tomorrow with a >> > > >> zImage header and report back. >> > > >> >> > > >> Thanks, >> > > >> Sander >> > > > >> > > > >> > > >> > > ELF does seem to be supported but then it gets stuck further down in >> > > domain builder code ( I also attached the entire output) : >> > > >> > > domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-unknown: >> > > 0x80000000 -> 0x80010010 >> > > xc: error: panic: xc_dom_core.c:608: xc_dom_find_arch_hooks: not found >> > > (type xen-3.0-unknown): Invalid kernel >> > > xc: error: panic: xc_dom_core.c:763: xc_dom_mem_init: arch hooks not >> > > set: Internal error >> > > libxl: error: libxl_dom.c:384:libxl__build_pv: xc_dom_mem_init failed: >> > > No such file or directory >> > > >> > > The vmlinux binary ( elf ) doesn''t boot neither and has the same >> > > problem. Could it have something to do with: >> > > "xc: detail: elf_xen_note_check: Not bothering with notes on ARM" >> > > Missing metadata? >> > > >> > > I''ll focus on getting an zImage working. >> > >> > The problem seems to be that dom->guest_type is set to >> > "xen-3.0-unknown" by xc_dom_guest_type at the end of >> > xc_dom_parse_elf_kernel. >> > In order to match the ARMv7 supported guest_type it should have been >> > "xen-3.0-armv7l", as you can see from >> > xc_dom_armzimageloader.c:xc_dom_parse_zimage_kernel. >> > >> > Maybe we just need to add the appropriate case in the switch in >> > xc_dom_guest_type (see below)? >> >> We certainly need to add something there, and your string seems as >> plausible as the next one ;-) >> >> I suppose we need to handle EM_AARCH64 at some point too... >> >> I reckon a warning print under the default case would have us then and >> other porters in the future... >> >> > >> > diff --git a/tools/libxc/xc_dom_elfloader.c b/tools/libxc/xc_dom_elfloader.c >> > index 6583859..911f316 100644 >> > --- a/tools/libxc/xc_dom_elfloader.c >> > +++ b/tools/libxc/xc_dom_elfloader.c >> > @@ -74,6 +74,8 @@ static char *xc_dom_guest_type(struct xc_dom_image *dom, >> > } >> > case EM_X86_64: >> > return "xen-3.0-x86_64"; >> > + case EM_ARM: >> > + return "xen-3.0-armv7l"; >> > default: >> > return "xen-3.0-unknown"; >> > } > > Sander, > if you confirm that this change solved your problem, I''ll resend as a > proper patch.Hi, This patch got me further in booting the domain but still got stuck. I was adding extra print''s to find out where exactly. Here is the stacktrace ( attached the whole output too) : domainbuilder: detail: xc_dom_build_image: called domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x80000000 -> 0x80011000 (pfn 0x80000 + 0x11 pages) domainbuilder: detail: xc_dom_pfn_to_ptr: pfn 80000 out of range (0x80000 > 0x2000) libxl: error: libxl_dom.c:392:libxl__build_pv: xc_dom_build_image failed: No such file or directory domainbuilder: detail: xc_dom_release: called libxl: error: libxl_create.c:908:domcreate_rebuild_done: cannot (re-)build domain: -3 libxl: debug: libxl_event.c:1569:libxl__ao_complete: ao 0x3aa28: complete, rc=-3 libxl: debug: libxl_create.c:1249:do_domain_create: ao 0x3aa28: inprogress: poller=0x34880, flags=ic libxl: debug: libxl_event.c:1541:libxl__ao__destroy: ao 0x3aa28: destroy It ends up in libxl_dom.c:392:libxl__build_pv: xc_dom_build_image failed where there is some x86 specific stuff I think. Thx _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org lists.xen.org/xen-devel
On 29 April 2013 15:48, Sander Bogaert <sander.bogaert@gmail.com> wrote:> On 29 April 2013 14:56, Stefano Stabellini > <stefano.stabellini@eu.citrix.com> wrote: >> On Mon, 29 Apr 2013, Ian Campbell wrote: >>> On Sun, 2013-04-28 at 15:04 +0100, Stefano Stabellini wrote: >>> > On Fri, 26 Apr 2013, Sander Bogaert wrote: >>> > > >> I was getting the number from xen/arch/arm/traps.c:706 ''''call >>> > > >> arm_hypercall_table[regs->r12].fn;" and the arm_hypercall_table in the >>> > > >> same file. It should have been #4 by that reasoning I know. >>> > > > >>> > > > That macro is such that the fact that something appears at position N In >>> > > > the list doesn''t mean it is position N in the resulting array, it >>> > > > expands into the syntax >>> > > > array[] = { >>> > > > [1] = a_thing >>> > > > [4] = another_thing >>> > > > } >>> > > > which initialises slots 1 and 4. >>> > > >>> > > Thanks, I overlooked that. >>> > > >>> > > >> >>> > > >> Also, I was trying an ELF file. I will try again tomorrow with a >>> > > >> zImage header and report back. >>> > > >> >>> > > >> Thanks, >>> > > >> Sander >>> > > > >>> > > > >>> > > >>> > > ELF does seem to be supported but then it gets stuck further down in >>> > > domain builder code ( I also attached the entire output) : >>> > > >>> > > domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-unknown: >>> > > 0x80000000 -> 0x80010010 >>> > > xc: error: panic: xc_dom_core.c:608: xc_dom_find_arch_hooks: not found >>> > > (type xen-3.0-unknown): Invalid kernel >>> > > xc: error: panic: xc_dom_core.c:763: xc_dom_mem_init: arch hooks not >>> > > set: Internal error >>> > > libxl: error: libxl_dom.c:384:libxl__build_pv: xc_dom_mem_init failed: >>> > > No such file or directory >>> > > >>> > > The vmlinux binary ( elf ) doesn''t boot neither and has the same >>> > > problem. Could it have something to do with: >>> > > "xc: detail: elf_xen_note_check: Not bothering with notes on ARM" >>> > > Missing metadata? >>> > > >>> > > I''ll focus on getting an zImage working. >>> > >>> > The problem seems to be that dom->guest_type is set to >>> > "xen-3.0-unknown" by xc_dom_guest_type at the end of >>> > xc_dom_parse_elf_kernel. >>> > In order to match the ARMv7 supported guest_type it should have been >>> > "xen-3.0-armv7l", as you can see from >>> > xc_dom_armzimageloader.c:xc_dom_parse_zimage_kernel. >>> > >>> > Maybe we just need to add the appropriate case in the switch in >>> > xc_dom_guest_type (see below)? >>> >>> We certainly need to add something there, and your string seems as >>> plausible as the next one ;-) >>> >>> I suppose we need to handle EM_AARCH64 at some point too... >>> >>> I reckon a warning print under the default case would have us then and >>> other porters in the future... >>> >>> > >>> > diff --git a/tools/libxc/xc_dom_elfloader.c b/tools/libxc/xc_dom_elfloader.c >>> > index 6583859..911f316 100644 >>> > --- a/tools/libxc/xc_dom_elfloader.c >>> > +++ b/tools/libxc/xc_dom_elfloader.c >>> > @@ -74,6 +74,8 @@ static char *xc_dom_guest_type(struct xc_dom_image *dom, >>> > } >>> > case EM_X86_64: >>> > return "xen-3.0-x86_64"; >>> > + case EM_ARM: >>> > + return "xen-3.0-armv7l"; >>> > default: >>> > return "xen-3.0-unknown"; >>> > } >> >> Sander, >> if you confirm that this change solved your problem, I''ll resend as a >> proper patch. > > Hi, > > This patch got me further in booting the domain but still got stuck. I > was adding extra print''s to find out where exactly. Here is the > stacktrace ( attached the whole output too) : > > domainbuilder: detail: xc_dom_build_image: called > domainbuilder: detail: xc_dom_alloc_segment: kernel : > 0x80000000 -> 0x80011000 (pfn 0x80000 + 0x11 pages) > domainbuilder: detail: xc_dom_pfn_to_ptr: pfn 80000 out of range > (0x80000 > 0x2000) > libxl: error: libxl_dom.c:392:libxl__build_pv: xc_dom_build_image > failed: No such file or directory > domainbuilder: detail: xc_dom_release: called > libxl: error: libxl_create.c:908:domcreate_rebuild_done: cannot > (re-)build domain: -3 > libxl: debug: libxl_event.c:1569:libxl__ao_complete: ao 0x3aa28: complete, rc=-3 > libxl: debug: libxl_create.c:1249:do_domain_create: ao 0x3aa28: > inprogress: poller=0x34880, flags=ic > libxl: debug: libxl_event.c:1541:libxl__ao__destroy: ao 0x3aa28: destroy > > It ends up in libxl_dom.c:392:libxl__build_pv: xc_dom_build_image > failed where there is some x86 specific stuff I think. > > ThxHi, I was wondering if you had any chance to have a look at this? Would it be a big patch to make ELF work? I''m trying to figure it out myself but it''s not that easy, I''m new to this :-) ( and I had some problems compiling the tools again ). Maybe some pointers could help me create this patch myself? Sander
On Wed, 2013-05-01 at 09:55 +0100, Sander Bogaert wrote:> > This patch got me further in booting the domain but still got stuck. I > > was adding extra print''s to find out where exactly. Here is the > > stacktrace ( attached the whole output too) : > > > > domainbuilder: detail: xc_dom_build_image: called > > domainbuilder: detail: xc_dom_alloc_segment: kernel : > > 0x80000000 -> 0x80011000 (pfn 0x80000 + 0x11 pages) > > domainbuilder: detail: xc_dom_pfn_to_ptr: pfn 80000 out of range > > (0x80000 > 0x2000)This could be due to confusion about where the RAM base starts. On x86 RAM starts at zero and so the domain builder had/has some assumptions about that in it. So your image appears to be linked at 0x8000000 (a common RAM base on ARM, and the one we currently expose to guest virtual machines, good) but the domain builder thinks memory is 0x0...0x2000000, bad. I added rambase_pfn to xc_dom_image to address this. tools/libxc/xc_dom_armzimageloader.c deals with it but I suspect the ELF loader doesn''t. You could probably hack an assignment in somewhere. I wonder if RAM base ought to be part of xc_dom_arch and set the default in xc_dom_image from that? We should also perhaps consider whether 0x80000000 is the right base to expose to guests.> > libxl: error: libxl_dom.c:392:libxl__build_pv: xc_dom_build_image > > failed: No such file or directory > > domainbuilder: detail: xc_dom_release: called > > libxl: error: libxl_create.c:908:domcreate_rebuild_done: cannot > > (re-)build domain: -3 > > libxl: debug: libxl_event.c:1569:libxl__ao_complete: ao 0x3aa28: complete, rc=-3 > > libxl: debug: libxl_create.c:1249:do_domain_create: ao 0x3aa28: > > inprogress: poller=0x34880, flags=ic > > libxl: debug: libxl_event.c:1541:libxl__ao__destroy: ao 0x3aa28: destroy > > > > It ends up in libxl_dom.c:392:libxl__build_pv: xc_dom_build_image > > failed where there is some x86 specific stuff I think. > > > > Thx > > Hi, > > I was wondering if you had any chance to have a look at this? Would it > be a big patch to make ELF work? I''m trying to figure it out myself > but it''s not that easy, I''m new to this :-) ( and I had some problems > compiling the tools again ). Maybe some pointers could help me create > this patch myself?I''m afraid this probably isn''t going to float to the top of my TODO list any time soon, sorry. If it doesn''t turn out to be the rambase thing I mention above then it''s worth noting that before we had libxl integration we had a hacky "xcbuild" binary[0] which at least at one point supported booting from ELF files. I wouldn''t recommend using xcbuild now but it might be useful to compare the libxc calls it makes with those made by libxl. (Fair warning: this might be bad advice and lead you down the wrong path, there are a couple of different ways you can use the libxc domain builder IIRC). [0] xenbits.xen.org/gitweb/?p=people/ianc/xen.git;a=shortlog;h=refs/heads/arm-xcbuild> > Sander
On 1 May 2013 11:27, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Wed, 2013-05-01 at 09:55 +0100, Sander Bogaert wrote: >> > This patch got me further in booting the domain but still got stuck. I >> > was adding extra print''s to find out where exactly. Here is the >> > stacktrace ( attached the whole output too) : >> > >> > domainbuilder: detail: xc_dom_build_image: called >> > domainbuilder: detail: xc_dom_alloc_segment: kernel : >> > 0x80000000 -> 0x80011000 (pfn 0x80000 + 0x11 pages) >> > domainbuilder: detail: xc_dom_pfn_to_ptr: pfn 80000 out of range >> > (0x80000 > 0x2000) > > This could be due to confusion about where the RAM base starts. > > On x86 RAM starts at zero and so the domain builder had/has some > assumptions about that in it. So your image appears to be linked at > 0x8000000 (a common RAM base on ARM, and the one we currently expose to > guest virtual machines, good) but the domain builder thinks memory is > 0x0...0x2000000, bad. > > I added rambase_pfn to xc_dom_image to address this. > tools/libxc/xc_dom_armzimageloader.c deals with it but I suspect the ELF > loader doesn''t. You could probably hack an assignment in somewhere. > > I wonder if RAM base ought to be part of xc_dom_arch and set the default > in xc_dom_image from that? > > We should also perhaps consider whether 0x80000000 is the right base to > expose to guests. > >> > libxl: error: libxl_dom.c:392:libxl__build_pv: xc_dom_build_image >> > failed: No such file or directory >> > domainbuilder: detail: xc_dom_release: called >> > libxl: error: libxl_create.c:908:domcreate_rebuild_done: cannot >> > (re-)build domain: -3 >> > libxl: debug: libxl_event.c:1569:libxl__ao_complete: ao 0x3aa28: complete, rc=-3 >> > libxl: debug: libxl_create.c:1249:do_domain_create: ao 0x3aa28: >> > inprogress: poller=0x34880, flags=ic >> > libxl: debug: libxl_event.c:1541:libxl__ao__destroy: ao 0x3aa28: destroy >> > >> > It ends up in libxl_dom.c:392:libxl__build_pv: xc_dom_build_image >> > failed where there is some x86 specific stuff I think. >> > >> > Thx >> >> Hi, >> >> I was wondering if you had any chance to have a look at this? Would it >> be a big patch to make ELF work? I''m trying to figure it out myself >> but it''s not that easy, I''m new to this :-) ( and I had some problems >> compiling the tools again ). Maybe some pointers could help me create >> this patch myself? > > I''m afraid this probably isn''t going to float to the top of my TODO list > any time soon, sorry. > > If it doesn''t turn out to be the rambase thing I mention above then it''s > worth noting that before we had libxl integration we had a hacky > "xcbuild" binary[0] which at least at one point supported booting from > ELF files. I wouldn''t recommend using xcbuild now but it might be useful > to compare the libxc calls it makes with those made by libxl. (Fair > warning: this might be bad advice and lead you down the wrong path, > there are a couple of different ways you can use the libxc domain > builder IIRC). > > > [0] > xenbits.xen.org/gitweb/?p=people/ianc/xen.git;a=shortlog;h=refs/heads/arm-xcbuild >> >> Sander > >I thought I send a small update: After finally understanding what needed to be done ( and some compilation struggles ) I made the suggested change. Basically I just added one line to set the rambase when loading an ELF ( see below ). This seems to work for my custom binary, as far as I can tell from the logs it boots but immediatly crashes ( see attached log ). For the Linux ELF binary, linuxvm this doesnt work ( see log ). I think it''s because it has some address hardcoded in it? So my questions: Can I get a small confirmation my custom binary boots and crashes after a succesful boot? :-) Any tips on the linux kernel case? I don''t really need this but maybe I can whip up a real patch and submit it ( it would be a very small one but it''s a start )? My changes: diff --git a/tools/libxc/xc_dom_elfloader.c b/tools/libxc/xc_dom_elfloader.c index 911f316..9b6a847 100644 --- a/tools/libxc/xc_dom_elfloader.c +++ b/tools/libxc/xc_dom_elfloader.c @@ -31,6 +31,8 @@ #define XEN_VER "xen-3.0" +#define ARM_GUEST_RAM_BASE 0x80000000 + /* ------------------------------------------------------------------------ */ static void log_callback(struct elf_binary *elf, void *caller_data, @@ -297,6 +299,9 @@ static int xc_dom_parse_elf_kernel(struct xc_dom_image *dom) dom->kernel_seg.vstart = dom->parms.virt_kstart; dom->kernel_seg.vend = dom->parms.virt_kend; + /* this should only be done for arm */ + dom->rambase_pfn = ARM_GUEST_RAM_BASE >> XC_PAGE_SHIFT; + if ( dom->parms.bsd_symtab ) xc_dom_load_elf_symtab(dom, elf, 0); _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org lists.xen.org/xen-devel
On Thu, 2013-05-02 at 15:55 +0100, Sander Bogaert wrote:> On 1 May 2013 11:27, Ian Campbell <Ian.Campbell@citrix.com> wrote: > After finally understanding what needed to be done ( and some > compilation struggles ) I made the suggested change. Basically I just > added one line to set the rambase when loading an ELF ( see below ). > This seems to work for my custom binary, as far as I can tell from the > logs it boots but immediatly crashes ( see attached log ). For the > Linux ELF binary, linuxvm this doesnt work ( see log ). I think it''s > because it has some address hardcoded in it? So my questions: > > Can I get a small confirmation my custom binary boots and crashes > after a succesful boot? :-)(XEN) Guest data abort: Translation fault at level 2 (XEN) gva=8ffffff8 What does the binary do? Looks like it has executed way outside its memory. What is the custom guest''s memory layout like?> Any tips on the linux kernel case? I don''t really need this but maybe > I can whip up a real patch and submit it ( it would be a very small > one but it''s a start )?I don''t think the Linux kernel ELF file is ever expected to be bootable as an independent thing, at least not for ARM, it relies on stuff in the zImage which wraps the raw binary. Ian.> My changes: > > diff --git a/tools/libxc/xc_dom_elfloader.c b/tools/libxc/xc_dom_elfloader.c > index 911f316..9b6a847 100644 > --- a/tools/libxc/xc_dom_elfloader.c > +++ b/tools/libxc/xc_dom_elfloader.c > @@ -31,6 +31,8 @@ > > #define XEN_VER "xen-3.0" > > +#define ARM_GUEST_RAM_BASE 0x80000000 > + > /* ------------------------------------------------------------------------ */ > > static void log_callback(struct elf_binary *elf, void *caller_data, > @@ -297,6 +299,9 @@ static int xc_dom_parse_elf_kernel(struct xc_dom_image *dom) > dom->kernel_seg.vstart = dom->parms.virt_kstart; > dom->kernel_seg.vend = dom->parms.virt_kend; > > + /* this should only be done for arm */ > + dom->rambase_pfn = ARM_GUEST_RAM_BASE >> XC_PAGE_SHIFT; > + > if ( dom->parms.bsd_symtab ) > xc_dom_load_elf_symtab(dom, elf, 0);
>> Can I get a small confirmation my custom binary boots and crashes >> after a succesful boot? :-) > > (XEN) Guest data abort: Translation fault at level 2 > (XEN) gva=8ffffff8 > > What does the binary do? Looks like it has executed way outside its > memory. What is the custom guest''s memory layout like? >This is still the same one as I initially mailed. It should print Hello World using the console hypercall. After playing around a bit ( the sp was set at 0x90000000 which was too far ) I got it running. I don''t see the Hello World however but I''ll look into that. Thanks Btw, when I configure the guest to have more than 474mb ram ( in the cfg file, ''memory = 475'' ) the domain creation fails. Is this to be expected? Is there documentation on how Xen is mapped on the board''s memory? I looked at some patches ( the ''hack'' 1:1 arndale patch too ) but it''s not entirely clear yet why this happens. I know uboot loads xen at 0x5000000 and we expect dom0 at 0x8000000. domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-armv7l: 0x80000000 -> 0x80010010 domainbuilder: detail: xc_dom_mem_init: mem 475 MB, pages 0x1db00 pages, 4k each domainbuilder: detail: xc_dom_mem_init: 0x1db00 pages domainbuilder: detail: xc_dom_boot_mem_init: called domainbuilder: detail: xc_dom_malloc : 950 kB (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=14 memflags=0 (120870 of 121600) xc: detail: Failed allocation for dom 14: 121600 extents of order 0 domainbuilder: detail: xc_dom_build_image: called domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x80000000 -> 0x80011000 (pfn 0x80000 + 0x11 pages) domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x80000+0x11 at 0xb6c2c000 xc: detail: elf_load_binary: phdr 0 at 0x0xb6c2c000 -> 0x0xb6c2c05c xc: detail: elf_load_binary: phdr 1 at 0x0xb6c3c000 -> 0x0xb6c3c010 domainbuilder: detail: alloc_magic_pages: called (XEN) p2m_populate_ram: L2 failed (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=14 memflags=0 (1 of 2) xc: detail: Failed allocation for dom 14: 2 extents of order 0 libxl: error: libxl_dom.c:392:libxl__build_pv: xc_dom_build_image failed: Device or resource busy domainbuilder: detail: xc_dom_release: called libxl: error: libxl_create.c:908:domcreate_rebuild_done: cannot (re-)build domain: -3
Possibly Parallel Threads
- xl console regression on xen-unstable
- no console when using xl toolstack xen 4.1.2
- Error: (4, ''Out of memory'', "xc_dom_boot_mem_init: can''t allocate low memory for domain\n")
- Re: [XenARM] XEN tools for ARM with Virtualization Extensions
- [PATCH] xen: arm: document which hypercalls (and subops) are supported on ARM