Xen devel - May 2013 - Re: [XenARM] Question about booting parameter of Mini-OS for ARM

On Mon, 6 May 2013, Chen Baozi wrote:
> On Mar 25, 2013, at 6:00 PM, Stefano Stabellini
<Stefano.Stabellini@eu.citrix.com> wrote:
> 
> > On Sun, 24 Mar 2013, Chen Baozi wrote:
> >> Hi all,
> >> 
> >> I''m reading Mini-OS''s codes and to estimate the
amount of work porting it to
> >> ARM (Ian''s GSoC idea this year).
> >> 
> >> While Xen is booting Mini-OS on x86 platform, it passes the
start_info_t to
> >> the guest through ESI register. And Mini-OS would use this
structure as the
> >> argument of start_kernel. However, I didn''t see codes
handle the
> >> start_info_t on ARM side. Instead, I see a more standard protocal
when
> >> booting ARM''s dom0, which follows linux kernel bootstrap
rules:
> >> 
> >> 	r0 = 0, r1 = machine nr, r2 = atags or dtb pointer
> >> 
> >> Does it mean that Xen for ARM does not use the start_info_t to
pass
> >> information when booting PV guest? 
> >> Or did I miss something important?
> > 
> > That''s right, we don''t use start_info_t on ARM, in
fact ARM guests are
> > not exactly like x86 PV guests.
> > The information present in the start_info page are either available
via
> > device tree or no used on ARM.
> 
> I''m thinking of parsing the dtb and fill the start_info structure
according to it before jump to the start_kernel. In that case, it won''t
break some original mini-os interface.
You can try that, or you can refactor the code dividing it into x86
specific and common stuff.

On Mon, 2013-05-06 at 11:40 +0100, Stefano Stabellini
wrote:
> On Mon, 6 May 2013, Chen Baozi wrote:
> > On Mar 25, 2013, at 6:00 PM, Stefano Stabellini
<Stefano.Stabellini@eu.citrix.com> wrote:
> > 
> > > On Sun, 24 Mar 2013, Chen Baozi wrote:
> > >> Hi all,
> > >> 
> > >> I''m reading Mini-OS''s codes and to estimate
the amount of work porting it to
> > >> ARM (Ian''s GSoC idea this year).
> > >> 
> > >> While Xen is booting Mini-OS on x86 platform, it passes the
start_info_t to
> > >> the guest through ESI register. And Mini-OS would use this
structure as the
> > >> argument of start_kernel. However, I didn''t see
codes handle the
> > >> start_info_t on ARM side. Instead, I see a more standard
protocal when
> > >> booting ARM''s dom0, which follows linux kernel
bootstrap rules:
> > >> 
> > >> 	r0 = 0, r1 = machine nr, r2 = atags or dtb pointer
> > >> 
> > >> Does it mean that Xen for ARM does not use the start_info_t
to pass
> > >> information when booting PV guest? 
> > >> Or did I miss something important?
> > > 
> > > That''s right, we don''t use start_info_t on ARM,
in fact ARM guests are
> > > not exactly like x86 PV guests.
> > > The information present in the start_info page are either
available via
> > > device tree or no used on ARM.
> > 
> > I''m thinking of parsing the dtb and fill the start_info
structure
> according to it before jump to the start_kernel. In that case, it
> won''t break some original mini-os interface.
If you go this path then trying to do it in arch_init would be
preferable. Nothing in start_kernel before arch_init seems like it
should need start info.

This is effectively what we do for Linux guests too -- populate a stunt
start info early on.
> You can try that, or you can refactor the code dividing it into x86
> specific and common stuff.
There is bound to be a bunch of this sort of refactoring required, just
because mini-os in practice has only really supported x86 until now and
even with the best intentions arch specific stuff tends to leak out
unless you are actively using something on multiple arches.

Ian.

On May 7, 2013, at 4:39 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2013-05-06 at 11:40 +0100, Stefano Stabellini wrote:
>> On Mon, 6 May 2013, Chen Baozi wrote:
>>> On Mar 25, 2013, at 6:00 PM, Stefano Stabellini
<Stefano.Stabellini@eu.citrix.com> wrote:
>>> 
>>>> On Sun, 24 Mar 2013, Chen Baozi wrote:
>>>>> Hi all,
>>>>> 
>>>>> I''m reading Mini-OS''s codes and to
estimate the amount of work porting it to
>>>>> ARM (Ian''s GSoC idea this year).
>>>>> 
>>>>> While Xen is booting Mini-OS on x86 platform, it passes the
start_info_t to
>>>>> the guest through ESI register. And Mini-OS would use this
structure as the
>>>>> argument of start_kernel. However, I didn''t see
codes handle the
>>>>> start_info_t on ARM side. Instead, I see a more standard
protocal when
>>>>> booting ARM''s dom0, which follows linux kernel
bootstrap rules:
>>>>> 
>>>>> 	r0 = 0, r1 = machine nr, r2 = atags or dtb pointer
>>>>> 
>>>>> Does it mean that Xen for ARM does not use the start_info_t
to pass
>>>>> information when booting PV guest? 
>>>>> Or did I miss something important?
>>>> 
>>>> That''s right, we don''t use start_info_t on
ARM, in fact ARM guests are
>>>> not exactly like x86 PV guests.
>>>> The information present in the start_info page are either
available via
>>>> device tree or no used on ARM.
>>> 
>>> I''m thinking of parsing the dtb and fill the start_info
structure
>> according to it before jump to the start_kernel. In that case, it
>> won''t break some original mini-os interface.
> 
> If you go this path then trying to do it in arch_init would be
> preferable. Nothing in start_kernel before arch_init seems like it
> should need start info.
> 
> This is effectively what we do for Linux guests too -- populate a stunt
> start info early on.
> 
>> You can try that, or you can refactor the code dividing it into x86
>> specific and common stuff.
> 
> There is bound to be a bunch of this sort of refactoring required, just
> because mini-os in practice has only really supported x86 until now and
> even with the best intentions arch specific stuff tends to leak out
> unless you are actively using something on multiple arches.
Yes. Right now I''m testing "make" to complete the
"infrastructure" headers and interfaces that arm64 needs. It turns out
that it lacks some basic type definition such as "uint64_t", which is
introduced from headers files of xen hypervisor. I checked x86''s
implementation. Those types are defined at include/public/arch-x86/xen-x86_32.h.
If we follows x86''s convention, we should define them in xen
hypervisor''s tree. I don''t think it is a good idea. Any ideas?

> 
> Ian.
> 
>

Chen Baozi, le Tue 07 May 2013 23:52:04 +0800, a écrit :
> it lacks some basic type definition such as "uint64_t",
> which is introduced from headers files of xen hypervisor.
? I see it coming from extras/mini-os/include/types.h

Samuel

On May 7, 2013, at 11:54 PM, Samuel Thibault
<samuel.thibault@ens-lyon.org> wrote:
> Chen Baozi, le Tue 07 May 2013 23:52:04 +0800, a écrit :
>> it lacks some basic type definition such as "uint64_t",
>> which is introduced from headers files of xen hypervisor.
> 
> ? I see it coming from extras/mini-os/include/types.h
Oh, my bad! I didn''t check this file. Yes. The origin types.h lacks
logic to deal with arm64 situation.

Thanks a lot!

Baozi.

On Mar 25, 2013, at 6:00 PM, Stefano Stabellini
<Stefano.Stabellini@eu.citrix.com> wrote:
> On Sun, 24 Mar 2013, Chen Baozi wrote:
>> Hi all,
>> 
>> I''m reading Mini-OS''s codes and to estimate the
amount of work porting it to
>> ARM (Ian''s GSoC idea this year).
>> 
>> While Xen is booting Mini-OS on x86 platform, it passes the
start_info_t to
>> the guest through ESI register. And Mini-OS would use this structure as
the
>> argument of start_kernel. However, I didn''t see codes handle
the
>> start_info_t on ARM side. Instead, I see a more standard protocal when
>> booting ARM''s dom0, which follows linux kernel bootstrap
rules:
>> 
>> 	r0 = 0, r1 = machine nr, r2 = atags or dtb pointer
>> 
>> Does it mean that Xen for ARM does not use the start_info_t to pass
>> information when booting PV guest? 
>> Or did I miss something important?
> 
> That''s right, we don''t use start_info_t on ARM, in fact
ARM guests are
> not exactly like x86 PV guests.
> The information present in the start_info page are either available via
> device tree or no used on ARM.
I noticed that in arch_domain_create() for arm there are codes
allocating xen heap pages for domain->shared_info:

arch_domain_create:
	if ((d->shared_info = alloc_xenheap_pages(0, 0)) == NULL)	

But it seems it is used nowhere and even not mapped by libxc:

int arch_setup_bootlate(struct xc_dom_image *dom)
{
	/* XXX
	 *   map shared info
         *   map grant tables
         *   setup shared info
         */
        return 0;
}

So my question is what is the role of shared_info page in arm? 
And is there any relations between shared_info and start_info?

Thanks.

Chen Baozi

On Mon, 2013-05-13 at 14:12 +0100, Chen Baozi wrote:
> On Mar 25, 2013, at 6:00 PM, Stefano Stabellini
<Stefano.Stabellini@eu.citrix.com> wrote:
> > On Sun, 24 Mar 2013, Chen Baozi wrote:
> >> Hi all,
> >> 
> >> I''m reading Mini-OS''s codes and to estimate the
amount of work porting it to
> >> ARM (Ian''s GSoC idea this year).
> >> 
> >> While Xen is booting Mini-OS on x86 platform, it passes the
start_info_t to
> >> the guest through ESI register. And Mini-OS would use this
structure as the
> >> argument of start_kernel. However, I didn''t see codes
handle the
> >> start_info_t on ARM side. Instead, I see a more standard protocal
when
> >> booting ARM''s dom0, which follows linux kernel bootstrap
rules:
> >> 
> >> 	r0 = 0, r1 = machine nr, r2 = atags or dtb pointer
> >> 
> >> Does it mean that Xen for ARM does not use the start_info_t to
pass
> >> information when booting PV guest? 
> >> Or did I miss something important?
> > 
> > That''s right, we don''t use start_info_t on ARM, in
fact ARM guests are
> > not exactly like x86 PV guests.
> > The information present in the start_info page are either available
via
> > device tree or no used on ARM.
> 
> I noticed that in arch_domain_create() for arm there are codes
> allocating xen heap pages for domain->shared_info:
> 
> arch_domain_create:
> 	if ((d->shared_info = alloc_xenheap_pages(0, 0)) == NULL)	
> 
> But it seems it is used nowhere and even not mapped by libxc:
> 
> int arch_setup_bootlate(struct xc_dom_image *dom)
> {
> 	/* XXX
> 	 *   map shared info
>          *   map grant tables
>          *   setup shared info
>          */
>         return 0;
> }
This comment is out of date, in fact on ARM the guest itself takes care
of mapping these things by using XENMEM_add_to_physmap with
XENMAPSPACE_shared_info et al.
> So my question is what is the role of shared_info page in arm? 
It is mostly the same as on x86 with a couple of exceptions which spring
to mind. Firstly XEN_LEGACY_MAX_VCPUS == 1 on ARM so guests are required
to use VCPUOP_register_vcpu_info for secondary CPUs instead of relying
on some number of vcpu_info''s being present in the shared info.
Secondly
we don''t actually implement the wall-clock stuff there (yet).

The bit we currently use is the evtchn_* stuff.
> And is there any relations between shared_info and start_info?
Not really no, one is start of day and one is runtime.

Ian.