Xen devel - Feb 2007 - [Xen-ia64-devel] [RFC][PATCH]mini-os: big-endian mini-os on ia64

Hi,

currently I''am porting the mini-os to ia64. A first version is already
in the
ia64xen-tree.
I would like to have a big-endian mini-os running on the ia64 little-endian 
hypervisor.
My target is to have only changes in the domU. This requires to swap all the 
domU data the hypervisor/dom0 see und use.
Attached is a patch with a first draft. I use the macro SWAP() to swap the 
right pieces (on x86 of course empty).
Currently netfront is not supported.
With this patch and small changes in the ia64xen hypervisor + loader
I''am able
to run a big andian mini-os on a ia64xen hypervisor.
The patch contains only changes in the common mini-os parts. The ia64 part in 
the tree is already fine with that.
The patches for the hypervisor will come with an extra patch.
Please have a look and send comments or check in if there are no questions.
Thanks.

Dietmar.


_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

On 20/2/07 14:00, "Dietmar Hahn"
<dietmar.hahn@fujitsu-siemens.com> wrote:
> I would like to have a big-endian mini-os running on the ia64 little-endian
> hypervisor.
What''s the motivation for wanting to do this?

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Keir,

Am Dienstag, 20. Februar 2007 15:36 schrieb Keir Fraser:
> On 20/2/07 14:00, "Dietmar Hahn"
<dietmar.hahn@fujitsu-siemens.com> wrote:
> > I would like to have a big-endian mini-os running on the ia64
> > little-endian hypervisor.
>
> What''s the motivation for wanting to do this?
>
>  -- Keir
I do some conceptual studies to get our mainframe OS running with xen and 
ia64. This OS is running big endian, so ...
My thoughts were to publish the results - maybe another one can use it.
Thanks.

Dietmar.

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

Would you like to have this applied to mainstream? Personally, I don''t 
think it''s too useful there. But I''m open to convincing.

Thanks
Gregor


Dietmar Hahn wrote:
> Hi Keir,
> 
> Am Dienstag, 20. Februar 2007 15:36 schrieb Keir Fraser:
>> On 20/2/07 14:00, "Dietmar Hahn"
<dietmar.hahn@fujitsu-siemens.com> wrote:
>>> I would like to have a big-endian mini-os running on the ia64
>>> little-endian hypervisor.
>> What''s the motivation for wanting to do this?
>>
>>  -- Keir
> 
> I do some conceptual studies to get our mainframe OS running with xen and 
> ia64. This OS is running big endian, so ...
> My thoughts were to publish the results - maybe another one can use it.
> Thanks.
> 
> Dietmar.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Am Dienstag, 20. Februar 2007 20:03 schrieb Grzegorz
Milos:
> Would you like to have this applied to mainstream? Personally, I
don''t
> think it''s too useful there. But I''m open to convincing.
Sure, I would like to have this in mainstream.
Then I can download the xen-source, set big-endian flag in ia64 makerules, try 
make and can test the big endian domU ;-) Ok, this was a joke!
Why not have it in the mainstream? Xen is developing very fast. After x86 now 
we have ia64 and ppc. ia64 is able to run big and little endian (and this is 
used in reality - HP-UX for ia64 is big -endian).
And with Gerd Hoffmann''s great new domain builder it''s an
easiness to start a
big-endian domU on little-endian dom0.
We did some changes to mini-os. Maybe one of the ppc people may port the 
mini-os - more changes may be needed.
I think the SWAP()''s are not that much. Maybe I can reduce this by
moving some
into the ia64-hypercall interface. What remain are the accesses to the ring 
buffers (not that simple in netfront because of the xen-macros).
Perhaps some other guys contribute their opinion.
Thanks.

Dietmar.
>
> Thanks
> Gregor
>
> Dietmar Hahn wrote:
> > Hi Keir,
> >
> > Am Dienstag, 20. Februar 2007 15:36 schrieb Keir Fraser:
> >> On 20/2/07 14:00, "Dietmar Hahn"
<dietmar.hahn@fujitsu-siemens.com>
wrote:
> >>> I would like to have a big-endian mini-os running on the ia64
> >>> little-endian hypervisor.
> >>
> >> What''s the motivation for wanting to do this?
> >>
> >>  -- Keir
> >
> > I do some conceptual studies to get our mainframe OS running with xen
and
> > ia64. This OS is running big endian, so ...
> > My thoughts were to publish the results - maybe another one can use
it.
> > Thanks.
> >
> > Dietmar.
_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

On 21/2/07 13:04, "Dietmar Hahn"
<dietmar.hahn@fujitsu-siemens.com> wrote:
> I think the SWAP()''s are not that much. Maybe I can reduce this by
moving some
> into the ia64-hypercall interface. What remain are the accesses to the ring
> buffers (not that simple in netfront because of the xen-macros).
> Perhaps some other guys contribute their opinion.
I would actually argue that they''re scattered all over the place and it
will
be hard to be sure that we have all the SWAPs we need and only the SWAPs we
need. Particularly to continue ensuring that given that most development
will be with minios having same endianness as Xen.

In short: it''s ugly and unless someone really cares it will rot.

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Am Mittwoch, 21. Februar 2007 14:51 schrieb Keir Fraser:
> On 21/2/07 13:04, "Dietmar Hahn"
<dietmar.hahn@fujitsu-siemens.com> wrote:
> > I think the SWAP()''s are not that much. Maybe I can reduce
this by moving
> > some into the ia64-hypercall interface. What remain are the accesses
to
> > the ring buffers (not that simple in netfront because of the
xen-macros).
> > Perhaps some other guys contribute their opinion.
>
> I would actually argue that they''re scattered all over the place
and it
> will be hard to be sure that we have all the SWAPs we need and only the
> SWAPs we need. Particularly to continue ensuring that given that most
> development will be with minios having same endianness as Xen.
>
> In short: it''s ugly and unless someone really cares it will rot.
>
>  -- Keir
Now that the environment (libxc, xen-ia64) is ready to start a simple BE domU, 
I want to start a new discussion about BE able mini-os.
What I want to have is a mini-os, where everybody whith ia64 hardware can 
build and run a BE mini-os beside LE mini-os (or other domU''s) on
xen-ia64
hypervisor. If you say at this point: no interrest for such a thing, than we 
can stop this discussion here.
On the other side we should discuss how this can be realized.
My target with the first patch was to minimize the impact on common mini-os 
source as small as possible and that the hypervisor shouldn''t be
touched by
BE/LE problematic. Therewidth I did only the SWAPs at the right place in the 
mini-os. Maybe that SWAPs are ugly but the SWAPs have to be done somewhere 
and the best place in my opinion is in the domU at the lowest level functions 
on top of the hypervisor interface - in gnttab, xenbus, xencons, event.
The other way would be building wrappers around all the accesses to 
domU/hypervisor interfaces and hide the SWAPs there. But this seems a little 
bit overkill at this stage.
Oh and yes I will care about this in the mini-os!
Thanks.

Dietmar

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

On 27/2/07 09:55, "Dietmar Hahn"
<dietmar.hahn@fujitsu-siemens.com> wrote:
> What I want to have is a mini-os, where everybody whith ia64 hardware can
> build and run a BE mini-os beside LE mini-os (or other domU''s) on
xen-ia64
> hypervisor. If you say at this point: no interrest for such a thing, than
we
> can stop this discussion here.
I don;t think we''d have a problem with incorportaing support for
ia64-be if
there''s a good reason for it (a better reason than "because
it''s possible").
> The other way would be building wrappers around all the accesses to
> domU/hypervisor interfaces and hide the SWAPs there. But this seems a
little
> bit overkill at this stage.
It would be less ugly and I think less prone to missing some open-coded
accesses. Open-coding the SWAP()s is pretty grim.

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> > What I want to have is a mini-os, where everybody whith ia64 hardware
can
> > build and run a BE mini-os beside LE mini-os (or other
domU''s) on
> > xen-ia64 hypervisor. If you say at this point: no interrest for such a
> > thing, than we can stop this discussion here.
Is there going to be a port of a full-featured BE OS to Xen/IA64?  Would 
existing OSes be able to be paravirtualised, or would it need to be HVM?

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Am Dienstag, 27. Februar 2007 13:07 schrieb Mark
Williamson:
> Is there going to be a port of a full-featured BE OS to Xen/IA64?  Would
> existing OSes be able to be paravirtualised, or would it need to be HVM?
Currently we have a prototype of our OS (BS2000 - S/390 like mainframe os) 
running in big-endian mode on ia64 hardware. We have built a virtual 
environment within linux to let the OS running. Linux is used as the 
IO-system. Therewidth we have some experiences with the SWAP stuff. The 
target is to substitute linux with dom0 and to full paravirtualise the OS.
At the moment I do some groundwork in preparation for the port.
Thanks.

Dietmar.

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

Am Dienstag, 27. Februar 2007 11:51 schrieb Keir Fraser:
> I don;t think we''d have a problem with incorportaing support for
ia64-be if
> there''s a good reason for it (a better reason than "because
it''s
> possible").
I understand this.
> > The other way would be building wrappers around all the accesses to
> > domU/hypervisor interfaces and hide the SWAPs there. But this seems a
> > little bit overkill at this stage.
>
> It would be less ugly and I think less prone to missing some open-coded
> accesses. Open-coding the SWAP()s is pretty grim.
Yes I see this. It''s simply more work and more code is touched but from
the
design view it''s a lot better.
If this is OK for you, I will try this and send a new patch as a proposal.
Thanks.

Dietmar.

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

On 28/2/07 08:25, "Dietmar Hahn"
<dietmar.hahn@fujitsu-siemens.com> wrote:
>> I don;t think we''d have a problem with incorportaing support
for ia64-be if
>> there''s a good reason for it (a better reason than
"because it''s
>> possible").
> I understand this.
Doing this for an OS that has pre-existing dependencies on being big-endian
(like your BS2000, presumably) I can understand. But I don''t see why
adding
contrary-endianness support to minios is part of your roadmap when your end
goal is the porting of a completely different OS? If it''s part of a
work-scoping exercise then maybe that''s understandable, but I
don''t see why
we''d necessarily take the resulting minios modifications upstream.
>> It would be less ugly and I think less prone to missing some open-coded
>> accesses. Open-coding the SWAP()s is pretty grim.
> Yes I see this. It''s simply more work and more code is touched but
from the
> design view it''s a lot better.
> If this is OK for you, I will try this and send a new patch as a proposal.
*If* we decide that this is a worthwhile exercise at all for minios, then I
think this has to be the way to go.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 28/2/07 08:21, "Dietmar Hahn"
<dietmar.hahn@fujitsu-siemens.com> wrote:
> Currently we have a prototype of our OS (BS2000 - S/390 like mainframe os)
> running in big-endian mode on ia64 hardware.
Couldn''t you run BS2000 in little-endian mode? How hardcoded is the
endianness assumption (e.g., all over your ia64-specific additions)?

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Am Mittwoch, 28. Februar 2007 09:37 schrieb Keir Fraser:
> On 28/2/07 08:25, "Dietmar Hahn"
<dietmar.hahn@fujitsu-siemens.com> wrote:
> >> I don;t think we''d have a problem with incorportaing
support for ia64-be
> >> if there''s a good reason for it (a better reason than
"because it''s
> >> possible").
> >
> > I understand this.
>
> Doing this for an OS that has pre-existing dependencies on being big-endian
> (like your BS2000, presumably) I can understand. But I don''t see
why adding
> contrary-endianness support to minios is part of your roadmap when your end
> goal is the porting of a completely different OS? If it''s part of
a
> work-scoping exercise then maybe that''s understandable, but I
don''t see why
> we''d necessarily take the resulting minios modifications upstream.
I did the mini-os port to ia64 as a starting point to get familiar with the 
internals of xen. The big-endian stuff in mini-os was a "work-scoping 
exercise" for me, but after getting big-endianess extensions into ia64-xen 
this was an offer to all developers interested in testing/developing the 
big-endian feature in xen-ia64.
> >> It would be less ugly and I think less prone to missing some
open-coded
> >> accesses. Open-coding the SWAP()s is pretty grim.
> >
> > Yes I see this. It''s simply more work and more code is
touched but from
> > the design view it''s a lot better.
> > If this is OK for you, I will try this and send a new patch as a
> > proposal.
>
> *If* we decide that this is a worthwhile exercise at all for minios, then I
> think this has to be the way to go.
I understand this.
Thanks.

Dietmar.

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

Am Mittwoch, 28. Februar 2007 09:42 schrieb Keir Fraser:
> On 28/2/07 08:21, "Dietmar Hahn"
<dietmar.hahn@fujitsu-siemens.com> wrote:
> > Currently we have a prototype of our OS (BS2000 - S/390 like mainframe
> > os) running in big-endian mode on ia64 hardware.
>
> Couldn''t you run BS2000 in little-endian mode? How hardcoded is
the
> endianness assumption (e.g., all over your ia64-specific additions)?
No, BS2000 in little-endian mode is not possible. We talk about millions of  
lines of code and billions of dollars customer application software.
It''s
much better to prepare the base for big-endian.
Thanks.

Dietmar.

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

>> What I want to have is a mini-os, where everybody whith ia64 hardware
can
>> build and run a BE mini-os beside LE mini-os (or other domU''s)
on xen-ia64
>> hypervisor. If you say at this point: no interrest for such a thing,
than we
>> can stop this discussion here.
> 
> I don;t think we''d have a problem with incorportaing support for
ia64-be if
> there''s a good reason for it (a better reason than "because
it''s possible").
> 
>> The other way would be building wrappers around all the accesses to
>> domU/hypervisor interfaces and hide the SWAPs there. But this seems a
little
>> bit overkill at this stage.
> 
> It would be less ugly and I think less prone to missing some open-coded
> accesses. Open-coding the SWAP()s is pretty grim.
> 
One solution to the rotting problem would be write regression tests. 
High level tests (like for example netfront test) would be quite good at 
picking missing SWAPs, since they exercise a fair amount of Xen/Dom0 
interfaces. Still it''s quite hard to check the coverage (anybody
happens
to be an expert on coverage testing?).

I too dislike scattering SWAPs all over the code, but I guess having a 
nice set of wrappers would at least confine the problem. Still ... not 
that fond of it.

Cheers
Gregor


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Gregor,

Am Donnerstag, 1. März 2007 23:08 schrieb Grzegorz
Milos:
> One solution to the rotting problem would be write regression tests.
> High level tests (like for example netfront test) would be quite good at
> picking missing SWAPs, since they exercise a fair amount of Xen/Dom0
> interfaces. Still it''s quite hard to check the coverage (anybody
happens
> to be an expert on coverage testing?).
>
> I too dislike scattering SWAPs all over the code, but I guess having a
> nice set of wrappers would at least confine the problem. Still ... not
> that fond of it.
>
> Cheers
> Gregor
I send this as a private mail to you.
First thanks for opening a possible door to the SWAP stuff.
I don''t full understand your rotting problem.
With recent binutils (see 
http://sources.redhat.com/ml/binutils/2004-07/msg00110.html) everybody on 
ia64 hardware can build big-endian images on a little-endian linux machine. 
Only some special make flags in mini-os/arch/ia64/arch.mk (and the SWAPs) are 
needed to build a big-endian mini-os on ia64-linux. So a running big-endian 
mini-os on xen-ia64 is already a regression test itself I think.
So the most currently used interfaces are checked with a running mini-os. 
Maybe for some more interface regression tests are possible/needed.
What do you think? Where is my mistake in thinking about this?
Thanks.

Dietmar

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Sorry, this should only go to 
Grzegorz Milos <gm281@cam.ac.uk>
Thanks.

Dietmar.

Am Donnerstag, 15. März 2007 11:55 schrieb Dietmar Hahn:
> Hi Gregor,
>
> Am Donnerstag, 1. März 2007 23:08 schrieb Grzegorz Milos:
> > One solution to the rotting problem would be write regression tests.
> > High level tests (like for example netfront test) would be quite good
at
> > picking missing SWAPs, since they exercise a fair amount of Xen/Dom0
> > interfaces. Still it''s quite hard to check the coverage
(anybody happens
> > to be an expert on coverage testing?).
> >
> > I too dislike scattering SWAPs all over the code, but I guess having a
> > nice set of wrappers would at least confine the problem. Still ... not
> > that fond of it.
> >
> > Cheers
> > Gregor
>
> I send this as a private mail to you.
> First thanks for opening a possible door to the SWAP stuff.
> I don''t full understand your rotting problem.
> With recent binutils (see
> http://sources.redhat.com/ml/binutils/2004-07/msg00110.html) everybody on
> ia64 hardware can build big-endian images on a little-endian linux machine.
> Only some special make flags in mini-os/arch/ia64/arch.mk (and the SWAPs)
> are needed to build a big-endian mini-os on ia64-linux. So a running
> big-endian mini-os on xen-ia64 is already a regression test itself I think.
> So the most currently used interfaces are checked with a running mini-os.
> Maybe for some more interface regression tests are possible/needed.
> What do you think? Where is my mistake in thinking about this?
> Thanks.
>
> Dietmar
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel