Xen devel - Mar 2007 - [PATCH] [HVM] Patches to make HVM capable of running OS/2.

This is a set of patches that makes it possible to run OS/2 in a fully
virtualized guest. Because of the limited real-mode support on Intel VT,
this will currently only work on AMD-V  CPUs.
One of the patches (the smsw instruction emulation) is very "stop-gap"
and
ugly as sin, but arguably even that is better than the current case which
blissfully does the wrong thing and then resumes execution from the wrong
EIP. :) The "right thing" is probably to add the lmsw/smsw
instructions to
x86_emulate, and use that instead, unless something about how the control
register intercepts & emulation makes x86_emulate the wrong tool for the
job. Suggestions welcome, because I do want to make a proper general-case
fix for this, rather than the ugly special-case kludge provided here.

Patch descriptions:

Patch 1:
vpic-prio-fix.patch

This is plain bugfix - IRQ priority calculation is currently broken in the
virtual vpic. The priority shift should be a right-rotation, not a
left-rotation.

Patch 2:
mmio_ops.patch

Just two additional mmio ops that OS/2 needs emulated.

Patch 3:
qemu-dm_xchg.patch

Adds support for IOREQ_TYPE_XCHG in qemu-dm.

Patch 4:
ropmbios-e801-fix.patch

This is a "minimally intrusive" way to fix an issue that I think
should
really be fixed in a different way.
Currently, the shared info, ioreq and buffered_io pages are mapped into the
guest''s memory space as the three highest page frames. They are
"protected"
from use by being marked as reserved in the e820 ram map. However, legacy
software won''t know about the e820 call, and since the older e801 call
reports all ram, including the shared pages, the guest OS will end up using
them as regular ram with disastrous results.
This patch makes the older e801 bios call report one 64kb block less memory,
thus "protecting" the shared pages from older OS''s in a
similar manner to
the e820 call, at the expense of 52kb of wasted ram (the e801 call reports
memory in 64kb blocks, so no way around this).

However, while I think shared_info might be needed by PV-on-HVM drivers, I
don''t see why the ioreq and buffered_iopage pages should be
guest-accessible
at all. Rather they should be shared only between HV and Dom0, since their
use is strictly for the qemu-dm device model, which should happen entirely
"out of sight" of a fully virtualized guest.

Patch 5:

svm_smsw_modrm.patch

This is the abovementioned ugly patch.
The current SVM emulation of the smsw instruction makes the assumption that
the destination for the msw is always a register. While i think this is true
for the newer MOV from CRx, the legacy smsw can also copy to mem. Since
I''ve
only encountered one specific usage of msw->mem this, which is
msw->segment
base + offset, I''ve put in handling only of this special case. Consider
this
one very temporary.







_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 16/3/07 12:07, "Trolle Selander" <trolle.selander@gmail.com>
wrote:
> This is a "minimally intrusive" way to fix an issue that I think
should really
> be fixed in a different way.
> Currently, the shared info, ioreq and buffered_io pages are mapped into the
> guest''s memory space as the three highest page frames. They are
"protected"
> from use by being marked as reserved in the e820 ram map. However, legacy
> software won''t know about the e820 call, and since the older e801
call reports
> all ram, including the shared pages, the guest OS will end up using them as
> regular ram with disastrous results.
> This patch makes the older e801 bios call report one 64kb block less
memory,
> thus "protecting" the shared pages from older OS''s in a
similar manner to the
> e820 call, at the expense of 52kb of wasted ram (the e801 call reports
memory
> in 64kb blocks, so no way around this).
I think we should reserve a 64kB block just below 0xF0000000 (by marking
E820_RESERVED) and put these Œspecial pages¹ in there. The fact they¹re not
currently mentioned by the e820 map at all is pretty bad ‹ an OS might
decide to remap PCI devices on top of them, for example!

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 3/16/07, Keir Fraser <keir@xensource.com>
wrote:
>
>  On 16/3/07 12:07, "Trolle Selander"
<trolle.selander@gmail.com> wrote:
>
> This is a "minimally intrusive" way to fix an issue that I think
should
> really be fixed in a different way.
> Currently, the shared info, ioreq and buffered_io pages are mapped into
> the guest''s memory space as the three highest page frames. They
are
> "protected" from use by being marked as reserved in the e820 ram
map.
> However, legacy software won''t know about the e820 call, and since
the older
> e801 call reports all ram, including the shared pages, the guest OS will
end
> up using them as regular ram with disastrous results.
> This patch makes the older e801 bios call report one 64kb block less
> memory, thus "protecting" the shared pages from older
OS''s in a similar
> manner to the e820 call, at the expense of 52kb of wasted ram (the e801
call
> reports memory in 64kb blocks, so no way around this).
>
>
> I think we should reserve a 64kB block just below 0xF0000000 (by marking
> E820_RESERVED) and put these ''special pages'' in there.
The fact they''re not
> currently mentioned by the e820 map at all is pretty bad — an OS might
> decide to remap PCI devices on top of them, for example!
>
>  -- Keir
>
I thought they were marked as reserved in the e820 map right now, but now
that I checked, you''re right that they''re not mentioned, which
means they''re
actually nearly as "unprotected" from a modern OS as from a
pre-e820-aware
one like OS/2. Nasty.
In any case, I still don''t see why the ioreq and buffered io pages
should be
inside the guest''s memory space at all. What''s the issue with
keeping them
completely outside the guest''s visible RAM and only shared between HV
&
Dom0?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 16/3/07 12:45, "Trolle Selander" <trolle.selander@gmail.com>
wrote:
> I thought they were marked as reserved in the e820 map right now, but now
that
> I checked, you''re right that they''re not mentioned, which
means they''re
> actually nearly as "unprotected" from a modern OS as from a
pre-e820-aware one
> like OS/2. Nasty.
> In any case, I still don''t see why the ioreq and buffered io pages
should be
> inside the guest''s memory space at all. What''s the issue
with keeping them
> completely outside the guest''s visible RAM and only shared between
HV & Dom0?
If the pages belong to the domU then they have to be part of its
pseudophysical address space, otherwise dom0 cannot map them (since HVM
pages are always mapped by pfn, not by mfn).

We could make the pages belong to dom0, or to Xen, I suppose, but that¹s not
the road we¹ve gone down and there¹s not really any reason to change now. We
should just keep the pages out of the guest¹s way so he doesn¹t accidentally
use them as RAM or map on top of them!

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 3/16/07, Keir Fraser <keir@xensource.com>
wrote:
>
>
>
>
> On 16/3/07 12:45, "Trolle Selander"
<trolle.selander@gmail.com> wrote:
>
> I thought they were marked as reserved in the e820 map right now, but now
> that I checked, you''re right that they''re not mentioned,
which means they''re
> actually nearly as "unprotected" from a modern OS as from a
pre-e820-aware
> one like OS/2. Nasty.
> In any case, I still don''t see why the ioreq and buffered io pages
should
> be inside the guest''s memory space at all. What''s the
issue with keeping
> them completely outside the guest''s visible RAM and only shared
between HV &
> Dom0?
>
>
> If the pages belong to the domU then they have to be part of its
> pseudophysical address space, otherwise dom0 cannot map them (since HVM
> pages are always mapped by pfn, not by mfn).
>
> We could make the pages belong to dom0, or to Xen, I suppose, but
that''s
> not the road we''ve gone down and there''s not really any
reason to change
> now. We should just keep the pages out of the guest''s way so he
doesn''t
> accidentally use them as RAM or map on top of them!
>
>  -- Keir
>
Keeping them out of the guest''s way is good enough for my current
practical
concerns, and this was the path my patch took, after all.
For the record, though, I do think that the most correct thing would be to
have the iopage & buffered_iopages owned by dom0, since they don''t
"belong"
to the domU anymore than any other data structure used by the qemu-dm
process.
In fact, one could argue that domU access to these pages could be a
theoretical way for a compromised domU to attack a process running as root
in dom0. From what I''ve seen, the worst that can practically be done at
the
moment is making qemu-dm lock up while eating 100% cpu (by setting the
buffered_iopage->read_pointer > buffered_iopage->read_pointer) but more
evil
minds than mine might be able to figure out a way to exploit this in a worse
way.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Possibly, but I think that allocating, registering and tracking the pages if
they were owned by dom0 would be harder. I don;t have a clear idea how we¹d
do it without changes to the dom0 kernel.

Meanwhile, domUs have plenty of other shared-memory protocols with dom0
kernel and root processes. It just needs some care to make sure the
interface is sufficiently narrow and arguments are well checked. Burning
100% CPU is not considered a successful attack (although it would of course
be annoying!). You can detect it and fix it up without rebooting the system,
for example.

Anyway, I didn¹t take your e801 patch, but checked in my own fix as
changeset 14415. It should hit the main public tree in a few hours, or you
can see it in the staging tree sooner
(http://xenbits.xensource.com/staging/xen-unstable.hg).

All your other patches are in except the smsw one. I¹m looking at that now.

 -- Keir

On 16/3/07 18:22, "Trolle Selander" <trolle.selander@gmail.com>
wrote:
> Keeping them out of the guest''s way is good enough for my current
practical
> concerns, and this was the path my patch took, after all.
> For the record, though, I do think that the most correct thing would be to
> have the iopage & buffered_iopages owned by dom0, since they
don''t "belong" to
> the domU anymore than any other data structure used by the qemu-dm process.
> In fact, one could argue that domU access to these pages could be a
> theoretical way for a compromised domU to attack a process running as root
in
> dom0. From what I''ve seen, the worst that can practically be done
at the
> moment is making qemu-dm lock up while eating 100% cpu (by setting the
> buffered_iopage->read_pointer > buffered_iopage->read_pointer) but
more evil
> minds than mine might be able to figure out a way to exploit this in a
worse
> way. 



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>
> Meanwhile, domUs have plenty of other shared-memory protocols with dom0
> kernel and root processes. It just needs some care to make sure the
> interface is sufficiently narrow and arguments are well checked. Burning
> 100% CPU is not considered a successful attack (although it would of course
> be annoying!). You can detect it and fix it up without rebooting the
system,
> for example.
>
Indeed, that one is very easy to guard against in the code, too. Simply
making the iopage handler loop on <= instead of != would do it, although
it''s probably better to just insert a sanity before entering the loop,
and
printing a warning to the log that the iopage may have been corrupted if bad
values are detected.
In any case, the current "qemu-dm in dom0" device model is unlikely to
live
forever. Both the two future alternatives I''ve seen discussed - the
stub
domain and the "reflection" one suggested earlier this month - would
do away
with any concerns I have. With the "reflection" model, the device
emulation
would actually run inside the HVM, in which case the io pages should
certainly be owned by the domU, and in the stub domain case, I suppose it
wouldn''t matter as much whether the domU or the stub domain owned the
pages
in any case.

All your other patches are in except the smsw one.
>
Great news.

I''m looking at that now.
>
I apologize to your eyes, and hope it won''t ruin your weekend. ;)
That one shouldn''t go in in its current state, at least.

/Trolle


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 16/3/07 20:11, "Trolle Selander" <trolle.selander@gmail.com>
wrote:
> 
>>  I''m looking at that now.
> 
> I apologize to your eyes, and hope it won''t ruin your weekend. ;)
> That one shouldn''t go in in its current state, at least.
Well, we¹ll see. I think even in it¹s current form it¹s probably no worse
than what¹s there right now. I¹ll see if there¹s any robustifying to be done
(better handling of cases that are not yet supported, in particular). Of
course ultimately we will replace the whole lot with a call into
x86_emulate().

Adding all this special-case code will make it all the sweeter when the day
comes that we replace it all with just x86_emulate(). :-)

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Thanks very much for the contribution. I am now
running Serenity System''s eComStation. I will receive
a new AMD Athlon 64 X2 machine in a couple weeks. I
will try your patches then.

Do you have a copy of eComStation?



 
____________________________________________________________________________________
The fish are biting. 
Get more visitors on your site using Yahoo! Search Marketing.
http://searchmarketing.yahoo.com/arp/sponsoredsearch_v2.php

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

My testing and development so far has been on Warp 4 with FP12. Later
fixpack levels and eCS may or may not work - I simply haven''t had time
to
test yet, though I will get around to it eventually. The top priority for me
is to make sure that my client''s specific OS/2 version and their
application
software works.
There are some minor outstanding issues, the most noticeable of which is
incorrect handling of page-spanning REPZ MOVS emulation, which causes the
boot image to be garbled. I haven''t seen this code ever get hit again
after
the initial boot, so it is effectively a "cosmetic" issue in practice.
I''ll
make a fix for that as soon as I have time, since I already know exactly
what''s going wrong and where.

The baseline VGA driver is amazingly slow. I haven''t tested with a
"full"
CL5446 driver yet, but even the generic GRADD VGA is a massive improvement
and provides acceptable performance, at least for my own current purposes.
Most likely, eCS has a proper CL5446 driver out of the box, so with a bit of
luck, you should be able to get as good graphics performance as it''s
possible to get out of HVM Xen right now.

By the time you get your new system, all the necessary patches may be in
xen-unstable already. All but one are already in. In any case, I''ll be
curious to hear how things work with eCS.

/Trolle

On 3/21/07, ecs user <user_ecs@yahoo.com> wrote:
>
>
> Thanks very much for the contribution. I am now
> running Serenity System''s eComStation. I will receive
> a new AMD Athlon 64 X2 machine in a couple weeks. I
> will try your patches then.
>
> Do you have a copy of eComStation?
>
>
>
>
>
>
____________________________________________________________________________________
> The fish are biting.
> Get more visitors on your site using Yahoo! Search Marketing.
> http://searchmarketing.yahoo.com/arp/sponsoredsearch_v2.php
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> Trolle Selander <trolle.selander@gmail.com> wrote:
> The baseline VGA driver is amazingly slow. I haven''t
> tested with a "full" CL5446 driver yet, but even the
> generic GRADD VGA
In QEMU I have been able to get above vga resolution
using gengradd 
> is a massive improvement and provides acceptable 
> performance, at least for my own current purposes.
> Most likely, eCS has a proper CL5446 driver out of
> the box,
Was not able to get a native CL5446 driver to work
> with a bit of luck, you should be able to get as
good
> graphics performance as it''s  possible to get out of
> HVM Xen right now.
I think I tried Scitech/IBM Display Doctor SE but I do
not remember the results. You might try this

http://hobbes.nmsu.edu/cgi-bin/h-viewer?sh=1&fname=/pub/os2/system/drivers/video/ibmsdd704.zip





 
____________________________________________________________________________________
Don''t pick lemons.
See all the new 2007 cars at Yahoo! Autos.
http://autos.yahoo.com/new_cars.html 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel