Xen devel - May 2006 - X86_emulate to be moved into qemu...

On 17 May 2006, at 21:17, Petersson, Mats wrote:
> When using x86_emulate.c inside qemu, we''d need to feed in the
virtual
> address, but we also need to translate to (guest-)physical address. Any
> hints or tricks for this, or do I need to read the page-table and get
> the info that way [and CAN I even do that]? [And I''m sorry if this
> shows
> my complete and utter ignorance of how Xen and QEMU operates together,
> but I''m afraid that I''m still learning these things].
Yes, you could walk pagetables. But equally you can pass in your 
translated CR2 value --- i.e., pass in a pseudophysical address. That 
address will then be passed to the special read/write hook functions, 
so you avoid needing to do the translation inside those.
> Another interesting question is what we do with the dozen or so
"new"
> include-files that are needed to make x86_emulate.c compile inside the
> tools/ioemu directory. At the moment, I''ve just created
directories
> inside tools/libxc and linked the necessary header files into those
> directories... Is that the proper solution?
What header files are those? It builds in tools/test/ without so many 
header files.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> Hmmm... wouldn''t it be nice if we didn''t have a bespoke, 
> buggy & incomplete emulator for hvm mmio. ;-)
In the process of solving this one, I''ve got to (another) of the
blocking points... 

At the moment, we translate the virtual address from the guest in the
page-fault-handler to guest physical address via the gva_to_gpa()
function in svm.c/vmx.c: xxx_do_page_fault(). 

When using x86_emulate.c inside qemu, we''d need to feed in the virtual
address, but we also need to translate to (guest-)physical address. Any
hints or tricks for this, or do I need to read the page-table and get
the info that way [and CAN I even do that]? [And I''m sorry if this
shows
my complete and utter ignorance of how Xen and QEMU operates together,
but I''m afraid that I''m still learning these things]. 

Another interesting question is what we do with the dozen or so "new"
include-files that are needed to make x86_emulate.c compile inside the
tools/ioemu directory. At the moment, I''ve just created directories
inside tools/libxc and linked the necessary header files into those
directories... Is that the proper solution?

--
Mats


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 5/17/06, Petersson, Mats <Mats.Petersson@amd.com>
wrote:
> > Hmmm... wouldn''t it be nice if we didn''t have a
bespoke,
> > buggy & incomplete emulator for hvm mmio. ;-)
>
> In the process of solving this one, I''ve got to (another) of the
> blocking points...
If Xen is going to need an x86 emulator, there is also need of a small
x86 emulator based on klibc as part of the general Linux kernel.
People often put video cards with x86 ROMs into PPC/Itanium machines
since the x86 versions are half the price. The general idea is that
during the boot process the emulator would run the x86 ROM and
initialize these cards. The same code would be used to initialize
secondary cards on all architectures. After klibc gets added to the
kernel tree I''d also like to see the emulator and video reset program
added.

-- 
Jon Smirl
jonsmirl@gmail.com

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> -----Original Message-----
> From: Jon Smirl [mailto:jonsmirl@gmail.com] 
> Sent: 17 May 2006 21:46
> To: Petersson, Mats
> Cc: Keir Fraser; Xen devel list; Benjamin Herrenschmidt
> Subject: Re: [Xen-devel] X86_emulate to be moved into qemu...
> 
> On 5/17/06, Petersson, Mats <Mats.Petersson@amd.com> wrote:
> > > Hmmm... wouldn''t it be nice if we didn''t have a
bespoke, buggy &
> > > incomplete emulator for hvm mmio. ;-)
> >
> > In the process of solving this one, I''ve got to (another) of
the
> > blocking points...
> 
> If Xen is going to need an x86 emulator, there is also need of a small
> x86 emulator based on klibc as part of the general Linux kernel.
> People often put video cards with x86 ROMs into PPC/Itanium 
> machines since the x86 versions are half the price. The 
> general idea is that during the boot process the emulator 
> would run the x86 ROM and initialize these cards. The same 
> code would be used to initialize secondary cards on all 
> architectures. After klibc gets added to the kernel tree I''d 
> also like to see the emulator and video reset program added.
Ehm, yes, but this is a much smaller emulator, as it''s only emulating
instructions that are caught by page faults - so only instructions that
are doing memory accesses. 

I think you could use the full x86-emulator that is part of the QEMU
project to achive what you''re after, but the x86_emualte_memop() that
we''re talking about here would not be even close to sufficient for
running the BIOS on a non-x86 processor. 

--
Mats
> 
> --
> Jon Smirl
> jonsmirl@gmail.com
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 5/17/06, Petersson, Mats <Mats.Petersson@amd.com>
wrote:
> > -----Original Message-----
> > From: Jon Smirl [mailto:jonsmirl@gmail.com]
> > Sent: 17 May 2006 21:46
> > To: Petersson, Mats
> > Cc: Keir Fraser; Xen devel list; Benjamin Herrenschmidt
> > Subject: Re: [Xen-devel] X86_emulate to be moved into qemu...
> >
> > On 5/17/06, Petersson, Mats <Mats.Petersson@amd.com> wrote:
> > > > Hmmm... wouldn''t it be nice if we didn''t
have a bespoke, buggy &
> > > > incomplete emulator for hvm mmio. ;-)
> > >
> > > In the process of solving this one, I''ve got to
(another) of the
> > > blocking points...
> >
> > If Xen is going to need an x86 emulator, there is also need of a small
> > x86 emulator based on klibc as part of the general Linux kernel.
> > People often put video cards with x86 ROMs into PPC/Itanium
> > machines since the x86 versions are half the price. The
> > general idea is that during the boot process the emulator
> > would run the x86 ROM and initialize these cards. The same
> > code would be used to initialize secondary cards on all
> > architectures. After klibc gets added to the kernel tree I''d
> > also like to see the emulator and video reset program added.
>
> Ehm, yes, but this is a much smaller emulator, as it''s only
emulating
> instructions that are caught by page faults - so only instructions that
> are doing memory accesses.
>
> I think you could use the full x86-emulator that is part of the QEMU
> project to achive what you''re after, but the x86_emualte_memop()
that
> we''re talking about here would not be even close to sufficient for
> running the BIOS on a non-x86 processor.
It wasn''t clear from the earlier mail that this was only a partial
emulator. We are already using the emulator from the Linux Bios
project so we''ll stick with that. It would nice to reduce the number
of emulator code bases being used.

>
> --
> Mats
> >
> > --
> > Jon Smirl
> > jonsmirl@gmail.com
> >
> >
>
>

-- 
Jon Smirl
jonsmirl@gmail.com

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 17 May 2006, at 22:45, Jon Smirl wrote:
> It wasn''t clear from the earlier mail that this was only a partial
> emulator. We are already using the emulator from the Linux Bios
> project so we''ll stick with that. It would nice to reduce the
number
> of emulator code bases being used.
Different applications require quite different emulators. The 
scitechsoft one in linuxbios emulates true real-mode only (no protected 
mode; no big real mode) and it will run on non-x86 hardware (obviously 
a fundamental requirement for your use). For us protected mode is the 
primary mode of operation, and we get *significant* speedup and 
simplification by knowing that we are emulating on x86 (specifically we 
execute a regs-only version of the instruction we are
''emulating'',
hence no tedious emulation of flags etc -- basically we only emulate 
the data movement to/from memory).

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com 
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of 
> Keir Fraser
> Sent: 17 May 2006 21:15
> To: Petersson, Mats
> Cc: Xen devel list
> Subject: [Xen-devel] Re: X86_emulate to be moved into qemu...
> 
> 
> On 17 May 2006, at 21:17, Petersson, Mats wrote:
> 
> > When using x86_emulate.c inside qemu, we''d need to feed in 
> the virtual 
> > address, but we also need to translate to (guest-)physical address. 
> > Any hints or tricks for this, or do I need to read the 
> page-table and 
> > get the info that way [and CAN I even do that]? [And I''m 
> sorry if this 
> > shows my complete and utter ignorance of how Xen and QEMU operates 
> > together, but I''m afraid that I''m still learning
these things].
> 
> Yes, you could walk pagetables. But equally you can pass in 
> your translated CR2 value --- i.e., pass in a pseudophysical 
> address. That address will then be passed to the special 
> read/write hook functions, so you avoid needing to do the 
> translation inside those.
Hmm, yes. But we also want CS:rIP translated, which isn''t too bad. And
for movs we have both the source and destination to take into account.
ES:rDI isn''t bad, but xS:rSI isn''t as trivial, since
we''d have to scan
for segment overrides in that case - and now we''re starting to look
like
a bunch of decoding code in two places again... :-( [Admittedly not
quite as much] 
> 
> > Another interesting question is what we do with the dozen 
> or so "new"
> > include-files that are needed to make x86_emulate.c compile 
> inside the 
> > tools/ioemu directory. At the moment, I''ve just created
directories
> > inside tools/libxc and linked the necessary header files into those 
> > directories... Is that the proper solution?
> 
> What header files are those? It builds in tools/test/ without 
> so many header files.
''hg status'' says:
M ioemu/target-i386-dm/Makefile
M ioemu/target-i386-dm/helper2.c
? ioemu/target-i386-dm/x86_emulate.c
? libxc/asm-x86/bitops.h
? libxc/asm-x86/cache.h
? libxc/asm-x86/config.h
? libxc/asm-x86/desc.h
? libxc/asm-x86/mm.h
? libxc/asm-x86/page.h
? libxc/asm-x86/processor.h
? libxc/asm-x86/regs.h
? libxc/asm-x86/rwlock.h
? libxc/asm-x86/spinlock.h
? libxc/asm-x86/string.h
? libxc/asm-x86/system.h
? libxc/asm-x86/types.h
? libxc/asm-x86/uaccess.h
? libxc/asm-x86/x86_emulate.h
? libxc/public/arch-x86_32.h
? libxc/public/arch-x86_64.h
? libxc/public/xen-compat.h
? libxc/public/xen.h

--
Mats
> 
>   -- Keir
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 18 May 2006, at 10:33, Petersson, Mats wrote:
>> Yes, you could walk pagetables. But equally you can pass in
>> your translated CR2 value --- i.e., pass in a pseudophysical
>> address. That address will then be passed to the special
>> read/write hook functions, so you avoid needing to do the
>> translation inside those.
>
> Hmm, yes. But we also want CS:rIP translated, which isn''t too bad.
And
> for movs we have both the source and destination to take into account.
> ES:rDI isn''t bad, but xS:rSI isn''t as trivial, since
we''d have to scan
> for segment overrides in that case - and now we''re starting to
look
> like
> a bunch of decoding code in two places again... :-( [Admittedly not
> quite as much]
The code should be passed in a buffer. This will be needed for 
interfacing with VMX anyway, since it places the instruction in a handy 
on-chip buffer for convenient readout. That gets rid of any need for 
CS:RIP decoding.

Yes, we will need translation in a few places for dual-memory-operand 
instructions (MOVS, stack instructions, etc). So we will need to walk 
pagetables. It''s just that the *common* case will be to need no walk. 
The segment override is no problem -- it''s already implemented.

  -- Keir

>> What header files are those? It builds in tools/test/ without
>> so many header files.
> ''hg status'' says:
I''ll fix that.

  -- Keir
> M ioemu/target-i386-dm/Makefile
> M ioemu/target-i386-dm/helper2.c
> ? ioemu/target-i386-dm/x86_emulate.c
> ? libxc/asm-x86/bitops.h
> ? libxc/asm-x86/cache.h
> ? libxc/asm-x86/config.h
> ? libxc/asm-x86/desc.h
> ? libxc/asm-x86/mm.h
> ? libxc/asm-x86/page.h
> ? libxc/asm-x86/processor.h
> ? libxc/asm-x86/regs.h
> ? libxc/asm-x86/rwlock.h
> ? libxc/asm-x86/spinlock.h
> ? libxc/asm-x86/string.h
> ? libxc/asm-x86/system.h
> ? libxc/asm-x86/types.h
> ? libxc/asm-x86/uaccess.h
> ? libxc/asm-x86/x86_emulate.h
> ? libxc/public/arch-x86_32.h
> ? libxc/public/arch-x86_64.h
> ? libxc/public/xen-compat.h
> ? libxc/public/xen.h
>
> --
> Mats
>>
>>   -- Keir
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> -----Original Message-----
> From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk] 
> Sent: 18 May 2006 10:41
> To: Petersson, Mats
> Cc: Xen devel list
> Subject: Re: [Xen-devel] Re: X86_emulate to be moved into qemu...
> 
> 
> On 18 May 2006, at 10:33, Petersson, Mats wrote:
> 
> >> Yes, you could walk pagetables. But equally you can pass in your 
> >> translated CR2 value --- i.e., pass in a pseudophysical 
> address. That 
> >> address will then be passed to the special read/write hook 
> functions, 
> >> so you avoid needing to do the translation inside those.
> >
> > Hmm, yes. But we also want CS:rIP translated, which isn''t 
> too bad. And 
> > for movs we have both the source and destination to take 
> into account.
> > ES:rDI isn''t bad, but xS:rSI isn''t as trivial, since
we''d
> have to scan 
> > for segment overrides in that case - and now we''re starting
to look
> > like a bunch of decoding code in two places again... :-( 
> [Admittedly 
> > not quite as much]
> 
> The code should be passed in a buffer. This will be needed 
> for interfacing with VMX anyway, since it places the 
> instruction in a handy on-chip buffer for convenient readout. 
> That gets rid of any need for CS:RIP decoding.
> 
> Yes, we will need translation in a few places for 
> dual-memory-operand instructions (MOVS, stack instructions, 
> etc). So we will need to walk pagetables. It''s just that the 
> *common* case will be to need no walk. 
> The segment override is no problem -- it''s already implemented.
For the movs (in x86_emaulate.c) the segment override is currently
detected but ignored for protected mode operation (base is assumed to be
zero). This is why Minix doesn''t work properly [or at all, really] -
admittedly, I don''t think Minix is the most critical operating system
in
the world, but one part of fixing this up is to avoid having to fix
furhter "weirdness" in various operating systems, right?
> 
>   -- Keir
> 
> 
> >> What header files are those? It builds in tools/test/ 
> without so many 
> >> header files.
> > ''hg status'' says:
> 
> I''ll fix that.
Ok, thanks. 

--
Mats
> 
>   -- Keir
> 
> > M ioemu/target-i386-dm/Makefile
> > M ioemu/target-i386-dm/helper2.c
> > ? ioemu/target-i386-dm/x86_emulate.c
> > ? libxc/asm-x86/bitops.h
> > ? libxc/asm-x86/cache.h
> > ? libxc/asm-x86/config.h
> > ? libxc/asm-x86/desc.h
> > ? libxc/asm-x86/mm.h
> > ? libxc/asm-x86/page.h
> > ? libxc/asm-x86/processor.h
> > ? libxc/asm-x86/regs.h
> > ? libxc/asm-x86/rwlock.h
> > ? libxc/asm-x86/spinlock.h
> > ? libxc/asm-x86/string.h
> > ? libxc/asm-x86/system.h
> > ? libxc/asm-x86/types.h
> > ? libxc/asm-x86/uaccess.h
> > ? libxc/asm-x86/x86_emulate.h
> > ? libxc/public/arch-x86_32.h
> > ? libxc/public/arch-x86_64.h
> > ? libxc/public/xen-compat.h
> > ? libxc/public/xen.h
> >
> > --
> > Mats
> >>
> >>   -- Keir
> >>
> >>
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@lists.xensource.com
> >> http://lists.xensource.com/xen-devel
> >>
> >>
> >
> 
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 18 May 2006, at 10:50, Petersson, Mats wrote:
> For the movs (in x86_emaulate.c) the segment override is currently
> detected but ignored for protected mode operation (base is assumed to 
> be
> zero). This is why Minix doesn''t work properly [or at all, really]
-
> admittedly, I don''t think Minix is the most critical operating
system
> in
> the world, but one part of fixing this up is to avoid having to fix
> furhter "weirdness" in various operating systems, right?
Well, the core logic calls out to a macro that then ignores the base. 
The obvious thing to do is have add a new call-out hook in struct 
x86_mem_emulator to read base address of a specified selector. Or we 
could simply pass them in as an array, perhaps as structs allowing us 
to determine other useful info like stack segment address size.

If we do that then we get rid of all real vs. protected mode 
segmentation differences in the core emulator. We always call out or 
read from the array. That gets us support for big real mode too.

If you want to add extensions or flexibility to x86_emulate for this 
stuff, please try to trickle piecemeal patches to me. I prefer that to 
big-bang uber patches. I can also make sure it integrates properly with 
current uses of the emulator.

  -- Keir
>>
>>   -- Keir
>>
>>
>>>> What header files are those? It builds in tools/test/
>> without so many
>>>> header files.
>>> ''hg status'' says:
>>
>> I''ll fix that.
>
> Ok, thanks.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> -----Original Message-----
> From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk] 
> Sent: 18 May 2006 11:09
> To: Petersson, Mats
> Cc: Xen devel list
> Subject: Re: [Xen-devel] Re: X86_emulate to be moved into qemu...
> 
> 
> On 18 May 2006, at 10:50, Petersson, Mats wrote:
> 
> > For the movs (in x86_emaulate.c) the segment override is currently 
> > detected but ignored for protected mode operation (base is 
> assumed to 
> > be zero). This is why Minix doesn''t work properly [or at 
> all, really] 
> > - admittedly, I don''t think Minix is the most critical
operating
> > system in the world, but one part of fixing this up is to 
> avoid having 
> > to fix furhter "weirdness" in various operating systems,
right?
> 
> Well, the core logic calls out to a macro that then ignores the base. 
> The obvious thing to do is have add a new call-out hook in 
> struct x86_mem_emulator to read base address of a specified 
> selector. Or we could simply pass them in as an array, 
> perhaps as structs allowing us to determine other useful info 
> like stack segment address size.
I think passing in as part of the struct is the easiest way - since
it''s
got to know it from the VMC[BS] - but the current code would have to
call out to the respective SVM/VMX code to fetch that info. No big deal.

> 
> If we do that then we get rid of all real vs. protected mode 
> segmentation differences in the core emulator. We always call 
> out or read from the array. That gets us support for big real 
> mode too.
That''s what I''d like to see - just get the base address out of
an array,
and be done with it - no problems supporting whatever strange things
people come up with... 
> 
> If you want to add extensions or flexibility to x86_emulate 
> for this stuff, please try to trickle piecemeal patches to 
> me. I prefer that to big-bang uber patches. I can also make 
> sure it integrates properly with current uses of the emulator.
Yes, I''ll get it to work, with current support, inside QEMU first, and
then trickle in added features (such as segment support and FPU/MMX/SSE
support). Naturally, getting the work into QEMU will be a fairly big
patch, as it''s touching quite a few areas, and I can''t see any
obvious
way to split it up into smaller pieces... 

--
Mats
> 
>   -- Keir
> 
> >>
> >>   -- Keir
> >>
> >>
> >>>> What header files are those? It builds in tools/test/
> >> without so many
> >>>> header files.
> >>> ''hg status'' says:
> >>
> >> I''ll fix that.
> >
> > Ok, thanks.
> 
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> -----Original Message-----
> From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk] 
> Sent: 18 May 2006 11:09
> To: Petersson, Mats
> Cc: Xen devel list
> Subject: Re: [Xen-devel] Re: X86_emulate to be moved into qemu...
> 
> 
> On 18 May 2006, at 10:50, Petersson, Mats wrote:
> 
> > For the movs (in x86_emaulate.c) the segment override is currently 
> > detected but ignored for protected mode operation (base is 
> assumed to 
> > be zero). This is why Minix doesn''t work properly [or at 
> all, really] 
> > - admittedly, I don''t think Minix is the most critical
operating
> > system in the world, but one part of fixing this up is to 
> avoid having 
> > to fix furhter "weirdness" in various operating systems,
right?
> 
> Well, the core logic calls out to a macro that then ignores the base. 
> The obvious thing to do is have add a new call-out hook in 
> struct x86_mem_emulator to read base address of a specified 
> selector. Or we could simply pass them in as an array, 
> perhaps as structs allowing us to determine other useful info 
> like stack segment address size.
> 
> If we do that then we get rid of all real vs. protected mode 
> segmentation differences in the core emulator. We always call 
> out or read from the array. That gets us support for big real 
> mode too.
> 
> If you want to add extensions or flexibility to x86_emulate 
> for this stuff, please try to trickle piecemeal patches to 
> me. I prefer that to big-bang uber patches. I can also make 
> sure it integrates properly with current uses of the emulator.
I just had a couple of thoughs for "stepwise refinement" of the
x86_emulate. 


1. Add a pointer to a struct (or opaque void *) the x86_emulate_memop()
to allow us to pass extra data from HVM that can be used inside the
call-back functions when needed. For the current usage, that would be
null.

2. Add new interface functionality - add a "fetch_insn_byte" function
pointer, and use that instead of/inside the macro insn_fetch. This will
be necessary if we pass a translated CS:rIP to the QEMU version. Or if
we pass along a buffer of instruction bytes from the guest code, we''d
need to fetch from that. The current code doesn''t make any difference
between reading code-bytes or any other reads of guest memory... 

These two changes can be done before I start actually trying to plumb
things into the QEMU model.

Does it make sense to do it this way, and/or do you have some other
ideas?

--
Mats
> 
>   -- Keir
> 
> >>
> >>   -- Keir
> >>
> >>
> >>>> What header files are those? It builds in tools/test/
> >> without so many
> >>>> header files.
> >>> ''hg status'' says:
> >>
> >> I''ll fix that.
> >
> > Ok, thanks.
> 
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 18 May 2006, at 13:34, Petersson, Mats wrote:
> 1. Add a pointer to a struct (or opaque void *) the x86_emulate_memop()
> to allow us to pass extra data from HVM that can be used inside the
> call-back functions when needed. For the current usage, that would be
> null.
I was considering packing all the current emulator parameter into an 
args structure, then passing a pointer to that to the callback 
functions. That''ll let them get at potentially interesting things like 
execution mode, and they can use container_of() to get at a containing 
structure if there is other stuff of interest to them out side the 
scope of emulator parameters.

That would also clean up calls to the emulator (imo) as if we add many 
more parameters we''ll end up with unwieldy parameter lists. Packing a 
structure then making the emulator call would be cleaner as you''d 
assign to each argument structure field on a separate source code line, 
and the field your assigning to would be explicitly named (rather than 
having to work out what the ordering of parameters to the function is, 
as you do now).
> 2. Add new interface functionality - add a "fetch_insn_byte"
function
> pointer, and use that instead of/inside the macro insn_fetch. This will
> be necessary if we pass a translated CS:rIP to the QEMU version. Or if
> we pass along a buffer of instruction bytes from the guest code,
we''d
> need to fetch from that. The current code doesn''t make any
difference
> between reading code-bytes or any other reads of guest memory...
I think we should pass that buffer in as an array, plus count of bytes 
it contains. Two extra fields for the args sturcture. :-)

  -- keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> -----Original Message-----
> From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk] 
> Sent: 18 May 2006 13:43
> To: Petersson, Mats
> Cc: Xen devel list
> Subject: Re: [Xen-devel] Re: X86_emulate to be moved into qemu...
> 
> 
> On 18 May 2006, at 13:34, Petersson, Mats wrote:
> 
> > 1. Add a pointer to a struct (or opaque void *) the 
> > x86_emulate_memop() to allow us to pass extra data from HVM 
> that can 
> > be used inside the call-back functions when needed. For the current 
> > usage, that would be null.
> 
> I was considering packing all the current emulator parameter 
> into an args structure, then passing a pointer to that to the 
> callback functions. That''ll let them get at potentially 
> interesting things like execution mode, and they can use 
> container_of() to get at a containing structure if there is 
> other stuff of interest to them out side the scope of 
> emulator parameters.
> 
> That would also clean up calls to the emulator (imo) as if we 
> add many more parameters we''ll end up with unwieldy parameter 
> lists. Packing a structure then making the emulator call 
> would be cleaner as you''d assign to each argument structure 
> field on a separate source code line, and the field your 
> assigning to would be explicitly named (rather than having to 
> work out what the ordering of parameters to the function is, 
> as you do now).
So, that would be a struct that could have the "struct ioreq" field in
it, which is optionally filled in depending on where it''s called from,
for example?

Yes, I agree with this. 

I''ll work on something that uses this method, and I''ll send
you a patch
before I go any further. 
> 
> > 2. Add new interface functionality - add a 
> "fetch_insn_byte" function 
> > pointer, and use that instead of/inside the macro insn_fetch. This 
> > will be necessary if we pass a translated CS:rIP to the 
> QEMU version. 
> > Or if we pass along a buffer of instruction bytes from the 
> guest code, 
> > we''d need to fetch from that. The current code
doesn''t make any
> > difference between reading code-bytes or any other reads of 
> guest memory...
> 
> I think we should pass that buffer in as an array, plus count 
> of bytes it contains. Two extra fields for the args sturcture. :-)
Yes, that''s what I was planning on [and yes, the length can is handy,
in
case we run into the back end of a page and the next page isn''t there -
we don''t handle that well right now. Still, I''m not sure what
we do if
the next page isn''t there and the instruction ends on the next page,
but
I guess we''ll just have to inject back in a page-fault and see what
happens... ;-)]

--
Mats
> 
>   -- keir
> 
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 18 May 2006, at 13:58, Petersson, Mats wrote:
> So, that would be a struct that could have the "struct ioreq"
field in
> it, which is optionally filled in depending on where it''s called
from,
> for example?
>
> Yes, I agree with this.
>
> I''ll work on something that uses this method, and I''ll
send you a patch
> before I go any further.
My model would be that any *extra* stuff you want to just pass through 
to the callback functions, you''d do as follows (roughly):

/* This is a container for all emulation context. */
struct mydevice_emulator_context {
     /* The actual args structure for the emulator. */
     struct x86_emulate args;
     /* Extra handy stuff for this particular call site and set of 
callback functions. */
     struct ioreq *ioreq;
};

/* This is a call to the emulator. */
x86_emulate_memop(&context.args);

/* This is an example of how a callback func gets at the 
caller-specific goodies. */
int emulate_read(..., struct x86_emulate_memop *args) {
     struct mydevice_emulator_context *context;
     context = container_of(args, struct mydevice_emulator_context, 
args);
     ....
}


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com 
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of 
> Keir Fraser
> Sent: 18 May 2006 14:15
> To: Petersson, Mats
> Cc: Xen devel list
> Subject: Re: [Xen-devel] Re: X86_emulate to be moved into qemu...
> 
> 
> On 18 May 2006, at 13:58, Petersson, Mats wrote:
> 
> > So, that would be a struct that could have the "struct 
> ioreq" field in 
> > it, which is optionally filled in depending on where it''s 
> called from, 
> > for example?
> >
> > Yes, I agree with this.
> >
> > I''ll work on something that uses this method, and
I''ll send you a
> > patch before I go any further.
> 
> My model would be that any *extra* stuff you want to just 
> pass through to the callback functions, you''d do as follows
(roughly):
> 
> /* This is a container for all emulation context. */ struct 
> mydevice_emulator_context {
>      /* The actual args structure for the emulator. */
>      struct x86_emulate args;
>      /* Extra handy stuff for this particular call site and 
> set of callback functions. */
>      struct ioreq *ioreq;
> };
> 
> /* This is a call to the emulator. */
> x86_emulate_memop(&context.args);
> 
> /* This is an example of how a callback func gets at the 
> caller-specific goodies. */ int emulate_read(..., struct 
> x86_emulate_memop *args) {
>      struct mydevice_emulator_context *context;
>      context = container_of(args, struct 
> mydevice_emulator_context, args);
>      ....
> }
Ok, I''m with you. 

Attached is a quick hack at getting a single argument to
x86_emulate_memop() and related changes. Comments please...

--
Mats
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 18 May 2006, at 14:58, Petersson, Mats wrote:
> Ok, I''m with you.
>
> Attached is a quick hack at getting a single argument to
> x86_emulate_memop() and related changes. Comments please...
Yeah, if you clean the formatting up a bit and submit that, I''d apply 
it.

The only question I have is really what we should call the argument 
structure. I guess x86_emulator_args is descriptive. I wondered about 
simply calling it x86_emulate, with the expectation that if/when we 
extend the emulator to do more than just memops we will rename the 
emulator call to x86_emulate(). Then we would simply have the same name 
for the function and its argument structure, which would be neat and 
straightforward. Sticking _args on the end of the name just seems ugly 
to me for some reason, but maybe that''s just irrational.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi --

  Please excuse me jumping in late on this thread.  I''m missing a bit
of
context, so maybe I''m misunderstanding something.  But ...

  Since QEMU is single-threaded, won''t moving the x86_emulate code into
there be a huge SMP performance bottleneck?  What''s the rationale for
doing this?

Thanks,
Dave


On 5/17/06, Petersson, Mats <Mats.Petersson@amd.com>
wrote:
> When using x86_emulate.c inside qemu, we''d need to feed in the
virtual
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 19 May 2006, at 14:31, Dave Lively wrote:
>  Please excuse me jumping in late on this thread.  I''m missing a
bit of
> context, so maybe I''m misunderstanding something.  But ...
>
>  Since QEMU is single-threaded, won''t moving the x86_emulate code
into
> there be a huge SMP performance bottleneck?  What''s the rationale
for
> doing this?
It will still exist in Xen as well, and the Xen version will be used 
for device models implemented in Xen (e.g., PICs).

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com 
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of 
> Dave Lively
> Sent: 19 May 2006 14:32
> To: Petersson, Mats
> Cc: Xen devel list
> Subject: Re: [Xen-devel] X86_emulate to be moved into qemu...
> 
> Hi --
> 
>   Please excuse me jumping in late on this thread.  I''m 
> missing a bit of context, so maybe I''m misunderstanding 
> something.  But ...
> 
>   Since QEMU is single-threaded, won''t moving the x86_emulate 
> code into there be a huge SMP performance bottleneck?  What''s 
> the rationale for doing this?
It''s not ADDING (much of) a bottleneck, as the code that is in HVM at
the moment is calling QEMU anyways - the only difference is that we''d
be
calling QEMU with a slightly less decoded instruction format, in order
to re-use the x86_emulate''s ability to decode instructions - and when
expanding on the instruction set, we only need to do it once, rather
than the current scenario where we two different decoding sets, that use
very different format [actually, there''s a third copy of
decoding-logic,
which is a stripped down copy of x86_emulate.c, which is the instrlen.c
in svm, which we also would like to remove...]

I already looked at using the x86_emulate code inside the HVM code, but
that doesn''t work since we have to task-switch to QEMU to get the
hardware data (say we''re doing a RMW operation on the video memory - we
need to read video memory, then modify it, then write it back). Even if
it was possible to do a task-switch and actually return where the code
was called from, it would still require a SECOND task-switch to write
the modified data back... Although I guess it would be ever so slightly
less bothersome from a SMP perspective, perhaps... 

Your point is very valid, but I don''t see any other (sane) solution.
Making QEMU multi-threaded would of course make it a bit better... Not
sure how easy that would be, and how to deal with multiple accesses to
the same device from multiple processors - not that the real hardware is
any good at that either... ;-) 

--
Mats

> 
> Thanks,
> Dave
> 
> 
> On 5/17/06, Petersson, Mats <Mats.Petersson@amd.com> wrote:
> > When using x86_emulate.c inside qemu, we''d need to feed in 
> the virtual
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ahh, I see.  Thanks to both of you for the clarifications.
I misunderstood the scope of the change.  I agree it sounds
like a good idea.

Dave

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel