Xen devel - Jun 2005 - [Patch 2/2] Add dom0_get_sectioninfo for XEN/VTI

As discussed before on the list, unmodified domain is presented a
real-platform-like physical address space, including memory hole, MMIO,
etc. Device Model needs to map pages of other unmodified domain into its
own virtual space. To achieve that, DM needs to retrieve physical pfn ->
machine pfn information of targeted domain. However current
DOM0_GETMEMLIST only returns a list of active pages allocated to
targeted domain, which doesn''t include hole information.

Attached patch adds a new operation (DOM0_GET_SECTIONINFO), to retrieve
hierarchy of physical address space. Together with DOM0_GETMEMLIST, DM
can then construct correct information to access page from other
domains.

Xenolinux has no such issue since it''s physical memory is always
contiguous. ;-)

Signed-off-by Kevin Tian <Kevin.tian@intel.com>

Thanks,
Kevin

--- xeno-unstable.org/xen/include/public/dom0_ops.h	2005-06-16
17:42:48.000000000 +0800
+++ xeno-unstable/xen/include/public/dom0_ops.h	2005-06-16
17:46:09.000000000 +0800
@@ -371,6 +372,24 @@ typedef struct {
  *                 IA64 specific operations (START)        
  **********************************************************/
                 
+#define DOM0_GET_SECTIONINFO	DOM0_IA64_OP_BASE           
+typedef struct {                
+    memory_t	start;	/* start of memory hole */                
+    memory_t	end;	/* end of memory hole */                  
+} dom0_section_t;                                                
+                                                                 
+/* Request section info which describes memory holes in guest    
+ * physical layer, like mmio, etc. When sections is NULL, return 
+ * number of holes. When sections is valid, return section info  
+ * upon requested number.                                        
+ * (ia64 specific now, but should adapt to other arch with holes)
+ */                                                              
+typedef struct {                                                 
+    domid_t domain;		/* Domain to be affected */       
+    u32 section_nr;		/* How many holes existing? */    
+    dom0_section_t *sections;	/* List to contain section info */
+} dom0_get_sectioninfo_t;                                        
+                                                                 
 /**********************************************************      
  *                 IA64 specific operations (END)
  **********************************************************/
@@ -407,6 +426,7 @@ typedef struct {              
         dom0_microcode_t         microcode;      
         dom0_ioport_permission_t ioport_permission;
         dom0_getvcpucontext_t    getvcpucontext;
+        dom0_get_sectioninfo_t	  getsectioninfo;
     } u;
 } dom0_op_t;


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> +/* Request section info which describes memory holes in guest    
> + * physical layer, like mmio, etc. When sections is NULL, return
> + * number of holes. When sections is valid, return section info  
> + * upon requested number.                                        
> + * (ia64 specific now, but should adapt to other arch with holes)
> + */                                                              
> +typedef struct {                                                 
> +    domid_t domain;		/* Domain to be affected */       
> +    u32 section_nr;		/* How many holes existing? */    
> +    dom0_section_t *sections;	/* List to contain 
> section info */
> +} dom0_get_sectioninfo_t;                                        
I''d like to know more about how your propose using this. On x86 VT-x,
the domain builder created the e820 memory map for the domain, so it''s
already known outside Xen (including where the various devices live
within the mmio region, which Xen doesn''t know). 

In any event, can''t your pursuade your unmodified IA64 domains that the
MMIO region lives after the end of RAM? Or is it determined to have it
below 4GB or something? 
It may be convention to have it below 4GB, but does the code actually
break if its above?

Ian

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>-----Original Message-----
>From: xen-devel-bounces@lists.xensource.com
>[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Ian Pratt
>Sent: Friday, June 17, 2005 4:38 PM
>> +/* Request section info which describes memory holes in guest
>> + * physical layer, like mmio, etc. When sections is NULL, return
>> + * number of holes. When sections is valid, return section info
>> + * upon requested number.
>> + * (ia64 specific now, but should adapt to other arch with holes)
>> + */
>> +typedef struct {
>> +    domid_t domain;		/* Domain to be affected */
>> +    u32 section_nr;		/* How many holes existing? */
>> +    dom0_section_t *sections;	/* List to contain
>> section info */
>> +} dom0_get_sectioninfo_t;
>
>I''d like to know more about how your propose using this. On x86
VT-x,
>the domain builder created the e820 memory map for the domain, so
it''s
>already known outside Xen (including where the various devices live
>within the mmio region, which Xen doesn''t know).
Yes, domain builder knows everything to construct target domain. However
when talking device model, will domain builder pass important
information, especially memory layout (e820), to device model? Device
model has to acquire same layout as domain builder to map other
unmodified domain''s page.

Our proposal is simply as: 
- control panel constructs a memory layout (neither e820, nor acpi,
simply for non-memory hole)
- That memory layout is passed into Xen known by section info.
- That memory layout is also passed to guest firmware, with the latter
to construct ACPI table to domain later.
- Then when device model is up, it invokes new dom0 operation to get
memory layout to map foreign page

However, now I also suspected the necessity a bit. ;-b As you said, that
info is produced from outside, and consumed only outside too. There''s
really no need to bother Xen as a connector. Then cleaner way may be
just to pass layout info to device model from domain builder directly.
If not done that yet, change will be made on qemu then. Do you think so?
>
>In any event, can''t your pursuade your unmodified IA64 domains that
the
>MMIO region lives after the end of RAM? Or is it determined to have it
>below 4GB or something?
>It may be convention to have it below 4GB, but does the code actually
>break if its above?
Ideally the layout is decided by platform, and domain should depend on
EFI/ACPI table to make its life. However several issues I think as
problem now:

- Experienced users may feel strange to see a layout not like real
hardware, if they don''t realize running at VMM.

- Not familiar with qemu... If unmodified domain is configured with 4G
memory, and then all MMIO region is moved to >4G area, I''m not sure
whether some old 32bit DMA controller within qemu can work properly
without re-compile?

- On unmodified x86 domain with PAE enabled, can 32bit domain handle
MMIO >4G if following same policy to move MMIO beyond memory? (If device
model has no e820 map). I think the internal type for I/O may be always
32bit... :)

Thanks,
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> However, now I also suspected the necessity a bit. ;-b As you 
> said, that info is produced from outside, and consumed only 
> outside too. There''s really no need to bother Xen as a 
> connector. Then cleaner way may be just to pass layout info 
> to device model from domain builder directly.
> If not done that yet, change will be made on qemu then. Do 
> you think so?
I believe this is the right approach.
> >In any event, can''t your pursuade your unmodified IA64 
> domains that the 
> >MMIO region lives after the end of RAM? Or is it determined 
> to have it 
> >below 4GB or something?
> >It may be convention to have it below 4GB, but does the code 
> actually 
> >break if its above?
> 
> Ideally the layout is decided by platform, and domain should 
> depend on EFI/ACPI table to make its life. However several 
> issues I think as problem now:
> 
> - Experienced users may feel strange to see a layout not like 
> real hardware, if they don''t realize running at VMM.
>
> - Not familiar with qemu... If unmodified domain is 
> configured with 4G memory, and then all MMIO region is moved 
> to >4G area, I''m not sure whether some old 32bit DMA 
> controller within qemu can work properly without re-compile?
> 
> - On unmodified x86 domain with PAE enabled, can 32bit domain 
> handle MMIO >4G if following same policy to move MMIO beyond 
> memory? (If device model has no e820 map). I think the 
> internal type for I/O may be always 32bit... :)
That final point is interesting. Linux is careful thesedays to avoid
ever storing physcial address in 32bits, but its possible earlier
versions were not. 

Anyhow, this is all a discussion for 3.1. I don''t want to get
distracted
from finishing 3.0 right now, and encourage everyone else to think about
the 3.0 todo list too...

Ian




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>-----Original Message-----
>From: Ian Pratt [mailto:m+Ian.Pratt@cl.cam.ac.uk]
>Sent: Friday, June 17, 2005 6:55 PM
>To: Tian, Kevin; xen-devel@lists.xensource.com
>
>
>> However, now I also suspected the necessity a bit. ;-b As you
>> said, that info is produced from outside, and consumed only
>> outside too. There''s really no need to bother Xen as a
>> connector. Then cleaner way may be just to pass layout info
>> to device model from domain builder directly.
>> If not done that yet, change will be made on qemu then. Do
>> you think so?
>
>I believe this is the right approach.
So I gave up requesting for new dom0 operation now. :)
>> - On unmodified x86 domain with PAE enabled, can 32bit domain
>> handle MMIO >4G if following same policy to move MMIO beyond
>> memory? (If device model has no e820 map). I think the
>> internal type for I/O may be always 32bit... :)
>
>That final point is interesting. Linux is careful thesedays to avoid
>ever storing physcial address in 32bits, but its possible earlier
>versions were not.
>
>Anyhow, this is all a discussion for 3.1. I don''t want to get
distracted
>from finishing 3.0 right now, and encourage everyone else to think
about
>the 3.0 todo list too...
>
Agree.

Thanks,
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel