Xen devel - May 2007 - [RFC] [0/4] User-space grants for Console and XenStore

Following on from the introduction of a User-space grant table device  
("gntdev" - see http://lists.xensource.com/archives/html/xen-devel/ 
2007-03/msg01337.html for more details), I have modified the domain  
building process so that the grant table mechanism is used for  
communication with the console and XenStore daemons. This replaces  
the use of xc_map_foreign_range().

The development was carried out as follows:

* Added a new gnttabop, which installs grant table entries into the  
reserved section of the grant table.
* Modified the domain builder to install entries for the console and  
XenStore pages for each new domain (using fixed entry numbers).
* Modified the console and XenStore daemons to use  
xc_gnttab_map_grant_ref() in place of xc_map_foreign_range().

I''ve tested this locally (on x86_32 and x86_64) and it seems to work.  
There are a few remaining concerns that I have:

* Will this work on ia64 and PowerPC?
* How will this integrate with XSM? The install-grant operation is  
clearly as privileged as the ability to map arbitrary memory from a  
foreign domain, so could these share a hook?
* Solaris support? I don''t know enough about Solaris to know what  
changes must be made to gntdev for it to run there. Since these  
changes hit presumably portable code, would it be acceptable to have  
an #ifdef in the console and XenStore daemons?

The patch is structured as follows:

1. gnttabop.patch - Changes to the hypervisor to add the install- 
entry gnttabop.
2. libxc.patch - Changes to libxc to install grants for the console  
and XenStore pages.
3. xenconsoled.patch - Changes to the console daemon to use grant  
references.
4. xenstored.patch - Changes to the XenStore daemon to use grant  
references.

I would welcome your input on this.

Regards,

Derek Murray.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, May 02, 2007 at 12:15:18PM +0100, Derek Murray wrote:
> Following on from the introduction of a User-space grant table device  
> ("gntdev" - see
http://lists.xensource.com/archives/html/xen-devel/
> 2007-03/msg01337.html for more details), I have modified the domain  
>
> * Solaris support? I don''t know enough about Solaris to know what
> changes must be made to gntdev for it to run there. Since these  
It would be nice to have the old method still work, but if you feel the
need to there''s no problem in dropping the old code. We''re
always
playing catch-up anyway...

regards
john

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi John,

On 2 May 2007, at 13:15, John Levon wrote:
> It would be nice to have the old method still work, but if you feel  
> the
> need to there''s no problem in dropping the old code.
We''re always
> playing catch-up anyway...
One possibility would be to have a Makefile option that controls  
whether or not xc_map_foreign_range() is available, and swaps out the  
calls for xc_gnttab_map_grant_ref() where necessary. That way, if/ 
when Solaris gets gntdev, it will be easy to swap in the new code,  
and nothing will be broken in the mean time. I''d also be willing to  
provide any support needed for porting gntdev to Solaris.

Regards,

Derek.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 2/5/07 13:29, "Derek Murray" <Derek.Murray@cl.cam.ac.uk>
wrote:
>> It would be nice to have the old method still work, but if you feel
>> the
>> need to there''s no problem in dropping the old code.
We''re always
>> playing catch-up anyway...
> 
> One possibility would be to have a Makefile option that controls
> whether or not xc_map_foreign_range() is available, and swaps out the
> calls for xc_gnttab_map_grant_ref() where necessary. That way, if/
> when Solaris gets gntdev, it will be easy to swap in the new code,
> and nothing will be broken in the mean time. I''d also be willing
to
> provide any support needed for porting gntdev to Solaris.
It should be easy to support the old way and choose dynamically at run time.
We should somehow tag the values passed to or stored in xenstore explicitly
as being grant-refs, at the very least.

I would like to continue to avoid dependence on particular dom0 kernel
features, so new tool stack on old kernel (with no gnttab driver) should
really continue to work.

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 2 May 2007, at 13:46, Keir Fraser wrote:
> It should be easy to support the old way and choose dynamically at  
> run time.
> We should somehow tag the values passed to or stored in xenstore  
> explicitly
> as being grant-refs, at the very least.
At present, I don''t store the grant-refs in xenstore, because they  
are constant (always 0 for xenstore, 1 for the console). I suppose I  
can see an argument where two different tool stacks are used with  
different conventions, but the same console and xenstore daemons are  
used. The tricky part is getting the grant references out of the  
domain builder without changing the interface to xc_linux_build().  
Would it be reasonable to assume that the current versions of xend  
and libxc are consistent, and therefore the fixed grant references  
could be written into xenstore (from constants in the Python code) at  
the same time as the console and xenstore mfns are?
> I would like to continue to avoid dependence on particular dom0 kernel
> features, so new tool stack on old kernel (with no gnttab driver)  
> should
> really continue to work.
This should be quite simple: if the call to xc_gnttab_open() fails,  
it could set a flag which determines the method to be used. All that  
needs to be done in the mean time is to add stub functions to  
xc_solaris.c - I''ll do that.

Regards,

Derek.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 2/5/07 15:38, "Derek Murray" <Derek.Murray@cl.cam.ac.uk>
wrote:
> At present, I don''t store the grant-refs in xenstore, because they
> are constant (always 0 for xenstore, 1 for the console). I suppose I
> can see an argument where two different tool stacks are used with
> different conventions, but the same console and xenstore daemons are
> used. The tricky part is getting the grant references out of the
> domain builder without changing the interface to xc_linux_build().
> Would it be reasonable to assume that the current versions of xend
> and libxc are consistent, and therefore the fixed grant references
> could be written into xenstore (from constants in the Python code) at
> the same time as the console and xenstore mfns are?
I would accept that degree of binding between libxenctrl.so and the Python
binding Xc.so, yes. The existing xenstore arguments/fields might already be
unhelpfully called grant-ref, by the way. :-) Very stupid of me if so.
You''ll just have to be a bit imaginative about naming of new
args/fields in
that case.
> This should be quite simple: if the call to xc_gnttab_open() fails,
> it could set a flag which determines the method to be used. All that
> needs to be done in the mean time is to add stub functions to
> xc_solaris.c - I''ll do that.
It''s as simple as try grant-table method, fall back to direct-map
method,
then fail outright, isn''t it? Maybe that''s what you meant.

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 2 May 2007, at 15:48, Keir Fraser wrote:
> I would accept that degree of binding between libxenctrl.so and the  
> Python
> binding Xc.so, yes. The existing xenstore arguments/fields might  
> already be
> unhelpfully called grant-ref, by the way. :-) Very stupid of me if so.
> You''ll just have to be a bit imaginative about naming of new args/
> fields in
> that case.
Not quite so bad - the current keys are called "ring-ref", so
I''ll
probably add "ring-grant-ref" for the new entries.
> It''s as simple as try grant-table method, fall back to direct-map
> method,
> then fail outright, isn''t it? Maybe that''s what you
meant.
It''s even simpler in practice. If the gntdev isn''t available,
then
xc_gnttab_open() will fail (as long as there''s a stub in the Solaris  
version of libxc), so there will be no "xcg" handle with which to try
to map the grant reference. Therefore we can go straight to the  
direct map method. I suppose, in addition, if we do have an xcg  
handle and the grant mapping fails for some reason, we could still  
fall back to the direct map method, though this failure should  
definitely be noted.

Regards,

Derek.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, May 02, 2007 at 12:15:18PM +0100, Derek Murray
wrote:
> * Will this work on ia64 and PowerPC?
With the attached patches, I was able to boot dom0 and create/destroy domU.
I used gcc (GCC) 3.4.4 20050721 (Red Hat 3.4.4-2).

BTW save/restore doesn''t seem to work because when domain is
restored grant table is reset so that grant table entries should be
reinstalled again before introducing domain.

-- 
yamahata



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 7 May 2007, at 09:46, Isaku Yamahata wrote:
> On Wed, May 02, 2007 at 12:15:18PM +0100, Derek Murray wrote:
>> * Will this work on ia64 and PowerPC?
>
> With the attached patches, I was able to boot dom0 and create/ 
> destroy domU.
> I used gcc (GCC) 3.4.4 20050721 (Red Hat 3.4.4-2).
Thanks for these patches! I''ll incorporate them into the revised  
version of my patchset. It''s good to know that gntdev is working on  
these architectures.
> BTW save/restore doesn''t seem to work because when domain is
> restored grant table is reset so that grant table entries should be
> reinstalled again before introducing domain.
I''ll take care of this as well.

Regards,

Derek Murray.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, May 08, 2007 at 03:40:41PM +0100, Derek Murray
wrote:
> 
> On 7 May 2007, at 09:46, Isaku Yamahata wrote:
> 
> >On Wed, May 02, 2007 at 12:15:18PM +0100, Derek Murray wrote:
> >>* Will this work on ia64 and PowerPC?
> >
> >With the attached patches, I was able to boot dom0 and create/ 
> >destroy domU.
> >I used gcc (GCC) 3.4.4 20050721 (Red Hat 3.4.4-2).
> 
> Thanks for these patches! I''ll incorporate them into the revised  
> version of my patchset. It''s good to know that gntdev is working
on
> these architectures.
I forgot to say that I only tested on IA64.

-- 
yamahata

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel