Xen devel - Jun 2006 - Where can I find some tutorial or manual on how to use xenstore?

I found from xen block and network driver that xenstore seems to need python
code for initialization and configuration. Is this true?
But the python code seems to be complicated.

Is there any complete tutorial or manual on how to use xenstore?

Thanks!
-lily


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

In article <4d616cca0606190013h3922ae8i155de76067d35083@mail.gmail.com>
you wrote:
> [-- multipart/alternative, encoding 7bit, 2 lines --]
> 
>    [-- text/plain, encoding 7bit, charset: ISO-8859-1, 9 lines --]
> 
> I found from xen block and network driver that xenstore seems to need
python
> code for initialization and configuration. Is this true?
> But the python code seems to be complicated.
I believe that the answer is that xenstore needs asistance from
code running in userspace in domain U, and currently the only
implemntation of that code is in python.

-- 
Horms                                           http://www.vergenet.net/~horms/


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 6/19/06, Lily Huang <ushuanglily@gmail.com>
wrote:
> I found from xen block and network driver that xenstore seems to need
python
> code for initialization and configuration. Is this true?
> But the python code seems to be complicated.
>
> Is there any complete tutorial or manual on how to use xenstore?
Check out the tools/migrate/buscreate.c tool in
http://www.distlab.dk/hg/index.cgi/xen-evilman.hg , it creates a
domain with disk and network access, without needing python.

Jacob

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Thanks a lot!

This seems to be good for setting up block and network driver. But can the
same code used for new drivers?

BTW: why do you call your repository evilman? ;-)

lily

On 6/25/06, Jacob Gorm Hansen <jacobg@diku.dk>
wrote:
>
> On 6/19/06, Lily Huang <ushuanglily@gmail.com> wrote:
> > I found from xen block and network driver that xenstore seems to need
> python
> > code for initialization and configuration. Is this true?
> > But the python code seems to be complicated.
> >
> > Is there any complete tutorial or manual on how to use xenstore?
>
> Check out the tools/migrate/buscreate.c tool in
> http://www.distlab.dk/hg/index.cgi/xen-evilman.hg , it creates a
> domain with disk and network access, without needing python.
>
> Jacob
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 6/25/06, Lily Huang <ushuanglily@gmail.com>
wrote:
> Thanks a lot!
>
> This seems to be good for setting up block and network driver. But can the
> same code used for new drivers?
No, you will need to adapt it a bit, but you can likely use one of the
setup_vbd() or setup_vif() functions as template. Turning on the
printf-enabled version of the xs_w() macro will probably also provide
helpful information.
> BTW: why do you call your repository evilman? ;-)
That is a good question, please see the explanation in the paper "The
Laundromat Model for Autonomic Cluster Computing", which just appear
at ICAC06 and is available from my web page, at
http://www.diku.dk/~jacobg ;-)

Regards,
Jacob

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

It''s certainly possible, I''ve used this code as the basis for
a new
driver, which is similar to the vbd driver. The driver starts fine but I 
am unable to have it participate in a domain save/restore. I guess that 
this is because xend is not aware of  the new driver and the open 
devices supported by this driver and hence the driver does not receive 
resume calls when the domain is restored..

Is there any way that a new driver can make it''s existance known to
xend?

Nick
>
>Message: 3
>Date: Sun, 25 Jun 2006 15:00:49 -0400
>From: "Lily Huang" <ushuanglily@gmail.com>
>Subject: Re: [Xen-devel] Where can I find some tutorial or manual on
>	how to	use xenstore?
>To: "Jacob Gorm Hansen" <jacobg@diku.dk>
>Cc: xen-devel <xen-devel@lists.xensource.com>
>Message-ID:
>	<4d616cca0606251200q1103ec01q90799667882c4fcb@mail.gmail.com>
>Content-Type: text/plain; charset="iso-8859-1"
>
>Thanks a lot!
>
>This seems to be good for setting up block and network driver. But can the
>same code used for new drivers?
>
>BTW: why do you call your repository evilman? ;-)
>
>lily
>
>On 6/25/06, Jacob Gorm Hansen <jacobg@diku.dk> wrote:
>  
>
>>On 6/19/06, Lily Huang <ushuanglily@gmail.com> wrote:
>>    
>>
>>>I found from xen block and network driver that xenstore seems to
need
>>>      
>>>
>>python
>>    
>>
>>>code for initialization and configuration. Is this true?
>>>But the python code seems to be complicated.
>>>
>>>Is there any complete tutorial or manual on how to use xenstore?
>>>      
>>>
>>Check out the tools/migrate/buscreate.c tool in
>>http://www.distlab.dk/hg/index.cgi/xen-evilman.hg , it creates a
>>domain with disk and network access, without needing python.
>>
>>Jacob
>>
>>    
>>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Jun 28, 2006 at 12:08:33PM +0100, Nick Logan wrote:
> It''s certainly possible, I''ve used this code as the basis
for a new
> driver, which is similar to the vbd driver. The driver starts fine but I 
> am unable to have it participate in a domain save/restore. I guess that 
> this is because xend is not aware of  the new driver and the open 
> devices supported by this driver and hence the driver does not receive 
> resume calls when the domain is restored..
> 
> Is there any way that a new driver can make it''s existance known
to xend?
If you''ve managed to get the driver to start through xend, I''d
be surprised if
it''s xend''s fault that save/restore does not work.

In any case, the main driver integration point is at the bottom of
tools/python/xen/xend/XendDomainInfo.py, where you can register a subclass of
xen.xend.server.DevController.py to manage the device.  You also need to make
sure that the driver''s config is parsed properly by xm.

Take a look at Julian Chesterfield and Andrew Warfield''s recent blktap
patches
to see how driver tool integration is done:

http://lists.xensource.com/archives/html/xen-devel/2006-06/msg00738.html

Cheers,

Ewan.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Thanks Ewan,

See below.

Cheers,

Nick

Ewan Mellor wrote:
>On Wed, Jun 28, 2006 at 12:08:33PM +0100, Nick Logan wrote:
>
>  
>
>>It''s certainly possible, I''ve used this code as the
basis for a new
>>driver, which is similar to the vbd driver. The driver starts fine but I
>>am unable to have it participate in a domain save/restore. I guess that 
>>this is because xend is not aware of  the new driver and the open 
>>devices supported by this driver and hence the driver does not receive 
>>resume calls when the domain is restored..
>>
>>Is there any way that a new driver can make it''s existance
known to xend?
>>    
>>
>
>If you''ve managed to get the driver to start through xend,
I''d be surprised if
>it''s xend''s fault that save/restore does not work.
>  
>
The driver was started outside of xend, using a varient of Jacob''s 
buscreate program to set the necessary values in xenstore. As this a 3rd 
party driver, I''m looking for a solution that does not involve xm or 
xend changes, if that''s possible. I''ll take a look at the
blktap patches
to see if that helps.
>In any case, the main driver integration point is at the bottom of
>tools/python/xen/xend/XendDomainInfo.py, where you can register a subclass
of
>xen.xend.server.DevController.py to manage the device.  You also need to
make
>sure that the driver''s config is parsed properly by xm.
>
>Take a look at Julian Chesterfield and Andrew Warfield''s recent
blktap patches
>to see how driver tool integration is done:
>
>http://lists.xensource.com/archives/html/xen-devel/2006-06/msg00738.html
>
>Cheers,
>
>Ewan.
>  
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Jun 28, 2006 at 02:38:15PM +0100, Nick Logan wrote:
> The driver was started outside of xend, using a varient of Jacob''s
> buscreate program to set the necessary values in xenstore. As this a 3rd 
> party driver, I''m looking for a solution that does not involve xm
or
> xend changes, if that''s possible. I''ll take a look at the
blktap patches
> to see if that helps.
Xend is explicitly bringing the devices back up on restore.  If you
deliberately bypass it, then you are going to have to do that bringup
yourself.

Ewan.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ewan Mellor wrote:
>On Wed, Jun 28, 2006 at 02:38:15PM +0100, Nick Logan wrote:
>
>  
>
>>The driver was started outside of xend, using a varient of
Jacob''s
>>buscreate program to set the necessary values in xenstore. As this a 3rd
>>party driver, I''m looking for a solution that does not involve
xm or
>>xend changes, if that''s possible. I''ll take a look at
the blktap patches
>>to see if that helps.
>>    
>>
>
>Xend is explicitly bringing the devices back up on restore.  If you
>deliberately bypass it, then you are going to have to do that bringup
>yourself.
>
>Ewan.
>  
>
I guessed that would be the case. The bringup for a restore would be 
quite straighforward but more complex for migration. Has anyone 
suggested hooks for xend to deal with 3rd party drivers so that it could 
initialise and restore devices that are supported by these drivers?  
This would enable new drivers to be implemented without changes to xend.

Cheers,

Nick

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Jun 28, 2006 at 04:20:51PM +0100, Nick Logan wrote:
> Ewan Mellor wrote:
> 
> >On Wed, Jun 28, 2006 at 02:38:15PM +0100, Nick Logan wrote:
> >
> > 
> >
> >>The driver was started outside of xend, using a varient of
Jacob''s
> >>buscreate program to set the necessary values in xenstore. As this
a 3rd
> >>party driver, I''m looking for a solution that does not
involve xm or
> >>xend changes, if that''s possible. I''ll take a
look at the blktap patches
> >>to see if that helps.
> >>   
> >>
> >
> >Xend is explicitly bringing the devices back up on restore.  If you
> >deliberately bypass it, then you are going to have to do that bringup
> >yourself.
> >
> >Ewan.
> > 
> >
> I guessed that would be the case. The bringup for a restore would be 
> quite straighforward but more complex for migration. Has anyone 
> suggested hooks for xend to deal with 3rd party drivers so that it could 
> initialise and restore devices that are supported by these drivers?  
> This would enable new drivers to be implemented without changes to xend.
One would have to write a device handler that parsed generic config, and then
passed it through to the store unaltered, and then have hotplug scripts that
could cope with this unparsed config.  At the moment, the driver backends do
special-case things like generating a MAC address when none is supplied,
converting device names to their major:minor, that kind of thing.  There
isn''t
a generic device path; adding one wouldn''t be hard.

Ewan.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ewan Mellor wrote:
>On Wed, Jun 28, 2006 at 04:20:51PM +0100, Nick Logan wrote:
>
>  
>
>>Ewan Mellor wrote:
>>
>>    
>>
>>>On Wed, Jun 28, 2006 at 02:38:15PM +0100, Nick Logan wrote:
>>>
>>>
>>>
>>>      
>>>
>>>>The driver was started outside of xend, using a varient of
Jacob''s
>>>>buscreate program to set the necessary values in xenstore. As
this a 3rd
>>>>party driver, I''m looking for a solution that does not
involve xm or
>>>>xend changes, if that''s possible. I''ll take a
look at the blktap patches
>>>>to see if that helps.
>>>>  
>>>>
>>>>        
>>>>
>>>Xend is explicitly bringing the devices back up on restore.  If you
>>>deliberately bypass it, then you are going to have to do that
bringup
>>>yourself.
>>>
>>>Ewan.
>>>
>>>
>>>      
>>>
>>I guessed that would be the case. The bringup for a restore would be 
>>quite straighforward but more complex for migration. Has anyone 
>>suggested hooks for xend to deal with 3rd party drivers so that it could
>>initialise and restore devices that are supported by these drivers?  
>>This would enable new drivers to be implemented without changes to xend.
>>    
>>
>
>One would have to write a device handler that parsed generic config, and
then
>passed it through to the store unaltered, and then have hotplug scripts that
>could cope with this unparsed config.  At the moment, the driver backends do
>special-case things like generating a MAC address when none is supplied,
>converting device names to their major:minor, that kind of thing.  There
isn''t
>a generic device path; adding one wouldn''t be hard.
>
>Ewan.
>  
>
Can I just confirm what your proposal is:

Add a generic device config to the xm config file that would contain the 
driver id, virtual device, physical device .....
Add a generic device handler to xend that would pass the unparsed 
generic config through to xenstore.
Add a hotplug script that would parse the unparsed config in xenstore 
and start the driver.
When saving a domain, xend would save the generic device config.
xend would call restore for the generic devices.

If  that''s correct, I''ll find some time to investigate how
this could done.

Nick


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, Jun 29, 2006 at 02:53:26PM +0100, Nick Logan wrote:
> Ewan Mellor wrote:
> 
> >On Wed, Jun 28, 2006 at 04:20:51PM +0100, Nick Logan wrote:
> >
> > 
> >
> >>Ewan Mellor wrote:
> >>
> >>   
> >>
> >>>On Wed, Jun 28, 2006 at 02:38:15PM +0100, Nick Logan wrote:
> >>>
> >>>
> >>>
> >>>     
> >>>
> >>>>The driver was started outside of xend, using a varient of
Jacob''s
> >>>>buscreate program to set the necessary values in xenstore.
As this a
> >>>>3rd party driver, I''m looking for a solution that
does not involve xm
> >>>>or xend changes, if that''s possible. I''ll
take a look at the blktap
> >>>>patches to see if that helps.
> >>>> 
> >>>>
> >>>>       
> >>>>
> >>>Xend is explicitly bringing the devices back up on restore.  If
you
> >>>deliberately bypass it, then you are going to have to do that
bringup
> >>>yourself.
> >>>
> >>>Ewan.
> >>>
> >>>
> >>>     
> >>>
> >>I guessed that would be the case. The bringup for a restore would
be
> >>quite straighforward but more complex for migration. Has anyone 
> >>suggested hooks for xend to deal with 3rd party drivers so that it
could
> >>initialise and restore devices that are supported by these drivers?
> >>This would enable new drivers to be implemented without changes to
xend.
> >>   
> >>
> >
> >One would have to write a device handler that parsed generic config,
and
> >then
> >passed it through to the store unaltered, and then have hotplug scripts
> >that
> >could cope with this unparsed config.  At the moment, the driver
backends
> >do
> >special-case things like generating a MAC address when none is
supplied,
> >converting device names to their major:minor, that kind of thing. 
There
> >isn''t
> >a generic device path; adding one wouldn''t be hard.
> >
> >Ewan.
> > 
> >
> Can I just confirm what your proposal is:
> 
> Add a generic device config to the xm config file that would contain the 
> driver id, virtual device, physical device .....
> Add a generic device handler to xend that would pass the unparsed 
> generic config through to xenstore.
> Add a hotplug script that would parse the unparsed config in xenstore 
> and start the driver.
> When saving a domain, xend would save the generic device config.
> xend would call restore for the generic devices.
> 
> If  that''s correct, I''ll find some time to investigate
how this could done.
Yes, that''s it.  It sounds like something that would be very useful.

Cheers,

Ewan.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

I also agree that this would be very useful, and I think the drivers
are stable enough now that it''s a good idea.

In addition to the generic xend integration, it would be quite good if
the skeleton frontend/backend contained minimal demonstrator code to
set up a device channel (a.k.a. shared memory page and event mapping)
between the two VMs.

We''ve talked about having something like this in the tree in the past,
it would clearly be useful for people wanting to write new drivers,
but I think there was fear that it would rot too quickly (and it
certainly would have in the past).  If the skeleton driver did
something simple but measurable (for correctness) like ping-pong
between the frontend and backend, we could add it to XenRT and make
sure that it stayed up to date.

just a thought...

a.

On 6/29/06, Ewan Mellor <ewan@xensource.com>
wrote:
> On Thu, Jun 29, 2006 at 02:53:26PM +0100, Nick Logan wrote:
>
> > Ewan Mellor wrote:
> >
> > >On Wed, Jun 28, 2006 at 04:20:51PM +0100, Nick Logan wrote:
> > >
> > >
> > >
> > >>Ewan Mellor wrote:
> > >>
> > >>
> > >>
> > >>>On Wed, Jun 28, 2006 at 02:38:15PM +0100, Nick Logan
wrote:
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>>The driver was started outside of xend, using a
varient of Jacob''s
> > >>>>buscreate program to set the necessary values in
xenstore. As this a
> > >>>>3rd party driver, I''m looking for a solution
that does not involve xm
> > >>>>or xend changes, if that''s possible.
I''ll take a look at the blktap
> > >>>>patches to see if that helps.
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>Xend is explicitly bringing the devices back up on
restore.  If you
> > >>>deliberately bypass it, then you are going to have to do
that bringup
> > >>>yourself.
> > >>>
> > >>>Ewan.
> > >>>
> > >>>
> > >>>
> > >>>
> > >>I guessed that would be the case. The bringup for a restore
would be
> > >>quite straighforward but more complex for migration. Has
anyone
> > >>suggested hooks for xend to deal with 3rd party drivers so
that it could
> > >>initialise and restore devices that are supported by these
drivers?
> > >>This would enable new drivers to be implemented without
changes to xend.
> > >>
> > >>
> > >
> > >One would have to write a device handler that parsed generic
config, and
> > >then
> > >passed it through to the store unaltered, and then have hotplug
scripts
> > >that
> > >could cope with this unparsed config.  At the moment, the driver
backends
> > >do
> > >special-case things like generating a MAC address when none is
supplied,
> > >converting device names to their major:minor, that kind of thing. 
There
> > >isn''t
> > >a generic device path; adding one wouldn''t be hard.
> > >
> > >Ewan.
> > >
> > >
> > Can I just confirm what your proposal is:
> >
> > Add a generic device config to the xm config file that would contain
the
> > driver id, virtual device, physical device .....
> > Add a generic device handler to xend that would pass the unparsed
> > generic config through to xenstore.
> > Add a hotplug script that would parse the unparsed config in xenstore
> > and start the driver.
> > When saving a domain, xend would save the generic device config.
> > xend would call restore for the generic devices.
> >
> > If  that''s correct, I''ll find some time to
investigate how this could done.
>
> Yes, that''s it.  It sounds like something that would be very
useful.
>
> Cheers,
>
> Ewan.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>> Can I just confirm what your proposal is:
>>
>> Add a generic device config to the xm config file that would contain
the
>> driver id, virtual device, physical device .....
>> Add a generic device handler to xend that would pass the unparsed 
>> generic config through to xenstore.
>> Add a hotplug script that would parse the unparsed config in xenstore 
>> and start the driver.
>> When saving a domain, xend would save the generic device config.
>> xend would call restore for the generic devices.
>>
>> If  that''s correct, I''ll find some time to
investigate how this could done.
>>     
>
> Yes, that''s it.  It sounds like something that would be very
useful.
>
> Cheers,
>
> Ewan.
>   
I can''t promise that I''ll get onto this straightaway, but if
we''re going
to go ahead with a new driver, we''ll have to have some way of dealing 
with save/restore and migration. This looks like a way to solve the problem.

Cheers,

Nick

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel