Xen devel - Jul 2008 - Re: [Qemu-devel] [PATCH 0/7] merge some xen bits into qemu

Gerd Hoffmann writes ("[Qemu-devel] [PATCH 0/7] merge some xen bits into
qemu"):
> Here are a bunch of patches which start adding xen support to qemu.
> Overview (individual patches have longer descriptions):
Just to clarify: as far as I can tell from the description,
this code has scant relationship with Xen upstream.

I''m generally in favour of pushing functionality out of the Xen branch
of qemu into upstream.  But going by Gerd Hoffman''s description I
don''t think that''s what his branch is.  His code
doesn''t appear to be
related to the Xen branch of qemu that we are using.

For example,
> With the first four patches in place upstream qemu can replace
xen''s
> qemu-dm for paravirtual domains.  The block and nic backend drivers are
> full userspace implementations using the grant table device (gntdev).
we only use qemu-dm in paravirtualised domains for certain marginal
functions - in many cases it is not used at all.  Certainly the
functionality Gerd describes, such as xen backend block and network
drivers, are not in our qemu tree and we do not intend for them to be
there.

In a Xen installation this functionality is in the dom0 (host) kernel.


As far as I can see much of Gerd Hoffman''s intended submission is
effectively an _emulator_ for Xen guests.  That is, it emulates a Xen
host without being one, so that a Xen domU can be run without the Xen
hypervisor.  Am I right, Gerd ?

I don''t think there''s anything wrong with that from a qemu
point of
view.  Obviously we think that the genuine Xen hypervisor is much
better :-) but it is right for people to have a choice, and having
qemu emulate more environments is generally good.

But if this functionality is to go into qemu upstream perhaps it
should be distinguished from `real Xen'' functions, because otherwise
users are going to become very very confused.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Jackson wrote:
> Gerd Hoffmann writes ("[Qemu-devel] [PATCH 0/7] merge some xen bits
into qemu"):
>   
>> Here are a bunch of patches which start adding xen support to qemu.
>> Overview (individual patches have longer descriptions):
>>     
>
> Just to clarify: as far as I can tell from the description,
> this code has scant relationship with Xen upstream.
>
> I''m generally in favour of pushing functionality out of the Xen
branch
> of qemu into upstream.  But going by Gerd Hoffman''s description I
> don''t think that''s what his branch is.  His code
doesn''t appear to be
> related to the Xen branch of qemu that we are using.
>   
I think it''s more closely related to Xenite and Xenner.  Gerd: are you 
planning on folding in domain creation?  Right now it appears to be a 
helper launched after the domain creation.
> For example,
>
>   
>> With the first four patches in place upstream qemu can replace
xen''s
>> qemu-dm for paravirtual domains.  The block and nic backend drivers are
>> full userspace implementations using the grant table device (gntdev).
>>     
>
> we only use qemu-dm in paravirtualised domains for certain marginal
> functions - in many cases it is not used at all.  Certainly the
> functionality Gerd describes, such as xen backend block and network
> drivers, are not in our qemu tree and we do not intend for them to be
> there.
>   
There''s no reason they couldn''t be though.  It''s just
like blktap.
> In a Xen installation this functionality is in the dom0 (host) kernel.
>
>
> As far as I can see much of Gerd Hoffman''s intended submission is
> effectively an _emulator_ for Xen guests.  That is, it emulates a Xen
> host without being one, so that a Xen domU can be run without the Xen
> hypervisor.  Am I right, Gerd ?
>   
No, it''s definitely for use with Xen (hypervisor).  But it''s
different
architecturally from how Xen uses QEMU in xen-unstable.

Personally, I really like the way Gerd has the code structured.  It 
seems like it could be used as almost a drop-in replacement for qemu-dm 
for PV guests.  HVM guests would require more work.  Of course, the 
net/disk support can be completely optionally.

With that said, it would be silly to have two Xen implementations within 
QEMU so there has to be some general agreement in the Xen community 
about how QEMU is going to be used before merging would make sense.  
That probably requires some discussion about how the HVM support would 
fit into this architecture.

Regards,

Anthony Liguori


> I don''t think there''s anything wrong with that from a
qemu point of
> view.  Obviously we think that the genuine Xen hypervisor is much
> better :-) but it is right for people to have a choice, and having
> qemu emulate more environments is generally good.
>
> But if this functionality is to go into qemu upstream perhaps it
> should be distinguished from `real Xen'' functions, because
otherwise
> users are going to become very very confused.
>
> Ian.
>
>
>   

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Jackson wrote:
> Gerd Hoffmann writes ("[Qemu-devel] [PATCH 0/7] merge some xen bits
into qemu"):
>> Here are a bunch of patches which start adding xen support to qemu.
>> Overview (individual patches have longer descriptions):
> 
> Just to clarify: as far as I can tell from the description,
> this code has scant relationship with Xen upstream.
Patches #3 + #4 are largely based on qemu-dm code (as noted in the
individual patch  descriptions).
> I''m generally in favour of pushing functionality out of the Xen
branch
> of qemu into upstream.  But going by Gerd Hoffman''s description I
> don''t think that''s what his branch is.  His code
doesn''t appear to be
> related to the Xen branch of qemu that we are using.
I want to merge the *functionality*.  IMHO that doesn''t mean that it
must be the *code* used by Xen at the moment.
> For example,
> 
>> With the first four patches in place upstream qemu can replace
xen''s
>> qemu-dm for paravirtual domains.  The block and nic backend drivers are
>> full userspace implementations using the grant table device (gntdev).
> 
> we only use qemu-dm in paravirtualised domains for certain marginal
> functions - in many cases it is not used at all.
It''s used for xen console (optionally, can also be handled by
xenconsoled) and the virtual framebuffer.
> Certainly the
> functionality Gerd describes, such as xen backend block and network
> drivers, are not in our qemu tree and we do not intend for them to be
> there.
I want them be there though.  You can use them or not, that is your call.
> In a Xen installation this functionality is in the dom0 (host) kernel.
That is only half the story.  The block backend can run in userspace too
(when using blktap).  The block backend driver should be pretty much
identical to blktap functionality-wise.  The implementation is quite
different though.
> As far as I can see much of Gerd Hoffman''s intended submission is
> effectively an _emulator_ for Xen guests.  That is, it emulates a Xen
> host without being one, so that a Xen domU can be run without the Xen
> hypervisor.  Am I right, Gerd ?
That is part of the longterm plan, yes.  I want qemu being able to do
both, run as device model for xen and also to boot xen guest images
without xen, using emulation.  If the intention would have been
emulation only I wouldn''t have Cc''ed xen-devel for patch
review.

The emulation bits are not in that patchset btw.
> But if this functionality is to go into qemu upstream perhaps it
> should be distinguished from `real Xen'' functions, because
otherwise
> users are going to become very very confused.
Huh?

cheers,
  Gerd

-- 
http://kraxel.fedorapeople.org/xenner/

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Anthony Liguori writes ("Re: [Qemu-devel] [PATCH 0/7] merge some xen bits
into qemu"):
> I think it''s more closely related to Xenite and Xenner.  Gerd: are
you
> planning on folding in domain creation?  Right now it appears to be a 
> helper launched after the domain creation.
...
> No, it''s definitely for use with Xen (hypervisor).  But
it''s different
> architecturally from how Xen uses QEMU in xen-unstable.
Xenner is an emulator for allowing Xen domUs to be booted without the
Xen hypervisor.

Xennite is an experimental replacement for the Xen userland management
stack in dom0: it moves more functionality from the Xen tools in dom0
into the qemu-dm process.  This is moving in almost the opposite
direction to Xen upstream is moving: we are moving qemu-dm into its
own tiny domain, so that the qemu code doesn''t need to run as a
process in dom0; this has important security and scalability
advantages.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, 28 Jul 2008, Ian Jackson wrote:
> Xennite is an experimental replacement for the Xen userland management
> stack in dom0: it moves more functionality from the Xen tools in dom0
> into the qemu-dm process.  This is moving in almost the opposite
> direction to Xen upstream is moving: we are moving qemu-dm into its
> own tiny domain, so that the qemu code doesn''t need to run as a
> process in dom0; this has important security and scalability
> advantages.
I think your userbase prefers the way Red Hat goes. If you do a reality
check on the current Python implementation and its memory usage, it is so
far from an ESX equivalent that I put my money on any Qemu userspace
version.

So if you say this new domain will not take at least 128MB extra memory,
that could be interesting.


Stefan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Anthony Liguori wrote:
> I think it''s more closely related to Xenite and Xenner.  Gerd: are
you
> planning on folding in domain creation?  Right now it appears to be a
> helper launched after the domain creation.
Not sure yet.  Given that xen upstream most likely wouldn''t follow a
xenite-like model I''m not sure how useful it would be. 
Shouldn''t be
that much extra code though, and as long as it is an opt-in upstream xen
should be happy too ...
> It
> seems like it could be used as almost a drop-in replacement for qemu-dm
> for PV guests.
Yes, that is the intention.
> HVM guests would require more work.
Yes.  Not addressed (yet).  hvm is more code, more invasive and I also
hope that Glauber Costa''s work on a cpu abstraction layer (for
emulation/kqemu/kvm) makes it easier to add xen hvm to the picture.

cheers,
  Gerd

-- 
http://kraxel.fedorapeople.org/xenner/

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Jackson wrote:
> Anthony Liguori writes ("Re: [Qemu-devel] [PATCH 0/7] merge some xen
bits into qemu"):
>   
>> I think it''s more closely related to Xenite and Xenner.  Gerd:
are you
>> planning on folding in domain creation?  Right now it appears to be a 
>> helper launched after the domain creation.
>>     
> ...
>   
>> No, it''s definitely for use with Xen (hypervisor).  But
it''s different
>> architecturally from how Xen uses QEMU in xen-unstable.
>>     
>
> Xenner is an emulator for allowing Xen domUs to be booted without the
> Xen hypervisor.
>   
Or "shim".  It''s almost architecturally identical to the
XenSource
developed "shim" for Hyper-V.  It seems like a popular thing to do
these
days :-)
> Xennite is an experimental replacement for the Xen userland management
> stack in dom0: it moves more functionality from the Xen tools in dom0
> into the qemu-dm process.  This is moving in almost the opposite
> direction to Xen upstream is moving: we are moving qemu-dm into its
> own tiny domain, so that the qemu code doesn''t need to run as a
> process in dom0; this has important security and scalability
> advantages.
>   
Are there separate requirements for stub domains verses Xennite?

Gerd''s code is different than what''s upstream in QEMU but the
question
is whether it''s reconcilable with what''s upstream.  If not,
what makes
it that way?

Regards,

Anthony Liguori
> Ian.
>   

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan de Konink, le Mon 28 Jul 2008 17:22:39 +0200, a écrit
:
> > This is moving in almost the opposite
> > direction to Xen upstream is moving: we are moving qemu-dm into its
> > own tiny domain, so that the qemu code doesn''t need to run as
a
> > process in dom0; this has important security and scalability
> > advantages.
> 
> I think your userbase prefers the way Red Hat goes. If you do a reality
> check on the current Python implementation and its memory usage, it is so
> far from an ESX equivalent that I put my money on any Qemu userspace
> version.
> 
> So if you say this new domain will not take at least 128MB extra memory,
> that could be interesting.
Err...  Currently the default allocated memory is 32MB because there are
still some bloats, but there is no reason why qemu-dm in its own domain
should take much more than qemu-dm in dom0.  Currently it should be able
to fit within 16MB.

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Gerd Hoffmann, le Mon 28 Jul 2008 16:43:54 +0200, a écrit
:
> Ian Jackson wrote:
> > I''m generally in favour of pushing functionality out of the
Xen branch
> > of qemu into upstream.  But going by Gerd Hoffman''s
description I
> > don''t think that''s what his branch is.  His code
doesn''t appear to be
> > related to the Xen branch of qemu that we are using.
> 
> I want to merge the *functionality*.  IMHO that doesn''t mean that
it
> must be the *code* used by Xen at the moment.
Well, the patch comments weren''t so clear about that.

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, 29 Jul 2008, Samuel Thibault wrote:
> Stefan de Konink, le Mon 28 Jul 2008 17:22:39 +0200, a écrit :
> > > This is moving in almost the opposite
> > > direction to Xen upstream is moving: we are moving qemu-dm into
its
> > > own tiny domain, so that the qemu code doesn''t need to
run as a
> > > process in dom0; this has important security and scalability
> > > advantages.
> >
> > I think your userbase prefers the way Red Hat goes. If you do a
reality
> > check on the current Python implementation and its memory usage, it is
so
> > far from an ESX equivalent that I put my money on any Qemu userspace
> > version.
> >
> > So if you say this new domain will not take at least 128MB extra
memory,
> > that could be interesting.
>
> Err...  Currently the default allocated memory is 32MB because there are
> still some bloats, but there is no reason why qemu-dm in its own domain
> should take much more than qemu-dm in dom0.  Currently it should be able
> to fit within 16MB.
And you don''t count the 331MB virtual memory the process takes, and
every
tapdisk that is created?

But I would love to see the 16MB version...

Stefan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan de Konink, le Tue 29 Jul 2008 01:27:24 +0200, a écrit
:
> On Tue, 29 Jul 2008, Samuel Thibault wrote:
> 
> > Stefan de Konink, le Mon 28 Jul 2008 17:22:39 +0200, a écrit :
> > > > This is moving in almost the opposite
> > > > direction to Xen upstream is moving: we are moving qemu-dm
into its
> > > > own tiny domain, so that the qemu code doesn''t need
to run as a
> > > > process in dom0; this has important security and scalability
> > > > advantages.
> > >
> > > I think your userbase prefers the way Red Hat goes. If you do a
reality
> > > check on the current Python implementation and its memory usage,
it is so
> > > far from an ESX equivalent that I put my money on any Qemu
userspace
> > > version.
> > >
> > > So if you say this new domain will not take at least 128MB extra
memory,
> > > that could be interesting.
> >
> > Err...  Currently the default allocated memory is 32MB because there
are
> > still some bloats, but there is no reason why qemu-dm in its own
domain
> > should take much more than qemu-dm in dom0.  Currently it should be
able
> > to fit within 16MB.
> 
> And you don''t count the 331MB virtual memory the process takes,
and every
> tapdisk that is created?
That depends on what you have configured of course. Virtual memory
doesn''t account at all, most of it just belongs to the guest.
> But I would love to see the 16MB version...
Well, just configure the use of stubdom-dm, and to see the free memory
(of the allocated 32MB), compile with #define LIBC_VERBOSE uncommented
in extras/mini-os/lib/sys.c

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan de Konink wrote:
> And you don''t count the 331MB virtual memory the process takes,
and every
> tapdisk that is created?
Tapdisk doesn''t need physical memory.  It is address space used to map
in guest pages.

cheers,
  Gerd

-- 
http://kraxel.fedorapeople.org/xenner/

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, Jul 28, 2008 at 03:58:15PM +0100, Ian Jackson
wrote:
> Anthony Liguori writes ("Re: [Qemu-devel] [PATCH 0/7] merge some xen
bits into qemu"):
> > I think it''s more closely related to Xenite and Xenner. 
Gerd: are you
> > planning on folding in domain creation?  Right now it appears to be a 
> > helper launched after the domain creation.
> ...
> > No, it''s definitely for use with Xen (hypervisor).  But
it''s different
> > architecturally from how Xen uses QEMU in xen-unstable.
> 
> Xenner is an emulator for allowing Xen domUs to be booted without the
> Xen hypervisor.
> 
> Xennite is an experimental replacement for the Xen userland management
> stack in dom0: it moves more functionality from the Xen tools in dom0
> into the qemu-dm process.  This is moving in almost the opposite
> direction to Xen upstream is moving: we are moving qemu-dm into its
> own tiny domain, so that the qemu code doesn''t need to run as a
> process in dom0; this has important security and scalability
> advantages.
Yes, to be clear that the Xennite code is *not* an official Xen project. 

It is my experimental work and I don''t expect anyone to use it for real
in the near future. It does however align very well with Gerd''s serious
Xenner project (which is incredibly useful tool allowing admins to mix
and match Xen and KVM) since both Xenner & Xennite have much same needs
in  terms of QEMU support code

The motiviation with ''Xennite'' is to make the process of
launching
and managing an instance of a Xen guest, as close as possible to that
of KVM. Start QEMU to launch the Xen guest, interact with the monitor
to control it, and kill QEMU to destroy the Xen guest. I''d class it as
a proof-of-concept since I never got around to making stuff like save
and restore work - I postponed further work until Xen re-synced with a
new QEMU codebase.  We could have a long discussion about the pros/cons
of this idea vs the upstream Xen move to QEMU in stub domains, but
that''s
off topic for this list, so I''ll leave it :-)  

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, 29 Jul 2008, Gerd Hoffmann wrote:
> Stefan de Konink wrote:
>
> > And you don''t count the 331MB virtual memory the process
takes, and every
> > tapdisk that is created?
>
> Tapdisk doesn''t need physical memory.  It is address space used to
map
> in guest pages.
Maybe you use a different version of Xen... I''m using 3.2.1 until the
Dell
patches of the 2.6.19 kernel are backported, or I get the Vanilla kernel
running on 64bits.


root      3005  0.0  0.1   8404  1012 ?        S    Jul26   0:00 xenstored
--pid-file /var/run/xenstore.pid
root      3009  0.0  1.8  99416  9936 ?        S    Jul26   0:00 python
/usr/sbin/xend start
root      3011  0.0  0.1  20540   680 ?        Sl   Jul26   0:00
xenconsoled
root      3013  0.0 10.7 242876 56492 ?        Sl   Jul26   0:01 python
/usr/sbin/xend start
root      3014  0.0  0.1  27252   904 ?        Ssl  Jul26   0:00
blktapctrl
root     13985  0.0  0.1  27332   832 ?        Sl   11:17   0:00 tapdisk
/var/run/tap/tapctrlwrite1 /var/run/tap/tap

So this looks like each disk added counts for a small megabyte of memory.
I wonder if it wouldn''t be possible to run some sort of tapdisk service
that adds a mapping, instead of a lot of different processes.

The amount of memory for xend seem to increased with 7MB (from 49MB) now
this is not even close to the 32MB or 16MB value that was mentioned
before.


Stefan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan de Konink, le Tue 29 Jul 2008 10:33:05 +0200, a écrit
:
> On Tue, 29 Jul 2008, Gerd Hoffmann wrote:
> > Stefan de Konink wrote:
> > > And you don''t count the 331MB virtual memory the process
takes, and every
> > > tapdisk that is created?
> >
> > Tapdisk doesn''t need physical memory.  It is address space
used to map
> > in guest pages.
> 
> Maybe you use a different version of Xen... I''m using 3.2.1 until
the Dell
> patches of the 2.6.19 kernel are backported, or I get the Vanilla kernel
> running on 64bits.
> 
> 
> root      3005  0.0  0.1   8404  1012 ?        S    Jul26   0:00 xenstored
--pid-file /var/run/xenstore.pid
> root      3009  0.0  1.8  99416  9936 ?        S    Jul26   0:00 python
/usr/sbin/xend start
> root      3011  0.0  0.1  20540   680 ?        Sl   Jul26   0:00
xenconsoled
> root      3013  0.0 10.7 242876 56492 ?        Sl   Jul26   0:01 python
/usr/sbin/xend start
> root      3014  0.0  0.1  27252   904 ?        Ssl  Jul26   0:00 blktapctrl
> root     13985  0.0  0.1  27332   832 ?        Sl   11:17   0:00 tapdisk
/var/run/tap/tapctrlwrite1 /var/run/tap/tap
Please provide the columns headers, else it''s impossible to say
anything about these numbers.
> The amount of memory for xend seem to increased with 7MB (from 49MB) now
> this is not even close to the 32MB or 16MB value that was mentioned
> before.
Please do not confuse with what I said about stubdom-dm, which is to
replace qemu-dm only, not xend.

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME
COMMAND
> > root      3005  0.0  0.1   8404  1012 ?        S    Jul26   0:00
xenstored --pid-file /var/run/xenstore.pid
> > root      3009  0.0  1.8  99416  9936 ?        S    Jul26   0:00
python /usr/sbin/xend start
> > root      3011  0.0  0.1  20540   680 ?        Sl   Jul26   0:00
xenconsoled
> > root      3013  0.0 10.7 242876 56492 ?        Sl   Jul26   0:01
python /usr/sbin/xend start
> > root      3014  0.0  0.1  27252   904 ?        Ssl  Jul26   0:00
blktapctrl
> > root     13985  0.0  0.1  27332   832 ?        Sl   11:17   0:00
tapdisk /var/run/tap/tapctrlwrite1 /var/run/tap/tap
>
> Please provide the columns headers, else it''s impossible to say
> anything about these numbers.
Done.
> > The amount of memory for xend seem to increased with 7MB (from 49MB)
now
> > this is not even close to the 32MB or 16MB value that was mentioned
> > before.
>
> Please do not confuse with what I said about stubdom-dm, which is to
> replace qemu-dm only, not xend.
Then it is very simple; I hope that ''Xennite'' or what ever the
number of
the beast is, hopefully has lower demands. If stubdom is only to be more
secure, please... that is the only thing I do not worry about these
days, I see far bigger bugs.

With VMWare releasing ESXi for free; I really wonder what the impact will
be on the userbase of Xen, especially the ''big
implementations'', that like
small solutions.


Stefan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel