Xen users - Dec 2007 - paravirt_ops DomU, does it mean performance?

Hi,

I was wondering if using a paravirt_ops enabled kernel (say 2.6.23) as a
Paravirtual DomU would mean higher performance out of the box, or if we
still need to compile some xen specific device drivers for network and block
devices.

Similarly, if HVM domains can run at the same performance as PV domains just
by using PV drivers for network and block, why are we bothering ourselves
with PV?  Why don''t we -always- use HVM DomU''s with PV
drivers?

Thanks for views on this,

Br,

Emre Erenoglu
erenoglu@gmail.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> I was wondering if using a paravirt_ops enabled kernel (say 2.6.23) as a
> Paravirtual DomU would mean higher performance out of the box, or if we
> still need to compile some xen specific device drivers for network and
> block devices.
The paravirt_ops kernel should run with similar performance to the XenLinux 
kernel from the main Xen tree.  It includes paravirtualisation for memory, 
SMP, etc as well as paravirtualised network and block devices.  It
doesn''t
include support for suspend/resume, paravirt framebuffer, etc.
> Similarly, if HVM domains can run at the same performance as PV domains
> just by using PV drivers for network and block, why are we bothering
> ourselves with PV?  Why don''t we -always- use HVM DomU''s
with PV drivers?
I suspect the performance of PV domains ought still to be a bit better than 
for HVM - at least for some workloads.  PV eliminates the need for shadow 
pagetables and reduces the number of things that need to be trapped and 
emulated.  PV can also run without hardware support, which is useful for some 
people.

However, HVM solutions from Intel and AMD are going to keep getting more 
features and more optimisation.  Eventually they''re going to do things
so
fast, and offer so many capabilities that we''re probably going to want
to use
HVM everywhere - at least some parts of it.  There''s no reason we
can''t
combine HVM and PV techniques though...  in principle we could get a PV guest 
to run in an HVM so that we could use some combination of PV and HVM 
features.  Folks from Intel have been looking at something they call
"hybrid
virtualisation" which combines techniques from both PV and HVM.

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Mark Williamson wrote:
>> I was wondering if using a paravirt_ops enabled kernel (say 2.6.23) as
a
>> Paravirtual DomU would mean higher performance out of the box, or if we
>> still need to compile some xen specific device drivers for network and
>> block devices.
>>     
>
> The paravirt_ops kernel should run with similar performance to the XenLinux
> kernel from the main Xen tree.  It includes paravirtualisation for memory, 
> SMP, etc as well as paravirtualised network and block devices.  It
doesn''t
> include support for suspend/resume, paravirt framebuffer, etc.
>
>   
>> Similarly, if HVM domains can run at the same performance as PV domains
>> just by using PV drivers for network and block, why are we bothering
>> ourselves with PV?  Why don''t we -always- use HVM
DomU''s with PV drivers?
>>     
>
> I suspect the performance of PV domains ought still to be a bit better than
> for HVM - at least for some workloads.  PV eliminates the need for shadow 
> pagetables and reduces the number of things that need to be trapped and 
> emulated. 
Is it still true to say that HVM uses shadow pagetables in the light of 
AMD nested pagetables?  Or is my understanding wrong?

--Sadique
>  PV can also run without hardware support, which is useful for some 
> people.
>
> However, HVM solutions from Intel and AMD are going to keep getting more 
> features and more optimisation.  Eventually they''re going to do
things so
> fast, and offer so many capabilities that we''re probably going to
want to use
> HVM everywhere - at least some parts of it.  There''s no reason we
can''t
> combine HVM and PV techniques though...  in principle we could get a PV
guest
> to run in an HVM so that we could use some combination of PV and HVM 
> features.  Folks from Intel have been looking at something they call
"hybrid
> virtualisation" which combines techniques from both PV and HVM.
>
> Cheers,
> Mark
>
>   

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> > I suspect the performance of PV domains ought still to be a bit better
> > than for HVM - at least for some workloads.  PV eliminates the need
for
> > shadow pagetables and reduces the number of things that need to be
> > trapped and emulated.
>
> Is it still true to say that HVM uses shadow pagetables in the light of
> AMD nested pagetables?  Or is my understanding wrong?
It''s true that HVM doesn''t always have to be shadow-pagetable
based.  I''m not
clear whether AMD NPT-capable hardware is on sale yet (it was in the original 
SVM spec but not shipped in the first SVM processors).  Intel have brought 
out hardware assisted paging specs also, so we should soon not require shadow 
PTs on Intel hardware either.  Most HVM hardware currently out there has to 
use shadow PTs, though.

This is an example of something we might want to use for PV guests in Xen, if 
it gets fast enough, in order to avoid trapping into Xen for pagetable 
updates.  This should be doable, even for older guests, since as long as they 
support the "hypercall transfer page" then they can potentially have
their
hypercall implementations redefined when they are loaded.

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue December 11 2007 1:47:47 pm Mark Williamson
wrote:
> The paravirt_ops kernel should run with similar performance to the XenLinux
> kernel from the main Xen tree.  It includes paravirtualisation for memory,
> SMP, etc as well as paravirtualised network and block devices.  It
doesn''t
> include support for suspend/resume, paravirt framebuffer, etc.
When you say ''paravirt framebuffer'', do you mean console
performance is going
to be as bad as for HVM?

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> > The paravirt_ops kernel should run with similar performance to the
> > XenLinux kernel from the main Xen tree.  It includes
paravirtualisation
> > for memory, SMP, etc as well as paravirtualised network and block
> > devices.  It doesn''t include support for suspend/resume,
paravirt
> > framebuffer, etc.
>
> When you say ''paravirt framebuffer'', do you mean console
performance is
> going to be as bad as for HVM?
Well, more that it doesn''t have a graphical console at all!  Paravirt
guests
can''t use the emulated framebuffer that HVM uses, they have to use a
paravirt
framebuffer if they want to output graphically.  The paravirt framebuffer 
doesn''t perform significantly better than the HVM console as far as I
know,
so not having it available may not be too painful to you...

You can still take the usual approach of using a network-based remote desktop 
approach, which is likely to give you better performance than either the HVM 
framebuffer or the paravirt framebuffer!

Using VNC or X11 over SSH is probably the obvious remote desktop solution, but 
I think that nomachine X (or freenx, if you can make it work) might be worth 
a look as well.  Nomachine have a "free" product but I''m not
sure what the
licensing restrictions are.

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi All,

I''m again a bit confused. I''m trying to boot up a 2.6.24
kernel on a HVM
DomU, and the kernel tells me "booting paravirtualized kernel on bare
hardware", but I can see no paravirtual devices in /dev.

Which devices shall it be? What are the names of the modules? Is there a
clear name for them or does each distro choose a name for itself?

I found names starting with virtio_* but I''m not sure if they are the
correct ones, at least in this 2.6.24 (from development version of Pardus),
modprobbing them gives no output in dmesg nor any devices in /dev

Or maybe we need to put some special configutation options in the config
file of the HVM domU?

Thanks,

Emre

On Wed, Dec 26, 2007 at 4:43 PM, Mark Williamson <
mark.williamson@cl.cam.ac.uk> wrote:
> > > The paravirt_ops kernel should run with similar performance to
the
> > > XenLinux kernel from the main Xen tree.  It includes
paravirtualisation
> > > for memory, SMP, etc as well as paravirtualised network and block
> > > devices.  It doesn''t include support for suspend/resume,
paravirt
> > > framebuffer, etc.
> >
> > When you say ''paravirt framebuffer'', do you mean
console performance is
> > going to be as bad as for HVM?
>
> Well, more that it doesn''t have a graphical console at all! 
Paravirt
> guests
> can''t use the emulated framebuffer that HVM uses, they have to use
a
> paravirt
> framebuffer if they want to output graphically.  The paravirt framebuffer
> doesn''t perform significantly better than the HVM console as far
as I know,
> so not having it available may not be too painful to you...
>
> You can still take the usual approach of using a network-based remote
> desktop
> approach, which is likely to give you better performance than either the
> HVM
> framebuffer or the paravirt framebuffer!
>
> Using VNC or X11 over SSH is probably the obvious remote desktop solution,
> but
> I think that nomachine X (or freenx, if you can make it work) might be
> worth
> a look as well.  Nomachine have a "free" product but I''m
not sure what the
> licensing restrictions are.
>
> Cheers,
> Mark
>
> --
> Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
> Mark: To answer a question with a question: What use is a skateboard?
> Dave: Skateboards have wheels.
> Mark: My wheel has a wheel!
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>


-- 
Emre Erenoglu
erenoglu@gmail.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Sunday May 25 2008 09:28:21 pm Emre ERENOGLU wrote:
> I''m again a bit confused. I''m trying to boot up a 2.6.24
kernel on a HVM
> DomU, and the kernel tells me "booting paravirtualized kernel on bare
> hardware", but I can see no paravirtual devices in /dev.
>
> Which devices shall it be? What are the names of the modules? Is there a
> clear name for them or does each distro choose a name for itself?
>
> I found names starting with virtio_* but I''m not sure if they are
the
> correct ones, at least in this 2.6.24 (from development version of Pardus),
> modprobbing them gives no output in dmesg nor any devices in /dev
>
> Or maybe we need to put some special configutation options in the config
> file of the HVM domU?
Hvm won''t get performance w/o something like the unmodified drivers you
installed awhile back. Hvm domus *think* they are ''booting [...] on
bare
hardware'', and are not aware that they are running under a hypervisor.
The pv
drivers do that. Pvops is just alternative approach to standard xen code to 
running under a hypervisor as a pv domu. It doesn''t make an hvm domain 
xen-aware.
> On Wed, Dec 26, 2007 at 4:43 PM, Mark Williamson 
<mark.williamson@cl.cam.ac.uk> wrote:
> > > The paravirt_ops kernel should run with similar performance to
the
> > >
>  > > XenLinux kernel from the main Xen tree.  It includes
>  > > paravirtualisation for memory, SMP, etc as well as
paravirtualised
>  > > network and block devices.  It doesn''t include support
for
>  > > suspend/resume, paravirt framebuffer, etc.
>  >
>  > When you say ''paravirt framebuffer'', do you mean
console performance is
>  > going to be as bad as for HVM?
>
> Well, more that it doesn''t have a graphical console at all!
 Paravirt
> guests can''t use the emulated framebuffer that HVM uses, they have
to use a
> paravirt framebuffer if they want to output graphically.  The paravirt
> framebuffer doesn''t perform significantly better than the HVM
console as
> far as I know, so not having it available may not be too painful to you...
I''ve mentioned before that what Fedora 9 (2.6.25 kernel) does is use
the
standard Xorg fbdev module in a pv domain, but it uses ''qemu-dm -M
xenpv ...''
to run the domain. (Actually, Fedora has been doing that since F8, fully 
patched, not dvd version.) Even their non-xen 2.6.25 kernel won''t boot
in a
pv domu. It doesn''t enable the xen frontends in the kernel config, but
even
when I''ve tried to do that, the kernel won''t boot. (See the
thread ''Creating
an fc8 2.6.23 domu'' in late Feb., early Mar.) Apparently there is still
going
to be a lot of work to merge xen into mainline.

Don''t know what other distros are doing w/ pvops kernels as far as
frame
buffers go.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Thanks Jim,

On Mon, May 26, 2008 at 4:20 AM, jim burns <jim_burn@bellsouth.net> wrote:
> On Sunday May 25 2008 09:28:21 pm Emre ERENOGLU wrote:
> > I''m again a bit confused. I''m trying to boot up a
2.6.24 kernel on a HVM
> > DomU, and the kernel tells me "booting paravirtualized kernel on
bare
> > hardware", but I can see no paravirtual devices in /dev.
> >
> > Which devices shall it be? What are the names of the modules? Is there
a
> > clear name for them or does each distro choose a name for itself?
> >
> > I found names starting with virtio_* but I''m not sure if they
are the
> > correct ones, at least in this 2.6.24 (from development version of
> Pardus),
> > modprobbing them gives no output in dmesg nor any devices in /dev
> >
> > Or maybe we need to put some special configutation options in the
config
> > file of the HVM domU?
>
> Hvm won''t get performance w/o something like the unmodified
drivers you
> installed awhile back. Hvm domus *think* they are ''booting [...]
on bare
> hardware'', and are not aware that they are running under a
hypervisor. The
> pv
> drivers do that. Pvops is just alternative approach to standard xen code to
> running under a hypervisor as a pv domu. It doesn''t make an hvm
domain
> xen-aware.
>
Got it.  I can load the PV drivers up, but I was too optimistic to think
that the pv_ops would automatically bring some PV drivers in place and
especially after seeing that kernel message in dmesg. But dissapointed now
with your explanation.

> > On Wed, Dec 26, 2007 at 4:43 PM, Mark Williamson
> <mark.williamson@cl.cam.ac.uk> wrote:
> > > > The paravirt_ops kernel should run with similar performance
to the
> > > >
> >  > > XenLinux kernel from the main Xen tree.  It includes
> >  > > paravirtualisation for memory, SMP, etc as well as
paravirtualised
> >  > > network and block devices.  It doesn''t include
support for
> >  > > suspend/resume, paravirt framebuffer, etc.
> >  >
> >  > When you say ''paravirt framebuffer'', do you
mean console performance
> is
> >  > going to be as bad as for HVM?
> >
> > Well, more that it doesn''t have a graphical console at all! 
Paravirt
> > guests can''t use the emulated framebuffer that HVM uses, they
have to use
> a
> > paravirt framebuffer if they want to output graphically.  The paravirt
> > framebuffer doesn''t perform significantly better than the HVM
console as
> > far as I know, so not having it available may not be too painful to
> you...
>
> I''ve mentioned before that what Fedora 9 (2.6.25 kernel) does is
use the
> standard Xorg fbdev module in a pv domain, but it uses ''qemu-dm -M
xenpv
> ...''
> to run the domain. (Actually, Fedora has been doing that since F8, fully
> patched, not dvd version.) Even their non-xen 2.6.25 kernel won''t
boot in a
> pv domu. It doesn''t enable the xen frontends in the kernel config,
but even
> when I''ve tried to do that, the kernel won''t boot. (See
the thread
> ''Creating
> an fc8 2.6.23 domu'' in late Feb., early Mar.) Apparently there is
still
> going
> to be a lot of work to merge xen into mainline.

I don''t understand. Mark says that paravirt_ops kernel shall run at
near
performance to XenLinux kernel, but you say that it''s not possible to
boot
such kernel. My experience is the same, I couldn''t boot this 2.6.24 as
a PV
domU. So where do we use this pv_ops?

My understanding was, given the necessary KERNEL_OPTIONS are enabled, the
kernel shall boot fine as a Xen PV DomU, thus it would have been enough to
write this inside the domu config file file kernel= line???

Emre Erenoglu


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Monday May 26 2008 05:02:23 am you wrote:
> Apparently there is still going
>  to be a lot of work to merge xen into mainline.
>
> I don''t understand. Mark says that paravirt_ops kernel shall run
at near
> performance to XenLinux kernel, but you say that it''s not possible
to boot
> such kernel. My experience is the same, I couldn''t boot this
2.6.24 as a PV
> domU. So where do we use this pv_ops?
>
> My understanding was, given the necessary KERNEL_OPTIONS are enabled, the
> kernel shall boot fine as a Xen PV DomU, thus it would have been enough to
> write this inside the domu config file file kernel= line???
It''s not possible *yet*, at least for Fedora. I''ve seen some
people on this
list say they were running 2.6.24 as a pv domu, from some other distro. 
Either that distro is still forward porting the standard xen patches, or they 
got pvops to work for them (and them alone). Fedora seems more inclined to a 
slow but correct approach - that is acceptable to the mainline kernel folks, 
so that their code will eventually be merged.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users