I compiled a pv_ops dom0 kernel from Jeremy''s latest git tree and upgraded Xen to 3.4.1 following one of Boris'' blog entries. From the Xen perspective everything seems to work as expected. But, I want to be able to run both Xen and KVM existing guests in a single cloud on a farm of VT-enabled machines. So after booting into the new pv_ops dom0 kernel as a test I tried starting one of my KVM guests. It ran extremely slow so I knew immediately that it wasnt'' using the VT acceleration. I then looked at /proc/cpuinfo and saw that the VT cpu flag (svm on this machine) was not there. The machine has a VT processor and hardware virtualization is enabled in the BIOS and I rebooted to check with a regular kernel and it of course the guest runs with VT acceleration there. My question is how can I get pv_ops dom0 kernel or Xen 3.4.1 to pass the cpu VT flags through so that existing KVM guests will run with cpu VT acceleration at full speed? -Gerry _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fajar A. Nugraha
2009-Dec-29 03:31 UTC
Re: [Xen-users] pv_ops dom0 kernel and VT cpu extensions
On Tue, Dec 29, 2009 at 8:11 AM, Gerry Reno <greno@verizon.net> wrote:> My question is how can I > get pv_ops dom0 kernel or Xen 3.4.1 to pass the cpu VT flags through so that > existing KVM guests will run with cpu VT acceleration at full speed?AFAIK you can''t. The same reason why you can''t get Virtualbox and KVM to use VT together. Only one virtualization technology can use VT at the same time. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fajar A. Nugraha wrote:> On Tue, Dec 29, 2009 at 8:11 AM, Gerry Reno <greno@verizon.net> wrote: > >> My question is how can I >> get pv_ops dom0 kernel or Xen 3.4.1 to pass the cpu VT flags through so that >> existing KVM guests will run with cpu VT acceleration at full speed? >> > > AFAIK you can''t. The same reason why you can''t get Virtualbox and KVM > to use VT together. > Only one virtualization technology can use VT at the same time. > >As physical boxes gain more and more processing capability it makes no sense to restrict a physical machine to only a single hypervisor. Libvirt will support nested VM''s that pass through the VT capabilities to the next level. So it makes sense that Xen should be able to do the same thing and pass through the cpu VT capabilities to other hypervisors. Is there some law of the universe that prevents this? -Gerry _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Following that logic we will need to start working on a vmmm (virtual machine monitor monitor) to handle multiple vmm''s :) Skickat från min iPhone Dec 28, 2009 kl. 11:01 PM skrev Gerry Reno <greno@verizon.net>:> Fajar A. Nugraha wrote: >> On Tue, Dec 29, 2009 at 8:11 AM, Gerry Reno <greno@verizon.net> >> wrote: >> >>> My question is how can I >>> get pv_ops dom0 kernel or Xen 3.4.1 to pass the cpu VT flags >>> through so that >>> existing KVM guests will run with cpu VT acceleration at full speed? >>> >> >> AFAIK you can''t. The same reason why you can''t get Virtualbox and KVM >> to use VT together. >> Only one virtualization technology can use VT at the same time. >> >> > As physical boxes gain more and more processing capability it makes > no sense to restrict a physical machine to only a single hypervisor. > > Libvirt will support nested VM''s that pass through the VT > capabilities to the next level. So it makes sense that Xen should > be able to do the same thing and pass through the cpu VT > capabilities to other hypervisors. Is there some law of the > universe that prevents this? > > -Gerry > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fajar A. Nugraha
2009-Dec-29 04:24 UTC
Re: [Xen-users] pv_ops dom0 kernel and VT cpu extensions
On Tue, Dec 29, 2009 at 11:01 AM, Gerry Reno <greno@verizon.net> wrote:> As physical boxes gain more and more processing capability it makes no sense > to restrict a physical machine to only a single hypervisor. > > Libvirt will support nested VM''s that pass through the VT capabilities to > the next level. So it makes sense that Xen should be able to do the same > thing and pass through the cpu VT capabilities to other hypervisors. Is > there some law of the universe that prevents this?While KVM could probably do it, Xen currently can''t. You might be able to get more answers about "why" and possible time line to support it (if any available) from xen-devel. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Chris wrote:> Following that logic we will need to start working on a vmmm (virtual > machine monitor monitor) to handle multiple vmm''s :) > > Skickat från min iPhone > > Dec 28, 2009 kl. 11:01 PM skrev Gerry Reno <greno@verizon.net>: > >> Fajar A. Nugraha wrote: >>> On Tue, Dec 29, 2009 at 8:11 AM, Gerry Reno <greno@verizon.net> wrote: >>> >>>> My question is how can I >>>> get pv_ops dom0 kernel or Xen 3.4.1 to pass the cpu VT flags >>>> through so that >>>> existing KVM guests will run with cpu VT acceleration at full speed? >>>> >>> >>> AFAIK you can''t. The same reason why you can''t get Virtualbox and KVM >>> to use VT together. >>> Only one virtualization technology can use VT at the same time. >>> >>> >> As physical boxes gain more and more processing capability it makes >> no sense to restrict a physical machine to only a single hypervisor. >> >> Libvirt will support nested VM''s that pass through the VT >> capabilities to the next level. So it makes sense that Xen should be >> able to do the same thing and pass through the cpu VT capabilities to >> other hypervisors. Is there some law of the universe that prevents >> this? >> >> -Gerry >>Chris, please.... don''t top post on the list! What you say about monitor monitor is not quite what is needed but somewhat along those lines. We need some type of small hypervisor-monitor / scheduler-kernel that would exist in Ring 0 and mediate between hypervisors. Then all hypervisors/kernels could be in Ring 1. Domains in Ring 2. All apps in Ring 3. Something like this. -Gerry _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fajar A. Nugraha wrote:> On Tue, Dec 29, 2009 at 11:01 AM, Gerry Reno <greno@verizon.net> wrote: > >> As physical boxes gain more and more processing capability it makes no sense >> to restrict a physical machine to only a single hypervisor. >> >> Libvirt will support nested VM''s that pass through the VT capabilities to >> the next level. So it makes sense that Xen should be able to do the same >> thing and pass through the cpu VT capabilities to other hypervisors. Is >> there some law of the universe that prevents this? >> > > While KVM could probably do it, Xen currently can''t. You might be able > to get more answers about "why" and possible time line to support it > (if any available) from xen-devel. >I''ll bring the thread over to xen-devel. -Gerry _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen
2009-Dec-29 16:09 UTC
Re: [Xen-users] pv_ops dom0 kernel and VT cpu extensions
On Tue, Dec 29, 2009 at 11:24:17AM +0700, Fajar A. Nugraha wrote:> On Tue, Dec 29, 2009 at 11:01 AM, Gerry Reno <greno@verizon.net> wrote: > > As physical boxes gain more and more processing capability it makes no sense > > to restrict a physical machine to only a single hypervisor. > > > > Libvirt will support nested VM''s that pass through the VT capabilities to > > the next level. So it makes sense that Xen should be able to do the same > > thing and pass through the cpu VT capabilities to other hypervisors. Is > > there some law of the universe that prevents this? > > While KVM could probably do it, Xen currently can''t. You might be able > to get more answers about "why" and possible time line to support it > (if any available) from xen-devel. >There was discussion about ''nested virtualization'' with Xen at the latest Xen Summit iirc. That still doesn''t mean dom0 should have VT visible to it. in Xen world dom0 is just a PV (paravirtualized) virtual machine guest, so you can''t run KVM in dom0. If you want to run KVM under Xen HVM guest, then that should be possible when ''nested virtualization'' is supported in Xen. -- Pasi _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen wrote:> On Tue, Dec 29, 2009 at 11:24:17AM +0700, Fajar A. Nugraha wrote: > >> On Tue, Dec 29, 2009 at 11:01 AM, Gerry Reno <greno@verizon.net> wrote: >> >>> As physical boxes gain more and more processing capability it makes no sense >>> to restrict a physical machine to only a single hypervisor. >>> >>> Libvirt will support nested VM''s that pass through the VT capabilities to >>> the next level. So it makes sense that Xen should be able to do the same >>> thing and pass through the cpu VT capabilities to other hypervisors. Is >>> there some law of the universe that prevents this? >>> >> While KVM could probably do it, Xen currently can''t. You might be able >> to get more answers about "why" and possible time line to support it >> (if any available) from xen-devel. >> >> > > There was discussion about ''nested virtualization'' with Xen at the > latest Xen Summit iirc. > > That still doesn''t mean dom0 should have VT visible to it. in Xen world > dom0 is just a PV (paravirtualized) virtual machine guest, > so you can''t run KVM in dom0. > > If you want to run KVM under Xen HVM guest, then that should be > possible when ''nested virtualization'' is supported in Xen. > > -- Pasi >Thanks Pasi, that''s interesting and I''ll try it when it is available. What would be best though is if Xen just passed the VT flags so that you can run KVM in dom0. As hardware continues to grow more powerful it won''t be long before we see as commonplace 64-core cpu, mult-terabytes of memory. This will be far more than any one hypervisor/kernel/os needs so we will need a way to boot/run multiple hypervisors on bare-metal. And that is why some type of agnostic hypervisor-mediator-scheduler needs to run in Ring 0 and then all the hypervisors/kernels in Ring 1. -Gerry _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen
2009-Dec-29 18:16 UTC
Re: [Xen-users] pv_ops dom0 kernel and VT cpu extensions
On Tue, Dec 29, 2009 at 11:29:27AM -0500, Gerry Reno wrote:> Pasi Kärkkäinen wrote: > >On Tue, Dec 29, 2009 at 11:24:17AM +0700, Fajar A. Nugraha wrote: > > > >>On Tue, Dec 29, 2009 at 11:01 AM, Gerry Reno <greno@verizon.net> wrote: > >> > >>>As physical boxes gain more and more processing capability it makes no > >>>sense > >>>to restrict a physical machine to only a single hypervisor. > >>> > >>>Libvirt will support nested VM''s that pass through the VT capabilities to > >>>the next level. So it makes sense that Xen should be able to do the same > >>>thing and pass through the cpu VT capabilities to other hypervisors. Is > >>>there some law of the universe that prevents this? > >>> > >>While KVM could probably do it, Xen currently can''t. You might be able > >>to get more answers about "why" and possible time line to support it > >>(if any available) from xen-devel. > >> > >> > > > >There was discussion about ''nested virtualization'' with Xen at the > >latest Xen Summit iirc. > > > >That still doesn''t mean dom0 should have VT visible to it. in Xen world > >dom0 is just a PV (paravirtualized) virtual machine guest, > >so you can''t run KVM in dom0. > > > >If you want to run KVM under Xen HVM guest, then that should be > >possible when ''nested virtualization'' is supported in Xen. > > > >-- Pasi > > > Thanks Pasi, that''s interesting and I''ll try it when it is available. > What would be best though is if Xen just passed the VT flags so that you > can run KVM in dom0. >There was also talks about converting dom0 to HVM guest instead of PV guest; that might make it possible to run KVM in dom0. Nested virtualization with Xen slides: http://www.xen.org/files/xensummit_intel09/xensummit-nested-virt.pdf HVM dom0 slides: http://www.xen.org/files/xensummit_intel09/nakajima.pdf -- Pasi> As hardware continues to grow more powerful it won''t be long before we > see as commonplace 64-core cpu, mult-terabytes of memory. This will be > far more than any one hypervisor/kernel/os needs so we will need a way > to boot/run multiple hypervisors on bare-metal. And that is why some > type of agnostic hypervisor-mediator-scheduler needs to run in Ring 0 > and then all the hypervisors/kernels in Ring 1. > > -Gerry >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users