Xen users - Feb 2007 - Consolidating multiple machines. Feasible?

Hello.

I''m new to xen so please point out incorrect terminology or any faulty 
assumptions I seem to be making.

I''m wondering if it is feasible to consolidate various separate
machines onto
a single piece of hardware.  

The desired outcome would be to have several ''stations''
(consisting of
monitor, sound, kbd & mouse and ideally some IO devices eg optical drives, 
card readers, etc.) all virtualised on the one machine.

Additionally there''d be a requirement for utility machine(s) eg
firewall, myth
backend, mail server, whatever.  Hopefully these headless/inputless linux 
machines can be virtualized in a straight forward manner.

There are multiple usage scenarios for the stations.  These include a linux 
development workstation, a windows workstation, linux based HTPC, windows 
based gaming and maybe linux based gaming.

Ideally (pretty much) any usage would be possible on any station, and stations 
could borrow hardware from unused stations.  I''m mainly thinking
graphics
cards here, if station1 isn''t using it''s card you could start
up a SLI
windows machine on station2.

I''d likely set up a AMD Socket F based system so would have hardware 
virtualisation and IOMMU.


Some questions that have occurred to me:

Given hardware based virtualisation, are devices (eg graphics, sound, tv 
cards...) in domUs going to work just like it was the native OS?

Can individual USB devices be reserved for use by a domU?

Can domUs be nested?  E.g. run a windows machine in the domU linux workstation 
(in a window).

Can hardware be relinquished by a domU?  Perhaps through some sort of tricky 
virtual machine migration?

What happens if you try to give reserved hardware to a domU when another domU 
already has it?

It seems desirable to have as little as possible running on dom0.  Are there 
management tools which handle headless dom0 and the need to shutdown a 
certain domU to free up hardware so the desired domU can be started up?

I think I read somewhere that using IOMMU requires the entirety of a domU to 
be locked in memory because devices can''t retry writes to main memory. 
Is
that correct?


I''m sure there''s other things I would need to consider, so
don''t restrict
yourself to what I actually ask! ;)

Cheers
Daniel

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wednesday 14 February 2007, Daniel McAllansmith
wrote:
> Given hardware based virtualisation, are devices (eg graphics, sound, tv
> cards...) in domUs going to work just like it was the native OS?
no.  ''hardware based virtualisation'' only means that Intel and
AMD added some
tricks to their processors to overcome some limitations that used to make CPU 
virtualisation much harder than it is on almost any other CPU.

only CPU and RAM are virtualised; the rest of the hardware (net, block device, 
screen, keyboard, PCI, USB) is all emulated, using code borrowed from qemu

you can select PCI devices to be accessed by a domU instead of by dom0; but 
it''s very experimental on HVM domUs (works on PV domUs).  even with
that,
screen cards are usually the thoughest to reassign (IOW: no)
> Can individual USB devices be reserved for use by a domU?
there''s some experimental code for that, but AFAIK, not ready yet.
> Can domUs be nested?  E.g. run a windows machine in the domU linux
> workstation (in a window).
not on pure Xen; but you could run Qemu on a domU.  no accelerator, 
unfortunately (that means the slowest variety of qemu)
> Can hardware be relinquished by a domU?  Perhaps through some sort of
> tricky virtual machine migration?
no.  PCI initialisation is very complex; you''d need a guest OS that
knows how
to do that and a lot of help from the hypervisor.  much easier is to use it 
from dom0 and share it like on a network.
> What happens if you try to give reserved hardware to a domU when another
> domU already has it?
you can''t.  the hypervisor won''t allow. (and the userland
tools check for
conflicts before starting the domU)
> It seems desirable to have as little as possible running on dom0.  Are
> there management tools which handle headless dom0 and the need to shutdown
> a certain domU to free up hardware so the desired domU can be started up?
not that i''ve heard.  anyway, the hardware in question would have to be
hard-reset to return it to power-up state, so that the new guest OS would 
find it as expected.  i don''t know if that''s possible without
interfering
with the whole PCI bus.


-- 
Javier


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hello, Javier.

Thanks for the response.

On Friday 16 February 2007 02:05, Javier Guerra wrote:
> On Wednesday 14 February 2007, Daniel McAllansmith wrote:
> > Given hardware based virtualisation, are devices (eg graphics, sound,
tv
> > cards...) in domUs going to work just like it was the native OS?
>
> no.  ''hardware based virtualisation'' only means that
Intel and AMD added
> some tricks to their processors to overcome some limitations that used to
> make CPU virtualisation much harder than it is on almost any other CPU.
>
> only CPU and RAM are virtualised; the rest of the hardware (net, block
> device, screen, keyboard, PCI, USB) is all emulated, using code borrowed
> from qemu
>
> you can select PCI devices to be accessed by a domU instead of by dom0; but
> it''s very experimental on HVM domUs (works on PV domUs).  even
with that,
> screen cards are usually the thoughest to reassign (IOW: no)
As far as I understand things AMDs Pacifica virtualisation added an IOMMU 
which can translate DMA transfers for HyperTransport or PCI devices.

I thought that use of the IOMMU would allow unmodified drivers in a unmodified 
domU OS access to the device (where the device has been hidden from domO).
This should allow 3D acceleration in a domU windows OS, just as can be 
achieved now using a paravirtualisation-aware driver in a domU linux.

Am I misunderstanding the purpose of the IOMMU, or does Xen not support it 
yet?

>
> > Can individual USB devices be reserved for use by a domU?
>
> there''s some experimental code for that, but AFAIK, not ready yet.
From further reading I see that xen 2 supported passthrough of usb devices but 
this has bitrotted.

I see some attempts to allocate an entire separate physical USB controller to 
different domO/domU(s) but that has been unstable.  Is this a known problem?
>
> > Can domUs be nested?  E.g. run a windows machine in the domU linux
> > workstation (in a window).
>
> not on pure Xen; but you could run Qemu on a domU.  no accelerator,
> unfortunately (that means the slowest variety of qemu)
So this is just the same setup on domU as running Qemu on a non-xen machine?
>
> > Can hardware be relinquished by a domU?  Perhaps through some sort of
> > tricky virtual machine migration?
>
> no.  PCI initialisation is very complex; you''d need a guest OS
that knows
> how to do that and a lot of help from the hypervisor.  much easier is to
> use it from dom0 and share it like on a network.
>
[snip]
>
> > It seems desirable to have as little as possible running on dom0.  Are
> > there management tools which handle headless dom0 and the need to
> > shutdown a certain domU to free up hardware so the desired domU can be
> > started up?
>
> not that i''ve heard.  anyway, the hardware in question would have
to be
> hard-reset to return it to power-up state, so that the new guest OS would
> find it as expected.  i don''t know if that''s possible
without interfering
> with the whole PCI bus.
Would hot-pluggable devices be a solution to the initialisation and reset 
problems?

It sounds as if a domU shutting down does not reset the state of any PCI 
devices it was using, so those devices are unusable until the whole machine 
is restarted.  Is that correct?


Is it necessary for the memory allocated to domUs to be locked in?  Does it 
make a difference if para- or hardware- virtualisation is used?


Thanks
Daniel

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users