Xen users - Jun 2006 - Can we run more virtual machine than the memory avaible?

Hi,

I''d like to create a virtual network with xen who have more tha 20  
hosts. Every host has 30MB in his configuration file. The total of  
memory is

20X30 = 600MB

+

64MB for the dom0
--------------------------
= 664MB

but I only have 512MB on my workstation. Can this virtual network  
possible with xen? Or you need to have enough ram!?

Thank you

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> -----Original Message-----
> From: xen-users-bounces@lists.xensource.com 
> [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of 
> Rachid Abdoun
> Sent: 22 June 2006 14:06
> To: Xen-users@lists.xensource.com
> Cc: EHRENSBERGER Juergen
> Subject: [Xen-users] Can we run more virtual machine than the 
> memory avaible?
> 
> Hi,
> 
> I''d like to create a virtual network with xen who have more tha 20
> hosts. Every host has 30MB in his configuration file. The total of  
> memory is
> 
> 20X30 = 600MB
> 
> +
> 
> 64MB for the dom0
> --------------------------
> = 664MB
> 
> but I only have 512MB on my workstation. Can this virtual network  
> possible with xen? Or you need to have enough ram!?
You need enough RAM. 

What you COULD do, assuming the GUEST OS supports it, is to reduce the
amount of RAM you give each guest, and let the guest swap to disk. 

The reason you can''t overcommit memory from Xen''s side of
things is that
the Xen hypervisor doesn''t actually know how (or what) to swap to/from
disk. This is, primarily, because we don''t actually want Xen to do this
(since it wouldn''t actually know which parts of memory the OS is using
for what...), but also because the hypervisor doesn''t have a clue what
a
hard-disk is - there''s no drivers for any storage-device in Xen. 

--
Mats
> 
> Thank you
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 
> 


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

It''s a big problem for me if you create a virtual network with 100  
hosts (30MB per host) you need 3 Go per host!!  I think this is the  
disadvantage  of xen


Le 22 juin 06 à 15:22, Harald Schioeberg a écrit :
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Thats not possible, you''ll need at least 664MB + 32 MB for the
> hypervisor + some overhead, xen won''t use exactly 30MB but a
little
> bit
> more.
>
> one thing: if you want to use the thing to simmulate some network
> behaviour: xen currently supports only 3 virtual interfaces per domU,
> that cripples my virtual network as well.
>
> harald
>
>
> Rachid Abdoun wrote:
>> Hi,
>>
>> I''d like to create a virtual network with xen who have more
tha 20
>> hosts. Every host has 30MB in his configuration file. The total of
>> memory is
>>
>> 20X30 = 600MB
>>
>> +
>>
>> 64MB for the dom0
>> --------------------------
>> = 664MB
>>
>> but I only have 512MB on my workstation. Can this virtual network
>> possible with xen? Or you need to have enough ram!?
>>
>> Thank you
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2.1 (GNU/Linux)
>
> iD8DBQFEmpmOJgyxs71kcx4RAmwEAKCAXO8kKBxdhZrNP5TWhP6QfbQCygCfcewF
> GbnXcFYWxm/QqFA/++pCTm4> =Qm6I
> -----END PGP SIGNATURE-----

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> -----Original Message-----
> From: xen-users-bounces@lists.xensource.com 
> [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of 
> Rachid Abdoun
> Sent: 22 June 2006 14:57
> To: Harald Schioeberg
> Cc: Xen-users@lists.xensource.com
> Subject: Re: [Xen-users] Can we run more virtual machine than 
> the memory avaible?
> 
> It''s a big problem for me if you create a virtual network with 100
> hosts (30MB per host) you need 3 Go per host!!  I think this is the  
> disadvantage  of xen
It may be a disadvantage, but it happens to be the design choosen for Xen. Xen
has limitations, which may cause it to be a disadvantage in some respects.

But I''m sure having 100 machines with 30MB of memory each would be even
more of a disadvantage, right?

Also, if you''re trying to run 100 virtual machines, I''m sure
you don''t REALLY want to do that on a single CPU, so we''re not
talking about a very basic machine...

I''m actually not aware of any virtual machine system where you are
allowed to overcommit memory... But I haven''t really been looking for
them either...

Can you suggest how this should be solved? 

There are LOTS of highly complicated situations to solve. 

We could obviously use Dom0 for the swapping process (at least in the sense of
using it''s disk drivers), so that''s not a big issue.

However, which pages can be swapped and which can''t isn''t so
easy to determine unless you are aware of what the OS knows about the memory -
for example, if the hypervisor decides that the page we''ve just given
to the network card should be swapped out and replaced with some other page, it
wouldn''t work very well - receiving a packet would destroy the new
contents of the page, sending a packet would send incorrect data. The hypervisor
has NO KNOWLEDGE of what''s in which page, so it''s not able to
choose a page correctly. Of course, the OS has some special call that is called
when the system is to "lock" a page in memory, and in a
para-virtualized environment, this could be intercepted and used to prevent the
pages from being paged out. But in the case of a unmodified OS, this would
definitely not work, as there''s not a single special operation to track
this information - the OS would just have some internal table of some sort
(binary tree, hash-list, linear array, bitmap or whatever the OS designer
decided was the best way to track a bunch of memory pages...)

I don''t think this will be implemented in Xen for a LONG time. 

--
Mats
> 
> 
> Le 22 juin 06 à 15:22, Harald Schioeberg a écrit :
> 
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Thats not possible, you''ll need at least 664MB + 32 MB for
the
> > hypervisor + some overhead, xen won''t use exactly 30MB but 
> a little  
> > bit
> > more.
> >
> > one thing: if you want to use the thing to simmulate some network
> > behaviour: xen currently supports only 3 virtual interfaces 
> per domU,
> > that cripples my virtual network as well.
> >
> > harald
> >
> >
> > Rachid Abdoun wrote:
> >> Hi,
> >>
> >> I''d like to create a virtual network with xen who have
more tha 20
> >> hosts. Every host has 30MB in his configuration file. The total of
> >> memory is
> >>
> >> 20X30 = 600MB
> >>
> >> +
> >>
> >> 64MB for the dom0
> >> --------------------------
> >> = 664MB
> >>
> >> but I only have 512MB on my workstation. Can this virtual network
> >> possible with xen? Or you need to have enough ram!?
> >>
> >> Thank you
> >>
> >> _______________________________________________
> >> Xen-users mailing list
> >> Xen-users@lists.xensource.com
> >> http://lists.xensource.com/xen-users
> >>
> >
> >
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.2.1 (GNU/Linux)
> >
> > iD8DBQFEmpmOJgyxs71kcx4RAmwEAKCAXO8kKBxdhZrNP5TWhP6QfbQCygCfcewF
> > GbnXcFYWxm/QqFA/++pCTm4> > =Qm6I
> > -----END PGP SIGNATURE-----
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 
> 


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> > It''s a big problem for me if you create a virtual network
with 100
> > hosts (30MB per host) you need 3 Go per host!!  I think this is the
> > disadvantage  of xen
>
> It may be a disadvantage, but it happens to be the design choosen for Xen.
> Xen has limitations, which may cause it to be a disadvantage in some
> respects.
Yes.

Note that you can still swap within the domains, and the domains themselves 
are able to make better (well, more informed) resource management decisions 
regarding swapping that the rest of the machine.

It''d be nice to support bursty memory usage patterns (e.g. if only a
few
virtual machines are wanting lots of memory then give it to them).  This 
should be possible to automate using the balloon driver, without requiring 
actual overcommit.
> We could obviously use Dom0 for the swapping process (at least in the sense
> of using it''s disk drivers), so that''s not a big issue.
Yep, we''d need some sort of Xen<->dom0 interface for doing the
swapping, and
updating the memory map.

It''d certainly make things a bit complicated.  For instance,
I''d imagine it
would make it less straightforward for guests to manage their own pagetables 
(who gets control over the "present" bit?), and running guests on
shadow
pagetables would be less efficient.
> The hypervisor has
> NO KNOWLEDGE of what''s in which page, so it''s not able to
choose a page
> correctly.
Yep.  In fact, if guests can swap too it''s even worse: double paging
can occur
if a guest wants to page out data that''s already paged out by the
hypervisor.
The hypervisor must fault it back in because the guest accessed it, but the 
guest immediately pages it out again.  Ouch.
> Of course, the OS has some special call that is called when the 
> system is to "lock" a page in memory, and in a para-virtualized
> environment, this could be intercepted and used to prevent the pages from
> being paged out. But in the case of a unmodified OS, this would definitely
> not work, as there''s not a single special operation to track this
> information - the OS would just have some internal table of some sort
> (binary tree, hash-list, linear array, bitmap or whatever the OS designer
> decided was the best way to track a bunch of memory pages...)
Well, locking shouldn''t be a problem except in domains with physical
device
access.  You wouldn''t want to page those!

However, there are still the semantic issues that performance-related 
information is being lost because it''s not visible to the hypervisor. 
I
believe IBM have / are considering some features in z/VM to allow guests to 
hint the characteristics of their pages to the hypervisor so that it can make 
better decisions.  The Disco hypervisor used to use a weird swap device to 
avoid double paging (not sure how applicable that would be to Xen, though).  
And I suspect there''s some interesting work that could be done in
trying to
infer characteristics of guest pages without the guest hinting.

VMware ESX hypervisor allows overcommit of memory (don''t know about
other
VMwares, but I suspect they might allow swapping, they''re not
hypervisors
tho).  It also uses a combination of unifying identical pages, and balloon 
drivers to maximise memory efficiency without resorting to swap, though.  
It''s quite a complicated scheme, but it''d more obvious in ESX
since they use
shadow pagetables and have device drivers in the hypervisor already.

Cheers,
Mark
> I don''t think this will be implemented in Xen for a LONG time.
>
> --
> Mats
>
> > Le 22 juin 06 à 15:22, Harald Schioeberg a écrit :
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA1
> > >
> > > Thats not possible, you''ll need at least 664MB + 32 MB
for the
> > > hypervisor + some overhead, xen won''t use exactly 30MB
but
> >
> > a little
> >
> > > bit
> > > more.
> > >
> > > one thing: if you want to use the thing to simmulate some network
> > > behaviour: xen currently supports only 3 virtual interfaces
> >
> > per domU,
> >
> > > that cripples my virtual network as well.
> > >
> > > harald
> > >
> > > Rachid Abdoun wrote:
> > >> Hi,
> > >>
> > >> I''d like to create a virtual network with xen who
have more tha 20
> > >> hosts. Every host has 30MB in his configuration file. The
total of
> > >> memory is
> > >>
> > >> 20X30 = 600MB
> > >>
> > >> +
> > >>
> > >> 64MB for the dom0
> > >> --------------------------
> > >> = 664MB
> > >>
> > >> but I only have 512MB on my workstation. Can this virtual
network
> > >> possible with xen? Or you need to have enough ram!?
> > >>
> > >> Thank you
> > >>
> > >> _______________________________________________
> > >> Xen-users mailing list
> > >> Xen-users@lists.xensource.com
> > >> http://lists.xensource.com/xen-users
> > >
> > > -----BEGIN PGP SIGNATURE-----
> > > Version: GnuPG v1.4.2.1 (GNU/Linux)
> > >
> > > iD8DBQFEmpmOJgyxs71kcx4RAmwEAKCAXO8kKBxdhZrNP5TWhP6QfbQCygCfcewF
> > > GbnXcFYWxm/QqFA/++pCTm4> > > =Qm6I
> > > -----END PGP SIGNATURE-----
> >
> > _______________________________________________
> > Xen-users mailing list
> > Xen-users@lists.xensource.com
> > http://lists.xensource.com/xen-users
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users