Xen users - Dec 2009 - How many guest hosts per physical host

Hi,

I have a Dell machine, duel core xeon, with 16GB RAM. We are planning to run
Xen on it. The storage can be increased as required. For every guest host
the memory that is going to be allocated is 1 GB. These guest hosts are
going to run web applications (apache/tomcat) for which 1 GB Ram should be
more than sufficient.

Can I create more than 16 guest hosts in my physical host? If the total
number of host with 1GB RAM each exceeds 16, does XEN have any feature to
mange the memory efficiently so that  memory will be allocated to guest host
based on the load on guest hosts at a particular point of time?

Thanks,
SV
-- 
View this message in context:
http://old.nabble.com/How-many-guest-hosts-per-physical-host-tp26806206p26806206.html
Sent from the Xen - User mailing list archive at Nabble.com.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wed, Dec 16, 2009 at 11:39 AM, svarghese <vargh@yahoo.com>
wrote:
> Can I create more than 16 guest hosts in my physical host?
Xen supports running a lot of domUs. Last time I experiment with this
I was able to start over 300 PV domUs with 16 MBs memory each. At that
point xenstored uses lots of CPU, and starting another domU takes some
time, so I stopped it there. Note that I use LVM on dom0 as domU''s
disk.

If you''re looking for maximum number of SUPPORTED domUs, you should
ask your linux vendor (if you''re using bundled Xen).

In your particular setup, you''d most likely be memory-bound, so the
answer would be NO. Not if each domU uses 1 GB. You need to reduce the
ammount of memory for each domU. dom0 needs some memory for itself, so
you can''t allocate all 16GB for domUs.

> If the total
> number of host with 1GB RAM each exceeds 16, does XEN have any feature to
> mange the memory efficiently so that  memory will be allocated to guest
host
> based on the load on guest hosts at a particular point of time?
Not by default. You can try
http://blog.xen.org/index.php/2008/08/27/xen-33-feature-memory-overcommit/
though.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Fri, Dec 18, 2009 at 12:51 PM, Fajar A. Nugraha <fajar@fajar.net>
wrote:
> On Wed, Dec 16, 2009 at 11:39 AM, svarghese <vargh@yahoo.com> wrote:
> > Can I create more than 16 guest hosts in my physical host?
>
> Xen supports running a lot of domUs. Last time I experiment with this
> I was able to start over 300 PV domUs with 16 MBs memory each. At that
> point xenstored uses lots of CPU, and starting another domU takes some
> time, so I stopped it there. Note that I use LVM on dom0 as domU''s
> disk.
>
> If you''re looking for maximum number of SUPPORTED domUs, you
should
> ask your linux vendor (if you''re using bundled Xen).
>
> In your particular setup, you''d most likely be memory-bound, so
the
> answer would be NO. Not if each domU uses 1 GB. You need to reduce the
> ammount of memory for each domU. dom0 needs some memory for itself, so
> you can''t allocate all 16GB for domUs.
>
>
> > If the total
> > number of host with 1GB RAM each exceeds 16, does XEN have any feature
to
> > mange the memory efficiently so that  memory will be allocated to
guest
> host
> > based on the load on guest hosts at a particular point of time?
>
> Not by default. You can try
> http://blog.xen.org/index.php/2008/08/27/xen-33-feature-memory-overcommit/
> though.
>
> --
> Fajar
>
>
I think it''s important to differentiate the number of VMs you can START
and
the number that can possibly RUN on the system! I imagine there''s a
limit to
the number you can start since there''s always a limit somewhere. As far
as
the number you can run that depends on the load of each VM.

I have one machine that''s an 8 core Xeon (32 GB ram) and am running 42
Full
CentOS 5.4 PV DomUs with Guis and 512 MB ea. The first 10 take about 30
seconds to boot each, the next 10 take about a minute to boot each, the next
10 take 2 minutes each etc... If I reboot one VM after the other 40 are up
it takes several minutes before it''s alive again. They do work OK and
provide an interactive environment for an entire classroom but if everyone
opens Firefox at the same time you can imagine how little we get done for
about 5 minutes.

I''m looking to free up resources everywhere I can but it really looks
like
I''m CPU bound at this point. Having all 40 VMs running even if
they''re not
being used for anything slows down the others.

I was playing with trying to get Xen to do qcow2 and have a base image in a
ramdisk for booting/rebooting purposes but failed miserably. Not only does
Xen not have full qcow2 support the code it does have doesn''t work
properly.
Not to mention it won''t even load a VM disk image out of a ramdisk
anyway
even without Qcow2.

I''m testing KVM this week in the same environment. So I guess the
answer to
your question is "it all depends". :-)

Grant McWilliams

Some people, when confronted with a problem, think "I know, I''ll
use
Windows."
Now they have two problems.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> Hi,
> 
> I have a Dell machine, duel core xeon, with 16GB RAM. We are planning
> to run
> Xen on it. The storage can be increased as required. For every guest
> host
> the memory that is going to be allocated is 1 GB. These guest hosts are
> going to run web applications (apache/tomcat) for which 1 GB Ram should
> be
> more than sufficient.
> 
> Can I create more than 16 guest hosts in my physical host? If the total
> number of host with 1GB RAM each exceeds 16, does XEN have any feature
> to
> mange the memory efficiently so that  memory will be allocated to guest
> host
> based on the load on guest hosts at a particular point of time?
The short answer is no.  You cannot overprovision RAM in XEN.
The slightly longer answer is that overprovisioning RAM is a bad idea.  RAM is
cheap.
The long answer is yes.  You can provision PV linux domUs with starting memory,
maximum memory, etc, and they can balloon as needed.

There are real, significant issues with the way ballooning works - or at least,
there were when I last tinkered with it.  But I personally question the idea of
over-committing a cheap resource and then introducing some black magic to manage
it.

Just buy another box when you fill this one up; or better yet, do a shared
storage solution so that when you buy another box, you can move domU''s
to it so that you can take the original box down for maintenance.  Scheduling 15
simultaneous outages to replace one hard drive is a PITA.

Nathan


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Grant McWilliams wrote:
> I have one machine that''s an 8 core Xeon (32 GB ram) and am
running 42
> Full CentOS 5.4 PV DomUs with Guis and 512 MB ea. The first 10 take
If you use the same linux distro and version on every virtual machine,
you can handle much more machines if you use other virtualisation
technology like virtuozzo, openvz, vserver or the new lxc containers.
The last one is integrated in recent kernel as far as i know.
These all run all VMs on one kernel. The VMs have no own kernel and
therefore you can share memory. You can also share filesystem caches if
you bind mount parts of your host installation. With that you can really
run hundreds of virtual machines with your hardware.
Drawback: Running different operating systems is nearly impossible.
Patching is hell.
But it should be possible to use both: xen and lxc. Opensuse 11.2 Xen
kernel has lxc requirements compiled in.





_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> -----Original Message-----
> From: xen-users-bounces@lists.xensource.com [mailto:xen-users-
> bounces@lists.xensource.com] On Behalf Of Florian Gleixner
> Sent: Friday, December 18, 2009 3:53 PM
> To: Grant McWilliams
> Cc: xen-users@lists.xensource.com
> Subject: Re: [Xen-users] How many guest hosts per physical host
> 
> Grant McWilliams wrote:
> 
> > I have one machine that's an 8 core Xeon (32 GB ram) and am
running
> 42
> > Full CentOS 5.4 PV DomUs with Guis and 512 MB ea. The first 10 take
> 
> If you use the same linux distro and version on every virtual machine,
> you can handle much more machines if you use other virtualisation
> technology like virtuozzo, openvz, vserver or the new lxc containers.
> The last one is integrated in recent kernel as far as i know.
> These all run all VMs on one kernel. The VMs have no own kernel and
> therefore you can share memory. You can also share filesystem caches if
> you bind mount parts of your host installation. With that you can
> really run hundreds of virtual machines with your hardware.
> Drawback: Running different operating systems is nearly impossible.
> Patching is hell.
> But it should be possible to use both: xen and lxc. Opensuse 11.2 Xen
> kernel has lxc requirements compiled in.
> 
Incidentally, I use Virtuozzo for Windows 2003 virtualization.  I get absolutely
AMAZING density on it, way better than I could get with Xen.  It's also
expensive, and there's very little user community if you have problems.

Best Regards,
Nathan Eisenberg

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

>
> >
> Incidentally, I use Virtuozzo for Windows 2003 virtualization.  I get
> absolutely AMAZING density on it, way better than I could get with Xen.
>  It''s also expensive, and there''s very little user
community if you have
> problems.
>
> Best Regards,
> Nathan Eisenberg
>
Using the name Virtuozzo and Virtualization in the same sentence is being a
bit optimistic. As far as density - on mysqlbench I''m losing less than
1% to
the exact same server running it native. In order to get more density than
that I think you''d have to be doing something amazing unless your
"virtualized" servers are faster than the real one. So how well does
that
live migration between Windows servers work with Virtuozzo?

As far as the cost, I have 190 Xen servers and I''ve never paid a dime.
As
far as the community I''d think that 80 messages a day on the Xen lists
would
be considered more than very little.

Grant McWilliams

Some people, when confronted with a problem, think "I know, I''ll
use
Windows."
Now they have two problems.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Grant McWilliams wrote:
>     >
>     Incidentally, I use Virtuozzo for Windows 2003 virtualization.  I
>     get absolutely AMAZING density on it, way better than I could get
>     with Xen.  It''s also expensive, and there''s very
little user
>     community if you have problems.
> 
>     Best Regards,
>     Nathan Eisenberg
> 
> 
> Using the name Virtuozzo and Virtualization in the same sentence is
> being a bit optimistic. As far as density - on mysqlbench I''m
losing
> less than 1% to the exact same server running it native. In order to get
> more density than that I think you''d have to be doing something
amazing
> unless your "virtualized" servers are faster than the real one.
So how
> well does that live migration between Windows servers work with Virtuozzo?
> 
I think density here is the number of virtual machines per physical
machine by given cpu/ram/disk resources. You can share cpu and probably
disk with xen, but you can hardly share ram. If you use OS
virtualisation (virtuozzo, lxc, solaris containers) every VM can use the
whole ram - you can limit the VMs if you like. OS virtualisation hat
many limitations and problems but you can run a incredible number of
machines on a machine.

Live migration is possible with some products but i have no idea how
good it is. OS virtualisations has other applications than hypervisor
based techniques.

> As far as the cost, I have 190 Xen servers and I''ve never paid a
dime.
> As far as the community I''d think that 80 messages a day on the
Xen
> lists would be considered more than very little.
> 
I personally think that lxc containers will get a boost soon. The really
nice is: you can combine and run lxc containers within a xen or kvm VM -
 theoretically.





_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Thanks a lots for all the valuable answers.

I guess I should keep the numbers below 16 so that I don''t overrun the
existing 16GB physical RAM.

Thanks.


Nathan Eisenberg wrote:
> 
>> Hi,
>> 
>> I have a Dell machine, duel core xeon, with 16GB RAM. We are planning
>> to run
>> Xen on it. The storage can be increased as required. For every guest
>> host
>> the memory that is going to be allocated is 1 GB. These guest hosts are
>> going to run web applications (apache/tomcat) for which 1 GB Ram should
>> be
>> more than sufficient.
>> 
>> Can I create more than 16 guest hosts in my physical host? If the total
>> number of host with 1GB RAM each exceeds 16, does XEN have any feature
>> to
>> mange the memory efficiently so that  memory will be allocated to guest
>> host
>> based on the load on guest hosts at a particular point of time?
> 
> The short answer is no.  You cannot overprovision RAM in XEN.
> The slightly longer answer is that overprovisioning RAM is a bad idea. 
> RAM is cheap.
> The long answer is yes.  You can provision PV linux domUs with starting
> memory, maximum memory, etc, and they can balloon as needed.
> 
> There are real, significant issues with the way ballooning works - or at
> least, there were when I last tinkered with it.  But I personally question
> the idea of over-committing a cheap resource and then introducing some
> black magic to manage it.  
> 
> Just buy another box when you fill this one up; or better yet, do a shared
> storage solution so that when you buy another box, you can move
domU''s to
> it so that you can take the original box down for maintenance.  Scheduling
> 15 simultaneous outages to replace one hard drive is a PITA.
> 
> Nathan
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 
> 
-- 
View this message in context:
http://old.nabble.com/How-many-guest-hosts-per-physical-host-tp26806206p26876775.html
Sent from the Xen - User mailing list archive at Nabble.com.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> Using the name Virtuozzo and Virtualization in the same sentence is 
> being a bit optimistic. As far as density - on mysqlbench I'm losing 
> less than 1% to the exact same server running it native. In order to 
> get more density than that I think you'd have to be doing something
amazing unless your "virtualized" servers are faster than the real
one. So how well does that live migration between Windows servers work with
Virtuozzo?
>
> As far as the cost, I have 190 Xen servers and I've never paid a dime. 
> As far as the community I'd think that 80 messages a day on the Xen
lists would be considered more than very little.
---

Grant,

I think you may have misinterpreted my statement to mean that Xen is expensive
and the user community is very small.  This is exactly the opposite of what I
was saying: I was speaking about the downsides of Virtuozzo, and the advantages
that Xen has over it.

To be absolutely clear: Xen has an excellent, thriving user community, is free,
and works fantastically.  Virtuozzo is neat for density (# of VMs running on a
piece of hardware), but it’s not free, has no cohesive community, and lacks some
of the features that Xen brings to the table.

Like you, I have many XEN servers deployed in production, and my job would be
MUCH harder without it.

Best Regards,
Nathan Eisenberg
Sr. Systems Administrator - Atlas Networks, LLC

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

That''s easy as 2+2=4!  ;-)

You have 16384M of RAM memory in your Dell....

In your grub''s menu.lst file, setup up this:

xenhopt=dom0_mem=512M

Now you have 15872M for your domUs... Let''s see:

Setup each domU with 992M of RAM (~1G) so, you got 992*16=15872.

Don''t forget to disable ballon in you /etc/xen/xend-config.sxp!

(dom0-min-mem 0)
(enable-dom0-ballooning no)

Cheers!
Thiago

2009/12/16 svarghese <vargh@yahoo.com>
>
> Hi,
>
> I have a Dell machine, duel core xeon, with 16GB RAM. We are planning to
> run
> Xen on it. The storage can be increased as required. For every guest host
> the memory that is going to be allocated is 1 GB. These guest hosts are
> going to run web applications (apache/tomcat) for which 1 GB Ram should be
> more than sufficient.
>
> Can I create more than 16 guest hosts in my physical host? If the total
> number of host with 1GB RAM each exceeds 16, does XEN have any feature to
> mange the memory efficiently so that  memory will be allocated to guest
> host
> based on the load on guest hosts at a particular point of time?
>
> Thanks,
> SV
> --
> View this message in context:
>
http://old.nabble.com/How-many-guest-hosts-per-physical-host-tp26806206p26806206.html
> Sent from the Xen - User mailing list archive at Nabble.com.
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

I''m using Debian Lenny as my dom0s! So I can run update-grub to enable
the
dom0_mem option...
If you are using other distro, see its own manual!

-
 Thiago

2009/12/21 Thiago Camargo Martins Cordeiro <thiagocmartinsc@gmail.com>
> That''s easy as 2+2=4!  ;-)
>
> You have 16384M of RAM memory in your Dell....
>
> In your grub''s menu.lst file, setup up this:
>
> xenhopt=dom0_mem=512M
>
> Now you have 15872M for your domUs... Let''s see:
>
> Setup each domU with 992M of RAM (~1G) so, you got 992*16=15872.
>
> Don''t forget to disable ballon in you /etc/xen/xend-config.sxp!
>
> (dom0-min-mem 0)
> (enable-dom0-ballooning no)
>
> Cheers!
> Thiago
>
> 2009/12/16 svarghese <vargh@yahoo.com>
>
>
>> Hi,
>>
>> I have a Dell machine, duel core xeon, with 16GB RAM. We are planning
to
>> run
>> Xen on it. The storage can be increased as required. For every guest
host
>> the memory that is going to be allocated is 1 GB. These guest hosts are
>> going to run web applications (apache/tomcat) for which 1 GB Ram should
be
>> more than sufficient.
>>
>> Can I create more than 16 guest hosts in my physical host? If the total
>> number of host with 1GB RAM each exceeds 16, does XEN have any feature
to
>> mange the memory efficiently so that  memory will be allocated to guest
>> host
>> based on the load on guest hosts at a particular point of time?
>>
>> Thanks,
>> SV
>> --
>> View this message in context:
>>
http://old.nabble.com/How-many-guest-hosts-per-physical-host-tp26806206p26806206.html
>> Sent from the Xen - User mailing list archive at Nabble.com.
>>
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
>
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue, Dec 22, 2009 at 2:26 AM, Thiago Camargo Martins Cordeiro
<thiagocmartinsc@gmail.com> wrote:
> That''s easy as 2+2=4!  ;-)
Well, not exactly :P
>
> You have 16384M of RAM memory in your Dell....
>
> In your grub''s menu.lst file, setup up this:
>
> xenhopt=dom0_mem=512M
>
> Now you have 15872M for your domUs... Let''s see:
>
> Setup each domU with 992M of RAM (~1G) so, you got 992*16=15872.
the hypervisor itself uses some memory. Also, as Pasi mention in
another thread, there''s additional memory overhead when allocating
memory and every vcpu for domUs. So you''re still missing some
variables.

In general it''s correct though. If you set each domU to use lower
memory (about 900MB should do) you can still have a total of 16 domUs.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users