Scott Classen wrote:>1. Since I only have 8 threads maybe is this too many domUs? Will they >be constantly fighting over CPU cycles?Just like multi-tasking without virtual machines, if there are more tasks to run than there are cores to run them on then the tasks will be time shared between the available cores. Some people like to pin a single core for the sole use of Dom0 so as to guarantee no contention and thus best response times for Dom0 tasks (which includes shuffling network packets about.>2. Should dom0 be the most minimal installation needed to administer >the domUs? Do the resources allocated to dom0 affect the performance >of the domUs?It is "traditional" that Dom0 should have the bare minimum of software installed and running. Compromise Dom0 and you also compromise every guest - thus from a security point of view it''s best to minimise the attack vector. However, if you want, then you can run a full graphical desktop in Dom0 and a few guests as well. particularly useful for experimentation as you can run your normal desktop stuff without tying up a machine for Xen.>3. Should I leave only 1 GB RAM for dom0 and give the rest to the domUs?General advice seems to be to explicitly limit the ram available to Dom0 (via boot string argument). Several memory structures are sized according to the amount of memory at boot time - and so having Dom0 have access to huge amounts of memory which will actually be used by guests means that it sizes it''s data structures accordingly and wastes some memory. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hello xen gurus, I''ve recently acquired a new computer for the express purpose of exploring xen. Eventually I would like to explore global shared filesystems and the ability to migrate running domUs between different physical computers, but for now I''m just trying to get a sense of how many domUs I can run on a single Xeon and generally exploring the capabilities and limitations of xen. my test computer: Supermico X8SAXS Motherboard. A single Xeon L5630 2.13GHz CPU - 4 cores/8 threads 12GB RAM 9650SE- 2LP 3ware RAID car with two 1TB drives in RAID1 configuration. dom0 and all domUs are using the 3ware. domUs are using a disk image file (i.e. disk = [ ''tap:aio:/var/lib/xen/images/xen1.img,xvda,w'', ] ) I''ve installed stock CentOS 5.5 xenified kernel (2.6.18-194.26.1.el5xen) and 4 identical domUs with 2VCPUs and 2GB RAM each: dom0 = 8 VCPU 4GB RAM xen1 = 2 VCPU 2GB RAM xen2 = 2 VCPU 2GB RAM xen3 = 2 VCPU 2GB RAM xen4 = 2 VCPU 2GB RAM My questions: 1. Since I only have 8 threads maybe is this too many domUs? Will they be constantly fighting over CPU cycles? 2. Should dom0 be the most minimal installation needed to administer the domUs? Do the resources allocated to dom0 affect the performance of the domUs? 3. Should I leave only 1 GB RAM for dom0 and give the rest to the domUs? Thanks, for any general advice on setting up xen. Scott _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Thu, 11 Nov 2010, Scott Classen wrote:> Hello xen gurus, > > I''ve recently acquired a new computer for the express purpose of > exploring xen. Eventually I would like to explore global shared > filesystems and the ability to migrate running domUs between different > physical computers, but for now I''m just trying to get a sense of how > many domUs I can run on a single Xeon and generally exploring the > capabilities and limitations of xen. > > my test computer: > Supermico X8SAXS Motherboard. > A single Xeon L5630 2.13GHz CPU - 4 cores/8 threads > 12GB RAM > 9650SE- 2LP 3ware RAID car with two 1TB drives in RAID1 configuration. > dom0 and all domUs are using the 3ware. domUs are using a disk image > file (i.e. disk = [ ''tap:aio:/var/lib/xen/images/xen1.img,xvda,w'', ] ) > > I''ve installed stock CentOS 5.5 xenified kernel > (2.6.18-194.26.1.el5xen) and 4 identical domUs with 2VCPUs and 2GB RAM > each: > > dom0 = 8 VCPU 4GB RAM > > xen1 = 2 VCPU 2GB RAM > xen2 = 2 VCPU 2GB RAM > xen3 = 2 VCPU 2GB RAM > xen4 = 2 VCPU 2GB RAM > > My questions: > 1. Since I only have 8 threads maybe is this too many domUs? Will they > be constantly fighting over CPU cycles?The answer depends on how busy the domU''s are. If they are idle most of the time then you can "over subscribe" and run many more vcpu''s than you have real cpu''s.> 2. Should dom0 be the most minimal installation needed to administer > the domUs? Do the resources allocated to dom0 affect the performance > of the domUs?Best practies are that you make it a minimal install. Most people will tell you that you should pin a cpu or 2 to the dom0 and allocate memory also. This can be done by adding "dom0_mem=XXXXM dom0_max_vcpus=X dom0_vpus_pin" to the kernel line in your dom0 grub config (be sure to change the X''s to an appropriate value). This will insure that your dom0 doesn''t compete for resources with the domU''s. Like this: kernel /boot/xen-3.4.3.gz dom0_mem=1536M dom0_max_vcpus=1 dom0_vpus_pin> 3. Should I leave only 1 GB RAM for dom0 and give the rest to the domUs?See the above. I have found that 1-2 GB of ram works well.> Thanks, for any general advice on setting up xen.I recommend you have a look at this book as a starting point ( I am not affiliated with the author or the publisher) It is a good general read but is outdated (But then aren''t all the xen docs). It touches on most of your questions. http://nostarch.com/xen.htm Good luck. Tom. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Henry Pepper wrote:>As to the Dom0 pinning, I was under the impression that not only will >you have to pin Dom0 to e.g. cpu0, but you will have to ensure that no >other VPCU can run on cpu0. So that would mean that all other domains >will have to be pinned to CPU 1 through 7.Correct. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
The Xen scheduler is handling VCPU scheduling. As I understand it, the scheduler will by default switch VCPUS every 30ms. This includes Dom0. So if you have more VCPUs than CPUs some VCPUs will be off CPU in some periods. As to the Dom0 pinning, I was under the impression that not only will you have to pin Dom0 to e.g. cpu0, but you will have to ensure that no other VPCU can run on cpu0. So that would mean that all other domains will have to be pinned to CPU 1 through 7. Using the Xen Credit scheduler you can give some domains more CPU time, than other Domains. Meaning that if one domU1 has 256 credits and domU2 has 128, then domU1 will, over time, get twice as much CPU time. Note that if domU1 has nothing to do in a period, then domU2 will get all the CPU time. Also note that I believe that the domain credit will be split among the VPCUs in the domain, so if domU1 has 4 VCPUs then each VPCU would have 64 credits. again if the some of those VCPU doesn''t use up their credit, then the rest of the VCPUs in the domain will get to use the remaining credits. Please note that it is only CPU allocation that the scheduler takes care of, I/O sharing is handled by the Dom0 OS. So if the Dom0 OS is up to fairly share disk I/O a single domU could starve all other domUs. Also it seems that local Dom0 I/O operations has a tendency to take precedence. There is also the book: ''The Definitive Guide to the Xen Hypervisor'' http://my.safaribooksonline.com/9780132349710 Henry On Thu, Nov 11, 2010 at 10:14 PM, Scott Classen <sclassen@lbl.gov> wrote:> Hello xen gurus, > > I''ve recently acquired a new computer for the express purpose of > exploring xen. Eventually I would like to explore global shared > filesystems and the ability to migrate running domUs between different > physical computers, but for now I''m just trying to get a sense of how > many domUs I can run on a single Xeon and generally exploring the > capabilities and limitations of xen. > > my test computer: > Supermico X8SAXS Motherboard. > A single Xeon L5630 2.13GHz CPU - 4 cores/8 threads > 12GB RAM > 9650SE- 2LP 3ware RAID car with two 1TB drives in RAID1 configuration. > dom0 and all domUs are using the 3ware. domUs are using a disk image > file (i.e. disk = [ ''tap:aio:/var/lib/xen/images/xen1.img,xvda,w'', ] ) > > I''ve installed stock CentOS 5.5 xenified kernel > (2.6.18-194.26.1.el5xen) and 4 identical domUs with 2VCPUs and 2GB RAM > each: > > dom0 = 8 VCPU 4GB RAM > > xen1 = 2 VCPU 2GB RAM > xen2 = 2 VCPU 2GB RAM > xen3 = 2 VCPU 2GB RAM > xen4 = 2 VCPU 2GB RAM > > My questions: > 1. Since I only have 8 threads maybe is this too many domUs? Will they > be constantly fighting over CPU cycles? > > 2. Should dom0 be the most minimal installation needed to administer > the domUs? Do the resources allocated to dom0 affect the performance > of the domUs? > > 3. Should I leave only 1 GB RAM for dom0 and give the rest to the domUs? > > Thanks, for any general advice on setting up xen. > > Scott > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users