The manual states the following `If you are running IO intensive tasks, its typically better to dedicate either a hyperthread or whole core to running domain 0, and hence pin other domains so that they can''t use CPU 0` Right now I have a host that has 4 procs (dual xeon, with HT) and dom0 us showing all 4 procs. Since IO is whats thrashed first on these things (at least with in my env) i would like to dedicated 1 logical proc to dom0 and have the guests use the other 3. First question is how can i tell dom0 to not use all 4 procs? I have tryed vcpu-set 0 0 but that didnt change anything. I could vcpu-pin cpu 0-3 to cpu0 but that seams like a bad idea because now dom0 thinks it has 4 cpu''s but actually it''s 1. [root@localhost bin]# xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 2771.4 any cpu Domain-0 0 1 1 -b- 2837.8 any cpu Domain-0 0 2 2 -b- 2159.9 any cpu Domain-0 0 3 3 r-- 2283.8 any cpu So first question is, how can i get dom0 to stop using all 4 procs and to only use 1? Second whats the best way to have guests use the rest of the cpu? Do i need to write a script that runs through each guest settting it manually to 1, 2, or 3? Another question, i started up 20 guests and all guests uses cpu 1 or 3, none of them used 0 or 2. Why? Thanks Shaun _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Wed, Feb 22, 2006 at 02:41:22PM -0800, Shaun wrote:> The manual states the following > > `If you are running IO intensive tasks, its typically better to dedicate > either a hyperthread or whole core to running domain 0, and hence pin other > domains so that they can''t use CPU 0` > > Right now I have a host that has 4 procs (dual xeon, with HT) and dom0 us > showing all 4 procs.Well, you have 2 procs with HT, which is different from 4 real procs.> Since IO is whats thrashed first on these things (at > least with in my env) i would like to dedicated 1 logical proc to dom0 and > have the guests use the other 3. First question is how can i tell dom0 to > not use all 4 procs? I have tryed vcpu-set 0 0 but that didnt change > anything. I could vcpu-pin cpu 0-3 to cpu0 but that seams like a bad idea > because now dom0 thinks it has 4 cpu''s but actually it''s 1. > > [root@localhost bin]# xm vcpu-list > Name ID VCPU CPU State Time(s) CPU > Affinity > Domain-0 0 0 0 -b- 2771.4 any cpu > Domain-0 0 1 1 -b- 2837.8 any cpu > Domain-0 0 2 2 -b- 2159.9 any cpu > Domain-0 0 3 3 r-- 2283.8 any cpu > > So first question is, how can i get dom0 to stop using all 4 procs and to > only use 1?set max_cpu=1 on the Dom0 kernel boot line.> Second whats the best way to have guests use the rest of the > cpu? Do i need to write a script that runs through each guest settting it > manually to 1, 2, or 3?Manual script is probably needed.> Another question, i started up 20 guests and all guests uses cpu 1 or 3, > none of them used 0 or 2. Why?Because xen is aware enough to know that HTs aren''t real CPUs, and you''ll actually get very bad performance running unrelated domains on competing HT on the same CPU (this was found by trial and error). Thus it is allocating to CPU1 because it assumes Dom0 gets the dedicated CPU0 HT, and CPU3 just to keep anything off the other HT on the second real proc. -Sean -- Sean Dague IBM Linux Technology Center email: japh@us.ibm.com Open Hypervisor Team alt: sldague@us.ibm.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Wednesday 22 February 2006 23:41, Shaun wrote:> The manual states the following > > `If you are running IO intensive tasks, its typically better to dedicate > either a hyperthread or whole core to running domain 0, and hence pin other > domains so that they can''t use CPU 0` >[...]> So first question is, how can i get dom0 to stop using all 4 procs and to > only use 1? Second whats the best way to have guests use the rest of the > cpu? Do i need to write a script that runs through each guest settting it > manually to 1, 2, or 3?/etc/xen/xend-config.sxp: .. # In SMP system, dom0 will use dom0-cpus # of CPUS # If dom0-cpus = 0, dom0 will take all cpus available (dom0-cpus 1) .. in /etc/xen/somedomain .. # Which CPU to start domain on? cpus = "2,3,1" # leave to Xen to pick # Number of Virtual CPUS to use, default is 1 vcpus = 3 .. /etc/xen/someother .. cpus="1,2,3" ..> Another question, i started up 20 guests and all guests uses cpu 1 or 3, > none of them used 0 or 2. Why?-- xebec mediafactory GmbH | Siemensstrasse 1 | 93055 Regensburg | Germany phone: +49.941.56814-123 | fax: +49.941.56814-299 email: e.bachmann@xebec.de | internet: http://www.xebec.de _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> /etc/xen/xend-config.sxp: > .. > # In SMP system, dom0 will use dom0-cpus # of CPUS > # If dom0-cpus = 0, dom0 will take all cpus available > (dom0-cpus 1)Tryed that, doesnt do anything, also according to the XenTodoList (http://wiki.xensource.com/xenwiki/XenTodoList) that directive isnt even used yet... `3 on xend start, shrink dom0 VCPUs and memory down to constants specified in xend-config (dom0-vcpus=1 dom0-mem=XXX)` Thanks, Shaun _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
So then should I disable HyperThreading? If so, that sucks since now only 1 cpu is available for guests in my setup. Thanks for your responce! ~Shaun _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Also... here''s my grub conf...
title CenOS-Xen (2.6.12.6-xen0)
root (hd0,0)
kernel /xen-3.gz dom0_mem=512000 com1=115200,8n1 max_cpu=1
module /vmlinuz-2.6.12.6-xen0 root=/dev/VolGroup00/LogVol00 ro
console=tty1 console=ttyS0,115200n8
module /initrd-2.6.12.6-xen0.img
and a xm vcpu-list....
[root@localhost ~]# xm vcpu-list
Name ID VCPU CPU State Time(s) CPU
Affinity
Domain-0 0 0 0 r-- 10.0 any cpu
Domain-0 0 1 1 -b- 5.5 any cpu
Domain-0 0 2 2 -b- 7.1 any cpu
Domain-0 0 3 3 -b- 5.0 any cpu
still showing all 4 cpu''s
~Shaun
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Hi Shaun, If I understood you want Domain0 using only #CPU0, right? I got this by doing the following: module /vmlinuz-2.6.12.6-xen0 root=/dev/hda1 ro console=tty1 max_cpus=1 I saw you''re using in the line that you specify the Xen-3 kernel. Perhaps, this is the source of the problem. Regards, Eli module /vmlinuz-2.6.12.6-xen0 root=/dev/VolGroup00/LogVol00 ro console=tty1 max_cpus=1 Shaun wrote:> Also... here''s my grub conf... > > title CenOS-Xen (2.6.12.6-xen0) > root (hd0,0) > kernel /xen-3.gz dom0_mem=512000 com1=115200,8n1 max_cpu=1 > module /vmlinuz-2.6.12.6-xen0 root=/dev/VolGroup00/LogVol00 ro > console=tty1 console=ttyS0,115200n8 > module /initrd-2.6.12.6-xen0.img > > > and a xm vcpu-list.... > > [root@localhost ~]# xm vcpu-list > Name ID VCPU CPU State Time(s) CPU > Affinity > Domain-0 0 0 0 r-- 10.0 any cpu > Domain-0 0 1 1 -b- 5.5 any cpu > Domain-0 0 2 2 -b- 7.1 any cpu > Domain-0 0 3 3 -b- 5.0 any cpu > > still showing all 4 cpu''s > > ~Shaun > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
though my problem may have been that i had max_cpu=1 when you said max_cpus=1 but tryed max_cpus=1 and still no go.... _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Shaun, I introduced a typo :-) I meant: maxcpus=1 (without the underscore _) Regards, Eli Shaun wrote:> though my problem may have been that i had max_cpu=1 when you said > max_cpus=1 but tryed max_cpus=1 and still no go.... > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
yep, that one made my dom0 only have 1 cpu... yay... now to test and make sure the guests use the others... :) "This stuff should be added to the docs!" ~Shaun _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Negitive on that one, setting maxcpus=1 does tell dom0 to only use 1cpu but all guests can only use the same cpu, trying to pin a guest to cpu 1 or 2 or 3 fails with a error(22, ''Invalid argument'') but a vcpu-pin 0 0 works (oviously). The docs talk about how it''s a IO performance improvment to do what i''m trying to do but it seams like this hasnt been implemented yet!? at least not in 3.x... ~Shaun _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
My problem was i was setting maxcpus=1 on the kernel line, should have been on the module line. Also xm vcpu-list will still show all cpu''s regardless it just shows the ones it''s not using as in a paused state. xm list will show VCPU as 1. thanks to Sean Dague for the help today. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users