Hi, there''s no vm config file form Dom0, right? So how does XEN assign virtual CPUs to real CPUs? I see this on a two Dual-Core Opteron machine: # xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 1 r-- 2927.8 any cpu Domain-0 0 1 2 -b- 2551.1 any cpu Domain-0 0 2 0 -b- 2477.0 any cpu Domain-0 0 3 0 -b- 2231.4 any cpu So VCPU 2 and 3 use the same CPU (0), even though there''s another CPU (3). Why? On another machine (same model) things look better: # xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 142.9 any cpu Domain-0 0 1 2 -b- 90.0 any cpu Domain-0 0 2 1 -b- 81.6 any cpu Domain-0 0 3 3 r-- 134.6 any cpu Regards, Ulrich _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> there''s no vm config file form Dom0, right? So how does XEN assign virtual > CPUs to real CPUs? I see this on a two Dual-Core Opteron machine: > # xm vcpu-list > Name ID VCPU CPU State Time(s) CPU > Affinity Domain-0 0 0 1 r-- 2927.8 > any cpu Domain-0 0 1 2 -b- 2551.1 > any cpu Domain-0 0 2 0 -b- 2477.0 > any cpu Domain-0 0 3 0 -b- 2231.4 > any cpuWell, you have "affinity = any cpu" for all your VCPUs, so they''re probably running on whatever CPU is free. Maybe CPU 3 was being used be another domain at the time, so the VCPUs had been migrated? I''ve not looked into the code here for a while, though, so I could be wrong. If you run the command again under different workload conditions, you might find that the vcpu-list command returns different results... I think you can vcpu-pin VCPUs to logical processors if you want to do the mapping manually (e.g. you want stricter performance partitioning at the expense of not using as many available cycles on all CPUs as possible). Cheers, Mark> So VCPU 2 and 3 use the same CPU (0), even though there''s another CPU (3). > Why? > > On another machine (same model) things look better: > > # xm vcpu-list > Name ID VCPU CPU State Time(s) CPU > Affinity Domain-0 0 0 0 -b- 142.9 > any cpu Domain-0 0 1 2 -b- 90.0 > any cpu Domain-0 0 2 1 -b- 81.6 > any cpu Domain-0 0 3 3 r-- 134.6 > any cpu > > Regards, > Ulrich > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users-- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Thursday, 26 October 2006 01:22:56 +0100, Mark Williamson wrote: Hi Mark.> I think you can vcpu-pin VCPUs to logical processors if you want to do > the mapping manually (e.g. you want stricter performance partitioning > at the expense of not using as many available cycles on all CPUs as > possible).Do You know if pinning can be accomplished in boot time? I didn''t see that in xmdomain.cfg(5). Regards, Daniel -- Daniel Bareiro - System Administrator Fingerprint: BFB3 08D6 B4D1 31B2 72B9 29CE 6696 BF1B 14E6 1D37 Powered by Debian GNU/Linux Etch - Linux user #188.598 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> > I think you can vcpu-pin VCPUs to logical processors if you want to do > > the mapping manually (e.g. you want stricter performance partitioning > > at the expense of not using as many available cycles on all CPUs as > > possible). > > Do You know if pinning can be accomplished in boot time? I didn''t see > that in xmdomain.cfg(5).Looking at the code (tools/python/xen/xm/create.py) I think the correct argument is a list of logical CPUs (i.e. cores, or hyperthreads if you have them) that the domain''s vCPUs should run on, e.g.: cpus = [''0'', ''1''] To run the vCPUs on logical processors 0 and 1. Cheers, Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Thursday, 26 October 2006 04:51:06 +0100, Mark Williamson wrote: Hi Mark.> > Do You know if pinning can be accomplished in boot time? I didn''t > > see that in xmdomain.cfg(5).> Looking at the code (tools/python/xen/xm/create.py) I think the > correct argument is a list of logical CPUs (i.e. cores, or > hyperthreads if you have them) that the domain''s vCPUs should run on, > e.g.:> cpus = [''0'', ''1'']> To run the vCPUs on logical processors 0 and 1.According to xmdomain.cfg(5), something like cpus = "0,1". Even so, I''m not sure that it will pin the vcpus to different physical cpus. Regards, Daniel -- Daniel Bareiro - System Administrator Fingerprint: BFB3 08D6 B4D1 31B2 72B9 29CE 6696 BF1B 14E6 1D37 Powered by Debian GNU/Linux Etch - Linux user #188.598 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
People have talked about different things in this thread so I thought I should clarify the options available to set CPU affinity to various VCPUs for both dom0 and domUs: The config file variable "cpus" defines a cpu affinity mask which applies to all VCPUs of a domain. So all VCPUs for that domain will be assigned the same CPU mask. I suggest you have at least as many CPUs set in the mask as you have VCPUs. You can also set this per VCPU after a domU has started using the "xm vcpu-pin" command. In 3.0.3 and unstable, you can use vcpu-pin on dom0 as well. Also in 3.0.3 and unstable, you can use the Xen boot argument "dom0_vcpus_pin" to pin VCPU N or dom0 on physical CPU N. Also to note is that the credit scheduler is already moving things around for you to keep runnable VCPUs running. No physical CPU will sit idle while there is a runnable VCPU, dom0 or domU. Cheers, Emmanuel _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Friday, 27 October 2006 14:29:11 +0100, Emmanuel Ackaouy wrote:> People have talked about different things in this thread so > I thought I should clarify the options available to set CPU > affinity to various VCPUs for both dom0 and domUs:Thanks for the explanation.> The config file variable "cpus" defines a cpu affinity mask > which applies to all VCPUs of a domain. So all VCPUs for that > domain will be assigned the same CPU mask.> I suggest you have at least as many CPUs set in the mask as > you have VCPUs.Yes. I did it. Is there some manner to setting the vcpus for the dom0?> You can also set this per VCPU after a domU has started using > the "xm vcpu-pin" command.If it cannot be automatized in boot time, then a script could perform this work: xenhost1:~# cat vcpu-pin #!/bin/sh echo -n "Pinning domUs..." /usr/sbin/xm vcpu-pin 1 0 0 /usr/sbin/xm vcpu-pin 1 1 3 echo "done." xenhost1:~# update-rc.d vcpu-pin start 40 S> In 3.0.3 and unstable, you can use vcpu-pin on dom0 as well. > Also in 3.0.3 and unstable, you can use the Xen boot argument > "dom0_vcpus_pin" to pin VCPU N or dom0 on physical CPU N.Interesting... but I''m not sure if I have understood the difference between the vcpu-pin use for the dom0 and dom0_vcpu_pin boot argument. do they produce the same result but dom0_vcpus_pin is a Grub argument? Regards, Daniel -- Daniel Bareiro - System Administrator Fingerprint: BFB3 08D6 B4D1 31B2 72B9 29CE 6696 BF1B 14E6 1D37 Powered by Debian GNU/Linux Etch - Linux user #188.598 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sun, Oct 29, 2006 at 05:29:30PM -0300, Daniel Bareiro wrote:> > In 3.0.3 and unstable, you can use vcpu-pin on dom0 as well. > > Also in 3.0.3 and unstable, you can use the Xen boot argument > > "dom0_vcpus_pin" to pin VCPU N or dom0 on physical CPU N. > > Interesting... but I''m not sure if I have understood the difference > between the vcpu-pin use for the dom0 and dom0_vcpu_pin boot argument. > do they produce the same result but dom0_vcpus_pin is a Grub argument?When using dom0_vcpus_pin at boot time, your dom0 VCPUs are created and start running on their assigned CPUs right away. They never run elsewhere. Also, once dom0_vcpus_pin has been specified, the dom0 VCPUs are "glued" in place: You can no longer use vcpu-pin to change their affinity mask or move them around. Cheers, Emmanuel. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users