> Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64, > 16 procs, 16 GB RAM under SLES 10 RC1. > > At startup, dom0-cpus is set to 0 (the default) in the xend-config.sxp > file, and dom0 uses all 16 cpus. ''xm top'' shows 16 procs, ''xm list'' > shows dom0 using 16 cpus, and ''xm vcpu-list'' shows 16 cpus assigned to > dom0. > > When we try to reduce the procs for dom0 to 1 using ''xm vcpu-set 0 1'', > the number of procs, as shown by /proc/cpuinfo, reduces to 9. ''xm > list'' shows dom0 using 9 procs, and ''xm vcpu-list'' shows that procs > 1-7 have been released, while procs 0 and 8 through 15 are still > assigned. Nothing is shown on the serial port. > > Can anyone shed some light on why the rest of the procs might not be > released? Thanks. >The following is output from ''xm list'' and ''xm vcpu-list'': m0075:~ # xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 15912 16 r----- 849.3 m0075:~ # xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 438.1 any cpu Domain-0 0 1 1 -b- 4.0 any cpu Domain-0 0 2 2 -b- 5.4 any cpu Domain-0 0 3 3 -b- 4.5 any cpu Domain-0 0 4 4 -b- 8.4 any cpu Domain-0 0 5 5 -b- 6.1 any cpu Domain-0 0 6 6 -b- 5.0 any cpu Domain-0 0 7 7 -b- 4.5 any cpu Domain-0 0 8 8 -b- 30.3 any cpu Domain-0 0 9 9 -b- 14.6 any cpu Domain-0 0 10 10 r-- 29.1 any cpu Domain-0 0 11 11 -b- 15.1 any cpu Domain-0 0 12 12 -b- 71.6 any cpu Domain-0 0 13 13 -b- 177.0 any cpu Domain-0 0 14 14 -b- 14.5 any cpu Domain-0 0 15 15 -b- 21.9 any cpu m0075:~ # xm vcpu-set 0 1 m0075:~ # xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 15912 9 r----- 853.2 m0075:~ # xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 439.3 any cpu Domain-0 0 1 - --p 4.0 any cpu Domain-0 0 2 - --p 5.6 any cpu Domain-0 0 3 - --p 4.5 any cpu Domain-0 0 4 - --p 8.4 any cpu Domain-0 0 5 - --p 6.1 any cpu Domain-0 0 6 - --p 5.0 any cpu Domain-0 0 7 - --p 4.5 any cpu Domain-0 0 8 8 -b- 30.3 any cpu Domain-0 0 9 9 -b- 15.0 any cpu Domain-0 0 10 10 r-- 29.3 any cpu Domain-0 0 11 11 -b- 15.2 any cpu Domain-0 0 12 12 -b- 72.7 any cpu Domain-0 0 13 13 -b- 177.5 any cpu Domain-0 0 14 14 -b- 14.5 any cpu Domain-0 0 15 15 -b- 22.0 any cpu brian carb unisys corporation - malvern, pa brian.carb@unisys.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Set (dom-cpus=1) in etc/xen/xend-config.sxp. - Padma - ________________________________ From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Carb, Brian A Sent: Thursday, June 08, 2006 1:12 PM To: xen-devel@lists.xensource.com Cc: McAfee, Tommie M; Krysan,Susan; Puthiyaparambil,Aravindh; Subrahmanian,Raj; Vessey,Bruce A; Carb, Brian A Subject: [Xen-devel] Unable to set dom0 vcpu to 1 Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64, 16 procs, 16 GB RAM under SLES 10 RC1. At startup, dom0-cpus is set to 0 (the default) in the xend-config.sxp file, and dom0 uses all 16 cpus. ''xm top'' shows 16 procs, ''xm list'' shows dom0 using 16 cpus, and ''xm vcpu-list'' shows 16 cpus assigned to dom0. When we try to reduce the procs for dom0 to 1 using ''xm vcpu-set 0 1'', the number of procs, as shown by /proc/cpuinfo, reduces to 9. ''xm list'' shows dom0 using 9 procs, and ''xm vcpu-list'' shows that procs 1-7 have been released, while procs 0 and 8 through 15 are still assigned. Nothing is shown on the serial port. Can anyone shed some light on why the rest of the procs might not be released? Thanks. The following is output from ''xm list'' and ''xm vcpu-list'': m0075:~ # xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 15912 16 r----- 849.3 m0075:~ # xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 438.1 any cpu Domain-0 0 1 1 -b- 4.0 any cpu Domain-0 0 2 2 -b- 5.4 any cpu Domain-0 0 3 3 -b- 4.5 any cpu Domain-0 0 4 4 -b- 8.4 any cpu Domain-0 0 5 5 -b- 6.1 any cpu Domain-0 0 6 6 -b- 5.0 any cpu Domain-0 0 7 7 -b- 4.5 any cpu Domain-0 0 8 8 -b- 30.3 any cpu Domain-0 0 9 9 -b- 14.6 any cpu Domain-0 0 10 10 r-- 29.1 any cpu Domain-0 0 11 11 -b- 15.1 any cpu Domain-0 0 12 12 -b- 71.6 any cpu Domain-0 0 13 13 -b- 177.0 any cpu Domain-0 0 14 14 -b- 14.5 any cpu Domain-0 0 15 15 -b- 21.9 any cpu m0075:~ # xm vcpu-set 0 1 m0075:~ # xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 15912 9 r----- 853.2 m0075:~ # xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 439.3 any cpu Domain-0 0 1 - --p 4.0 any cpu Domain-0 0 2 - --p 5.6 any cpu Domain-0 0 3 - --p 4.5 any cpu Domain-0 0 4 - --p 8.4 any cpu Domain-0 0 5 - --p 6.1 any cpu Domain-0 0 6 - --p 5.0 any cpu Domain-0 0 7 - --p 4.5 any cpu Domain-0 0 8 8 -b- 30.3 any cpu Domain-0 0 9 9 -b- 15.0 any cpu Domain-0 0 10 10 r-- 29.3 any cpu Domain-0 0 11 11 -b- 15.2 any cpu Domain-0 0 12 12 -b- 72.7 any cpu Domain-0 0 13 13 -b- 177.5 any cpu Domain-0 0 14 14 -b- 14.5 any cpu Domain-0 0 15 15 -b- 22.0 any cpu brian carb unisys corporation - malvern, pa brian.carb@unisys.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Padma, Shouldn''t the xm vcpu-set work regardless of the dom0-cpus value? Raj ________________________________ From: Apparao, Padmashree K [mailto:padmashree.k.apparao@intel.com] Sent: Thursday, June 08, 2006 5:03 PM To: Carb, Brian A; xen-devel@lists.xensource.com Cc: McAfee, Tommie M; Krysan, Susan; Puthiyaparambil, Aravindh; Subrahmanian, Raj; Vessey, Bruce A Subject: RE: [Xen-devel] Unable to set dom0 vcpu to 1 Set (dom-cpus=1) in etc/xen/xend-config.sxp. - Padma - ________________________________ From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Carb, Brian A Sent: Thursday, June 08, 2006 1:12 PM To: xen-devel@lists.xensource.com Cc: McAfee, Tommie M; Krysan,Susan; Puthiyaparambil,Aravindh; Subrahmanian,Raj; Vessey,Bruce A; Carb, Brian A Subject: [Xen-devel] Unable to set dom0 vcpu to 1 Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64, 16 procs, 16 GB RAM under SLES 10 RC1. At startup, dom0-cpus is set to 0 (the default) in the xend-config.sxp file, and dom0 uses all 16 cpus. ''xm top'' shows 16 procs, ''xm list'' shows dom0 using 16 cpus, and ''xm vcpu-list'' shows 16 cpus assigned to dom0. When we try to reduce the procs for dom0 to 1 using ''xm vcpu-set 0 1'', the number of procs, as shown by /proc/cpuinfo, reduces to 9. ''xm list'' shows dom0 using 9 procs, and ''xm vcpu-list'' shows that procs 1-7 have been released, while procs 0 and 8 through 15 are still assigned. Nothing is shown on the serial port. Can anyone shed some light on why the rest of the procs might not be released? Thanks. The following is output from ''xm list'' and ''xm vcpu-list'': m0075:~ # xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 15912 16 r----- 849.3 m0075:~ # xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 438.1 any cpu Domain-0 0 1 1 -b- 4.0 any cpu Domain-0 0 2 2 -b- 5.4 any cpu Domain-0 0 3 3 -b- 4.5 any cpu Domain-0 0 4 4 -b- 8.4 any cpu Domain-0 0 5 5 -b- 6.1 any cpu Domain-0 0 6 6 -b- 5.0 any cpu Domain-0 0 7 7 -b- 4.5 any cpu Domain-0 0 8 8 -b- 30.3 any cpu Domain-0 0 9 9 -b- 14.6 any cpu Domain-0 0 10 10 r-- 29.1 any cpu Domain-0 0 11 11 -b- 15.1 any cpu Domain-0 0 12 12 -b- 71.6 any cpu Domain-0 0 13 13 -b- 177.0 any cpu Domain-0 0 14 14 -b- 14.5 any cpu Domain-0 0 15 15 -b- 21.9 any cpu m0075:~ # xm vcpu-set 0 1 m0075:~ # xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 15912 9 r----- 853.2 m0075:~ # xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 439.3 any cpu Domain-0 0 1 - --p 4.0 any cpu Domain-0 0 2 - --p 5.6 any cpu Domain-0 0 3 - --p 4.5 any cpu Domain-0 0 4 - --p 8.4 any cpu Domain-0 0 5 - --p 6.1 any cpu Domain-0 0 6 - --p 5.0 any cpu Domain-0 0 7 - --p 4.5 any cpu Domain-0 0 8 8 -b- 30.3 any cpu Domain-0 0 9 9 -b- 15.0 any cpu Domain-0 0 10 10 r-- 29.3 any cpu Domain-0 0 11 11 -b- 15.2 any cpu Domain-0 0 12 12 -b- 72.7 any cpu Domain-0 0 13 13 -b- 177.5 any cpu Domain-0 0 14 14 -b- 14.5 any cpu Domain-0 0 15 15 -b- 22.0 any cpu brian carb unisys corporation - malvern, pa brian.carb@unisys.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I think it should too, but I ended up doing this to achieve what I needed. ( only 1 vcpu for dom0) - Padma - ________________________________ From: Subrahmanian, Raj [mailto:raj.subrahmanian@unisys.com] Sent: Thursday, June 08, 2006 2:16 PM To: Apparao, Padmashree K; Carb, Brian A; xen-devel@lists.xensource.com Cc: McAfee, Tommie M; Krysan, Susan; Puthiyaparambil, Aravindh; Vessey, Bruce A Subject: RE: [Xen-devel] Unable to set dom0 vcpu to 1 Padma, Shouldn''t the xm vcpu-set work regardless of the dom0-cpus value? Raj ________________________________ From: Apparao, Padmashree K [mailto:padmashree.k.apparao@intel.com] Sent: Thursday, June 08, 2006 5:03 PM To: Carb, Brian A; xen-devel@lists.xensource.com Cc: McAfee, Tommie M; Krysan, Susan; Puthiyaparambil, Aravindh; Subrahmanian, Raj; Vessey, Bruce A Subject: RE: [Xen-devel] Unable to set dom0 vcpu to 1 Set (dom-cpus=1) in etc/xen/xend-config.sxp. - Padma - ________________________________ From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Carb, Brian A Sent: Thursday, June 08, 2006 1:12 PM To: xen-devel@lists.xensource.com Cc: McAfee, Tommie M; Krysan,Susan; Puthiyaparambil,Aravindh; Subrahmanian,Raj; Vessey,Bruce A; Carb, Brian A Subject: [Xen-devel] Unable to set dom0 vcpu to 1 Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64, 16 procs, 16 GB RAM under SLES 10 RC1. At startup, dom0-cpus is set to 0 (the default) in the xend-config.sxp file, and dom0 uses all 16 cpus. ''xm top'' shows 16 procs, ''xm list'' shows dom0 using 16 cpus, and ''xm vcpu-list'' shows 16 cpus assigned to dom0. When we try to reduce the procs for dom0 to 1 using ''xm vcpu-set 0 1'', the number of procs, as shown by /proc/cpuinfo, reduces to 9. ''xm list'' shows dom0 using 9 procs, and ''xm vcpu-list'' shows that procs 1-7 have been released, while procs 0 and 8 through 15 are still assigned. Nothing is shown on the serial port. Can anyone shed some light on why the rest of the procs might not be released? Thanks. The following is output from ''xm list'' and ''xm vcpu-list'': m0075:~ # xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 15912 16 r----- 849.3 m0075:~ # xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 438.1 any cpu Domain-0 0 1 1 -b- 4.0 any cpu Domain-0 0 2 2 -b- 5.4 any cpu Domain-0 0 3 3 -b- 4.5 any cpu Domain-0 0 4 4 -b- 8.4 any cpu Domain-0 0 5 5 -b- 6.1 any cpu Domain-0 0 6 6 -b- 5.0 any cpu Domain-0 0 7 7 -b- 4.5 any cpu Domain-0 0 8 8 -b- 30.3 any cpu Domain-0 0 9 9 -b- 14.6 any cpu Domain-0 0 10 10 r-- 29.1 any cpu Domain-0 0 11 11 -b- 15.1 any cpu Domain-0 0 12 12 -b- 71.6 any cpu Domain-0 0 13 13 -b- 177.0 any cpu Domain-0 0 14 14 -b- 14.5 any cpu Domain-0 0 15 15 -b- 21.9 any cpu m0075:~ # xm vcpu-set 0 1 m0075:~ # xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 15912 9 r----- 853.2 m0075:~ # xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 439.3 any cpu Domain-0 0 1 - --p 4.0 any cpu Domain-0 0 2 - --p 5.6 any cpu Domain-0 0 3 - --p 4.5 any cpu Domain-0 0 4 - --p 8.4 any cpu Domain-0 0 5 - --p 6.1 any cpu Domain-0 0 6 - --p 5.0 any cpu Domain-0 0 7 - --p 4.5 any cpu Domain-0 0 8 8 -b- 30.3 any cpu Domain-0 0 9 9 -b- 15.0 any cpu Domain-0 0 10 10 r-- 29.3 any cpu Domain-0 0 11 11 -b- 15.2 any cpu Domain-0 0 12 12 -b- 72.7 any cpu Domain-0 0 13 13 -b- 177.5 any cpu Domain-0 0 14 14 -b- 14.5 any cpu Domain-0 0 15 15 -b- 22.0 any cpu brian carb unisys corporation - malvern, pa brian.carb@unisys.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
* Carb, Brian A <Brian.Carb@unisys.com> [2006-06-08 15:13]:> > Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64, > > 16 procs, 16 GB RAM under SLES 10 RC1. > > > > At startup, dom0-cpus is set to 0 (the default) in the xend-config.sxp > > file, and dom0 uses all 16 cpus. ''xm top'' shows 16 procs, ''xm list'' > > shows dom0 using 16 cpus, and ''xm vcpu-list'' shows 16 cpus assigned to > > dom0. > > > > When we try to reduce the procs for dom0 to 1 using ''xm vcpu-set 0 1'', > > the number of procs, as shown by /proc/cpuinfo, reduces to 9. ''xm > > list'' shows dom0 using 9 procs, and ''xm vcpu-list'' shows that procs > > 1-7 have been released, while procs 0 and 8 through 15 are still > > assigned. Nothing is shown on the serial port. > > > > Can anyone shed some light on why the rest of the procs might not be > > released? Thanks.On large smp boxes, it takes some time to unplug all of the processors as any processes that are running on those cpus have to be migrated to another cpu. On a 32-way, it took 15 seconds to go from 32 to 1. As I ran xm list, you can see the number of active vcpus reduce over time. [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 21 r----- 4494.6 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 19 r----- 4495.4 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 17 r----- 4496.2 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 16 r----- 4496.8 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 16 r----- 4497.4 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 15 r----- 4497.6 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 14 r----- 4498.1 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 13 r----- 4498.6 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 12 r----- 4499.2 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 12 r----- 4499.4 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 10 r----- 4500.3 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 10 r----- 4500.7 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 9 r----- 4501.0 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 8 r----- 4501.5 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 7 r----- 4502.0 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 6 r----- 4502.5 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 4 r----- 4503.4 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 1 r----- 4504.4 -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx (512) 838-9253 T/L: 678-9253 ryanh@us.ibm.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Thanks Ryan. We''ve experienced the wait as well (we contributed the timeout loop in changeset 10043 for xm-test 01_enforce_dom0_cpus_basic_pos). For some reason, though, vcpu-set is not releasing procs above #7, even when we wait a long time. brian carb unisys corporation - malvern, pa brian.carb@unisys.com -----Original Message----- From: Ryan Harper [mailto:ryanh@us.ibm.com] Sent: Monday, June 12, 2006 11:12 AM To: Carb, Brian A Cc: xen-devel@lists.xensource.com; McAfee, Tommie M; Krysan, Susan; Puthiyaparambil, Aravindh; Subrahmanian, Raj; Vessey, Bruce A Subject: Re: [Xen-devel] Unable to set dom0 vcpu to 1 * Carb, Brian A <Brian.Carb@unisys.com> [2006-06-08 15:13]:> > Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64, > > 16 procs, 16 GB RAM under SLES 10 RC1. > > > > At startup, dom0-cpus is set to 0 (the default) in the > > xend-config.sxp file, and dom0 uses all 16 cpus. ''xm top'' shows 16procs, ''xm list''> > shows dom0 using 16 cpus, and ''xm vcpu-list'' shows 16 cpus assigned > > to dom0. > > > > When we try to reduce the procs for dom0 to 1 using ''xm vcpu-set 0 > > 1'', the number of procs, as shown by /proc/cpuinfo, reduces to 9. > > ''xm list'' shows dom0 using 9 procs, and ''xm vcpu-list'' shows that > > procs > > 1-7 have been released, while procs 0 and 8 through 15 are still > > assigned. Nothing is shown on the serial port. > > > > Can anyone shed some light on why the rest of the procs might not be> > released? Thanks.On large smp boxes, it takes some time to unplug all of the processors as any processes that are running on those cpus have to be migrated to another cpu. On a 32-way, it took 15 seconds to go from 32 to 1. As I ran xm list, you can see the number of active vcpus reduce over time. [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 21 r----- 4494.6 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 19 r----- 4495.4 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 17 r----- 4496.2 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 16 r----- 4496.8 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 16 r----- 4497.4 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 15 r----- 4497.6 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 14 r----- 4498.1 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 13 r----- 4498.6 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 12 r----- 4499.2 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 12 r----- 4499.4 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 10 r----- 4500.3 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 10 r----- 4500.7 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 9 r----- 4501.0 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 8 r----- 4501.5 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 7 r----- 4502.0 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 6 r----- 4502.5 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 4 r----- 4503.4 [root@x460-3 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 507 1 r----- 4504.4 -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx (512) 838-9253 T/L: 678-9253 ryanh@us.ibm.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
* Carb, Brian A <Brian.Carb@unisys.com> [2006-06-12 10:44]:> Thanks Ryan. We''ve experienced the wait as well (we contributed the > timeout loop in changeset 10043 for xm-test > 01_enforce_dom0_cpus_basic_pos). For some reason, though, vcpu-set is > not releasing procs above #7, even when we wait a long time.There was a recent patch dealing with hotplug cpu and rcu sent to the list by Harry Butterworth. I think it has been committed yet but you will probably want to try again with that patch. -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx (512) 838-9253 T/L: 678-9253 ryanh@us.ibm.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Actually, in our machine, no matter what I write in /etc/xen/xend-config.sxp, dom0 alaways uses the 32 cpus. On 6/8/06, Apparao, Padmashree K <padmashree.k.apparao@intel.com> wrote:> > > > > I think it should too, but I ended up doing this to achieve what I needed. > ( only 1 vcpu for dom0) > > > > - Padma > > - > > > > ________________________________ > > > From: Subrahmanian, Raj [mailto:raj.subrahmanian@unisys.com] > Sent: Thursday, June 08, 2006 2:16 PM > To: Apparao, Padmashree K; Carb, Brian A; xen-devel@lists.xensource.com > Cc: McAfee, Tommie M; Krysan, Susan; Puthiyaparambil, Aravindh; Vessey, > Bruce A > > Subject: RE: [Xen-devel] Unable to set dom0 vcpu to 1 > > > > > > > > Padma, > > Shouldn''t the xm vcpu-set work regardless of the dom0-cpus value? > > Raj > > > ________________________________ > > > From: Apparao, Padmashree K [mailto:padmashree.k.apparao@intel.com] > Sent: Thursday, June 08, 2006 5:03 PM > To: Carb, Brian A; xen-devel@lists.xensource.com > Cc: McAfee, Tommie M; Krysan, Susan; Puthiyaparambil, Aravindh; > Subrahmanian, Raj; Vessey, Bruce A > Subject: RE: [Xen-devel] Unable to set dom0 vcpu to 1 > > Set (dom-cpus=1) in etc/xen/xend-config.sxp. > > > > - Padma > > - > > > > > > ________________________________ > > > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > Carb, Brian A > Sent: Thursday, June 08, 2006 1:12 PM > To: xen-devel@lists.xensource.com > Cc: McAfee, Tommie M; Krysan,Susan; Puthiyaparambil,Aravindh; > Subrahmanian,Raj; Vessey,Bruce A; Carb, Brian A > Subject: [Xen-devel] Unable to set dom0 vcpu to 1 > > > > Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64, 16 > procs, 16 GB RAM under SLES 10 RC1. > > At startup, dom0-cpus is set to 0 (the default) in the xend-config.sxp file, > and dom0 uses all 16 cpus. ''xm top'' shows 16 procs, ''xm list'' shows dom0 > using 16 cpus, and ''xm vcpu-list'' shows 16 cpus assigned to dom0. > > When we try to reduce the procs for dom0 to 1 using ''xm vcpu-set 0 1'', the > number of procs, as shown by /proc/cpuinfo, reduces to 9. ''xm list'' shows > dom0 using 9 procs, and ''xm vcpu-list'' shows that procs 1-7 have been > released, while procs 0 and 8 through 15 are still assigned. Nothing is > shown on the serial port. > > Can anyone shed some light on why the rest of the procs might not be > released? Thanks. > > The following is output from ''xm list'' and ''xm vcpu-list'': > > m0075:~ # xm list > Name ID Mem(MiB) VCPUs State > Time(s) > Domain-0 0 15912 16 r----- > 849.3 > m0075:~ # xm vcpu-list > Name ID VCPU CPU State > Time(s) CPU Affinity > Domain-0 0 0 0 -b- > 438.1 any cpu > Domain-0 0 1 1 -b- > 4.0 any cpu > Domain-0 0 2 2 -b- > 5.4 any cpu > Domain-0 0 3 3 -b- > 4.5 any cpu > Domain-0 0 4 4 -b- > 8.4 any cpu > Domain-0 0 5 5 -b- > 6.1 any cpu > Domain-0 0 6 6 -b- > 5.0 any cpu > Domain-0 0 7 7 -b- > 4.5 any cpu > Domain-0 0 8 8 -b- > 30.3 any cpu > Domain-0 0 9 9 -b- > 14.6 any cpu > Domain-0 0 10 10 r-- > 29.1 any cpu > Domain-0 0 11 11 -b- > 15.1 any cpu > Domain-0 0 12 12 -b- > 71.6 any cpu > Domain-0 0 13 13 -b- > 177.0 any cpu > Domain-0 0 14 14 -b- > 14.5 any cpu > Domain-0 0 15 15 -b- > 21.9 any cpu > > m0075:~ # xm vcpu-set 0 1 > m0075:~ # xm list > Name ID Mem(MiB) VCPUs State > Time(s) > Domain-0 0 15912 9 r----- > 853.2 > m0075:~ # xm vcpu-list > Name ID VCPU CPU State > Time(s) CPU Affinity > Domain-0 0 0 0 -b- > 439.3 any cpu > Domain-0 0 1 - --p > 4.0 any cpu > Domain-0 0 2 - --p > 5.6 any cpu > Domain-0 0 3 - --p > 4.5 any cpu > Domain-0 0 4 - --p > 8.4 any cpu > Domain-0 0 5 - --p > 6.1 any cpu > Domain-0 0 6 - --p > 5.0 any cpu > Domain-0 0 7 - --p > 4.5 any cpu > Domain-0 0 8 8 -b- > 30.3 any cpu > Domain-0 0 9 9 -b- > 15.0 any cpu > Domain-0 0 10 10 r-- > 29.3 any cpu > Domain-0 0 11 11 -b- > 15.2 any cpu > Domain-0 0 12 12 -b- > 72.7 any cpu > Domain-0 0 13 13 -b- > 177.5 any cpu > Domain-0 0 14 14 -b- > 14.5 any cpu > Domain-0 0 15 15 -b- > 22.0 any cpu > > brian carb > unisys corporation - malvern, pa > brian.carb@unisys.com > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Thanks Ryan,
We''re still seeing this problem as of changeset 10333 (which I believe
includes that patch).
It seems as if the "vcpus" value as shown by "xm list --long
0" command
is getting set to 8, even when we boot dom0 with 16procs. In fact, vcpus
is still set to 8 even if we boot hyperthreaded to a 32x. Since the
release of procs with vcpu-set seems to key off this value, we can never
release more than 7.
--------------------------
m0075:~ # xm list --long 0
(domain
(domid 0)
(uuid 00000000-0000-0000-0000-000000000000)
(vcpus 8)
(cpu_weight 1.0)
(memory 2728)
(maxmem 15016)
(features )
(name Domain-0)
(on_poweroff destroy)
(on_reboot restart)
(on_crash restart)
(state r-----)
(shutdown_reason poweroff)
(cpu_time 1254.97341823)
(online_vcpus 16)
)
brian carb
unisys corporation - malvern, pa
brian.carb@unisys.com
-----Original Message-----
From: Ryan Harper [mailto:ryanh@us.ibm.com]
Sent: Monday, June 12, 2006 11:50 AM
To: Carb, Brian A
Cc: xen-devel@lists.xensource.com; McAfee, Tommie M; Krysan, Susan;
Puthiyaparambil, Aravindh; Subrahmanian, Raj; Vessey, Bruce A
Subject: Re: [Xen-devel] Unable to set dom0 vcpu to 1
* Carb, Brian A <Brian.Carb@unisys.com> [2006-06-12
10:44]:> Thanks Ryan. We''ve experienced the wait as well (we contributed
the
> timeout loop in changeset 10043 for xm-test
> 01_enforce_dom0_cpus_basic_pos). For some reason, though, vcpu-set is
> not releasing procs above #7, even when we wait a long time.
There was a recent patch dealing with hotplug cpu and rcu sent to the
list by Harry Butterworth. I think it has been committed yet but you
will probably want to try again with that patch.
--
Ryan Harper
Software Engineer; Linux Technology Center IBM Corp., Austin, Tx
(512) 838-9253 T/L: 678-9253
ryanh@us.ibm.com
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
* Carb, Brian A <Brian.Carb@unisys.com> [2006-06-15 10:12]:> Thanks Ryan, > > We''re still seeing this problem as of changeset 10333 (which I believe > includes that patch).Yeah, that was a long shot.> > It seems as if the "vcpus" value as shown by "xm list --long 0" command > is getting set to 8, even when we boot dom0 with 16procs. In fact, vcpus > is still set to 8 even if we boot hyperthreaded to a 32x. Since the > release of procs with vcpu-set seems to key off this value, we can never > release more than 7.the vcpus value for dom0 should be the number of vcpus the domain is configured to use; usually in the conf file. Domain0 is different since it has no conf file being built by xen instead of Xend. In this case, vcpus value is derived from max_vcpu_id, which is the highest vcpu that as been initialized. For dom0, this value should be the number of cpus xen enabled during boot. vcpu-set uses the vcpu_avail variable which is initialized from the vcpus variable, which, as mentioned able is determined by max_vcpu_id. e.g. max_vcpu_id=7, vcpus = 8, vcpu_avail = (1 << 8)-1, or 0xff I''d start fresh by clearing out xenstore: rm /var/lib/xenstored/tdb And rebooting. It should never be the case where vcpus for dom0 is < online vcpus as that means you are running vcpus that haven''t been initialized (max_vcpu_id indicates the max vcpu that has been allocated to run in the domain). It would be interesting if you can track what steps you took to get the machine/xenstore into this state and file a bug. -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx (512) 838-9253 T/L: 678-9253 ryanh@us.ibm.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel