Xen devel - May 2006 - RE: Setting dom0-cpus to 0 doesn''t use all cpus asdocumented

We discovered this when running the xm-test suite. One of the tests
(01_enforce_dom0_cpus) changes the number of cpus by editing the
xend-config.sxp file and restarting xend. Should this test be updated to
use the method Ian suggested, to vary the number of cpus, rather than
restarting xend?

brian carb (610-648-2437 or N2 385-2437)
unisys cmp technology lab
malvern, pa
brian.carb@unisys.com

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is thus for use only by the intended recipient. If you
received this in error, please contact the sender and delete the e-mail
and its attachments from all computers. 

-----Original Message-----
From: Ryan Harper [mailto:ryanh@us.ibm.com] 
Sent: Tuesday, May 09, 2006 11:45 AM
To: Ian Pratt
Cc: Carb, Brian A; Xen-devel@lists.xensource.com; Subrahmanian, Raj;
Krysan, Susan; Puthiyaparambil, Aravindh; Vessey, Bruce A
Subject: Re: [Xen-devel] Setting dom0-cpus to 0 doesn''t use all cpus
asdocumented

* Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk> [2006-05-09
10:07]:
> > The comments for dom0-cpus in the xend-config.sxp file indicate that
> > setting this value to 0 in an SMP system will cause dom0 to use all 
> > the available CPUs. However, it appears as if setting this value to 
> > 0 does not change the number of CPUs at all.
> > 
> > I''m running SLES10 Beta 11 and xen-unstable changeset 9960 on
a
> > Unisys ES7000 with 32 CPUs and 32GB memory. In xend-config.sxp, 
> > dom0-cpus is set to 0. When xen dom0 boots, it is using all 32 procs
> > (as reported by ''grep processor /proc/cpuinfo | wc
-l'').
> > 
> > If I edit the xend-config.sxp file, change the value of dom0-cpus to
> > 1, and restart xend, dom0 correctly uses 1 CPUs.
> > If I then change dom0-cpus to 0 and restart xend, dom0 still uses 1 
> > cpu. The only way to have dom0 use 32 cpus again is to either set 
> > the value of dom0-cpus to 32, or reboot xen.
> > 
> > Has anyone else seen this behavior?
> 
> You can just use the normal xm commands to vary the number of CPUs.
> Restarting xend is a rather brutal way of trying to achieve the same 
> thing...
Is it worth keeping dom0-cpus then?


--
Ryan Harper
Software Engineer; Linux Technology Center IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
ryanh@us.ibm.com

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

* Carb, Brian A <Brian.Carb@unisys.com> [2006-05-09
10:58]:
> We discovered this when running the xm-test suite. One of the tests
> (01_enforce_dom0_cpus) changes the number of cpus by editing the
> xend-config.sxp file and restarting xend. Should this test be updated to
> use the method Ian suggested, to vary the number of cpus, rather than
> restarting xend?
As Ian said, it is for setting the value when xend starts which is what
the test is about.  However, the test should bring dom0 vcpus to the
max and then test enforcement down to 1 rather than assuming that
dom0''s
current value can be reduced.  Or, bail on the test if the current value
of dom0''s online vcpus isn''t > 2.

-- 
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
ryanh@us.ibm.com

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

I chose to update the test to SKIP if dom0 isn''t proper configured
which at this point means SMP host and dom0 with at least 2 VCPUs
online.  Also I noticed that the test was still using vcpu-enable which
is no longer available via xm; we use vcpu-set.  This patch fixes the
issue where after successfully running the test, dom0''s vcpu state was
not restored.  Updated some comments and lines that exceeded 80 columns.

-- 
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
ryanh@us.ibm.com


diffstat output:
 01_enforce_dom0_cpus_basic_pos.py |   55 ++++++++++++++++++++++++--------------
 1 files changed, 36 insertions(+), 19 deletions(-)

Signed-off-by: Ryan Harper <ryanh@us.ibm.com>
---
# HG changeset patch
# User Ryan Harper <ryanh@us.ibm.com>
# Node ID 279b674363bc78a284284bd62b9a0e2b5c65acb6
# Parent  1e3977e029fddc1e53995a3502d590f8a07e5c62
Fix up enforce_dom0_cpu test case to check that dom0 has at least 2 vcpus
online.  Also fix bogus call to vcpu-enable; xm now uses vcpu-set as method for
enable/disable of vcpus.  Previously after a successful test run, this test
would not leave dom0 in the same state as it was when the test started.  Also
some comment language changes and 80 column formatting.

diff -r 1e3977e029fd -r 279b674363bc
tools/xm-test/tests/enforce_dom0_cpus/01_enforce_dom0_cpus_basic_pos.py
--- a/tools/xm-test/tests/enforce_dom0_cpus/01_enforce_dom0_cpus_basic_pos.py
Mon May  8 18:21:41 2006
+++ b/tools/xm-test/tests/enforce_dom0_cpus/01_enforce_dom0_cpus_basic_pos.py
Tue May  9 19:19:47 2006
@@ -4,20 +4,23 @@
 # Authors: Dan Smith <danms@us.ibm.com>
 #          Ryan Harper <ryanh@us.ibm.com>
 
-# 1) Make sure we have a multi cpu system
+# 1) Make sure we have a multi cpu system and dom0 has at 
+#    least 2 vcpus online.
 # 2) clone standard config (/etc/xen/xend-config.sxp) 
 # 3) modify clone with enforce_dom0_cpus=X
 # 4) restart xend with modified config
 # 5) check /proc/cpuinfo for cpu count
-# 6) check xm list -v to see that only 1 cpu is online for dom0
-# 7) Restart xend with default config
+# 6) check xm info ''VCPUs'' field to see that only
''enforce_dom0_cpus''
+#    number of cpus are online in dom0
+# 7) Restore initial dom0 vcpu state
+# 8) Restart xend with default config
 
 import sys
 import re
 import time
 import os
 
-# what value should dom0_cpus be enforced?
+# what value should dom0_cpus enforce?
 enforce_dom0_cpus=1
 
 from XmTestLib import *
@@ -25,12 +28,19 @@
 check_status = 1
 max_tries = 10
 
-# 1) Make sure we have a multi cpu system
+# 1) Make sure we have a multi cpu system and dom0 has at least 2 vcpus online.
 
 if smpConcurrencyLevel() <= 1:
     print "*** NOTE: This machine does not have more than one
physical"
     print "          or logical cpu.  The vcpu-disable test cannot be
run!"
     SKIP("Host not capable of running test")
+
+# count number of online vcpus in dom0
+dom0_online_vcpus = int(getDomInfo("Domain-0", "VCPUs"))
+if dom0_online_vcpus <= 1:
+    print "*** NOTE: DOM0 needs at least 2 VCPUs online to run this
test"
+    print "          Please enable additional vcpus if possible via xm
vcpu-set"
+    SKIP("Host state not capable of running test")
     
 # 2) clone standard config (/etc/xen/xend-config.sxp) 
 # 3) modify clone with enforce_dom0_cpus=1
@@ -65,26 +75,33 @@
 if output != str(enforce_dom0_cpus):
     os.unsetenv("XEND_CONFIG")
     restartXend()
-    FAIL("/proc/cpuinfo says xend didn''t enforce dom0_cpus (%s !=
%s)" %(output, enforce_dom0_cpus))
+    FAIL("/proc/cpuinfo says xend didn''t enforce dom0_cpus (%s !=
%s)"%(output,
+                                                            
enforce_dom0_cpus))
 
-# 7) count number of online cpus and see that it matches enforce value
-dom0vcpus = getVcpuInfo("Domain-0")
-num_online = len(filter(lambda x: x >= 0, dom0vcpus.values()))
+# 6) count number of online cpus and see that it matches enforce value
+num_online = int(getDomInfo("Domain-0", "VCPUs"))
 if num_online != enforce_dom0_cpus:
     os.unsetenv("XEND_CONFIG")
     restartXend()
-    FAIL("xm says xend didn''t enforce dom0_cpus (%s != %s)"
%(num_online, enforce_dom0_cpus))
+    FAIL("xm says xend didn''t enforce dom0_cpus (%s != %s)"
%(num_online,
+                                                            
enforce_dom0_cpus))
 
-# restore dead processors 
-for (k,v) in zip(dom0vcpus.keys(),dom0vcpus.values()):
-    if v == -1:
-        status, output = traceCommand("xm vcpu-enable 0 %s"%(k))
-        if check_status and status != 0:
-            os.unsetenv("XEND_CONFIG")
-            restartXend()
-            FAIL("\"%s\" returned invalid %i != 0"
%(cmd,status))
+# 7) restore dead processors 
+status, output = traceCommand("xm vcpu-set 0 %s"%(dom0_online_vcpus))
+if check_status and status != 0:
+    os.unsetenv("XEND_CONFIG")
+    restartXend()
+    FAIL("\"%s\" returned invalid %i != 0" %(cmd,status))
 
-# Restart xend with default config
+# check restore worked
+num_online = int(getDomInfo("Domain-0", "VCPUs"))
+if num_online != dom0_online_vcpus:
+    os.unsetenv("XEND_CONFIG")
+    restartXend()
+    FAIL("failed to restore dom0''s VCPUs")
+
+
+# 8) Restart xend with default config
 os.unsetenv("XEND_CONFIG")
 restartXend()
 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, May 09, 2006 at 02:43:04PM -0500, Ryan Harper wrote:
> I chose to update the test to SKIP if dom0 isn''t proper configured
> which at this point means SMP host and dom0 with at least 2 VCPUs
> online.  Also I noticed that the test was still using vcpu-enable which
> is no longer available via xm; we use vcpu-set.  This patch fixes the
> issue where after successfully running the test, dom0''s vcpu state
was
> not restored.  Updated some comments and lines that exceeded 80 columns.
Applied, thanks.

Ewan.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel