Xen devel - Nov 2005 - multi-processor odd behavior?

Hey folks,

I''m using a recent (dated last weekend) xen-unstable build on a machine
with two processors.  (Well, one processor with hyperthreading turned on.)

I''ve created two guest domains.  Inside each domain, I have a simple 
application ("eat") that''s a simple "for(;;);"
loop, to consume as much
CPU as possible.

I''m seeing odd behavior:

1. When I run "eat" inside one guest, "xm top" reports for
CPU(%):

  Domain-0:  0.5
   guest-1: 99.6
   guest-2:  0.0

2. When I run "eat" inside both guests:

  Domain-0:  0.5
   guest-1: 49.9
   guest-2: 49.9

  [This isn''t what I expect, which is for both domains to run near
100%.]

3. When I run "eat" inside both guests *and* in Domain-0:

  Domain-0: 97.6
   guest-1: 49.9
   guest-2: 49.9

This seems to indicate that one CPU is being reserved entirely for dom0, 
even when dom0 isn''t using it to capacity?  (That doesn''t seem
like the
intended behavior.)  Do I have a configuration option set wrong?  Or is 
this a known issue?  Or, is it fixed in the latest builds?

"xm top" shows "CPUs: 2 @ 2992MHz", and also indicates that
Domain-0 has 2
VCPUS while each of the guests has 1 VCPU.

Thanks,
JLG


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 18 Nov 2005, at 16:32, John L Griffin wrote:
> This seems to indicate that one CPU is being reserved entirely for 
> dom0,
> even when dom0 isn''t using it to capacity?  (That doesn''t
seem like the
> intended behavior.)  Do I have a configuration option set wrong?  Or is
> this a known issue?  Or, is it fixed in the latest builds?
The tools statically round-robin allocate vcpus to physical cpus as 
they are created, but there is currently no dynamic load balancing. So 
if your two guests happen to get allocated to the same pcpu, they will 
each get half a cpu even if other physical cpus are idling.

We have a prototype load balancer that we may check in after cleaning 
up and testing. An alternative, or perhaps complementary, approach is 
to implement ''take'' scheduling (idle cpus steal work from
loaded cpus).

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi John,

In addition to the default round robin allocation, we have special 
behaviour for HT. This behaviour is actually by design; from 
xen/common/dom0_ops.c:

----- 200 * If we''re on a HT system, we only use the first HT for dom0,
other 201 * domains will all share the second HT of each CPU. Since dom0 is 
on 202 * CPU 0, we favour high numbered CPUs in the event of a tie. 203 */ 
204 pro = smp_num_siblings - 1; 205 for ( i = pro; i < num_online_cpus(); i 
+= smp_num_siblings ) 206 if ( cnt[i] <= cnt[pro] ) 207 pro = i; -----

This is because you get IO performance gains (particularly for demanding 
loads, like small packets on a GigE network) by having dom0 run *truly* 
concurrently with the domUs it is servicing. This is because it saves 
context switching overhead.

The domUs are therefore running on the second HT. The CPU''s HT 
implementation should ensure that if dom0 is idle, the second HT will get 
all the raw horsepower of the processor.

If you really want to run domUs on both threads, you can use the pincpu 
command to manually shuffle them around.

Cheers,
Mark

On Nov 18 2005, John L Griffin wrote:
>Hey folks,
>
>I''m using a recent (dated last weekend) xen-unstable build on a
machine
>with two processors.  (Well, one processor with hyperthreading turned on.)
>
>I''ve created two guest domains.  Inside each domain, I have a
simple
>application ("eat") that''s a simple "for(;;);"
loop, to consume as much
>CPU as possible.
>
>I''m seeing odd behavior:
>
>1. When I run "eat" inside one guest, "xm top" reports
for CPU(%):
>
>  Domain-0:  0.5
>   guest-1: 99.6
>   guest-2:  0.0
>
>2. When I run "eat" inside both guests:
>
>  Domain-0:  0.5
>   guest-1: 49.9
>   guest-2: 49.9
>
>  [This isn''t what I expect, which is for both domains to run near
100%.]
>
>3. When I run "eat" inside both guests *and* in Domain-0:
>
>  Domain-0: 97.6
>   guest-1: 49.9
>   guest-2: 49.9
>
>This seems to indicate that one CPU is being reserved entirely for dom0, 
>even when dom0 isn''t using it to capacity?  (That doesn''t
seem like the
>intended behavior.)  Do I have a configuration option set wrong?  Or is 
>this a known issue?  Or, is it fixed in the latest builds?
>
>"xm top" shows "CPUs: 2 @ 2992MHz", and also indicates
that Domain-0 has 2
>VCPUS while each of the guests has 1 VCPU.
>
>Thanks,
>JLG
>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel