Xen devel - Apr 2006 - Odd behaviour of Xen guests on multiprocessor systems ?

I noticed an odd thing recenlty when running some experiments with several
xen guests on a multiprocessor host:-

I was running some tests with infinite while loops inside the guests and
used xentop on the host to see what was happening.
Firstly, xentop doesn''t give per-physical-cpu information for
multiprocessors, it gives VCPU information (have a separate posted query on
this).
During the tests I noticed that in some cases I, when I started the infinite
loops inside the guests, 2 guests appeared to be on the same
CPU - I inferred this because xentop would show 49.9% VCPU usage for each of
the 2 guests. In other cases, the 2 guests appeared to be on different
CPUs because xentop would show 99.9% VCPU usage for each. This state would
remain the same for however long I ran the experiment.

Can anyone explain why this is the case ? Once a guest VM is started on a
multiprocessor system, does it always stick to one particular physical
CPU ? If so why and doesn''t this have some serious implications for
provisioning for mulitple colocated VMs ?

Thanks and regards
Arjun


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

* Arjun <cse.syslab@gmail.com> [2006-04-25 19:57]:
> I noticed an odd thing recenlty when running some experiments with several
> xen guests on a multiprocessor host:-
> 
> I was running some tests with infinite while loops inside the guests and
> used xentop on the host to see what was happening.
> Firstly, xentop doesn''t give per-physical-cpu information for
> multiprocessors, it gives VCPU information (have a separate posted query on
> this).
> During the tests I noticed that in some cases I, when I started the
infinite
> loops inside the guests, 2 guests appeared to be on the same
> CPU - I inferred this because xentop would show 49.9% VCPU usage for each
of
> the 2 guests. In other cases, the 2 guests appeared to be on different
> CPUs because xentop would show 99.9% VCPU usage for each. This state would
> remain the same for however long I ran the experiment.
You should run xm vcpu-list.  This will give you a vcpu to cpu mapping
for each domain running.

[root@x460-3 ~]# xm list dom1
Name                              ID Mem(MiB) VCPUs State  Time(s)
dom1                               1      768     2 -b----     9.8

Two vcpu domain running..

[root@x460-3 ~]# xm vcpu-list dom1
Name                              ID  VCPU  CPU  State  Time(s)  CPU Affinity
dom1                               1     0    2   -b-       3.3  2
dom1                               1     1    3   -b-       3.0  3

I see that dom1 is using cpus 2 and 3.
> 
> Can anyone explain why this is the case ? Once a guest VM is started on a
> multiprocessor system, does it always stick to one particular physical
Xen does not migrate VCPUs between physical cpus automatically.  One can
force a migration via the xm vcpu-pin command.

[root@x460-3 ~]# xm vcpu-pin dom1 1 6
[root@x460-3 ~]# xm vcpu-list dom1
Name                              ID  VCPU  CPU  State  Time(s)  CPU Affinity
dom1                               1     0    2   -b-       8.4  2
dom1                               1     1    6   -b-       4.8  6
> CPU ? If so why and doesn''t this have some serious implications
for
> provisioning for mulitple colocated VMs ?
When a domain is created, Xen chooses processors with the fewest number
of VCPUs present.  There is certainly room for a user-space load
balancer which can migrate vcpus around according to load.


-- 
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
ryanh@us.ibm.com

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel