Xen users - Jan 2006 - CPU resource settings: allocation / repartition / scheduler

Hi,

We have an IBM xSeries 235 with 2 CPUs Xeon 3.0 Ghz + RAM 2.5 GB +
Hyperthreading enabled.
We have a Xen 3.0.0 installed + kernal-2.6.12.6 + Debian Sarge 3.1r1

We''d like to go deeply into the Xen CPU resource allocation and
repartition,
but we meet some issues and troubles.
I''m a bit confused because I don''t know whether I really
misuse /
misconfigure the system or are there some (known) bugs in that version ?

If there are some bugs, consider the following lines as feedback, or if
I''m
wrong with using and configuration, consider them as questions for more
explanation ! ;-)


- CPU allocation:
-----------------
. Unstability with "maxcpus=X" argument in my /boot/grub.menus.lst
  [reminder: 2 CPUS + Hyperthread = 4 CPUs]
  - if I remove that argument from the grub config file
    "xm list" --> Domain-0 will use 4 CPUs (maximum)
  - if I specify "maxcpus=N" in the grub config file (with N < MAX)
    "xm list" --> Domain-0 will use N CPUs
  - if I specify "maxcpus=4" in the grub config file (MAX = 4)
    "xm list" --> Domain-0 will use 4 CPUs (maximum)
  - if I specify "maxcpus=X" in the grub config file (with X > MAX)
    "xm list" --> Domain-0 will use 4 CPUs (maximum)

  Now the problem seems that we meet unstability issues !
  --> if Domain-0 uses the maximum number of CPU''s from my host
      "xm list" --> DomU machines seen as running but they
don''t
      fully boot ! (they hang during their filesystems mounting step...
      ... tt''s NOT a kernel panic, but just hanging !)
  --> is it normal ? is it a bug ? did I miss something in config ?

. "dom0-cpus" / "dom0-num-cpus" arguments in
/etc/xen/xend-config.sxp
  I really wonder what is the role and influence of those arguments.
  very confusing...

  --> is it "dom0-cpus" or "dom0-num-cpus" I must use in
Xen 3.0.0 ?
  --> is it related or independant of my "maxcpus" argument ?

. "xm vcpu-set Domain-0 X" command line doesn''t work.
  Ex: if "maxcpus=3", whatever the value I assign to X in my
      "xm vcpu-set Domain-0 X" command line, "xm list"
always
      sees Domain-0 with 3 running CPUs.
  --> is it normal ?
  --> I understand I can''t assign 4 CPUs to Domain-0 if maxcpus=3,
      but isn''t it possible to dynamically assign from 1 up to
"maxcpus"
      running CPUs to my Domain-0 ?

. "vcpus=N" argument in my domU config file
  I''ve had a strange surprise.
  I thought I could assign up to 4 (MAX) VCPUs for domU machines.
  Strangely, N can have a value up to 8 VCPUs (2 * MAX) !!!
  I have 2 VCPUs assigned to each CPU.
  --> could you please give me some explanation ?
  I thought it was due to "SMT enabled" value in my kernel-U config,
  but no way... whatever SMT enabled or disabled, I still have the
  possibility to assign 8 VCPUs !?!

. "xm vcpu-set Domain-U X" command line works fine
  Is it normal that I can''t assign a value to X, bigger
  than "vcpus" parameter value in my domU config file ?

. default value for vcpu-set dom0 / domU
  is it possible to set such parameters in dom0 / domU config files ?
  ie. maxcpus can be set to 3 but for some reasons we want it to
      boot with 2 VCPUs only ?


- CPU repartition:
------------------
. "xm vcpu-pin Domain-0 X Y" command line
  If I have a "maxcpus" value set between > 1 and < 4,
  the "xm vcpu-pin Domain-0 X Y" command line behaviour seems
  to be very strange. Sometimes it works, sometimes it bugs.
  Sometimes I get the "Error: (22, ''Invalid
argument'')" and
  the command doesn''t affect any vcpu-pin.
  Here is what I noticed:
  - The command is successful when the VCPU has the "b" state
  - An error occurs when the VCPU has the "r" state.
  I can undersand that behaviour, but:
  --> How to avoid that kind of error ?
      Is it possible to switch a VCPU from "r" to "b" state,
      in order to assign pins on demand ?
  --> Your experiences ? recommendations ? best practices ?

. "xm vcpu-pin Domain-U X Y" command line
  Same behaviour with "vcpus" argument + domU machines.

. default value for vcpu-pin for dom0 / domU:
  is it possible to set such parameters in dom0 / domU config,
  so that it''s set at boot time and we avoid error due to a
  VCPU "r" state ?



- CPU scheduler:
----------------
. Documentation for "sched-bvt" and "sched-sedf" are very
poor,
  I haven''t found detailed info for arguments and settings.
  --> In some docs I''ve read the "sched-bvt" is the
default,
      and in other docs I''ve read "sched-sedf".
      Which one is the default in Xen 3.0.0 ?
  --> "xm help --long" command line says those command need 6
arguments.
      Ok... but please do you have some more info for those arguments ?
      - role and meanings ? values to assign ?
      - your experiences and best practices ?

. What about "atropos" ? "round-robin" ? other algorithms ?
  Is it still adviced not to use it ?

. We''ve read about a scheduler called CKRM... is there any possible
  interfacing with Xen ?



And at last, a question for the Xen team.
As you can see in my list of questions, we would be eventually interested in
the Xen Optimizer.
Is there any unstable version we could use ?
Who can I contact @ Xen to have such information and follow Xen Optimizer
evolution ?



Thank you,
Samuel



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Samuel Torton <samuel <at> ncslab.com> writes:
> And at last, a question for the Xen team.
> As you can see in my list of questions, we would be eventually interested
in
> the Xen Optimizer.
> Is there any unstable version we could use ?
> Who can I contact  <at>  Xen to have such information and follow Xen
Optimizer
> evolution ?
> 
> Thank you,
> Samuel
> 
I am also very interested in xen3 cpu (and everything else) scheduling. The
documentation is indeed very limited. I´d also be interested in information on
XenOptimizer but it seems to be justnot ready???

During the last weeks I got quite unsure whether/wich xen is ready for
production
usage. Especially scheduling in 2.0 seems to be not DoS aware and the provided
xm tools arent enough for day-to-day production work. perhaps XenOptimizer can
change this...

dirk


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users