Xen devel - Dec 2009 - domU config file options for scheduling

Hi,

This is addressed mainly to George, but maybe someone else want to work
on it too.

I found very inconvenient to have to do:

xm sched-credit -d xen08 -c 50

after a VM starts, and I think there should be some configuration values
to be added into the startup file. Something like:

shedcreditweight = 20
shedcreditcap = 50

would be really useful and save a load of scripting work for everyone
(avoiding shell scripts to run after a VM is started).

George, would you have time to implement this? Any chance to have this
added for the 4.0 release? That''d be really a GREAT addition, with I
believe, minimum effort.

Also, I had a chat with George about having a kind of "burst" feature
added to the credit scheduler. My idea is that, by default, someone
would setup a credit-burst-time, and a credit-burst-cap, to any VM that
has a a credit-cap set. That could be set this way in the domU config file:

shedcreditweight = 20
shedcreditcap = 50
shedbursttime = 5000 <--- 5 seconds max burst CPU time
shedburstcap = 80 <--- 80% CPU max during the burst
shedrecovertime = 2000 <--- 2 second recovery time

Whenever a VM goes over its shedcreditcap, the burst-cap would be used
for a time no longer than the credit-burst-time. When this time is over,
then shedcreditcap would be used, until the VM uses less than
shedcreditcap for a time longer than shedrecovertime.

I do believe that this kind of scheduling would be REALLY useful for
avoiding that a VM abuses the CPU usage. For people doing Xen VM hosting
business, or cloud computing, that would be just great. The values that
I wrote above would be a typical use.

I had a look myself in the scheduler code, and it''s quite not obvious
where to patch. I think I have understood the burncredit() system, but I
didn''t get where the values are read for the cap and all. Did I
understood well that Xen uses 30ms cycles for the credit scheduler? If
that is the case, then I believe that computing what cap values to use
depending on the burst parameters for a given 30ms scheduling cycle
wouldn''t be such a big overhead.

If nobody wants to implement this, can someone gives me some pointers in
the sched_credit.c and all the .h structures? If I do stupid trials and
errors in my implementation at first, because I''d be a beginner at
patching Xen, would you guys have enough time to explain, and excuse a
newbie? Would that kind of patch be accepted if proven to work?

Cheers,

Thomas Goirand

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

I agree that there should be a way to add scheduling configuration to
the config file.  The main wrinkle is that each scheduler takes its
own scheduling parameters, which generally can''t be translated for
other schedulers. It would have to be up to the person making the
config file to make sure that there were appropriate info for other
schedulers if they might be used.

Unfortunately this would mainly have to do with python, which I''m by
no means an expert at.  Would anyone who knows the python like to
volunteer to do it?  Otherwise, I''ll see what I can do; it
shouldn''t
be too difficult to hack something together.

 -George

On Thu, Dec 10, 2009 at 7:58 AM, Thomas Goirand <thomas@goirand.fr>
wrote:
> Hi,
>
> This is addressed mainly to George, but maybe someone else want to work
> on it too.
>
> I found very inconvenient to have to do:
>
> xm sched-credit -d xen08 -c 50
>
> after a VM starts, and I think there should be some configuration values
> to be added into the startup file. Something like:
>
> shedcreditweight = 20
> shedcreditcap = 50
>
> would be really useful and save a load of scripting work for everyone
> (avoiding shell scripts to run after a VM is started).
>
> George, would you have time to implement this? Any chance to have this
> added for the 4.0 release? That''d be really a GREAT addition, with
I
> believe, minimum effort.
>
> Also, I had a chat with George about having a kind of "burst"
feature
> added to the credit scheduler. My idea is that, by default, someone
> would setup a credit-burst-time, and a credit-burst-cap, to any VM that
> has a a credit-cap set. That could be set this way in the domU config file:
>
> shedcreditweight = 20
> shedcreditcap = 50
> shedbursttime = 5000 <--- 5 seconds max burst CPU time
> shedburstcap = 80 <--- 80% CPU max during the burst
> shedrecovertime = 2000 <--- 2 second recovery time
>
> Whenever a VM goes over its shedcreditcap, the burst-cap would be used
> for a time no longer than the credit-burst-time. When this time is over,
> then shedcreditcap would be used, until the VM uses less than
> shedcreditcap for a time longer than shedrecovertime.
>
> I do believe that this kind of scheduling would be REALLY useful for
> avoiding that a VM abuses the CPU usage. For people doing Xen VM hosting
> business, or cloud computing, that would be just great. The values that
> I wrote above would be a typical use.
>
> I had a look myself in the scheduler code, and it''s quite not
obvious
> where to patch. I think I have understood the burncredit() system, but I
> didn''t get where the values are read for the cap and all. Did I
> understood well that Xen uses 30ms cycles for the credit scheduler? If
> that is the case, then I believe that computing what cap values to use
> depending on the burst parameters for a given 30ms scheduling cycle
> wouldn''t be such a big overhead.
>
> If nobody wants to implement this, can someone gives me some pointers in
> the sched_credit.c and all the .h structures? If I do stupid trials and
> errors in my implementation at first, because I''d be a beginner at
> patching Xen, would you guys have enough time to explain, and excuse a
> newbie? Would that kind of patch be accepted if proven to work?
>
> Cheers,
>
> Thomas Goirand
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, 10 Dec 2009, George Dunlap wrote:
> I agree that there should be a way to add scheduling configuration to
> the config file.  The main wrinkle is that each scheduler takes its
> own scheduling parameters, which generally can''t be translated for
> other schedulers. It would have to be up to the person making the
> config file to make sure that there were appropriate info for other
> schedulers if they might be used.
> 
> Unfortunately this would mainly have to do with python, which I''m
by
> no means an expert at.  Would anyone who knows the python like to
> volunteer to do it?  Otherwise, I''ll see what I can do; it
shouldn''t
> be too difficult to hack something together.
> 
You can try C with libxenlight ;)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

As for ''cap'' and ''weight'', they are already
configurable.
Adding the following to configuration will help.

cpu_weight = 20
cpu_cap = 50

Thanks,
Kouya

George Dunlap writes:
> I agree that there should be a way to add scheduling configuration to
> the config file.  The main wrinkle is that each scheduler takes its
> own scheduling parameters, which generally can''t be translated for
> other schedulers. It would have to be up to the person making the
> config file to make sure that there were appropriate info for other
> schedulers if they might be used.
> 
> Unfortunately this would mainly have to do with python, which I''m
by
> no means an expert at.  Would anyone who knows the python like to
> volunteer to do it?  Otherwise, I''ll see what I can do; it
shouldn''t
> be too difficult to hack something together.
> 
>  -George
> 
> On Thu, Dec 10, 2009 at 7:58 AM, Thomas Goirand <thomas@goirand.fr>
wrote:
> > Hi,
> >
> > This is addressed mainly to George, but maybe someone else want to
work
> > on it too.
> >
> > I found very inconvenient to have to do:
> >
> > xm sched-credit -d xen08 -c 50
> >
> > after a VM starts, and I think there should be some configuration
values
> > to be added into the startup file. Something like:
> >
> > shedcreditweight = 20
> > shedcreditcap = 50
> >
> > would be really useful and save a load of scripting work for everyone
> > (avoiding shell scripts to run after a VM is started).
> >
> > George, would you have time to implement this? Any chance to have this
> > added for the 4.0 release? That''d be really a GREAT addition,
with I
> > believe, minimum effort.
> >
> > Also, I had a chat with George about having a kind of
"burst" feature
> > added to the credit scheduler. My idea is that, by default, someone
> > would setup a credit-burst-time, and a credit-burst-cap, to any VM
that
> > has a a credit-cap set. That could be set this way in the domU config
file:
> >
> > shedcreditweight = 20
> > shedcreditcap = 50
> > shedbursttime = 5000 <--- 5 seconds max burst CPU time
> > shedburstcap = 80 <--- 80% CPU max during the burst
> > shedrecovertime = 2000 <--- 2 second recovery time
> >
> > Whenever a VM goes over its shedcreditcap, the burst-cap would be used
> > for a time no longer than the credit-burst-time. When this time is
over,
> > then shedcreditcap would be used, until the VM uses less than
> > shedcreditcap for a time longer than shedrecovertime.
> >
> > I do believe that this kind of scheduling would be REALLY useful for
> > avoiding that a VM abuses the CPU usage. For people doing Xen VM
hosting
> > business, or cloud computing, that would be just great. The values
that
> > I wrote above would be a typical use.
> >
> > I had a look myself in the scheduler code, and it''s quite not
obvious
> > where to patch. I think I have understood the burncredit() system, but
I
> > didn''t get where the values are read for the cap and all. Did
I
> > understood well that Xen uses 30ms cycles for the credit scheduler? If
> > that is the case, then I believe that computing what cap values to use
> > depending on the burst parameters for a given 30ms scheduling cycle
> > wouldn''t be such a big overhead.
> >
> > If nobody wants to implement this, can someone gives me some pointers
in
> > the sched_credit.c and all the .h structures? If I do stupid trials
and
> > errors in my implementation at first, because I''d be a
beginner at
> > patching Xen, would you guys have enough time to explain, and excuse a
> > newbie? Would that kind of patch be accepted if proven to work?
> >
> > Cheers,
> >
> > Thomas Goirand
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> >
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Kouya Shimura wrote:
> As for ''cap'' and ''weight'', they are
already configurable.
> Adding the following to configuration will help.
> 
> cpu_weight = 20
> cpu_cap = 50
> 
> Thanks,
> Kouya
I''ve spent a LONG TIME searching for this parameter, and
didn''t find
this info. Either it''s not documented, or not enough. Even George
didn''t
know about it.

Thomas


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel