Xen devel - Jul 2006 - credit scheduler: the policy of credit assignment

Hi Emmanuel,

I looked into the code of the credit scheduler and one question
has come up.

I''m not sure whether it is your intention that once the value of
credit_balance, which is the sum of credit of all domains on Xen,
goes to zero it may be stuck at zero. It will happen all of them
are cpu intensive domains and some of them turn into idle.

Every 30 msec, the credit scheduler gives them csched_priv.credit
milliseconds --- 30 msec * number of physical cpus ---. 30 msec later,
you will find they have consumed all of the time they gave, which
leads the value of credit_balance will keep zero. This means some
domains are assigned credit with negative value every time.

I know even in this case it will be balanced between domains
based on the weights but it would take quite long time to be balanced.
I feel the scheduler should give each domains larger credit than now
when credit_balance is small.

Am I something wrong about this?

Thanks,
Hirokazu Takahashi.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Jul 05, 2006 at 01:23:50PM +0900, Hirokazu Takahashi
wrote:
> Hi Emmanuel,
> 
> I looked into the code of the credit scheduler and one question
> has come up.
> 
> I''m not sure whether it is your intention that once the value of
> credit_balance, which is the sum of credit of all domains on Xen,
> goes to zero it may be stuck at zero. It will happen all of them
> are cpu intensive domains and some of them turn into idle.
> 
> Every 30 msec, the credit scheduler gives them csched_priv.credit
> milliseconds --- 30 msec * number of physical cpus ---. 30 msec later,
> you will find they have consumed all of the time they gave, which
> leads the value of credit_balance will keep zero. This means some
> domains are assigned credit with negative value every time.
> 
> I know even in this case it will be balanced between domains
> based on the weights but it would take quite long time to be balanced.
> I feel the scheduler should give each domains larger credit than now
> when credit_balance is small.
> 
> Am I something wrong about this?
Credit_balance only comes into play when active domains with positive
credit go idle. It''s a mechanism to converge the system towards its
stable state.

Are you suggesting that credit_balance, as it is used, should be the
sum of credit *prior* to incrementing active domains'' credits?

I''m not really sure I understand what you''re concerned about
here.
Can you elaborate and use a specific example to illustrate?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Emmanuel,
> > Hi Emmanuel,
> > 
> > I looked into the code of the credit scheduler and one question
> > has come up.
> > 
> > I''m not sure whether it is your intention that once the value
of
> > credit_balance, which is the sum of credit of all domains on Xen,
> > goes to zero it may be stuck at zero. It will happen all of them
> > are cpu intensive domains and some of them turn into idle.
> > 
> > Every 30 msec, the credit scheduler gives them csched_priv.credit
> > milliseconds --- 30 msec * number of physical cpus ---. 30 msec later,
> > you will find they have consumed all of the time they gave, which
> > leads the value of credit_balance will keep zero. This means some
> > domains are assigned credit with negative value every time.
> > 
> > I know even in this case it will be balanced between domains
> > based on the weights but it would take quite long time to be balanced.
> > I feel the scheduler should give each domains larger credit than now
> > when credit_balance is small.
> > 
> > Am I something wrong about this?
> 
> Credit_balance only comes into play when active domains with positive
> credit go idle. It''s a mechanism to converge the system towards
its
> stable state.
Yes, I think I understand the intention.
However, I don''t think the mechanism is effective enough.
> Are you suggesting that credit_balance, as it is used, should be the
> sum of credit *prior* to incrementing active domains'' credits?
I think it would be better.
> I''m not really sure I understand what you''re concerned
about here.
> Can you elaborate and use a specific example to illustrate?

Please suppose the system has 1 cpu and the sum of the credit goes
down to zero or quite small value unluckily.

           credit
Domain A     20
Domain B    -10        
Domain C    -10

30msec later, the domains consumed 30msec if they are cpu intensive
domains. The credits may become:

Domain A     0
Domain B    -15
Domain C    -15

Then, the credit scheduler divides 30 msec and gives it to each domains.
The credits may become:

Domain A     15
Domain B    -5
Domain C    -10

The sum of the credit keeps the same value.
I think there will be a chance to optimize it.

Thansk,
Hirokazu Takahashi.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, Jul 06, 2006 at 08:54:41AM +0900, Hirokazu Takahashi
wrote:
> > Credit_balance only comes into play when active domains with positive
> > credit go idle. It''s a mechanism to converge the system
towards its
> > stable state.
> 
> Yes, I think I understand the intention.
> However, I don''t think the mechanism is effective enough.
> 
> > Are you suggesting that credit_balance, as it is used, should be the
> > sum of credit *prior* to incrementing active domains''
credits?
> 
> I think it would be better.
> 
> > I''m not really sure I understand what you''re
concerned about here.
> > Can you elaborate and use a specific example to illustrate?
> 
> 
> Please suppose the system has 1 cpu and the sum of the credit goes
> down to zero or quite small value unluckily.
> 
>            credit
> Domain A     20
> Domain B    -10        
> Domain C    -10
I don''t understand why it is "unlucky" that the credit sum is
about zero?
> 30msec later, the domains consumed 30msec if they are cpu intensive
> domains. The credits may become:
> 
> Domain A     0
> Domain B    -15
> Domain C    -15
More likely, because time-slices are 30ms, it would be:

Domain A    -10
Domain B    -10        
Domain C    -10

> Then, the credit scheduler divides 30 msec and gives it to each domains.
> The credits may become:
> 
> Domain A     15
> Domain B    -5
> Domain C    -10
You''re not incrementing credits equally here so I assume you
mean that the domains have different weights and should
therefore get a different amount of CPU time.

If the domains had equal weights, you''d end up with:

Domain A    0
Domain B    0
Domain C    0
> The sum of the credit keeps the same value.
> I think there will be a chance to optimize it.
I don''t understand why you think it is bad to have a stable
active credit sum converging on zero. It''s the intent of the
scheduler to hand out as many credits as are being consumed.

As a matter of fact, having some domains credit positive and
others negative is the mechanism by which priorities change
and domains with higher weight get more CPU time. If all
domains were credit positive, they''d get the exact same
amount of CPU.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel