Xen devel - Aug 2010 - [RFC]PLE''s performance enhancement through improving scheduler.

The attached patch is just for RFC, not for check-in. Recently, we are working
on enhancing the hardware feature PLE through improving scheduler in Xen, and
the attached patch can improve system''s throughput significantly. With
standand virtualization benchmark(vConsolidate), the testing result shows  ~20%
performance gain.   In the implemenation, there are two points to enhance the
system''s scheduler.
The first one is that when PLE vmexit occurs,  scheduler de-schedules the vcpu
and put it in the second position of the runq instead of moving it to the tail
of runq so that it can be re-scheduled in a very short time. In this case, it 
can improve scheduler''s faireness and make the PLE-senstive guests
allocated with reasonable timeslice.  The other improvement is to boost other
vcpus'' priority of the same guest through moving them to the head of
the runq when PLE vmexit happens with one vcpu of the guest.  And we are also
improving the implementation to make it more robust and more pervasive, but
before the work is done, we also want to collect your guys'' ideas and
suggestions about it ?  Any comment is very appreciated!. Thanks!
Xiantao

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 18/08/2010 06:51, "Zhang, Xiantao" <xiantao.zhang@intel.com>
wrote:
> The attached patch is just for RFC, not for check-in. Recently, we are
working
> on enhancing the hardware feature PLE through improving scheduler in Xen,
and
> the attached patch can improve system''s throughput significantly.
With
> standand virtualization benchmark(vConsolidate), the testing result shows
> ~20% performance gain.   In the implemenation, there are two points to
enhance
> the system''s scheduler.
> The first one is that when PLE vmexit occurs,  scheduler de-schedules the
vcpu
> and put it in the second position of the runq instead of moving it to the
tail
> of runq so that it can be re-scheduled in a very short time. In this case,
it
> can improve scheduler''s faireness and make the PLE-senstive guests
allocated
> with reasonable timeslice.  The other improvement is to boost other
vcpus''
> priority of the same guest through moving them to the head of the runq when
> PLE vmexit happens with one vcpu of the guest.  And we are also improving
the
> implementation to make it more robust and more pervasive, but before the
work
> is done, we also want to collect your guys'' ideas and suggestions
about it ?
> Any comment is very appreciated!. Thanks!
> Xiantao
Please Cc George Dunlap on scheduler work.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xiantao,

Thanks for the patch.  You have two changes here:
* Yield goes back only 1 vcpu ("yield")
* Other vcpus of yielding VM get yanked to the front ("boost")

If you send them as two patches, we can talk about the changes /
accept them individually.

Both changes sound reasonable, but we often find in schedulers that
small changes can have unexpected results. :-)

I think the "yield" patch should be easy to accept.  The
"boost" patch
looks good but also seems riskier -- I''d want to do some more thinking
and testing to make sure there aren''t situations where it could cause
problems, or allow one VM to "game" the scheduler to get more time.

Would you mind running the vConsolidate test with each patch
separately, so we can see how each one affects the performance?

 -George

On Wed, Aug 18, 2010 at 7:39 AM, Keir Fraser <keir.fraser@eu.citrix.com>
wrote:
> On 18/08/2010 06:51, "Zhang, Xiantao"
<xiantao.zhang@intel.com> wrote:
>
>> The attached patch is just for RFC, not for check-in. Recently, we are
working
>> on enhancing the hardware feature PLE through improving scheduler in
Xen, and
>> the attached patch can improve system''s throughput
significantly. With
>> standand virtualization benchmark(vConsolidate), the testing result
shows
>> ~20% performance gain.   In the implemenation, there are two points to
enhance
>> the system''s scheduler.
>> The first one is that when PLE vmexit occurs,  scheduler de-schedules
the vcpu
>> and put it in the second position of the runq instead of moving it to
the tail
>> of runq so that it can be re-scheduled in a very short time. In this
case, it
>> can improve scheduler''s faireness and make the PLE-senstive
guests allocated
>> with reasonable timeslice.  The other improvement is to boost other
vcpus''
>> priority of the same guest through moving them to the head of the runq
when
>> PLE vmexit happens with one vcpu of the guest.  And we are also
improving the
>> implementation to make it more robust and more pervasive, but before
the work
>> is done, we also want to collect your guys'' ideas and
suggestions about it ?
>> Any comment is very appreciated!. Thanks!
>> Xiantao
>
> Please Cc George Dunlap on scheduler work.
>
>  -- Keir
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

George Dunlap wrote:
> Xiantao,
> 
> Thanks for the patch.  You have two changes here:
> * Yield goes back only 1 vcpu ("yield")
> * Other vcpus of yielding VM get yanked to the front ("boost")
> 
> If you send them as two patches, we can talk about the changes /
> accept them individually.
> Both changes sound reasonable, but we often find in schedulers that
> small changes can have unexpected results. :-)
> 
> I think the "yield" patch should be easy to accept.  The
"boost" patch
> looks good but also seems riskier -- I''d want to do some more
thinking
> and testing to make sure there aren''t situations where it could
cause
> problems, or allow one VM to "game" the scheduler to get more
time.
Agree, we also thought this maybe a risk to unconditionally boost other
vcpus'' priority. We will investigate more before it is finally done.
Basically, we thought the current scheduler is not fair for PLE-senstive guests,
because it always yields its schedule chances to other vcpus in the runq once
meet PLE vmexits, so only very little time is allocated for its run. Certainly,
we also need to avoid malicous guests stealing time through the boost mechanism.
> Would you mind running the vConsolidate test with each patch
> separately, so we can see how each one affects the performance?
Okay, we will perform the testing separately as you suggested.  
Xiantao

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel