Dario Faggioli
2011-Dec-16 16:29 UTC
[PATCH] All domains (including dom0) should be best effort upon creation.
All domains (including dom0) should be best effort upon creation.
In the sedf scheduler, while trying to guarantee to dom0 the proper
amount of CPU time for being able to do its job, we end up allocating
75% CPU-share to each and every of its VCPUs. This, combined with the
fact that such a scheduler has no load balancing logic at all (and
thus *all* dom0''s VCPUs just rise and stay on physical CPU #0), means
that for example on a 12-cores system we''re trying to exploit
75x12=900% of what we have on a PCPU, which is definitely too much!
Moreover, even if a cleverer mechanism for distributing VCPUs among
the available PCPUs (if not a proper load balancer) will be put in
place some day, it will still be difficult to decide how much guaranteed
CPU bandwidth each of the dom0''s VCPUs should be provided with, without
posing the system at risk of livelock or starvation, even during
boot time.
Therefore, since sedf is capable of some form of best-effort
scheduling, the best thing we can do is ask for this behaviour for
dom0, as it is for all other domains, right upon creation. It will
then be the sysadmin''s job to modify dom0''s scheduling
parameters
to better fit his particular needs, maybe after spreading the load
a bit by pinning VCPUs on PCPUs, or using cpupools, etc.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
diff -r 01c8b27e3d7d xen/common/sched_sedf.c
--- a/xen/common/sched_sedf.c Thu Dec 15 16:56:21 2011 +0000
+++ b/xen/common/sched_sedf.c Fri Dec 16 16:05:37 2011 +0100
@@ -359,19 +359,9 @@ static void *sedf_alloc_vdata(const stru
inf->latency = 0;
inf->status = EXTRA_AWARE | SEDF_ASLEEP;
inf->extraweight = 1;
-
- if ( v->domain->domain_id == 0 )
- {
- /* Domain0 gets 75% guaranteed (15ms every 20ms). */
- inf->period = MILLISECS(20);
- inf->slice = MILLISECS(15);
- }
- else
- {
- /* Best-effort extratime only. */
- inf->period = WEIGHT_PERIOD;
- inf->slice = 0;
- }
+ /* Upon creation all domain are best-effort. */
+ inf->period = WEIGHT_PERIOD;
+ inf->slice = 0;
inf->period_orig = inf->period; inf->slice_orig = inf->slice;
INIT_LIST_HEAD(&(inf->list));
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-------------------------------------------------------------------
Dario Faggioli, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
PhD Candidate, ReTiS Lab, Scuola Superiore Sant''Anna, Pisa (Italy)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
George Dunlap
2011-Dec-16 18:06 UTC
Re: [PATCH] All domains (including dom0) should be best effort upon creation.
On Fri, Dec 16, 2011 at 4:29 PM, Dario Faggioli <dario.faggioli@citrix.com> wrote:> All domains (including dom0) should be best effort upon creation. > > In the sedf scheduler, while trying to guarantee to dom0 the proper > amount of CPU time for being able to do its job, we end up allocating > 75% CPU-share to each and every of its VCPUs. This, combined with the > fact that such a scheduler has no load balancing logic at all (and > thus *all* dom0''s VCPUs just rise and stay on physical CPU #0), means > that for example on a 12-cores system we''re trying to exploit > 75x12=900% of what we have on a PCPU, which is definitely too much! > > Moreover, even if a cleverer mechanism for distributing VCPUs among > the available PCPUs (if not a proper load balancer) will be put in > place some day, it will still be difficult to decide how much guaranteed > CPU bandwidth each of the dom0''s VCPUs should be provided with, without > posing the system at risk of livelock or starvation, even during > boot time. > > Therefore, since sedf is capable of some form of best-effort > scheduling, the best thing we can do is ask for this behaviour for > dom0, as it is for all other domains, right upon creation. It will > then be the sysadmin''s job to modify dom0''s scheduling parameters > to better fit his particular needs, maybe after spreading the load > a bit by pinning VCPUs on PCPUs, or using cpupools, etc. > > Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>Looks reasonable to me. Acked-by: George Dunlap <george.dunlap@eu.citrix.com>> > > diff -r 01c8b27e3d7d xen/common/sched_sedf.c > --- a/xen/common/sched_sedf.c Thu Dec 15 16:56:21 2011 +0000 > +++ b/xen/common/sched_sedf.c Fri Dec 16 16:05:37 2011 +0100 > @@ -359,19 +359,9 @@ static void *sedf_alloc_vdata(const stru > inf->latency = 0; > inf->status = EXTRA_AWARE | SEDF_ASLEEP; > inf->extraweight = 1; > - > - if ( v->domain->domain_id == 0 ) > - { > - /* Domain0 gets 75% guaranteed (15ms every 20ms). */ > - inf->period = MILLISECS(20); > - inf->slice = MILLISECS(15); > - } > - else > - { > - /* Best-effort extratime only. */ > - inf->period = WEIGHT_PERIOD; > - inf->slice = 0; > - } > + /* Upon creation all domain are best-effort. */ > + inf->period = WEIGHT_PERIOD; > + inf->slice = 0; > > inf->period_orig = inf->period; inf->slice_orig = inf->slice; > INIT_LIST_HEAD(&(inf->list)); > > -- > <<This happens because I choose it to happen!>> (Raistlin Majere) > ------------------------------------------------------------------- > Dario Faggioli, http://retis.sssup.it/people/faggioli > Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) > PhD Candidate, ReTiS Lab, Scuola Superiore Sant''Anna, Pisa (Italy) > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >
Dario Faggioli
2012-Jan-04 16:03 UTC
Re: [PATCH] All domains (including dom0) should be best effort upon creation.
On Fri, 2011-12-16 at 18:06 +0000, George Dunlap wrote:> > Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> > > Looks reasonable to me. > > Acked-by: George Dunlap <george.dunlap@eu.citrix.com> > >Again... Should I do something else or can this be checked-in? :-) Thanks and Regards, Dario -- <<This happens because I choose it to happen!>> (Raistlin Majere) ------------------------------------------------------------------- Dario Faggioli, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) PhD Candidate, ReTiS Lab, Scuola Superiore Sant''Anna, Pisa (Italy) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel