Atsushi SAKAI
2007-May-09 02:49 UTC
[Xen-devel] [PATCH][RFC] consider weight for vcpu-pin case
Hi, This patch intends to consider weight for vcpu-pin case. (in other words, pcpu.credit > 30msec case.) In this case, weight parameter does not work appropriately. By applying this patch, weight parameter works. This patch consists 3-parts 1)Weight Calculation for vcpupin csched_consider_vcpupin() (add) 2)Credit Balance Calculation for vcpupin csched_consider_vcpupin_credit_balance() (add) 3)Other credit correction factor calculation. csched_acct() (modified like variable csched_corr_factor) For test If you want to test this patch, please test like following configuration at 4pcpu machine. Dom1 vcpu0.0-1 vcpu1.0-1 weight 256 Dom2 vcpu0.0-1 vcpu1.0-1 weight 512 N.B.)0-1 means pininfo(xm vcpu-pin) If this patch does not applied, xentop shows each cpu 100%. But If you use this patch, xentop shows 66%, 133% for each domain. Discussion Points 1) many loop is permitted? 2) arithmetic divide is still remain 4 points for each vcpu. 3) Is it enough tuned?(Need more tune) Any comments will be appreciated. c.f this problem is already questioned six month ago. http://lists.xensource.com/archives/html/xen-devel/2006-10/msg00365.html At this moment, I think this effect is very low. But a month ago, somebody suggested me this issue is important. (SMP like boot 8cpu with 2domain in 1pcpu share, weight does not effect. This also effects NUMA system weight.) Thanks Atsushi SAKAI _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Atsushi SAKAI
2007-Jun-08 00:17 UTC
Re: [Xen-devel] [PATCH][RFC] consider weight for vcpu-pin case
Hi, Keir Would you give me a comment on this? Thanks Atsushi SAKAI Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote:> Hi, > > This patch intends to consider weight for vcpu-pin case. > (in other words, pcpu.credit > 30msec case.) > In this case, weight parameter does not work appropriately. > By applying this patch, weight parameter works. > > This patch consists 3-parts > 1)Weight Calculation for vcpupin > csched_consider_vcpupin() (add) > > 2)Credit Balance Calculation for vcpupin > csched_consider_vcpupin_credit_balance() (add) > > 3)Other credit correction factor calculation. > csched_acct() (modified like variable csched_corr_factor) > > For test > If you want to test this patch, please test like following configuration > at 4pcpu machine. > Dom1 vcpu0.0-1 vcpu1.0-1 weight 256 > Dom2 vcpu0.0-1 vcpu1.0-1 weight 512 > N.B.)0-1 means pininfo(xm vcpu-pin) > If this patch does not applied, xentop shows each cpu 100%. > But If you use this patch, xentop shows 66%, 133% for each domain. > > Discussion Points > 1) many loop is permitted? > 2) arithmetic divide is still remain 4 points for each vcpu. > 3) Is it enough tuned?(Need more tune) > > > Any comments will be appreciated. > > c.f > this problem is already questioned six month ago. > http://lists.xensource.com/archives/html/xen-devel/2006-10/msg00365.html > At this moment, I think this effect is very low. > But a month ago, somebody suggested me this issue is important. > (SMP like boot 8cpu with 2domain in 1pcpu share, weight does not effect. > This also effects NUMA system weight.) > > Thanks > Atsushi SAKAI_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2007-Jun-08 08:26 UTC
Re: [Xen-devel] [PATCH][RFC] consider weight for vcpu-pin case
The patch fixes sched-credit vcpu weighting somehow, I guess. How does it do this? -- Keir On 8/6/07 01:17, "Atsushi SAKAI" <sakaia@jp.fujitsu.com> wrote:> Hi, Keir > > Would you give me a comment on this? > > Thanks > Atsushi SAKAI > > > Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote: > >> Hi, >> >> This patch intends to consider weight for vcpu-pin case. >> (in other words, pcpu.credit > 30msec case.) >> In this case, weight parameter does not work appropriately. >> By applying this patch, weight parameter works. >> >> This patch consists 3-parts >> 1)Weight Calculation for vcpupin >> csched_consider_vcpupin() (add) >> >> 2)Credit Balance Calculation for vcpupin >> csched_consider_vcpupin_credit_balance() (add) >> >> 3)Other credit correction factor calculation. >> csched_acct() (modified like variable csched_corr_factor) >> >> For test >> If you want to test this patch, please test like following configuration >> at 4pcpu machine. >> Dom1 vcpu0.0-1 vcpu1.0-1 weight 256 >> Dom2 vcpu0.0-1 vcpu1.0-1 weight 512 >> N.B.)0-1 means pininfo(xm vcpu-pin) >> If this patch does not applied, xentop shows each cpu 100%. >> But If you use this patch, xentop shows 66%, 133% for each domain. >> >> Discussion Points >> 1) many loop is permitted? >> 2) arithmetic divide is still remain 4 points for each vcpu. >> 3) Is it enough tuned?(Need more tune) >> >> >> Any comments will be appreciated. >> >> c.f >> this problem is already questioned six month ago. >> http://lists.xensource.com/archives/html/xen-devel/2006-10/msg00365.html >> At this moment, I think this effect is very low. >> But a month ago, somebody suggested me this issue is important. >> (SMP like boot 8cpu with 2domain in 1pcpu share, weight does not effect. >> This also effects NUMA system weight.) >> >> Thanks >> Atsushi SAKAI > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Atsushi SAKAI
2007-Jun-08 10:06 UTC
Re: [Xen-devel] [PATCH][RFC] consider weight for vcpu-pin case
Hi, Keir Thanks for your reply, Keir Fraser <keir@xensource.com> wrote:> The patch fixes sched-credit vcpu weighting somehow, I guess. How does it do > this?For example Xen boots on 8core machine(like Xeon Quad core X 2) Somebody plan to use 2pcpu for development use, and 2pcpu x 3 for customer use. example (d0 means Domain0 v0 means vcpu0) d0v0 d0v1 d0v2 d0v3 d0v4(not pinned) idle d1v0 d1v1 (pinned 0-1 with weight=256) running CPU intensive process. d2v0 d2v1 (pinned 0-1 with weight=512) running CPU intensive process. d3v0 d3v1 (pinned 2-3 with weight=256) idle d4v0 d4v1 (pinned 4-5 with weight=256) idle d5v0 d5v1 (pinned 6-7 with weight=256) idle (if you see in xentop) before after(applying this patch) d0 0.0 0.0 d1 100.0 66.0 d2 100.0 134.0 d3 0.0 0.0 d4 0.0 0.0 d5 0.0 0.0 In this case d1 credit sum is 80msec, and d2 credit sum is 160msec.(30msec x 8pcpu in Xen) But available CPU resources is just 60msec(since 2domains are pinned to 2pcpu). so each pcpu credit value is overflow(>30msec) This makes cannot allocate CPU resources based on CPU weight. Just equally allocate d1 and d2 CPU resources. (I think this is current CREDIT scheduler problem) Thanks Atsushi SAKAI _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2007-Jun-08 10:33 UTC
Re: [Xen-devel] [PATCH][RFC] consider weight for vcpu-pin case
At the very least the coding style needs to be cleaned up to match the rest of the file and to fit in 80 columns. Apart from that the new functions appear unfeasibly complicated and I''m a little worried abut how much time will be spent executing them each scheduling period? -- Keir On 8/6/07 11:06, "Atsushi SAKAI" <sakaia@jp.fujitsu.com> wrote:> Hi, Keir > > Thanks for your reply, > > Keir Fraser <keir@xensource.com> wrote: >> The patch fixes sched-credit vcpu weighting somehow, I guess. How does it do >> this? > > > For example Xen boots on 8core machine(like Xeon Quad core X 2) > Somebody plan to use 2pcpu for development use, and 2pcpu x 3 for customer > use. > > example (d0 means Domain0 v0 means vcpu0) > d0v0 d0v1 d0v2 d0v3 d0v4(not pinned) idle > d1v0 d1v1 (pinned 0-1 with weight=256) running CPU intensive process. > d2v0 d2v1 (pinned 0-1 with weight=512) running CPU intensive process. > d3v0 d3v1 (pinned 2-3 with weight=256) idle > d4v0 d4v1 (pinned 4-5 with weight=256) idle > d5v0 d5v1 (pinned 6-7 with weight=256) idle > > > (if you see in xentop) > before after(applying this patch) > d0 0.0 0.0 > d1 100.0 66.0 > d2 100.0 134.0 > d3 0.0 0.0 > d4 0.0 0.0 > d5 0.0 0.0 > > In this case d1 credit sum is 80msec, and d2 credit sum is 160msec.(30msec x > 8pcpu in Xen) > But available CPU resources is just 60msec(since 2domains are pinned to > 2pcpu). > so each pcpu credit value is overflow(>30msec) > This makes cannot allocate CPU resources based on CPU weight. > Just equally allocate d1 and d2 CPU resources. > (I think this is current CREDIT scheduler problem) > > Thanks > Atsushi SAKAI > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Atsushi SAKAI
2007-Jun-08 11:42 UTC
Re: [Xen-devel] [PATCH][RFC] consider weight for vcpu-pin case
Hi, Keir I measured it with xentrace on IA64 (Montecito 4pcpu cpu1.4GHz ITC350MHz(in x86 TSC)). As you know, credit is calculated on BSP with every 30msec. and its overhead of this patch is about 500count. This corresponds 1.4usec/30msec(1/20000) overhead on BSP (other AP is not affected by this patch.) Anyway, I clean up the code at first. Thanks Atsushi SAKAI Keir Fraser <keir@xensource.com> wrote:> > At the very least the coding style needs to be cleaned up to match the rest > of the file and to fit in 80 columns. Apart from that the new functions > appear unfeasibly complicated and I''m a little worried abut how much time > will be spent executing them each scheduling period? > > -- Keir > > On 8/6/07 11:06, "Atsushi SAKAI" <sakaia@jp.fujitsu.com> wrote: > > > Hi, Keir > > > > Thanks for your reply, > > > > Keir Fraser <keir@xensource.com> wrote: > >> The patch fixes sched-credit vcpu weighting somehow, I guess. How does it do > >> this? > > > > > > For example Xen boots on 8core machine(like Xeon Quad core X 2) > > Somebody plan to use 2pcpu for development use, and 2pcpu x 3 for customer > > use. > > > > example (d0 means Domain0 v0 means vcpu0) > > d0v0 d0v1 d0v2 d0v3 d0v4(not pinned) idle > > d1v0 d1v1 (pinned 0-1 with weight=256) running CPU intensive process. > > d2v0 d2v1 (pinned 0-1 with weight=512) running CPU intensive process. > > d3v0 d3v1 (pinned 2-3 with weight=256) idle > > d4v0 d4v1 (pinned 4-5 with weight=256) idle > > d5v0 d5v1 (pinned 6-7 with weight=256) idle > > > > > > (if you see in xentop) > > before after(applying this patch) > > d0 0.0 0.0 > > d1 100.0 66.0 > > d2 100.0 134.0 > > d3 0.0 0.0 > > d4 0.0 0.0 > > d5 0.0 0.0 > > > > In this case d1 credit sum is 80msec, and d2 credit sum is 160msec.(30msec x > > 8pcpu in Xen) > > But available CPU resources is just 60msec(since 2domains are pinned to > > 2pcpu). > > so each pcpu credit value is overflow(>30msec) > > This makes cannot allocate CPU resources based on CPU weight. > > Just equally allocate d1 and d2 CPU resources. > > (I think this is current CREDIT scheduler problem) > > > > Thanks > > Atsushi SAKAI > > > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel