Is scheduling broken in xen-unstable? Here''s what I did... Added sched=rrobin to my kernel and started xen Ran xm rrobin, unknown command Ran xm help rrobin, unkown command So round robin is throw out So I tried the default bvt... Without appending it to my kernel, I ran... xm bvt_ctxallow 1 - Error: Internal server error With it appended to my kernel, I run.. xm bvt_ctxallow 1 - Computer hard crashes Any suggestions? Thanks, David _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> Added sched=rrobin to my kernel and started xen Ran xm > rrobin, unknown command Ran xm help rrobin, unkown commandsched=rrobin is not in unstable anymore. The default is the SEDF scheduler, but you can still set sched=bvt As I recall, there is some documentation on SEDF in tools/misc We should file a bug that using a scheduler op on a non existent scheduler does bad things. Ian> So round robin is throw out > > So I tried the default bvt... > Without appending it to my kernel, I ran... xm bvt_ctxallow 1 - > Error: Internal server error > > With it appended to my kernel, I run.. > xm bvt_ctxallow 1 - Computer hard crashes > > Any suggestions? > > Thanks, > David > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Submitted bug to bugzilla. I am now changing my scheduling and have noticed some different results, perhaps you could help me in my studies.... I am running CPU intensive VMs and am trying to find out at what scheduling they''ll run the best (since the simulation being run over a long duration, short periods and slices are not important). I have 8 VMs so my first few tests were running in periods of 1, 2, and 10 seconds, divided among the 8 domUs... So I typed Xm sedf (1,8) 1,2,10e9 125,250,1250e6 0 0 0 (ie, xm sedf 1 1000000000 125000000 0 0 0) In this case, I excluded scheduling for dom0... Could you please help me refine my scheduling. Thanks, David -----Original Message----- From: Ian Pratt [mailto:m+Ian.Pratt@cl.cam.ac.uk] Sent: Monday, July 25, 2005 11:50 AM To: Wolinsky, David; xen-devel@lists.xensource.com Cc: ian.pratt@cl.cam.ac.uk Subject: RE: [Xen-devel] Scheduling> Added sched=rrobin to my kernel and started xen Ran xm rrobin, unknown> command Ran xm help rrobin, unkown commandsched=rrobin is not in unstable anymore. The default is the SEDF scheduler, but you can still set sched=bvt As I recall, there is some documentation on SEDF in tools/misc We should file a bug that using a scheduler op on a non existent scheduler does bad things. Ian> So round robin is throw out > > So I tried the default bvt... > Without appending it to my kernel, I ran... xm bvt_ctxallow 1 - > Error: Internal server error > > With it appended to my kernel, I run.. > xm bvt_ctxallow 1 - Computer hard crashes > > Any suggestions? > > Thanks, > David > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>I am now changing my scheduling and have noticed some different results,>perhaps you could help me in my studies.... > >I am running CPU intensive VMs and am trying to find out at what >scheduling they''ll run the best (since the simulation being run over a >long duration, short periods and slices are not important). I have 8 >VMs so my first few tests were running in periods of 1, 2, and 10 >seconds, divided among the 8 domUs... > > >So I typed >Xm sedf (1,8) 1,2,10e9 125,250,1250e6 0 0 0 >(ie, xm sedf 1 1000000000 125000000 0 0 0) > >In this case, I excluded scheduling for dom0... > >Could you please help me refine my scheduling. > > >Sure. Actually I have not thought of such long periods and slices and in fact there are parts in the code that limit the slice and period lengths to roughly 4 seconds (due to arithmetic overflow), as I guessed that those long periods would be quite exotic. In fact I think you might be better of to use the scheduler in extra-time mode, that means you don''t guarantee time to the domains, but rather split the remaining (i.e. when all realtime domains have finished) time into pieces of various sizes. I guess this might be more appropriate to you, I assume that your long running simulation is not a real-time application? So you might just try to do xm sedf (1,8) 0 0 0 1 w With w specifying a weight for the domain. This works intuitively, so a domain with weight 4 gets twice the amount of CPU time as one with weight 2, which gets 2/5 of one with weight 5. I hope that this suits your needs, if not, let me know and I''ll change some of the arithmetic code. BTW: What did actually happen when you used the above command? Stephan>-----Original Message----- >From: Ian Pratt [mailto:m+Ian.Pratt@cl.cam.ac.uk] >Sent: Monday, July 25, 2005 11:50 AM >To: Wolinsky, David; xen-devel@lists.xensource.com >Cc: ian.pratt@cl.cam.ac.uk >Subject: RE: [Xen-devel] Scheduling > > > > >>Added sched=rrobin to my kernel and started xen Ran xm rrobin, unknown >> >> > > > >>command Ran xm help rrobin, unkown command >> >> > >sched=rrobin is not in unstable anymore. > >The default is the SEDF scheduler, but you can still set sched=bvt > >As I recall, there is some documentation on SEDF in tools/misc > >We should file a bug that using a scheduler op on a non existent >scheduler does bad things. > >Ian > > > >>So round robin is throw out >> >>So I tried the default bvt... >>Without appending it to my kernel, I ran... xm bvt_ctxallow 1 - >>Error: Internal server error >> >>With it appended to my kernel, I run.. >>xm bvt_ctxallow 1 - Computer hard crashes >> >>Any suggestions? >> >>Thanks, >>David >> >> >> >> > >_______________________________________________ >Xen-devel mailing list >Xen-devel@lists.xensource.com >http://lists.xensource.com/xen-devel > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Jul 28 2005, David_Wolinsky@Dell.com wrote: David, I''m away over the long weekend, will look into that in more detail when I''ll be back on Tuesday.>Yeah its not a real time application. And while weighting is a nice, >how does it deal with the period? If you could point to where in the >code the scheduler is, I would love to look through it an get a better >idea of how it works.The weighting doesn''t deal with periods, its parameter is the size of an EXTRA_QUANTUM. The interesting code is in the deschedule_extra_domain and do_extra_schedule (function names from my memory...). Have a look there. The basic principle is that each acquisition of a extra-quantum ''costs'' a price (called score in the code) and the domain with the lowest due payment gets the next quantum. This price is inverse to the weight. To avoid overflows, rather than incrementing the own due amount of ''payment'', the amount of other domains can be decreased. I hope this makes some sense. It should become more clear, when you have a look at the code. Please be aware that there are two extraqueues, both operate by the same principle, but the computation of the costs is different. You will want to have a look at the EXTRA_UTIL_Q, (or L1 extraq).> My short term goal was to see if scheduling could >create an overhead... Here were my results... > Period Slice BOPs > 8 VM 1 ms 125 us 6858 >..What does BOP stand for? I use it for begin of period, but how does that fit into your measurement?>With the base numbers being this.... > >Host 32403.5 >1 VM 32057 >...What does the second number mean?>But I figure these numbers might help you in your task, please >let me know if you do anything drastic to the scheduler between now and >next Wednesday... If so I''ll re-run my tests, and hold off publishing my >analysis.Well not to the scheduler itself, but there will be a load balancer in the near future, which runs outside the scheduler though.>Also any what happens with a number > 4 sec?Not sure, there could be some arithmetic overflows in some of the score estimation functions, because I multiply two time values. As they are in nanoseconds, 4 secs ~ 2^32 ns -> if you multiply them you will get an arithmetic overflow (max. result size is 64 bits). Pre right-shifting will improve the range, but of course will loose some precision (but who needs nanoseconds in this context anyways?) Result of this will be that the L0 extraq might do some interesting stuff, although only to a limited extent, so it''s def. not critical and hard to notice! Cheers, Stephan>-----Original Message----- >From: Stephan Diestelhorst [mailto:sd386@cl.cam.ac.uk] >Sent: Tuesday, July 26, 2005 5:44 PM >To: Wolinsky, David >Cc: m+Ian.Pratt@cl.cam.ac.uk; xen-devel@lists.xensource.com >Subject: Re: [Xen-devel] Scheduling > >>I am now changing my scheduling and have noticed some different >>results, > >>perhaps you could help me in my studies.... >> >>I am running CPU intensive VMs and am trying to find out at what >>scheduling they''ll run the best (since the simulation being run over a >>long duration, short periods and slices are not important). I have 8 >>VMs so my first few tests were running in periods of 1, 2, and 10 >>seconds, divided among the 8 domUs... >> >> >>So I typed >>Xm sedf (1,8) 1,2,10e9 125,250,1250e6 0 0 0 (ie, xm sedf 1 1000000000 >>125000000 0 0 0) >> >>In this case, I excluded scheduling for dom0... >> >>Could you please help me refine my scheduling. >> >> >> >Sure. Actually I have not thought of such long periods and slices and in >fact there are parts in the code that limit the slice and period lengths >to roughly 4 seconds (due to arithmetic overflow), as I guessed that >those long periods would be quite exotic. In fact I think you might be >better of to use the scheduler in extra-time mode, that means you don''t >guarantee time to the domains, but rather split the remaining (i.e. when >all realtime domains have finished) time into pieces of various sizes. >I guess this might be more appropriate to you, I assume that your long >running simulation is not a real-time application? > >So you might just try to do >xm sedf (1,8) 0 0 0 1 w >With w specifying a weight for the domain. This works intuitively, so a >domain with weight 4 gets twice the amount of CPU time as one with >weight 2, which gets 2/5 of one with weight 5. > >I hope that this suits your needs, if not, let me know and I''ll change >some of the arithmetic code. >BTW: What did actually happen when you used the above command? > >Stephan > >>-----Original Message----- >>From: Ian Pratt [mailto:m+Ian.Pratt@cl.cam.ac.uk] >>Sent: Monday, July 25, 2005 11:50 AM >>To: Wolinsky, David; xen-devel@lists.xensource.com >>Cc: ian.pratt@cl.cam.ac.uk >>Subject: RE: [Xen-devel] Scheduling >> >> >> >> >>>Added sched=rrobin to my kernel and started xen Ran xm rrobin, unknown >>> >>> >> >> >> >>>command Ran xm help rrobin, unkown command >>> >>> >> >>sched=rrobin is not in unstable anymore. >> >>The default is the SEDF scheduler, but you can still set sched=bvt >> >>As I recall, there is some documentation on SEDF in tools/misc >> >>We should file a bug that using a scheduler op on a non existent >>scheduler does bad things. >> >>Ian >> >> >> >>>So round robin is throw out >>> >>>So I tried the default bvt... >>>Without appending it to my kernel, I ran... xm bvt_ctxallow 1 - >>>Error: Internal server error >>> >>>With it appended to my kernel, I run.. >>>xm bvt_ctxallow 1 - Computer hard crashes >>> >>>Any suggestions? >>> >>>Thanks, >>>David >>> >>> >>> >>> >> >>_______________________________________________ >>Xen-devel mailing list >>Xen-devel@lists.xensource.com >>http://lists.xensource.com/xen-devel >> >> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Sorry for the late response, I too myself just got in today... Just to answer a few questions, BOPs stands for business operations per second Host - real system, ie, linux 2.6.12.x running natively Also note, I do not include dom0 as a "VM" since we''re not benchmarking its performance... that is, 1 VM = 1 domU + 1 dom0, ..., 8 VM = 8 domU + 1 dom0 As I am still catching up on other tasks, I will respond / look into the code later. Thanks, David _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel