Hi, I sent RFC and a patch-set of I/O framework for virtual disk in order to introduce QoS in end of July. > http://lists.xensource.com/archives/html/xen-devel/2007-07/msg00863.html But no one responded to it. I think that when many virtual machines run on one physical machine, QoS of machine resources will be necessary. For example, in server consolidation, integrated servers have various roles and are different in the desired resources. So, distribution of resources is very important for stabilization. As other example, it will be avoided to be affected by jobs of other people in consolidation of office platform. CPU and memory are controlled (allocated) at Xen hypervisor. (This is OS agnostic) Is similar controller for I/O not necessary? Linux adopts CFQ which is fair scheduler which is suitable for desktop system. But, privileged domain will be not only Linux in future. I think that such function should be developed in Xen hypervisor or virtual block driver framework. Does anyone have opinions or idea? Please reply comments. -------------------------------------------------- Satoshi UCHIDA _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
First impression was a lot of plumbing and not much in the way of moving parts. Also of course we have CFQ on Linux -- I suspect the kinds of changes you are suggesting would not be popular with kernel maintainers since they will argue there is already an I/O scheduling subsystem. :-) On the other hand, if you want to run a block driver in a driver domain (and so outside dom0) then having a programmatic scheduling interface via xenstore is quite nice... -- Keir On 22/8/07 10:32, "Satoshi Uchida" <s-uchida@ap.jp.nec.com> wrote:> Hi, > > I sent RFC and a patch-set of I/O framework for virtual disk in order to > introduce QoS in end of July. >> http://lists.xensource.com/archives/html/xen-devel/2007-07/msg00863.html > But no one responded to it. > > I think that when many virtual machines run on one physical machine, QoS of > machine resources will be necessary. > For example, in server consolidation, integrated servers have various roles > and are different in the desired resources. > So, distribution of resources is very important for stabilization. > As other example, it will be avoided to be affected by jobs of other people in > consolidation of office platform. > > CPU and memory are controlled (allocated) at Xen hypervisor. (This is OS > agnostic) > Is similar controller for I/O not necessary? > > Linux adopts CFQ which is fair scheduler which is suitable for desktop system. > But, privileged domain will be not only Linux in future. > I think that such function should be developed in Xen hypervisor or virtual > block driver framework. > > Does anyone have opinions or idea? > Please reply comments. > > -------------------------------------------------- > Satoshi UCHIDA > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Satoshi Uchida
2007-Aug-24 08:27 UTC
RE: [Xen-devel] Is QoS of virtual disk not necessary?
Hi, Keir. Thanks for comment.> First impression was a lot of plumbing and not much in the way of > moving parts.What does this means? Are "iomgr_xx" procedures too much for the turn-based control module? E.g. the iomgr_oo_abort_request_fn procedure is not used in turn-based control module.> Also of course we have CFQ on Linux -- I suspect the kinds of changes > you are suggesting would not be popular with kernel maintainers since > they will argue there is already an I/O scheduling subsystem. :-)I think that using I/O scheduling subsystem is one of solutions. But, I fear two things. One is that CFQ can be used only if a driver domain (or dom0) is only Linux. In future, driver domains will be choose any types of OS, and they don''t have same I/O scheduling mechanism. And there are also many I/O schedulers in Linux, and driver domains will be choose some schedulers. Therefore, I/O control is not dependent with I/O scheduler in specific type of OS. Another is that CFQ is developed for desktop system and for private environments. So, this may not be suitable in virtualization environments. And, a setting parameter of CFQ is too simple, namely it have only 8-level priority ranks. Therefore, it is difficult to apply CFQ into huge virtualization system. E.g. for many domains, it is difficult to set them by a percentage. Therefore, I think that it is better to develop OS-agnostic I/O control.> On the other hand, if you want to run a block driver in a driver > domain (and so outside dom0) then having a programmatic scheduling > interface via xenstore is quite nice...Does this mean that interfaces should not be implemented by insmod or rewriting sysfs entries, but should be implemented by xenstore and xm commands? Thanks. --------------------------------------------- Satoshi UCHIDA _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
hi how many processors does xen support in hardware for the practical system? what is the max number of processors in the practice system running under xen ? Thanks in advance _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
pradeep singh rautela
2007-Aug-24 10:06 UTC
Re: [Xen-devel] how many processors does xen support in hardware ?
On 8/24/07, tgh <tianguanhua@ncic.ac.cn> wrote:> hi > how many processors does xen support in hardware for the practical > system?Thats would be 255 i guess. thanks> what is the max number of processors in the practice system running > under xen ?> > > Thanks in advance > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- -- pradeep singh rautela "Genius is 1% inspiration, and 99% perspiration" - not me :) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
tgh
2007-Aug-24 11:31 UTC
Re: [Xen-devel] how many processors does xen support in hardware ?
pradeep singh rautela 写道:> On 8/24/07, tgh <tianguanhua@ncic.ac.cn> wrote: > >> hi >> how many processors does xen support in hardware for the practical >> system? >> > > Thats would be 255 i guess. >thank you for your reply but I want to kown the max number of hardware cpus the xen could support, or the xen''s scalability in the aspect of cpu number,not in the aspect of the VM number thanks> thanks > >> what is the max number of processors in the practice system running >> under xen ? >> > > >> Thanks in advance >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >> > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Puthiyaparambil, Aravindh
2007-Aug-24 13:34 UTC
RE: [Xen-devel] how many processors does xen support in hardware?
Xen x86_32/PAE supports 32 logical CPUs. This is a hard limit and there is no easy way to increase this. Xen x86_64 supports 64 logical CPUs. You need to compile Xen with max_phys_cpus=64 to enable this support. If you want to go beyond 126 LCPUs some patching is needed. I guess you can theoretically go up to 255 but I think you might run out of GDT slots before that. Aravindh Puthiyaparambil Virtualization Engineer Virtual Systems Development Unisys, Tredyffrin PA -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of tgh Sent: Friday, August 24, 2007 7:31 AM To: pradeep singh rautela Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] how many processors does xen support in hardware? pradeep singh rautela 写道:> On 8/24/07, tgh <tianguanhua@ncic.ac.cn> wrote: > >> hi >> how many processors does xen support in hardware for the practical >> system? >> > > Thats would be 255 i guess. >thank you for your reply but I want to kown the max number of hardware cpus the xen could support, or the xen''s scalability in the aspect of cpu number,not in the aspect of the VM number thanks> thanks > >> what is the max number of processors in the practice system running >> under xen ? >> > > >> Thanks in advance >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >> > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
pradeep singh rautela
2007-Aug-24 14:32 UTC
Re: [Xen-devel] how many processors does xen support in hardware?
On 8/24/07, Puthiyaparambil, Aravindh <aravindh.puthiyaparambil@unisys.com> wrote:> Xen x86_32/PAE supports 32 logical CPUs. This is a hard limit and there is no easy way to increase this. > > Xen x86_64 supports 64 logical CPUs. You need to compile Xen with max_phys_cpus=64 to enable this support. If you want to go beyond 126 LCPUs some patching is needed. > > I guess you can theoretically go up to 255 but I think you might run out of GDT slots before that.Right, so why keep 255 there at all? I mean why hard code it to a max 255 then?Put it at a number a little near or higher than what Xen is actually capable of. thanks> > Aravindh Puthiyaparambil > Virtualization Engineer > Virtual Systems Development > Unisys, Tredyffrin PA > > -----Original Message----- > From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of tgh > Sent: Friday, August 24, 2007 7:31 AM > To: pradeep singh rautela > Cc: xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] how many processors does xen support in hardware? > > pradeep singh rautela 写道: > > On 8/24/07, tgh <tianguanhua@ncic.ac.cn> wrote: > > > >> hi > >> how many processors does xen support in hardware for the practical > >> system? > >> > > > > Thats would be 255 i guess. > > > thank you for your reply > but I want to kown the max number of hardware cpus the xen could > support, or the xen's scalability in the aspect of cpu number,not in the > aspect of the VM number > > thanks > > > > > > > thanks > > > >> what is the max number of processors in the practice system running > >> under xen ? > >> > > > > > >> Thanks in advance > >> > >> > >> _______________________________________________ > >> Xen-devel mailing list > >> Xen-devel@lists.xensource.com > >> http://lists.xensource.com/xen-devel > >> > >> > > > > > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- -- pradeep singh rautela "Genius is 1% inspiration, and 99% perspiration" - not me :) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2007-Aug-24 14:49 UTC
Re: [Xen-devel] how many processors does xen support in hardware?
Where did you get this number 255 from? K. On 24/8/07 15:32, "pradeep singh rautela" <rautelap@gmail.com> wrote:> On 8/24/07, Puthiyaparambil, Aravindh<aravindh.puthiyaparambil@unisys.com>> wrote: > Xen x86_32/PAE supports 32 logical CPUs. This is a hard limit and > there is no easy way to increase this. > > Xen x86_64 supports 64 logical > CPUs. You need to compile Xen with max_phys_cpus=64 to enable this support. If > you want to go beyond 126 LCPUs some patching is needed. > > I guess you can > theoretically go up to 255 but I think you might run out of GDT slots before > that.Right, so why keep 255 there at all? I mean why hard code it to a> max255 then?Put it at a number a little near or higher than what Xen> isactually capable of. thanks> > Aravindh Puthiyaparambil > Virtualization > Engineer > Virtual Systems Development > Unisys, Tredyffrin PA > > > -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of tgh > Sent: > Friday, August 24, 2007 7:31 AM > To: pradeep singh rautela > Cc: > xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] how many processors > does xen support in hardware? > > pradeep singh rautela 写道: > > On 8/24/07, > tgh <tianguanhua@ncic.ac.cn> wrote: > > > >> hi > >> how many processors > does xen support in hardware for the practical > >> system? > >> > > > > Thats > would be 255 i guess. > > > thank you for your reply > but I want to kown the > max number of hardware cpus the xen could > support, or the xen''s scalability > in the aspect of cpu number,not in the > aspect of the VM number > > > thanks > > > > > > > thanks > > > >> what is the max number of processors in > the practice system running > >> under xen ? > >> > > > > > >> Thanks in > advance > >> > >> > >> _______________________________________________ > >> > Xen-devel mailing list > >> Xen-devel@lists.xensource.com > >> > http://lists.xensource.com/xen-devel > >> > >> > > > > > > > > > > _______________________________________________ > Xen-devel mailing list > > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-->-- pradeep singh rautela "Genius is 1% inspiration, and 99% perspiration" -> not me :)> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Puthiyaparambil, Aravindh
2007-Aug-24 15:02 UTC
RE: [Xen-devel] how many processors does xen support in hardware?
It is not hard coded to 255. For example if you want to up the number to 96, then build Xen x86_64 with "make max_phys_cpus=96" and so on. But like I mentioned anything above 126 will require some patching. BTW (echoing Keir''s question), why do you think it is hard coded to 255? Aravindh -----Original Message----- From: pradeep singh rautela [mailto:rautelap@gmail.com] Sent: Friday, August 24, 2007 10:33 AM To: Puthiyaparambil, Aravindh Cc: tgh; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] how many processors does xen support in hardware? On 8/24/07, Puthiyaparambil, Aravindh <aravindh.puthiyaparambil@unisys.com> wrote:> Xen x86_32/PAE supports 32 logical CPUs. This is a hard limit and there is no easy way to increase this. > > Xen x86_64 supports 64 logical CPUs. You need to compile Xen with max_phys_cpus=64 to enable this support. If you want to go beyond 126 LCPUs some patching is needed. > > I guess you can theoretically go up to 255 but I think you might run out of GDT slots before that.Right, so why keep 255 there at all? I mean why hard code it to a max 255 then?Put it at a number a little near or higher than what Xen is actually capable of. thanks> > Aravindh Puthiyaparambil > Virtualization Engineer > Virtual Systems Development > Unisys, Tredyffrin PA > > -----Original Message----- > From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of tgh > Sent: Friday, August 24, 2007 7:31 AM > To: pradeep singh rautela > Cc: xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] how many processors does xen support in hardware? > > pradeep singh rautela 写道: > > On 8/24/07, tgh <tianguanhua@ncic.ac.cn> wrote: > > > >> hi > >> how many processors does xen support in hardware for the practical > >> system? > >> > > > > Thats would be 255 i guess. > > > thank you for your reply > but I want to kown the max number of hardware cpus the xen could > support, or the xen''s scalability in the aspect of cpu number,not in the > aspect of the VM number > > thanks > > > > > > > thanks > > > >> what is the max number of processors in the practice system running > >> under xen ? > >> > > > > > >> Thanks in advance > >> > >> > >> _______________________________________________ > >> Xen-devel mailing list > >> Xen-devel@lists.xensource.com > >> http://lists.xensource.com/xen-devel > >> > >> > > > > > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- -- pradeep singh rautela "Genius is 1% inspiration, and 99% perspiration" - not me :) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 24/8/07 09:27, "Satoshi Uchida" <s-uchida@ap.jp.nec.com> wrote:> Another is that CFQ is developed for desktop system and for private > environments. > So, this may not be suitable in virtualization environments. > And, a setting parameter of CFQ is too simple, namely it have only 8-level > priority ranks. > Therefore, it is difficult to apply CFQ into huge virtualization system. > E.g. for many domains, it is difficult to set them by a percentage. > > Therefore, I think that it is better to develop OS-agnostic I/O control.Another nice thing would be that if we do not use CFQ then we do not need a kernel thread per VBD. We could support one kernel thread per blkback and one kernel thread per VBD. I don''t know if both these models can be neatly supported by a single consistent set of iomgr hooks.>> On the other hand, if you want to run a block driver in a driver >> domain (and so outside dom0) then having a programmatic scheduling >> interface via xenstore is quite nice... > > Does this mean that interfaces should not be implemented by insmod or > rewriting sysfs entries, but should be implemented by xenstore and xm > commands?Hmmm... Well actually all the blkdev stats are exported thru sysfs right now, so already there are things that do not work well when blkback is in a driver domain (e.g., xentop). Perhaps we should export everything thru sysfs, but then provide a way to proxy that info through xenstore too, as an optional extra (either a user-space daemon, or we could make it a kernel driver option). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:> > Therefore, I think that it is better to develop OS-agnostic I/O control. > > Another nice thing would be that if we do not use CFQ then we do not need a > kernel thread per VBD.It would be probably easy to extend CFQ2 to use an user passed identifier instead of per task for sharing if that''s your goal. For a kernel driver like blkback you could just switch around multiple current->io_contexts. And if CFQ2 can manage hundreds of processes I don''t see why it couldn''t manage hundreds of guests. Although you probably should not put that many on a single device anyways; or rather if it''s a single device for that many it''s likely a RAM backed big storage box that doesn''t need much scheduling anyways. The claim that CFQ2 is desktop oriented in this thread earlier is also not true. The desktop oriented scheduler is more AS and CFQ2 is widely and successfully used in server applications. -Andi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 24/8/07 18:26, "Andi Kleen" <andi@firstfloor.org> wrote:>> Another nice thing would be that if we do not use CFQ then we do not need a >> kernel thread per VBD. > > It would be probably easy to extend CFQ2 to use an user passed identifier > instead of per task for sharing if that''s your goal. For a kernel driver > like blkback you could just switch around multiple current->io_contexts.Does current->io_context get latched before submit_bh() (or whatever similar function it is we use) returns? Is this true even in older 2.6 kernels with old CFQ? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
pradeep singh
2007-Aug-24 18:00 UTC
Re: [Xen-devel] how many processors does xen support in hardware?
On Friday 24 August 2007 20:19:07 Keir Fraser wrote:> Where did you get this number 255 from?Sorry i meant, while i compile the xen kernel i can configure kernel upto a max of 255. right? And i guess that means the number of CPUs you can compile your SMP kernel to work with. what did i miss? :-/ thanks --pradeep> > K. > > On 24/8/07 15:32, "pradeep singh rautela" <rautelap@gmail.com> wrote: > > On 8/24/07, Puthiyaparambil, Aravindh > > <aravindh.puthiyaparambil@unisys.com> > > > wrote: > > Xen x86_32/PAE supports 32 logical CPUs. This is a hard limit and > > there is no easy way to increase this. > > > > Xen x86_64 supports 64 logical > > CPUs. You need to compile Xen with max_phys_cpus=64 to enable this > > support. If you want to go beyond 126 LCPUs some patching is needed. > > > > I guess you can > > theoretically go up to 255 but I think you might run out of GDT slots > > before that. > > Right, so why keep 255 there at all? I mean why hard code it to a > > > max > > 255 then?Put it at a number a little near or higher than what Xen > > > is > > actually capable of. > > thanks > > > Aravindh Puthiyaparambil > > Virtualization > > Engineer > > Virtual Systems Development > > Unisys, Tredyffrin PA > > > > > > -----Original Message----- > > From: xen-devel-bounces@lists.xensource.com > > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of tgh > > Sent: > > Friday, August 24, 2007 7:31 AM > > To: pradeep singh rautela > > Cc: > > xen-devel@lists.xensource.com > > Subject: Re: [Xen-devel] how many processors > > does xen support in hardware? > > > > pradeep singh rautela 写道: > > > On 8/24/07, > > > > tgh <tianguanhua@ncic.ac.cn> wrote: > > >> hi > > >> how many processors > > > > does xen support in hardware for the practical > > > > >> system? > > > > > > Thats > > > > would be 255 i guess. > > > > thank you for your reply > > but I want to kown the > > max number of hardware cpus the xen could > > support, or the xen''s scalability > > in the aspect of cpu number,not in the > > aspect of the VM number > > > > > > thanks > > > > > thanks > > > > > >> what is the max number of processors in > > > > the practice system running > > > > >> under xen ? > > >> > > >> > > >> > > >> Thanks in > > > > advance > > > > >> _______________________________________________ > > > > Xen-devel mailing list > > > > >> Xen-devel@lists.xensource.com > > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > > > > > > > > _______________________________________________ > > Xen-devel mailing list > > > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > -- > > -- > pradeep singh rautela > > "Genius is 1% inspiration, and 99% perspiration" - > > > not me :) > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2007-Aug-24 18:04 UTC
Re: [Xen-devel] how many processors does xen support in hardware?
Oh, right, yes that is Linux''s configurable maximum (itself fairly arbitrary really). You can''t actually use that number on Xen, so don''t set it so high. Simple really. :-) -- Keir On 24/8/07 19:00, "pradeep singh" <rautelap@gmail.com> wrote:> On Friday 24 August 2007 20:19:07 Keir Fraser wrote: >> Where did you get this number 255 from? > > Sorry i meant, while i compile the xen kernel i can configure kernel upto a > max of 255. right? > > And i guess that means the number of CPUs you can compile your SMP kernel to > work with. > what did i miss? :-/ > > thanks > --pradeep >> >> K. >> >> On 24/8/07 15:32, "pradeep singh rautela" <rautelap@gmail.com> wrote: >>> On 8/24/07, Puthiyaparambil, Aravindh >> >> <aravindh.puthiyaparambil@unisys.com> >> >>> wrote: >>> Xen x86_32/PAE supports 32 logical CPUs. This is a hard limit and >>> there is no easy way to increase this. >>> >>> Xen x86_64 supports 64 logical >>> CPUs. You need to compile Xen with max_phys_cpus=64 to enable this >>> support. If you want to go beyond 126 LCPUs some patching is needed. >>> >>> I guess you can >>> theoretically go up to 255 but I think you might run out of GDT slots >>> before that. >> >> Right, so why keep 255 there at all? I mean why hard code it to a >> >>> max >> >> 255 then?Put it at a number a little near or higher than what Xen >> >>> is >> >> actually capable of. >> >> thanks >> >>> Aravindh Puthiyaparambil >>> Virtualization >>> Engineer >>> Virtual Systems Development >>> Unisys, Tredyffrin PA >>> >>> >>> -----Original Message----- >>> From: xen-devel-bounces@lists.xensource.com >>> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of tgh >>> Sent: >>> Friday, August 24, 2007 7:31 AM >>> To: pradeep singh rautela >>> Cc: >>> xen-devel@lists.xensource.com >>> Subject: Re: [Xen-devel] how many processors >>> does xen support in hardware? >>> >>> pradeep singh rautela 写道: >>>> On 8/24/07, >>> >>> tgh <tianguanhua@ncic.ac.cn> wrote: >>>>> hi >>>>> how many processors >>> >>> does xen support in hardware for the practical >>> >>>>> system? >>>> >>>> Thats >>> >>> would be 255 i guess. >>> >>> thank you for your reply >>> but I want to kown the >>> max number of hardware cpus the xen could >>> support, or the xen''s scalability >>> in the aspect of cpu number,not in the >>> aspect of the VM number >>> >>> >>> thanks >>> >>>> thanks >>>> >>>>> what is the max number of processors in >>> >>> the practice system running >>> >>>>> under xen ? >>>>> >>>>> >>>>> >>>>> Thanks in >>> >>> advance >>> >>>>> _______________________________________________ >>> >>> Xen-devel mailing list >>> >>>>> Xen-devel@lists.xensource.com >>> >>> http://lists.xensource.com/xen-devel >>> >>> >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel >> >> -- >> >> -- >> pradeep singh rautela >> >> "Genius is 1% inspiration, and 99% perspiration" - >> >>> not me :) >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
pradeep singh
2007-Aug-24 18:15 UTC
Re: [Xen-devel] how many processors does xen support in hardware?
On Friday 24 August 2007 23:34:27 Keir Fraser wrote:> Oh, right, yes that is Linux''s configurable maximum (itself fairly > arbitrary really). You can''t actually use that number on Xen, so don''t set > it so high. Simple really. :-)Simple?...heh :-) I''ll keep that in mind. thanks for insight. --pradeep> > -- Keir > > On 24/8/07 19:00, "pradeep singh" <rautelap@gmail.com> wrote: > > On Friday 24 August 2007 20:19:07 Keir Fraser wrote: > >> Where did you get this number 255 from? > > > > Sorry i meant, while i compile the xen kernel i can configure kernel upto > > a max of 255. right? > > > > And i guess that means the number of CPUs you can compile your SMP kernel > > to work with. > > what did i miss? :-/ > > > > thanks > > --pradeep > > > >> K. > >>[snip] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, Aug 24, 2007 at 06:40:54PM +0100, Keir Fraser wrote:> On 24/8/07 18:26, "Andi Kleen" <andi@firstfloor.org> wrote: > > >> Another nice thing would be that if we do not use CFQ then we do not need a > >> kernel thread per VBD. > > > > It would be probably easy to extend CFQ2 to use an user passed identifier > > instead of per task for sharing if that''s your goal. For a kernel driver > > like blkback you could just switch around multiple current->io_contexts. > > Does current->io_context get latched before submit_bh() (or whatever similar > function it is we use) returns? Is this true even in older 2.6 kernels with > old CFQ?It obviously has to because you could exit in the next call. The context is reference counted and lazily freed. -Andi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
tgh
2007-Aug-25 07:29 UTC
Re: [Xen-devel] how many processors does xen support in hardware?
hi did someone make the experiment for the xen''s scalability in the aspect of the cpu number supported? and what about the performence with 16 cpus , 32 cpus , and 64cpus, say ,comparing with the native linux? pradeep singh 写道:> On Friday 24 August 2007 23:34:27 Keir Fraser wrote: > >> Oh, right, yes that is Linux''s configurable maximum (itself fairly >> arbitrary really). You can''t actually use that number on Xen, so don''t set >> it so high. Simple really. :-) >> > > Simple?...heh :-) > I''ll keep that in mind. > > thanks for insight. > --pradeep > >> -- Keir >> >> On 24/8/07 19:00, "pradeep singh" <rautelap@gmail.com> wrote: >> >>> On Friday 24 August 2007 20:19:07 Keir Fraser wrote: >>> >>>> Where did you get this number 255 from? >>>> >>> Sorry i meant, while i compile the xen kernel i can configure kernel upto >>> a max of 255. right? >>> >>> And i guess that means the number of CPUs you can compile your SMP kernel >>> to work with. >>> what did i miss? :-/ >>> >>> thanks >>> --pradeep >>> >>> >>>> K. >>>> >>>> > [snip] > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > > > . > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
tgh
2007-Aug-26 13:26 UTC
Re: [Xen-devel] how many processors does xen support in hardware?
Thank you for your reply and did you take any scalability experiments for xen under hardware platform with 16cpus or more cpus for datacenter application or any other applications? and what about the performance of xen ,comparing with the native linux? Thanks Mats Petersson 写道:> At 08:29 25/08/2007, you wrote: >> hi >> did someone make the experiment for the xen''s scalability in the aspect >> of the cpu number supported? and what about the performence with 16 cpus >> , 32 cpus , and 64cpus, say ,comparing with the native linux? > > > This would be very dependant on the workload type - something that > does a lot of page-table updates will be more affected by "xen > overhead" than something that does no page-table updates. Other than > that, a para-virtual guest should be fairly close to the native setup. > > Of course, finding machines with more than 32 cores is pretty > difficult, as 32 takes 8 sockets of quad-cores, and that''s pretty much > the limit on "standard" motherboards - and even those are pretty darn > expensive. > > -- > Mats > > > > >> pradeep singh $B<LF;(B: >> > On Friday 24 August 2007 23:34:27 Keir Fraser wrote: >> > >> >> Oh, right, yes that is Linux''s configurable maximum (itself fairly >> >> arbitrary really). You can''t actually use that number on Xen, so >> don''t set >> >> it so high. Simple really. :-) >> >> >> > >> > Simple?...heh :-) >> > I''ll keep that in mind. >> > >> > thanks for insight. >> > --pradeep >> > >> >> -- Keir >> >> >> >> On 24/8/07 19:00, "pradeep singh" <rautelap@gmail.com> wrote: >> >> >> >>> On Friday 24 August 2007 20:19:07 Keir Fraser wrote: >> >>> >> >>>> Where did you get this number 255 from? >> >>>> >> >>> Sorry i meant, while i compile the xen kernel i can configure >> kernel upto >> >>> a max of 255. right? >> >>> >> >>> And i guess that means the number of CPUs you can compile your >> SMP kernel >> >>> to work with. >> >>> what did i miss? :-/ >> >>> >> >>> thanks >> >>> --pradeep >> >>> >> >>> >> >>>> K. >> >>>> >> >>>> >> > [snip] >> > >> > >> > >> > _______________________________________________ >> > Xen-devel mailing list >> > Xen-devel@lists.xensource.com >> > http://lists.xensource.com/xen-devel >> > >> > >> > . >> > >> > >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel > > > > . >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Satoshi Uchida
2007-Aug-28 09:07 UTC
RE: [Xen-devel] Is QoS of virtual disk not necessary?
Hi,> > On 24/8/07 09:27, "Satoshi Uchida" <s-uchida@ap.jp.nec.com> wrote: > > > > > Therefore, I think that it is better to develop OS-agnostic > I/O control. > > Another nice thing would be that if we do not use CFQ then we > do not need a kernel thread per VBD. We could support one > kernel thread per blkback and one kernel thread per VBD. > > I don''t know if both these models can be neatly supported by > a single consistent set of iomgr hooks.First, I fell that integrating into one kernel thread is a nice ways. I/O requests can be controlled only by implementing any queuing techniques, and it is not need to control threads However, usage of thread per VBD has advantage of simplifying to control event channels (ring buffer) and to manage information of VBDs (rd-reqs wr-reqs, ...), etc. So, I think that it is not needed to unite their threads.> > Does this mean that interfaces should not be implemented by > insmod or > > rewriting sysfs entries, but should be implemented by > xenstore and xm > > commands? > > Hmmm... Well actually all the blkdev stats are exported thru > sysfs right now, so already there are things that do not work > well when blkback is in a driver domain (e.g., xentop). > Perhaps we should export everything thru sysfs, but then > provide a way to proxy that info through xenstore too, as an > optional extra (either a user-space daemon, or we could make > it a kernel driver option). >OK. It is nice to implement interface via xenstore. The xenstore is implemented on any OS (Linux or Solaris) and it will be implemented on other OSs in future. Maybe, there are sysfs in only Linux and there are not sysfs in other OS. -- Perhaps there are similar systems such as devfs. -- Therefore, Using sysfs interface becomes OS-specific, but using xenstore is OS-agnostic. Thanks, -------------------------------------- Satoshi UCHIDA _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Satoshi Uchida
2007-Aug-28 09:09 UTC
[Xen-devel] RE: Is QoS of virtual disk not necessary?
> It would be probably easy to extend CFQ2 to use an user > passed identifier instead of per task for sharing if that''s > your goal. For a kernel driver like blkback you could just > switch around multiple current->io_contexts. > > And if CFQ2 can manage hundreds of processes I don''t see why > it couldn''t manage hundreds of guests. Although you probably > should not put that many on a single device anyways; or > rather if it''s a single device for that many it''s likely a > RAM backed big storage box that doesn''t need much scheduling anyways. >CFQ2 is CFQ version 2 or new CFQ? In linux documentation ( Documentation/block/ioprio.txt ), I see that CFQ is version 3. > With the introduction of cfq v3 If CFQ2 is CFQ version 2, which is current CFQ version 2? or version 3? As you say, improving io_contexts may be possible to control I/O requests. Maybe, in XenEnterprise, I guess that I/O requests is controlled by CFQ, namely, I/O priority is set to blkback or balktap threads. However, CFQ is implemented in only Linux and not in other OSs. So, privileged domains or driver domains are restricted with only Linux. In current, privileged domain (domain 0) is only Linux, but is this situation continued in future? I think that users use also other OSs as driver domains. Therefore, I suggested IOMGR. By control mechanism is developed at backend drivers, I/O control setting will become OS independent. -- But, it is needed to implement for other OSs.> The claim that CFQ2 is desktop oriented in this thread > earlier is also not true. The desktop oriented scheduler is > more AS and CFQ2 is widely and successfully used in server > applications.I read on web that "CFQ is made for multimedia desktop usage only” said Andrea. http://www.cs.rice.edu/~ssiyer/r/antsched/shines.html (Perhaps version 1 ?) And following document is found in Kconfig.iosched. config IOSCHED_CFQ tristate "CFQ I/O scheduler" default y ---help--- The CFQ I/O scheduler tries to distribute bandwidth equally among all processes in the system. It should provide a fair working environment, suitable for desktop systems. This is the default I/O scheduler. Therefore, I seem that "CFQ is suitable for desktop system". However, its report is old, and I don''t know surely whether current CFQ is desktop oriented or not. Thanks, -------------------------------------- Satoshi UCHIDA _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> CFQ2 is CFQ version 2 or new CFQ? > In linux documentation ( Documentation/block/ioprio.txt ), I see that CFQ is version 3. > > With the introduction of cfq v3 > If CFQ2 is CFQ version 2, which is current CFQ version 2? or version 3?Linux kernel subsystems rarely have fixed meaningfull major release numbers. They just improve continuously in small steps with each kernel release.> However, CFQ is implemented in only Linux and not in other OSs. > So, privileged domains or driver domains are restricted with only Linux. > In current, privileged domain (domain 0) is only Linux, but is this situation continued in future? > I think that users use also other OSs as driver domains. > Therefore, I suggested IOMGR. > By control mechanism is developed at backend drivers, I/O control setting will become OS independent. > -- But, it is needed to implement for other OSs.Xen depends on Dom0 IO facilities for a lot of things. Following your argument to the logical conclusion you would need to reimplement a lot more code (like firewalling or network QoS) in Xen too. Just doesn''t seem to be a wise investment of precious development resources to me, especially since most Dom0 OSes probably already have mostly equivalent facilities. Of course without hard data it is a stab in the dark. Did you do some benchmarking if there is a real fairness problem currently?> > The claim that CFQ2 is desktop oriented in this thread > > earlier is also not true. The desktop oriented scheduler is > > more AS and CFQ2 is widely and successfully used in server > > applications. > > I read on web that "CFQ is made for multimedia desktop usage only” said Andrea. > http://www.cs.rice.edu/~ssiyer/r/antsched/shines.html > (Perhaps version 1 ?)Perhaps.> And following document is found in Kconfig.iosched. > config IOSCHED_CFQ > tristate "CFQ I/O scheduler" > default y > ---help--- > The CFQ I/O scheduler tries to distribute bandwidth equally > among all processes in the system. It should provide a fair > working environment, suitable for desktop systems. > This is the default I/O scheduler. > > Therefore, I seem that "CFQ is suitable for desktop system". > However, its report is old, and I don''t know surely whether current CFQ is desktop oriented or not.First server (and as well as desktop) workloads are quite varied. It is hard to say anything ultimative and there can be always special cases and exceptions. Also desktop and server workloads can overlap. The main difference is probably just that servers tend to have a lot more spindles, but that normally just requires less scheduling then. CFQ2 is the default scheduler in the main server oriented distributions (SLES, RHEL) so it''s not doing that badly. It''s also default in most desktop oriented distributions (Fedora, OpenSUSE, Ubuntu, etc.) as far as I know. The main reason why AS (which showing nice gains in some ways) didn''t really catch on was that it was somewhat slower in some server workloads In some cases special workloads prefer other schedulers, but they often want the noop scheduler then (e.g. some highend databases tend to prefer to be not scheduled by the OS at all while others like MySQL tend to prefer some help from the OS) -Andi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel