Core parking is a power control feature and it can co-work with NPTM to control system power budget through online/offline some CPUs in the system. These patches implement core parking feature for xen. They consist of 2 parts: dom0 patches and xen hypervisor patches. At dom0 side, patches include [Patch 1/3] intercept native pad (Processor Aggregator Device) logic, providing a native interface for natvie platform and a paravirt template for paravirt platform, so that os can implicitly hook to proper ops accordingly; [Patch 2/3] redirect paravirt template to Xen pv ops; [Patch 3/3] implement Xen pad logic, and when getting pad device notification, it hypercalls to Xen hypervisor for core parking. Due to the characteristic of xen continue_hypercall_on_cpu, dom0 seperately send/get core parking request/result; At Xen hypervisor side, patches include [Patch 1/2] implement hypercall through which dom0 send core parking request, and get core parking result; [Patch 2/2] implement Xen core parking. Different core parking sequence has different power/performance result, due to cpu socket/core/thread topology. This patch provide power-first and performance-first policies, users can choose core parking policy on their own demand, considering power and performance tradeoff. cc Shaohua, core parking feature author at linux kernel side. Thanks, Jinsong
>>> On 17.02.12 at 09:54, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: > Core parking is a power control feature and it can co-work with NPTM to > control system power budget through online/offline some CPUs in the system. > These patches implement core parking feature for xen. They consist of 2 > parts: dom0 patches and xen hypervisor patches. > > At dom0 side, patches include > [Patch 1/3] intercept native pad (Processor Aggregator Device) logic, > providing a native interface for natvie platform and a paravirt template for > paravirt platform, so that os can implicitly hook to proper ops accordingly; > [Patch 2/3] redirect paravirt template to Xen pv ops; > [Patch 3/3] implement Xen pad logic, and when getting pad device > notification, it hypercalls to Xen hypervisor for core parking. Due to the > characteristic of xen continue_hypercall_on_cpu, dom0 seperately send/get > core parking request/result; > > At Xen hypervisor side, patches include > [Patch 1/2] implement hypercall through which dom0 send core parking > request, and get core parking result; > [Patch 2/2] implement Xen core parking. Different core parking sequence has > different power/performance result, due to cpu socket/core/thread topology. > This patch provide power-first and performance-first policies, users can choose > core parking policy on their own demand, considering power and performance > tradeoff.Does this really need to be implemented in the hypervisor? All this boils down to is a wrapper around cpu_down() and cpu_up(), which have hypercall interfaces already. So I''d rather see this as being an extension to Dom0''s pCPU management patches (which aren''t upstream afaict)... Jan
Jan Beulich wrote:>>>> On 17.02.12 at 09:54, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >> Core parking is a power control feature and it can co-work with NPTM >> to control system power budget through online/offline some CPUs in >> the system. These patches implement core parking feature for xen. >> They consist of 2 >> parts: dom0 patches and xen hypervisor patches. >> >> At dom0 side, patches include >> [Patch 1/3] intercept native pad (Processor Aggregator Device) logic, >> providing a native interface for natvie platform and a paravirt >> template for paravirt platform, so that os can implicitly hook to >> proper ops accordingly; [Patch 2/3] redirect paravirt template to >> Xen pv ops; [Patch 3/3] implement Xen pad logic, and when getting >> pad device >> notification, it hypercalls to Xen hypervisor for core parking. Due >> to the characteristic of xen continue_hypercall_on_cpu, dom0 >> seperately send/get >> core parking request/result; >> >> At Xen hypervisor side, patches include >> [Patch 1/2] implement hypercall through which dom0 send core parking >> request, and get core parking result; >> [Patch 2/2] implement Xen core parking. Different core parking >> sequence has different power/performance result, due to cpu >> socket/core/thread topology. This patch provide power-first and >> performance-first policies, users can choose core parking policy on >> their own demand, considering power and performance tradeoff. > > Does this really need to be implemented in the hypervisor? All this > boils down to is a wrapper around cpu_down() and cpu_up(), which > have hypercall interfaces already. So I''d rather see this as being > an extension to Dom0''s pCPU management patches (which aren''t > upstream afaict)... > > JanIt''s a design choice. Core parking is not only a wrapper around cpu_down/up, it also involves policy algorithms which depend on physical cpu topology and cpu_online/present_map, etc. Implement core parking at dom0 side need expose all those information to dom0, with potential issues (like coherence), while dom0 still need do same work as hypervisor. Our idea is to keep dom0 as ACPI parser, then hypercall and do rest things at hypervisor side. Thanks, Jinsong
>>> On 17.02.12 at 18:48, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: > Jan Beulich wrote: >>>>> On 17.02.12 at 09:54, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >>> Core parking is a power control feature and it can co-work with NPTM >>> to control system power budget through online/offline some CPUs in >>> the system. These patches implement core parking feature for xen. >>> They consist of 2 >>> parts: dom0 patches and xen hypervisor patches. >>> >>> At dom0 side, patches include >>> [Patch 1/3] intercept native pad (Processor Aggregator Device) logic, >>> providing a native interface for natvie platform and a paravirt >>> template for paravirt platform, so that os can implicitly hook to >>> proper ops accordingly; [Patch 2/3] redirect paravirt template to >>> Xen pv ops; [Patch 3/3] implement Xen pad logic, and when getting >>> pad device >>> notification, it hypercalls to Xen hypervisor for core parking. Due >>> to the characteristic of xen continue_hypercall_on_cpu, dom0 >>> seperately send/get >>> core parking request/result; >>> >>> At Xen hypervisor side, patches include >>> [Patch 1/2] implement hypercall through which dom0 send core parking >>> request, and get core parking result; >>> [Patch 2/2] implement Xen core parking. Different core parking >>> sequence has different power/performance result, due to cpu >>> socket/core/thread topology. This patch provide power-first and >>> performance-first policies, users can choose core parking policy on >>> their own demand, considering power and performance tradeoff. >> >> Does this really need to be implemented in the hypervisor? All this >> boils down to is a wrapper around cpu_down() and cpu_up(), which >> have hypercall interfaces already. So I''d rather see this as being >> an extension to Dom0''s pCPU management patches (which aren''t >> upstream afaict)... >> >> Jan > > It''s a design choice. Core parking is not only a wrapper around cpu_down/up, > it also involves policy algorithms which depend on physical cpu topology and > cpu_online/present_map, etc. Implement core parking at dom0 side need expose > all those information to dom0, with potential issues (like coherence), while > dom0 still need do same work as hypervisor. > Our idea is to keep dom0 as ACPI parser, then hypercall and do rest things > at hypervisor side.Actually, after some more thought, I don''t even think this ought to be implemented in the Dom0 kernel, but in user space altogether. Afaict all information necessary is already being exposed. Jan
Jan Beulich wrote:>>>> On 17.02.12 at 18:48, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >> Jan Beulich wrote: >>>>>> On 17.02.12 at 09:54, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>> wrote: >>>> Core parking is a power control feature and it can co-work with >>>> NPTM to control system power budget through online/offline some >>>> CPUs in the system. These patches implement core parking feature >>>> for xen. They consist of 2 parts: dom0 patches and xen hypervisor >>>> patches. >>>> >>>> At dom0 side, patches include >>>> [Patch 1/3] intercept native pad (Processor Aggregator Device) >>>> logic, providing a native interface for natvie platform and a >>>> paravirt template for paravirt platform, so that os can implicitly >>>> hook to proper ops accordingly; [Patch 2/3] redirect paravirt >>>> template to Xen pv ops; [Patch 3/3] implement Xen pad logic, and >>>> when getting pad device notification, it hypercalls to Xen >>>> hypervisor for core parking. Due to the characteristic of xen >>>> continue_hypercall_on_cpu, dom0 seperately send/get core parking >>>> request/result; >>>> >>>> At Xen hypervisor side, patches include >>>> [Patch 1/2] implement hypercall through which dom0 send core >>>> parking request, and get core parking result; >>>> [Patch 2/2] implement Xen core parking. Different core parking >>>> sequence has different power/performance result, due to cpu >>>> socket/core/thread topology. This patch provide power-first and >>>> performance-first policies, users can choose core parking policy on >>>> their own demand, considering power and performance tradeoff. >>> >>> Does this really need to be implemented in the hypervisor? All this >>> boils down to is a wrapper around cpu_down() and cpu_up(), which >>> have hypercall interfaces already. So I''d rather see this as being >>> an extension to Dom0''s pCPU management patches (which aren''t >>> upstream afaict)... >>> >>> Jan >> >> It''s a design choice. Core parking is not only a wrapper around >> cpu_down/up, it also involves policy algorithms which depend on >> physical cpu topology and cpu_online/present_map, etc. Implement >> core parking at dom0 side need expose all those information to dom0, >> with potential issues (like coherence), while dom0 still need do >> same work as hypervisor. >> Our idea is to keep dom0 as ACPI parser, then hypercall and do rest >> things at hypervisor side. > > Actually, after some more thought, I don''t even think this ought to > be implemented in the Dom0 kernel, but in user space altogether. > Afaict all information necessary is already being exposed. >No, user space lack necessary information. If I didn''t misunderstand, it has some dom0-side dependencies not ready now, like 1. sysfs interface, and exposing xen pcpu topology and maps; 2. intecept pad notify and call usermodehelper; 3. a daemon to monitor/policy core parking (daemon enable when linux run as pvops under xen (kernel acpi_pad disable now), daemon disable when linux run under baremetal (kernel acpi_pad enable now)) Seems keep same approach as native kernel which handle acpi_pad in kernel side (for us, in hypervisor side) is a reasonable choice. Per my understanding core parking is a co-work part of NPTM, the whole process is basically a remote controller-microengine-bios-kernel process, not necessarily involve user action. Thanks, Jinsong
Liu, Jinsong wrote:> Jan Beulich wrote: >>>>> On 17.02.12 at 18:48, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>> wrote: >>> Jan Beulich wrote: >>>>>>> On 17.02.12 at 09:54, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>>> wrote: >>>>> Core parking is a power control feature and it can co-work with >>>>> NPTM to control system power budget through online/offline some >>>>> CPUs in the system. These patches implement core parking feature >>>>> for xen. They consist of 2 parts: dom0 patches and xen hypervisor >>>>> patches. >>>>> >>>>> At dom0 side, patches include >>>>> [Patch 1/3] intercept native pad (Processor Aggregator Device) >>>>> logic, providing a native interface for natvie platform and a >>>>> paravirt template for paravirt platform, so that os can implicitly >>>>> hook to proper ops accordingly; [Patch 2/3] redirect paravirt >>>>> template to Xen pv ops; [Patch 3/3] implement Xen pad logic, and >>>>> when getting pad device notification, it hypercalls to Xen >>>>> hypervisor for core parking. Due to the characteristic of xen >>>>> continue_hypercall_on_cpu, dom0 seperately send/get core parking >>>>> request/result; >>>>> >>>>> At Xen hypervisor side, patches include >>>>> [Patch 1/2] implement hypercall through which dom0 send core >>>>> parking request, and get core parking result; >>>>> [Patch 2/2] implement Xen core parking. Different core parking >>>>> sequence has different power/performance result, due to cpu >>>>> socket/core/thread topology. This patch provide power-first and >>>>> performance-first policies, users can choose core parking policy >>>>> on their own demand, considering power and performance tradeoff. >>>> >>>> Does this really need to be implemented in the hypervisor? All this >>>> boils down to is a wrapper around cpu_down() and cpu_up(), which >>>> have hypercall interfaces already. So I''d rather see this as being >>>> an extension to Dom0''s pCPU management patches (which aren''t >>>> upstream afaict)... >>>> >>>> Jan >>> >>> It''s a design choice. Core parking is not only a wrapper around >>> cpu_down/up, it also involves policy algorithms which depend on >>> physical cpu topology and cpu_online/present_map, etc. Implement >>> core parking at dom0 side need expose all those information to dom0, >>> with potential issues (like coherence), while dom0 still need do >>> same work as hypervisor. Our idea is to keep dom0 as ACPI parser, >>> then hypercall and do rest things at hypervisor side. >> >> Actually, after some more thought, I don''t even think this ought to >> be implemented in the Dom0 kernel, but in user space altogether. >> Afaict all information necessary is already being exposed. >> > > No, user space lack necessary information. If I didn''t misunderstand, > it has some dom0-side dependencies not ready now, like > 1. sysfs interface, and exposing xen pcpu topology and maps; > 2. intecept pad notify and call usermodehelper; > 3. a daemon to monitor/policy core parking (daemon enable when linux > run as pvops under xen (kernel acpi_pad disable now), daemon disable > when linux run under baremetal (kernel acpi_pad enable now)) > > Seems keep same approach as native kernel which handle acpi_pad in > kernel side (for us, in hypervisor side) is a reasonable choice. Per > my understanding core parking is a co-work part of NPTM, the whole > process is basically a remote controller-microengine-bios-kernel > process, not necessarily involve user action. >Any comments? Thanks, Jinsong
Liu, Jinsong wrote:> Liu, Jinsong wrote: >> Jan Beulich wrote: >>>>>> On 17.02.12 at 18:48, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>> wrote: >>>> Jan Beulich wrote: >>>>>>>> On 17.02.12 at 09:54, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>>>> wrote: >>>>>> Core parking is a power control feature and it can co-work with >>>>>> NPTM to control system power budget through online/offline some >>>>>> CPUs in the system. These patches implement core parking feature >>>>>> for xen. They consist of 2 parts: dom0 patches and xen >>>>>> hypervisor patches. >>>>>> >>>>>> At dom0 side, patches include >>>>>> [Patch 1/3] intercept native pad (Processor Aggregator Device) >>>>>> logic, providing a native interface for natvie platform and a >>>>>> paravirt template for paravirt platform, so that os can >>>>>> implicitly hook to proper ops accordingly; [Patch 2/3] redirect >>>>>> paravirt template to Xen pv ops; [Patch 3/3] implement Xen pad >>>>>> logic, and when getting pad device notification, it hypercalls >>>>>> to Xen hypervisor for core parking. Due to the characteristic of >>>>>> xen continue_hypercall_on_cpu, dom0 seperately send/get core >>>>>> parking request/result; >>>>>> >>>>>> At Xen hypervisor side, patches include >>>>>> [Patch 1/2] implement hypercall through which dom0 send core >>>>>> parking request, and get core parking result; >>>>>> [Patch 2/2] implement Xen core parking. Different core parking >>>>>> sequence has different power/performance result, due to cpu >>>>>> socket/core/thread topology. This patch provide power-first and >>>>>> performance-first policies, users can choose core parking policy >>>>>> on their own demand, considering power and performance tradeoff. >>>>> >>>>> Does this really need to be implemented in the hypervisor? All >>>>> this boils down to is a wrapper around cpu_down() and cpu_up(), >>>>> which have hypercall interfaces already. So I''d rather see this >>>>> as being an extension to Dom0''s pCPU management patches (which >>>>> aren''t upstream afaict)... >>>>> >>>>> Jan >>>> >>>> It''s a design choice. Core parking is not only a wrapper around >>>> cpu_down/up, it also involves policy algorithms which depend on >>>> physical cpu topology and cpu_online/present_map, etc. Implement >>>> core parking at dom0 side need expose all those information to >>>> dom0, with potential issues (like coherence), while dom0 still >>>> need do same work as hypervisor. Our idea is to keep dom0 as ACPI >>>> parser, then hypercall and do rest things at hypervisor side. >>> >>> Actually, after some more thought, I don''t even think this ought to >>> be implemented in the Dom0 kernel, but in user space altogether. >>> Afaict all information necessary is already being exposed. >>> >> >> No, user space lack necessary information. If I didn''t misunderstand, >> it has some dom0-side dependencies not ready now, like >> 1. sysfs interface, and exposing xen pcpu topology and maps; >> 2. intecept pad notify and call usermodehelper; >> 3. a daemon to monitor/policy core parking (daemon enable when linux >> run as pvops under xen (kernel acpi_pad disable now), daemon disable >> when linux run under baremetal (kernel acpi_pad enable now)) >> >> Seems keep same approach as native kernel which handle acpi_pad in >> kernel side (for us, in hypervisor side) is a reasonable choice. Per >> my understanding core parking is a co-work part of NPTM, the whole >> process is basically a remote controller-microengine-bios-kernel >> process, not necessarily involve user action. >> > > Any comments? >Sorry, forgot to re-attach patches :-) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
>>> On 29.02.12 at 13:41, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: > Liu, Jinsong wrote: >> Jan Beulich wrote: >>>>>> On 17.02.12 at 18:48, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>> wrote: >>>> Jan Beulich wrote: >>>>>>>> On 17.02.12 at 09:54, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>>>> wrote: >>>>>> Core parking is a power control feature and it can co-work with >>>>>> NPTM to control system power budget through online/offline some >>>>>> CPUs in the system. These patches implement core parking feature >>>>>> for xen. They consist of 2 parts: dom0 patches and xen hypervisor >>>>>> patches. >>>>>> >>>>>> At dom0 side, patches include >>>>>> [Patch 1/3] intercept native pad (Processor Aggregator Device) >>>>>> logic, providing a native interface for natvie platform and a >>>>>> paravirt template for paravirt platform, so that os can implicitly >>>>>> hook to proper ops accordingly; [Patch 2/3] redirect paravirt >>>>>> template to Xen pv ops; [Patch 3/3] implement Xen pad logic, and >>>>>> when getting pad device notification, it hypercalls to Xen >>>>>> hypervisor for core parking. Due to the characteristic of xen >>>>>> continue_hypercall_on_cpu, dom0 seperately send/get core parking >>>>>> request/result; >>>>>> >>>>>> At Xen hypervisor side, patches include >>>>>> [Patch 1/2] implement hypercall through which dom0 send core >>>>>> parking request, and get core parking result; >>>>>> [Patch 2/2] implement Xen core parking. Different core parking >>>>>> sequence has different power/performance result, due to cpu >>>>>> socket/core/thread topology. This patch provide power-first and >>>>>> performance-first policies, users can choose core parking policy >>>>>> on their own demand, considering power and performance tradeoff. >>>>> >>>>> Does this really need to be implemented in the hypervisor? All this >>>>> boils down to is a wrapper around cpu_down() and cpu_up(), which >>>>> have hypercall interfaces already. So I''d rather see this as being >>>>> an extension to Dom0''s pCPU management patches (which aren''t >>>>> upstream afaict)... >>>>> >>>>> Jan >>>> >>>> It''s a design choice. Core parking is not only a wrapper around >>>> cpu_down/up, it also involves policy algorithms which depend on >>>> physical cpu topology and cpu_online/present_map, etc. Implement >>>> core parking at dom0 side need expose all those information to dom0, >>>> with potential issues (like coherence), while dom0 still need do >>>> same work as hypervisor. Our idea is to keep dom0 as ACPI parser, >>>> then hypercall and do rest things at hypervisor side. >>> >>> Actually, after some more thought, I don''t even think this ought to >>> be implemented in the Dom0 kernel, but in user space altogether. >>> Afaict all information necessary is already being exposed. >>> >> >> No, user space lack necessary information. If I didn''t misunderstand, >> it has some dom0-side dependencies not ready now, like >> 1. sysfs interface, and exposing xen pcpu topology and maps; >> 2. intecept pad notify and call usermodehelper; >> 3. a daemon to monitor/policy core parking (daemon enable when linux >> run as pvops under xen (kernel acpi_pad disable now), daemon disable >> when linux run under baremetal (kernel acpi_pad enable now)) >> >> Seems keep same approach as native kernel which handle acpi_pad in >> kernel side (for us, in hypervisor side) is a reasonable choice. Per >> my understanding core parking is a co-work part of NPTM, the whole >> process is basically a remote controller-microengine-bios-kernel >> process, not necessarily involve user action. >> > > Any comments?No - I continue to disagree that this needs to be done outside of user space (the fact that certain necessary kernel pieces aren''t in pv-ops is no excuse, nor is it that native does this in the kernel - that would at most allow for implementing it in the kernel, but still won''t justify doing it in the hypervisor). Jan
Jan Beulich wrote:>>>> On 29.02.12 at 13:41, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >> Liu, Jinsong wrote: >>> Jan Beulich wrote: >>>>>>> On 17.02.12 at 18:48, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>>> wrote: >>>>> Jan Beulich wrote: >>>>>>>>> On 17.02.12 at 09:54, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>>>>> wrote: >>>>>>> Core parking is a power control feature and it can co-work with >>>>>>> NPTM to control system power budget through online/offline some >>>>>>> CPUs in the system. These patches implement core parking feature >>>>>>> for xen. They consist of 2 parts: dom0 patches and xen >>>>>>> hypervisor patches. >>>>>>> >>>>>>> At dom0 side, patches include >>>>>>> [Patch 1/3] intercept native pad (Processor Aggregator Device) >>>>>>> logic, providing a native interface for natvie platform and a >>>>>>> paravirt template for paravirt platform, so that os can >>>>>>> implicitly hook to proper ops accordingly; [Patch 2/3] redirect >>>>>>> paravirt template to Xen pv ops; [Patch 3/3] implement Xen pad >>>>>>> logic, and when getting pad device notification, it hypercalls >>>>>>> to Xen hypervisor for core parking. Due to the characteristic >>>>>>> of xen continue_hypercall_on_cpu, dom0 seperately send/get core >>>>>>> parking request/result; >>>>>>> >>>>>>> At Xen hypervisor side, patches include >>>>>>> [Patch 1/2] implement hypercall through which dom0 send core >>>>>>> parking request, and get core parking result; >>>>>>> [Patch 2/2] implement Xen core parking. Different core parking >>>>>>> sequence has different power/performance result, due to cpu >>>>>>> socket/core/thread topology. This patch provide power-first and >>>>>>> performance-first policies, users can choose core parking policy >>>>>>> on their own demand, considering power and performance tradeoff. >>>>>> >>>>>> Does this really need to be implemented in the hypervisor? All >>>>>> this boils down to is a wrapper around cpu_down() and cpu_up(), >>>>>> which have hypercall interfaces already. So I''d rather see this >>>>>> as being an extension to Dom0''s pCPU management patches (which >>>>>> aren''t upstream afaict)... >>>>>> >>>>>> Jan >>>>> >>>>> It''s a design choice. Core parking is not only a wrapper around >>>>> cpu_down/up, it also involves policy algorithms which depend on >>>>> physical cpu topology and cpu_online/present_map, etc. Implement >>>>> core parking at dom0 side need expose all those information to >>>>> dom0, with potential issues (like coherence), while dom0 still >>>>> need do same work as hypervisor. Our idea is to keep dom0 as ACPI >>>>> parser, then hypercall and do rest things at hypervisor side. >>>> >>>> Actually, after some more thought, I don''t even think this ought to >>>> be implemented in the Dom0 kernel, but in user space altogether. >>>> Afaict all information necessary is already being exposed. >>>> >>> >>> No, user space lack necessary information. If I didn''t >>> misunderstand, it has some dom0-side dependencies not ready now, >>> like >>> 1. sysfs interface, and exposing xen pcpu topology and maps; >>> 2. intecept pad notify and call usermodehelper; >>> 3. a daemon to monitor/policy core parking (daemon enable when linux >>> run as pvops under xen (kernel acpi_pad disable now), daemon disable >>> when linux run under baremetal (kernel acpi_pad enable now)) >>> >>> Seems keep same approach as native kernel which handle acpi_pad in >>> kernel side (for us, in hypervisor side) is a reasonable choice. Per >>> my understanding core parking is a co-work part of NPTM, the whole >>> process is basically a remote controller-microengine-bios-kernel >>> process, not necessarily involve user action. >>> >> >> Any comments? > > No - I continue to disagree that this needs to be done outside of > user space (the fact that certain necessary kernel pieces aren''t in > pv-ops is no excuse, nor is it that native does this in the kernel - > that would at most allow for implementing it in the kernel, but still > won''t justify doing it in the hypervisor). >Jan, could you elaborate more your thoughts? like - the pros of user space approach (and cons, if it has); - the disadvantages of hypervisor approach; Thanks, Jinsong
>>> On 01.03.12 at 09:20, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: > Jan Beulich wrote: >>>>> On 29.02.12 at 13:41, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >>> Liu, Jinsong wrote: >>>> Jan Beulich wrote: >>>>>>>> On 17.02.12 at 18:48, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>>>> wrote: >>>>>> Jan Beulich wrote: >>>>>>>>>> On 17.02.12 at 09:54, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>>>>>> wrote: >>>>>>>> Core parking is a power control feature and it can co-work with >>>>>>>> NPTM to control system power budget through online/offline some >>>>>>>> CPUs in the system. These patches implement core parking feature >>>>>>>> for xen. They consist of 2 parts: dom0 patches and xen >>>>>>>> hypervisor patches. >>>>>>>> >>>>>>>> At dom0 side, patches include >>>>>>>> [Patch 1/3] intercept native pad (Processor Aggregator Device) >>>>>>>> logic, providing a native interface for natvie platform and a >>>>>>>> paravirt template for paravirt platform, so that os can >>>>>>>> implicitly hook to proper ops accordingly; [Patch 2/3] redirect >>>>>>>> paravirt template to Xen pv ops; [Patch 3/3] implement Xen pad >>>>>>>> logic, and when getting pad device notification, it hypercalls >>>>>>>> to Xen hypervisor for core parking. Due to the characteristic >>>>>>>> of xen continue_hypercall_on_cpu, dom0 seperately send/get core >>>>>>>> parking request/result; >>>>>>>> >>>>>>>> At Xen hypervisor side, patches include >>>>>>>> [Patch 1/2] implement hypercall through which dom0 send core >>>>>>>> parking request, and get core parking result; >>>>>>>> [Patch 2/2] implement Xen core parking. Different core parking >>>>>>>> sequence has different power/performance result, due to cpu >>>>>>>> socket/core/thread topology. This patch provide power-first and >>>>>>>> performance-first policies, users can choose core parking policy >>>>>>>> on their own demand, considering power and performance tradeoff. >>>>>>> >>>>>>> Does this really need to be implemented in the hypervisor? All >>>>>>> this boils down to is a wrapper around cpu_down() and cpu_up(), >>>>>>> which have hypercall interfaces already. So I''d rather see this >>>>>>> as being an extension to Dom0''s pCPU management patches (which >>>>>>> aren''t upstream afaict)... >>>>>>> >>>>>>> Jan >>>>>> >>>>>> It''s a design choice. Core parking is not only a wrapper around >>>>>> cpu_down/up, it also involves policy algorithms which depend on >>>>>> physical cpu topology and cpu_online/present_map, etc. Implement >>>>>> core parking at dom0 side need expose all those information to >>>>>> dom0, with potential issues (like coherence), while dom0 still >>>>>> need do same work as hypervisor. Our idea is to keep dom0 as ACPI >>>>>> parser, then hypercall and do rest things at hypervisor side. >>>>> >>>>> Actually, after some more thought, I don''t even think this ought to >>>>> be implemented in the Dom0 kernel, but in user space altogether. >>>>> Afaict all information necessary is already being exposed. >>>>> >>>> >>>> No, user space lack necessary information. If I didn''t >>>> misunderstand, it has some dom0-side dependencies not ready now, >>>> like >>>> 1. sysfs interface, and exposing xen pcpu topology and maps; >>>> 2. intecept pad notify and call usermodehelper; >>>> 3. a daemon to monitor/policy core parking (daemon enable when linux >>>> run as pvops under xen (kernel acpi_pad disable now), daemon disable >>>> when linux run under baremetal (kernel acpi_pad enable now)) >>>> >>>> Seems keep same approach as native kernel which handle acpi_pad in >>>> kernel side (for us, in hypervisor side) is a reasonable choice. Per >>>> my understanding core parking is a co-work part of NPTM, the whole >>>> process is basically a remote controller-microengine-bios-kernel >>>> process, not necessarily involve user action. >>>> >>> >>> Any comments? >> >> No - I continue to disagree that this needs to be done outside of >> user space (the fact that certain necessary kernel pieces aren''t in >> pv-ops is no excuse, nor is it that native does this in the kernel - >> that would at most allow for implementing it in the kernel, but still >> won''t justify doing it in the hypervisor). >> > > Jan, could you elaborate more your thoughts? like > - the pros of user space approach (and cons, if it has); > - the disadvantages of hypervisor approach;Whenever a user space implementation is possible (and not too cumbersome), I think it ought to be preferred. Even more so when it involves policy decisions. Jan
Jan Beulich wrote:>>>> On 01.03.12 at 09:20, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >> Jan Beulich wrote: >>>>>> On 29.02.12 at 13:41, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>> wrote: >>>> Liu, Jinsong wrote: >>>>> Jan Beulich wrote: >>>>>>>>> On 17.02.12 at 18:48, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>>>>> wrote: >>>>>>> Jan Beulich wrote: >>>>>>>>>>> On 17.02.12 at 09:54, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>>>>>>> wrote: >>>>>>>>> Core parking is a power control feature and it can co-work >>>>>>>>> with NPTM to control system power budget through >>>>>>>>> online/offline some CPUs in the system. These patches >>>>>>>>> implement core parking feature for xen. They consist of 2 >>>>>>>>> parts: dom0 patches and xen hypervisor patches. >>>>>>>>> >>>>>>>>> At dom0 side, patches include >>>>>>>>> [Patch 1/3] intercept native pad (Processor Aggregator Device) >>>>>>>>> logic, providing a native interface for natvie platform and a >>>>>>>>> paravirt template for paravirt platform, so that os can >>>>>>>>> implicitly hook to proper ops accordingly; [Patch 2/3] >>>>>>>>> redirect paravirt template to Xen pv ops; [Patch 3/3] >>>>>>>>> implement Xen pad logic, and when getting pad device >>>>>>>>> notification, it hypercalls to Xen hypervisor for core >>>>>>>>> parking. Due to the characteristic of xen >>>>>>>>> continue_hypercall_on_cpu, dom0 seperately send/get core >>>>>>>>> parking request/result; >>>>>>>>> >>>>>>>>> At Xen hypervisor side, patches include >>>>>>>>> [Patch 1/2] implement hypercall through which dom0 send core >>>>>>>>> parking request, and get core parking result; >>>>>>>>> [Patch 2/2] implement Xen core parking. Different core parking >>>>>>>>> sequence has different power/performance result, due to cpu >>>>>>>>> socket/core/thread topology. This patch provide power-first >>>>>>>>> and performance-first policies, users can choose core parking >>>>>>>>> policy on their own demand, considering power and performance >>>>>>>>> tradeoff. >>>>>>>> >>>>>>>> Does this really need to be implemented in the hypervisor? All >>>>>>>> this boils down to is a wrapper around cpu_down() and cpu_up(), >>>>>>>> which have hypercall interfaces already. So I''d rather see this >>>>>>>> as being an extension to Dom0''s pCPU management patches (which >>>>>>>> aren''t upstream afaict)... >>>>>>>> >>>>>>>> Jan >>>>>>> >>>>>>> It''s a design choice. Core parking is not only a wrapper around >>>>>>> cpu_down/up, it also involves policy algorithms which depend on >>>>>>> physical cpu topology and cpu_online/present_map, etc. Implement >>>>>>> core parking at dom0 side need expose all those information to >>>>>>> dom0, with potential issues (like coherence), while dom0 still >>>>>>> need do same work as hypervisor. Our idea is to keep dom0 as >>>>>>> ACPI parser, then hypercall and do rest things at hypervisor >>>>>>> side. >>>>>> >>>>>> Actually, after some more thought, I don''t even think this ought >>>>>> to be implemented in the Dom0 kernel, but in user space >>>>>> altogether. Afaict all information necessary is already being >>>>>> exposed. >>>>>> >>>>> >>>>> No, user space lack necessary information. If I didn''t >>>>> misunderstand, it has some dom0-side dependencies not ready now, >>>>> like >>>>> 1. sysfs interface, and exposing xen pcpu topology and maps; >>>>> 2. intecept pad notify and call usermodehelper; >>>>> 3. a daemon to monitor/policy core parking (daemon enable when >>>>> linux run as pvops under xen (kernel acpi_pad disable now), >>>>> daemon disable when linux run under baremetal (kernel acpi_pad >>>>> enable now)) >>>>> >>>>> Seems keep same approach as native kernel which handle acpi_pad in >>>>> kernel side (for us, in hypervisor side) is a reasonable choice. >>>>> Per my understanding core parking is a co-work part of NPTM, the >>>>> whole process is basically a remote >>>>> controller-microengine-bios-kernel process, not necessarily >>>>> involve user action. >>>>> >>>> >>>> Any comments? >>> >>> No - I continue to disagree that this needs to be done outside of >>> user space (the fact that certain necessary kernel pieces aren''t in >>> pv-ops is no excuse, nor is it that native does this in the kernel - >>> that would at most allow for implementing it in the kernel, but >>> still won''t justify doing it in the hypervisor). >>> >> >> Jan, could you elaborate more your thoughts? like >> - the pros of user space approach (and cons, if it has); >> - the disadvantages of hypervisor approach; > > Whenever a user space implementation is possible (and not too > cumbersome), I think it ought to be preferred. Even more so when it involves policy decisions. >Unfortunately, yes, though cumbersome is not basic reason user space approach is not preferred. Core parking is a power management staff, based on dynamic physical details like cpu topologies and maps owned by hypervisor. It''s natural to implement it at hypervisor side, like what other xen power management staffs do. Based on same reason does native linux choose to implement it at kernel, not at user space. Thanks, Jinsong
>>> On 01.03.12 at 12:14, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: > Unfortunately, yes, though cumbersome is not basic reason user space > approach is not preferred. > Core parking is a power management staff, based on dynamic physical details > like cpu topologies and maps owned by hypervisor. It''s natural to implementCPU topology is available to user space, and as far as I recall your hypervisor patch didn''t really manipulate any maps - all it did was pick what CPU to bring up/down, and then carry out that decision.> it at hypervisor side, like what other xen power management staffs do. BasedOther power management stuff can''t be easily done in user space.> on same reason does native linux choose to implement it at kernel, not at > user space.Sorry, no, I don''t consider Linux having taken (or going to take - after all I can''t find the word "park" when grepping 3.3-rc5''s drivers/acpi/, so I assume the patch isn''t in yet) a questionable decision a reason for Xen to follow blindly. Jan
Jan Beulich wrote:>>>> On 01.03.12 at 12:14, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >> Unfortunately, yes, though cumbersome is not basic reason user space >> approach is not preferred. Core parking is a power management staff, >> based on dynamic physical details like cpu topologies and maps owned >> by hypervisor. It''s natural to implement > > CPU topology is available to user space, and as far as I recall your > hypervisor patch didn''t really manipulate any maps - all it did was > pick what CPU to bring up/down, and then carry out that decision.No. threads_per_core and cores_per_socket exposed to userspace is pointless to us (and, it''s questionable need fixup). Core parking depends on following physical info (no matter where it implement): 1. cpu_online_map; 2. cpu_present_map; 3. cpu_core_mask; 4. cpu_sibling_mask; all of them are *dynamic*, especially, 3/4 are varied per cpu and per online/offline ops. For userspace, consider - userspace need keep syncing with hypervisor, after each online/offline ops and for each cpu; - extending new policy need expose whatever physical info it need; technically it maybe possible to do so, but I doubt it''s really necessary.> >> it at hypervisor side, like what other xen power management staffs >> do. Based > > Other power management stuff can''t be easily done in user space.I think it''s more appropriate not to separate 1 logic into 2 hierarchies.> >> on same reason does native linux choose to implement it at kernel, >> not at user space. > > Sorry, no, I don''t consider Linux having taken (or going to take - > after all I can''t find the word "park" when grepping 3.3-rc5''s > drivers/acpi/, so I assume the patch isn''t in yet) a questionable > decision a reason for > Xen to follow blindly. >Kernel named it as acpi pad (Processor Aggregator Device). Xen hypervisor didn''t name it as ''pad'' because it''s too acpi specific, core parking is a more general name. Thanks, Jinsong
>>> On 01.03.12 at 15:31, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: > Jan Beulich wrote: >>>>> On 01.03.12 at 12:14, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >>> Unfortunately, yes, though cumbersome is not basic reason user space >>> approach is not preferred. Core parking is a power management staff, >>> based on dynamic physical details like cpu topologies and maps owned >>> by hypervisor. It''s natural to implement >> >> CPU topology is available to user space, and as far as I recall your >> hypervisor patch didn''t really manipulate any maps - all it did was >> pick what CPU to bring up/down, and then carry out that decision. > > No. threads_per_core and cores_per_socket exposed to userspace is pointless > to us (and, it''s questionable need fixup).Sure this would be insufficient. But what do you think did XEN_SYSCTL_topologyinfo get added for?> Core parking depends on following physical info (no matter where it > implement): > 1. cpu_online_map; > 2. cpu_present_map; > 3. cpu_core_mask; > 4. cpu_sibling_mask; > all of them are *dynamic*, especially, 3/4 are varied per cpu and per > online/offline ops.Afaict all of these can be reconstructed using (mostly sysctl) hypercalls. Jan
I would really doubt the need to create a new interface of receiving ACPI event and sending to user land (other than existing native kernel) specifically for Xen. What''s the benefit and why kernel people should buy-in that? Core parking is a platform feature, not virtualization feature. Naturally following native approach is the most efficient. Why do you want to create yet another interface for Xen to do that? Shan Haitao 2012/3/1 Jan Beulich <JBeulich@suse.com>:>>>> On 01.03.12 at 15:31, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >> Jan Beulich wrote: >>>>>> On 01.03.12 at 12:14, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >>>> Unfortunately, yes, though cumbersome is not basic reason user space >>>> approach is not preferred. Core parking is a power management staff, >>>> based on dynamic physical details like cpu topologies and maps owned >>>> by hypervisor. It''s natural to implement >>> >>> CPU topology is available to user space, and as far as I recall your >>> hypervisor patch didn''t really manipulate any maps - all it did was >>> pick what CPU to bring up/down, and then carry out that decision. >> >> No. threads_per_core and cores_per_socket exposed to userspace is pointless >> to us (and, it''s questionable need fixup). > > Sure this would be insufficient. But what do you think did > XEN_SYSCTL_topologyinfo get added for? > >> Core parking depends on following physical info (no matter where it >> implement): >> 1. cpu_online_map; >> 2. cpu_present_map; >> 3. cpu_core_mask; >> 4. cpu_sibling_mask; >> all of them are *dynamic*, especially, 3/4 are varied per cpu and per >> online/offline ops. > > Afaict all of these can be reconstructed using (mostly sysctl) > hypercalls. > > Jan > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On 02/03/2012 09:42, "Haitao Shan" <maillists.shan@gmail.com> wrote:> I would really doubt the need to create a new interface of receiving > ACPI event and sending to user land (other than existing native > kernel) specifically for Xen. What''s the benefit and why kernel people > should buy-in that? > Core parking is a platform feature, not virtualization feature. > Naturally following native approach is the most efficient. Why do you > want to create yet another interface for Xen to do that?While I sympathise with your position rather more than Jan does, the fact is that it''s *you* who are suggesting yet-another-Xen-interface. Whereas doing it in userspace requires only existing hypercalls I believe. -- Keir> Shan Haitao > > 2012/3/1 Jan Beulich <JBeulich@suse.com>: >>>>> On 01.03.12 at 15:31, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >>> Jan Beulich wrote: >>>>>>> On 01.03.12 at 12:14, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >>>>> Unfortunately, yes, though cumbersome is not basic reason user space >>>>> approach is not preferred. Core parking is a power management staff, >>>>> based on dynamic physical details like cpu topologies and maps owned >>>>> by hypervisor. It''s natural to implement >>>> >>>> CPU topology is available to user space, and as far as I recall your >>>> hypervisor patch didn''t really manipulate any maps - all it did was >>>> pick what CPU to bring up/down, and then carry out that decision. >>> >>> No. threads_per_core and cores_per_socket exposed to userspace is pointless >>> to us (and, it''s questionable need fixup). >> >> Sure this would be insufficient. But what do you think did >> XEN_SYSCTL_topologyinfo get added for? >> >>> Core parking depends on following physical info (no matter where it >>> implement): >>> 1. cpu_online_map; >>> 2. cpu_present_map; >>> 3. cpu_core_mask; >>> 4. cpu_sibling_mask; >>> all of them are *dynamic*, especially, 3/4 are varied per cpu and per >>> online/offline ops. >> >> Afaict all of these can be reconstructed using (mostly sysctl) >> hypercalls. >> >> Jan >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel
>>> On 02.03.12 at 10:42, Haitao Shan <maillists.shan@gmail.com> wrote: > I would really doubt the need to create a new interface of receiving > ACPI event and sending to user land (other than existing native > kernel) specifically for Xen. What''s the benefit and why kernel people > should buy-in that?Receiving ACPI events??? I don''t recall having seen anything like that in the proposed patch (since if that was the case, I would probably agree that this is better done in kernel/hypervisor). Was what got sent out perhaps just a small fraction of what is intended, and was it forgotten to mention this? Jan> Core parking is a platform feature, not virtualization feature. > Naturally following native approach is the most efficient. Why do you > want to create yet another interface for Xen to do that? > > Shan Haitao > > 2012/3/1 Jan Beulich <JBeulich@suse.com>: >>>>> On 01.03.12 at 15:31, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >>> Jan Beulich wrote: >>>>>>> On 01.03.12 at 12:14, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >>>>> Unfortunately, yes, though cumbersome is not basic reason user space >>>>> approach is not preferred. Core parking is a power management staff, >>>>> based on dynamic physical details like cpu topologies and maps owned >>>>> by hypervisor. It''s natural to implement >>>> >>>> CPU topology is available to user space, and as far as I recall your >>>> hypervisor patch didn''t really manipulate any maps - all it did was >>>> pick what CPU to bring up/down, and then carry out that decision. >>> >>> No. threads_per_core and cores_per_socket exposed to userspace is pointless >>> to us (and, it''s questionable need fixup). >> >> Sure this would be insufficient. But what do you think did >> XEN_SYSCTL_topologyinfo get added for? >> >>> Core parking depends on following physical info (no matter where it >>> implement): >>> 1. cpu_online_map; >>> 2. cpu_present_map; >>> 3. cpu_core_mask; >>> 4. cpu_sibling_mask; >>> all of them are *dynamic*, especially, 3/4 are varied per cpu and per >>> online/offline ops. >> >> Afaict all of these can be reconstructed using (mostly sysctl) >> hypercalls. >> >> Jan >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel
Jan Beulich wrote:>>>> On 02.03.12 at 10:42, Haitao Shan <maillists.shan@gmail.com> wrote: >> I would really doubt the need to create a new interface of receiving >> ACPI event and sending to user land (other than existing native >> kernel) specifically for Xen. What''s the benefit and why kernel >> people should buy-in that? > > Receiving ACPI events??? I don''t recall having seen anything like that > in the proposed patch (since if that was the case, I would probably > agree that this is better done in kernel/hypervisor). Was what got > sent out perhaps just a small fraction of what is intended, and was it > forgotten to mention this? > > JanYes, it does need to intercept ACPI PAD notify then handle it similar way as native kernel did. The logic is at dom0 ''[PATCH 3/3] Xen pad logic and notification'' as attached. Sorry I didn''t give explicit and detailed explanation at the patch description (defaultly I thought you know native acpi pad, so I ignored the background story :-) I will add more detailed description at dom0 patch 3/3. Thanks, Jinsong> >> Core parking is a platform feature, not virtualization feature. >> Naturally following native approach is the most efficient. Why do you >> want to create yet another interface for Xen to do that? >> >> Shan Haitao >> >> 2012/3/1 Jan Beulich <JBeulich@suse.com>: >>>>>> On 01.03.12 at 15:31, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>> wrote: >>>> Jan Beulich wrote: >>>>>>>> On 01.03.12 at 12:14, "Liu, Jinsong" <jinsong.liu@intel.com> >>>>>>>> wrote: >>>>>> Unfortunately, yes, though cumbersome is not basic reason user >>>>>> space approach is not preferred. Core parking is a power >>>>>> management staff, based on dynamic physical details like cpu >>>>>> topologies and maps owned by hypervisor. It''s natural to >>>>>> implement >>>>> >>>>> CPU topology is available to user space, and as far as I recall >>>>> your hypervisor patch didn''t really manipulate any maps - all it >>>>> did was pick what CPU to bring up/down, and then carry out that >>>>> decision. >>>> >>>> No. threads_per_core and cores_per_socket exposed to userspace is >>>> pointless to us (and, it''s questionable need fixup). >>> >>> Sure this would be insufficient. But what do you think did >>> XEN_SYSCTL_topologyinfo get added for? >>> >>>> Core parking depends on following physical info (no matter where >>>> it implement): >>>> 1. cpu_online_map; >>>> 2. cpu_present_map; >>>> 3. cpu_core_mask; >>>> 4. cpu_sibling_mask; >>>> all of them are *dynamic*, especially, 3/4 are varied per cpu and >>>> per online/offline ops. >>> >>> Afaict all of these can be reconstructed using (mostly sysctl) >>> hypercalls. >>> >>> Jan >>> >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xen.org >>> http://lists.xen.org/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
>>> On 04.03.12 at 16:48, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: > Yes, it does need to intercept ACPI PAD notify then handle it similar way as > native kernel did. The logic is at dom0 ''[PATCH 3/3] Xen pad logic and > notification'' as attached.And I know now why I didn''t look at this in any detail - it modifies a file that doesn''t even exist in 3.3-rc5 (drivers/xen/xen_acpi_pad.c), nor does it mention what tree/branch this would be against. Plus, ACPI_PROCESSOR_AGGREGATOR is a tristate symbol, yet your code assumes it to be boolean. But yes, given the way this is implemented I withdraw my reservations against the hypervisor side changes, as doing this completely in the Dom0 kernel would be cumbersome (due to the need to call sysctl functions), and doing it entirely in user space would indeed only be justified if the native kernel did it this way too. Jan
Jan Beulich wrote:>>>> On 04.03.12 at 16:48, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >> Yes, it does need to intercept ACPI PAD notify then handle it >> similar way as native kernel did. The logic is at dom0 ''[PATCH 3/3] >> Xen pad logic and notification'' as attached. > > And I know now why I didn''t look at this in any detail - it modifies a > file that doesn''t even exist in 3.3-rc5 (drivers/xen/xen_acpi_pad.c), > nor does it mention what tree/branch this would be against. Plus, > ACPI_PROCESSOR_AGGREGATOR is a tristate symbol, yet your code > assumes it to be boolean.Yes, tristate/bool is an issue of core parking dom0-side patches which has involved some discussion how to solve it in a reasonable way (as attached). I will update it later.> > But yes, given the way this is implemented I withdraw my reservations > against the hypervisor side changes, as doing this completely in the > Dom0 kernel would be cumbersome (due to the need to call sysctl > functions), and doing it entirely in user space would indeed only be > justified if the native kernel did it this way too. > > JanThanks Jan, Keir, and Haitao! _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Thanks, Keir. We did create new hypercalls. But for new interface(mentioned in my previous mail), I mean the mechanisms for kernel to notify user-space for core parking decision. This does *not* exist in kernel. If we add it specifically for Xen, I don''t think kernel people would buy-in that. Shan Haitao 2012/3/2 Keir Fraser <keir@xen.org>:> On 02/03/2012 09:42, "Haitao Shan" <maillists.shan@gmail.com> wrote: > >> I would really doubt the need to create a new interface of receiving >> ACPI event and sending to user land (other than existing native >> kernel) specifically for Xen. What''s the benefit and why kernel people >> should buy-in that? >> Core parking is a platform feature, not virtualization feature. >> Naturally following native approach is the most efficient. Why do you >> want to create yet another interface for Xen to do that? > > While I sympathise with your position rather more than Jan does, the fact is > that it''s *you* who are suggesting yet-another-Xen-interface. Whereas doing > it in userspace requires only existing hypercalls I believe. > > -- Keir > >> Shan Haitao >> >> 2012/3/1 Jan Beulich <JBeulich@suse.com>: >>>>>> On 01.03.12 at 15:31, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >>>> Jan Beulich wrote: >>>>>>>> On 01.03.12 at 12:14, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >>>>>> Unfortunately, yes, though cumbersome is not basic reason user space >>>>>> approach is not preferred. Core parking is a power management staff, >>>>>> based on dynamic physical details like cpu topologies and maps owned >>>>>> by hypervisor. It''s natural to implement >>>>> >>>>> CPU topology is available to user space, and as far as I recall your >>>>> hypervisor patch didn''t really manipulate any maps - all it did was >>>>> pick what CPU to bring up/down, and then carry out that decision. >>>> >>>> No. threads_per_core and cores_per_socket exposed to userspace is pointless >>>> to us (and, it''s questionable need fixup). >>> >>> Sure this would be insufficient. But what do you think did >>> XEN_SYSCTL_topologyinfo get added for? >>> >>>> Core parking depends on following physical info (no matter where it >>>> implement): >>>> 1. cpu_online_map; >>>> 2. cpu_present_map; >>>> 3. cpu_core_mask; >>>> 4. cpu_sibling_mask; >>>> all of them are *dynamic*, especially, 3/4 are varied per cpu and per >>>> online/offline ops. >>> >>> Afaict all of these can be reconstructed using (mostly sysctl) >>> hypercalls. >>> >>> Jan >>> >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xen.org >>> http://lists.xen.org/xen-devel > >
Sorry, yes, I also had missed the ACPI interaction. On 05/03/2012 21:27, "Haitao Shan" <maillists.shan@gmail.com> wrote:> Thanks, Keir. We did create new hypercalls. > But for new interface(mentioned in my previous mail), I mean the > mechanisms for kernel to notify user-space for core parking decision. > This does *not* exist in kernel. If we add it specifically for Xen, I > don''t think kernel people would buy-in that. > > Shan Haitao > > 2012/3/2 Keir Fraser <keir@xen.org>: >> On 02/03/2012 09:42, "Haitao Shan" <maillists.shan@gmail.com> wrote: >> >>> I would really doubt the need to create a new interface of receiving >>> ACPI event and sending to user land (other than existing native >>> kernel) specifically for Xen. What''s the benefit and why kernel people >>> should buy-in that? >>> Core parking is a platform feature, not virtualization feature. >>> Naturally following native approach is the most efficient. Why do you >>> want to create yet another interface for Xen to do that? >> >> While I sympathise with your position rather more than Jan does, the fact is >> that it''s *you* who are suggesting yet-another-Xen-interface. Whereas doing >> it in userspace requires only existing hypercalls I believe. >> >> -- Keir >> >>> Shan Haitao >>> >>> 2012/3/1 Jan Beulich <JBeulich@suse.com>: >>>>>>> On 01.03.12 at 15:31, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >>>>> Jan Beulich wrote: >>>>>>>>> On 01.03.12 at 12:14, "Liu, Jinsong" <jinsong.liu@intel.com> wrote: >>>>>>> Unfortunately, yes, though cumbersome is not basic reason user space >>>>>>> approach is not preferred. Core parking is a power management staff, >>>>>>> based on dynamic physical details like cpu topologies and maps owned >>>>>>> by hypervisor. It''s natural to implement >>>>>> >>>>>> CPU topology is available to user space, and as far as I recall your >>>>>> hypervisor patch didn''t really manipulate any maps - all it did was >>>>>> pick what CPU to bring up/down, and then carry out that decision. >>>>> >>>>> No. threads_per_core and cores_per_socket exposed to userspace is >>>>> pointless >>>>> to us (and, it''s questionable need fixup). >>>> >>>> Sure this would be insufficient. But what do you think did >>>> XEN_SYSCTL_topologyinfo get added for? >>>> >>>>> Core parking depends on following physical info (no matter where it >>>>> implement): >>>>> 1. cpu_online_map; >>>>> 2. cpu_present_map; >>>>> 3. cpu_core_mask; >>>>> 4. cpu_sibling_mask; >>>>> all of them are *dynamic*, especially, 3/4 are varied per cpu and per >>>>> online/offline ops. >>>> >>>> Afaict all of these can be reconstructed using (mostly sysctl) >>>> hypercalls. >>>> >>>> Jan >>>> >>>> >>>> _______________________________________________ >>>> Xen-devel mailing list >>>> Xen-devel@lists.xen.org >>>> http://lists.xen.org/xen-devel >> >>