Hi list, Is anybody working on virtualization of the CPU Performance Monitoring Unit? There are 2 PMU related projects listed on GSoC 2012 page. http://wiki.xen.org/wiki/Archived/GSoC_2012_Ideas - Virtualization of the CPU Performance Monitoring Unit - Perf support for Xen I''m interested on these 2 projects. Regards, Lin Ming
On 04/06/2012 10:41 AM, Lin Ming wrote:> Hi list, > > Is anybody working on virtualization of the CPU Performance Monitoring Unit? > > There are 2 PMU related projects listed on GSoC 2012 page. > http://wiki.xen.org/wiki/Archived/GSoC_2012_Ideas > > - Virtualization of the CPU Performance Monitoring Unit > - Perf support for Xen > > I''m interested on these 2 projects.Hi Lin, 1. I don''t think Xen was accepted as an organization for 2012 GSOC. See http://lists.xen.org/archives/html/xen-devel/2012-03/msg02080.html. 2. The PMU project description in the wiki is vague. I know HVM guests support virtualized PMU. Please check vpmu.c files in /hvm, /svm, and /vmx directories. You better ask mentors for details (maybe this is XCP specific?). -Wei> > Regards, > Lin Ming > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >
Konrad Rzeszutek Wilk
2012-Apr-06 18:36 UTC
Re: Virtualization of the CPU Performance Monitoring Unit
On Fri, Apr 06, 2012 at 11:43:00AM -0500, Wei Huang wrote:> On 04/06/2012 10:41 AM, Lin Ming wrote: > >Hi list, > > > >Is anybody working on virtualization of the CPU Performance Monitoring Unit? > > > >There are 2 PMU related projects listed on GSoC 2012 page. > >http://wiki.xen.org/wiki/Archived/GSoC_2012_Ideas > > > >- Virtualization of the CPU Performance Monitoring Unit > >- Perf support for Xen > > > >I''m interested on these 2 projects. > Hi Lin, > > 1. I don''t think Xen was accepted as an organization for 2012 GSOC. > See > http://lists.xen.org/archives/html/xen-devel/2012-03/msg02080.html. > 2. The PMU project description in the wiki is vague. I know HVM > guests support virtualized PMU. Please check vpmu.c files in /hvm, > /svm, and /vmx directories. You better ask mentors for details > (maybe this is XCP specific?).This is dom0 related. So being able to use perf to instrument the hypervisor (and the guests from dom0).
On Sat, Apr 7, 2012 at 12:43 AM, Wei Huang <wei.huang2@amd.com> wrote:> On 04/06/2012 10:41 AM, Lin Ming wrote: >> >> Hi list, >> >> Is anybody working on virtualization of the CPU Performance Monitoring >> Unit? >> >> There are 2 PMU related projects listed on GSoC 2012 page. >> http://wiki.xen.org/wiki/Archived/GSoC_2012_Ideas >> >> - Virtualization of the CPU Performance Monitoring Unit >> - Perf support for Xen >> >> I''m interested on these 2 projects. > > Hi Lin, > > 1. I don''t think Xen was accepted as an organization for 2012 GSOC. See > http://lists.xen.org/archives/html/xen-devel/2012-03/msg02080.html.It doesn''t matter. I just want to find something valuable to do.> 2. The PMU project description in the wiki is vague. I know HVM guests > support virtualized PMU. Please check vpmu.c files in /hvm, /svm, and /vmx > directories. You better ask mentors for details (maybe this is XCP > specific?).(CC mentors) I tested PMU/Perf support with xen-unstable, dom0 3.3 kernel, domU 3.4-rc1 kernel. Here is the result on an Intel SandyBrige machine. - In dom0 # dmesg |grep -i "Performance Event" Performance Events: unsupported p6 CPU model 42 no PMU driver, software events only. Hardware events are not supported yet in dom0. - In domU # dmesg |grep -i "Performance Events" Performance Events: 16-deep LBR, SandyBridge events, Intel PMU driver. Seems domU has support for hardware events. But "perf" does not work on domU. Run "perf top", but no data was collected. # cat /proc/interrupts |grep "PMI" PMI: 0 0 Performance monitoring interrupts No PMU interrupts. BTW, I also tested KVM-Qemu. "perf" works well on KVM-Qemu. Thanks, Lin Ming
On Sat, Apr 7, 2012 at 2:36 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:> On Fri, Apr 06, 2012 at 11:43:00AM -0500, Wei Huang wrote: >> On 04/06/2012 10:41 AM, Lin Ming wrote: >> >Hi list, >> > >> >Is anybody working on virtualization of the CPU Performance Monitoring Unit? >> > >> >There are 2 PMU related projects listed on GSoC 2012 page. >> >http://wiki.xen.org/wiki/Archived/GSoC_2012_Ideas >> > >> >- Virtualization of the CPU Performance Monitoring Unit >> >- Perf support for Xen >> > >> >I''m interested on these 2 projects. >> Hi Lin, >> >> 1. I don''t think Xen was accepted as an organization for 2012 GSOC. >> See >> http://lists.xen.org/archives/html/xen-devel/2012-03/msg02080.html. >> 2. The PMU project description in the wiki is vague. I know HVM >> guests support virtualized PMU. Please check vpmu.c files in /hvm, >> /svm, and /vmx directories. You better ask mentors for details >> (maybe this is XCP specific?). > > This is dom0 related. So being able to use perf to instrument > the hypervisor (and the guests from dom0).For guests instrument: How about use perf on guests(domU) directly? Thanks, Lin Ming
Ian Campbell
2012-Apr-10 09:52 UTC
Re: Virtualization of the CPU Performance Monitoring Unit
On Sun, 2012-04-08 at 07:58 +0100, Lin Ming wrote:> On Sat, Apr 7, 2012 at 2:36 AM, Konrad Rzeszutek Wilk > <konrad.wilk@oracle.com> wrote: > > On Fri, Apr 06, 2012 at 11:43:00AM -0500, Wei Huang wrote: > >> On 04/06/2012 10:41 AM, Lin Ming wrote: > >> >Hi list, > >> > > >> >Is anybody working on virtualization of the CPU Performance Monitoring Unit? > >> > > >> >There are 2 PMU related projects listed on GSoC 2012 page. > >> >http://wiki.xen.org/wiki/Archived/GSoC_2012_Ideas > >> > > >> >- Virtualization of the CPU Performance Monitoring Unit > >> >- Perf support for Xen > >> > > >> >I''m interested on these 2 projects. > >> Hi Lin, > >> > >> 1. I don''t think Xen was accepted as an organization for 2012 GSOC. > >> See > >> http://lists.xen.org/archives/html/xen-devel/2012-03/msg02080.html. > >> 2. The PMU project description in the wiki is vague. I know HVM > >> guests support virtualized PMU. Please check vpmu.c files in /hvm, > >> /svm, and /vmx directories. You better ask mentors for details > >> (maybe this is XCP specific?). > > > > This is dom0 related. So being able to use perf to instrument > > the hypervisor (and the guests from dom0). > > For guests instrument: > > How about use perf on guests(domU) directly?I guess that would be a subset of the dom0 support? e.g. the ability for a PV guest to run perf on itself. Ian.
Marcus Granado
2012-Apr-10 14:45 UTC
Re: Virtualization of the CPU Performance Monitoring Unit
On 08/04/12 07:48, Lin Ming wrote:>> 2. The PMU project description in the wiki is vague. I know HVM guests >> support virtualized PMU. Please check vpmu.c files in /hvm, /svm, and /vmx >> directories. You better ask mentors for details (maybe this is XCP >> specific?).Yes, the description is vague, we should update the page. The idea was to enable users to run hardware profilers in dom0 and in any domU, PV or HVM. This would include implementing what is missing for dom0/PV domains, and to confirm in HVM domains that profilers such as oprofile, hwpmc, intel performance counter monitor at least are running without problems in the existing vPMU implementation. The original idea was to add support to XCP, but please feel free to add support to xen-unstable and the latest linux kernel; XCP will use those at some point. cheers, Marcus
On Tue, 2012-04-10 at 10:52 +0100, Ian Campbell wrote:> On Sun, 2012-04-08 at 07:58 +0100, Lin Ming wrote: > > On Sat, Apr 7, 2012 at 2:36 AM, Konrad Rzeszutek Wilk > > <konrad.wilk@oracle.com> wrote: > > > On Fri, Apr 06, 2012 at 11:43:00AM -0500, Wei Huang wrote: > > >> On 04/06/2012 10:41 AM, Lin Ming wrote: > > >> >Hi list, > > >> > > > >> >Is anybody working on virtualization of the CPU Performance Monitoring Unit? > > >> > > > >> >There are 2 PMU related projects listed on GSoC 2012 page. > > >> >http://wiki.xen.org/wiki/Archived/GSoC_2012_Ideas > > >> > > > >> >- Virtualization of the CPU Performance Monitoring Unit > > >> >- Perf support for Xen > > >> > > > >> >I''m interested on these 2 projects. > > >> Hi Lin, > > >> > > >> 1. I don''t think Xen was accepted as an organization for 2012 GSOC. > > >> See > > >> http://lists.xen.org/archives/html/xen-devel/2012-03/msg02080.html. > > >> 2. The PMU project description in the wiki is vague. I know HVM > > >> guests support virtualized PMU. Please check vpmu.c files in /hvm, > > >> /svm, and /vmx directories. You better ask mentors for details > > >> (maybe this is XCP specific?). > > > > > > This is dom0 related. So being able to use perf to instrument > > > the hypervisor (and the guests from dom0). > > > > For guests instrument: > > > > How about use perf on guests(domU) directly? > > I guess that would be a subset of the dom0 support? e.g. the ability for > a PV guest to run perf on itself.Yes.> > Ian. > >
On Tue, 2012-04-10 at 15:45 +0100, Marcus Granado wrote:> On 08/04/12 07:48, Lin Ming wrote: > >> 2. The PMU project description in the wiki is vague. I know HVM guests > >> support virtualized PMU. Please check vpmu.c files in /hvm, /svm, and /vmx > >> directories. You better ask mentors for details (maybe this is XCP > >> specific?). > Yes, the description is vague, we should update the page. The idea was > to enable users to run hardware profilers in dom0 and in any domU, PV or > HVM. This would include implementing what is missing for dom0/PV > domains, and to confirm in HVM domains that profilers such as oprofile, > hwpmc, intel performance counter monitor at least are running without > problems in the existing vPMU implementation. The original idea was to > add support to XCP, but please feel free to add support to xen-unstable > and the latest linux kernel; XCP will use those at some point.Thanks for the clarification. I''ll investigate this first.> cheers, > Marcus > >
Konrad Rzeszutek Wilk
2012-Apr-11 13:46 UTC
Re: Virtualization of the CPU Performance Monitoring Unit
On Sun, Apr 08, 2012 at 02:48:25PM +0800, Lin Ming wrote:> On Sat, Apr 7, 2012 at 12:43 AM, Wei Huang <wei.huang2@amd.com> wrote: > > On 04/06/2012 10:41 AM, Lin Ming wrote: > >> > >> Hi list, > >> > >> Is anybody working on virtualization of the CPU Performance Monitoring > >> Unit? > >> > >> There are 2 PMU related projects listed on GSoC 2012 page. > >> http://wiki.xen.org/wiki/Archived/GSoC_2012_Ideas > >> > >> - Virtualization of the CPU Performance Monitoring Unit > >> - Perf support for Xen > >> > >> I''m interested on these 2 projects. > > > > Hi Lin, > > > > 1. I don''t think Xen was accepted as an organization for 2012 GSOC. See > > http://lists.xen.org/archives/html/xen-devel/2012-03/msg02080.html. > > It doesn''t matter. > I just want to find something valuable to do. > > > 2. The PMU project description in the wiki is vague. I know HVM guests > > support virtualized PMU. Please check vpmu.c files in /hvm, /svm, and /vmx > > directories. You better ask mentors for details (maybe this is XCP > > specific?). > > (CC mentors) > > I tested PMU/Perf support with xen-unstable, dom0 3.3 kernel, domU > 3.4-rc1 kernel. > Here is the result on an Intel SandyBrige machine. > > - In dom0 > > # dmesg |grep -i "Performance Event" > Performance Events: unsupported p6 CPU model 42 no PMU driver, > software events only. > > Hardware events are not supported yet in dom0. > > - In domU > # dmesg |grep -i "Performance Events" > Performance Events: 16-deep LBR, SandyBridge events, Intel PMU driver. > > Seems domU has support for hardware events. > But "perf" does not work on domU.That isn''t actually true. If you run it, you will see it working in the guest - it just that it does not use the performence counters but instead uses the timer to sample data.> Run "perf top", but no data was collected.Hm, I am able to collect data using Fedora Core 16 PV guest. For dom0 or domU? For dom0 there is a bug somewhere where the machine crashes after 30 seconds or so - hadn''t actually gotten to the bottom of it. There was an email thread: https://lkml.org/lkml/2012/2/12/74 about this. Patches are most welcome!> > # cat /proc/interrupts |grep "PMI" > PMI: 0 0 Performance monitoring interrupts > > No PMU interrupts. > > BTW, I also tested KVM-Qemu. > "perf" works well on KVM-Qemu. > > Thanks, > Lin Ming > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
Konrad Rzeszutek Wilk
2012-Apr-11 14:00 UTC
Re: Virtualization of the CPU Performance Monitoring Unit
On Tue, Apr 10, 2012 at 03:45:30PM +0100, Marcus Granado wrote:> On 08/04/12 07:48, Lin Ming wrote: > >>2. The PMU project description in the wiki is vague. I know HVM guests > >>support virtualized PMU. Please check vpmu.c files in /hvm, /svm, and /vmx > >>directories. You better ask mentors for details (maybe this is XCP > >>specific?). > Yes, the description is vague, we should update the page. The idea was > to enable users to run hardware profilers in dom0 and in any domU, PV or > HVM. This would include implementing what is missing for dom0/PV > domains, and to confirm in HVM domains that profilers such as oprofile, > hwpmc, intel performance counter monitor at least are running without > problems in the existing vPMU implementation. The original idea was to > add support to XCP, but please feel free to add support to xen-unstable > and the latest linux kernel; XCP will use those at some point.Hey Marcos, I think we (you, Lin and me) should talk - if Lin is very interested in this, it might make sense to focus first on the existing supported ''perf'' framework that Linux has - instead of the oprofile variant. Lin, Marcos, what day/time would work for an IRC meeting on irc.freenode.net?> cheers, > Marcus > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Wed, 2012-04-11 at 09:46 -0400, Konrad Rzeszutek Wilk wrote:> On Sun, Apr 08, 2012 at 02:48:25PM +0800, Lin Ming wrote: > > On Sat, Apr 7, 2012 at 12:43 AM, Wei Huang <wei.huang2@amd.com> wrote: > > > On 04/06/2012 10:41 AM, Lin Ming wrote: > > >> > > >> Hi list, > > >> > > >> Is anybody working on virtualization of the CPU Performance Monitoring > > >> Unit? > > >> > > >> There are 2 PMU related projects listed on GSoC 2012 page. > > >> http://wiki.xen.org/wiki/Archived/GSoC_2012_Ideas > > >> > > >> - Virtualization of the CPU Performance Monitoring Unit > > >> - Perf support for Xen > > >> > > >> I''m interested on these 2 projects. > > > > > > Hi Lin, > > > > > > 1. I don''t think Xen was accepted as an organization for 2012 GSOC. See > > > http://lists.xen.org/archives/html/xen-devel/2012-03/msg02080.html. > > > > It doesn''t matter. > > I just want to find something valuable to do. > > > > > 2. The PMU project description in the wiki is vague. I know HVM guests > > > support virtualized PMU. Please check vpmu.c files in /hvm, /svm, and /vmx > > > directories. You better ask mentors for details (maybe this is XCP > > > specific?). > > > > (CC mentors) > > > > I tested PMU/Perf support with xen-unstable, dom0 3.3 kernel, domU > > 3.4-rc1 kernel. > > Here is the result on an Intel SandyBrige machine. > > > > - In dom0 > > > > # dmesg |grep -i "Performance Event" > > Performance Events: unsupported p6 CPU model 42 no PMU driver, > > software events only. > > > > Hardware events are not supported yet in dom0. > > > > - In domU > > # dmesg |grep -i "Performance Events" > > Performance Events: 16-deep LBR, SandyBridge events, Intel PMU driver. > > > > Seems domU has support for hardware events. > > But "perf" does not work on domU. > > That isn''t actually true. If you run it, you will see it working > in the guest - it just that it does not use the performence counters > but instead uses the timer to sample data.Right, I mean "hardware event" does not work. Hardware event, for example, perf top -e cycles, does not work. Software event, for example, perf top -e cpu-clock, works.> > > Run "perf top", but no data was collected. > > Hm, I am able to collect data using Fedora Core 16 PV guest. > For dom0 or domU? For dom0 there is a bug somewhere whereFor domU HVM guest. I have problem to run domU PV guest. Still looking at it.> the machine crashes after 30 seconds or so - hadn''t actually > gotten to the bottom of it. There was an email thread: > https://lkml.org/lkml/2012/2/12/74 about this. > > Patches are most welcome!I''ll see if I can reproduce this problem on dom0. Thanks, Lin Ming
On Wed, 2012-04-11 at 10:00 -0400, Konrad Rzeszutek Wilk wrote:> On Tue, Apr 10, 2012 at 03:45:30PM +0100, Marcus Granado wrote: > > On 08/04/12 07:48, Lin Ming wrote: > > >>2. The PMU project description in the wiki is vague. I know HVM guests > > >>support virtualized PMU. Please check vpmu.c files in /hvm, /svm, and /vmx > > >>directories. You better ask mentors for details (maybe this is XCP > > >>specific?). > > Yes, the description is vague, we should update the page. The idea was > > to enable users to run hardware profilers in dom0 and in any domU, PV or > > HVM. This would include implementing what is missing for dom0/PV > > domains, and to confirm in HVM domains that profilers such as oprofile, > > hwpmc, intel performance counter monitor at least are running without > > problems in the existing vPMU implementation. The original idea was to > > add support to XCP, but please feel free to add support to xen-unstable > > and the latest linux kernel; XCP will use those at some point. > > Hey Marcos, > > I think we (you, Lin and me) should talk - if Lin is very interested in this, > it might make sense to focus first on the existing supported ''perf'' framework > that Linux has - instead of the oprofile variant.Yes, I''m interested.> > Lin, Marcos, what day/time would work for an IRC meeting on > irc.freenode.net?I''m on ##xen now with nick name "mlin". 10pm to 12pm(Shanghai, China time) is OK for me.
On Wed, Apr 11, 2012 at 10:30 PM, Lin Ming <mlin@ss.pku.edu.cn> wrote: [....]>> That isn''t actually true. If you run it, you will see it working >> in the guest - it just that it does not use the performence counters >> but instead uses the timer to sample data. > > Right, I mean "hardware event" does not work. > > Hardware event, for example, perf top -e cycles, does not work.Just found that vpmu is disabled by default. You need to pass xen boot parameter "vpmu" to make hardware event work.> Software event, for example, perf top -e cpu-clock, works.So both hardware and software event work in DomU. Great!> >> >> > Run "perf top", but no data was collected. >> >> Hm, I am able to collect data using Fedora Core 16 PV guest. >> For dom0 or domU? For dom0 there is a bug somewhere where > > For domU HVM guest. > I have problem to run domU PV guest. Still looking at it. > >> the machine crashes after 30 seconds or so - hadn''t actually >> gotten to the bottom of it. There was an email thread: >> https://lkml.org/lkml/2012/2/12/74 about this. >> >> Patches are most welcome!Here are the patches. https://lkml.org/lkml/2012/4/15/12 Regards, Lin Ming
Konrad Rzeszutek Wilk
2012-Apr-16 17:39 UTC
Re: Virtualization of the CPU Performance Monitoring Unit
On Mon, Apr 16, 2012 at 04:16:07PM +0800, Lin Ming wrote:> On Wed, Apr 11, 2012 at 10:30 PM, Lin Ming <mlin@ss.pku.edu.cn> wrote: > > [....] > > >> That isn''t actually true. If you run it, you will see it working > >> in the guest - it just that it does not use the performence counters > >> but instead uses the timer to sample data. > > > > Right, I mean "hardware event" does not work. > > > > Hardware event, for example, perf top -e cycles, does not work. > > Just found that vpmu is disabled by default. > You need to pass xen boot parameter "vpmu" to make hardware event work.Oh, I wonder why it was disabled by default? Wei, would you know by any chance?> > > Software event, for example, perf top -e cpu-clock, works. > > So both hardware and software event work in DomU. > Great!Excellent!> > > > >> > >> > Run "perf top", but no data was collected. > >> > >> Hm, I am able to collect data using Fedora Core 16 PV guest. > >> For dom0 or domU? For dom0 there is a bug somewhere where > > > > For domU HVM guest. > > I have problem to run domU PV guest. Still looking at it. > > > >> the machine crashes after 30 seconds or so - hadn''t actually > >> gotten to the bottom of it. There was an email thread: > >> https://lkml.org/lkml/2012/2/12/74 about this. > >> > >> Patches are most welcome! > > Here are the patches. > https://lkml.org/lkml/2012/4/15/12Let me play with them a bit. At first glance they look ok - but I recall Peter Z saying something about not implementing the IRQ WORKER, but I can''t recall the reasons.> > Regards, > Lin Ming
Dietmar Hahn
2012-Apr-17 06:11 UTC
Re: Virtualization of the CPU Performance Monitoring Unit
Am Montag 16 April 2012, 13:39:36 schrieb Konrad Rzeszutek Wilk:> On Mon, Apr 16, 2012 at 04:16:07PM +0800, Lin Ming wrote: > > On Wed, Apr 11, 2012 at 10:30 PM, Lin Ming <mlin@ss.pku.edu.cn> wrote: > > > > [....] > > > > >> That isn''t actually true. If you run it, you will see it working > > >> in the guest - it just that it does not use the performence counters > > >> but instead uses the timer to sample data. > > > > > > Right, I mean "hardware event" does not work. > > > > > > Hardware event, for example, perf top -e cycles, does not work. > > > > Just found that vpmu is disabled by default. > > You need to pass xen boot parameter "vpmu" to make hardware event work. > > Oh, I wonder why it was disabled by default? Wei, would you know > by any chance?This had to do with a problem in the intel nehalem processors which could cause endless interrupt loops in the hypervisor if a hvm guest uses the performance counters so Keir proposed to add the vpmu boot flag. http://lists.xen.org/archives/html/xen-devel/2009-10/msg01460.html and http://lists.xen.org/archives/html/xen-devel/2009-11/msg00088.html Dietmar> > > > > > Software event, for example, perf top -e cpu-clock, works. > > > > So both hardware and software event work in DomU. > > Great! > > Excellent! > > > > > > > >> > > >> > Run "perf top", but no data was collected. > > >> > > >> Hm, I am able to collect data using Fedora Core 16 PV guest. > > >> For dom0 or domU? For dom0 there is a bug somewhere where > > > > > > For domU HVM guest. > > > I have problem to run domU PV guest. Still looking at it. > > > > > >> the machine crashes after 30 seconds or so - hadn''t actually > > >> gotten to the bottom of it. There was an email thread: > > >> https://lkml.org/lkml/2012/2/12/74 about this. > > >> > > >> Patches are most welcome! > > > > Here are the patches. > > https://lkml.org/lkml/2012/4/15/12 > > Let me play with them a bit. At first glance they look ok - but I recall > Peter Z saying something about not implementing the IRQ WORKER, but I can''t > recall the reasons. > > > > Regards, > > Lin Ming > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >-- Company details: http://ts.fujitsu.com/imprint.html