Ruslan Nikolaev
2011-May-12 19:36 UTC
[Xen-devel] Perfctr-Xen framework for permonace analysis
Hi I want to make an announcement about new perfomance monitoring framework. Perfctr-Xen framework that enables per-thread performance analysis in Xen. Current version is capable of properly virtualizing counters in both paravirtualized and HVM modes. It is based on perfctr (which is a library and kernel module for non-virtualized guests), ported to Xen, and extended to work properly in virtualized environment. Both accumulative and interrupt modes counting (profiling) are supported. The advantage of Perfctr-Xen is that it does not require specific HVM extensions which are needed for vpmu driver, can work in paravirtualized mode, and it also quite universal: works with many common tools such as PAPI, HPCToolkit, TAU PerfExplorer. It supports proper per-domain and per-thread virtualization. It is light-weight, supports wide range of CPUs, does not require save-and-restore for accumulative mode of counting (it uses counter offsetting), avoids expensive hypercalls and counter re-programming in certain circumstances (when threads are counting the same type of events). In addition, some techniques are employed to account for the overhead caused by the framework itself. This makes measurements quite accurate. Perfctr-Xen consists of series of patches that need to be applied to Xen, Linux, perfctr. There are available at: http://people.cs.vt.edu/~rnikola/ The code is available under LGPL. It would be great to discuss if and how it can be integrated into Xen. The publication regarding Perfctr-Xen is at: http://portal.acm.org/citation.cfm?id=1952687 Thanks, Ruslan Nikolaev _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-May-12 20:09 UTC
Re: [Xen-devel] Perfctr-Xen framework for permonace analysis
On 12/05/2011 20:36, "Ruslan Nikolaev" <nruslan_devel@yahoo.com> wrote:> Hi > > I want to make an announcement about new perfomance monitoring framework. > > Perfctr-Xen framework that enables per-thread performance analysis in Xen. > Current version is capable of properly virtualizing counters in both > paravirtualized and HVM modes. It is based on perfctr (which is a library and > kernel module for non-virtualized guests), ported to Xen, and extended to work > properly in virtualized environment. Both accumulative and interrupt modes > counting (profiling) are supported. > > The advantage of Perfctr-Xen is that it does not require specific HVM > extensions which are needed for vpmu driver, can work in paravirtualized mode, > and it also quite universal: works with many common tools such as PAPI, > HPCToolkit, TAU PerfExplorer. It supports proper per-domain and per-thread > virtualization. It is light-weight, supports wide range of CPUs, does not > require save-and-restore for accumulative mode of counting (it uses counter > offsetting), avoids expensive hypercalls and counter re-programming in certain > circumstances (when threads are counting the same type of events). In > addition, some techniques are employed to account for the overhead caused by > the framework itself. This makes measurements quite accurate. > > Perfctr-Xen consists of series of patches that need to be applied to Xen, > Linux, perfctr. There are available at: > http://people.cs.vt.edu/~rnikola/ > > The code is available under LGPL. It would be great to discuss if and how it > can be integrated into Xen.Could it reasonably replace the oprofile stuff we have already? I wouldn''t want yet another perfctr subsystem/interface unless it supplants an existing one. We need a revolving door policy here I think. -- Keir> The publication regarding Perfctr-Xen is at: > http://portal.acm.org/citation.cfm?id=1952687 > > Thanks, > Ruslan Nikolaev > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ruslan Nikolaev
2011-May-12 20:35 UTC
Re: [Xen-devel] Perfctr-Xen framework for permonace analysis
Well, at least it can peacefully coexist with oprofile. AFAIK, XenOProf has some limitations (system wide monitoring only), and works only with OProfile. Many people would also prefer to use PAPI/HPCToolkit/PerfExplorer. HPCToolkit itself is quite sophisticated framework with elaborated functionality. It is quite comparable to OProfile. PerfExplorer is also a very interesting project. PAPI is used by developers to do wide range of measurements which are not necessarily possible with OProfile. The key feature of Perfctr-Xen is that it allows to do all levels of performance monitoring, it does not restrict to profiling. This framework and xenoprof provide orthogonal solutions which both can be used in different circumstances. Thanks, Ruslan Nikolaev --- On Fri, 5/13/11, Keir Fraser <keir.xen@gmail.com> wrote:> From: Keir Fraser <keir.xen@gmail.com> > Subject: Re: [Xen-devel] Perfctr-Xen framework for permonace analysis > To: "Ruslan Nikolaev" <nruslan_devel@yahoo.com>, xen-devel@lists.xensource.com > Date: Friday, May 13, 2011, 12:09 AM > On 12/05/2011 20:36, "Ruslan > Nikolaev" <nruslan_devel@yahoo.com> > wrote: > > > Hi > > > > I want to make an announcement about new perfomance > monitoring framework. > > > > Perfctr-Xen framework that enables per-thread > performance analysis in Xen. > > Current version is capable of properly virtualizing > counters in both > > paravirtualized and HVM modes. It is based on perfctr > (which is a library and > > kernel module for non-virtualized guests), ported to > Xen, and extended to work > > properly in virtualized environment. Both accumulative > and interrupt modes > > counting (profiling) are supported. > > > > The advantage of Perfctr-Xen is that it does not > require specific HVM > > extensions which are needed for vpmu driver, can work > in paravirtualized mode, > > and it also quite universal: works with many common > tools such as PAPI, > > HPCToolkit, TAU PerfExplorer. It supports proper > per-domain and per-thread > > virtualization. It is light-weight, supports wide > range of CPUs, does not > > require save-and-restore for accumulative mode of > counting (it uses counter > > offsetting), avoids expensive hypercalls and counter > re-programming in certain > > circumstances (when threads are counting the same type > of events). In > > addition, some techniques are employed to account for > the overhead caused by > > the framework itself. This makes measurements quite > accurate. > > > > Perfctr-Xen consists of series of patches that need to > be applied to Xen, > > Linux, perfctr. There are available at: > > http://people.cs.vt.edu/~rnikola/ > > > > The code is available under LGPL. It would be great to > discuss if and how it > > can be integrated into Xen. > > Could it reasonably replace the oprofile stuff we have > already? I wouldn''t > want yet another perfctr subsystem/interface unless it > supplants an existing > one. We need a revolving door policy here I think. > > -- Keir > > > The publication regarding Perfctr-Xen is at: > > http://portal.acm.org/citation.cfm?id=1952687 > > > > Thanks, > > Ruslan Nikolaev > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
George Dunlap
2011-May-13 13:57 UTC
Re: [Xen-devel] Perfctr-Xen framework for permonace analysis
On Thu, May 12, 2011 at 9:35 PM, Ruslan Nikolaev <nruslan_devel@yahoo.com> wrote:> Well, at least it can peacefully coexist with oprofile. AFAIK, XenOProf has some limitations (system wide monitoring only), and works only with OProfile. Many people would also prefer to use PAPI/HPCToolkit/PerfExplorer. HPCToolkit itself is quite sophisticated framework with elaborated functionality. It is quite comparable to OProfile. PerfExplorer is also a very interesting project. PAPI is used by developers to do wide range of measurements which are not necessarily possible with OProfile. The key feature of Perfctr-Xen is that it allows to do all levels of performance monitoring, it does not restrict to profiling.xenoprof enables the normal Linux oprofile, which can be used on processes, as well as on domains and on Xen itself. I don''t remember if oprofile allows collection of performance counters on individual processes though; I suspect not. I''m a bit torn -- if people are used to using oprofile under Linux, it may be advantageous to keep that, rather than replacing it outright with a new tool. But as you said about the scheduler, Keir, oprofile gets little enough love as it is... -George> > This framework and xenoprof provide orthogonal solutions which both can be used in different circumstances. > > Thanks, > Ruslan Nikolaev > > --- On Fri, 5/13/11, Keir Fraser <keir.xen@gmail.com> wrote: > >> From: Keir Fraser <keir.xen@gmail.com> >> Subject: Re: [Xen-devel] Perfctr-Xen framework for permonace analysis >> To: "Ruslan Nikolaev" <nruslan_devel@yahoo.com>, xen-devel@lists.xensource.com >> Date: Friday, May 13, 2011, 12:09 AM >> On 12/05/2011 20:36, "Ruslan >> Nikolaev" <nruslan_devel@yahoo.com> >> wrote: >> >> > Hi >> > >> > I want to make an announcement about new perfomance >> monitoring framework. >> > >> > Perfctr-Xen framework that enables per-thread >> performance analysis in Xen. >> > Current version is capable of properly virtualizing >> counters in both >> > paravirtualized and HVM modes. It is based on perfctr >> (which is a library and >> > kernel module for non-virtualized guests), ported to >> Xen, and extended to work >> > properly in virtualized environment. Both accumulative >> and interrupt modes >> > counting (profiling) are supported. >> > >> > The advantage of Perfctr-Xen is that it does not >> require specific HVM >> > extensions which are needed for vpmu driver, can work >> in paravirtualized mode, >> > and it also quite universal: works with many common >> tools such as PAPI, >> > HPCToolkit, TAU PerfExplorer. It supports proper >> per-domain and per-thread >> > virtualization. It is light-weight, supports wide >> range of CPUs, does not >> > require save-and-restore for accumulative mode of >> counting (it uses counter >> > offsetting), avoids expensive hypercalls and counter >> re-programming in certain >> > circumstances (when threads are counting the same type >> of events). In >> > addition, some techniques are employed to account for >> the overhead caused by >> > the framework itself. This makes measurements quite >> accurate. >> > >> > Perfctr-Xen consists of series of patches that need to >> be applied to Xen, >> > Linux, perfctr. There are available at: >> > http://people.cs.vt.edu/~rnikola/ >> > >> > The code is available under LGPL. It would be great to >> discuss if and how it >> > can be integrated into Xen. >> >> Could it reasonably replace the oprofile stuff we have >> already? I wouldn''t >> want yet another perfctr subsystem/interface unless it >> supplants an existing >> one. We need a revolving door policy here I think. >> >> -- Keir >> >> > The publication regarding Perfctr-Xen is at: >> > http://portal.acm.org/citation.cfm?id=1952687 >> > >> > Thanks, >> > Ruslan Nikolaev >> > >> > _______________________________________________ >> > Xen-devel mailing list >> > Xen-devel@lists.xensource.com >> > http://lists.xensource.com/xen-devel >> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
George Dunlap
2011-May-13 13:59 UTC
Re: [Xen-devel] Perfctr-Xen framework for permonace analysis
On Thu, May 12, 2011 at 8:36 PM, Ruslan Nikolaev <nruslan_devel@yahoo.com> wrote:> Hi > > I want to make an announcement about new perfomance monitoring framework. > > Perfctr-Xen framework that enables per-thread performance analysis in Xen. Current version is capable of properly virtualizing counters in both paravirtualized and HVM modes. It is based on perfctr (which is a library and kernel module for non-virtualized guests), ported to Xen, and extended to work properly in virtualized environment. Both accumulative and interrupt modes counting (profiling) are supported. > > The advantage of Perfctr-Xen is that it does not require specific HVM extensions which are needed for vpmu driver, can work in paravirtualized mode, and it also quite universal: works with many common tools such as PAPI, HPCToolkit, TAU PerfExplorer. It supports proper per-domain and per-thread virtualization. It is light-weight, supports wide range of CPUs, does not require save-and-restore for accumulative mode of counting (it uses counter offsetting), avoids expensive hypercalls and counter re-programming in certain circumstances (when threads are counting the same type of events). In addition, some techniques are employed to account for the overhead caused by the framework itself. This makes measurements quite accurate. > > Perfctr-Xen consists of series of patches that need to be applied to Xen, Linux, perfctr. There are available at: > http://people.cs.vt.edu/~rnikola/ > > The code is available under LGPL. It would be great to discuss if and how it can be integrated into Xen. > > The publication regarding Perfctr-Xen is at: > http://portal.acm.org/citation.cfm?id=1952687Do you have this paper, or perhaps a pre-print, posted somewhere not behind a paywall? Or a different document describing the library in detail? Most of us don''t have subscriptions to ACM... -George _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ruslan Nikolaev
2011-May-13 23:03 UTC
Re: [Xen-devel] Perfctr-Xen framework for permonace analysis
--- On Fri, 5/13/11, George Dunlap <George.Dunlap@eu.citrix.com> wrote:> From: George Dunlap <George.Dunlap@eu.citrix.com> > Subject: Re: [Xen-devel] Perfctr-Xen framework for permonace analysis > To: "Ruslan Nikolaev" <nruslan_devel@yahoo.com> > Cc: xen-devel@lists.xensource.com > Date: Friday, May 13, 2011, 5:59 PM > On Thu, May 12, 2011 at 8:36 PM, > Ruslan Nikolaev > <nruslan_devel@yahoo.com> > wrote: > > Hi > > > > I want to make an announcement about new perfomance > monitoring framework. > > > > Perfctr-Xen framework that enables per-thread > performance analysis in Xen. Current version is capable of > properly virtualizing counters in both paravirtualized and > HVM modes. It is based on perfctr (which is a library and > kernel module for non-virtualized guests), ported to Xen, > and extended to work properly in virtualized environment. > Both accumulative and interrupt modes counting (profiling) > are supported. > > > > The advantage of Perfctr-Xen is that it does not > require specific HVM extensions which are needed for vpmu > driver, can work in paravirtualized mode, and it also quite > universal: works with many common tools such as PAPI, > HPCToolkit, TAU PerfExplorer. It supports proper per-domain > and per-thread virtualization. It is light-weight, supports > wide range of CPUs, does not require save-and-restore for > accumulative mode of counting (it uses counter offsetting), > avoids expensive hypercalls and counter re-programming in > certain circumstances (when threads are counting the same > type of events). In addition, some techniques are employed > to account for the overhead caused by the framework itself. > This makes measurements quite accurate. > > > > Perfctr-Xen consists of series of patches that need to > be applied to Xen, Linux, perfctr. There are available at: > > http://people.cs.vt.edu/~rnikola/ > > > > The code is available under LGPL. It would be great to > discuss if and how it can be integrated into Xen. > > > > The publication regarding Perfctr-Xen is at: > > http://portal.acm.org/citation.cfm?id=1952687 > > Do you have this paper, or perhaps a pre-print, posted > somewhere not > behind a paywall? Or a different document describing > the library in > detail? Most of us don''t have subscriptions to > ACM...Ok. I will post a link. Ruslan.> > -George > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ruslan Nikolaev
2011-May-16 20:44 UTC
Re: [Xen-devel] Perfctr-Xen framework for permonace analysis
The author''s paper is available on my website: http://people.cs.vt.edu/~rnikola/?page_id=159 Please, let me know what you think. Thanks, Ruslan Nikolaev --- On Fri, 5/13/11, George Dunlap <George.Dunlap@eu.citrix.com> wrote:> From: George Dunlap <George.Dunlap@eu.citrix.com> > Subject: Re: [Xen-devel] Perfctr-Xen framework for permonace analysis > To: "Ruslan Nikolaev" <nruslan_devel@yahoo.com> > Cc: xen-devel@lists.xensource.com > Date: Friday, May 13, 2011, 5:59 PM > On Thu, May 12, 2011 at 8:36 PM, > Ruslan Nikolaev > <nruslan_devel@yahoo.com> > wrote: > > Hi > > > > I want to make an announcement about new perfomance > monitoring framework. > > > > Perfctr-Xen framework that enables per-thread > performance analysis in Xen. Current version is capable of > properly virtualizing counters in both paravirtualized and > HVM modes. It is based on perfctr (which is a library and > kernel module for non-virtualized guests), ported to Xen, > and extended to work properly in virtualized environment. > Both accumulative and interrupt modes counting (profiling) > are supported. > > > > The advantage of Perfctr-Xen is that it does not > require specific HVM extensions which are needed for vpmu > driver, can work in paravirtualized mode, and it also quite > universal: works with many common tools such as PAPI, > HPCToolkit, TAU PerfExplorer. It supports proper per-domain > and per-thread virtualization. It is light-weight, supports > wide range of CPUs, does not require save-and-restore for > accumulative mode of counting (it uses counter offsetting), > avoids expensive hypercalls and counter re-programming in > certain circumstances (when threads are counting the same > type of events). In addition, some techniques are employed > to account for the overhead caused by the framework itself. > This makes measurements quite accurate. > > > > Perfctr-Xen consists of series of patches that need to > be applied to Xen, Linux, perfctr. There are available at: > > http://people.cs.vt.edu/~rnikola/ > > > > The code is available under LGPL. It would be great to > discuss if and how it can be integrated into Xen. > > > > The publication regarding Perfctr-Xen is at: > > http://portal.acm.org/citation.cfm?id=1952687 > > Do you have this paper, or perhaps a pre-print, posted > somewhere not > behind a paywall? Or a different document describing > the library in > detail? Most of us don''t have subscriptions to > ACM... > > -George > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2011-Aug-15 15:21 UTC
Re: [Xen-devel] Perfctr-Xen framework for permonace analysis
On Thu, May 12, 2011 at 12:36:07PM -0700, Ruslan Nikolaev wrote:> Hi > > I want to make an announcement about new perfomance monitoring framework. > > Perfctr-Xen framework that enables per-thread performance analysis in Xen. Current version is capable of properly virtualizing counters in both paravirtualized and HVM modes. It is based on perfctr (which is a library and kernel module for non-virtualized guests), ported to Xen, and extended to work properly in virtualized environment. Both accumulative and interrupt modes counting (profiling) are supported. > > The advantage of Perfctr-Xen is that it does not require specific HVM extensions which are needed for vpmu driver, can work in paravirtualized mode, and it also quite universal: works with many common tools such as PAPI, HPCToolkit, TAU PerfExplorer. It supports proper per-domain and per-thread virtualization. It is light-weight, supports wide range of CPUs, does not require save-and-restore for accumulative mode of counting (it uses counter offsetting), avoids expensive hypercalls and counter re-programming in certain circumstances (when threads are counting the same type of events). In addition, some techniques are employed to account for the overhead caused by the framework itself. This makes measurements quite accurate. > > Perfctr-Xen consists of series of patches that need to be applied to Xen, Linux, perfctr. There are available at: > http://people.cs.vt.edu/~rnikola/ > > The code is available under LGPL. It would be great to discuss if and how it can be integrated into Xen.So I read your guys paper and found it interesting. But I am bit confused about the baremetal state of perf monitoring tool. Which one is in the upstream kernel (3.0) that runs when you use ''perftop'' and like?> > The publication regarding Perfctr-Xen is at: > http://portal.acm.org/citation.cfm?id=1952687 > > Thanks, > Ruslan Nikolaev > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel