Andrew Evans
2010-Jan-29 01:37 UTC
[Xen-devel] profiling active RHEL 4 PV domains with xenoprof
Hi, Is it necessary to run a xenoprof-patched oprofile in a PV domU being actively profiled? I ask because http://xenoprof.sourceforge.net/xenoprof_2.0.txt seems to say no (no mention of --xen switch) but http://www.xen.org/files/summit_3/xenoprof_tutorial.pdf seems to say yes (specifies use of --xen in domUs). I''m running RHEL 4 in the domU I want to actively profile, and RHEL 4 provides the ancient oprofile-0.8.1 package, with no support for the --xen switch. I''ve gotten oprofile 0.9.6 with xenoprof patches built on RHEL 4 but haven''t been able to test it due to XENOPROF_start failing (see other message). Another thing I''m confused about: the above xenoprof tutorial also specifies use of the --event switch in domUs, saying they must use the same event as dom0. Makes sense, but my domUs are PV, and oprofile says only timer interrupts are usable. So must I also use timer interrupts on dom0? thanks, -Andrew _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
George Dunlap
2010-Jan-29 12:41 UTC
Re: [Xen-devel] profiling active RHEL 4 PV domains with xenoprof
It''s been a long time since I played with oprofile; I think it could use some serious attention, but it''s been on my "list of important things to do someday" for about 2 years now. :-) Re the PV kernels: are you sure that the PV kernel has support for Xen profiling? PV xenoprofile makes hypercalls to Xen to set up the performance event counters. But if that isn''t set up properly, or for some other reason doesn''t work right, the guest oprofile may default back to using only timers, for which it doesn''t need Xen''s help (but which are less accurate especially in a virtual environment). -George On Fri, Jan 29, 2010 at 1:37 AM, Andrew Evans <andrewe@yahoo-inc.com> wrote:> Hi, > > Is it necessary to run a xenoprof-patched oprofile in a PV domU being > actively profiled? I ask because > > http://xenoprof.sourceforge.net/xenoprof_2.0.txt > > seems to say no (no mention of --xen switch) but > > http://www.xen.org/files/summit_3/xenoprof_tutorial.pdf > > seems to say yes (specifies use of --xen in domUs). I''m running RHEL 4 > in the domU I want to actively profile, and RHEL 4 provides the ancient > oprofile-0.8.1 package, with no support for the --xen switch. I''ve > gotten oprofile 0.9.6 with xenoprof patches built on RHEL 4 but haven''t > been able to test it due to XENOPROF_start failing (see other message). > > Another thing I''m confused about: the above xenoprof tutorial also > specifies use of the --event switch in domUs, saying they must use the > same event as dom0. Makes sense, but my domUs are PV, and oprofile says > only timer interrupts are usable. So must I also use timer interrupts on > dom0? > > thanks, > > -Andrew > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Santos, Jose Renato G
2010-Jan-29 17:04 UTC
RE: [Xen-devel] profiling active RHEL 4 PV domains with xenoprof
> -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > Andrew Evans > Sent: Thursday, January 28, 2010 5:38 PM > To: xen-devel@lists.xensource.com > Subject: [Xen-devel] profiling active RHEL 4 PV domains with xenoprof > > Hi, > > Is it necessary to run a xenoprof-patched oprofile in a PV > domU being actively profiled? I ask becauseYes, you need need xenoprof support both in the kernel and in the OProfile user level tools in anctive profiled domain.> > http://xenoprof.sourceforge.net/xenoprof_2.0.txt > > seems to say no (no mention of --xen switch) but > > http://www.xen.org/files/summit_3/xenoprof_tutorial.pdf > > seems to say yes (specifies use of --xen in domUs). I''m > running RHEL 4 in the domU I want to actively profile, and > RHEL 4 provides the ancient > oprofile-0.8.1 package, with no support for the --xen switch. > I''ve gotten oprofile 0.9.6 with xenoprof patches built onAre you sure Oprofile user level tools has xenoprof support? The latest patch avaialable in http://xenoprof.sourceforge.net/ is for oprofile 0.9.5 but you claim to be using 0.9.6 Try using 0.9.5 and apply the patch.> RHEL 4 but haven''t been able to test it due to XENOPROF_start > failing (see other message). >> Another thing I''m confused about: the above xenoprof tutorial > also specifies use of the --event switch in domUs, saying > they must use the same event as dom0. Makes sense, but my > domUs are PV, and oprofile says only timer interrupts are > usable. So must I also use timer interrupts on dom0? >No, you should use the same event in dom0 and domU and not timer mode. If Oprofile is reporting timer mode in domU there is something wrong in its OProfile configuration; either you don''t have the right patches or you are not correctly synchronyzing dom0 and domU opcontrol commands. Renato> thanks, > > -Andrew > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dulloor
2010-Jan-29 17:50 UTC
Re: [Xen-devel] profiling active RHEL 4 PV domains with xenoprof
I didn''t test active profiling much, because I was interested in profiling only the guest kernel. It could be the patches. I will take a look and get back. -dulloor On Fri, Jan 29, 2010 at 12:04 PM, Santos, Jose Renato G <joserenato.santos@hp.com> wrote:> > >> -----Original Message----- >> From: xen-devel-bounces@lists.xensource.com >> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of >> Andrew Evans >> Sent: Thursday, January 28, 2010 5:38 PM >> To: xen-devel@lists.xensource.com >> Subject: [Xen-devel] profiling active RHEL 4 PV domains with xenoprof >> >> Hi, >> >> Is it necessary to run a xenoprof-patched oprofile in a PV >> domU being actively profiled? I ask because > > Yes, you need need xenoprof support both in the kernel and in the OProfile user level tools in anctive profiled domain. >> >> http://xenoprof.sourceforge.net/xenoprof_2.0.txt >> >> seems to say no (no mention of --xen switch) but >> >> http://www.xen.org/files/summit_3/xenoprof_tutorial.pdf >> >> seems to say yes (specifies use of --xen in domUs). I''m >> running RHEL 4 in the domU I want to actively profile, and >> RHEL 4 provides the ancient >> oprofile-0.8.1 package, with no support for the --xen switch. >> I''ve gotten oprofile 0.9.6 with xenoprof patches built on > > > Are you sure Oprofile user level tools has xenoprof support? > The latest patch avaialable in http://xenoprof.sourceforge.net/ is for oprofile 0.9.5 but you claim to be using 0.9.6 > Try using 0.9.5 and apply the patch. > >> RHEL 4 but haven''t been able to test it due to XENOPROF_start >> failing (see other message). >> > >> Another thing I''m confused about: the above xenoprof tutorial >> also specifies use of the --event switch in domUs, saying >> they must use the same event as dom0. Makes sense, but my >> domUs are PV, and oprofile says only timer interrupts are >> usable. So must I also use timer interrupts on dom0? >> > > No, you should use the same event in dom0 and domU and not timer mode. > If Oprofile is reporting timer mode in domU there is something wrong in its OProfile configuration; either you don''t have the right patches or you are not correctly synchronyzing dom0 and domU opcontrol commands. > > > Renato > >> thanks, >> >> -Andrew >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Andrew Evans
2010-Jan-29 18:00 UTC
Re: [Xen-devel] profiling active RHEL 4 PV domains with xenoprof
[This email is either empty or too large to be displayed at this time]
Andrew Evans
2010-Jan-29 21:11 UTC
Re: [Xen-devel] profiling active RHEL 4 PV domains with xenoprof
On 01/29/10 04:41, George Dunlap wrote:> Re the PV kernels: are you sure that the PV kernel has support for Xen > profiling? PV xenoprofile makes hypercalls to Xen to set up the > performance event counters. But if that isn''t set up properly, or for > some other reason doesn''t work right, the guest oprofile may default > back to using only timers, for which it doesn''t need Xen''s help (but > which are less accurate especially in a virtual environment). >I''m pretty sure RHEL 4''s 2.6.9 kernel doesn''t have the xenoprof patches, but I haven''t looked yet. I''m going to try the 2.6.18 xenU kernel I built from the xen-unstable tree and see how that goes. thanks, -Andrew _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Andrew Evans
2010-Jan-29 21:14 UTC
Re: [Xen-devel] profiling active RHEL 4 PV domains with xenoprof
On 01/29/10 09:50, Dulloor wrote:> I didn''t test active profiling much, because I was interested in > profiling only the guest kernel. It could be the patches. I will take > a look and get back. >Thanks! Don''t spend much time on it yet though. I''m pretty sure my PV guest kernel didn''t have the xenoprof patches (RHEL 4 2.6.9-67.ELxenU). I''m going to try the 2.6.18 xenU kernel I built from xen-unstable and see how that goes. -Andrew _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Andrew Evans
2010-Jan-30 00:24 UTC
Re: [Xen-devel] profiling active RHEL 4 PV domains with xenoprof
Big thanks to Renato and George -- I now have OProfile working for active PV domains. The key was that my guest kernel didn''t have the xenoprof patches. I built the 2.6.18-xenU kernel/modules from xen-unstable and pushed those to my domU. I had to `make menuconfig` to set CONFIG_PROFILING=y and CONFIG_OPROFILE=m after doing the `make KERNELS=linux-2.6-xenU kernels` from the top of my xen-unstable.hg tree. I also had to mkinitrd again to create a ramdisk compatible with the new kernel. Now `opreport -l` in dom0 shows output like this: CPU: Core 2, speed 2500.01 MHz (estimated) Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (Unhalted core cycles) count 1000000 samples % app name symbol name 219 12.1734 vmlinux-unknown (no symbols) 158 8.7827 e1000e (no symbols) 115 6.3924 vmlinux .text.read_sb_page 94 5.2251 xen-unknown (no symbols) 72 4.0022 libpython2.4.so.1.0 (no symbols) 49 2.7237 vmlinux xfrm_state_find 44 2.4458 vmlinux .text.md_ioctl 37 2.0567 bridge (no symbols) 34 1.8899 netbk (no symbols) I''m running a different kernel in the domU to the one in my dom0, and it doesn''t appear to be possible to pass multiple vmlinux paths to opcontrol. Renato''s http://www.xen.org/files/summit_3/xenoprof_tutorial.pdf uses the same kernel in dom0 and domU. Is that a requirement, or is there a way for me to point oprofiled or opreport to my guest kernel''s debug symbols too? thanks, -Andrew _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Santos, Jose Renato G
2010-Jan-31 21:20 UTC
RE: [Xen-devel] profiling active RHEL 4 PV domains with xenoprof
> -----Original Message----- > From: Andrew Evans [mailto:andrewe@yahoo-inc.com] > Sent: Friday, January 29, 2010 4:24 PM > To: xen-devel@lists.xensource.com > Cc: Santos, Jose Renato G; George Dunlap > Subject: Re: [Xen-devel] profiling active RHEL 4 PV domains > with xenoprof > > I''m running a different kernel in the domU to the one in my > dom0, and it doesn''t appear to be possible to pass multiple > vmlinux paths to opcontrol. Renato''s > http://www.xen.org/files/summit_3/xenoprof_tutorial.pdf uses > the same kernel in dom0 and domU. Is that a requirement, or > is there a way for me to point oprofiled or opreport to my > guest kernel''s debug symbols too? >You can use different kernels for different domains. Each active domain should specify its own kernel image using the --vmlinux option in the opcontrol command. That means you need to have the image available in the domain file system (normally this is not required as the guest image is loaded from dom0 file system) Also remember that the samples from a domU guest will not be reported in dom0. You need to run opreport in each guest to obtain their samples. Renato> thanks, > > -Andrew > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Andrew Evans
2010-Feb-01 18:40 UTC
Re: [Xen-devel] profiling active RHEL 4 PV domains with xenoprof
On 01/31/10 13:20, Santos, Jose Renato G wrote:> Also remember that the samples from a domU guest will not be reported > in dom0. You need to run opreport in each guest to obtain their samples.Thanks, that''s what I was missing. I thought domU samples were automagically propagated up to dom0 somehow, since the examples I''ve seen didn''t say anything about running opreport in domUs that have been actively profiled. thanks, -Andrew _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel