Thanks in part to Jose''s help I''ve gotten oprofile working on multiple domains with unstable. After reading the original Xenoprof paper, I have a question about how hypervisor samples are handled. If my understanding is correct, all samples pass through the hypervisor where they are then dispatched to the profiler in the corresponding domain. This makes sense in the case where code running in a given domain makes a hypercall. However, what happens when a hardware interrupt triggers code execution in the hypervisor? Where do such samples end up getting dispatched to and recorded? Thanks, David Carr _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
The sample is delivered to the VCPU (and associated domain) that is currently running on the CPU when the sample is taken, as computed by the Xen macro "current". That means that samples taken when a harware interrupt is running are associated with the VCPU/domain that was interrupted by the HW interrupt. Renato>> -----Original Message----- >> From: xen-devel-bounces@lists.xensource.com >> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of >> David Carr >> Sent: Tuesday, May 23, 2006 2:12 PM >> To: xen-devel@lists.xensource.com >> Subject: [Xen-devel] Xenoprof and the hypervisor >> >> Thanks in part to Jose''s help I''ve gotten oprofile working >> on multiple domains with unstable. After reading the >> original Xenoprof paper, I have a question about how >> hypervisor samples are handled. If my understanding is >> correct, all samples pass through the hypervisor where they >> are then dispatched to the profiler in the corresponding >> domain. This makes sense in the case where code running in >> a given domain makes a hypercall. >> However, what happens when a hardware interrupt triggers >> code execution in the hypervisor? Where do such samples end >> up getting dispatched to and recorded? >> >> Thanks, >> David Carr >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tuesday 23 May 2006 17:13, Santos, Jose Renato G wrote:> The sample is delivered to the VCPU (and associated domain) that is > currently running on the CPU when the sample is taken, > as computed by the Xen macro "current". That means that samples > taken when a harware interrupt is running are associated > with the VCPU/domain that was interrupted by the HW interrupt. > Renato >So let''s suppose what we are trying to do is to optimize or measure the performace of the hypervisor itself? What we would like to happen in that case would be that we would to collect all of the samples that found that the hypervisor was running when the nmi occurred and ship these samples off to, say, dom0 for later collection. At the moment, if a fully virtualized guest is taking most of the cycles (say> 97.5% of the cpu), and without passive domain support in xenoprof, thenwhat happens is that 97.5% of the samples are discarded, because the fully virtualized domain is not being profiled (is_profiled(vcpu->domain) is false in xenoprof_log_event()). So, I''d just like those samples handed to dom0, if possible. I''ve tried to do this, but it appears oprofile then throws away the samples (needs to be looked at some more to be sure). So what''s the easiest way to get those samples collected and out to dom0? -- Ray Bryant AMD Performance Labs Austin, Tx 512-602-0038 (o) 512-507-7807 (c) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ray, Xiaowei Yang from Intel posted a patch a few weeks ago that enable passive domain support. Passive domain support does exactly what you want. It enables you to specify the fully virtualized guest as a passive domain (using --passive_domains=<domid> option in opcontrol in dom0). With this patch, all samples collected when the passive_domain is running are delivered to dom0. Xiaowei also added support to decode the samples in kernel space to the specific kernel symbols of the passive domains (you need to pass the kernel image as an option in opcontrol). With this you are able to get all samples in the system. The only limitation is that oprofile will not be able to decode user level samples for the passive domains. It will aggregate all samples into under a single "user-level" category, but all xen and kernel samples will be mapped to their associated symbols. I hope this helps Renato>> -----Original Message----- >> From: Ray Bryant [mailto:raybry@mpdtxmail.amd.com] >> Sent: Wednesday, May 24, 2006 12:46 PM >> To: xen-devel@lists.xensource.com >> Cc: Santos, Jose Renato G; David Carr >> Subject: Re: [Xen-devel] Xenoprof and the hypervisor >> >> On Tuesday 23 May 2006 17:13, Santos, Jose Renato G wrote: >> > The sample is delivered to the VCPU (and associated >> domain) that is >> > currently running on the CPU when the sample is taken, as >> computed by >> > the Xen macro "current". That means that samples taken >> when a harware >> > interrupt is running are associated with the VCPU/domain that was >> > interrupted by the HW interrupt. >> > Renato >> > >> >> So let''s suppose what we are trying to do is to optimize or >> measure the >> performace of the hypervisor itself? What we would like to >> happen in that >> case would be that we would to collect all of the samples >> that found that the hypervisor was running when the nmi >> occurred and ship these samples off to, say, dom0 for later >> collection. >> >> At the moment, if a fully virtualized guest is taking most >> of the cycles (say >> > 97.5% of the cpu), and without passive domain support in xenoprof, >> > then >> what happens is that 97.5% of the samples are discarded, >> because the fully virtualized domain is not being profiled >> (is_profiled(vcpu->domain) is false in xenoprof_log_event()). >> >> So, I''d just like those samples handed to dom0, if possible. >> I''ve tried to >> do this, but it appears oprofile then throws away the >> samples (needs to be looked at some more to be sure). >> >> So what''s the easiest way to get those samples collected and >> out to dom0? >> -- >> Ray Bryant >> AMD Performance Labs Austin, Tx >> 512-602-0038 (o) 512-507-7807 (c) >> >>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel