Wei Huang
2011-Feb-11 16:29 UTC
[Xen-devel] [PATCH 4/5][RFC] lwp: adding support for AMD lightweight profiling
Add xsave/xrstor support for LWP Because LWP is not tracked by TS, which is used in current xsave/xrstor implementation for FPU/AVX. As a result, we have to save and restore LWP in extended area whenever vcpu is re-scheduled. To avoid unnecessary lwp save/restore, this patch checks the LWP_CBADDR to determine whether LWP has been turned on. Signed-off-by: Wei Huang <wei.huang2@amd.com> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2011-Feb-14 09:12 UTC
Re: [Xen-devel] [PATCH 4/5][RFC] lwp: adding support for AMD lightweight profiling
>>> On 11.02.11 at 17:29, Wei Huang <wei.huang2@amd.com> wrote: >--- a/xen/arch/x86/i387.c Thu Feb 10 16:25:09 2011 -0600 >+++ b/xen/arch/x86/i387.c Thu Feb 10 16:55:27 2011 -0600 >@@ -65,6 +65,55 @@ > static void init_fpu(void); > static void restore_fpu(struct vcpu *v); > >+/* Save AMD LWP */ >+void xsave_lwp(struct vcpu *v) >+{ >+ uint64_t lwpcb; >+ bool_t ts; >+ struct xsave_struct *xsave_area = v->arch.xsave_area; >+ >+ if ( cpu_has_lwp ) >+ { >+ /* Has LWP been used? */ >+ rdmsrl(MSR_AMD_LWP_CBADDR, lwpcb);There''s no way to track LWP-using state for a vCPU, is there? rdmsr seems pretty expensive for being used in the context switch unconditionally (on CPUs supporting LWP)...>+ if ( !lwpcb ) { >+ /* Guest might have turned off LWP. So clean the bit here. */ >+ xsave_area->xsave_hdr.xstate_bv &= ~XSTATE_LWP; >+ return; >+ } >+ >+ /* Disable TS temporarily to avoid recursion. */ >+ ts = read_cr0() & X86_CR0_TS; >+ clts(); >+ xsave(v, XSTATE_LWP); >+ if ( ts ) >+ stts();Together with the xrstor_lwp() ones, quite a few manipulations of CR0, and hence making context switch between two LWP-using vcpus pretty expensive. I''m sure some of this redundancy can be eliminated. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Huang2, Wei
2011-Feb-14 15:55 UTC
RE: [Xen-devel] [PATCH 4/5][RFC] lwp: adding support for AMD lightweight profiling
Hi Jan, No there is no quick way (like TS bit) to keep track LWP state. Here is an excerpt from lwp spec: "LWP does not support the "lazy" state save and restore that is possible for floating point and SSE state. It does not interact with the CR0.TS bit. Operating systems that support LWP must always do an XSAVE to preserve the old thread''s LWP context and an XRSTOR to set up the new LWP context. The OS can continue to do a lazy switch of the FP and SSE state by ensuring that the corresponding bits in EDX:EAX are clear when it executes the XSAVE and XRSTOR to handle the LWP context." Thanks, -Wei -----Original Message----- From: Jan Beulich [mailto:JBeulich@novell.com] Sent: Monday, February 14, 2011 3:12 AM To: Huang2, Wei Cc: Gang Wei; xen-devel@lists.xensource.com; KeirFraser Subject: Re: [Xen-devel] [PATCH 4/5][RFC] lwp: adding support for AMD lightweight profiling>>> On 11.02.11 at 17:29, Wei Huang <wei.huang2@amd.com> wrote: >--- a/xen/arch/x86/i387.c Thu Feb 10 16:25:09 2011 -0600 >+++ b/xen/arch/x86/i387.c Thu Feb 10 16:55:27 2011 -0600 >@@ -65,6 +65,55 @@ > static void init_fpu(void); > static void restore_fpu(struct vcpu *v); > >+/* Save AMD LWP */ >+void xsave_lwp(struct vcpu *v) >+{ >+ uint64_t lwpcb; >+ bool_t ts; >+ struct xsave_struct *xsave_area = v->arch.xsave_area; >+ >+ if ( cpu_has_lwp ) >+ { >+ /* Has LWP been used? */ >+ rdmsrl(MSR_AMD_LWP_CBADDR, lwpcb);There''s no way to track LWP-using state for a vCPU, is there? rdmsr seems pretty expensive for being used in the context switch unconditionally (on CPUs supporting LWP)...>+ if ( !lwpcb ) { >+ /* Guest might have turned off LWP. So clean the bit here. */ >+ xsave_area->xsave_hdr.xstate_bv &= ~XSTATE_LWP; >+ return; >+ } >+ >+ /* Disable TS temporarily to avoid recursion. */ >+ ts = read_cr0() & X86_CR0_TS; >+ clts(); >+ xsave(v, XSTATE_LWP); >+ if ( ts ) >+ stts();Together with the xrstor_lwp() ones, quite a few manipulations of CR0, and hence making context switch between two LWP-using vcpus pretty expensive. I''m sure some of this redundancy can be eliminated. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2011-Feb-14 16:17 UTC
RE: [Xen-devel] [PATCH 4/5][RFC] lwp: adding support for AMD lightweight profiling
>>> On 14.02.11 at 16:55, "Huang2, Wei" <Wei.Huang2@amd.com> wrote: > No there is no quick way (like TS bit) to keep track LWP state. Here is an > excerpt from lwp spec: > > "LWP does not support the "lazy" state save and restore that is possible for > floating point and SSE state. It does not interact with the CR0.TS bit. > Operating systems that support LWP must always do an XSAVE to preserve the > old thread''s LWP context and an XRSTOR to set up the new LWP context. The OS > can continue to do a lazy switch of the FP and SSE state by ensuring that the > corresponding bits in EDX:EAX are clear when it executes the XSAVE and XRSTOR > to handle the LWP context."I certainly wasn''t thinking of CR0.TS, but after reading patch 5 my question should have been re-phrased into whether the intercept of the control MSR can''t be used for tracking purposes. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Huang2, Wei
2011-Feb-14 17:04 UTC
RE: [Xen-devel] [PATCH 4/5][RFC] lwp: adding support for AMD lightweight profiling
This might be possible. The frequency of MSR read/write inside guest VM will be less compared with vcpu context switch. I will try it in next spin. -Wei -----Original Message----- From: Jan Beulich [mailto:JBeulich@novell.com] Sent: Monday, February 14, 2011 10:18 AM To: Huang2, Wei Cc: Gang Wei; xen-devel@lists.xensource.com; KeirFraser Subject: RE: [Xen-devel] [PATCH 4/5][RFC] lwp: adding support for AMD lightweight profiling>>> On 14.02.11 at 16:55, "Huang2, Wei" <Wei.Huang2@amd.com> wrote: > No there is no quick way (like TS bit) to keep track LWP state. Here is an > excerpt from lwp spec: > > "LWP does not support the "lazy" state save and restore that is possible for > floating point and SSE state. It does not interact with the CR0.TS bit. > Operating systems that support LWP must always do an XSAVE to preserve the > old thread''s LWP context and an XRSTOR to set up the new LWP context. The OS > can continue to do a lazy switch of the FP and SSE state by ensuring that the > corresponding bits in EDX:EAX are clear when it executes the XSAVE and XRSTOR > to handle the LWP context."I certainly wasn''t thinking of CR0.TS, but after reading patch 5 my question should have been re-phrased into whether the intercept of the control MSR can''t be used for tracking purposes. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel