Xen devel - Jan 2011 - AMD crash on restore. bug in MTRR restore. 4.0.2-rc2-pre

The mtrr save code on my AMD system reads the first MTRR as
0x00000000f0000000,0x000000fff8000800. When it tries to restore in
mtrr_var_range_msr_set, 0x000000fff8000800 is rejected as invalid and
not restored.

Commenting out the check against the mask, as well as Tim''s patch to
fix
the segment registers makes the output of xen-hvmctx match before the
save and after the restore.

Windows still crashes on restore though so I don''t know if this has
anything to do with it.

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Additionally, WinDbg tells me that MSR''s 175, and 176 are wiped out by
save/restore:

immediately before suspend
kd> rdmsr 174
msr[174] = 00000000`00000008
kd> rdmsr 175
msr[175] = 00000000`8039e000
kd> rdmsr 176
msr[176] = 00000000`816a4950

immediately after restore
kd> rdmsr 174
msr[174] = 00000000`00000000
kd> rdmsr 175
msr[175] = 00000000`00000000
kd> rdmsr 176
msr[176] = 00000000`00000000

although you''re never too sure with WinDbg...

James
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
> bounces@lists.xensource.com] On Behalf Of James Harper
> Sent: Wednesday, 26 January 2011 17:19
> To: xen-devel@lists.xensource.com
> Cc: Tim Deegan
> Subject: [Xen-devel] AMD crash on restore. bug in MTRR restore.
4.0.2-rc2-pre
> 
> The mtrr save code on my AMD system reads the first MTRR as
> 0x00000000f0000000,0x000000fff8000800. When it tries to restore in
> mtrr_var_range_msr_set, 0x000000fff8000800 is rejected as invalid and
> not restored.
> 
> Commenting out the check against the mask, as well as Tim''s patch
to
fix
> the segment registers makes the output of xen-hvmctx match before the
> save and after the restore.
> 
> Windows still crashes on restore though so I don''t know if this
has
> anything to do with it.
> 
> James
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

By saving the sysenter msrs in my PV drivers before the suspend
hypercall and restoring them after, save/restore now works fine.

Does nobody else see a save/restore failure on AMD as a blocker to
4.0.2? It it something particular to my exact AMD CPU and OS (Windows
2008 x32)?

James
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
> bounces@lists.xensource.com] On Behalf Of James Harper
> Sent: Thursday, 27 January 2011 13:32
> To: xen-devel@lists.xensource.com
> Cc: Tim Deegan
> Subject: RE: [Xen-devel] AMD crash on restore. bug in MTRR
restore.4.0.2-rc2-
> pre
> 
> Additionally, WinDbg tells me that MSR''s 175, and 176 are wiped
out by
> save/restore:
> 
> immediately before suspend
> kd> rdmsr 174
> msr[174] = 00000000`00000008
> kd> rdmsr 175
> msr[175] = 00000000`8039e000
> kd> rdmsr 176
> msr[176] = 00000000`816a4950
> 
> immediately after restore
> kd> rdmsr 174
> msr[174] = 00000000`00000000
> kd> rdmsr 175
> msr[175] = 00000000`00000000
> kd> rdmsr 176
> msr[176] = 00000000`00000000
> 
> although you''re never too sure with WinDbg...
> 
> James
> 
> > -----Original Message-----
> > From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
> > bounces@lists.xensource.com] On Behalf Of James Harper
> > Sent: Wednesday, 26 January 2011 17:19
> > To: xen-devel@lists.xensource.com
> > Cc: Tim Deegan
> > Subject: [Xen-devel] AMD crash on restore. bug in MTRR restore.
> 4.0.2-rc2-pre
> >
> > The mtrr save code on my AMD system reads the first MTRR as
> > 0x00000000f0000000,0x000000fff8000800. When it tries to restore in
> > mtrr_var_range_msr_set, 0x000000fff8000800 is rejected as invalid
and
> > not restored.
> >
> > Commenting out the check against the mask, as well as Tim''s
patch to
> fix
> > the segment registers makes the output of xen-hvmctx match before
the
> > save and after the restore.
> >
> > Windows still crashes on restore though so I don''t know if
this has
> > anything to do with it.
> >
> > James
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>>> On 27.01.11 at 04:49, "James Harper"
<james.harper@bendigoit.com.au> wrote:
> By saving the sysenter msrs in my PV drivers before the suspend
> hypercall and restoring them after, save/restore now works fine.
> 
> Does nobody else see a save/restore failure on AMD as a blocker to
> 4.0.2? It it something particular to my exact AMD CPU and OS (Windows
> 2008 x32)?
The 32-bit-ness of your Windows may matter: Looking at
xen/arch/hvm/svm/svm.c, I cannot see how
v->arch.hvm_svm.guest_sysenter_* would ever get set
in this case (given that the intercept for the respective MSRs
gets disabled in svm_update_guest_efer()), yet
svm_vmcb_save() reads this field unconditionally, whereas I
would think v->arch.hvm_svm.vmcb->sysenter_* should be
read/written in that case.

That code seems to go back to c/s 19856, fixing a boot
problem introduced in c/s 19648, which is the one that
removed the reads of the VMCB fields. Christoph?

Jan
>> -----Original Message-----
>> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
>> bounces@lists.xensource.com] On Behalf Of James Harper
>> Sent: Thursday, 27 January 2011 13:32
>> To: xen-devel@lists.xensource.com 
>> Cc: Tim Deegan
>> Subject: RE: [Xen-devel] AMD crash on restore. bug in MTRR
> restore.4.0.2-rc2-
>> pre
>> 
>> Additionally, WinDbg tells me that MSR''s 175, and 176 are
wiped out by
>> save/restore:
>> 
>> immediately before suspend
>> kd> rdmsr 174
>> msr[174] = 00000000`00000008
>> kd> rdmsr 175
>> msr[175] = 00000000`8039e000
>> kd> rdmsr 176
>> msr[176] = 00000000`816a4950
>> 
>> immediately after restore
>> kd> rdmsr 174
>> msr[174] = 00000000`00000000
>> kd> rdmsr 175
>> msr[175] = 00000000`00000000
>> kd> rdmsr 176
>> msr[176] = 00000000`00000000
>> 
>> although you''re never too sure with WinDbg...
>> 
>> James


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi James,

Could you try the attached patch? The issue is because we don''t
intercept SYSENTER_xx MSRs under 32bit mode (guest mode). As a result,
hvm_svm.guest_sysenter_xx contain invalid values. So save/restore will
eventually fail by using these values. This patch solves the problem.

If it works, please let me know and I will submit a formal one.

Best,
-Wei

On Wed, 2011-01-26 at 21:49 -0600, James Harper wrote:
> By saving the sysenter msrs in my PV drivers before the suspend
> hypercall and restoring them after, save/restore now works fine.
> 
> Does nobody else see a save/restore failure on AMD as a blocker to
> 4.0.2? It it something particular to my exact AMD CPU and OS (Windows
> 2008 x32)?
> 
> James
> 
> > -----Original Message-----
> > From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
> > bounces@lists.xensource.com] On Behalf Of James Harper
> > Sent: Thursday, 27 January 2011 13:32
> > To: xen-devel@lists.xensource.com
> > Cc: Tim Deegan
> > Subject: RE: [Xen-devel] AMD crash on restore. bug in MTRR
> restore.4.0.2-rc2-
> > pre
> > 
> > Additionally, WinDbg tells me that MSR''s 175, and 176 are
wiped out by
> > save/restore:
> > 
> > immediately before suspend
> > kd> rdmsr 174
> > msr[174] = 00000000`00000008
> > kd> rdmsr 175
> > msr[175] = 00000000`8039e000
> > kd> rdmsr 176
> > msr[176] = 00000000`816a4950
> > 
> > immediately after restore
> > kd> rdmsr 174
> > msr[174] = 00000000`00000000
> > kd> rdmsr 175
> > msr[175] = 00000000`00000000
> > kd> rdmsr 176
> > msr[176] = 00000000`00000000
> > 
> > although you''re never too sure with WinDbg...
> > 
> > James
> > 
> > > -----Original Message-----
> > > From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
> > > bounces@lists.xensource.com] On Behalf Of James Harper
> > > Sent: Wednesday, 26 January 2011 17:19
> > > To: xen-devel@lists.xensource.com
> > > Cc: Tim Deegan
> > > Subject: [Xen-devel] AMD crash on restore. bug in MTRR restore.
> > 4.0.2-rc2-pre
> > >
> > > The mtrr save code on my AMD system reads the first MTRR as
> > > 0x00000000f0000000,0x000000fff8000800. When it tries to restore
in
> > > mtrr_var_range_msr_set, 0x000000fff8000800 is rejected as invalid
> and
> > > not restored.
> > >
> > > Commenting out the check against the mask, as well as
Tim''s patch to
> > fix
> > > the segment registers makes the output of xen-hvmctx match before
> the
> > > save and after the restore.
> > >
> > > Windows still crashes on restore though so I don''t know
if this has
> > > anything to do with it.
> > >
> > > James
> > >
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xensource.com
> > > http://lists.xensource.com/xen-devel
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel