Xen devel - May 2006 - [PATCH][SVM][1/2] fix SVM 64bit hv cores>0 reboot/hang issue

SVM patch for 64bit hv, to reset the ss, es, ds host selectors to NULL
during a context switch to the SVM domain''s vcpu.  More detailed
description of the problem below, any alternate solutions and thoughts
are appreciated.

This patch also initializes the tlb_control to 1 for the initial
do_launch().

Applies cleanly to 9906.
Please apply.

Signed-off-by: Tom Woller <thomas.woller@amd.com>

NOTES:
This issue occurs when creating an unmodified HVM/SVM guest on cores>0,
with a 64bit hypervisor, and SMP Dom0.   The system will reboot or hang
on the initial  vmrun/launch".

The reboot has been traced to a microcode induced processor shutdown.
The shutdown is triggered when the host selectors are being restored
during the initial "vmexit".  The microcode performs a consistency
check
on each host selector restored.  Each selectors'' GDT entry being
restored must be accessible to the core performing the VMRUN/VMEXIT.  In
this failing case the host selectors being restored are DS/ES == 0x2B
(from Dom0 vcpu), and are contained in physical GDT pages/entries that
are not present in memory for core 1 - which results in a processor
shutdown.  

Basically, each host selector that is restored (during a SVM VMEXIT)
must be accessible - i.e. those GDT pages must be mapped in for the core
performing the vmexit.  A NULL selector is not checked.  

An alternate solution (besides setting ss,es,ds to NULL) would be to
determine where to place code that will ensure that each required GDT
page, associated with the host selectors to be restored, is valid and
physically present.   Are ss, es, ds set to 0x2B needed in this 64bit
context, or is the solution of setting these selectors to NULL
sufficient?   
   
We placed test code, into the svm_ctxt_switch_to() function, that
touches each GDT page,  but this code is running in Interrupt context,
so page faults result in a CPU1 FATAL TRAP 14 (page fault) during
INTERRUPT CONTEXT. 
We also tried running this same test code during the svm do_launch() but
the result is a GP fault and system crash also.

Now, cursory testing on VT boxes indicates that the 0x2B selectors are
also restored on VMX, but the VT microcode does not seem to validate
these values, and therefore does not cause a processor shutdown.  

These registers are officially ignored in 64bit mode, and zero''ing them
out seems to be a solution that is functional, but we''re unsure as to
the use of the __USER_DS 0x2B values in ds, es selectors in 64bit mode. 

Appreciate any information/thoughts,
Tom


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 3 May 2006, at 22:17, Woller, Thomas wrote:
> Now, cursory testing on VT boxes indicates that the 0x2B selectors are
> also restored on VMX, but the VT microcode does not seem to validate
> these values, and therefore does not cause a processor shutdown.
>
> These registers are officially ignored in 64bit mode, and zero''ing
them
> out seems to be a solution that is functional, but we''re unsure as
to
> the use of the __USER_DS 0x2B values in ds, es selectors in 64bit mode.
Zeroing them out is probably the best thing to do. Xen doesn''t bother 
to change them while it executes because ds/es/ss are effectively 
unused in 64-bit mode. The fact they happen to contain __USER_DS is 
utterly unimportant.

You probably need to worry about fs and gs as well. They''re not causing
you problems now because guests don''t usually set those registers to 
non-zero values, but if they ever do then you''ll presumably have 
problems?

So, I think the patch is good but needs to zero fs/gs as well. I''ll 
wait for further advice before applying this patch.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> 
> Zeroing them out is probably the best thing to do. Xen 
> doesn''t bother to change them while it executes because 
> ds/es/ss are effectively unused in 64-bit mode. The fact they 
> happen to contain __USER_DS is utterly unimportant.
Good to hear that...
> 
> You probably need to worry about fs and gs as well. They''re 
> not causing you problems now because guests don''t usually set 
> those registers to non-zero values, but if they ever do then 
> you''ll presumably have problems?
Just to be clear here, the host registers are what is giving us 
problems. When the microcode loads the host selectors in the vmexit
microcode logic, then the consistency check fails on ds,es (0x2B)
due to GDT entries not in memory at that moment. 

I think that we''ll go with not modifying gs,fs now - they are currently
NULL in the host context.
> I''ll wait for further advice before applying this patch.
So, if you would, please apply the previous patch, exactly as is.

Also, if you could please apply to the 3.0-testing tree, that would
help also.  

Thanks,
Tom


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 4 May 2006, at 19:30, Woller, Thomas wrote:
>> You probably need to worry about fs and gs as well. They''re
>> not causing you problems now because guests don''t usually set
>> those registers to non-zero values, but if they ever do then
>> you''ll presumably have problems?
> Just to be clear here, the host registers are what is giving us
> problems. When the microcode loads the host selectors in the vmexit
> microcode logic, then the consistency check fails on ds,es (0x2B)
> due to GDT entries not in memory at that moment.
>
> I think that we''ll go with not modifying gs,fs now - they are
currently
> NULL in the host context.
Okay, I think you''re right. I''ll apply the patch.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel