Xen devel - Oct 2013 - [PATCH 4/4 V2] XSA-60 security hole: flush cache when vmentry back to UC guest

From 561fa67689b40b60b22437ccac3945a92458c00b Mon Sep 17 00:00:00 2001
From: Liu Jinsong <jinsong.liu@intel.com>
Date: Thu, 31 Oct 2013 06:38:15 +0800
Subject: [PATCH 4/4 V2] XSA-60 security hole: flush cache when vmentry back to
UC guest

This patch flush cache when vmentry back to UC guest, to prevent
cache polluted by hypervisor access guest memory during UC mode.

The elegant way to do this is, simply add wbinvd just before vmentry.
However, currently wbinvd before vmentry will mysteriously trigger
lapic timer interrupt storm, hung booting stage for 10s ~ 60s. We still
didn''t dig out the root cause of interrupt storm, so currently this
patch add flag indicating hypervisor access UC guest memory to prevent
interrupt storm -- though it still leaves aspects un-addressed, i.e.
speculative reads, and multi-vCPU issues, etc.

Whenever the interrupt storm got root caused and fixed, the protection
flag can be removed -- that would be final clean and elegant approach
dealing with cache flushing before vmentry.

Suggested-by: Jan Beulich <jbeulich@suse.com>
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Liu Jinsong <jinsong.liu@intel.com>
---
 xen/arch/x86/hvm/hvm.c         |    6 ++++++
 xen/arch/x86/hvm/vmx/vmx.c     |    7 +++++++
 xen/include/asm-x86/hvm/vcpu.h |    1 +
 3 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index df021de..8fc396c 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -2483,6 +2483,9 @@ static enum hvm_copy_result __hvm_copy(
         return HVMCOPY_unhandleable;
 #endif
 
+    if ( unlikely(curr->arch.hvm_vcpu.cache_mode == NO_FILL_CACHE_MODE) )
+        curr->arch.hvm_vcpu.hypervisor_access_uc_hvm_memory = 1;
+
     while ( todo > 0 )
     {
         count = min_t(int, PAGE_SIZE - (addr & ~PAGE_MASK), todo);
@@ -2596,6 +2599,9 @@ static enum hvm_copy_result __hvm_clear(paddr_t addr, int
size)
         return HVMCOPY_unhandleable;
 #endif
 
+    if ( unlikely(curr->arch.hvm_vcpu.cache_mode == NO_FILL_CACHE_MODE) )
+        curr->arch.hvm_vcpu.hypervisor_access_uc_hvm_memory = 1;
+
     while ( todo > 0 )
     {
         count = min_t(int, PAGE_SIZE - (addr & ~PAGE_MASK), todo);
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index d846a9c..2674410 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -2974,6 +2974,13 @@ void vmx_vmenter_helper(const struct cpu_user_regs *regs)
     struct hvm_vcpu_asid *p_asid;
     bool_t need_flush;
 
+    /* In case hypervisor access hvm memory when guest uc mode */
+    if ( unlikely(curr->arch.hvm_vcpu.hypervisor_access_uc_hvm_memory) )
+    {
+        curr->arch.hvm_vcpu.hypervisor_access_uc_hvm_memory = 0;
+        wbinvd();
+    }
+
     if ( !cpu_has_vmx_vpid )
         goto out;
     if ( nestedhvm_vcpu_in_guestmode(curr) )
diff --git a/xen/include/asm-x86/hvm/vcpu.h b/xen/include/asm-x86/hvm/vcpu.h
index e8b8cd7..58ed509 100644
--- a/xen/include/asm-x86/hvm/vcpu.h
+++ b/xen/include/asm-x86/hvm/vcpu.h
@@ -159,6 +159,7 @@ struct hvm_vcpu {
 
     /* Which cache mode is this VCPU in (CR0:CD/NW)? */
     u8                  cache_mode;
+    bool_t              hypervisor_access_uc_hvm_memory;
 
     struct hvm_vcpu_io  hvm_io;
 
-- 
1.7.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

>>> On 30.10.13 at 18:17, "Liu, Jinsong"
<jinsong.liu@intel.com> wrote:
> From 561fa67689b40b60b22437ccac3945a92458c00b Mon Sep 17 00:00:00 2001
> From: Liu Jinsong <jinsong.liu@intel.com>
> Date: Thu, 31 Oct 2013 06:38:15 +0800
> Subject: [PATCH 4/4 V2] XSA-60 security hole: flush cache when vmentry back
> to UC guest
VMX maintainers: The whole series is awaiting your comments or
acks.

Thanks, Jan

On Tue, Nov 5, 2013 at 6:30 AM, Jan Beulich <JBeulich@suse.com>
wrote:
>>>> On 30.10.13 at 18:17, "Liu, Jinsong"
<jinsong.liu@intel.com> wrote:
>> From 561fa67689b40b60b22437ccac3945a92458c00b Mon Sep 17 00:00:00 2001
>> From: Liu Jinsong <jinsong.liu@intel.com>
>> Date: Thu, 31 Oct 2013 06:38:15 +0800
>> Subject: [PATCH 4/4 V2] XSA-60 security hole: flush cache when vmentry
back
>> to UC guest
>
> VMX maintainers: The whole series is awaiting your comments or
> acks.
>
> Thanks, Jan
>
It seems sensible for the time being.

Acked-by: Jun Nakajima <jun.nakajima@intel.com>


-- 
Jun
Intel Open Source Technology Center