Linux Virtualization - Jun 2020 - [PATCH v3 25/75] x86/sev-es: Add support for handling IOIO exceptions

On Tue, Apr 28, 2020 at 05:16:35PM +0200, Joerg Roedel
wrote:
> From: Tom Lendacky <thomas.lendacky at amd.com>
> 
> Add support for decoding and handling #VC exceptions for IOIO events.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> [ jroedel at suse.de: Adapted code to #VC handling framework ]
> Co-developed-by: Joerg Roedel <jroedel at suse.de>
> Signed-off-by: Joerg Roedel <jroedel at suse.de>
> ---
>  arch/x86/boot/compressed/sev-es.c |  32 +++++
>  arch/x86/kernel/sev-es-shared.c   | 202 ++++++++++++++++++++++++++++++
>  2 files changed, 234 insertions(+)
> 
> diff --git a/arch/x86/boot/compressed/sev-es.c
b/arch/x86/boot/compressed/sev-es.c
> index 1241697dd156..17765e471e28 100644
> --- a/arch/x86/boot/compressed/sev-es.c
> +++ b/arch/x86/boot/compressed/sev-es.c
> @@ -23,6 +23,35 @@
...
> +static enum es_result vc_handle_ioio(struct ghcb *ghcb, struct es_em_ctxt
*ctxt)
> +{
> +	struct pt_regs *regs = ctxt->regs;
> +	u64 exit_info_1, exit_info_2;
> +	enum es_result ret;
> +
> +	ret = vc_ioio_exitinfo(ctxt, &exit_info_1);
> +	if (ret != ES_OK)
> +		return ret;
> +
> +	if (exit_info_1 & IOIO_TYPE_STR) {
> +		int df = (regs->flags & X86_EFLAGS_DF) ? -1 : 1;
> +		unsigned int io_bytes, exit_bytes;
> +		unsigned int ghcb_count, op_count;
> +		unsigned long es_base;
> +		u64 sw_scratch;
> +
> +		/*
> +		 * For the string variants with rep prefix the amount of in/out
> +		 * operations per #VC exception is limited so that the kernel
> +		 * has a chance to take interrupts an re-schedule while the
> +		 * instruction is emulated.
Doesn't this also suppress single-step #DBs?
> +		 */
> +		io_bytes   = (exit_info_1 >> 4) & 0x7;
> +		ghcb_count = sizeof(ghcb->shared_buffer) / io_bytes;
> +
> +		op_count    = (exit_info_1 & IOIO_REP) ? regs->cx : 1;
> +		exit_info_2 = min(op_count, ghcb_count);
> +		exit_bytes  = exit_info_2 * io_bytes;
> +
> +		es_base = insn_get_seg_base(ctxt->regs, INAT_SEG_REG_ES);
> +
> +		if (!(exit_info_1 & IOIO_TYPE_IN)) {
> +			ret = vc_insn_string_read(ctxt,
> +					       (void *)(es_base + regs->si),
SEV(-ES) is 64-bit only, why bother with the es_base charade?
> +					       ghcb->shared_buffer, io_bytes,
> +					       exit_info_2, df);
df handling is busted, it's aways non-zero.  Same goes for the SI/DI
adjustments below.
> +			if (ret)
> +				return ret;
> +		}
> +
> +		sw_scratch = __pa(ghcb) + offsetof(struct ghcb, shared_buffer);
> +		ghcb_set_sw_scratch(ghcb, sw_scratch);
> +		ret = sev_es_ghcb_hv_call(ghcb, ctxt, SVM_EXIT_IOIO,
> +				   exit_info_1, exit_info_2);
> +		if (ret != ES_OK)
> +			return ret;
Batching the memory accesses and I/O accesses separately is technically
wrong, e.g. a #DB on a memory access will result in bogus data being shown
in the debugger.  In practice it seems unlikely to matter, but I'm curious
as to why string I/O is supported in the first place.  I didn't think there
was that much string I/O in the kernel?
> +
> +		/* Everything went well, write back results */
> +		if (exit_info_1 & IOIO_TYPE_IN) {
> +			ret = vc_insn_string_write(ctxt,
> +						(void *)(es_base + regs->di),
> +						ghcb->shared_buffer, io_bytes,
> +						exit_info_2, df);
> +			if (ret)
> +				return ret;
> +
> +			if (df)
> +				regs->di -= exit_bytes;
> +			else
> +				regs->di += exit_bytes;
> +		} else {
> +			if (df)
> +				regs->si -= exit_bytes;
> +			else
> +				regs->si += exit_bytes;
> +		}
> +
> +		if (exit_info_1 & IOIO_REP)
> +			regs->cx -= exit_info_2;
> +
> +		ret = regs->cx ? ES_RETRY : ES_OK;
> +
> +	} else {
> +		int bits = (exit_info_1 & 0x70) >> 1;
> +		u64 rax = 0;
> +
> +		if (!(exit_info_1 & IOIO_TYPE_IN))
> +			rax = lower_bits(regs->ax, bits);
> +
> +		ghcb_set_rax(ghcb, rax);
> +
> +		ret = sev_es_ghcb_hv_call(ghcb, ctxt, SVM_EXIT_IOIO, exit_info_1, 0);
> +		if (ret != ES_OK)
> +			return ret;
> +
> +		if (exit_info_1 & IOIO_TYPE_IN) {
> +			if (!ghcb_is_valid_rax(ghcb))
> +				return ES_VMM_ERROR;
> +			regs->ax = lower_bits(ghcb->save.rax, bits);
> +		}
> +	}
> +
> +	return ret;
> +}
> -- 
> 2.17.1
>

Hi Jean,

On Tue, May 19, 2020 at 11:20:55PM -0700, Sean Christopherson
wrote:
> On Tue, Apr 28, 2020 at 05:16:35PM +0200, Joerg Roedel wrote:
> > +		/*
> > +		 * For the string variants with rep prefix the amount of in/out
> > +		 * operations per #VC exception is limited so that the kernel
> > +		 * has a chance to take interrupts an re-schedule while the
> > +		 * instruction is emulated.
> 
> Doesn't this also suppress single-step #DBs?
Yes it does.
> 
> > +		 */
> > +		io_bytes   = (exit_info_1 >> 4) & 0x7;
> > +		ghcb_count = sizeof(ghcb->shared_buffer) / io_bytes;
> > +
> > +		op_count    = (exit_info_1 & IOIO_REP) ? regs->cx : 1;
> > +		exit_info_2 = min(op_count, ghcb_count);
> > +		exit_bytes  = exit_info_2 * io_bytes;
> > +
> > +		es_base = insn_get_seg_base(ctxt->regs, INAT_SEG_REG_ES);
> > +
> > +		if (!(exit_info_1 & IOIO_TYPE_IN)) {
> > +			ret = vc_insn_string_read(ctxt,
> > +					       (void *)(es_base + regs->si),
> 
> SEV(-ES) is 64-bit only, why bother with the es_base charade?
User-space can also cause IOIO #VC exceptions, and user-space can be
32-bit legacy code with segments, so es_base has to be taken into
account.

> 
> > +					       ghcb->shared_buffer, io_bytes,
> > +					       exit_info_2, df);
> 
> df handling is busted, it's aways non-zero.  Same goes for the SI/DI
> adjustments below.
Right, this is fixed now.
> Batching the memory accesses and I/O accesses separately is technically
> wrong, e.g. a #DB on a memory access will result in bogus data being shown
> in the debugger.  In practice it seems unlikely to matter, but I'm
curious
> as to why string I/O is supported in the first place.  I didn't think
there
> was that much string I/O in the kernel?
True, #DBs won't be correct anymore. Currently debugging is not
supported in SEV-ES guests anyway, but if it is supported the #DB
exception would happen in the #VC handler and not on the original
instruction.


Regards,

	Joerg

On Wed, Jun 03, 2020 at 04:23:25PM +0200, Joerg Roedel
wrote:
> > > +		 */
> > > +		io_bytes   = (exit_info_1 >> 4) & 0x7;
> > > +		ghcb_count = sizeof(ghcb->shared_buffer) / io_bytes;
> > > +
> > > +		op_count    = (exit_info_1 & IOIO_REP) ? regs->cx : 1;
> > > +		exit_info_2 = min(op_count, ghcb_count);
> > > +		exit_bytes  = exit_info_2 * io_bytes;
> > > +
> > > +		es_base = insn_get_seg_base(ctxt->regs, INAT_SEG_REG_ES);
> > > +
> > > +		if (!(exit_info_1 & IOIO_TYPE_IN)) {
> > > +			ret = vc_insn_string_read(ctxt,
> > > +					       (void *)(es_base + regs->si),
> > 
> > SEV(-ES) is 64-bit only, why bother with the es_base charade?
> 
> User-space can also cause IOIO #VC exceptions, and user-space can be
> 32-bit legacy code with segments, so es_base has to be taken into
> account.
Is there actually a use case for this?  Exposing port IO to userspace
doesn't exactly improve security.

Given that i386 ABI requires EFLAGS.DF=0 upon function entry/exit, i.e. is
the de facto default, the DF bug implies this hasn't been tested.  And I
don't see how this could possibly have worked for SEV given that the kernel
unrolls string I/O because the VMM can't emulate string I/O.  Presumably
someone would have complained if they "needed" to run legacy crud. 
The
host and guest obviously need major updates, so supporting e.g. DPDK with
legacy virtio seems rather silly.
> > > +					       ghcb->shared_buffer, io_bytes,
> > > +					       exit_info_2, df);
> > 
> > df handling is busted, it's aways non-zero.  Same goes for the
SI/DI
> > adjustments below.
> 
> Right, this is fixed now.
> 
> > Batching the memory accesses and I/O accesses separately is
technically
> > wrong, e.g. a #DB on a memory access will result in bogus data being
shown
> > in the debugger.  In practice it seems unlikely to matter, but I'm
curious
> > as to why string I/O is supported in the first place.  I didn't
think there
> > was that much string I/O in the kernel?
> 
> True, #DBs won't be correct anymore. Currently debugging is not
> supported in SEV-ES guests anyway, but if it is supported the #DB
> exception would happen in the #VC handler and not on the original
> instruction.
As in, the guest can't debug itself?  Or the host can't debug the guest?