Linux Virtualization - Aug 2007 - [PATCH] Add I/O hypercalls for i386 paravirt

In general, I/O in a virtual guest is subject to performance problems.  
The I/O can not be completed physically, but must be virtualized.  This 
means trapping and decoding port I/O instructions from the guest OS.  
Not only is the trap for a #GP heavyweight, both in the processor and 
the hypervisor (which usually has a complex #GP path), but this forces 
the hypervisor to decode the individual instruction which has faulted.  
Worse, even with hardware assist such as VT, the exit reason alone is 
not sufficient to determine the true nature of the faulting instruction, 
requiring a complex and costly instruction decode and simulation.

This patch provides hypercalls for the i386 port I/O instructions, which 
vastly helps guests which use native-style drivers.  For certain VMI 
workloads, this provides a performance boost of up to 30%.  We expect 
KVM and lguest to be able to achieve similar gains on I/O intensive 
workloads.

This patch is against 2.6.23-rc2-mm2, and should be targeted for 2.6.24.

Zach
-------------- next part --------------
A non-text attachment was scrubbed...
Name: i386-mm-paravirt-io-ops.patch
Type: text/x-patch
Size: 9003 bytes
Desc: not available
Url :
http://lists.linux-foundation.org/pipermail/virtualization/attachments/20070821/f8983e4f/i386-mm-paravirt-io-ops.bin

Zachary Amsden wrote:
> In general, I/O in a virtual guest is subject to performance
> problems.  The I/O can not be completed physically, but must be
> virtualized.  This means trapping and decoding port I/O instructions
> from the guest OS.  Not only is the trap for a #GP heavyweight, both
> in the processor and the hypervisor (which usually has a complex #GP
> path), but this forces the hypervisor to decode the individual
> instruction which has faulted.  Worse, even with hardware assist such
> as VT, the exit reason alone is not sufficient to determine the true
> nature of the faulting instruction, requiring a complex and costly
> instruction decode and simulation.
>
> This patch provides hypercalls for the i386 port I/O instructions,
> which vastly helps guests which use native-style drivers.  For certain
> VMI workloads, this provides a performance boost of up to 30%.  We
> expect KVM and lguest to be able to achieve similar gains on I/O
> intensive workloads.
>

Won't these workloads be better off using paravirtualized drivers? 
i.e., do the native drivers with paravirt I/O instructions get anywhere
near the performance of paravirt drivers?


-- 
Do not meddle in the internals of kernels, for they are subtle and quick to
panic.

Zachary Amsden wrote:
> In general, I/O in a virtual guest is subject to performance problems. 
> The I/O can not be completed physically, but must be virtualized.  This
> means trapping and decoding port I/O instructions from the guest OS. 
> Not only is the trap for a #GP heavyweight, both in the processor and
> the hypervisor (which usually has a complex #GP path), but this forces
> the hypervisor to decode the individual instruction which has faulted. 
> Worse, even with hardware assist such as VT, the exit reason alone is
> not sufficient to determine the true nature of the faulting instruction,
> requiring a complex and costly instruction decode and simulation.
> 
> This patch provides hypercalls for the i386 port I/O instructions, which
> vastly helps guests which use native-style drivers.  For certain VMI
> workloads, this provides a performance boost of up to 30%.  We expect
> KVM and lguest to be able to achieve similar gains on I/O intensive
> workloads.
> 
What about cost on hardware?

	-hpa

On Tue, Aug 21, 2007 at 10:23:14PM -0700, Zachary Amsden
wrote:
> In general, I/O in a virtual guest is subject to performance problems.  
> The I/O can not be completed physically, but must be virtualized.  This 
> means trapping and decoding port I/O instructions from the guest OS.  
> Not only is the trap for a #GP heavyweight, both in the processor and 
> the hypervisor (which usually has a complex #GP path), but this forces 
> the hypervisor to decode the individual instruction which has faulted.  
Is that really that expensive? Hard to imagine.

e.g. you could always have a fast check for inb/outb at the beginning
of the #GP handler. And is your initial #GP entry really more expensive
than a hypercall? 
> Worse, even with hardware assist such as VT, the exit reason alone is 
> not sufficient to determine the true nature of the faulting instruction, 
> requiring a complex and costly instruction decode and simulation.
It's unclear to me why that should be that costly.

Worst case it's a switch()

-Andi

Zachary Amsden wrote:
> This patch provides hypercalls for the i386 port I/O instructions,
> which vastly helps guests which use native-style drivers.  For certain
> VMI workloads, this provides a performance boost of up to 30%.  We
> expect KVM and lguest to be able to achieve similar gains on I/O
> intensive workloads.
Two comments:

- I should dust off my "break up paravirt_ops" patch, and this would
fit
nicely into it (I think we already discussed this)

- What happens if you *don't* want to pv some of the io instructions? 
What if you have a device which is directly exposed to the guest?

    J

On Tue, 2007-08-21 at 22:23 -0700, Zachary Amsden wrote:
> In general, I/O in a virtual guest is subject to performance problems.  
> The I/O can not be completed physically, but must be virtualized.  This 
> means trapping and decoding port I/O instructions from the guest OS.  
> Not only is the trap for a #GP heavyweight, both in the processor and 
> the hypervisor (which usually has a complex #GP path), but this forces 
> the hypervisor to decode the individual instruction which has faulted.  
> Worse, even with hardware assist such as VT, the exit reason alone is 
> not sufficient to determine the true nature of the faulting instruction, 
> requiring a complex and costly instruction decode and simulation.
 .../...

How about userland ? Things like X do IO's typically... You still need
to trap/emulate for these no ?

Cheers,
Ben.