In general, I/O in a virtual guest is subject to performance problems. The I/O can not be completed physically, but must be virtualized. This means trapping and decoding port I/O instructions from the guest OS. Not only is the trap for a #GP heavyweight, both in the processor and the hypervisor (which usually has a complex #GP path), but this forces the hypervisor to decode the individual instruction which has faulted. Worse, even with hardware assist such as VT, the exit reason alone is not sufficient to determine the true nature of the faulting instruction, requiring a complex and costly instruction decode and simulation. This patch provides hypercalls for the i386 port I/O instructions, which vastly helps guests which use native-style drivers. For certain VMI workloads, this provides a performance boost of up to 30%. We expect KVM and lguest to be able to achieve similar gains on I/O intensive workloads. This patch is against 2.6.23-rc2-mm2, and should be targeted for 2.6.24. Zach -------------- next part -------------- A non-text attachment was scrubbed... Name: i386-mm-paravirt-io-ops.patch Type: text/x-patch Size: 9003 bytes Desc: not available Url : http://lists.linux-foundation.org/pipermail/virtualization/attachments/20070821/f8983e4f/i386-mm-paravirt-io-ops.bin
Zachary Amsden wrote:> In general, I/O in a virtual guest is subject to performance > problems. The I/O can not be completed physically, but must be > virtualized. This means trapping and decoding port I/O instructions > from the guest OS. Not only is the trap for a #GP heavyweight, both > in the processor and the hypervisor (which usually has a complex #GP > path), but this forces the hypervisor to decode the individual > instruction which has faulted. Worse, even with hardware assist such > as VT, the exit reason alone is not sufficient to determine the true > nature of the faulting instruction, requiring a complex and costly > instruction decode and simulation. > > This patch provides hypercalls for the i386 port I/O instructions, > which vastly helps guests which use native-style drivers. For certain > VMI workloads, this provides a performance boost of up to 30%. We > expect KVM and lguest to be able to achieve similar gains on I/O > intensive workloads. >Won't these workloads be better off using paravirtualized drivers? i.e., do the native drivers with paravirt I/O instructions get anywhere near the performance of paravirt drivers? -- Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Zachary Amsden wrote:> In general, I/O in a virtual guest is subject to performance problems. > The I/O can not be completed physically, but must be virtualized. This > means trapping and decoding port I/O instructions from the guest OS. > Not only is the trap for a #GP heavyweight, both in the processor and > the hypervisor (which usually has a complex #GP path), but this forces > the hypervisor to decode the individual instruction which has faulted. > Worse, even with hardware assist such as VT, the exit reason alone is > not sufficient to determine the true nature of the faulting instruction, > requiring a complex and costly instruction decode and simulation. > > This patch provides hypercalls for the i386 port I/O instructions, which > vastly helps guests which use native-style drivers. For certain VMI > workloads, this provides a performance boost of up to 30%. We expect > KVM and lguest to be able to achieve similar gains on I/O intensive > workloads. >What about cost on hardware? -hpa
On Tue, Aug 21, 2007 at 10:23:14PM -0700, Zachary Amsden wrote:> In general, I/O in a virtual guest is subject to performance problems. > The I/O can not be completed physically, but must be virtualized. This > means trapping and decoding port I/O instructions from the guest OS. > Not only is the trap for a #GP heavyweight, both in the processor and > the hypervisor (which usually has a complex #GP path), but this forces > the hypervisor to decode the individual instruction which has faulted.Is that really that expensive? Hard to imagine. e.g. you could always have a fast check for inb/outb at the beginning of the #GP handler. And is your initial #GP entry really more expensive than a hypercall?> Worse, even with hardware assist such as VT, the exit reason alone is > not sufficient to determine the true nature of the faulting instruction, > requiring a complex and costly instruction decode and simulation.It's unclear to me why that should be that costly. Worst case it's a switch() -Andi
Zachary Amsden wrote:> This patch provides hypercalls for the i386 port I/O instructions, > which vastly helps guests which use native-style drivers. For certain > VMI workloads, this provides a performance boost of up to 30%. We > expect KVM and lguest to be able to achieve similar gains on I/O > intensive workloads.Two comments: - I should dust off my "break up paravirt_ops" patch, and this would fit nicely into it (I think we already discussed this) - What happens if you *don't* want to pv some of the io instructions? What if you have a device which is directly exposed to the guest? J
On Tue, 2007-08-21 at 22:23 -0700, Zachary Amsden wrote:> In general, I/O in a virtual guest is subject to performance problems. > The I/O can not be completed physically, but must be virtualized. This > means trapping and decoding port I/O instructions from the guest OS. > Not only is the trap for a #GP heavyweight, both in the processor and > the hypervisor (which usually has a complex #GP path), but this forces > the hypervisor to decode the individual instruction which has faulted. > Worse, even with hardware assist such as VT, the exit reason alone is > not sufficient to determine the true nature of the faulting instruction, > requiring a complex and costly instruction decode and simulation..../... How about userland ? Things like X do IO's typically... You still need to trap/emulate for these no ? Cheers, Ben.
Possibly Parallel Threads
- [PATCH] Add I/O hypercalls for i386 paravirt
- [RFC, PATCH 0/24] VMI i386 Linux virtualization interface proposal
- [RFC, PATCH 0/24] VMI i386 Linux virtualization interface proposal
- Paravirt-ops VMI / Xen / lrustyvisor merge status
- Paravirt-ops VMI / Xen / lrustyvisor merge status