On Thu, Oct 29, 2015 at 05:18:56PM +0100, David Woodhouse wrote:> On Thu, 2015-10-29 at 11:01 +0200, Michael S. Tsirkin wrote: > > > > Example: you have a mix of assigned devices and virtio devices. You > > don't trust your assigned device vendor not to corrupt your memory so > > you want to limit the damage your assigned device can do to your > > guest, > > so you use an IOMMU for that. Thus existing iommu=pt within guest is > > out. > > > > But you trust your hypervisor (you have no choice anyway), > > and you don't want the overhead of tweaking IOMMU > > on data path for virtio. Thus iommu=on is out too. > > That's not at all special for virtio or guest VMs. Even with real > hardware, we might want performance from *some* devices, and security > from others. See the DMA_ATTR_IOMMU_BYPASS which is currently being > discussed.Right. So let's wait for that discussion to play out?> But of course the easy answer in *your* case it just to ask the > hypervisor not to put the virtio devices behind an IOMMU at all. Which > we were planning to remain the default behaviour.One can't do this for x86 ATM, can one?> In all cases, the DMA API shall do the right thing.I have no problem with that. For example, can we teach the DMA API on intel x86 to use PT for virtio by default? That would allow merging Andy's patches with full compatibility with old guests and hosts.> -- > dwmw2 > >-- MST
On Sun, Nov 08, 2015 at 12:37:47PM +0200, Michael S. Tsirkin wrote:> I have no problem with that. For example, can we teach > the DMA API on intel x86 to use PT for virtio by default? > That would allow merging Andy's patches with > full compatibility with old guests and hosts.Well, the only incompatibility comes from an experimental qemu feature, more explicitly from a bug in that features implementation. So why should we work around that in the kernel? I think it is not too hard to fix qemu to generate a correct DMAR table which excludes the virtio devices from iommu translation. Joerg
On Sun, 2015-11-08 at 12:37 +0200, Michael S. Tsirkin wrote:> On Thu, Oct 29, 2015 at 05:18:56PM +0100, David Woodhouse wrote: > > On Thu, 2015-10-29 at 11:01 +0200, Michael S. Tsirkin wrote: > > > > > > But you trust your hypervisor (you have no choice anyway), > > > and you don't want the overhead of tweaking IOMMU > > > on data path for virtio. Thus iommu=on is out too. > > > > That's not at all special for virtio or guest VMs. Even with real > > hardware, we might want performance from *some* devices, and security > > from others. See the DMA_ATTR_IOMMU_BYPASS which is currently being > > discussed. > > Right. So let's wait for that discussion to play out?That discussion is purely about a requested optimisation. This one is about correctness.> > But of course the easy answer in *your* case it just to ask the > > hypervisor not to put the virtio devices behind an IOMMU at all. Which > > we were planning to remain the default behaviour. > > One can't do this for x86 ATM, can one?The converse is true, in fact ? currently, there's no way to tell? qemu-system-x86 that you *do* want it to put the virtio devices behind the emulated IOMMU, as it has no support for that. Which is a bit sad really, since the DMAR table that qemu advertises to the guest does *tell* the guest that the virtio devices are behind the emulated IOMMU. In the short term, we'll be fixing the DMAR table, and still not actually making it possible to put the virtio devices behind the emulated IOMMU. In the fullness of time, however, we *will* be fixing the qemu IOMMU code so that it can translate for virtio devices ? and for assigned physical devices, which I believe are also broken at the moment when qemu emulates an IOMMU.> > In all cases, the DMA API shall do the right thing. > > I have no problem with that. For example, can we teach > the DMA API on intel x86 to use PT for virtio by default? > That would allow merging Andy's patches with > full compatibility with old guests and hosts.A quirk so that we *notice* the bug in the existing qemu DMAR table, and disbelieve what it says about the virtio devices? Alternatively, we could just recognise that the emulated IOMMU support in qemu is an experimental feature and doesn't work right, yet. Are people really using it in anger? If we do want to do a quirk, then we should make it get it right for assigned devices too. To start with, do you want to try to express the criteria for "the DMAR table lies and <this> device is actually untranslated" in a form of prose which could reasonably be translated into code? -- David Woodhouse Open Source Technology Centre David.Woodhouse at intel.com Intel Corporation -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5691 bytes Desc: not available URL: <http://lists.linuxfoundation.org/pipermail/virtualization/attachments/20151108/d8d37fe4/attachment.bin>
On Sun, Nov 08, 2015 at 12:49:46PM +0100, Joerg Roedel wrote:> On Sun, Nov 08, 2015 at 12:37:47PM +0200, Michael S. Tsirkin wrote: > > I have no problem with that. For example, can we teach > > the DMA API on intel x86 to use PT for virtio by default? > > That would allow merging Andy's patches with > > full compatibility with old guests and hosts. > > Well, the only incompatibility comes from an experimental qemu feature, > more explicitly from a bug in that features implementation. So why > should we work around that in the kernel? I think it is not too hard to > fix qemu to generate a correct DMAR table which excludes the virtio > devices from iommu translation. > > > JoergIt's not that easy - you'd have to dedicate some buses for iommu bypass, and teach management tools to only put virtio there - but it's possible. This will absolutely address guests that don't need to set up IOMMU for virtio devices, and virtio that bypasses the IOMMU. But the problem is that we do want to *allow* guests to set up IOMMU for virtio devices. In that case, these are two other usecases: A- monolitic virtio within QEMU: iommu only needed for VFIO -> guest should always use iommu=pt iommu=on works but is just useless overhead. B- modular out of process virtio outside QEMU: iommu needed for VFIO or kernel driver -> guest should use iommu=pt or iommu=on depending on security/performance requirements Note that there could easily be a mix of these in the same system. So for these cases we do need QEMU to specify to guest that IOMMU covers the virtio devices. Also, once one does this, the default on linux is iommu=on and not pt, which works but ATM is very slow. This poses three problems: 1. How do we address the different needs of A and B? One way would be for virtio to pass the information to guest using some virtio specific way, and have drivers specify what kind of DMA access they want. 2. (Kind of a subset of 1) once we do allow IOMMU, how do we make sure most guests use the more sensible iommu=pt. 3. Once we do allow IOMMU, how can we keep existing guests work in this configuration? Creating different hypervisor configurations depending on guest is very nasty. Again, one way would be some virtio specific interface. I'd rather we figured the answers to this before merging Andy's patches because I'm concerned that instead of 1 broken configuration (virtio always bypasses IOMMU) we'll get two bad configurations (in the second one, virtio uses the slow default with no gain in security). Suggestions wellcome. -- MST