Andy Lutomirski
2015-Jul-28 16:11 UTC
[PATCH v4 0/4] virtio: Clean up scatterlists and use the DMA API
On Jul 28, 2015 6:11 AM, "Jan Kiszka" <jan.kiszka at siemens.com> wrote:> > On 2015-07-28 15:06, Michael S. Tsirkin wrote: > > On Tue, Jul 28, 2015 at 02:46:20PM +0200, Paolo Bonzini wrote: > >> > >> > >> On 28/07/2015 12:12, Benjamin Herrenschmidt wrote: > >>>>> That is an experimental feature (it's x-iommu), so it can change. > >>>>> > >>>>> The plan was: > >>>>> > >>>>> - for PPC, virtio never honors IOMMU > >>>>> > >>>>> - for non-PPC, either have virtio always honor IOMMU, or enforce that > >>>>> virtio is not under IOMMU. > >>>>> > >>> I dislike having PPC special cased. > >>> > >>> In fact, today x86 guests also assume that virtio bypasses IOMMU I > >>> believe. In fact *all* guests do. > >> > >> This doesn't matter much, since the only guests that implement an IOMMU > >> in QEMU are (afaik) PPC and x86, and x86 does not yet promise any kind > >> of stability. > > > > Hmm I think Jan (cc) said it was already used out there. > > Yes, no known issues with vt-d emulation for almost a year now. Error > reporting could be improved, and interrupt remapping is still missing, > but those are minor issues in this context. > > In my testing setups, I also have virtio devices in use, passed through > to an L2 guest, but only in 1:1 mapping so that their broken IOMMU > support causes no practical problems. >How are you getting 1:1 to work? Is it something that L0 QEMU can advertise to L1? If so, can we just do that unconditionally, which would make my patch work? I have no objection to 1:1 devices in general. It's only devices that the PCI code on the guest identifies as not 1:1 but that are nonetheless 1:1 that cause problems.
Jan Kiszka
2015-Jul-28 16:44 UTC
[PATCH v4 0/4] virtio: Clean up scatterlists and use the DMA API
On 2015-07-28 18:11, Andy Lutomirski wrote:> On Jul 28, 2015 6:11 AM, "Jan Kiszka" <jan.kiszka at siemens.com> wrote: >> >> On 2015-07-28 15:06, Michael S. Tsirkin wrote: >>> On Tue, Jul 28, 2015 at 02:46:20PM +0200, Paolo Bonzini wrote: >>>> >>>> >>>> On 28/07/2015 12:12, Benjamin Herrenschmidt wrote: >>>>>>> That is an experimental feature (it's x-iommu), so it can change. >>>>>>> >>>>>>> The plan was: >>>>>>> >>>>>>> - for PPC, virtio never honors IOMMU >>>>>>> >>>>>>> - for non-PPC, either have virtio always honor IOMMU, or enforce that >>>>>>> virtio is not under IOMMU. >>>>>>> >>>>> I dislike having PPC special cased. >>>>> >>>>> In fact, today x86 guests also assume that virtio bypasses IOMMU I >>>>> believe. In fact *all* guests do. >>>> >>>> This doesn't matter much, since the only guests that implement an IOMMU >>>> in QEMU are (afaik) PPC and x86, and x86 does not yet promise any kind >>>> of stability. >>> >>> Hmm I think Jan (cc) said it was already used out there. >> >> Yes, no known issues with vt-d emulation for almost a year now. Error >> reporting could be improved, and interrupt remapping is still missing, >> but those are minor issues in this context. >> >> In my testing setups, I also have virtio devices in use, passed through >> to an L2 guest, but only in 1:1 mapping so that their broken IOMMU >> support causes no practical problems. >> > > How are you getting 1:1 to work? Is it something that L0 QEMU can > advertise to L1? If so, can we just do that unconditionally, which > would make my patch work?The guest hypervisor is Jailhouse and the guest is the root cell that loaded the hypervisor, thus continues with identity mappings. You usually don't have 1:1 mapping with other setups - maybe with some Xen configuration? Dunno.> > I have no objection to 1:1 devices in general. It's only devices that > the PCI code on the guest identifies as not 1:1 but that are > nonetheless 1:1 that cause problems.The ability to have virtio on systems with IOMMU in place makes testing much more efficient for us. Ideally, we would have it in non-identity mapping scenarios as well, e.g. to start secondary Linux instances in the test VMs, giving them their own virtio devices. And we will eventually have this need on ARM as well. Virtio needs to be backward compatible, so the change to put these devices under IOMMU control could be advertised during feature negotiations and controlled on QEMU side via a device property. Newer guest drivers would have to acknowledge that they support virtio via IOMMUs. Older ones would refuse to work, and the admin could instead spawn VMs with this feature disabled. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux
Andy Lutomirski
2015-Jul-28 17:10 UTC
[PATCH v4 0/4] virtio: Clean up scatterlists and use the DMA API
On Tue, Jul 28, 2015 at 9:44 AM, Jan Kiszka <jan.kiszka at siemens.com> wrote:> The ability to have virtio on systems with IOMMU in place makes testing > much more efficient for us. Ideally, we would have it in non-identity > mapping scenarios as well, e.g. to start secondary Linux instances in > the test VMs, giving them their own virtio devices. And we will > eventually have this need on ARM as well. > > Virtio needs to be backward compatible, so the change to put these > devices under IOMMU control could be advertised during feature > negotiations and controlled on QEMU side via a device property. Newer > guest drivers would have to acknowledge that they support virtio via > IOMMUs. Older ones would refuse to work, and the admin could instead > spawn VMs with this feature disabled. >The trouble is that this is really a property of the bus and not of the device. If you build a virtio device that physically plugs into a PCIe slot, the device has no concept of an IOMMU in the first place. Similarly, if you take an L0-provided IOMMU-supporting device and pass it through to L2 using current QEMU on L1 (with Q35 emulation and iommu enabled), then, from L2's perspective, the device is 1:1 no matter what the device thinks. IOW, I think the original design was wrong and now we have to deal with it. I think the best solution would be to teach QEMU to fix its ACPI tables so that 1:1 virtio devices are actually exposed as 1:1. --Andy
Seemingly Similar Threads
- [PATCH v4 0/4] virtio: Clean up scatterlists and use the DMA API
- [PATCH v4 0/4] virtio: Clean up scatterlists and use the DMA API
- [PATCH v4 0/4] virtio: Clean up scatterlists and use the DMA API
- [PATCH v4 0/4] virtio: Clean up scatterlists and use the DMA API
- [PATCH v4 0/4] virtio: Clean up scatterlists and use the DMA API