Linux Virtualization - Sep 2014 - [PATCH v5 2/3] virtio_pci: Use the DMA API for virtqueues when possible

On Sep 19, 2014 2:33 PM, "Benjamin Herrenschmidt"
<benh at kernel.crashing.org> wrote:
>
> On Wed, 2014-09-17 at 09:07 -0700, Andy Lutomirski wrote:
> > It shouldn't.  That being said, at some point this problem will
need
> > solving on PPC, and this patch doesn't help much, other than
adding
> > the virtio_ring piece.
> >
> > I'd really like to see the generic or arch IOMMU code handle this
so
> > that the PPC special case in the virtio-pci driver can go away.
> >
> > Another reason that this kind of special casing doesn't belong in
> > virtio-pci: if anyone ever binds vfio to a virtio-pci device, if the
> > DMA API doesn't work right, then the kernel will go boom or be
> > compromised.
>
> Well, that's also for these reasons that I think your patch
shouldn't go
> through those virtio_* variants that test that "use_dma_api" flag
or
> whatever you called it last time I looked :-)
>
> You should just unconditionally call the DMA API and we should fixup the
> special cases by overriding the hooks.
Unconditionally for virtio-pci or unconditionally for all drivers?  I
wouldn't be surprised if some of the virtio-mmio archs have trouble,
too.  Aarch64, at least, seems to be okay with using the DMA API on
virtio_mmio devices, even with IOMMU support compiled in.

Are there other relevant architectures that are easy to test?
>
> I can help you do the override hack for PPC for now, until we can do
> something saner accross all architectures.
Sure.

The question is: should the patches go in to 3.18 as is our should
they wait?  It would be straightforward to remove the use_dma_api
switch PPC, s390, and virtio_mmio are ready.

--Andy
>
> The main problem is s390 but I had a quick chat with Utz Bacher earlier
> this week and he seemed to think having s390 use the DMA API shouldn't
> be a big deal, they can also hookup some transparent set of ops if
> necessary.
>
> Cheers,
> Ben.
>
>

On Fri, 2014-09-19 at 22:59 -0700, Andy Lutomirski
wrote:
> Sure.
> 
> The question is: should the patches go in to 3.18 as is our should
> they wait?  It would be straightforward to remove the use_dma_api
> switch PPC, s390, and virtio_mmio are ready.
I don't mind the patches going in now in their current form with one
exception (see below), but I'm keen on moving all the way to an
unconditional use of the DMA API with a simple tweaking of the dma ops
of the device based on need/iommu/etc...

What I might do is first experiment with powerpc to try to set
use_dma_api to true and do the dma_ops manipulation.

The exception I mentioned is that I would really like the virtio device
to expose via whatever transport we chose to use (though capability
exchange sounds like a reasonable one) whether the "server"
implementation is bypassing IOMMUs or not instead on relying on client
side heuristics.

IE. Basically, we are trying to "guess" with an ifdef CONFIG_PPC, what
is essentially an attribute of the server-side, ie, whether is bypasses
the iommu for the PCI bus it resides on.

I believe all the arguments about whether this should be a bus property
or whether the x86 case can be worked around via ACPI tables etc... are
all moot. Today, qemu implementation can put virtio devices on busses
with an iommu and bypass it, so at the very least for backward
compatibility, we should expose that appropriately from the "server"
side.

Cheers,
Ben.

On Sun, 2014-09-21 at 15:03 +1000, Benjamin Herrenschmidt wrote:
> The exception I mentioned is that I would really like the virtio device
> to expose via whatever transport we chose to use (though capability
> exchange sounds like a reasonable one) whether the "server"
> implementation is bypassing IOMMUs or not instead on relying on client
> side heuristics.
> 
> IE. Basically, we are trying to "guess" with an ifdef CONFIG_PPC,
what
> is essentially an attribute of the server-side, ie, whether is bypasses
> the iommu for the PCI bus it resides on.
> I believe all the arguments about whether this should be a bus property
> or whether the x86 case can be worked around via ACPI tables etc... are
> all moot. Today, qemu implementation can put virtio devices on busses
> with an iommu and bypass it, so at the very least for backward
> compatibility, we should expose that appropriately from the
"server"
> side.
And of course, since we are talking about backward compatibility with
existing qemus here, the capability should be the opposite, ie "honor
iommu", with the assumption that without it, the implementation bypasses
it, which reflects what the current qemu implementation does on any
architecture, whether you configure the bus to have an iommu emulated on
it or not.

Cheers,
Ben.