Linux Virtualization - Apr 2018 - [RFC] virtio: Use DMA MAP API for devices without an IOMMU

On Fri, 2018-04-06 at 00:16 -0700, Christoph Hellwig
wrote:
> On Fri, Apr 06, 2018 at 08:23:10AM +0530, Anshuman Khandual wrote:
> > On 04/06/2018 02:48 AM, Benjamin Herrenschmidt wrote:
> > > On Thu, 2018-04-05 at 21:34 +0300, Michael S. Tsirkin wrote:
> > > > > In this specific case, because that would make qemu
expect an iommu,
> > > > > and there isn't one.
> > > > 
> > > > 
> > > > I think that you can set iommu_platform in qemu without an
iommu.
> > > 
> > > No I mean the platform has one but it's not desirable for it
to be used
> > > due to the performance hit.
> > 
> > Also the only requirement is to bounce the I/O buffers through SWIOTLB
> > implemented as DMA API which the virtio core understands. There is no
> > need for an IOMMU to be involved for the device representation in this
> > case IMHO.
> 
> This whole virtio translation issue is a mess.  I think we need to
> switch it to the dma API, and then quirk the legacy case to always
> use the direct mapping inside the dma API.
Fine with using a dma API always on the Linux side, but we do want to
special case virtio still at the arch and qemu side to have a "direct
mapping" mode. Not sure how (special flags on PCI devices) to avoid
actually going through an emulated IOMMU on the qemu side, because that
slows things down, esp. with vhost.

IE, we can't I think just treat it the same as a physical device.

Cheers,
Ben.

On Fri, Apr 06, 2018 at 06:37:18PM +1000, Benjamin Herrenschmidt
wrote:
> > > implemented as DMA API which the virtio core understands. There
is no
> > > need for an IOMMU to be involved for the device representation in
this
> > > case IMHO.
> > 
> > This whole virtio translation issue is a mess.  I think we need to
> > switch it to the dma API, and then quirk the legacy case to always
> > use the direct mapping inside the dma API.
> 
> Fine with using a dma API always on the Linux side, but we do want to
> special case virtio still at the arch and qemu side to have a "direct
> mapping" mode. Not sure how (special flags on PCI devices) to avoid
> actually going through an emulated IOMMU on the qemu side, because that
> slows things down, esp. with vhost.
> 
> IE, we can't I think just treat it the same as a physical device.
We should have treated it like a physical device from the start, but
that device has unfortunately sailed.

But yes, we'll need a per-device quirk that says 'don't attach an
iommu'.

On 04/15/2018 05:41 PM, Christoph Hellwig wrote:
> On Fri, Apr 06, 2018 at 06:37:18PM +1000, Benjamin Herrenschmidt wrote:
>>>> implemented as DMA API which the virtio core understands. There
is no
>>>> need for an IOMMU to be involved for the device representation
in this
>>>> case IMHO.
>>>
>>> This whole virtio translation issue is a mess.  I think we need to
>>> switch it to the dma API, and then quirk the legacy case to always
>>> use the direct mapping inside the dma API.
>>
>> Fine with using a dma API always on the Linux side, but we do want to
>> special case virtio still at the arch and qemu side to have a
"direct
>> mapping" mode. Not sure how (special flags on PCI devices) to
avoid
>> actually going through an emulated IOMMU on the qemu side, because that
>> slows things down, esp. with vhost.
>>
>> IE, we can't I think just treat it the same as a physical device.
> 
> We should have treated it like a physical device from the start, but
> that device has unfortunately sailed.
> 
> But yes, we'll need a per-device quirk that says 'don't attach
an
> iommu'.
How about doing it per platform basis as suggested in this RFC through
an arch specific callback. Because all the virtio devices in the given
platform would require and exercise this option (to avail bounce buffer
mechanism for secure guests as an example). So the flag basically is a
platform specific one not a device specific one.