Linux Virtualization - Apr 2023 - [PATCH vhost v6 08/11] virtio_ring: introduce virtqueue_dma_dev()

On Tue, Apr 11, 2023 at 10:09:40AM +0800, Jason Wang
wrote:
> We want to support AF_XDP for virtio-net. It means AF_XDP needs to
> know the dma device to perform DMA mapping. So we introduce a helper
> to expose the dma dev of the virtio device.
The whole virtio architecture is based around the core code doing
the DMA mapping.  I can't see how you can just export a helper to
expose the dma device.  You'd have to complete rework the layering
of the virtio code if you want to do it in the upper level drivers.
And why would you want to do this?  The low-level code is the only
piece that can actually know if you need to do a dma mapping.  All
the kernel subsystems that don't do it inside the low-level drivers
or helpers closely associtated are a giant and hard to fix map
(see usb for the prime exhibit).

So the first question is:  why do you want this for XF_ADP, and
the next question will be how to do that without making a complete
mess.
> This works fine as long as VIRTIO_F_ACCESS_PLATFORM is negotiated. But
> when it is not, the virtio driver needs to use a physical address so
> we want to expose the virtio device without dma_ops in the hope that
> it will go for direct mapping where the physical address is used. But
> it may not work on some specific setups (arches that assume an IOMMU
> or have arch dma ops).
The DMA device for virtio_pci is the underlying PCI device, always.
!VIRTIO_F_ACCESS_PLATFORM means there is no dma device at all.  Because
of all these things you can't just expose a pointer to the dma_device
as that is just a completely wrong way of thinking about the problem.

On Tue, Apr 11, 2023 at 11:31?AM Christoph Hellwig <hch at infradead.org>
wrote:
>
> On Tue, Apr 11, 2023 at 10:09:40AM +0800, Jason Wang wrote:
> > We want to support AF_XDP for virtio-net. It means AF_XDP needs to
> > know the dma device to perform DMA mapping. So we introduce a helper
> > to expose the dma dev of the virtio device.
>
> The whole virtio architecture is based around the core code doing
> the DMA mapping.  I can't see how you can just export a helper to
> expose the dma device.  You'd have to complete rework the layering
> of the virtio code if you want to do it in the upper level drivers.
> And why would you want to do this?  The low-level code is the only
> piece that can actually know if you need to do a dma mapping.  All
> the kernel subsystems that don't do it inside the low-level drivers
> or helpers closely associtated are a giant and hard to fix map
> (see usb for the prime exhibit).
>
> So the first question is:  why do you want this for XF_ADP,
Xuan, is it possible to set up the DMA mapping inside the virtio
driver itself? I vaguely remember at least the RX buffer mapping is
done by the driver. If this is true, we can avoid exposing DMA details
to the upper layer.
> and
> the next question will be how to do that without making a complete
> mess.
>
> > This works fine as long as VIRTIO_F_ACCESS_PLATFORM is negotiated. But
> > when it is not, the virtio driver needs to use a physical address so
> > we want to expose the virtio device without dma_ops in the hope that
> > it will go for direct mapping where the physical address is used. But
> > it may not work on some specific setups (arches that assume an IOMMU
> > or have arch dma ops).
>
> The DMA device for virtio_pci is the underlying PCI device, always.
> !VIRTIO_F_ACCESS_PLATFORM means there is no dma device at all.  Because
> of all these things you can't just expose a pointer to the dma_device
> as that is just a completely wrong way of thinking about the problem.
Ok, so if there's no DMA at all we should avoid using the DMA API
completely. This means we should check dma_dev against NULL in
virtio_has_dma_quirk().

Thanks
>