Linux Virtualization - Apr 2023 - [PATCH vhost v6 08/11] virtio_ring: introduce virtqueue_dma_dev()

On Mon, 10 Apr 2023 23:20:59 -0700, Christoph Hellwig <hch at
infradead.org> wrote:
> On Tue, Apr 11, 2023 at 02:11:17PM +0800, Xuan Zhuo wrote:
> > NO, all dma maping is done inside xdp socket. That is done
> > when setup.
>
> That is just completely broken, virtio or not.  Highlevel code like
> sockets must never do dma mappings themselves.
AF_XDP provides some API for net driver. This API will do dma map or dma sync.

cc AF_XDP maintainers.

If we cannot set Virtio Deivce to a direct DMA device, then we can only make
some modifications in AF_XDP.

Thanks.

> _______________________________________________
> Virtualization mailing list
> Virtualization at lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/virtualization

On Tue, Apr 11, 2023 at 02:28:00PM +0800, Xuan Zhuo
wrote:
> > That is just completely broken, virtio or not.  Highlevel code like
> > sockets must never do dma mappings themselves.
> 
> AF_XDP provides some API for net driver. This API will do dma map or dma
sync.
> 
> cc AF_XDP maintainers.
So in that case AF_XDP doesn't actually require a dma device as you
claimed early and gets the layering right after all.  Just don't use that
API from a network driver that doesn't need to do dma mappings like
virtio for the !platform_access case then.

Here, this has cc the maintainers of AF_XDP.

For on the same page, I summarize it.

We discusses this question last at [1]. We planned to pass one device to xsk.
Then xsk can use the DMA API on this device. The device can be one hw
device(such as PCI, mmio) or virtio device. If it is one hw device, no problem.
If it is one virtio device, then we expect that DMA API on that will return
direct dma map, because the dma_ops of the virtio device is NULL. But on some
platform such as sparc64 or some case of x86, the arch has own dma_ops. So we
wrong.

And as the discuss of this thread, we cannot get direct dma address from
virtio device by DMA API. So, we need xsk to expose the DMA ops to the
virtio-net driver when virtio core can not use the DMA API.

All the processing of dma inside xsk:
1. map/unmap at enable/disable
2. dma sync

Solution:
1. xsk uses phy memory and passes the dma sync when the dev is null.
2. xsk uses the callbacks passed by driver to do the dma.

If I miss something, please point out.

Thanks.

[1] https://lore.kernel.org/virtualization/1677727282.6062915-2-xuanzhuo at
linux.alibaba.com/