Jason Wang
2023-Apr-11 03:56 UTC
[PATCH vhost v6 08/11] virtio_ring: introduce virtqueue_dma_dev()
On Tue, Apr 11, 2023 at 11:31?AM Christoph Hellwig <hch at infradead.org> wrote:> > On Tue, Apr 11, 2023 at 10:09:40AM +0800, Jason Wang wrote: > > We want to support AF_XDP for virtio-net. It means AF_XDP needs to > > know the dma device to perform DMA mapping. So we introduce a helper > > to expose the dma dev of the virtio device. > > The whole virtio architecture is based around the core code doing > the DMA mapping. I can't see how you can just export a helper to > expose the dma device. You'd have to complete rework the layering > of the virtio code if you want to do it in the upper level drivers. > And why would you want to do this? The low-level code is the only > piece that can actually know if you need to do a dma mapping. All > the kernel subsystems that don't do it inside the low-level drivers > or helpers closely associtated are a giant and hard to fix map > (see usb for the prime exhibit). > > So the first question is: why do you want this for XF_ADP,Xuan, is it possible to set up the DMA mapping inside the virtio driver itself? I vaguely remember at least the RX buffer mapping is done by the driver. If this is true, we can avoid exposing DMA details to the upper layer.> and > the next question will be how to do that without making a complete > mess. > > > This works fine as long as VIRTIO_F_ACCESS_PLATFORM is negotiated. But > > when it is not, the virtio driver needs to use a physical address so > > we want to expose the virtio device without dma_ops in the hope that > > it will go for direct mapping where the physical address is used. But > > it may not work on some specific setups (arches that assume an IOMMU > > or have arch dma ops). > > The DMA device for virtio_pci is the underlying PCI device, always. > !VIRTIO_F_ACCESS_PLATFORM means there is no dma device at all. Because > of all these things you can't just expose a pointer to the dma_device > as that is just a completely wrong way of thinking about the problem.Ok, so if there's no DMA at all we should avoid using the DMA API completely. This means we should check dma_dev against NULL in virtio_has_dma_quirk(). Thanks>
Christoph Hellwig
2023-Apr-11 04:09 UTC
[PATCH vhost v6 08/11] virtio_ring: introduce virtqueue_dma_dev()
On Tue, Apr 11, 2023 at 11:56:47AM +0800, Jason Wang wrote:> > The DMA device for virtio_pci is the underlying PCI device, always. > > !VIRTIO_F_ACCESS_PLATFORM means there is no dma device at all. Because > > of all these things you can't just expose a pointer to the dma_device > > as that is just a completely wrong way of thinking about the problem. > > Ok, so if there's no DMA at all we should avoid using the DMA API > completely. This means we should check dma_dev against NULL in > virtio_has_dma_quirk().No nee to add a check to virtio_has_dma_quirk. But looking at virtio_has_dma_quirk shows that virtio-gpu is pretty messed up already as well. It can't really poke into the DMA details. We'll need core virtio helpers for allocating and syncing a sg_table instead and make virtio_has_dma_quirk private to the core.
Xuan Zhuo
2023-Apr-11 06:11 UTC
[PATCH vhost v6 08/11] virtio_ring: introduce virtqueue_dma_dev()
On Tue, 11 Apr 2023 11:56:47 +0800, Jason Wang <jasowang at redhat.com> wrote:> On Tue, Apr 11, 2023 at 11:31?AM Christoph Hellwig <hch at infradead.org> wrote: > > > > On Tue, Apr 11, 2023 at 10:09:40AM +0800, Jason Wang wrote: > > > We want to support AF_XDP for virtio-net. It means AF_XDP needs to > > > know the dma device to perform DMA mapping. So we introduce a helper > > > to expose the dma dev of the virtio device. > > > > The whole virtio architecture is based around the core code doing > > the DMA mapping. I can't see how you can just export a helper to > > expose the dma device. You'd have to complete rework the layering > > of the virtio code if you want to do it in the upper level drivers. > > And why would you want to do this? The low-level code is the only > > piece that can actually know if you need to do a dma mapping. All > > the kernel subsystems that don't do it inside the low-level drivers > > or helpers closely associtated are a giant and hard to fix map > > (see usb for the prime exhibit). > > > > So the first question is: why do you want this for XF_ADP, > > Xuan, is it possible to set up the DMA mapping inside the virtio > driver itself? I vaguely remember at least the RX buffer mapping is > done by the driver. If this is true, we can avoid exposing DMA details > to the upper layer.NO, all dma maping is done inside xdp socket. That is done when setup. When adding to RX Ring, xdp socket will call DMA SYNC.> > > and > > the next question will be how to do that without making a complete > > mess. > > > > > This works fine as long as VIRTIO_F_ACCESS_PLATFORM is negotiated. But > > > when it is not, the virtio driver needs to use a physical address so > > > we want to expose the virtio device without dma_ops in the hope that > > > it will go for direct mapping where the physical address is used. But > > > it may not work on some specific setups (arches that assume an IOMMU > > > or have arch dma ops). > > > > The DMA device for virtio_pci is the underlying PCI device, always. > > !VIRTIO_F_ACCESS_PLATFORM means there is no dma device at all. Because > > of all these things you can't just expose a pointer to the dma_device > > as that is just a completely wrong way of thinking about the problem. > > Ok, so if there's no DMA at all we should avoid using the DMA API > completely. This means we should check dma_dev against NULL in > virtio_has_dma_quirk(). > > Thanks > > > > > _______________________________________________ > Virtualization mailing list > Virtualization at lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/virtualization