Virtualization - Jan 2022 - [PATCH 0/6] virtio: support advance DMA

On Fri, Jan 07, 2022 at 02:33:00PM +0800, Xuan Zhuo
wrote:
> virtqueue_add() only supports virtual addresses, dma is completed in
> virtqueue_add().
> 
> In some scenarios (such as the AF_XDP scenario), DMA is completed in
advance, so
> it is necessary for us to support passing the DMA address to
virtqueue_add().
> 
> This patch set stipulates that if sg->dma_address is not NULL, use this
> address as the DMA address. And record this information in extra->flags,
> which can be skipped when executing dma unmap.
> 
>     extra->flags |= VRING_DESC_F_PREDMA;
> 
> But the indirect desc does not have a corresponding extra, so the second
and
> third patches of this patch set are to allocate the corresponding extra
while
> allocating the indirect desc. Each desc must have a corresponding extra
because
> it is possible in an sgs some are advance DMA, while others are virtual
> addresses. So we must allocate an extra for each indirect desc.

I didn't realize AF_XDP didn't have space to stuff the header into.
Jason, is that expected? It would be best to fix that, performance is
best if header is linear with the data ...
Or maybe we can reduce the use of indirect somewhat, at least while
the ring is mostly empty?
> Xuan Zhuo (6):
>   virtio: rename vring_unmap_state_packed() to
>     vring_unmap_extra_packed()
>   virtio: split: alloc indirect desc with extra
>   virtio: packed: alloc indirect desc with extra
>   virtio: split: virtqueue_add_split() support dma address
>   virtio: packed: virtqueue_add_packed() support dma address
>   virtio: add api virtio_dma_map() for advance dma
> 
>  drivers/virtio/virtio_ring.c | 387 ++++++++++++++++++++---------------
>  include/linux/virtio.h       |   9 +
>  2 files changed, 232 insertions(+), 164 deletions(-)
> 
> --
> 2.31.0

On Mon, Jan 10, 2022 at 5:59 PM Michael S. Tsirkin <mst at redhat.com>
wrote:
>
> On Fri, Jan 07, 2022 at 02:33:00PM +0800, Xuan Zhuo wrote:
> > virtqueue_add() only supports virtual addresses, dma is completed in
> > virtqueue_add().
> >
> > In some scenarios (such as the AF_XDP scenario), DMA is completed in
advance, so
> > it is necessary for us to support passing the DMA address to
virtqueue_add().
> >
> > This patch set stipulates that if sg->dma_address is not NULL, use
this
> > address as the DMA address. And record this information in
extra->flags,
> > which can be skipped when executing dma unmap.
> >
> >     extra->flags |= VRING_DESC_F_PREDMA;
> >
> > But the indirect desc does not have a corresponding extra, so the
second and
> > third patches of this patch set are to allocate the corresponding
extra while
> > allocating the indirect desc. Each desc must have a corresponding
extra because
> > it is possible in an sgs some are advance DMA, while others are
virtual
> > addresses. So we must allocate an extra for each indirect desc.
>
>
> I didn't realize AF_XDP didn't have space to stuff the header into.
> Jason, is that expected?
I might be wrong but it looks to me AF_XDP allows to reserve
sufficient headroom via xdp_umem_reg_v1.
> It would be best to fix that, performance is
> best if header is linear with the data ...
This looks like a must otherwise we may meet trouble in zerocopy receive.

Thanks
> Or maybe we can reduce the use of indirect somewhat, at least while
> the ring is mostly empty?
>
> > Xuan Zhuo (6):
> >   virtio: rename vring_unmap_state_packed() to
> >     vring_unmap_extra_packed()
> >   virtio: split: alloc indirect desc with extra
> >   virtio: packed: alloc indirect desc with extra
> >   virtio: split: virtqueue_add_split() support dma address
> >   virtio: packed: virtqueue_add_packed() support dma address
> >   virtio: add api virtio_dma_map() for advance dma
> >
> >  drivers/virtio/virtio_ring.c | 387
++++++++++++++++++++---------------
> >  include/linux/virtio.h       |   9 +
> >  2 files changed, 232 insertions(+), 164 deletions(-)
> >
> > --
> > 2.31.0
>