On Tue, 22 Feb 2022 12:02:14 +0800, Jason Wang <jasowang at redhat.com>
wrote:>
> ? 2022/2/21 ??7:23, Xuan Zhuo ??:
> > On Mon, 21 Feb 2022 11:32:52 +0800, Jason Wang <jasowang at
redhat.com> wrote:
> >> On Fri, Feb 18, 2022 at 5:00 PM Xuan Zhuo <xuanzhuo at
linux.alibaba.com> wrote:
> >>> On Thu, 17 Feb 2022 15:19:44 +0800, Jason Wang <jasowang at
redhat.com> wrote:
> >>>> On Thu, Feb 10, 2022 at 4:51 PM Xuan Zhuo <xuanzhuo at
linux.alibaba.com> wrote:
> >>>>> virtqueue_add() only supports virtual addresses, dma
is completed in
> >>>>> virtqueue_add().
> >>>>>
> >>>>> In some scenarios (such as the AF_XDP scenario), DMA
is completed in advance, so
> >>>>> it is necessary for us to support passing the DMA
address to virtqueue_add().
> >>>> I'd suggest rename this feature as "unmanaged
DMA".
> >>> OK
> >>>
> >>>>> Record this predma information in extra->flags,
which can be skipped when
> >>>>> executing dma unmap.
> >>>> Question still, can we use per-virtqueue flag instead of
per
> >>>> descriptor flag? If my memory is correct, the answer is
yes in the
> >>>> discussion for the previous version.
> >>>>
> >>> Yes.
> >>>
> >>> per-virtqueue? I guess it should be per-submit.
> >>>
> >>> This patch set only adds a flag to desc_extra[head].flags, so
that we can know
> >>> if we need to unmap dma when we detach.
> >> I meant if we can manage to make it per virtqueue, there's no
need to
> >> maintain per buffer flag.
> >>
> > Rethinking this question, I feel there is no essential difference
between per
> > virtqueue and per sgs.
> >
> > per virtqueue:
> > 1. add buf:
> > a. check vq->premapped for map every sg
> > 2. detach:
> > a. check vq->premaped for unmap
> >
> > per sgs:
> > 1. add buf:
> > a. check function parameter "premapped" for map
every sg
> > b. add flag to extra[head].flag
> >
> > 2. detach:
> > a: check extra[head].flag for unmap
> >
> >
> > Thanks.
>
>
> Per-virtqueue is still a little bit easier at the first glance.
>
> Actually, per-sg have one advantage: it can be used without virtqueue
> reset (to allow switching between the two modes). But I'm not sure
> whether we had such requirements.
>
> I think to answer this question, we probably need a real use case (if we
> can come up with a case that is more lightweight than AF_XDP, that would
> be even better).
Sadly, I didn't think of other scenarios. Hope someone can give a scenario.
For per virtqueue, virtio-net will also switch to premapped. Because the tx
queue is shared.
But in the process of implementing this, I encountered a troublesome problem. We
need to record the dma address in virtnet. For tx, since skb contains multiple
frags, there will be many dma addresses. When unmap in virtnet It will be more
troublesome. Because we have to regain these dma addresses.
I think of two ways:
1. Let virtio return the addr of each desc when detached.
2. Allocate a block of memory for each sq/rq to hold the dma address.
Thanks.
>
> Thanks
>
>
> >
> >
> >> So we know something that needs to be mapped by virtio core
itself,
> >> e.g the indirect page. Other than this, all the rest could be
> >> pre-mapped.
> >>
> >> For vnet header, it could be mapped by virtio-net which could be
still
> >> treated as pre mapped DMA since it's not the virtio ring code.
> >>
> >> Anything I miss here?
> >>
> >> Thanks
> >>
> >>
> >>> Thanks.
> >>>
> >>>> Thanks
> >>>>
> >>>>> v1:
> >>>>> 1. All sgs requested at one time are required to
be unified PREDMA, and several
> >>>>> of them are not supported to be PREDMA
> >>>>> 2. virtio_dma_map() is removed from this patch set
and will be submitted
> >>>>> together with the next time AF_XDP supports
virtio dma
> >>>>> 3. Added patch #2 #3 to remove the check for flags
when performing unmap
> >>>>> indirect desc
> >>>>>
> >>>>> Xuan Zhuo (6):
> >>>>> virtio: rename vring_unmap_state_packed() to
> >>>>> vring_unmap_extra_packed()
> >>>>> virtio: remove flags check for unmap split indirect
desc
> >>>>> virtio: remove flags check for unmap packed
indirect desc
> >>>>> virtio: virtqueue_add() support predma
> >>>>> virtio: split: virtqueue_add_split() support dma
address
> >>>>> virtio: packed: virtqueue_add_packed() support dma
address
> >>>>>
> >>>>> drivers/virtio/virtio_ring.c | 199
++++++++++++++++++++++-------------
> >>>>> 1 file changed, 126 insertions(+), 73 deletions(-)
> >>>>>
> >>>>> --
> >>>>> 2.31.0
> >>>>>
>