Jason Wang
2020-Dec-30 08:41 UTC
[RFC v2 09/13] vduse: Add support for processing vhost iotlb message
On 2020/12/30 ??3:09, Yongji Xie wrote:> On Wed, Dec 30, 2020 at 2:11 PM Jason Wang <jasowang at redhat.com> wrote: >> >> On 2020/12/29 ??6:26, Yongji Xie wrote: >>> On Tue, Dec 29, 2020 at 5:11 PM Jason Wang <jasowang at redhat.com> wrote: >>>> >>>> ----- Original Message ----- >>>>> On Mon, Dec 28, 2020 at 4:43 PM Jason Wang <jasowang at redhat.com> wrote: >>>>>> On 2020/12/28 ??4:14, Yongji Xie wrote: >>>>>>>> I see. So all the above two questions are because VHOST_IOTLB_INVALIDATE >>>>>>>> is expected to be synchronous. This need to be solved by tweaking the >>>>>>>> current VDUSE API or we can re-visit to go with descriptors relaying >>>>>>>> first. >>>>>>>> >>>>>>> Actually all vdpa related operations are synchronous in current >>>>>>> implementation. The ops.set_map/dma_map/dma_unmap should not return >>>>>>> until the VDUSE_UPDATE_IOTLB/VDUSE_INVALIDATE_IOTLB message is replied >>>>>>> by userspace. Could it solve this problem? >>>>>> I was thinking whether or not we need to generate IOTLB_INVALIDATE >>>>>> message to VDUSE during dma_unmap (vduse_dev_unmap_page). >>>>>> >>>>>> If we don't, we're probably fine. >>>>>> >>>>> It seems not feasible. This message will be also used in the >>>>> virtio-vdpa case to notify userspace to unmap some pages during >>>>> consistent dma unmapping. Maybe we can document it to make sure the >>>>> users can handle the message correctly. >>>> Just to make sure I understand your point. >>>> >>>> Do you mean you plan to notify the unmap of 1) streaming DMA or 2) >>>> coherent DMA? >>>> >>>> For 1) you probably need a workqueue to do that since dma unmap can >>>> be done in irq or bh context. And if usrspace does't do the unmap, it >>>> can still access the bounce buffer (if you don't zap pte)? >>>> >>> I plan to do it in the coherent DMA case. >> >> Any reason for treating coherent DMA differently? >> > Now the memory of the bounce buffer is allocated page by page in the > page fault handler. So it can't be used in coherent DMA mapping case > which needs some memory with contiguous virtual addresses. I can use > vmalloc() to do allocation for the bounce buffer instead. But it might > cause some memory waste. Any suggestion?I may miss something. But I don't see a relationship between the IOTLB_UNMAP and vmalloc().> >>> It's true that userspace can >>> access the dma buffer if userspace doesn't do the unmap. But the dma >>> pages would not be freed and reused unless user space called munmap() >>> for them. >> >> I wonder whether or not we could recycle IOVA in this case to avoid the >> IOTLB_UMAP message. >> > We can achieve that if we use vmalloc() to do allocation for the > bounce buffer which can be used in coherent DMA mapping case. But > looks like we still have no way to avoid the IOTLB_UMAP message in > vhost-vdpa case.I think that's fine. For virtio-vdpa, from VDUSE userspace perspective, it works like a driver that is using SWIOTLB in this case. Thanks> > Thanks, > Yongji >