Virtualization - Mar 2021 - [RFC v4 06/11] vduse: Implement an MMU-based IOMMU driver

On 2021/3/5 3:59 ??, Yongji Xie wrote:
> On Fri, Mar 5, 2021 at 3:27 PM Jason Wang <jasowang at redhat.com>
wrote:
>>
>> On 2021/3/5 3:13 ??, Yongji Xie wrote:
>>> On Fri, Mar 5, 2021 at 2:52 PM Jason Wang <jasowang at
redhat.com> wrote:
>>>> On 2021/3/5 2:15 ??, Yongji Xie wrote:
>>>>
>>>> Sorry if I've asked this before.
>>>>
>>>> But what's the reason for maintaing a dedicated IOTLB here?
I think we
>>>> could reuse vduse_dev->iommu since the device can not be
used by both
>>>> virtio and vhost in the same time or use
vduse_iova_domain->iotlb for
>>>> set_map().
>>>>
>>>> The main difference between domain->iotlb and dev->iotlb
is the way to
>>>> deal with bounce buffer. In the domain->iotlb case, bounce
buffer
>>>> needs to be mapped each DMA transfer because we need to get the
bounce
>>>> pages by an IOVA during DMA unmapping. In the dev->iotlb
case, bounce
>>>> buffer only needs to be mapped once during initialization,
which will
>>>> be used to tell userspace how to do mmap().
>>>>
>>>> Also, since vhost IOTLB support per mapping token (opauqe), can
we use
>>>> that instead of the bounce_pages *?
>>>>
>>>> Sorry, I didn't get you here. Which value do you mean to
store in the
>>>> opaque pointer?
>>>>
>>>> So I would like to have a way to use a single IOTLB for manage
all kinds
>>>> of mappings. Two possible ideas:
>>>>
>>>> 1) map bounce page one by one in vduse_dev_map_page(), in
>>>> VDUSE_IOTLB_GET_FD, try to merge the result if we had the same
fd. Then
>>>> for bounce pages, userspace still only need to map it once and
we can
>>>> maintain the actual mapping by storing the page or pa in the
opaque
>>>> field of IOTLB entry.
>>>>
>>>> Looks like userspace still needs to unmap the old region and
map a new
>>>> region (size is changed) with the fd in each VDUSE_IOTLB_GET_FD
ioctl.
>>>>
>>>>
>>>> I don't get here. Can you give an example?
>>>>
>>> For example, userspace needs to process two I/O requests (one page
per
>>> request). To process the first request, userspace uses
>>> VDUSE_IOTLB_GET_FD ioctl to query the iova region (0 ~ 4096) and
mmap
>>> it.
>>
>> I think in this case we should let VDUSE_IOTLB_GET_FD return the
maximum
>> range as far as they are backed by the same fd.
>>
> But now the bounce page is mapped one by one. The second page (4096 ~
> 8192) might not be mapped when userspace is processing the first
> request. So the maximum range is 0 ~ 4096 at that time.
>
> Thanks,
> Yongji

A question, if I read the code correctly, VDUSE_IOTLB_GET_FD will return 
the whole bounce map range which is setup in vduse_dev_map_page()? So my 
understanding is that usersapce may choose to map all its range via mmap().

So if we 'map' bounce page one by one in vduse_dev_map_page(). (Here 
'map' means using multiple itree entries instead of a single one). Then 
in the VDUSE_IOTLB_GET_FD we can keep traversing itree (dev->iommu) 
until the range is backed by a different file.

With this, there's no userspace visible changes and there's no need for 
the domain->iotlb?

Thanks

>