Virtualization - Oct 2022 - Issue with VDUSE (QSD vduse-blk export) and vhost-vdpa

Hi Xie,
I was testing libblkio [1] with QSD vduse-blk export and had some 
issues.

In a nutshell, QSD prints me the following messages when using 
vhost-vdpa to access the device:

  Failed to get vq[0] iova mapping
  Failed to update vring for vq[0]

This happens only with vhost-vdpa, using virtio-vdpa instead the device 
works fine.
I'm using Linux v6.0 and QEMU master (commit 
214a8da23651f2472b296b3293e619fd58d9e212).

I haven't had much time to investigate, I hope to do it next week, but 
maybe it's much faster for you.

I saw that ioctl(VDUSE_IOTLB_GET_FD) in libvduse.c returns -1 (EPERM), 
so IIUC in the kernel vduse_dev_broken() was called, and the device is 
in a broken state.


We will use libblkio in QEMU [2] to access vDPA devices via vhost-vdpa.  
But I'm doing these tests without QEMU for now, using an example inside 
the libblkio repo:

# Build libblkio and examples
    # Fedora/CentOS/RHEL
    dnf install -y git meson rust cargo python3-docutils rustfmt
    # Debian/Ubuntu
    apt-get install -y git meson rustc cargo python3-docutils

    git clone https://gitlab.com/libblkio/libblkio.git

    cd libblkio
    git checkout v1.1.0

    meson setup build
    meson compile -C build


# On terminal 1
    modprobe vduse
    modprobe vhost-vdpa

    qemu-img create -f qcow2 -o preallocation=full /path/to/test.qcow2 1g

    qemu-storage-daemon \
      --blockdev
file,filename=/path/to/test.qcow2,cache.direct=on,aio=native,node-name=file \
      --blockdev qcow2,file=file,node-name=qcow2 \
      --object iothread,id=iothread0 \
      --export
vduse-blk,id=vduse0,name=vduse0,num-queues=1,node-name=qcow2,writable=on,iothread=iothread0


# On terminal 2
    vdpa dev add name vduse0 mgmtdev vduse

    cd libblkio/build

    # blkio-bench executes
    ./examples/blkio-bench virtio-blk-vhost-vdpa \
      path=/dev/vhost-vdpa-0 --runtime=5 --readwrite=randread

    # after this step, QSD (running on terminal 1) prints the following
messages:
      Failed to get vq[0] iova mapping
      Failed to update vring for vq[0]

I don't know if I'm doing something wrong or in libblkio we have some 
issue, but using vdpa-sim-blk works correctly, so maybe there is 
something in vduse that is missing.

Any help or suggestion is welcome :-)

Thanks,
Stefano

[1] https://libblkio.gitlab.io/libblkio/
[2] 
https://lore.kernel.org/qemu-devel/20221013185908.1297568-1-stefanha at
redhat.com/