Virtualization - Dec 2020 - [External] Re: [PATCH 0/7] Introduce vdpa management tool

> From: Yongji Xie <xieyongji at bytedance.com>
> Sent: Tuesday, December 1, 2020 3:26 PM
> 
> On Tue, Dec 1, 2020 at 2:25 PM Jason Wang <jasowang at redhat.com>
wrote:
> >
> >
> > On 2020/11/30 ??3:07, Yongji Xie wrote:
> > >>> Thanks for adding me, Jason!
> > >>>
> > >>> Now I'm working on a v2 patchset for VDUSE (vDPA
Device in
> > >>> Userspace) [1]. This tool is very useful for the vduse
device. So
> > >>> I'm considering integrating this into my v2 patchset.
But there is
> > >>> one problem?
> > >>>
> > >>> In this tool, vdpa device config action and enable action
are
> > >>> combined into one netlink msg: VDPA_CMD_DEV_NEW. But in
vduse
> > >>> case, it needs to be splitted because a chardev should be
created
> > >>> and opened by a userspace process before we enable the
vdpa device
> > >>> (call vdpa_register_device()).
> > >>>
> > >>> So I'd like to know whether it's possible (or
have some plans) to
> > >>> add two new netlink msgs something like:
VDPA_CMD_DEV_ENABLE
> and
> > >>> VDPA_CMD_DEV_DISABLE to make the config path more
flexible.
> > >>>
> > >> Actually, we've discussed such intermediate step in some
early
> > >> discussion. It looks to me VDUSE could be one of the users of
this.
> > >>
> > >> Or I wonder whether we can switch to use anonymous inode(fd)
for
> > >> VDUSE then fetching it via an VDUSE_GET_DEVICE_FD ioctl?
> > >>
> > > Yes, we can. Actually the current implementation in VDUSE is like
> > > this.  But seems like this is still a intermediate step. The fd
> > > should be binded to a name or something else which need to be
> > > configured before.
> >
> >
> > The name could be specified via the netlink. It looks to me the real
> > issue is that until the device is connected with a userspace, it
can't
> > be used. So we also need to fail the enabling if it doesn't
opened.
> >
> 
> Yes, that's true. So you mean we can firstly try to fetch the fd binded
to a
> name/vduse_id via an VDUSE_GET_DEVICE_FD, then use the
> name/vduse_id as a attribute to create vdpa device? It looks fine to me.
I probably do not well understand. I tried reading patch [1] and few things do
not look correct as below.
Creating the vdpa device on the bus device and destroying the device from the
workqueue seems unnecessary and racy.

It seems vduse driver needs 
This is something should be done as part of the vdpa dev add command, instead of
connecting two sides separately and ensuring race free access to it.

So VDUSE_DEV_START and VDUSE_DEV_STOP should possibly be avoided.

$ vdpa dev add parentdev vduse_mgmtdev type net name foo2

When above command is executed it creates necessary vdpa device foo2 on the bus.
When user binds foo2 device with the vduse driver, in the probe(), it creates
respective char device to access it from user space.
Depending on which driver foo2 device is bound it, it can be used, either via
(a) existing vhost stack  or (b) some vdpa Netdev driver? (not sure its current
state), or (c) vduse user space.

This will have sane model to me without races unless I am missing something
fundamental here.
This way there are not two ways to create vdpa devices from user space.
Consumers can be of different types (vhost, vduse etc) of the bus device as
above mentioned.

[1] https://www.spinics.net/lists/linux-mm/msg231581.html