Linux Virtualization - Sep 2019 - [PATCH] vhost: introduce mdev based hardware backend

On Fri, Sep 27, 2019 at 11:27:12AM +0800, Jason Wang
wrote:
> 
> On 2019/9/26 ??9:14, Tiwei Bie wrote:
> > On Thu, Sep 26, 2019 at 04:35:18AM -0400, Michael S. Tsirkin wrote:
> > > On Thu, Sep 26, 2019 at 12:54:27PM +0800, Tiwei Bie wrote:
> > [...]
> > > > diff --git a/include/uapi/linux/vhost.h
b/include/uapi/linux/vhost.h
> > > > index 40d028eed645..5afbc2f08fa3 100644
> > > > --- a/include/uapi/linux/vhost.h
> > > > +++ b/include/uapi/linux/vhost.h
> > > > @@ -116,4 +116,12 @@
> > > >   #define VHOST_VSOCK_SET_GUEST_CID	_IOW(VHOST_VIRTIO, 0x60,
__u64)
> > > >   #define VHOST_VSOCK_SET_RUNNING		_IOW(VHOST_VIRTIO, 0x61,
int)
> > > > +/* VHOST_MDEV specific defines */
> > > > +
> > > > +#define VHOST_MDEV_SET_STATE	_IOW(VHOST_VIRTIO, 0x70,
__u64)
> > > > +
> > > > +#define VHOST_MDEV_S_STOPPED	0
> > > > +#define VHOST_MDEV_S_RUNNING	1
> > > > +#define VHOST_MDEV_S_MAX	2
> > > > +
> > > >   #endif
> > > So assuming we have an underlying device that behaves like
virtio:
> > I think they are really good questions/suggestions. Thanks!
> > 
> > > 1. Should we use SET_STATUS maybe?
> > I like this idea. I will give it a try.
> > 
> > > 2. Do we want a reset ioctl?
> > I think it is helpful. If we use SET_STATUS, maybe we
> > can use it to support the reset.
> > 
> > > 3. Do we want ability to enable rings individually?
> > I will make it possible at least in the vhost layer.
> 
> 
> Note the API support e.g set_vq_ready().
virtio spec calls this "enabled" so let's stick to that.
> 
> > 
> > > 4. Does device need to limit max ring size?
> > > 5. Does device need to limit max number of queues?
> > I think so. It's helpful to have ioctls to report the max
> > ring size and max number of queues.
> 
> 
> An issue is the max number of queues is done through a device specific way,
> usually device configuration space. This is supported by the transport API,
> but how to expose it to userspace may need more thought.
> 
> Thanks
an ioctl for device config?  But for v1 I'd be quite happy to just have
a minimal working device with 2 queues.
> 
> > 
> > Thanks!

On 2019/9/27 ??5:41, Michael S. Tsirkin wrote:
> On Fri, Sep 27, 2019 at 11:27:12AM +0800, Jason Wang wrote:
>> On 2019/9/26 ??9:14, Tiwei Bie wrote:
>>> On Thu, Sep 26, 2019 at 04:35:18AM -0400, Michael S. Tsirkin wrote:
>>>> On Thu, Sep 26, 2019 at 12:54:27PM +0800, Tiwei Bie wrote:
>>> [...]
>>>>> diff --git a/include/uapi/linux/vhost.h
b/include/uapi/linux/vhost.h
>>>>> index 40d028eed645..5afbc2f08fa3 100644
>>>>> --- a/include/uapi/linux/vhost.h
>>>>> +++ b/include/uapi/linux/vhost.h
>>>>> @@ -116,4 +116,12 @@
>>>>>    #define VHOST_VSOCK_SET_GUEST_CID	_IOW(VHOST_VIRTIO,
0x60, __u64)
>>>>>    #define VHOST_VSOCK_SET_RUNNING		_IOW(VHOST_VIRTIO,
0x61, int)
>>>>> +/* VHOST_MDEV specific defines */
>>>>> +
>>>>> +#define VHOST_MDEV_SET_STATE	_IOW(VHOST_VIRTIO, 0x70,
__u64)
>>>>> +
>>>>> +#define VHOST_MDEV_S_STOPPED	0
>>>>> +#define VHOST_MDEV_S_RUNNING	1
>>>>> +#define VHOST_MDEV_S_MAX	2
>>>>> +
>>>>>    #endif
>>>> So assuming we have an underlying device that behaves like
virtio:
>>> I think they are really good questions/suggestions. Thanks!
>>>
>>>> 1. Should we use SET_STATUS maybe?
>>> I like this idea. I will give it a try.
>>>
>>>> 2. Do we want a reset ioctl?
>>> I think it is helpful. If we use SET_STATUS, maybe we
>>> can use it to support the reset.
>>>
>>>> 3. Do we want ability to enable rings individually?
>>> I will make it possible at least in the vhost layer.
>>
>> Note the API support e.g set_vq_ready().
> virtio spec calls this "enabled" so let's stick to that.

Ok.

>
>>>> 4. Does device need to limit max ring size?
>>>> 5. Does device need to limit max number of queues?
>>> I think so. It's helpful to have ioctls to report the max
>>> ring size and max number of queues.
>>
>> An issue is the max number of queues is done through a device specific
way,
>> usually device configuration space. This is supported by the transport
API,
>> but how to expose it to userspace may need more thought.
>>
>> Thanks
> an ioctl for device config?  But for v1 I'd be quite happy to just have
> a minimal working device with 2 queues.

I'm fully agree, and it will work as long as VIRTIO_NET_F_MQ and 
VIRTIO_NET_F_CTRL_VQ is not advertised by the mdev device.

Thanks

>
>>> Thanks!

On Fri, Sep 27, 2019 at 08:17:47PM +0800, Jason Wang
wrote:
> 
> On 2019/9/27 ??5:41, Michael S. Tsirkin wrote:
> > On Fri, Sep 27, 2019 at 11:27:12AM +0800, Jason Wang wrote:
> > > On 2019/9/26 ??9:14, Tiwei Bie wrote:
> > > > On Thu, Sep 26, 2019 at 04:35:18AM -0400, Michael S. Tsirkin
wrote:
> > > > > On Thu, Sep 26, 2019 at 12:54:27PM +0800, Tiwei Bie
wrote:
> > > > [...]
> > > > > > diff --git a/include/uapi/linux/vhost.h
b/include/uapi/linux/vhost.h
> > > > > > index 40d028eed645..5afbc2f08fa3 100644
> > > > > > --- a/include/uapi/linux/vhost.h
> > > > > > +++ b/include/uapi/linux/vhost.h
> > > > > > @@ -116,4 +116,12 @@
> > > > > >    #define VHOST_VSOCK_SET_GUEST_CID
_IOW(VHOST_VIRTIO, 0x60, __u64)
> > > > > >    #define VHOST_VSOCK_SET_RUNNING	
_IOW(VHOST_VIRTIO, 0x61, int)
> > > > > > +/* VHOST_MDEV specific defines */
> > > > > > +
> > > > > > +#define VHOST_MDEV_SET_STATE	_IOW(VHOST_VIRTIO,
0x70, __u64)
> > > > > > +
> > > > > > +#define VHOST_MDEV_S_STOPPED	0
> > > > > > +#define VHOST_MDEV_S_RUNNING	1
> > > > > > +#define VHOST_MDEV_S_MAX	2
> > > > > > +
> > > > > >    #endif
> > > > > So assuming we have an underlying device that behaves
like virtio:
> > > > I think they are really good questions/suggestions. Thanks!
> > > > 
> > > > > 1. Should we use SET_STATUS maybe?
> > > > I like this idea. I will give it a try.
> > > > 
> > > > > 2. Do we want a reset ioctl?
> > > > I think it is helpful. If we use SET_STATUS, maybe we
> > > > can use it to support the reset.
> > > > 
> > > > > 3. Do we want ability to enable rings individually?
> > > > I will make it possible at least in the vhost layer.
> > > 
> > > Note the API support e.g set_vq_ready().
> > virtio spec calls this "enabled" so let's stick to that.
> 
> 
> Ok.
> 
> 
> > 
> > > > > 4. Does device need to limit max ring size?
> > > > > 5. Does device need to limit max number of queues?
> > > > I think so. It's helpful to have ioctls to report the
max
> > > > ring size and max number of queues.
> > > 
> > > An issue is the max number of queues is done through a device
specific way,
> > > usually device configuration space. This is supported by the
transport API,
> > > but how to expose it to userspace may need more thought.
> > > 
> > > Thanks
> > an ioctl for device config?  But for v1 I'd be quite happy to just
have
> > a minimal working device with 2 queues.
> 
> 
> I'm fully agree, and it will work as long as VIRTIO_NET_F_MQ and
> VIRTIO_NET_F_CTRL_VQ is not advertised by the mdev device.
> 
> Thanks
Hmm this means we need to validate the features bits,
not just pass them through to the hardware.
Problem is, how do we add more feature bits later,
without testing all hardware?
I guess this means the device specific driver must do it.

> 
> > 
> > > > Thanks!