Michael S. Tsirkin
2019-Sep-27 12:46 UTC
[PATCH] vhost: introduce mdev based hardware backend
On Fri, Sep 27, 2019 at 08:17:47PM +0800, Jason Wang wrote:> > On 2019/9/27 ??5:41, Michael S. Tsirkin wrote: > > On Fri, Sep 27, 2019 at 11:27:12AM +0800, Jason Wang wrote: > > > On 2019/9/26 ??9:14, Tiwei Bie wrote: > > > > On Thu, Sep 26, 2019 at 04:35:18AM -0400, Michael S. Tsirkin wrote: > > > > > On Thu, Sep 26, 2019 at 12:54:27PM +0800, Tiwei Bie wrote: > > > > [...] > > > > > > diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h > > > > > > index 40d028eed645..5afbc2f08fa3 100644 > > > > > > --- a/include/uapi/linux/vhost.h > > > > > > +++ b/include/uapi/linux/vhost.h > > > > > > @@ -116,4 +116,12 @@ > > > > > > #define VHOST_VSOCK_SET_GUEST_CID _IOW(VHOST_VIRTIO, 0x60, __u64) > > > > > > #define VHOST_VSOCK_SET_RUNNING _IOW(VHOST_VIRTIO, 0x61, int) > > > > > > +/* VHOST_MDEV specific defines */ > > > > > > + > > > > > > +#define VHOST_MDEV_SET_STATE _IOW(VHOST_VIRTIO, 0x70, __u64) > > > > > > + > > > > > > +#define VHOST_MDEV_S_STOPPED 0 > > > > > > +#define VHOST_MDEV_S_RUNNING 1 > > > > > > +#define VHOST_MDEV_S_MAX 2 > > > > > > + > > > > > > #endif > > > > > So assuming we have an underlying device that behaves like virtio: > > > > I think they are really good questions/suggestions. Thanks! > > > > > > > > > 1. Should we use SET_STATUS maybe? > > > > I like this idea. I will give it a try. > > > > > > > > > 2. Do we want a reset ioctl? > > > > I think it is helpful. If we use SET_STATUS, maybe we > > > > can use it to support the reset. > > > > > > > > > 3. Do we want ability to enable rings individually? > > > > I will make it possible at least in the vhost layer. > > > > > > Note the API support e.g set_vq_ready(). > > virtio spec calls this "enabled" so let's stick to that. > > > Ok. > > > > > > > > > 4. Does device need to limit max ring size? > > > > > 5. Does device need to limit max number of queues? > > > > I think so. It's helpful to have ioctls to report the max > > > > ring size and max number of queues. > > > > > > An issue is the max number of queues is done through a device specific way, > > > usually device configuration space. This is supported by the transport API, > > > but how to expose it to userspace may need more thought. > > > > > > Thanks > > an ioctl for device config? But for v1 I'd be quite happy to just have > > a minimal working device with 2 queues. > > > I'm fully agree, and it will work as long as VIRTIO_NET_F_MQ and > VIRTIO_NET_F_CTRL_VQ is not advertised by the mdev device. > > ThanksHmm this means we need to validate the features bits, not just pass them through to the hardware. Problem is, how do we add more feature bits later, without testing all hardware? I guess this means the device specific driver must do it.> > > > > > > Thanks!
On 2019/9/27 ??8:46, Michael S. Tsirkin wrote:> On Fri, Sep 27, 2019 at 08:17:47PM +0800, Jason Wang wrote: >> On 2019/9/27 ??5:41, Michael S. Tsirkin wrote: >>> On Fri, Sep 27, 2019 at 11:27:12AM +0800, Jason Wang wrote: >>>> On 2019/9/26 ??9:14, Tiwei Bie wrote: >>>>> On Thu, Sep 26, 2019 at 04:35:18AM -0400, Michael S. Tsirkin wrote: >>>>>> On Thu, Sep 26, 2019 at 12:54:27PM +0800, Tiwei Bie wrote: >>>>> [...] >>>>>>> diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h >>>>>>> index 40d028eed645..5afbc2f08fa3 100644 >>>>>>> --- a/include/uapi/linux/vhost.h >>>>>>> +++ b/include/uapi/linux/vhost.h >>>>>>> @@ -116,4 +116,12 @@ >>>>>>> #define VHOST_VSOCK_SET_GUEST_CID _IOW(VHOST_VIRTIO, 0x60, __u64) >>>>>>> #define VHOST_VSOCK_SET_RUNNING _IOW(VHOST_VIRTIO, 0x61, int) >>>>>>> +/* VHOST_MDEV specific defines */ >>>>>>> + >>>>>>> +#define VHOST_MDEV_SET_STATE _IOW(VHOST_VIRTIO, 0x70, __u64) >>>>>>> + >>>>>>> +#define VHOST_MDEV_S_STOPPED 0 >>>>>>> +#define VHOST_MDEV_S_RUNNING 1 >>>>>>> +#define VHOST_MDEV_S_MAX 2 >>>>>>> + >>>>>>> #endif >>>>>> So assuming we have an underlying device that behaves like virtio: >>>>> I think they are really good questions/suggestions. Thanks! >>>>> >>>>>> 1. Should we use SET_STATUS maybe? >>>>> I like this idea. I will give it a try. >>>>> >>>>>> 2. Do we want a reset ioctl? >>>>> I think it is helpful. If we use SET_STATUS, maybe we >>>>> can use it to support the reset. >>>>> >>>>>> 3. Do we want ability to enable rings individually? >>>>> I will make it possible at least in the vhost layer. >>>> Note the API support e.g set_vq_ready(). >>> virtio spec calls this "enabled" so let's stick to that. >> >> Ok. >> >> >>>>>> 4. Does device need to limit max ring size? >>>>>> 5. Does device need to limit max number of queues? >>>>> I think so. It's helpful to have ioctls to report the max >>>>> ring size and max number of queues. >>>> An issue is the max number of queues is done through a device specific way, >>>> usually device configuration space. This is supported by the transport API, >>>> but how to expose it to userspace may need more thought. >>>> >>>> Thanks >>> an ioctl for device config? But for v1 I'd be quite happy to just have >>> a minimal working device with 2 queues. >> >> I'm fully agree, and it will work as long as VIRTIO_NET_F_MQ and >> VIRTIO_NET_F_CTRL_VQ is not advertised by the mdev device. >> >> Thanks > Hmm this means we need to validate the features bits, > not just pass them through to the hardware. > Problem is, how do we add more feature bits later, > without testing all hardware? > I guess this means the device specific driver must do it. >That looks not good, maybe a virtio device id based features blacklist in vhost-mdev. Then MQ and CTRL_VQ could be filtered out by vhost-mdev. Thanks>>>>> Thanks!
Michael S. Tsirkin
2019-Sep-27 13:23 UTC
[PATCH] vhost: introduce mdev based hardware backend
On Fri, Sep 27, 2019 at 09:17:56PM +0800, Jason Wang wrote:> > On 2019/9/27 ??8:46, Michael S. Tsirkin wrote: > > On Fri, Sep 27, 2019 at 08:17:47PM +0800, Jason Wang wrote: > > > On 2019/9/27 ??5:41, Michael S. Tsirkin wrote: > > > > On Fri, Sep 27, 2019 at 11:27:12AM +0800, Jason Wang wrote: > > > > > On 2019/9/26 ??9:14, Tiwei Bie wrote: > > > > > > On Thu, Sep 26, 2019 at 04:35:18AM -0400, Michael S. Tsirkin wrote: > > > > > > > On Thu, Sep 26, 2019 at 12:54:27PM +0800, Tiwei Bie wrote: > > > > > > [...] > > > > > > > > diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h > > > > > > > > index 40d028eed645..5afbc2f08fa3 100644 > > > > > > > > --- a/include/uapi/linux/vhost.h > > > > > > > > +++ b/include/uapi/linux/vhost.h > > > > > > > > @@ -116,4 +116,12 @@ > > > > > > > > #define VHOST_VSOCK_SET_GUEST_CID _IOW(VHOST_VIRTIO, 0x60, __u64) > > > > > > > > #define VHOST_VSOCK_SET_RUNNING _IOW(VHOST_VIRTIO, 0x61, int) > > > > > > > > +/* VHOST_MDEV specific defines */ > > > > > > > > + > > > > > > > > +#define VHOST_MDEV_SET_STATE _IOW(VHOST_VIRTIO, 0x70, __u64) > > > > > > > > + > > > > > > > > +#define VHOST_MDEV_S_STOPPED 0 > > > > > > > > +#define VHOST_MDEV_S_RUNNING 1 > > > > > > > > +#define VHOST_MDEV_S_MAX 2 > > > > > > > > + > > > > > > > > #endif > > > > > > > So assuming we have an underlying device that behaves like virtio: > > > > > > I think they are really good questions/suggestions. Thanks! > > > > > > > > > > > > > 1. Should we use SET_STATUS maybe? > > > > > > I like this idea. I will give it a try. > > > > > > > > > > > > > 2. Do we want a reset ioctl? > > > > > > I think it is helpful. If we use SET_STATUS, maybe we > > > > > > can use it to support the reset. > > > > > > > > > > > > > 3. Do we want ability to enable rings individually? > > > > > > I will make it possible at least in the vhost layer. > > > > > Note the API support e.g set_vq_ready(). > > > > virtio spec calls this "enabled" so let's stick to that. > > > > > > Ok. > > > > > > > > > > > > > 4. Does device need to limit max ring size? > > > > > > > 5. Does device need to limit max number of queues? > > > > > > I think so. It's helpful to have ioctls to report the max > > > > > > ring size and max number of queues. > > > > > An issue is the max number of queues is done through a device specific way, > > > > > usually device configuration space. This is supported by the transport API, > > > > > but how to expose it to userspace may need more thought. > > > > > > > > > > Thanks > > > > an ioctl for device config? But for v1 I'd be quite happy to just have > > > > a minimal working device with 2 queues. > > > > > > I'm fully agree, and it will work as long as VIRTIO_NET_F_MQ and > > > VIRTIO_NET_F_CTRL_VQ is not advertised by the mdev device. > > > > > > Thanks > > Hmm this means we need to validate the features bits, > > not just pass them through to the hardware. > > Problem is, how do we add more feature bits later, > > without testing all hardware? > > I guess this means the device specific driver must do it. > > > > That looks not good, maybe a virtio device id based features blacklist in > vhost-mdev. Then MQ and CTRL_VQ could be filtered out by vhost-mdev. > > ThanksTwo implementations of e.g. virtio net can have different features whitelisted. So I think there's no way but let the driver do it. We should probably provide a standard place in the ops for driver to supply the whitelist, to make sure drivers don't forget.> > > > > > > Thanks!