Virtualization - Feb 2022 - [PATCH v4 09/14] virtio: queue_reset: struct virtio_config_ops add callbacks for queue_reset

? 2022/2/9 ??8:28, Xuan Zhuo ??:
> Performing reset on a queue is divided into four steps:
>
> 1. reset_vq: reset one vq
> 2. recycle the buffer from vq by virtqueue_detach_unused_buf()
> 3. release the ring of the vq by vring_release_virtqueue()
> 4. enable_reset_vq: re-enable the reset queue
>
> So add two callbacks reset_vq, enable_reset_vq to struct
> virtio_config_ops.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo at linux.alibaba.com>
> ---
>   include/linux/virtio_config.h | 13 +++++++++++++
>   1 file changed, 13 insertions(+)
>
> diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> index 4d107ad31149..0d01a64f2576 100644
> --- a/include/linux/virtio_config.h
> +++ b/include/linux/virtio_config.h
> @@ -74,6 +74,17 @@ struct virtio_shm_region {
>    * @set_vq_affinity: set the affinity for a virtqueue (optional).
>    * @get_vq_affinity: get the affinity for a virtqueue (optional).
>    * @get_shm_region: get a shared memory region based on the index.
> + * @reset_vq: reset a queue individually

This needs to be marked as optional I think.

> + *	vq: the virtqueue
> + *	Returns 0 on success or error status

It looks to me the caller should also guarantee that the vring is not 
accsed by any functions e.g NAPI.

> + *	After successfully calling this, be sure to call
> + *	virtqueue_detach_unused_buf() to recycle the buffer in the ring, and
> + *	then call vring_release_virtqueue() to release the vq ring.
> + * @enable_reset_vq: enable a reset queue
> + *	vq: the virtqueue
> + *	ring_num: specify ring num for the vq to be re-enabled. 0 means use the
> + *	          default value. MUST be a power of 2.

Note that we don't have power of 2 requirement for packed virtqueue.

And I wonder if it's cleaner to have a find_vq() ops instead to dealing 
with the re-allocation and possible size change, or have a dedicated 
helper to set vring size so driver can do.

reset_vq()

virtqueue_set_vring_size()

enable_reset_vq()

> + *	Returns 0 on success or error status
>    */
>   typedef void vq_callback_t(struct virtqueue *);
>   struct virtio_config_ops {
> @@ -100,6 +111,8 @@ struct virtio_config_ops {
>   			int index);
>   	bool (*get_shm_region)(struct virtio_device *vdev,
>   			       struct virtio_shm_region *region, u8 id);
> +	int (*reset_vq)(struct virtqueue *vq);
> +	int (*enable_reset_vq)(struct virtqueue *vq, u16 ring_num);

Note that the current implement is best-effort, so it's not guarantee 
that we can have a vring with ring_num, we may get less under memory 
pressure or even fail. We probably need to have a pamater to mandate the 
ring_num otherwise user may surprise to see a decreased size of the ring 
when a increasing is actually requested.

Thanks

>   };
>   
>   /* If driver didn't advertise the feature, it will never appear. */

On Fri, 11 Feb 2022 14:49:03 +0800, Jason Wang <jasowang at redhat.com>
wrote:
>
> ? 2022/2/9 ??8:28, Xuan Zhuo ??:
> > Performing reset on a queue is divided into four steps:
> >
> > 1. reset_vq: reset one vq
> > 2. recycle the buffer from vq by virtqueue_detach_unused_buf()
> > 3. release the ring of the vq by vring_release_virtqueue()
> > 4. enable_reset_vq: re-enable the reset queue
> >
> > So add two callbacks reset_vq, enable_reset_vq to struct
> > virtio_config_ops.
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo at linux.alibaba.com>
> > ---
> >   include/linux/virtio_config.h | 13 +++++++++++++
> >   1 file changed, 13 insertions(+)
> >
> > diff --git a/include/linux/virtio_config.h
b/include/linux/virtio_config.h
> > index 4d107ad31149..0d01a64f2576 100644
> > --- a/include/linux/virtio_config.h
> > +++ b/include/linux/virtio_config.h
> > @@ -74,6 +74,17 @@ struct virtio_shm_region {
> >    * @set_vq_affinity: set the affinity for a virtqueue (optional).
> >    * @get_vq_affinity: get the affinity for a virtqueue (optional).
> >    * @get_shm_region: get a shared memory region based on the index.
> > + * @reset_vq: reset a queue individually
>
>
> This needs to be marked as optional I think.
OK.
>
>
> > + *	vq: the virtqueue
> > + *	Returns 0 on success or error status
>
>
> It looks to me the caller should also guarantee that the vring is not
> accsed by any functions e.g NAPI.
OK.
>
>
> > + *	After successfully calling this, be sure to call
> > + *	virtqueue_detach_unused_buf() to recycle the buffer in the ring,
and
> > + *	then call vring_release_virtqueue() to release the vq ring.
> > + * @enable_reset_vq: enable a reset queue
> > + *	vq: the virtqueue
> > + *	ring_num: specify ring num for the vq to be re-enabled. 0 means
use the
> > + *	          default value. MUST be a power of 2.
>
>
> Note that we don't have power of 2 requirement for packed virtqueue.

So the following check here does not seem reasonable. (virtio_pci_modern.c)

static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
				  struct virtio_pci_vq_info *info,
				  unsigned index,
				  void (*callback)(struct virtqueue *vq),
				  const char *name,
				  bool ctx,
				  u16 msix_vec)
{

	struct virtio_pci_modern_device *mdev = &vp_dev->mdev;
	struct virtqueue *vq;
	u16 num;
	int err;

	if (index >= vp_modern_get_num_queues(mdev))
		return ERR_PTR(-ENOENT);

	/* Check if queue is either not available or already active. */
	num = vp_modern_get_queue_size(mdev, index);
	if (!num || vp_modern_get_queue_enable(mdev, index))
		return ERR_PTR(-ENOENT);

	if (num & (num - 1)) {
		dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", num);
		return ERR_PTR(-EINVAL);
	}
	^^^^^^^^^^^^^^^^^^^^^^^
>
> And I wonder if it's cleaner to have a find_vq() ops instead to dealing
> with the re-allocation and possible size change, or have a dedicated
> helper to set vring size so driver can do.
>
> reset_vq()
>
> virtqueue_set_vring_size()
>
> enable_reset_vq()
I like to add a dedicated helper.
>
>
> > + *	Returns 0 on success or error status
> >    */
> >   typedef void vq_callback_t(struct virtqueue *);
> >   struct virtio_config_ops {
> > @@ -100,6 +111,8 @@ struct virtio_config_ops {
> >   			int index);
> >   	bool (*get_shm_region)(struct virtio_device *vdev,
> >   			       struct virtio_shm_region *region, u8 id);
> > +	int (*reset_vq)(struct virtqueue *vq);
> > +	int (*enable_reset_vq)(struct virtqueue *vq, u16 ring_num);
>
>
> Note that the current implement is best-effort, so it's not guarantee
> that we can have a vring with ring_num, we may get less under memory
> pressure or even fail. We probably need to have a pamater to mandate the
> ring_num otherwise user may surprise to see a decreased size of the ring
> when a increasing is actually requested.
1. We can add a helper to specify max ring num.
2. Or after specifying ring num, in case of failure, return directly.

I prefer #1

Thanks.
>
> Thanks
>
>
> >   };
> >
> >   /* If driver didn't advertise the feature, it will never appear.
*/
>