Virtualization - May 2023 - [PATCH v4] virtio_net: suppress cpu stall when free_unused_bufs

On Thu, May 4, 2023 at 10:27?AM Wenliang Wang
<wangwenliang.1995 at bytedance.com> wrote:
>
> For multi-queue and large ring-size use case, the following error
> occurred when free_unused_bufs:
> rcu: INFO: rcu_sched self-detected stall on CPU.
>
> Fixes: 986a4f4d452d ("virtio_net: multiqueue support")
> Signed-off-by: Wenliang Wang <wangwenliang.1995 at bytedance.com>
> ---
> v2:
> -add need_resched check.
> -apply same logic to sq.
> v3:
> -use cond_resched instead.
> v4:
> -add fixes tag
> ---
>  drivers/net/virtio_net.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index 8d8038538fc4..a12ae26db0e2 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -3560,12 +3560,14 @@ static void free_unused_bufs(struct virtnet_info
*vi)
>                 struct virtqueue *vq = vi->sq[i].vq;
>                 while ((buf = virtqueue_detach_unused_buf(vq)) != NULL)
>                         virtnet_sq_free_unused_buf(vq, buf);
> +               cond_resched();
Does this really address the case when the virtqueue is very large?

Thanks
>         }
>
>         for (i = 0; i < vi->max_queue_pairs; i++) {
>                 struct virtqueue *vq = vi->rq[i].vq;
>                 while ((buf = virtqueue_detach_unused_buf(vq)) != NULL)
>                         virtnet_rq_free_unused_buf(vq, buf);
> +               cond_resched();
>         }
>  }
>
> --
> 2.20.1
>

On Fri, 5 May 2023 11:28:25 +0800, Jason Wang <jasowang at redhat.com>
wrote:
> On Thu, May 4, 2023 at 10:27?AM Wenliang Wang
> <wangwenliang.1995 at bytedance.com> wrote:
> >
> > For multi-queue and large ring-size use case, the following error
> > occurred when free_unused_bufs:
> > rcu: INFO: rcu_sched self-detected stall on CPU.
> >
> > Fixes: 986a4f4d452d ("virtio_net: multiqueue support")
> > Signed-off-by: Wenliang Wang <wangwenliang.1995 at
bytedance.com>
> > ---
> > v2:
> > -add need_resched check.
> > -apply same logic to sq.
> > v3:
> > -use cond_resched instead.
> > v4:
> > -add fixes tag
> > ---
> >  drivers/net/virtio_net.c | 2 ++
> >  1 file changed, 2 insertions(+)
> >
> > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > index 8d8038538fc4..a12ae26db0e2 100644
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -3560,12 +3560,14 @@ static void free_unused_bufs(struct
virtnet_info *vi)
> >                 struct virtqueue *vq = vi->sq[i].vq;
> >                 while ((buf = virtqueue_detach_unused_buf(vq)) !=
NULL)
> >                         virtnet_sq_free_unused_buf(vq, buf);
> > +               cond_resched();
>
> Does this really address the case when the virtqueue is very large?
Yes, I also have this question. I think cond_resched() should be called every
time a certain number of buffers are processed.

Thanks.
>
> Thanks
>
> >         }
> >
> >         for (i = 0; i < vi->max_queue_pairs; i++) {
> >                 struct virtqueue *vq = vi->rq[i].vq;
> >                 while ((buf = virtqueue_detach_unused_buf(vq)) !=
NULL)
> >                         virtnet_rq_free_unused_buf(vq, buf);
> > +               cond_resched();
> >         }
> >  }
> >
> > --
> > 2.20.1
> >
>

On Fri, May 05, 2023 at 11:28:25AM +0800, Jason Wang
wrote:
> On Thu, May 4, 2023 at 10:27?AM Wenliang Wang
> <wangwenliang.1995 at bytedance.com> wrote:
> >
> > For multi-queue and large ring-size use case, the following error
> > occurred when free_unused_bufs:
> > rcu: INFO: rcu_sched self-detected stall on CPU.
> >
> > Fixes: 986a4f4d452d ("virtio_net: multiqueue support")
> > Signed-off-by: Wenliang Wang <wangwenliang.1995 at
bytedance.com>
> > ---
> > v2:
> > -add need_resched check.
> > -apply same logic to sq.
> > v3:
> > -use cond_resched instead.
> > v4:
> > -add fixes tag
> > ---
> >  drivers/net/virtio_net.c | 2 ++
> >  1 file changed, 2 insertions(+)
> >
> > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > index 8d8038538fc4..a12ae26db0e2 100644
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -3560,12 +3560,14 @@ static void free_unused_bufs(struct
virtnet_info *vi)
> >                 struct virtqueue *vq = vi->sq[i].vq;
> >                 while ((buf = virtqueue_detach_unused_buf(vq)) !=
NULL)
> >                         virtnet_sq_free_unused_buf(vq, buf);
> > +               cond_resched();
> 
> Does this really address the case when the virtqueue is very large?
> 
> Thanks

it does in that a very large queue is still just 64k in size.
we might however have 64k of these queues.
> >         }
> >
> >         for (i = 0; i < vi->max_queue_pairs; i++) {
> >                 struct virtqueue *vq = vi->rq[i].vq;
> >                 while ((buf = virtqueue_detach_unused_buf(vq)) !=
NULL)
> >                         virtnet_rq_free_unused_buf(vq, buf);
> > +               cond_resched();
> >         }
> >  }
> >
> > --
> > 2.20.1
> >