Linux Virtualization - Nov 2019 - [PATCH] virtio_console: allocate inbufs in add_port() only if it is needed

When we hot unplug a virtserialport and then try to hot plug again,
it fails:

(qemu) chardev-add socket,id=serial0,path=/tmp/serial0,server,nowait
(qemu) device_add virtserialport,bus=virtio-serial0.0,nr=2,\
                  chardev=serial0,id=serial0,name=serial0
(qemu) device_del serial0
(qemu) device_add virtserialport,bus=virtio-serial0.0,nr=2,\
                  chardev=serial0,id=serial0,name=serial0
kernel error:
  virtio-ports vport2p2: Error allocating inbufs
qemu error:
  virtio-serial-bus: Guest failure in adding port 2 for device \
                     virtio-serial0.0

This happens because buffers for the in_vq are allocated when the port is
added but are not released when the port is unplugged.

They are only released when virtconsole is removed (see a7a69ec0d8e4)

To avoid the problem and to be symmetric, we could allocate all the buffers
in init_vqs() as they are released in remove_vqs(), but it sounds like
a waste of memory.

Rather than that, this patch changes add_port() logic to only allocate the
buffers if the in_vq has available free slots.

Fixes: a7a69ec0d8e4 ("virtio_console: free buffers after reset")
Cc: mst at redhat.com
Signed-off-by: Laurent Vivier <lvivier at redhat.com>
---
 drivers/char/virtio_console.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c
index 7270e7b69262..77105166fe01 100644
--- a/drivers/char/virtio_console.c
+++ b/drivers/char/virtio_console.c
@@ -1421,12 +1421,17 @@ static int add_port(struct ports_device *portdev, u32
id)
 	spin_lock_init(&port->outvq_lock);
 	init_waitqueue_head(&port->waitqueue);
 
-	/* Fill the in_vq with buffers so the host can send us data. */
-	nr_added_bufs = fill_queue(port->in_vq, &port->inbuf_lock);
-	if (!nr_added_bufs) {
-		dev_err(port->dev, "Error allocating inbufs\n");
-		err = -ENOMEM;
-		goto free_device;
+	/* if the in_vq has not already been filled (the port has already been
+	 * used and unplugged), fill the in_vq with buffers so the host can
+	 * send us data.
+	 */
+	if (port->in_vq->num_free != 0) {
+		nr_added_bufs = fill_queue(port->in_vq, &port->inbuf_lock);
+		if (!nr_added_bufs) {
+			dev_err(port->dev, "Error allocating inbufs\n");
+			err = -ENOMEM;
+			goto free_device;
+		}
 	}
 
 	if (is_rproc_serial(port->portdev->vdev))
-- 
2.21.0

Any comments?

Thanks,
Laurent

On 18/10/2019 18:47, Laurent Vivier wrote:
> When we hot unplug a virtserialport and then try to hot plug again,
> it fails:
> 
> (qemu) chardev-add socket,id=serial0,path=/tmp/serial0,server,nowait
> (qemu) device_add virtserialport,bus=virtio-serial0.0,nr=2,\
>                   chardev=serial0,id=serial0,name=serial0
> (qemu) device_del serial0
> (qemu) device_add virtserialport,bus=virtio-serial0.0,nr=2,\
>                   chardev=serial0,id=serial0,name=serial0
> kernel error:
>   virtio-ports vport2p2: Error allocating inbufs
> qemu error:
>   virtio-serial-bus: Guest failure in adding port 2 for device \
>                      virtio-serial0.0
> 
> This happens because buffers for the in_vq are allocated when the port is
> added but are not released when the port is unplugged.
> 
> They are only released when virtconsole is removed (see a7a69ec0d8e4)
> 
> To avoid the problem and to be symmetric, we could allocate all the buffers
> in init_vqs() as they are released in remove_vqs(), but it sounds like
> a waste of memory.
> 
> Rather than that, this patch changes add_port() logic to only allocate the
> buffers if the in_vq has available free slots.
> 
> Fixes: a7a69ec0d8e4 ("virtio_console: free buffers after reset")
> Cc: mst at redhat.com
> Signed-off-by: Laurent Vivier <lvivier at redhat.com>
> ---
>  drivers/char/virtio_console.c | 17 +++++++++++------
>  1 file changed, 11 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c
> index 7270e7b69262..77105166fe01 100644
> --- a/drivers/char/virtio_console.c
> +++ b/drivers/char/virtio_console.c
> @@ -1421,12 +1421,17 @@ static int add_port(struct ports_device *portdev,
u32 id)
>  	spin_lock_init(&port->outvq_lock);
>  	init_waitqueue_head(&port->waitqueue);
>  
> -	/* Fill the in_vq with buffers so the host can send us data. */
> -	nr_added_bufs = fill_queue(port->in_vq, &port->inbuf_lock);
> -	if (!nr_added_bufs) {
> -		dev_err(port->dev, "Error allocating inbufs\n");
> -		err = -ENOMEM;
> -		goto free_device;
> +	/* if the in_vq has not already been filled (the port has already been
> +	 * used and unplugged), fill the in_vq with buffers so the host can
> +	 * send us data.
> +	 */
> +	if (port->in_vq->num_free != 0) {
> +		nr_added_bufs = fill_queue(port->in_vq, &port->inbuf_lock);
> +		if (!nr_added_bufs) {
> +			dev_err(port->dev, "Error allocating inbufs\n");
> +			err = -ENOMEM;
> +			goto free_device;
> +		}
>  	}
>  
>  	if (is_rproc_serial(port->portdev->vdev))
>

On Fri, Oct 18, 2019 at 06:47:18PM +0200, Laurent Vivier
wrote:
> When we hot unplug a virtserialport and then try to hot plug again,
> it fails:
> 
> (qemu) chardev-add socket,id=serial0,path=/tmp/serial0,server,nowait
> (qemu) device_add virtserialport,bus=virtio-serial0.0,nr=2,\
>                   chardev=serial0,id=serial0,name=serial0
> (qemu) device_del serial0
> (qemu) device_add virtserialport,bus=virtio-serial0.0,nr=2,\
>                   chardev=serial0,id=serial0,name=serial0
> kernel error:
>   virtio-ports vport2p2: Error allocating inbufs
> qemu error:
>   virtio-serial-bus: Guest failure in adding port 2 for device \
>                      virtio-serial0.0
> 
> This happens because buffers for the in_vq are allocated when the port is
> added but are not released when the port is unplugged.
> 
> They are only released when virtconsole is removed (see a7a69ec0d8e4)
> 
> To avoid the problem and to be symmetric, we could allocate all the buffers
> in init_vqs() as they are released in remove_vqs(), but it sounds like
> a waste of memory.
> 
> Rather than that, this patch changes add_port() logic to only allocate the
> buffers if the in_vq has available free slots.
> 
> Fixes: a7a69ec0d8e4 ("virtio_console: free buffers after reset")
> Cc: mst at redhat.com
> Signed-off-by: Laurent Vivier <lvivier at redhat.com>
> ---
>  drivers/char/virtio_console.c | 17 +++++++++++------
>  1 file changed, 11 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c
> index 7270e7b69262..77105166fe01 100644
> --- a/drivers/char/virtio_console.c
> +++ b/drivers/char/virtio_console.c
> @@ -1421,12 +1421,17 @@ static int add_port(struct ports_device *portdev,
u32 id)
>  	spin_lock_init(&port->outvq_lock);
>  	init_waitqueue_head(&port->waitqueue);
>  
> -	/* Fill the in_vq with buffers so the host can send us data. */
> -	nr_added_bufs = fill_queue(port->in_vq, &port->inbuf_lock);
> -	if (!nr_added_bufs) {
> -		dev_err(port->dev, "Error allocating inbufs\n");
> -		err = -ENOMEM;
> -		goto free_device;
> +	/* if the in_vq has not already been filled (the port has already been
> +	 * used and unplugged), fill the in_vq with buffers so the host can
> +	 * send us data.
> +	 */
> +	if (port->in_vq->num_free != 0) {
> +		nr_added_bufs = fill_queue(port->in_vq, &port->inbuf_lock);
> +		if (!nr_added_bufs) {
> +			dev_err(port->dev, "Error allocating inbufs\n");
> +			err = -ENOMEM;
> +			goto free_device;
> +		}
>  	}
>  
>  	if (is_rproc_serial(port->portdev->vdev))
Well fill_queue will just add slots as long as it can.
So on a full queue it does nothing. How does this patch help?
> -- 
> 2.21.0

On 06/11/2019 14:56, Michael S. Tsirkin wrote:
> On Fri, Oct 18, 2019 at 06:47:18PM +0200, Laurent Vivier wrote:
>> When we hot unplug a virtserialport and then try to hot plug again,
>> it fails:
>>
>> (qemu) chardev-add socket,id=serial0,path=/tmp/serial0,server,nowait
>> (qemu) device_add virtserialport,bus=virtio-serial0.0,nr=2,\
>>                   chardev=serial0,id=serial0,name=serial0
>> (qemu) device_del serial0
>> (qemu) device_add virtserialport,bus=virtio-serial0.0,nr=2,\
>>                   chardev=serial0,id=serial0,name=serial0
>> kernel error:
>>   virtio-ports vport2p2: Error allocating inbufs
>> qemu error:
>>   virtio-serial-bus: Guest failure in adding port 2 for device \
>>                      virtio-serial0.0
>>
>> This happens because buffers for the in_vq are allocated when the port
is
>> added but are not released when the port is unplugged.
>>
>> They are only released when virtconsole is removed (see a7a69ec0d8e4)
>>
>> To avoid the problem and to be symmetric, we could allocate all the
buffers
>> in init_vqs() as they are released in remove_vqs(), but it sounds like
>> a waste of memory.
>>
>> Rather than that, this patch changes add_port() logic to only allocate
the
>> buffers if the in_vq has available free slots.
>>
>> Fixes: a7a69ec0d8e4 ("virtio_console: free buffers after
reset")
>> Cc: mst at redhat.com
>> Signed-off-by: Laurent Vivier <lvivier at redhat.com>
>> ---
>>  drivers/char/virtio_console.c | 17 +++++++++++------
>>  1 file changed, 11 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/char/virtio_console.c
b/drivers/char/virtio_console.c
>> index 7270e7b69262..77105166fe01 100644
>> --- a/drivers/char/virtio_console.c
>> +++ b/drivers/char/virtio_console.c
>> @@ -1421,12 +1421,17 @@ static int add_port(struct ports_device
*portdev, u32 id)
>>  	spin_lock_init(&port->outvq_lock);
>>  	init_waitqueue_head(&port->waitqueue);
>>  
>> -	/* Fill the in_vq with buffers so the host can send us data. */
>> -	nr_added_bufs = fill_queue(port->in_vq, &port->inbuf_lock);
>> -	if (!nr_added_bufs) {
>> -		dev_err(port->dev, "Error allocating inbufs\n");
>> -		err = -ENOMEM;
>> -		goto free_device;
>> +	/* if the in_vq has not already been filled (the port has already
been
>> +	 * used and unplugged), fill the in_vq with buffers so the host can
>> +	 * send us data.
>> +	 */
>> +	if (port->in_vq->num_free != 0) {
>> +		nr_added_bufs = fill_queue(port->in_vq,
&port->inbuf_lock);
>> +		if (!nr_added_bufs) {
>> +			dev_err(port->dev, "Error allocating inbufs\n");
>> +			err = -ENOMEM;
>> +			goto free_device;
>> +		}
>>  	}
>>  
>>  	if (is_rproc_serial(port->portdev->vdev))
> 
> Well fill_queue will just add slots as long as it can.
> So on a full queue it does nothing. How does this patch help?
Yes, but in this case it returns 0 and so add_port() fails and exits
with -ENOMEM and the device is freed. It's what this patch tries to avoid.

Thanks,
Laurent