similar to: [PATCH 3.4 22/24] virtio-blk: Use block layer provided spinlock

When user hot-unplug a disk which is busy serving I/O, __blk_run_queue might be unable to drain all the requests. As a result, the blk_drain_queue() would loop forever and blk_cleanup_queue would not return. So hot-unplug will fail. This patch adds a callback in blk_drain_queue() for low lever driver to abort requests. Currently, this is useful for virtio-blk to do cleanup in hot-unplug. Cc:

When user hot-unplug a disk which is busy serving I/O, __blk_run_queue might be unable to drain all the requests. As a result, the blk_drain_queue() would loop forever and blk_cleanup_queue would not return. So hot-unplug will fail. This patch adds a callback in blk_drain_queue() for low lever driver to abort requests. Currently, this is useful for virtio-blk to do cleanup in hot-unplug. Cc:

This patch set fixes the race when hot-unplug stressed disk. Asias He (3): virtio-blk: Call del_gendisk() before disable guest kick virtio-blk: Reset device after blk_cleanup_queue() virtio-blk: Use block layer provided spinlock drivers/block/virtio_blk.c | 25 ++++++------------------- 1 file changed, 6 insertions(+), 19 deletions(-) -- 1.7.10.2

This patch set fixes the race when hot-unplug stressed disk. Asias He (3): virtio-blk: Call del_gendisk() before disable guest kick virtio-blk: Reset device after blk_cleanup_queue() virtio-blk: Use block layer provided spinlock drivers/block/virtio_blk.c | 25 ++++++------------------- 1 file changed, 6 insertions(+), 19 deletions(-) -- 1.7.10.2

Block drivers like nbd and rbd unlock struct request_queue->queue_lock in their request_fn. I'd like to do the same in virtio_blk. After happily posting the patch, Michael Tsirkin pointed out an issue that I can't explain. This may affect existing block drivers that unlock the queue_lock too. What happens when the block device is removed (hot unplug or kernel module unloaded) while

Block drivers like nbd and rbd unlock struct request_queue->queue_lock in their request_fn. I'd like to do the same in virtio_blk. After happily posting the patch, Michael Tsirkin pointed out an issue that I can't explain. This may affect existing block drivers that unlock the queue_lock too. What happens when the block device is removed (hot unplug or kernel module unloaded) while

If we reset the virtio-blk device before the requests already dispatched to the virtio-blk driver from the block layer are finised, we will stuck in blk_cleanup_queue() and the remove will fail. blk_cleanup_queue() calls blk_drain_queue() to drain all requests queued before DEAD marking. However it will never success if the device is already stopped. We'll have q->in_flight[] > 0, so

If we reset the virtio-blk device before the requests already dispatched to the virtio-blk driver from the block layer are finised, we will stuck in blk_cleanup_queue() and the remove will fail. blk_cleanup_queue() calls blk_drain_queue() to drain all requests queued before DEAD marking. However it will never success if the device is already stopped. We'll have q->in_flight[] > 0, so

Holding the vblk->lock across kick causes poor scalability in SMP guests. If one CPU is doing virtqueue kick and another CPU touches the vblk->lock it will have to spin until virtqueue kick completes. This patch reduces system% CPU utilization in SMP guests that are running multithreaded I/O-bound workloads. The improvements are small but show as iops and SMP are increased. Khoa Huynh

Holding the vblk->lock across kick causes poor scalability in SMP guests. If one CPU is doing virtqueue kick and another CPU touches the vblk->lock it will have to spin until virtqueue kick completes. This patch reduces system% CPU utilization in SMP guests that are running multithreaded I/O-bound workloads. The improvements are small but show as iops and SMP are increased. Khoa Huynh

Rusty, currently virtio_blk uses one major number per device. While this works quite well on most systems it is wasteful and will exhaust major numbers on larger installations. This patch allocates a major number on init and will use 16 minor numbers for each disk. That will allow ~64k virtio_blk disks. Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com> ---

Rusty, currently virtio_blk uses one major number per device. While this works quite well on most systems it is wasteful and will exhaust major numbers on larger installations. This patch allocates a major number on init and will use 16 minor numbers for each disk. That will allow ~64k virtio_blk disks. Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com> ---

3.4-stable review patch. If anyone has any objections, please let me know. ------------------ From: Asias He <asias at redhat.com> commit 483001c765af6892b3fc3726576cb42f17d1d6b5 upstream. blk_cleanup_queue() will call blk_drian_queue() to drain all the requests before queue DEAD marking. If we reset the device before blk_cleanup_queue() the drain would fail. 1) if the queue is stopped

3.4-stable review patch. If anyone has any objections, please let me know. ------------------ From: Asias He <asias at redhat.com> commit 483001c765af6892b3fc3726576cb42f17d1d6b5 upstream. blk_cleanup_queue() will call blk_drian_queue() to drain all the requests before queue DEAD marking. If we reset the device before blk_cleanup_queue() the drain would fail. 1) if the queue is stopped

This patch adds support for an optional stats vq that works similary to the stats vq provided by virtio-balloon. The purpose of this change is to allow collection of statistics about working virtio-blk devices to easily analyze performance without having to tap into the guest. Cc: Rusty Russell <rusty at rustcorp.com.au> Cc: "Michael S. Tsirkin" <mst at redhat.com> Cc:

This patch adds support for an optional stats vq that works similary to the stats vq provided by virtio-balloon. The purpose of this change is to allow collection of statistics about working virtio-blk devices to easily analyze performance without having to tap into the guest. Cc: Rusty Russell <rusty at rustcorp.com.au> Cc: "Michael S. Tsirkin" <mst at redhat.com> Cc:

> +static void do_virtblk_request(request_queue_t *q) > +{ > + struct virtio_blk *vblk = NULL; > + struct request *req; > + struct virtblk_req *vbr; > + > + while ((req = elv_next_request(q)) != NULL) { > + vblk = req->rq_disk->private_data; > + > + vbr = mempool_alloc(vblk->pool, GFP_ATOMIC); > + if (!vbr) > + goto stop; > + > +

> +static void do_virtblk_request(request_queue_t *q) > +{ > + struct virtio_blk *vblk = NULL; > + struct request *req; > + struct virtblk_req *vbr; > + > + while ((req = elv_next_request(q)) != NULL) { > + vblk = req->rq_disk->private_data; > + > + vbr = mempool_alloc(vblk->pool, GFP_ATOMIC); > + if (!vbr) > + goto stop; > + > +

This flag should be set by a virtio transport driver, when it was notified about a lost device, before the remove callback of a backend driver is triggered. A backend driver can test this flag in order to perform specific actions that might be appropriate wrt the device loss. In case of a device loss further request queueing should be prevented by setting appropriate queue flags prior to

Code is added to the remove callback to verify if a device was lost. In case of a device loss further request queueing should be prevented by setting appropriate queue flags prior to invoking del_gendisk(). Blocking of request queueing leads to appropriate I/O errors when data are tried to be synched. Trying to synch data to a lost block device doesn't make too much sense. If the current