similar to: Race condition during hotplug when dropping block queue lock

Holding the vblk->lock across kick causes poor scalability in SMP guests. If one CPU is doing virtqueue kick and another CPU touches the vblk->lock it will have to spin until virtqueue kick completes. This patch reduces system% CPU utilization in SMP guests that are running multithreaded I/O-bound workloads. The improvements are small but show as iops and SMP are increased. Khoa Huynh

Holding the vblk->lock across kick causes poor scalability in SMP guests. If one CPU is doing virtqueue kick and another CPU touches the vblk->lock it will have to spin until virtqueue kick completes. This patch reduces system% CPU utilization in SMP guests that are running multithreaded I/O-bound workloads. The improvements are small but show as iops and SMP are increased. Khoa Huynh

3.4-stable review patch. If anyone has any objections, please let me know. ------------------ From: Asias He <asias at redhat.com> commit 2c95a3290919541b846bee3e0fbaa75860929f53 upstream. Block layer will allocate a spinlock for the queue if the driver does not provide one in blk_init_queue(). The reason to use the internal spinlock is that blk_cleanup_queue() will switch to use the

3.4-stable review patch. If anyone has any objections, please let me know. ------------------ From: Asias He <asias at redhat.com> commit 2c95a3290919541b846bee3e0fbaa75860929f53 upstream. Block layer will allocate a spinlock for the queue if the driver does not provide one in blk_init_queue(). The reason to use the internal spinlock is that blk_cleanup_queue() will switch to use the

This function was only used by btrfs code in btrfs_abort_devices() (seems in a wrong way). It was removed in commit d07eb9117050c9ed3f78296ebcc06128b52693be, So, Let''s remove the dead code to avoid any confusion. Changes in v2: update commit log, btrfs_abort_devices() was removed already. Cc: Jens Axboe <axboe@kernel.dk> Cc: linux-kernel@vger.kernel.org Cc: Chris Mason

3.4-stable review patch. If anyone has any objections, please let me know. ------------------ From: Asias He <asias at redhat.com> commit 483001c765af6892b3fc3726576cb42f17d1d6b5 upstream. blk_cleanup_queue() will call blk_drian_queue() to drain all the requests before queue DEAD marking. If we reset the device before blk_cleanup_queue() the drain would fail. 1) if the queue is stopped

3.4-stable review patch. If anyone has any objections, please let me know. ------------------ From: Asias He <asias at redhat.com> commit 483001c765af6892b3fc3726576cb42f17d1d6b5 upstream. blk_cleanup_queue() will call blk_drian_queue() to drain all the requests before queue DEAD marking. If we reset the device before blk_cleanup_queue() the drain would fail. 1) if the queue is stopped

Not really interesting yet, this just gets us to the state where single queue boots on a current kernel. Signed-off-by: Jens Axboe <axboe at kernel.dk> --- block/Kconfig | 5 + block/Kconfig.iosched | 2 + block/blk-core.c | 427 ++++++++++++++++++-------------------- block/blk-exec.c | 14 +- block/blk-flush.c

Not really interesting yet, this just gets us to the state where single queue boots on a current kernel. Signed-off-by: Jens Axboe <axboe at kernel.dk> --- block/Kconfig | 5 + block/Kconfig.iosched | 2 + block/blk-core.c | 427 ++++++++++++++++++-------------------- block/blk-exec.c | 14 +- block/blk-flush.c

Rather than handing a scatterlist[] and out and in numbers to virtqueue_add_buf(), hand two separate ones which can be chained. I shall refrain from ranting about what a disgusting hack chained scatterlists are. I'll just note that this doesn't make things simpler (see diff). The scatterlists we use can be too large for the stack, so we put them in our device struct and reuse them. But

Rather than handing a scatterlist[] and out and in numbers to virtqueue_add_buf(), hand two separate ones which can be chained. I shall refrain from ranting about what a disgusting hack chained scatterlists are. I'll just note that this doesn't make things simpler (see diff). The scatterlists we use can be too large for the stack, so we put them in our device struct and reuse them. But

This patch set fixes the race when hot-unplug stressed disk. Asias He (3): virtio-blk: Call del_gendisk() before disable guest kick virtio-blk: Reset device after blk_cleanup_queue() virtio-blk: Use block layer provided spinlock drivers/block/virtio_blk.c | 25 ++++++------------------- 1 file changed, 6 insertions(+), 19 deletions(-) -- 1.7.10.2

This patch set fixes the race when hot-unplug stressed disk. Asias He (3): virtio-blk: Call del_gendisk() before disable guest kick virtio-blk: Reset device after blk_cleanup_queue() virtio-blk: Use block layer provided spinlock drivers/block/virtio_blk.c | 25 ++++++------------------- 1 file changed, 6 insertions(+), 19 deletions(-) -- 1.7.10.2

When user hot-unplug a disk which is busy serving I/O, __blk_run_queue might be unable to drain all the requests. As a result, the blk_drain_queue() would loop forever and blk_cleanup_queue would not return. So hot-unplug will fail. This patch adds a callback in blk_drain_queue() for low lever driver to abort requests. Currently, this is useful for virtio-blk to do cleanup in hot-unplug. Cc:

When user hot-unplug a disk which is busy serving I/O, __blk_run_queue might be unable to drain all the requests. As a result, the blk_drain_queue() would loop forever and blk_cleanup_queue would not return. So hot-unplug will fail. This patch adds a callback in blk_drain_queue() for low lever driver to abort requests. Currently, this is useful for virtio-blk to do cleanup in hot-unplug. Cc:

If we reset the virtio-blk device before the requests already dispatched to the virtio-blk driver from the block layer are finised, we will stuck in blk_cleanup_queue() and the remove will fail. blk_cleanup_queue() calls blk_drain_queue() to drain all requests queued before DEAD marking. However it will never success if the device is already stopped. We'll have q->in_flight[] > 0, so

If we reset the virtio-blk device before the requests already dispatched to the virtio-blk driver from the block layer are finised, we will stuck in blk_cleanup_queue() and the remove will fail. blk_cleanup_queue() calls blk_drain_queue() to drain all requests queued before DEAD marking. However it will never success if the device is already stopped. We'll have q->in_flight[] > 0, so

> +static void do_virtblk_request(request_queue_t *q) > +{ > + struct virtio_blk *vblk = NULL; > + struct request *req; > + struct virtblk_req *vbr; > + > + while ((req = elv_next_request(q)) != NULL) { > + vblk = req->rq_disk->private_data; > + > + vbr = mempool_alloc(vblk->pool, GFP_ATOMIC); > + if (!vbr) > + goto stop; > + > +

> +static void do_virtblk_request(request_queue_t *q) > +{ > + struct virtio_blk *vblk = NULL; > + struct request *req; > + struct virtblk_req *vbr; > + > + while ((req = elv_next_request(q)) != NULL) { > + vblk = req->rq_disk->private_data; > + > + vbr = mempool_alloc(vblk->pool, GFP_ATOMIC); > + if (!vbr) > + goto stop; > + > +

Deleting the disk and partitions in virtblk_remove() via del_gendisk() causes never ending waits when trying to synch dirty inode pages. A dummy virtqueue_notify() in virtblk_remove() is used to detect a host notification error, latter occurs when block device was hot-unplugged. When the dummy host kick failed blk_cleanup_queue() should be invoked prior to del_gendisk(). Signed-off-by: Heinz