search for: blk_mq_start_stopped_hw_queu

...19,9 @@ static int virtblk_restore(struct virtio_device *vdev) > if (ret) > return ret; > > + if (vblk->num_vqs != vblk->tag_set.nr_hw_queues) > + blk_mq_update_nr_hw_queues(&vblk->tag_set, vblk->num_vqs); > + > virtio_device_ready(vdev); > > blk_mq_start_stopped_hw_queues(vblk->disk->queue, true); > This should never happen; it'd be a configuration problem. Paolo

...19,9 @@ static int virtblk_restore(struct virtio_device *vdev) > if (ret) > return ret; > > + if (vblk->num_vqs != vblk->tag_set.nr_hw_queues) > + blk_mq_update_nr_hw_queues(&vblk->tag_set, vblk->num_vqs); > + > virtio_device_ready(vdev); > > blk_mq_start_stopped_hw_queues(vblk->disk->queue, true); > This should never happen; it'd be a configuration problem. Paolo

On Thu, Feb 13, 2020 at 8:38 PM Halil Pasic <pasic at linux.ibm.com> wrote: > > Since nobody else is going to restart our hw_queue for us, the > blk_mq_start_stopped_hw_queues() is in virtblk_done() is not sufficient > necessarily sufficient to ensure that the queue will get started again. > In case of global resource outage (-ENOMEM because mapping failure, > because of swiotlb full) our virtqueue may be empty and we can get > stuck with a stopped hw_queue...

On Thu, Feb 13, 2020 at 8:38 PM Halil Pasic <pasic at linux.ibm.com> wrote: > > Since nobody else is going to restart our hw_queue for us, the > blk_mq_start_stopped_hw_queues() is in virtblk_done() is not sufficient > necessarily sufficient to ensure that the queue will get started again. > In case of global resource outage (-ENOMEM because mapping failure, > because of swiotlb full) our virtqueue may be empty and we can get > stuck with a stopped hw_queue...

...> On Tue, 18 Feb 2020 10:21:18 +0800 > Ming Lei <tom.leiming at gmail.com> wrote: > > > On Thu, Feb 13, 2020 at 8:38 PM Halil Pasic <pasic at linux.ibm.com> wrote: > > > > > > Since nobody else is going to restart our hw_queue for us, the > > > blk_mq_start_stopped_hw_queues() is in virtblk_done() is not sufficient > > > necessarily sufficient to ensure that the queue will get started again. > > > In case of global resource outage (-ENOMEM because mapping failure, > > > because of swiotlb full) our virtqueue may be empty and we can get >...

Two patches are handling new edge cases introduced by doing DMA mappings (which can fail) in virtio core. I stumbled upon this while stress testing I/O for Protected Virtual Machines. I deliberately chose a tiny swiotlb size and have generated load with fio. With more than one virtio-blk disk in use I experienced hangs. The goal of this series is to fix those hangs. Halil Pasic (2):

Two patches are handling new edge cases introduced by doing DMA mappings (which can fail) in virtio core. I stumbled upon this while stress testing I/O for Protected Virtual Machines. I deliberately chose a tiny swiotlb size and have generated load with fio. With more than one virtio-blk disk in use I experienced hangs. The goal of this series is to fix those hangs. Halil Pasic (2):

..._work+0x360/0x360 [ 54.769303] ? kthread_bind+0x10/0x10 [ 54.769305] ret_from_fork+0x35/0x40 [ 54.769307] ---[ end trace 923a87a9ce0e777a ]--- Thank you very much! Dongli Zhang On 2/13/20 4:37 AM, Halil Pasic wrote: > Since nobody else is going to restart our hw_queue for us, the > blk_mq_start_stopped_hw_queues() is in virtblk_done() is not sufficient > necessarily sufficient to ensure that the queue will get started again. > In case of global resource outage (-ENOMEM because mapping failure, > because of swiotlb full) our virtqueue may be empty and we can get > stuck with a stopped hw_queue...

...+++ b/drivers/block/virtio_blk.c @@ -819,6 +819,9 @@ static int virtblk_restore(struct virtio_device *vdev) if (ret) return ret; + if (vblk->num_vqs != vblk->tag_set.nr_hw_queues) + blk_mq_update_nr_hw_queues(&vblk->tag_set, vblk->num_vqs); + virtio_device_ready(vdev); blk_mq_start_stopped_hw_queues(vblk->disk->queue, true); -- 2.7.4

...truct virtio_device *vdev) >> if (ret) >> return ret; >> >> + if (vblk->num_vqs != vblk->tag_set.nr_hw_queues) >> + blk_mq_update_nr_hw_queues(&vblk->tag_set, vblk->num_vqs); >> + >> virtio_device_ready(vdev); >> >> blk_mq_start_stopped_hw_queues(vblk->disk->queue, true); >> > > This should never happen; it'd be a configuration problem. > Do you mean all hosts have to be configured with the same number of ->num_vqs? What about cases like migrating a guest from HostA to HostB while HostB is much more powerful...

Since nobody else is going to restart our hw_queue for us, the blk_mq_start_stopped_hw_queues() is in virtblk_done() is not sufficient necessarily sufficient to ensure that the queue will get started again. In case of global resource outage (-ENOMEM because mapping failure, because of swiotlb full) our virtqueue may be empty and we can get stuck with a stopped hw_queue. Let us not stop t...

On Tue, 18 Feb 2020 10:21:18 +0800 Ming Lei <tom.leiming at gmail.com> wrote: > On Thu, Feb 13, 2020 at 8:38 PM Halil Pasic <pasic at linux.ibm.com> wrote: > > > > Since nobody else is going to restart our hw_queue for us, the > > blk_mq_start_stopped_hw_queues() is in virtblk_done() is not sufficient > > necessarily sufficient to ensure that the queue will get started again. > > In case of global resource outage (-ENOMEM because mapping failure, > > because of swiotlb full) our virtqueue may be empty and we can get > > stuck wit...

...+++ b/drivers/block/virtio_blk.c @@ -819,6 +819,9 @@ static int virtblk_restore(struct virtio_device *vdev) if (ret) return ret; + if (vblk->num_vqs != vblk->tag_set.nr_hw_queues) + blk_mq_update_nr_hw_queues(&vblk->tag_set, vblk->num_vqs); + virtio_device_ready(vdev); blk_mq_start_stopped_hw_queues(vblk->disk->queue, true); -- 2.7.4

...ush_work(&vblk->config_work); blk_mq_stop_hw_queues(vblk->disk->queue); @@ -823,7 +809,6 @@ static int virtblk_restore(struct virtio_device *vdev) struct virtio_blk *vblk = vdev->priv; int ret; - vblk->config_enable = true; ret = init_vq(vdev->priv); if (!ret) blk_mq_start_stopped_hw_queues(vblk->disk->queue, true); -- MST

...eue_get_buf(vblk->vq[qid], &len)) != NULL) { > blk_mq_complete_request(vbr->req); > req_done = true; > } > @@ -151,7 +157,7 @@ static void virtblk_done(struct virtqueue *vq) > /* In case queue is stopped waiting for more buffers. */ > if (req_done) > blk_mq_start_stopped_hw_queues(vblk->disk->queue, true); > - spin_unlock_irqrestore(&vblk->vq_lock, flags); > + spin_unlock_irqrestore(&vblk->vq_lock[qid], flags); > } > > static int virtio_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *req) > @@ -160,6 +166,7 @@ static int virti...

...eue_get_buf(vblk->vq[qid], &len)) != NULL) { > blk_mq_complete_request(vbr->req); > req_done = true; > } > @@ -151,7 +157,7 @@ static void virtblk_done(struct virtqueue *vq) > /* In case queue is stopped waiting for more buffers. */ > if (req_done) > blk_mq_start_stopped_hw_queues(vblk->disk->queue, true); > - spin_unlock_irqrestore(&vblk->vq_lock, flags); > + spin_unlock_irqrestore(&vblk->vq_lock[qid], flags); > } > > static int virtio_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *req) > @@ -160,6 +166,7 @@ static int virti...

...ush_work(&vblk->config_work); blk_mq_stop_hw_queues(vblk->disk->queue); @@ -823,7 +809,6 @@ static int virtblk_restore(struct virtio_device *vdev) struct virtio_blk *vblk = vdev->priv; int ret; - vblk->config_enable = true; ret = init_vq(vdev->priv); if (!ret) blk_mq_start_stopped_hw_queues(vblk->disk->queue, true); -- MST

Hi, These patches try to support multi virtual queues(multi-vq) in one virtio-blk device, and maps each virtual queue(vq) to blk-mq's hardware queue. With this approach, both scalability and performance on virtio-blk device can get improved. For verifying the improvement, I implements virtio-blk multi-vq over qemu's dataplane feature, and both handling host notification from each vq and

Hi, These patches try to support multi virtual queues(multi-vq) in one virtio-blk device, and maps each virtual queue(vq) to blk-mq's hardware queue. With this approach, both scalability and performance on virtio-blk device can get improved. For verifying the improvement, I implements virtio-blk multi-vq over qemu's dataplane feature, and both handling host notification from each vq and

...get_buf(vblk->vqs[qid].vq, &len)) != NULL) { > blk_mq_complete_request(vbr->req); > req_done = true; > } > @@ -151,7 +159,7 @@ static void virtblk_done(struct virtqueue *vq) > /* In case queue is stopped waiting for more buffers. */ > if (req_done) > blk_mq_start_stopped_hw_queues(vblk->disk->queue, true); > - spin_unlock_irqrestore(&vblk->vq_lock, flags); > + spin_unlock_irqrestore(&vblk->vqs[qid].lock, flags); > } > > static int virtio_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *req) > @@ -160,6 +168,7 @@ static int virt...