search for: virtblk_done

On Thu, Feb 13, 2020 at 8:38 PM Halil Pasic <pasic at linux.ibm.com> wrote: > > Since nobody else is going to restart our hw_queue for us, the > blk_mq_start_stopped_hw_queues() is in virtblk_done() is not sufficient > necessarily sufficient to ensure that the queue will get started again. > In case of global resource outage (-ENOMEM because mapping failure, > because of swiotlb full) our virtqueue may be empty and we can get > stuck with a stopped hw_queue. > > Let us not...

On Thu, Feb 13, 2020 at 8:38 PM Halil Pasic <pasic at linux.ibm.com> wrote: > > Since nobody else is going to restart our hw_queue for us, the > blk_mq_start_stopped_hw_queues() is in virtblk_done() is not sufficient > necessarily sufficient to ensure that the queue will get started again. > In case of global resource outage (-ENOMEM because mapping failure, > because of swiotlb full) our virtqueue may be empty and we can get > stuck with a stopped hw_queue. > > Let us not...

...15/486720 files, 222526/1944576 blocks BUG: sleeping function called from invalid context at kernel/locking/mutex.c:586 in_atomic(): 1, irqs_disabled(): 1, pid: 0, name: swapper/1 2 locks held by swapper/1/0: #0: (&(&vblk->vq_lock)->rlock){-.-...}, at: [<ffffffffa0039042>] virtblk_done+0x42/0xe0 [virtio_blk] #1: (&(&bitmap->counts.lock)->rlock){-.....}, at: [<ffffffff81633718>] bitmap_endwrite+0x68/0x240 irq event stamp: 33518 hardirqs last enabled at (33515): [<ffffffff8102544f>] default_idle+0x1f/0x230 hardirqs last disabled at (33516): [<fff...

...15/486720 files, 222526/1944576 blocks BUG: sleeping function called from invalid context at kernel/locking/mutex.c:586 in_atomic(): 1, irqs_disabled(): 1, pid: 0, name: swapper/1 2 locks held by swapper/1/0: #0: (&(&vblk->vq_lock)->rlock){-.-...}, at: [<ffffffffa0039042>] virtblk_done+0x42/0xe0 [virtio_blk] #1: (&(&bitmap->counts.lock)->rlock){-.....}, at: [<ffffffff81633718>] bitmap_endwrite+0x68/0x240 irq event stamp: 33518 hardirqs last enabled at (33515): [<ffffffff8102544f>] default_idle+0x1f/0x230 hardirqs last disabled at (33516): [<fff...

...0 > Ming Lei <tom.leiming at gmail.com> wrote: > > > On Thu, Feb 13, 2020 at 8:38 PM Halil Pasic <pasic at linux.ibm.com> wrote: > > > > > > Since nobody else is going to restart our hw_queue for us, the > > > blk_mq_start_stopped_hw_queues() is in virtblk_done() is not sufficient > > > necessarily sufficient to ensure that the queue will get started again. > > > In case of global resource outage (-ENOMEM because mapping failure, > > > because of swiotlb full) our virtqueue may be empty and we can get > > > stuck with a...

Hi, These patches try to support multi virtual queues(multi-vq) in one virtio-blk device, and maps each virtual queue(vq) to blk-mq's hardware queue. With this approach, both scalability and performance on virtio-blk device can get improved. For verifying the improvement, I implements virtio-blk multi-vq over qemu's dataplane feature, and both handling host notification from each vq and

Hi, These patches try to support multi virtual queues(multi-vq) in one virtio-blk device, and maps each virtual queue(vq) to blk-mq's hardware queue. With this approach, both scalability and performance on virtio-blk device can get improved. For verifying the improvement, I implements virtio-blk multi-vq over qemu's dataplane feature, and both handling host notification from each vq and

...k; > @@ -47,6 +50,10 @@ struct virtio_blk > > /* Ida index - used to track minor number allocations. */ > int index; > + > + /* num of vqs */ > + int num_vqs; > + struct virtio_blk_vq *vqs; > }; > > struct virtblk_req > @@ -133,14 +140,15 @@ static void virtblk_done(struct virtqueue *vq) > { > struct virtio_blk *vblk = vq->vdev->priv; > bool req_done = false; > + int qid = vq->index; > struct virtblk_req *vbr; > unsigned long flags; > unsigned int len; > > - spin_lock_irqsave(&vblk->vq_lock, flags); >...

...k; > @@ -47,6 +50,10 @@ struct virtio_blk > > /* Ida index - used to track minor number allocations. */ > int index; > + > + /* num of vqs */ > + int num_vqs; > + struct virtio_blk_vq *vqs; > }; > > struct virtblk_req > @@ -133,14 +140,15 @@ static void virtblk_done(struct virtqueue *vq) > { > struct virtio_blk *vblk = vq->vdev->priv; > bool req_done = false; > + int qid = vq->index; > struct virtblk_req *vbr; > unsigned long flags; > unsigned int len; > > - spin_lock_irqsave(&vblk->vq_lock, flags); >...

...162.407880] <Interrupt> [ 162.408400] lock(&journal->j_state_lock); [ 162.409287] [ 162.409287] *** DEADLOCK *** [ 162.409287] [ 162.410447] 2 locks held by swapper/0/0: [ 162.411248] #0: (&(&vblk->vqs[i].lock)->rlock){-.-.}, at: [<00000000b75553ae>] virtblk_done+0x50/0xf8 [ 162.413101] #1: (rcu_read_lock){....}, at: [<000000002bf2a216>] hctx_lock+0x1c/0xe8 [ 162.414630] [ 162.414630] stack backtrace: [ 162.415492] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.16.0-rc2 #1 [ 162.429624] Hardware name: linux,dummy-virt (DT) [ 162.430631] Call trac...

...162.407880] <Interrupt> [ 162.408400] lock(&journal->j_state_lock); [ 162.409287] [ 162.409287] *** DEADLOCK *** [ 162.409287] [ 162.410447] 2 locks held by swapper/0/0: [ 162.411248] #0: (&(&vblk->vqs[i].lock)->rlock){-.-.}, at: [<00000000b75553ae>] virtblk_done+0x50/0xf8 [ 162.413101] #1: (rcu_read_lock){....}, at: [<000000002bf2a216>] hctx_lock+0x1c/0xe8 [ 162.414630] [ 162.414630] stack backtrace: [ 162.415492] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.16.0-rc2 #1 [ 162.429624] Hardware name: linux,dummy-virt (DT) [ 162.430631] Call trac...

..._t vq_lock[MAX_NUM_VQ]; /* The disk structure for the kernel. */ struct gendisk *disk; @@ -47,6 +49,9 @@ struct virtio_blk /* Ida index - used to track minor number allocations. */ int index; + + /* num of vqs */ + int num_vqs; }; struct virtblk_req @@ -133,14 +138,15 @@ static void virtblk_done(struct virtqueue *vq) { struct virtio_blk *vblk = vq->vdev->priv; bool req_done = false; + int qid = vq->index; struct virtblk_req *vbr; unsigned long flags; unsigned int len; - spin_lock_irqsave(&vblk->vq_lock, flags); + spin_lock_irqsave(&vblk->vq_lock[qid], fl...

..._t vq_lock[MAX_NUM_VQ]; /* The disk structure for the kernel. */ struct gendisk *disk; @@ -47,6 +49,9 @@ struct virtio_blk /* Ida index - used to track minor number allocations. */ int index; + + /* num of vqs */ + int num_vqs; }; struct virtblk_req @@ -133,14 +138,15 @@ static void virtblk_done(struct virtqueue *vq) { struct virtio_blk *vblk = vq->vdev->priv; bool req_done = false; + int qid = vq->index; struct virtblk_req *vbr; unsigned long flags; unsigned int len; - spin_lock_irqsave(&vblk->vq_lock, flags); + spin_lock_irqsave(&vblk->vq_lock[qid], fl...

...rnel. */ > struct gendisk *disk; > @@ -47,6 +49,9 @@ struct virtio_blk > > /* Ida index - used to track minor number allocations. */ > int index; > + > + /* num of vqs */ > + int num_vqs; > }; > > struct virtblk_req > @@ -133,14 +138,15 @@ static void virtblk_done(struct virtqueue *vq) > { > struct virtio_blk *vblk = vq->vdev->priv; > bool req_done = false; > + int qid = vq->index; > struct virtblk_req *vbr; > unsigned long flags; > unsigned int len; > > - spin_lock_irqsave(&vblk->vq_lock, flags); >...

...rnel. */ > struct gendisk *disk; > @@ -47,6 +49,9 @@ struct virtio_blk > > /* Ida index - used to track minor number allocations. */ > int index; > + > + /* num of vqs */ > + int num_vqs; > }; > > struct virtblk_req > @@ -133,14 +138,15 @@ static void virtblk_done(struct virtqueue *vq) > { > struct virtio_blk *vblk = vq->vdev->priv; > bool req_done = false; > + int qid = vq->index; > struct virtblk_req *vbr; > unsigned long flags; > unsigned int len; > > - spin_lock_irqsave(&vblk->vq_lock, flags); >...

Two patches are handling new edge cases introduced by doing DMA mappings (which can fail) in virtio core. I stumbled upon this while stress testing I/O for Protected Virtual Machines. I deliberately chose a tiny swiotlb size and have generated load with fio. With more than one virtio-blk disk in use I experienced hangs. The goal of this series is to fix those hangs. Halil Pasic (2):

Two patches are handling new edge cases introduced by doing DMA mappings (which can fail) in virtio core. I stumbled upon this while stress testing I/O for Protected Virtual Machines. I deliberately chose a tiny swiotlb size and have generated load with fio. With more than one virtio-blk disk in use I experienced hangs. The goal of this series is to fix those hangs. Halil Pasic (2):

...6 blocks > BUG: sleeping function called from invalid context at > kernel/locking/mutex.c:586 > in_atomic(): 1, irqs_disabled(): 1, pid: 0, name: swapper/1 > 2 locks held by swapper/1/0: > #0: (&(&vblk->vq_lock)->rlock){-.-...}, at: [<ffffffffa0039042>] > virtblk_done+0x42/0xe0 [virtio_blk] > #1: (&(&bitmap->counts.lock)->rlock){-.....}, at: > [<ffffffff81633718>] bitmap_endwrite+0x68/0x240 > irq event stamp: 33518 > hardirqs last enabled at (33515): [<ffffffff8102544f>] default_idle+0x1f/0x230 > hardirqs last disa...

...The disk structure for the kernel. */ struct gendisk *disk; @@ -47,6 +50,10 @@ struct virtio_blk /* Ida index - used to track minor number allocations. */ int index; + + /* num of vqs */ + int num_vqs; + struct virtio_blk_vq *vqs; }; struct virtblk_req @@ -133,14 +140,15 @@ static void virtblk_done(struct virtqueue *vq) { struct virtio_blk *vblk = vq->vdev->priv; bool req_done = false; + int qid = vq->index; struct virtblk_req *vbr; unsigned long flags; unsigned int len; - spin_lock_irqsave(&vblk->vq_lock, flags); + spin_lock_irqsave(&vblk->vqs[qid].lock, f...

...The disk structure for the kernel. */ struct gendisk *disk; @@ -47,6 +52,10 @@ struct virtio_blk /* Ida index - used to track minor number allocations. */ int index; + + /* num of vqs */ + int num_vqs; + struct virtio_blk_vq *vqs; }; struct virtblk_req @@ -133,14 +142,15 @@ static void virtblk_done(struct virtqueue *vq) { struct virtio_blk *vblk = vq->vdev->priv; bool req_done = false; + int qid = vq->index; struct virtblk_req *vbr; unsigned long flags; unsigned int len; - spin_lock_irqsave(&vblk->vq_lock, flags); + spin_lock_irqsave(&vblk->vqs[qid].lock, f...