search for: blk_mq_timeout_check

This patch should fix the bug reported in https://lkml.org/lkml/2014/9/11/249. Test is still pending. David Hildenbrand (1): blk-mq: Avoid race condition with uninitialized requests block/blk-mq.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) -- 1.8.5.5

This patch should fix the bug reported in https://lkml.org/lkml/2014/9/11/249. Test is still pending. David Hildenbrand (1): blk-mq: Avoid race condition with uninitialized requests block/blk-mq.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) -- 1.8.5.5

On 09/18/2014 11:04 AM, David Hildenbrand wrote: > This patch should fix the bug reported in https://lkml.org/lkml/2014/9/11/249. > > We have to initialize at least the atomic_flags and the cmd_flags when > allocating storage for the requests. > > Otherwise blk_mq_timeout_check() might dereference uninitialized pointers when > racing with the creation of a request. > > Also move the reset of cmd_flags for the initializing code to the point where a > request is freed. So we will never end up with pending flush request indicators > that might trigger derefer...

On 09/18/2014 11:04 AM, David Hildenbrand wrote: > This patch should fix the bug reported in https://lkml.org/lkml/2014/9/11/249. > > We have to initialize at least the atomic_flags and the cmd_flags when > allocating storage for the requests. > > Otherwise blk_mq_timeout_check() might dereference uninitialized pointers when > racing with the creation of a request. > > Also move the reset of cmd_flags for the initializing code to the point where a > request is freed. So we will never end up with pending flush request indicators > that might trigger derefer...

...1560ca>] kthread+0xee/0x108 >> [ 66.438627] [<000000000067c76e>] kernel_thread_starter+0x6/0xc >> [ 66.438628] [<000000000067c768>] kernel_thread_starter+0x0/0xc >> [ 66.438629] Last Breaking-Event-Address: >> [ 66.438631] [<00000000003edde8>] blk_mq_timeout_check+0x6c/0xb8 >> >> I looked into the dump, and the full function is (annotated by me to match the source code) >> r2= tags >> r3= tag (4e) >> Dump of assembler code for function blk_mq_tag_to_rq: >> 0x00000000003ed0f4 <+0>: lg %r1,96(%r2)...

...1560ca>] kthread+0xee/0x108 >> [ 66.438627] [<000000000067c76e>] kernel_thread_starter+0x6/0xc >> [ 66.438628] [<000000000067c768>] kernel_thread_starter+0x0/0xc >> [ 66.438629] Last Breaking-Event-Address: >> [ 66.438631] [<00000000003edde8>] blk_mq_timeout_check+0x6c/0xb8 >> >> I looked into the dump, and the full function is (annotated by me to match the source code) >> r2= tags >> r3= tag (4e) >> Dump of assembler code for function blk_mq_tag_to_rq: >> 0x00000000003ed0f4 <+0>: lg %r1,96(%r2)...

This patch should fix the bug reported in https://lkml.org/lkml/2014/9/11/249. We have to initialize at least the atomic_flags and the cmd_flags when allocating storage for the requests. Otherwise blk_mq_timeout_check() might dereference uninitialized pointers when racing with the creation of a request. Also move the reset of cmd_flags for the initializing code to the point where a request is freed. So we will never end up with pending flush request indicators that might trigger dereferences of invalid pointers...

...1. __blk_mq_alloc_request() gets a free tag (thus e.g. removing it from bitmap_tags) 2. __blk_mq_alloc_request() initializes is via blk_mq_rq_ctx_init(). The struct is filled with life and rq->q is set. When blk_mq_hw_ctx_check_timeout() is called: 1. blk_mq_tag_busy_iter() is used to call blk_mq_timeout_check() on all busy tags. 2. This is done by collecting all free tags using bt_for_each_free() and handing them to blk_mq_timeout_check(). This uses bitmap_tags. 3. blk_mq_timeout_check() calls blk_mq_tag_to_rq() to get the rq. Could we have a race between - getting the tag (turning it busy) and in...

...1. __blk_mq_alloc_request() gets a free tag (thus e.g. removing it from bitmap_tags) 2. __blk_mq_alloc_request() initializes is via blk_mq_rq_ctx_init(). The struct is filled with life and rq->q is set. When blk_mq_hw_ctx_check_timeout() is called: 1. blk_mq_tag_busy_iter() is used to call blk_mq_timeout_check() on all busy tags. 2. This is done by collecting all free tags using bt_for_each_free() and handing them to blk_mq_timeout_check(). This uses bitmap_tags. 3. blk_mq_timeout_check() calls blk_mq_tag_to_rq() to get the rq. Could we have a race between - getting the tag (turning it busy) and in...

...x524 [ 66.438625] [<00000000001560ca>] kthread+0xee/0x108 [ 66.438627] [<000000000067c76e>] kernel_thread_starter+0x6/0xc [ 66.438628] [<000000000067c768>] kernel_thread_starter+0x0/0xc [ 66.438629] Last Breaking-Event-Address: [ 66.438631] [<00000000003edde8>] blk_mq_timeout_check+0x6c/0xb8 I looked into the dump, and the full function is (annotated by me to match the source code) r2= tags r3= tag (4e) Dump of assembler code for function blk_mq_tag_to_rq: 0x00000000003ed0f4 <+0>: lg %r1,96(%r2) # r1 has now tags->rqs 0x00000000003ed0fa <+6>:...

...x524 [ 66.438625] [<00000000001560ca>] kthread+0xee/0x108 [ 66.438627] [<000000000067c76e>] kernel_thread_starter+0x6/0xc [ 66.438628] [<000000000067c768>] kernel_thread_starter+0x0/0xc [ 66.438629] Last Breaking-Event-Address: [ 66.438631] [<00000000003edde8>] blk_mq_timeout_check+0x6c/0xb8 I looked into the dump, and the full function is (annotated by me to match the source code) r2= tags r3= tag (4e) Dump of assembler code for function blk_mq_tag_to_rq: 0x00000000003ed0f4 <+0>: lg %r1,96(%r2) # r1 has now tags->rqs 0x00000000003ed0fa <+6>:...

...[<00000000001560ca>] kthread+0xee/0x108 > [ 66.438627] [<000000000067c76e>] kernel_thread_starter+0x6/0xc > [ 66.438628] [<000000000067c768>] kernel_thread_starter+0x0/0xc > [ 66.438629] Last Breaking-Event-Address: > [ 66.438631] [<00000000003edde8>] blk_mq_timeout_check+0x6c/0xb8 > > I looked into the dump, and the full function is (annotated by me to match the source code) > r2= tags > r3= tag (4e) > Dump of assembler code for function blk_mq_tag_to_rq: > 0x00000000003ed0f4 <+0>: lg %r1,96(%r2) # r1 has now...

...ead+0xee/0x108 >>> [ 66.438627] [<000000000067c76e>] kernel_thread_starter+0x6/0xc >>> [ 66.438628] [<000000000067c768>] kernel_thread_starter+0x0/0xc >>> [ 66.438629] Last Breaking-Event-Address: >>> [ 66.438631] [<00000000003edde8>] blk_mq_timeout_check+0x6c/0xb8 >>> >>> I looked into the dump, and the full function is (annotated by me to match the source code) >>> r2= tags >>> r3= tag (4e) >>> Dump of assembler code for function blk_mq_tag_to_rq: >>> 0x00000000003ed0f4 <+0>: lg...