thr3ads.net - Btrfs devel - bonnie triggers and endless numbers of stack traces [Aug 2011]

If this information is useful, please help other people find it:
Share via:

Bernd Schubert

2011-Aug-19 16:45 UTC

bonnie triggers and endless numbers of stack traces

Just for performance tests I run:

./bonnie++ -d /mnt/btrfs -s0 -n 1:256:256:1 -r 0

and this causes and endless number of stack traces. Those seem to 
come from:

use_block_rsv()

	ret = block_rsv_use_bytes(block_rsv, blocksize);
	if (!ret)
		return block_rsv;
	if (ret) {
		WARN_ON(1);
		ret = reserve_metadata_bytes(trans, root, block_rsv, blocksize,


Why is there a WARN_ON(1)? Running the bonnie benchmark is basically impossible
that.

Thanks,
Bernd

> Aug 19 18:30:56 fslab2 kernel: [  265.255644] Loglevel set to 9
> Aug 19 18:31:26 fslab2 kernel: [  295.490858] ------------[ cut here
]------------
> Aug 19 18:31:26 fslab2 kernel: [  295.495589] WARNING: at
fs/btrfs/extent-tree.c:5711 btrfs_alloc_free_block+0x200/0x360 [btrfs]()
> Aug 19 18:31:26 fslab2 kernel: [  295.504472] Hardware name: H8DCE
> Aug 19 18:31:26 fslab2 kernel: [  295.507750] Modules linked in: nfsd
ib_umad rdma_ucm rdma_cm iw_cm ib_addr ib_uverbs sg ib_ipoib ib_cm ib_sa ip
> v6 sd_mod crc_t10dif loop arcmsr md_mod ib_mthca ib_mad pcspkr ib_core
8250_pnp fuse af_packet nfs lockd fscache auth_rpcgss nfs_acl sunrpc btrfs
>  lzo_decompress lzo_compress zlib_deflate crc32c libcrc32c crypto_hash
crypto_algapi ata_generic pata_acpi e1000 pata_amd sata_nv libata scsi_mod
>  unix [last unloaded: scsi_wait_scan]
> Aug 19 18:31:26 fslab2 kernel: [  295.548618] Pid: 2074, comm: bonnie++ Not
tainted 3.1.0-rc2+ #34
> Aug 19 18:31:26 fslab2 kernel: [  295.554695] Call Trace:
> Aug 19 18:31:26 fslab2 kernel: [  295.557209]  [<ffffffff8105c677>] ?
console_unlock+0x227/0x290
> Aug 19 18:31:26 fslab2 kernel: [  295.563111]  [<ffffffff8105bb7f>]
warn_slowpath_common+0x7f/0xc0
> Aug 19 18:31:26 fslab2 kernel: [  295.569186]  [<ffffffff8105bbda>]
warn_slowpath_null+0x1a/0x20
> Aug 19 18:31:26 fslab2 kernel: [  295.575096]  [<ffffffffa013d0d0>]
btrfs_alloc_free_block+0x200/0x360 [btrfs]
> Aug 19 18:31:26 fslab2 kernel: [  295.582230]  [<ffffffffa0165d10>] ?
lock_delalloc_pages+0x1f0/0x1f0 [btrfs]
> Aug 19 18:31:26 fslab2 kernel: [  295.589280]  [<ffffffffa0127b6b>]
__btrfs_cow_block+0x14b/0x6e0 [btrfs]
> Aug 19 18:31:26 fslab2 kernel: [  295.595978]  [<ffffffffa0179144>] ?
btrfs_try_tree_write_lock+0x44/0x80 [btrfs]
> Aug 19 18:31:26 fslab2 kernel: [  295.603394]  [<ffffffffa0128217>]
btrfs_cow_block+0x117/0x260 [btrfs]
> Aug 19 18:31:26 fslab2 kernel: [  295.609920]  [<ffffffffa012e455>]
btrfs_search_slot+0x385/0xaa0 [btrfs]
> Aug 19 18:31:26 fslab2 kernel: [  295.616621]  [<ffffffffa0140f3f>]
btrfs_lookup_inode+0x2f/0xa0 [btrfs]
> Aug 19 18:31:26 fslab2 kernel: [  295.623236]  [<ffffffffa0190eb3>]
btrfs_update_delayed_inode+0x73/0x160 [btrfs]
> Aug 19 18:31:26 fslab2 kernel: [  295.630644]  [<ffffffff8137163e>] ?
mutex_unlock+0xe/0x10
> Aug 19 18:31:26 fslab2 kernel: [  295.636125]  [<ffffffffa0192088>]
btrfs_run_delayed_items+0xe8/0x120 [btrfs]
> Aug 19 18:31:26 fslab2 kernel: [  295.643254]  [<ffffffffa014a240>]
btrfs_commit_transaction+0x230/0x870 [btrfs]
> Aug 19 18:31:26 fslab2 kernel: [  295.650585]  [<ffffffffa0149de9>] ?
join_transaction+0x69/0x290 [btrfs]
> Aug 19 18:31:26 fslab2 kernel: [  295.657274]  [<ffffffff8107f410>] ?
wake_up_bit+0x40/0x40
> Aug 19 18:31:26 fslab2 kernel: [  295.662783]  [<ffffffff81171700>] ?
__sync_filesystem+0x90/0x90
> Aug 19 18:31:26 fslab2 kernel: [  295.668783]  [<ffffffffa0124ace>]
btrfs_sync_fs+0x5e/0xd0 [btrfs]
> Aug 19 18:31:26 fslab2 kernel: [  295.674951]  [<ffffffff811716ce>]
__sync_filesystem+0x5e/0x90
> Aug 19 18:31:26 fslab2 kernel: [  295.680764]  [<ffffffff8117171f>]
sync_one_sb+0x1f/0x30
> Aug 19 18:31:26 fslab2 kernel: [  295.686061]  [<ffffffff8114751f>]
iterate_supers+0x7f/0xe0
> Aug 19 18:31:26 fslab2 kernel: [  295.691613]  [<ffffffff81171775>]
sys_sync+0x45/0x70
> Aug 19 18:31:26 fslab2 kernel: [  295.696648]  [<ffffffff8137b4c2>]
system_call_fastpath+0x16/0x1b
> Aug 19 18:31:26 fslab2 kernel: [  295.702726] ---[ end trace
5328a9730b4cdff4 ]---
> Aug 19 18:31:26 fslab2 kernel: [  295.707533] ------------[ cut here
]------------
> Aug 19 18:31:26 fslab2 kernel: [  295.712230] WARNING: at
fs/btrfs/extent-tree.c:5711 btrfs_alloc_free_block+0x200/0x360 [btrfs]()
> Aug 19 18:31:26 fslab2 kernel: [  295.721114] Hardware name: H8DCE
> Aug 19 18:31:26 fslab2 kernel: [  295.724410] Modules linked in: nfsd
ib_umad rdma_ucm rdma_cm iw_cm ib_addr ib_uverbs sg ib_ipoib ib_cm ib_sa ip
> v6 sd_mod crc_t10dif loop arcmsr md_mod ib_mthca ib_mad pcspkr ib_core
8250_pnp fuse af_packet nfs lockd fscache auth_rpcgss nfs_acl sunrpc btrfs
>  lzo_decompress lzo_compress zlib_deflate crc32c libcrc32c crypto_hash
crypto_algapi ata_generic pata_acpi e1000 pata_amd sata_nv libata scsi_mod[...]
repeats at least a few thousand times and fills the logs...



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Bernd Schubert

2011-Aug-19 17:28 UTC

head link

Re: bonnie triggers and endless numbers of stack traces

I think we either should remove it or replace by WARN_ON_ONCE()


Remove WARN_ON(1) in a common code path

From: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>

Something like bonnie++ -d /mnt/btrfs -s0 -n 1:256:256:1 -r 0
will trigger lots of those WARN_ON(1), so lets remove it.


Signed-off-by: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>
---
 fs/btrfs/extent-tree.c |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 80d6148..1d1a8d0 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -5708,7 +5708,6 @@ use_block_rsv(struct btrfs_trans_handle *trans,
        if (!ret)
                return block_rsv;
        if (ret) {
-               WARN_ON(1);
                ret = reserve_metadata_bytes(trans, root, block_rsv, blocksize,
                                             0);
                if (!ret) {




--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Josef Bacik

2011-Aug-19 19:36 UTC

head link

Re: bonnie triggers and endless numbers of stack traces

On 08/19/2011 12:45 PM, Bernd Schubert wrote:> Just for performance tests I run:
> 
> ./bonnie++ -d /mnt/btrfs -s0 -n 1:256:256:1 -r 0
> 
> and this causes and endless number of stack traces. Those seem to 
> come from:
> 
> use_block_rsv()
> 
> 	ret = block_rsv_use_bytes(block_rsv, blocksize);
> 	if (!ret)
> 		return block_rsv;
> 	if (ret) {
> 		WARN_ON(1);
> 		ret = reserve_metadata_bytes(trans, root, block_rsv, blocksize,
> 
> 
> Why is there a WARN_ON(1)? Running the bonnie benchmark is basically
impossible
> that.
This is being worked on, if you really don''t like it pull my git tree
and test it out and see if the errors go away

git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work.git

Thanks,

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

David Sterba

2011-Aug-19 23:45 UTC

head link

Re: bonnie triggers and endless numbers of stack traces

On Fri, Aug 19, 2011 at 03:36:55PM -0400, Josef Bacik
wrote:> This is being worked on, if you really don''t like it pull my git
tree
> and test it out and see if the errors go away
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work.git
pulled on top of latest linus'', with top commit:
"Btrfs: fix space leak when we fail to make an allocation"

but I still see it, xfstests/013.


david
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Bernd Schubert

2011-Aug-22 08:17 UTC

head link

Re: bonnie triggers and endless numbers of stack traces

On 08/19/2011 09:36 PM, Josef Bacik wrote:> On 08/19/2011 12:45 PM, Bernd Schubert wrote:
>> Just for performance tests I run:
>>
>> ./bonnie++ -d /mnt/btrfs -s0 -n 1:256:256:1 -r 0
>>
>> and this causes and endless number of stack traces. Those seem to
>> come from:
>>
>> use_block_rsv()
>>
>> 	ret = block_rsv_use_bytes(block_rsv, blocksize);
>> 	if (!ret)
>> 		return block_rsv;
>> 	if (ret) {
>> 		WARN_ON(1);
>> 		ret = reserve_metadata_bytes(trans, root, block_rsv, blocksize,
>>
>>
>> Why is there a WARN_ON(1)? Running the bonnie benchmark is basically
impossible
>> that.
>
> This is being worked on, if you really don''t like it pull my git
tree
> and test it out and see if the errors go away
>
> git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work.git
Do you plan to fix it for 3.1? If not, any chance to at least update it 
to WARN_ON_ONCE?


Thanks,
Bernd
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

David Sterba

2011-Aug-29 18:42 UTC

head link

[PATCH debug] btrfs: ratelimit WARN_ON in use_block_rsv

A debugging helper, not really intended for merge.

From: David Sterba <dsterba@suse.cz>

Signed-off-by: David Sterba <dsterba@suse.cz>
---
 fs/btrfs/extent-tree.c |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 44a3107..c5c1e7d 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -23,6 +23,7 @@
 #include <linux/rcupdate.h>
 #include <linux/kthread.h>
 #include <linux/slab.h>
+#include <linux/ratelimit.h>
 #include "compat.h"
 #include "hash.h"
 #include "ctree.h"
@@ -5703,7 +5704,13 @@ use_block_rsv(struct btrfs_trans_handle *trans,
 	if (!ret)
 		return block_rsv;
 	if (ret) {
-		WARN_ON(1);
+		static DEFINE_RATELIMIT_STATE(_rs,
+				DEFAULT_RATELIMIT_INTERVAL,
+				/*DEFAULT_RATELIMIT_BURST*/ 2);
+		if (__ratelimit(&_rs)) {
+			printk(KERN_DEBUG "btrfs: block rsv returned %d\n", ret);
+			WARN_ON(1);
+		}
 		ret = reserve_metadata_bytes(trans, root, block_rsv, blocksize,
 					     0);
 		if (!ret) {
-- 
1.7.6

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Btrfs devel - Aug 2011 - bonnie triggers and endless numbers of stack traces

bonnie triggers and endless numbers of stack traces

Re: bonnie triggers and endless numbers of stack traces

Re: bonnie triggers and endless numbers of stack traces

Re: bonnie triggers and endless numbers of stack traces

Re: bonnie triggers and endless numbers of stack traces

[PATCH debug] btrfs: ratelimit WARN_ON in use_block_rsv