search for: btrfs_run_delayed_ref

Write dirty block groups may allocate new block, thus may add new delayed back ref. btrfs_run_delayed_refs may make some block groups dirty. commit_cowonly_roots does not handle the recursion properly, so some dirty blocks can be left unwritten at commit time. This patch move btrfs_run_delayed_refs into the loop that writes dirty block groups, and make the code not break out of the loop until there is...

...fffa02e1c34>] ? btrfs_search_slot+0x162/0x4cb [btrfs] [<ffffffffa02e2ea3>] ? btrfs_insert_empty_items+0x6a/0xba [btrfs] [<ffffffffa02e9bf3>] ? run_clustered_refs+0x370/0x682 [btrfs] [<ffffffffa032d201>] ? btrfs_find_ref_cluster+0xd/0x13c [btrfs] [<ffffffffa02e9fd6>] ? btrfs_run_delayed_refs+0xd1/0x17c [btrfs] [<ffffffffa02f8467>] ? btrfs_commit_transaction+0x38f/0x709 [btrfs] [<ffffffff8136f6e6>] ? _raw_spin_lock+0xe/0x10 [<ffffffffa02f79fe>] ? join_transaction.clone.23+0xc1/0x200 [btrfs] [<ffffffff81068ffb>] ? wake_up_bit+0x2a/0x2a [<ffffffffa02f28fd&...

Hello, I have a 5.5TB Btrfs filesystem on top of a md-raid 5 device. Now if i run some file operations like find, i get these messages. kernel is 2.6.38.5-1 on arch linux May 5 14:15:12 mail kernel: [13559.089713] parent transid verify failed on 3062073683968 wanted 5181 found 5188 May 5 14:15:12 mail kernel: [13559.089834] parent transid verify failed on 3062073683968 wanted 5181 found 5188

caching_thread()s do all their work under read access to extent_commit_sem. They give up on this read access only when need_resched() tells them, or when they exit. As a result, somebody that wants a WRITE access to this sem, might wait for a long time. Especially this is problematic in cache_block_group(), which can be called on critical paths like find_free_extent() and in commit path via

...0 [btrfs] [ 386.318592] [<ffffffffa00d6324>] run_delayed_tree_ref+0x114/0x190 [btrfs] [ 386.318623] [<ffffffffa00dac2e>] run_one_delayed_ref+0xde/0xf0 [btrfs] [ 386.318654] [<ffffffffa00dad76>] run_clustered_refs+0x136/0x3d0 [btrfs] [ 386.318685] [<ffffffffa00db100>] btrfs_run_delayed_refs+0xf0/0x2d0 [btrfs] [ 386.318721] [<ffffffffa00ecf99>] __btrfs_end_transaction+0xf9/0x3c0 [btrfs] [ 386.318753] [<ffffffffa00dcd35>] ? btrfs_drop_snapshot+0x545/0x600 [btrfs] [ 386.318789] [<ffffffffa00ed2b8>] btrfs_end_transaction_throttle+0x18/0x20 [btrfs] [ 386.318820]...

...:35:40 localhost kernel: [1655047.752803] [<ffffffff81051d06>] warn_slowpath_fmt+0x46/0x50 Jan 21 16:35:40 localhost kernel: [1655047.752814] [<ffffffffa016ed6d>] __btrfs_abort_transaction+0xad/0xc0 [btrfs] Jan 21 16:35:40 localhost kernel: [1655047.752826] [<ffffffffa0183d24>] btrfs_run_delayed_refs+0x3f4/0x440 [btrfs] Jan 21 16:35:40 localhost kernel: [1655047.752840] [<ffffffffa01b62a9>] ? merge_state+0xd9/0x150 [btrfs] Jan 21 16:35:40 localhost kernel: [1655047.752845] [<ffffffff8168035e>] ? _raw_spin_lock+0xe/0x20 Jan 21 16:35:40 localhost kernel: [1655047.752856] [<ffff...

...4:43 localhost kernel: [25554.948827] [<ffffffffa01a5b6d>] ? run_clustered_refs+0x11d/0x830 [btrfs] Oct 22 20:44:43 localhost kernel: [25554.948847] [<ffffffffa01a5dd9>] run_clustered_refs+0x389/0x830 [btrfs] Oct 22 20:44:43 localhost kernel: [25554.948867] [<ffffffffa01a6348>] btrfs_run_delayed_refs+0xc8/0x220 [btrfs] Oct 22 20:44:43 localhost kernel: [25554.948881] [<ffffffff813f29ce>] ? mutex_unlock+0xe/0x10 Oct 22 20:44:43 localhost kernel: [25554.948900] [<ffffffffa01b8381>] btrfs_commit_transaction+0x81/0x8a0 [btrfs] Oct 22 20:44:43 localhost kernel: [25554.948915] [<ff...

...f0>] btrfs_del_csums+0x2b0/0x300 [btrfs] Dec 11 17:49:04 SANOS1 kernel: [<ffffffffa0227149>] __btrfs_free_extent+0x639/0x7b0 [btrfs] Dec 11 17:49:04 SANOS1 kernel: [<ffffffffa022b34e>] run_clustered_refs+0x2be/0xa50 [btrfs] Dec 11 17:49:04 SANOS1 kernel: [<ffffffffa022bc76>] btrfs_run_delayed_refs+0x196/0x4c0 [btrfs] Dec 11 17:49:04 SANOS1 kernel: [<ffffffffa025cac8>] ? merge_state+0xd8/0x150 [btrfs] Dec 11 17:49:04 SANOS1 kernel: [<ffffffffa025c949>] ? free_extent_state+0x19/0x20 [btrfs] Dec 11 17:49:04 SANOS1 kernel: [<ffffffffa025d476>] ? clear_extent_bit+0x216/0x380...

...3 kernel: [158311.402427] [<ffffffffa01c0ea0>] ? run_clustered_refs+0xa74/0xa79 [btrfs] Nov 18 06:59:46 hsad-srv-03 kernel: [158311.402434] [<ffffffffa01b0940>] ? __btrfs_abort_transaction+0x4a/0xfc [btrfs] Nov 18 06:59:46 hsad-srv-03 kernel: [158311.402442] [<ffffffffa01c3f97>] ? btrfs_run_delayed_refs+0x239/0x3ab [btrfs] Nov 18 06:59:46 hsad-srv-03 kernel: [158311.402445] [<ffffffff810593c4>] ? should_resched+0x5/0x23 Nov 18 06:59:46 hsad-srv-03 kernel: [158311.402447] [<ffffffff813784b2>] ? mutex_lock+0x9/0x25 Nov 18 06:59:46 hsad-srv-03 kernel: [158311.402457] [<ffffffffa01d11e...

...arn_slowpath_common+0x78/0x8c [172816.293012] [<ffffffff8111ea5b>] ? __btrfs_free_extent+0x290/0x5c7 [172816.293014] [<ffffffff810b2490>] ? __slab_free+0xd1/0x236 [172816.293016] [<ffffffff81121d68>] ? run_clustered_refs+0x66c/0x6b8 [172816.293018] [<ffffffff81121e7d>] ? btrfs_run_delayed_refs+0xc9/0x173 [172816.293021] [<ffffffff8112faf0>] ? __btrfs_end_transaction+0x90/0x1dd [172816.293024] [<ffffffff810273b0>] ? should_resched+0x5/0x24 [172816.293027] [<ffffffff81166981>] ? btrfs_async_run_delayed_node_done+0x16c/0x1ca [172816.293029] [<ffffffff8114f20f>]...

...;ffffffff811da606>] ? btrfs_search_slot+0x2f5/0x624 kernel: [<ffffffff811dbc5a>] ? btrfs_insert_empty_items+0x5c/0xaf kernel: [<ffffffff811e5089>] ? run_clustered_refs+0x852/0x8e6 kernel: [<ffffffff811e4d20>] ? run_clustered_refs+0x4e9/0x8e6 kernel: [<ffffffff811e7f6b>] ? btrfs_run_delayed_refs+0x10d/0x289 kernel: [<ffffffff811f4ec6>] ? btrfs_commit_transaction+0x3a5/0x93c kernel: [<ffffffff810427f0>] ? abort_exclusive_wait+0x79/0x79 kernel: [<ffffffff811f5a8c>] ? start_transaction+0x311/0x408 kernel: [<ffffffff811eed7e>] ? transaction_kthread+0xd1/0x16d kernel: [...

Hi I am hitting this Warning reproducible, the workload is a ceph osd, kernel ist 3.1.0-rc5. Best Regards, martin [ 5472.099766] ------------[ cut here ]------------ [ 5472.099833] WARNING: at fs/btrfs/inode.c:2193 btrfs_orphan_commit_root+0xb0/0xc0 [btrfs]() [ 5472.099838] Hardware name: MS-96B3 [ 5472.099842] Modules linked in: radeon ttm drm_kms_helper drm i2c_algo_bit psmouse sp5100_tco

...7972] [<ffffffffa05a5278>] ? btrfs_find_ref_cluster+0x68/0x170 [btrfs] [ 469.037999] [<ffffffffa059f6e1>] btrfs_write_out_cache+0xb1/0xf0 [btrfs] [ 469.038021] [<ffffffffa055854f>] btrfs_write_dirty_block_groups+0x50f/0x5e0 [btrfs] [ 469.038043] [<ffffffffa0557f01>] ? btrfs_run_delayed_refs+0x1a1/0x2e0 [btrfs] [ 469.038072] [<ffffffffa05c2dd0>] commit_cowonly_roots+0x121/0x1eb [btrfs] [ 469.038099] [<ffffffffa0569aad>] btrfs_commit_transaction+0x59d/0xac0 [btrfs] [ 469.038105] [<ffffffff8107f610>] ? wake_up_bit+0x40/0x40 [ 469.038134] [<ffffffffa05b4d46&g...

...14:50:23 mithrandir kernel: [40871.705733] [<ffffffff81025042>] ? set_next_entity+0x3b/0x60 Jun 10 14:50:23 mithrandir kernel: [40871.705776] [<ffffffffa0db768f>] ? btrfs_find_ref_cluster+0xf6/0x137 [btrfs] Jun 10 14:50:23 mithrandir kernel: [40871.705822] [<ffffffffa0d77e38>] btrfs_run_delayed_refs+0xcc/0x17c [btrfs] Jun 10 14:50:23 mithrandir kernel: [40871.705871] [<ffffffffa0d85775>] btrfs_commit_transaction+0x72/0x655 [btrfs] Jun 10 14:50:23 mithrandir kernel: [40871.705920] [<ffffffffa0d84d89>] ? join_transaction.isra.23+0x203/0x20a [btrfs] Jun 10 14:50:23 mithrandir kerne...

...676.673653] ffff8800a9b3bba8 0000000000000001 ffff8801161e1660 0000000000000005 [ 676.673653] Call Trace: [ 676.673653] [<ffffffff810336f0>] ? get_parent_ip+0xf/0x40 [ 676.673653] [<ffffffff8123e641>] ? btrfs_find_ref_cluster+0xf6/0x137 [ 676.673653] [<ffffffff81201e25>] btrfs_run_delayed_refs+0xcc/0x17c [ 676.673653] [<ffffffff811f5d3a>] ? btrfs_free_path+0x22/0x26 [ 676.673653] [<ffffffff8120fa5e>] __btrfs_end_transaction+0x98/0x1f3 [ 676.673653] [<ffffffff8120fbf0>] btrfs_end_transaction_throttle+0x13/0x15 [ 676.673653] [<ffffffff81211a56>] __unlink_en...

...16:58:36 repository kernel: [ 146.190020] [<ffffffffa01ec994>] run_one_delayed_ref+0x64/0xe0 [btrfs] Jan 6 16:58:36 repository kernel: [ 146.190020] [<ffffffffa01ecacb>] run_clustered_refs+0xbb/0x220 [btrfs] Jan 6 16:58:36 repository kernel: [ 146.190020] [<ffffffffa01ecce0>] btrfs_run_delayed_refs+0xb0/0x200 [btrfs] Jan 6 16:58:36 repository kernel: [ 146.190020] [<ffffffffa01f8a6f>] btrfs_commit_transaction+0x7f/0x620 [btrfs] Jan 6 16:58:36 repository kernel: [ 146.190020] [<ffffffff815a7dbe>] ? mutex_lock+0x1e/0x50 Jan 6 16:58:36 repository kernel: [ 146.190020] [<fff...

On 06/30/2011 09:13 PM, Josef Bacik wrote: > On 06/30/2011 10:12 AM, Proskurin Kirill wrote: >> On 06/29/2011 08:14 PM, Josef Bacik wrote: >>>> Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse. >>>> Now I see [btrfs-transacti]& btrfs-endio-wri] 80-100% all the time and >>>> io performance looks like lower then before.

...able generic_file_aio_read do_sync_read vfs_read sys_read sysenter_do_call (chrome) 431.9ms sync_page wait_on_page_bit read_extent_buffer_pages btree_read_extent_buffer_pages read_tree_block read_block_for_search btrfs_search_slot lookup_inline_extent_backref __btrfs_free_extent run_clustered_refs btrfs_run_delayed_refs btrfs_commit_transaction (btrfs) Cheers. -- Felipe Contreras -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

...x81/0xe0 > >>>> Feb 14 00:39:20 lupus kernel: [ 2951.854261]  [<ffffffff812fce15>] > ? > >>>> btrfs_write_dirty_block_groups+0x245/0x500 > >>>> Feb 14 00:39:20 lupus kernel: [ 2951.854283]  [<ffffffff812fcb6a>] > ? > >>>> btrfs_run_delayed_refs+0x1ba/0x220 > >>>> Feb 14 00:39:20 lupus kernel: [ 2951.854304]  [<ffffffff8130afff>] > ? > >>>> commit_cowonly_roots+0xff/0x1d0 > >>>> Feb 14 00:39:20 lupus kernel: [ 2951.854323]  [<ffffffff8130c583>] > ? > >>>> btrfs_...

...cd #1 [36277.691919] Call Trace: [36277.691933] [<ffffffff8106382f>] warn_slowpath_common+0x7e/0x96 [36277.691942] [<ffffffff810638db>] warn_slowpath_fmt+0x41/0x43 [36277.691952] [<ffffffff8125e626>] __btrfs_abort_transaction+0x40/0x9d [36277.691963] [<ffffffff8126dd55>] btrfs_run_delayed_refs+0x267/0x34b [36277.691972] [<ffffffff8111e2f3>] ? virt_to_head_page+0x9/0x2c [36277.691982] [<ffffffff8127c241>] __btrfs_end_transaction+0x7f/0x21b [36277.691991] [<ffffffff8127c426>] btrfs_end_transaction+0x10/0x12 [36277.692000] [<ffffffff812810c0>] btrfs_finish_order...