From: Liu Bo <liub.liubo@gmail.com> This is a collection of some posted patches. It is also available in git://github.com/liubogithub/btrfs-work.git Liu Bo (5): Btrfs: fix a bug of per-file nocow Btrfs: update delayed ref''s tracepoints to show sequence Btrfs: fix a bug in parsing return value in logical resolve Btrfs: use helper for logical resolve Btrfs: use larger limit for translation of logical to inode liubo (1): Btrfs: cleanup for unused ref cache stuff fs/btrfs/backref.c | 26 +++++++++++++++++--------- fs/btrfs/backref.h | 3 ++- fs/btrfs/ctree.h | 3 --- fs/btrfs/disk-io.c | 5 ----- fs/btrfs/inode.c | 3 +-- fs/btrfs/ioctl.c | 24 +++++++----------------- fs/btrfs/scrub.c | 14 ++++++++------ fs/btrfs/send.c | 7 ++++--- include/trace/events/btrfs.h | 14 ++++++++++---- 9 files changed, 49 insertions(+), 50 deletions(-) -- 1.7.7.6 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Users report a bug, the reproducer is: $ mkfs.btrfs /dev/loop0 $ mount /dev/loop0 /mnt/btrfs/ $ mkdir /mnt/btrfs/dir $ chattr +C /mnt/btrfs/dir/ $ dd if=/dev/zero of=/mnt/btrfs/dir/foo bs=4K count=10; $ lsattr /mnt/btrfs/dir/foo ---------------C- /mnt/btrfs/dir/foo $ filefrag /mnt/btrfs/dir/foo /mnt/btrfs/dir/foo: 1 extent found ---> an extent $ dd if=/dev/zero of=/mnt/btrfs/dir/foo bs=4K count=1 seek=5 conv=notrunc,nocreat; sync $ filefrag /mnt/btrfs/dir/foo /mnt/btrfs/dir/foo: 3 extents found ---> with nocow, btrfs breaks the extent into three parts The new created file should not only inherit the NODATACOW flag, but also honor NODATASUM flag, because we must do COW on a file extent with checksum. Signed-off-by: Liu Bo <bo.li.liu@oracle.com> --- fs/btrfs/inode.c | 3 +-- fs/btrfs/ioctl.c | 3 ++- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index b2c3514..9bce6e9 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -4720,8 +4720,7 @@ static struct inode *btrfs_new_inode(struct btrfs_trans_handle *trans, if (S_ISREG(mode)) { if (btrfs_test_opt(root, NODATASUM)) BTRFS_I(inode)->flags |= BTRFS_INODE_NODATASUM; - if (btrfs_test_opt(root, NODATACOW) || - (BTRFS_I(dir)->flags & BTRFS_INODE_NODATACOW)) + if (btrfs_test_opt(root, NODATACOW)) BTRFS_I(inode)->flags |= BTRFS_INODE_NODATACOW; } diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 00ddf22..7390b25 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -141,7 +141,8 @@ void btrfs_inherit_iflags(struct inode *inode, struct inode *dir) } if (flags & BTRFS_INODE_NODATACOW) - BTRFS_I(inode)->flags |= BTRFS_INODE_NODATACOW; + BTRFS_I(inode)->flags |= BTRFS_INODE_NODATACOW | + BTRFS_INODE_NODATASUM; btrfs_update_iflags(inode); } -- 1.7.7.6 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
From: liubo <liubo2009@cn.fujitsu.com> As ref cache has been removed from btrfs, there is no user on its lock and its check. Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com> Signed-off-by: Liu Bo <bo.li.liu@oracle.com> --- fs/btrfs/ctree.h | 3 --- fs/btrfs/disk-io.c | 5 ----- 2 files changed, 0 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 0d195b5..4ae593c 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -1366,9 +1366,6 @@ struct btrfs_fs_info { struct rb_root defrag_inodes; atomic_t defrag_running; - spinlock_t ref_cache_lock; - u64 total_ref_cache_size; - /* * these three are in extended format (availability of single * chunks is denoted by BTRFS_AVAIL_ALLOC_BIT_SINGLE bit, other diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 22e98e0..bcd1703 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2000,7 +2000,6 @@ int open_ctree(struct super_block *sb, INIT_LIST_HEAD(&fs_info->caching_block_groups); spin_lock_init(&fs_info->delalloc_lock); spin_lock_init(&fs_info->trans_lock); - spin_lock_init(&fs_info->ref_cache_lock); spin_lock_init(&fs_info->fs_roots_radix_lock); spin_lock_init(&fs_info->delayed_iput_lock); spin_lock_init(&fs_info->defrag_inodes_lock); @@ -3211,10 +3210,6 @@ int close_ctree(struct btrfs_root *root) printk(KERN_INFO "btrfs: at unmount delalloc count %llu\n", (unsigned long long)fs_info->delalloc_bytes); } - if (fs_info->total_ref_cache_size) { - printk(KERN_INFO "btrfs: at umount reference cache size %llu\n", - (unsigned long long)fs_info->total_ref_cache_size); - } free_extent_buffer(fs_info->extent_root->node); free_extent_buffer(fs_info->extent_root->commit_root); -- 1.7.7.6 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Liu Bo
2012-Aug-31 05:36 UTC
[PATCH 3/6] Btrfs: update delayed ref''s tracepoints to show sequence
We''ve added a new field ''sequence'' to delayed ref node, so update related tracepoints. Signed-off-by: Liu Bo <bo.li.liu@oracle.com> --- include/trace/events/btrfs.h | 14 ++++++++++---- 1 files changed, 10 insertions(+), 4 deletions(-) diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h index 91b91e8..54fab04 100644 --- a/include/trace/events/btrfs.h +++ b/include/trace/events/btrfs.h @@ -445,6 +445,7 @@ TRACE_EVENT(btrfs_delayed_tree_ref, __field( u64, ref_root ) __field( int, level ) __field( int, type ) + __field( u64, seq ) ), TP_fast_assign( @@ -455,17 +456,19 @@ TRACE_EVENT(btrfs_delayed_tree_ref, __entry->ref_root = full_ref->root; __entry->level = full_ref->level; __entry->type = ref->type; + __entry->seq = ref->seq; ), TP_printk("bytenr = %llu, num_bytes = %llu, action = %s, " "parent = %llu(%s), ref_root = %llu(%s), level = %d, " - "type = %s", + "type = %s, seq = %llu", (unsigned long long)__entry->bytenr, (unsigned long long)__entry->num_bytes, show_ref_action(__entry->action), show_root_type(__entry->parent), show_root_type(__entry->ref_root), - __entry->level, show_ref_type(__entry->type)) + __entry->level, show_ref_type(__entry->type), + (unsigned long long)__entry->seq) ); TRACE_EVENT(btrfs_delayed_data_ref, @@ -485,6 +488,7 @@ TRACE_EVENT(btrfs_delayed_data_ref, __field( u64, owner ) __field( u64, offset ) __field( int, type ) + __field( u64, seq ) ), TP_fast_assign( @@ -496,11 +500,12 @@ TRACE_EVENT(btrfs_delayed_data_ref, __entry->owner = full_ref->objectid; __entry->offset = full_ref->offset; __entry->type = ref->type; + __entry->seq = ref->seq; ), TP_printk("bytenr = %llu, num_bytes = %llu, action = %s, " "parent = %llu(%s), ref_root = %llu(%s), owner = %llu, " - "offset = %llu, type = %s", + "offset = %llu, type = %s, seq = %llu", (unsigned long long)__entry->bytenr, (unsigned long long)__entry->num_bytes, show_ref_action(__entry->action), @@ -508,7 +513,8 @@ TRACE_EVENT(btrfs_delayed_data_ref, show_root_type(__entry->ref_root), (unsigned long long)__entry->owner, (unsigned long long)__entry->offset, - show_ref_type(__entry->type)) + show_ref_type(__entry->type), + (unsigned long long)__entry->seq) ); TRACE_EVENT(btrfs_delayed_ref_head, -- 1.7.7.6 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Liu Bo
2012-Aug-31 05:36 UTC
[PATCH 4/6] Btrfs: fix a bug in parsing return value in logical resolve
From: Liu Bo <liub.liubo@gmail.com> In logical resolve, we parse extent_from_logical()''s ''ret'' as a kind of flag. It is possible to lose our errors because (-EXXXX & BTRFS_EXTENT_FLAG_TREE_BLOCK) is true. I''m not sure if it is on purpose, it just looks too hacky if it is. I''d rather use a real flag and a ''ret'' to catch errors. Acked-by: Jan Schmidt <list.btrfs@jan-o-sch.net> Signed-off-by: Liu Bo <liub.liubo@gmail.com> --- fs/btrfs/backref.c | 24 ++++++++++++++++-------- fs/btrfs/backref.h | 3 ++- fs/btrfs/ioctl.c | 6 ++++-- fs/btrfs/scrub.c | 14 ++++++++------ fs/btrfs/send.c | 7 ++++--- 5 files changed, 34 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/backref.c b/fs/btrfs/backref.c index ff6475f..4e80ba3 100644 --- a/fs/btrfs/backref.c +++ b/fs/btrfs/backref.c @@ -1193,7 +1193,8 @@ char *btrfs_iref_to_path(struct btrfs_root *fs_root, struct btrfs_path *path, * tree blocks and <0 on error. */ int extent_from_logical(struct btrfs_fs_info *fs_info, u64 logical, - struct btrfs_path *path, struct btrfs_key *found_key) + struct btrfs_path *path, struct btrfs_key *found_key, + u64 *flags_ret) { int ret; u64 flags; @@ -1237,10 +1238,17 @@ int extent_from_logical(struct btrfs_fs_info *fs_info, u64 logical, (unsigned long long)found_key->objectid, (unsigned long long)found_key->offset, (unsigned long long)flags, item_size); - if (flags & BTRFS_EXTENT_FLAG_TREE_BLOCK) - return BTRFS_EXTENT_FLAG_TREE_BLOCK; - if (flags & BTRFS_EXTENT_FLAG_DATA) - return BTRFS_EXTENT_FLAG_DATA; + + WARN_ON(!flags_ret); + if (flags_ret) { + if (flags & BTRFS_EXTENT_FLAG_TREE_BLOCK) + *flags_ret = BTRFS_EXTENT_FLAG_TREE_BLOCK; + else if (flags & BTRFS_EXTENT_FLAG_DATA) + *flags_ret = BTRFS_EXTENT_FLAG_DATA; + else + BUG_ON(1); + return 0; + } return -EIO; } @@ -1432,15 +1440,15 @@ int iterate_inodes_from_logical(u64 logical, struct btrfs_fs_info *fs_info, { int ret; u64 extent_item_pos; + u64 flags = 0; struct btrfs_key found_key; int search_commit_root = path->search_commit_root; - ret = extent_from_logical(fs_info, logical, path, - &found_key); + ret = extent_from_logical(fs_info, logical, path, &found_key, &flags); btrfs_release_path(path); if (ret < 0) return ret; - if (ret & BTRFS_EXTENT_FLAG_TREE_BLOCK) + if (flags & BTRFS_EXTENT_FLAG_TREE_BLOCK) return -EINVAL; extent_item_pos = logical - found_key.objectid; diff --git a/fs/btrfs/backref.h b/fs/btrfs/backref.h index 032f4dc..4fda5d8 100644 --- a/fs/btrfs/backref.h +++ b/fs/btrfs/backref.h @@ -40,7 +40,8 @@ int inode_item_info(u64 inum, u64 ioff, struct btrfs_root *fs_root, struct btrfs_path *path); int extent_from_logical(struct btrfs_fs_info *fs_info, u64 logical, - struct btrfs_path *path, struct btrfs_key *found_key); + struct btrfs_path *path, struct btrfs_key *found_key, + u64 *flags); int tree_backref_for_extent(unsigned long *ptr, struct extent_buffer *eb, struct btrfs_extent_item *ei, u32 item_size, diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 7390b25..123ffee 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -3217,6 +3217,7 @@ static long btrfs_ioctl_logical_to_ino(struct btrfs_root *root, int ret = 0; int size; u64 extent_item_pos; + u64 flags = 0; struct btrfs_ioctl_logical_ino_args *loi; struct btrfs_data_container *inodes = NULL; struct btrfs_path *path = NULL; @@ -3246,10 +3247,11 @@ static long btrfs_ioctl_logical_to_ino(struct btrfs_root *root, goto out; } - ret = extent_from_logical(root->fs_info, loi->logical, path, &key); + ret = extent_from_logical(root->fs_info, loi->logical, path, &key, + &flags); btrfs_release_path(path); - if (ret & BTRFS_EXTENT_FLAG_TREE_BLOCK) + if (flags & BTRFS_EXTENT_FLAG_TREE_BLOCK) ret = -ENOENT; if (ret < 0) goto out; diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index b223620..5def223 100644 --- a/fs/btrfs/scrub.c +++ b/fs/btrfs/scrub.c @@ -352,13 +352,14 @@ static void scrub_print_warning(const char *errstr, struct scrub_block *sblock) struct extent_buffer *eb; struct btrfs_extent_item *ei; struct scrub_warning swarn; - u32 item_size; - int ret; + unsigned long ptr = 0; + u64 extent_item_pos; + u64 flags = 0; u64 ref_root; + u32 item_size; u8 ref_level; - unsigned long ptr = 0; const int bufsize = 4096; - u64 extent_item_pos; + int ret; path = btrfs_alloc_path(); @@ -375,7 +376,8 @@ static void scrub_print_warning(const char *errstr, struct scrub_block *sblock) if (!path || !swarn.scratch_buf || !swarn.msg_buf) goto out; - ret = extent_from_logical(fs_info, swarn.logical, path, &found_key); + ret = extent_from_logical(fs_info, swarn.logical, path, &found_key, + &flags); if (ret < 0) goto out; @@ -387,7 +389,7 @@ static void scrub_print_warning(const char *errstr, struct scrub_block *sblock) item_size = btrfs_item_size_nr(eb, path->slots[0]); btrfs_release_path(path); - if (ret & BTRFS_EXTENT_FLAG_TREE_BLOCK) { + if (flags & BTRFS_EXTENT_FLAG_TREE_BLOCK) { do { ret = tree_backref_for_extent(&ptr, eb, ei, item_size, &ref_root, &ref_level); diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index fb5ffe9..7186ae8 100644 --- a/fs/btrfs/send.c +++ b/fs/btrfs/send.c @@ -1143,6 +1143,7 @@ static int find_extent_clone(struct send_ctx *sctx, u64 logical; u64 num_bytes; u64 extent_item_pos; + u64 flags = 0; struct btrfs_file_extent_item *fi; struct extent_buffer *eb = path->nodes[0]; struct backref_ctx backref_ctx; @@ -1181,13 +1182,13 @@ static int find_extent_clone(struct send_ctx *sctx, } logical += btrfs_file_extent_offset(eb, fi); - ret = extent_from_logical(sctx->send_root->fs_info, - logical, tmp_path, &found_key); + ret = extent_from_logical(sctx->send_root->fs_info, logical, tmp_path, + &found_key, &flags); btrfs_release_path(tmp_path); if (ret < 0) goto out; - if (ret & BTRFS_EXTENT_FLAG_TREE_BLOCK) { + if (flags & BTRFS_EXTENT_FLAG_TREE_BLOCK) { ret = -EIO; goto out; } -- 1.7.7.6 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
We already have a helper, iterate_inodes_from_logical(), for logical resolve, so just use it. Signed-off-by: Liu Bo <bo.li.liu@oracle.com> --- fs/btrfs/ioctl.c | 19 +++---------------- 1 files changed, 3 insertions(+), 16 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 123ffee..7544134 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -3216,12 +3216,9 @@ static long btrfs_ioctl_logical_to_ino(struct btrfs_root *root, { int ret = 0; int size; - u64 extent_item_pos; - u64 flags = 0; struct btrfs_ioctl_logical_ino_args *loi; struct btrfs_data_container *inodes = NULL; struct btrfs_path *path = NULL; - struct btrfs_key key; if (!capable(CAP_SYS_ADMIN)) return -EPERM; @@ -3247,23 +3244,13 @@ static long btrfs_ioctl_logical_to_ino(struct btrfs_root *root, goto out; } - ret = extent_from_logical(root->fs_info, loi->logical, path, &key, - &flags); - btrfs_release_path(path); - - if (flags & BTRFS_EXTENT_FLAG_TREE_BLOCK) + ret = iterate_inodes_from_logical(loi->logical, root->fs_info, path, + build_ino_list, inodes); + if (ret == -EINVAL) ret = -ENOENT; if (ret < 0) goto out; - extent_item_pos = loi->logical - key.objectid; - ret = iterate_extent_inodes(root->fs_info, key.objectid, - extent_item_pos, 0, build_ino_list, - inodes); - - if (ret < 0) - goto out; - ret = copy_to_user((void *)(unsigned long)loi->inodes, (void *)(unsigned long)inodes, size); if (ret) -- 1.7.7.6 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Liu Bo
2012-Aug-31 05:36 UTC
[PATCH 6/6] Btrfs: use larger limit for translation of logical to inode
This is the change of the kernel side. Translation of logical to inode used to have an upper limit 4k on inode container''s size, but the limit is not large enough for a data with a great many of refs, so when resolving logical address, we can end up with "ioctl ret=0, bytes_left=0, bytes_missing=19944, cnt=510, missed=2493" This changes to regard 64k as the upper limit and use vmalloc instead of kmalloc to get memory more easily. Signed-off-by: Liu Bo <bo.li.liu@oracle.com> --- fs/btrfs/backref.c | 2 +- fs/btrfs/ioctl.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/backref.c b/fs/btrfs/backref.c index 4e80ba3..fb72931 100644 --- a/fs/btrfs/backref.c +++ b/fs/btrfs/backref.c @@ -1583,7 +1583,7 @@ struct btrfs_data_container *init_data_container(u32 total_bytes) size_t alloc_bytes; alloc_bytes = max_t(size_t, total_bytes, sizeof(*data)); - data = kmalloc(alloc_bytes, GFP_NOFS); + data = vmalloc(alloc_bytes); if (!data) return ERR_PTR(-ENOMEM); diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 7544134..2a303d5 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -3236,7 +3236,7 @@ static long btrfs_ioctl_logical_to_ino(struct btrfs_root *root, goto out; } - size = min_t(u32, loi->size, 4096); + size = min_t(u32, loi->size, 64 * 1024); inodes = init_data_container(size); if (IS_ERR(inodes)) { ret = PTR_ERR(inodes); @@ -3258,7 +3258,7 @@ static long btrfs_ioctl_logical_to_ino(struct btrfs_root *root, out: btrfs_free_path(path); - kfree(inodes); + vfree(inodes); kfree(loi); return ret; -- 1.7.7.6 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Aug 31, 2012 at 01:36:54PM +0800, Liu Bo wrote:> The new created file should not only inherit the NODATACOW flag, but also > honor NODATASUM flag, because we must do COW on a file extent with checksum.I was not entierly correct when suggesting to move NODATASUM to inherit_iflags. Se below.> > Signed-off-by: Liu Bo <bo.li.liu@oracle.com> > --- > fs/btrfs/inode.c | 3 +-- > fs/btrfs/ioctl.c | 3 ++- > 2 files changed, 3 insertions(+), 3 deletions(-) > > diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c > index b2c3514..9bce6e9 100644 > --- a/fs/btrfs/inode.c > +++ b/fs/btrfs/inode.c > @@ -4720,8 +4720,7 @@ static struct inode *btrfs_new_inode(struct btrfs_trans_handle *trans, > if (S_ISREG(mode)) {Originaly it applied to regular files only.> if (btrfs_test_opt(root, NODATASUM)) > BTRFS_I(inode)->flags |= BTRFS_INODE_NODATASUM; > - if (btrfs_test_opt(root, NODATACOW) || > - (BTRFS_I(dir)->flags & BTRFS_INODE_NODATACOW)) > + if (btrfs_test_opt(root, NODATACOW)) > BTRFS_I(inode)->flags |= BTRFS_INODE_NODATACOW; > } > > diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c > index 00ddf22..7390b25 100644 > --- a/fs/btrfs/ioctl.c > +++ b/fs/btrfs/ioctl.c > @@ -141,7 +141,8 @@ void btrfs_inherit_iflags(struct inode *inode, struct inode *dir) > } > > if (flags & BTRFS_INODE_NODATACOW) > - BTRFS_I(inode)->flags |= BTRFS_INODE_NODATACOW; > + BTRFS_I(inode)->flags |= BTRFS_INODE_NODATACOW | > + BTRFS_INODE_NODATASUM;But now this will happen to: * nodes (mknod) * directories -- NODATASUM for a dir does nothing, but does not feel right * symlinks -- same * subvolume roots -- same, and even more confusing what it does While the fix you suggested in http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg18292.html is simple, I''m still inclined to keep all flag inheritance at one place, so the fix I''m proposing is to do> if (flags & BTRFS_INODE_NODATACOW) > BTRFS_I(inode)->flags |= BTRFS_INODE_NODATACOW;if (S_ISREG(inode->i_mode)) BTRFS_I(inode)->flags |= BTRFS_INODE_NODATASUM; }> > btrfs_update_iflags(inode); > }-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html