similar to: [PATCH 0/6] Btrfs commit fixes, async subvol operations

Hi everyone, This patchset is the latest approach I''m using for the Ceph storage daemon to keep track of which data has safely committed to disk. The basic idea is to not use the (problematic) user transaction ioctls at all. Instead, the daemon quiesces its own write requests, initiates an async snapshot, and then continues. The snapshot approach is nice because it provides rollback.

I''ve been working to try to improve the write-ahead log''s performance, and I found that the bottleneck addresses in the checksum items, especially when we want to make a random write on a large file, e.g a 4G file. Then a idea for this suggested by Chris is to use sub transaction ids and just to log the part of inode that had changed since either the last log commit or the last

This is a first draft of a subvolume quota implementation. It is possible to limit subvolumes and any group of subvolumes and also to track the amount of space that will get freed when deleting snapshots. The current version is functionally incomplete, with the main missing feature being the initial scan and rescan of an existing filesystem. I put some effort into writing an introduction into

Hello, Ok here are all my fsync changes, including the fix for the put_transaction stuff that I posted earlier. I figure it'll be easier to just commit it as one thing then each individual peice. The additional thing that I did was add a radix tree that tracks the current outstanding transactions. I also added a field to the in memory btrfs inode that keeps track of the last transaction

There are several bugs at error path of create_snapshot() when the transaction commitment failed. - access the freed transaction handler. At the end of the transaction commitment, the transaction handler was freed, so we should not access it after the transaction commitment. - we were not aware of the error which happened during the snapshot creation if we submitted a async transaction

We calculate timeout (either 1 or MAX_SCHEDULE_TIMEOUT) based on whether num_writers > 1 or should_grow at the top of the loop. Then, much much later, we wait for that timeout if either num_writers or should_grow is true. However, it''s possible for a racing process (calling btrfs_end_transaction()) to decrement num_writers such that we wait forever instead of for 1. Fix this by

Hello, This patch series adds an initial implementation of restriper (it''s a clever name for relocation framework that allows to do selective profile changing and selective balancing with some goodies like pausing/resuming and reporting progress to the user. Profile changing is global (per-FS) so far, per-subvolume profiles require some discussion and can be implemented in future.

Hi Chris- Here''s a simpler version of the patch that drops the unrelated sync_fs stuff. thanks- sage The ''flushoncommit'' mount option forces any data dirtied by a write in a prior transaction to commit as part of the current commit. This makes the committed state a fully consistent view of the file system from the application''s perspective (i.e., it

The kernel side for rescan, which is needed if you want to enable qgroup tracking on a non-empty volume. The first patch splits btrfs_qgroup_account_ref into readable ans reusable units. The second patch adds the rescan implementation (refer to its commit message for a description of the algorithm). The third patch starts an automatic rescan when qgroups are enabled. It is only separated to

If we have particularly full nodes, we could call btrfs_node_blockptr up to 32 times, which is 32 pairs of kmap/kunmap, which _sucks_. So go ahead and map the extent buffer while we look for readahead targets. Thanks, Signed-off-by: Josef Bacik <josef@redhat.com> --- fs/btrfs/ctree.c | 23 +++++++++++++++++++++-- 1 files changed, 21 insertions(+), 2 deletions(-) diff --git

Reserve metadata space for extent tree, checksum tree and root tree Signed-off-by: Yan Zheng <zheng.yan@oracle.com> --- diff -urp 8/fs/btrfs/ctree.h 9/fs/btrfs/ctree.h --- 8/fs/btrfs/ctree.h 2010-04-18 10:26:38.327697818 +0800 +++ 9/fs/btrfs/ctree.h 2010-04-18 10:30:01.883697869 +0800 @@ -682,21 +682,15 @@ struct btrfs_space_info { u64 bytes_reserved; /* total bytes the allocator has

This patch finally gives us full proper -ENOSPC handling for btrfs. Now whenever you do a btrfs_start_transaction, you must specify the number of items you are planning to add/delete/modify. The worst case number of blocks that could be modified by changing that number of items will be calculated and checked against the amount of free space in the space_info where the root you are modifying

This series adds an initial implementation for scrub. It works quite straightforward. The usermode issues an ioctl for each device in the fs. For each device, it enumerates the allocated device chunks. For each chunk, the contained extents are enumerated and the data checksums fetched. The extents are read sequentially and the checksums verified. If an error occurs (checksum or EIO), a good copy

When btrfs_start_transaction() fails, we should call btrfs_std_error() properly for filesystem to readonly. (in this patch, forced readonly framework is used) Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com> --- fs/btrfs/file.c | 1 + fs/btrfs/inode.c | 34 +++++++++++++++++++++++++++------- fs/btrfs/ioctl.c | 11 ++++++++++- fs/btrfs/relocation.c | 4 +++-

The freeze rwsem is taken by sb_start_intwrite() and dropped during the commit_ or end_transaction(). In the async case, that happens in a worker thread. Tell lockdep the calling thread is releasing ownership of the rwsem and the async thread is picking it up. Josef and I worked out a more complicated solution that made the async commit thread join and potentially get a later transaction, but

Since there is a filesystem state, we should deal with it carefully at mount, umount and remount. - At mount, the FS state should be checked if there is error on these FS. If it does have, btrfsck is recommended. - At umount, the FS state should be saved into disk for consistency. Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com> --- fs/btrfs/disk-io.c | 47

Add a mount option user_subvol_rm_allowed that allows users to delete a (potentially non-empty!) subvol when they would otherwise we allowed to do an rmdir(2). We duplicate the may_delete() checks from the core VFS code to implement identical security checks (minus the directory size check). Signed-off-by: Sage Weil <sage@newdream.net> --- fs/btrfs/ctree.h | 1 + fs/btrfs/ioctl.c |

After running ceph on XFS for some time, I decided to try btrfs again. Performance with the current "for-linux-min" branch and big metadata is much better. The only problem (?) I''m still seeing is a warning that seems to occur from time to time: [87703.784552] ------------[ cut here ]------------ [87703.789759] WARNING: at fs/btrfs/inode.c:2103

Currently the fs sync function (super.c:btrfs_sync_fs()) doesn''t wait for delayed work to finish before returning success to the caller. This change fixes this, ensuring that there''s no data loss if a power failure happens right after fs sync returns success to the caller and before the next commit happens. Steps to reproduce the data loss issue: $ mkfs.btrfs -f /dev/sdb3 $

From: Anand <anand.jain@oracle.com> This patch adds creation-time to the snapshot list display, which would help user to better manage the snapshots when number of snapshots grow substantially. This patch is developed and on top of the send/receive btrfs and btrfs-progs repo at git://github.com/ablock84/linux-btrfs.git (send-v2) git://github.com/ablock84/btrfs-progs.git (send-v2)