Kernel 3.1-rc8 btrfs-progs-0.19 mount options: noatime,autodefrag (space_cache is enabled) There are snapshots present on the filesystem. When I do a btrfs fi defrag on a file, the file becomes much more fragmented. The end result can be a file with 20k times more fragments than before. Initially I thought the extents were just smaller but were next to each other, so I checked with both ''filefrag'' as well as ''filefrag -v''. Both reported the same number. I don''t know if this has anything to do with having snapshots at all because files that haven''t been snapshotted yet are affected by this. Disk space isn''t an issue since my 1TB disk isn''t even halfway filled, and moving the file around actually gives me fairly contiguous files. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Chester wrote:> Kernel 3.1-rc8 > btrfs-progs-0.19 > mount options: noatime,autodefrag (space_cache is enabled) > There are snapshots present on the filesystem. > > When I do a btrfs fi defrag on a file, the file becomes much more > fragmented. The end result can be a file with 20k times more fragments > than before. Initially I thought the extents were just smaller but > were next to each other, so I checked with both ''filefrag'' as well as > ''filefrag -v''. Both reported the same number. I don''t know if this has > anything to do with having snapshots at all because files that haven''t > been snapshotted yet are affected by this. Disk space isn''t an issue > since my 1TB disk isn''t even halfway filled, and moving the file > around actually gives me fairly contiguous files.Was any process doing heavy I/O work while you were defragging the file? Could you try to remount the fs without autodefrag, and defrag the file again? Firstly let''s see if autodefrag makes things worse. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Oct 11, 2011 at 1:19 AM, Li Zefan <lizf@cn.fujitsu.com> wrote:> Was any process doing heavy I/O work while you were defragging the file? > > Could you try to remount the fs without autodefrag, and defrag > the file again? Firstly let''s see if autodefrag makes things worse. >As far as I can tell, autodefrag seems to be making it worse. I turned off autodefrag but btrfs fi defrag is still having some trouble defragmenting highly fragmented files. For example, when I defragment a ~400mb file with 3 extents, it does a nice job and turns it into 1 extent. However, for more fragmented files (of the same size) say with 1k-2k extents, the defrag utility has a hard time. It either reduces the number of extents by a very small amount or it doesn''t change at all. I decided to run btrfs fi defrag on the same file again just to make sure nothing else can be done, but after running it again, the number of extents in that file actually DOUBLED! But this isn''t as bad as when autodefrag is on, where extents can increase by 20x tl;dr: with autodefrag off, btrfs fi defrag isn''t nearly as counter-productive as it is with autodefrag on. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
05:15, Chester wrote:> Okay, this pretty much confirms that autodefrag makes defrag totally > useless.. The above email was with autodefrag turned off.. > > With autodefrag on: > 339 MB file > initially 4 extents > > btrfs fi defrag filename > filefrag filename > > about 900 extents > > I did btrfs defrag two more times and ended up with 2048 extents > > I attached the output of filefrag -v, which shows a pretty peculiar pattern.This confirmed my speculation. I''ve fixed this bug a month ago, but the patch hasn''t hitted mainline. You can try it out: http://marc.info/?l=linux-btrfs&m=131495014823121&w=2 With this bug fixed, I think autodefrag won''t be problematic. -- Li Zefan -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Oct 11, 2011 at 9:37 PM, Li Zefan <lizf@cn.fujitsu.com> wrote:> > > This confirmed my speculation. I''ve fixed this bug a month ago, but > the patch hasn''t hitted mainline. > > You can try it out: > > http://marc.info/?l=linux-btrfs&m=131495014823121&w=2 > > With this bug fixed, I think autodefrag won''t be problematic. > > -- > Li ZefanYup. That patch fixes it for me. Thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Excerpts from Chester''s message of 2011-10-12 00:14:21 -0400:> On Tue, Oct 11, 2011 at 9:37 PM, Li Zefan <lizf@cn.fujitsu.com> wrote: > > > > > > This confirmed my speculation. I''ve fixed this bug a month ago, but > > the patch hasn''t hitted mainline. > > > > You can try it out: > > > > http://marc.info/?l=linux-btrfs&m=131495014823121&w=2 > > > > With this bug fixed, I think autodefrag won''t be problematic. > > > > -- > > Li Zefan > > Yup. That patch fixes it for me. Thanks.Thanks guys, This patch is queued up now for the next release (3.2) -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html