Pádraig Brady
2011-Feb-13 23:53 UTC
Re: [PATCH] copy: adjust fiemap handling of sparse files
Unfortunately, after checking BTRFS I see that fiemap behaves differently to EXT4. IMHO the EXT4 operation seems correct, and gives full info about the structure of a file, which cp for example can use to efficiently and accurately reproduce the structure at the destination. On EXT4 (on kernel-2.6.35.11-83.fc14.i686) there are no extents returned for holes in a file. However on btrfs it does return an extent for holes? So with btrfs there is no way to know an extent is allocated but unwritten (zero), so one must read and write all the data, rather than just fallocating the space in the destination. One can also see this with the following on btrfs: $ fallocate -l 100000000 unwritten $ truncate -s 100000000 sparse $ dd count=1000 bs=100000 if=/dev/zero of=zero $ filefrag -vs * Filesystem type is: 9123683e File size of sparse is 100000000 (24415 blocks, blocksize 4096) ext logical physical expected length flags 0 0 0 24415 unwritten,eof sparse: 1 extent found File size of unwritten is 100000000 (24415 blocks, blocksize 4096) ext logical physical expected length flags 0 0 68160 12207 1 12207 92560 80366 12208 eof unwritten: 2 extents found File size of zeros is 100000000 (24415 blocks, blocksize 4096) ext logical physical expected length flags 0 0 19360 20678 1 20678 43760 40037 3737 eof zeros: 2 extents found So is this expected? Has this already been changed to match ext4? For my own reference, I can probably skip performance tests on (older) btrfs by checking `filefrag` output. Also in `cp`, if we see an "unwritten extent" we should probably create a hole rather than an empty allocation by default. It''s better to decrease file allocation than increase it. cheers, Pádraig. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Jim Meyering
2011-Feb-14 06:05 UTC
Re: [PATCH] copy: adjust fiemap handling of sparse files
Pádraig Brady wrote:> Unfortunately, after checking BTRFS I see that fiemap > behaves differently to EXT4. IMHO the EXT4 operation > seems correct, and gives full info about the structure > of a file, which cp for example can use to efficiently > and accurately reproduce the structure at the destination. > > On EXT4 (on kernel-2.6.35.11-83.fc14.i686) there are no extents > returned for holes in a file. > > However on btrfs it does return an extent for holes? > So with btrfs there is no way to know an extent > is allocated but unwritten (zero), so one must > read and write all the data, rather than just > fallocating the space in the destination. > > One can also see this with the following on btrfs: > > $ fallocate -l 100000000 unwritten > $ truncate -s 100000000 sparse > $ dd count=1000 bs=100000 if=/dev/zero of=zero > > $ filefrag -vs * > Filesystem type is: 9123683e > File size of sparse is 100000000 (24415 blocks, blocksize 4096) > ext logical physical expected length flags > 0 0 0 24415 unwritten,eof > sparse: 1 extent found > File size of unwritten is 100000000 (24415 blocks, blocksize 4096) > ext logical physical expected length flags > 0 0 68160 12207 > 1 12207 92560 80366 12208 eof > unwritten: 2 extents found > File size of zeros is 100000000 (24415 blocks, blocksize 4096) > ext logical physical expected length flags > 0 0 19360 20678 > 1 20678 43760 40037 3737 eof > zeros: 2 extents found > > So is this expected?I heard it''s a bug in F14''s btrfs. To test btrfs properly, I had to use rawhide/F15.> Has this already been changed to match ext4?Yes, with a new-enough kernel.> For my own reference, I can probably skip performance > tests on (older) btrfs by checking `filefrag` output. > Also in `cp`, if we see an "unwritten extent" we should > probably create a hole rather than an empty allocation > by default. It''s better to decrease file allocation > than increase it.Makes sense. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html