Fragmentation on XFS

Jamie Lokier
Mon, 25 Feb 2008 21:46:57 -0000

Rob Bosch wrote:
> > Was that simply due to writing too-small block to NTFS?  In other
> > words, would increasing the size of write() calls have fixed it
> > instead, without leaving allocated but unused disk space in the case
> > of a user-abort with --partial, --partial-dir or --inplace?
>
> It could have been a function of the block size but I don't think so.
I
> never tested the strategies you list.  Under cygwin the posix_fallocate
> function is extremely efficient in that it immediately allocates the files
> and does no writing, yet still provides a single-extent file if it can be
> provided (just like fallocate if supported in the kernel).  Given that
> solved the problem I didn't pursue any other alternatives.

I'm thinking that if rsync is aborted after it calls posix_fallocate,
you might have some large amount of disk space used, but not indicated
in the file size.  That seems like a bad state to leave a filesystem
in, because it's mostly invisible.

That means the prealloc option should be off by default (on all systems).

I'm thinking if large writes would fix the NTFS problem, they wouldn't
leave the filesystem like that following an abort, so that could be
safely turned on by default, and everyone would benefit from improved
NTFS performance.

Another possibility is to incrementally preallocate large chunks,
e.g. 1MiB at a time while writing.  That wouldn't leave huge
preallocated space, and should improve NTFS performance enough, so
could be turned on by default perhaps.

-- Jamie