I believe that the changes to support --preallocate and --sparse together have broken --preallocate by itself (commit f3873b3d88b61167b106e7b9227a20147f8f6197) The previous behavior of --preallocate was to do just that: reserve blocks in the filesystem WITHOUT setting the size of the file to the final length. The reported filesize would change as the preallocated blocks were actually written. Unfortunately, the change causes a whole bunch of problems with our automated backup, and in particular with --append-verify or --inplace. The normal sequence we use in our large-file backup system is to combine preallocate with append-verify. When a partially written file is restarted, it no longer goes through proper verification (with append-verify). Worse, if you do only --inplace, then the entire file goes through verification rather than only the previously-transfered portion. I believe this is due to the filesize misrepresenting how much data was actually written. in syscall.c in appears maybe there is a logic error (that should be bitwise-OR, not logical-OR): OFF_T do_fallocate(int fd, OFF_T offset, OFF_T length) { int opts = inplace || preallocate_files ? 0 : DO_FALLOC_OPTIONS; -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20190114/ea6f33b6/attachment.html>
On Mon, Jan 14, 2019 at 6:11 PM TR Reardon via rsync <rsync at lists.samba.org> wrote:> I believe that the changes to support --preallocate and --sparse together > have broken --preallocate by itself (commit > f3873b3d88b61167b106e7b9227a20147f8f6197) >Indeed, those "opts" values were reversed, and thus fallocate wasn't told to keep the size 0. I've checked in a fix. ..wayne.. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20190115/47a08c07/attachment.html>
Just for clarity: the intention is still a logical-OR rather than bitwise-OR? If it's a new file on receiver, 'inplace' would still be set and this KEEP_SIZE would be lost? On Tue, Jan 15, 2019 at 12:06 PM Wayne Davison <wayne at opencoder.net> wrote:> > On Mon, Jan 14, 2019 at 6:11 PM TR Reardon via rsync <rsync at lists.samba.org> wrote: >> >> I believe that the changes to support --preallocate and --sparse together have broken --preallocate by itself (commit f3873b3d88b61167b106e7b9227a20147f8f6197) > > > Indeed, those "opts" values were reversed, and thus fallocate wasn't told to keep the size 0. I've checked in a fix. > > ..wayne..
Maybe Matching Threads
- file contents cause rsync to fail (with certains args and dir structure)
- [Bug 11588] New: missing option: preallocate for all files except for sparse
- Allow "--in-place" as an alternative option name for "--inplace"
- samba 3.6 client signing
- [Bug 13320] New: file contents cause rsync to fail (with certains args and dir structure)