Greetings,
when the daily system backup with rsync is performed, thousands of files (e.g.
from /usr directory),
which are not modified are uselessly transferrred to the backup storage every
day.
The rsync command is the following:
rsync --update -DHAErlptgo --relative --atimes --delete-during ${SRCPATH}
${BACKUPDIR}
Quick investigation shown, that these files are hard links, and the command
option related to this
behaviour is -H: without it the unmodified files are not copied. Seemingly this
situation is described
in the rsync manual:
"If incremental recursion is active (see --recursive), rsync may transfer
a missing
hard-linked file before it finds that another link for that contents exists
else‐
where in the hierarchy.... One way to avoid this inefficiency is to
disable
incremental recursion using the --no-inc-recursive option."
Adding --no-inc-recursive to the aforementioned command line doesn't help
(are --recursive and
--no-inc-recursive supposed to be used in one option set?). And I use
--recursive to copy entire
directory tree, including subdirectories. Could someone advise please how one
can prevent copying
unmodified hard-linked files?
Thanks,
A. Berger.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 What does --itemize-changes say about these files? Also, what is --atimes? Is this the Apple modified rsync for OSX? On 03/28/2016 07:51 AM, Albert Berger wrote:> Greetings, > > when the daily system backup with rsync is performed, thousands of > files (e.g. from /usr directory), which are not modified are > uselessly transferrred to the backup storage every day. The rsync > command is the following: > > rsync --update -DHAErlptgo --relative --atimes --delete-during > ${SRCPATH} ${BACKUPDIR} > > Quick investigation shown, that these files are hard links, and the > command option related to this behaviour is -H: without it the > unmodified files are not copied. Seemingly this situation is > described in the rsync manual: > > "If incremental recursion is active (see --recursive), rsync may > transfer a missing hard-linked file before it finds that another > link for that contents exists else‐ where in the hierarchy.... > One way to avoid this inefficiency is to disable incremental > recursion using the --no-inc-recursive option." > > Adding --no-inc-recursive to the aforementioned command line > doesn't help (are --recursive and --no-inc-recursive supposed to be > used in one option set?). And I use --recursive to copy entire > directory tree, including subdirectories. Could someone advise > please how one can prevent copying unmodified hard-linked files? > > Thanks, A. Berger. >- -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlb5H1sACgkQVKC1jlbQAQe0LwCg0yhgY81WywKO4NYioxCfbF3J qVcAn2P9hJiW9pXwMZutAmuOjIvdvvNZ =/Enu -----END PGP SIGNATURE-----
On Mon, Mar 28, 2016 at 08:11:07AM -0400, Kevin Korb wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > What does --itemize-changes say about these files? > > Also, what is --atimes? Is this the Apple modified rsync for OSX? > > On 03/28/2016 07:51 AM, Albert Berger wrote: > > Greetings, > > > > when the daily system backup with rsync is performed, thousands of > > files (e.g. from /usr directory), which are not modified are > > uselessly transferrred to the backup storage every day. The rsync > > command is the following: > > > > rsync --update -DHAErlptgo --relative --atimes --delete-during > > ${SRCPATH} ${BACKUPDIR} > > > > Quick investigation shown, that these files are hard links, and the > > command option related to this behaviour is -H: without it the > > unmodified files are not copied. Seemingly this situation is > > described in the rsync manual: > > > > "If incremental recursion is active (see --recursive), rsync may > > transfer a missing hard-linked file before it finds that another > > link for that contents exists else‐ where in the hierarchy.... > > One way to avoid this inefficiency is to disable incremental > > recursion using the --no-inc-recursive option." > > > > Adding --no-inc-recursive to the aforementioned command line > > doesn't help (are --recursive and --no-inc-recursive supposed to be > > used in one option set?). And I use --recursive to copy entire > > directory tree, including subdirectories. Could someone advise > > please how one can prevent copying unmodified hard-linked files? > > > > Thanks, A. Berger. > >As an example of copying hard-linked files, the /usr/bin contains the following two hard-linked sets for two inodes: inode 1: ld ld.bfd inode 2: g++ i686-pc-linux-gnu-c++ i686-pc-linux-gnu-g++ c++ Below is the --itemized-changes output for those two sets which is produced when /usr/bin doesn't have any modifications and is synced with the aforementioned command to its backup copy: hf......u.. /usr/bin/ld.bfd hf......u.. /usr/bin/i686-pc-linux-gnu-c++ hf......u.. /usr/bin/i686-pc-linux-gnu-g++ hf......u.. /usr/bin/g++ As can bee seen, one file in each set is not listed at all, and all other hard-linked files are marked for updating. Is this behaviour normal? --atimes is an option for preserving access times. This option is added by patching rsync source with a patch from rsync-patches-3.1.1.tar.gz file from rsync download page. The system is ArchLinux, kernel 4.4.5. Regards, A.Berger.