--no-whole-file would only make it even worse. It would have to read the remote file over the network in order to do the diff then it would write the whole file over the network anyway (--inplace would help a little). Local copies force --whole-file for a good reason. On 03/14/2018 10:05 AM, Ben RUBSON via rsync wrote:> On 14 Mar 2018, Lentes, Bernd via rsync wrote: > >> ----- On Mar 14, 2018, at 2:19 PM, Ben RUBSON ben.rubson at gmail.com wrote: >> >>> On 14 Mar 2018, Lentes, Bernd via rsync wrote: >>> >>>> I would now expect a rsync from the snap would transfer just some megay >>>> bytes to the file from the day before. >>>> But it doesn't: >>>> >>>> ha-idg-1:/cluster/guests/servers_alive # time rsync -av --stats >>>> sa.raw.snap /mnt/idg-2/SysAdmin_AG_Wurst/backup/cluster/test >>> >>> Hi Bernd, >>> >>> When doing rsync locally, diff alg is not involved, this is why file is >>> fully transferred. >>> >>> Ben >> >> Hi Ben, >> >> also when the target is a cifs share, it's still considered as local ? > > Yes as it's mounted locally. > >> Is there something i can do to get the diff algorithm used ? > > Perhaps --no-whole-file would do the trick ? > >> Copying via ssh to the cifs server is unfortunately not possible. >> >> >> Bernd >-- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 224 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/rsync/attachments/20180314/fbedd8c2/signature.sig>
Yes you're right, rsync would update only a few parts of the file, but network usage would be even worst. The only solution would finally be to have rsync on the target system. Ben On 14 Mar, Kevin Korb via rsync wrote:> --no-whole-file would only make it even worse. It would have to read > the remote file over the network in order to do the diff then it would > write the whole file over the network anyway (--inplace would help a > little). Local copies force --whole-file for a good reason. > > On 03/14/2018 10:05 AM, Ben RUBSON via rsync wrote: >> On 14 Mar 2018, Lentes, Bernd via rsync wrote: >> >>> ----- On Mar 14, 2018, at 2:19 PM, Ben RUBSON ben.rubson at gmail.com wrote: >>> >>>> On 14 Mar 2018, Lentes, Bernd via rsync wrote: >>>> >>>>> I would now expect a rsync from the snap would transfer just some megay >>>>> bytes to the file from the day before. >>>>> But it doesn't: >>>>> >>>>> ha-idg-1:/cluster/guests/servers_alive # time rsync -av --stats >>>>> sa.raw.snap /mnt/idg-2/SysAdmin_AG_Wurst/backup/cluster/test >>>> >>>> Hi Bernd, >>>> >>>> When doing rsync locally, diff alg is not involved, this is why file is >>>> fully transferred. >>>> >>>> Ben >>> >>> Hi Ben, >>> >>> also when the target is a cifs share, it's still considered as local ? >> >> Yes as it's mounted locally. >> >>> Is there something i can do to get the diff algorithm used ? >> >> Perhaps --no-whole-file would do the trick ? >> >>> Copying via ssh to the cifs server is unfortunately not possible. >>> >>> >>> Bernd
On 2018-03-14 10:07 AM, Kevin Korb via rsync wrote:> --no-whole-file would only make it even worse. It would have to read > the remote file over the network in order to do the diff then it would > write the whole file over the network anyway (--inplace would help a > little). Local copies force --whole-file for a good reason. > >The one possible exception is if the cifs share is mounted over an asymmetrical Internet connection.. (ie, the download speed is 10X faster than the upload.) -------------- next part -------------- A non-text attachment was scrubbed... Name: remi.vcf Type: text/x-vcard Size: 193 bytes Desc: not available URL: <http://lists.samba.org/pipermail/rsync/attachments/20180314/ac6e2af3/remi.vcf> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/rsync/attachments/20180314/ac6e2af3/signature.sig>
It would still be downloading the whole file only to write out a new one. Rsync writes out a whole new file because it assumes it is doing so locally. The source of that new file can be a combination of parts of the existing file and parts of the remote file but it still writes the whole thing. --inplace helps with that somewhat but you will probably still write all of the file after the first difference. On 03/14/2018 10:12 AM, Remi Gauvin via rsync wrote:> On 2018-03-14 10:07 AM, Kevin Korb via rsync wrote: >> --no-whole-file would only make it even worse. It would have to read >> the remote file over the network in order to do the diff then it would >> write the whole file over the network anyway (--inplace would help a >> little). Local copies force --whole-file for a good reason. >> >> > > The one possible exception is if the cifs share is mounted over an > asymmetrical Internet connection.. (ie, the download speed is 10X faster > than the upload.) > > >-- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 224 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/rsync/attachments/20180314/605f8a77/signature.sig>