Again, --partial only means don't delete the incomplete file if rsync is
aborted. Normally rsync will delete the incomplete file so you don't
have bogus files laying around.
When you rsync to or from a network mount to rsync that is a local copy.
To use rsync over the network either your source or your target would
be hostname:/path (for rsync over ssh) or hostname::module (for an
rsyncd server).
With a local copy rsync forces --whole-file because that is a simple
read the file from one place and write it to the other place. If you
were to override that rsync would have to read both versions of the file
(one of them over the network) compare chunks of the files, the chunks
that are correct would then be written back over the network to a new
file while the chunks that aren't correct would be written over the
network to complete that new file. This would be significantly more
network transferring.
Adding --checksum would only make things much worse. Rsync would
checksum every file on both ends (reading them over the network) before
doing exactly what it would have done without --checksum.
Essentially, rsync's differential transfer algorithm is designed to
minimize network IO at the expense of CPU usage and local disk IO. But
in order to do that you need rsync running on both ends of a network
connection communicating to each other. That way they can each do
hashing/checksumming on local files quickly and negotiate the smallest
amount of data to send over the network.
On 06/24/2016 03:22 PM, McDowell, Blake wrote:> Hi Kevin,
>
> I'm not a systems manager so my apologies if I'm a little lost
here. I'm an audiovisual conservator/archivist and I use rsync for
transferring files, a lot.
>
> Yes, I connect to the server and then it shows up as a disk on my desktop
and I run rsync between the external drive mounted on my computer and the now
mounted server. So, this would be a local copy? And, therefore, there is no way
to invoke --partial because --whole-file is forced?
>
> I don't quite understand what you mean by "rsync isn't doing
the networking", but if it is not doing this then is there no checksumming
occurring during the transfer? Unless, of course, -c.
>
> Thanks!
>
> -----Original Message-----
> From: rsync [mailto:rsync-bounces at lists.samba.org] On Behalf Of Kevin
Korb
> Sent: Friday, June 24, 2016 3:07 PM
> To: rsync at lists.samba.org
> Subject: Re: --partial not working?
>
> If you are doing a local only copy (rsync isn't networking) then
--whole-file is forced. There is no benefit of reading and checksumming files
to reduce network transfer when there is no network transfer.
>
> You said you were moving data to a remote server so I assume you are using
a network mount of some kind to make it a local copy instead of letting rsync do
the networking.
>
> Anyways, if rsync isn't doing the networking then forcing it to do a
delta transfer would only make it take even longer.
>
> On 06/24/2016 02:59 PM, McDowell, Blake wrote:
>> Hi Kevin,
>>
>> I haven't specified --whole-file. After entering an rsync command
the terminal always reads "delta-transmission disabled for local transfer
or --whole-file" but I assume that is just a standard phrase that always
appears.
>>
>> So, if I am running partial (-P) and not using --whole-file or
disabling the delta-transmission, why would an incomplete file be deleted and
the transfer start anew after an interruption?
>>
>> Blake
>>
>> -----Original Message-----
>> From: rsync [mailto:rsync-bounces at lists.samba.org] On Behalf Of
Kevin
>> Korb
>> Sent: Friday, June 24, 2016 12:03 PM
>> To: rsync at lists.samba.org
>> Subject: Re: --partial not working?
>>
>> Partial means don't delete the incomplete file. The file is being
transferred because it doesn't match. But rsync does delta-transfers so it
won't actually transfer the parts that match. It will just verify that they
do match.
>>
>> Unless of course --whole-file.
>>
>> On 06/24/2016 11:47 AM, McDowell, Blake wrote:
>>> Hello,
>>>
>>>
>>>
>>> I’m running rsync -avPhi to move large video files to a remote
server.
>>> Often we have to stop a transfer midway through to push something
>>> else to the server. My hope was that the -P flag would invoke
>>> --partial and the transfer would pick-up where it left off. This
does
>>> not seem to be happening. Rather, the transfer starts over from the
>>> beginning, erasing the partially transferred file.
>>>
>>>
>>> When I redo the transfer my I see this:
>>>
>>>> f.st..g....
afc2010039_crhp0026_mv06_concat_Williams_Cecil_J.mov
>>>
>>>
>>> I understand what all of that means, but I don't know why it
doesn't
>>> resume the partial transfer.
>>>
>>>
>>> Do I need to change my rsync command to allow it to resume a
transfer
>>> of a partially transferred file?
>>>
>>>
>>> Thanks,
>>>
>>> Blake
>>>
>>>
>>>
>>
>> --
>>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>> Kevin Korb Phone: (407) 252-6853
>> Systems Administrator Internet:
>> FutureQuest, Inc. Kevin at FutureQuest.net (work)
>> Orlando, Florida kmk at sanitarium.net (personal)
>> Web page: http://www.sanitarium.net/
>> PGP public key available on web site.
>>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._
>> .,
>>
>
> --
>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
> Kevin Korb Phone: (407) 252-6853
> Systems Administrator Internet:
> FutureQuest, Inc. Kevin at FutureQuest.net (work)
> Orlando, Florida kmk at sanitarium.net (personal)
> Web page: http://www.sanitarium.net/
> PGP public key available on web site.
>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>
>
--
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. Kevin at FutureQuest.net (work)
Orlando, Florida kmk at sanitarium.net (personal)
Web page: http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20160624/6b6b0b0e/signature.sig>