Hi Kevin, I haven't specified --whole-file. After entering an rsync command the terminal always reads "delta-transmission disabled for local transfer or --whole-file" but I assume that is just a standard phrase that always appears. So, if I am running partial (-P) and not using --whole-file or disabling the delta-transmission, why would an incomplete file be deleted and the transfer start anew after an interruption? Blake -----Original Message----- From: rsync [mailto:rsync-bounces at lists.samba.org] On Behalf Of Kevin Korb Sent: Friday, June 24, 2016 12:03 PM To: rsync at lists.samba.org Subject: Re: --partial not working? Partial means don't delete the incomplete file. The file is being transferred because it doesn't match. But rsync does delta-transfers so it won't actually transfer the parts that match. It will just verify that they do match. Unless of course --whole-file. On 06/24/2016 11:47 AM, McDowell, Blake wrote:> Hello, > > > > I’m running rsync -avPhi to move large video files to a remote server. > Often we have to stop a transfer midway through to push something > else to the server. My hope was that the -P flag would invoke > --partial and the transfer would pick-up where it left off. This does > not seem to be happening. Rather, the transfer starts over from the > beginning, erasing the partially transferred file. > > > When I redo the transfer my I see this: > >>f.st..g.... afc2010039_crhp0026_mv06_concat_Williams_Cecil_J.mov > > > I understand what all of that means, but I don't know why it doesn't > resume the partial transfer. > > > Do I need to change my rsync command to allow it to resume a transfer > of a partially transferred file? > > > Thanks, > > Blake > > >-- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
If you are doing a local only copy (rsync isn't networking) then --whole-file is forced. There is no benefit of reading and checksumming files to reduce network transfer when there is no network transfer. You said you were moving data to a remote server so I assume you are using a network mount of some kind to make it a local copy instead of letting rsync do the networking. Anyways, if rsync isn't doing the networking then forcing it to do a delta transfer would only make it take even longer. On 06/24/2016 02:59 PM, McDowell, Blake wrote:> Hi Kevin, > > I haven't specified --whole-file. After entering an rsync command the terminal always reads "delta-transmission disabled for local transfer or --whole-file" but I assume that is just a standard phrase that always appears. > > So, if I am running partial (-P) and not using --whole-file or disabling the delta-transmission, why would an incomplete file be deleted and the transfer start anew after an interruption? > > Blake > > -----Original Message----- > From: rsync [mailto:rsync-bounces at lists.samba.org] On Behalf Of Kevin Korb > Sent: Friday, June 24, 2016 12:03 PM > To: rsync at lists.samba.org > Subject: Re: --partial not working? > > Partial means don't delete the incomplete file. The file is being transferred because it doesn't match. But rsync does delta-transfers so it won't actually transfer the parts that match. It will just verify that they do match. > > Unless of course --whole-file. > > On 06/24/2016 11:47 AM, McDowell, Blake wrote: >> Hello, >> >> >> >> I’m running rsync -avPhi to move large video files to a remote server. >> Often we have to stop a transfer midway through to push something >> else to the server. My hope was that the -P flag would invoke >> --partial and the transfer would pick-up where it left off. This does >> not seem to be happening. Rather, the transfer starts over from the >> beginning, erasing the partially transferred file. >> >> >> When I redo the transfer my I see this: >> >>> f.st..g.... afc2010039_crhp0026_mv06_concat_Williams_Cecil_J.mov >> >> >> I understand what all of that means, but I don't know why it doesn't >> resume the partial transfer. >> >> >> Do I need to change my rsync command to allow it to resume a transfer >> of a partially transferred file? >> >> >> Thanks, >> >> Blake >> >> >> > > -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. Kevin at FutureQuest.net (work) > Orlando, Florida kmk at sanitarium.net (personal) > Web page: http://www.sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., >-- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/rsync/attachments/20160624/7e3faf43/signature.sig>
Hi Kevin, I'm not a systems manager so my apologies if I'm a little lost here. I'm an audiovisual conservator/archivist and I use rsync for transferring files, a lot. Yes, I connect to the server and then it shows up as a disk on my desktop and I run rsync between the external drive mounted on my computer and the now mounted server. So, this would be a local copy? And, therefore, there is no way to invoke --partial because --whole-file is forced? I don't quite understand what you mean by "rsync isn't doing the networking", but if it is not doing this then is there no checksumming occurring during the transfer? Unless, of course, -c. Thanks! -----Original Message----- From: rsync [mailto:rsync-bounces at lists.samba.org] On Behalf Of Kevin Korb Sent: Friday, June 24, 2016 3:07 PM To: rsync at lists.samba.org Subject: Re: --partial not working? If you are doing a local only copy (rsync isn't networking) then --whole-file is forced. There is no benefit of reading and checksumming files to reduce network transfer when there is no network transfer. You said you were moving data to a remote server so I assume you are using a network mount of some kind to make it a local copy instead of letting rsync do the networking. Anyways, if rsync isn't doing the networking then forcing it to do a delta transfer would only make it take even longer. On 06/24/2016 02:59 PM, McDowell, Blake wrote:> Hi Kevin, > > I haven't specified --whole-file. After entering an rsync command the terminal always reads "delta-transmission disabled for local transfer or --whole-file" but I assume that is just a standard phrase that always appears. > > So, if I am running partial (-P) and not using --whole-file or disabling the delta-transmission, why would an incomplete file be deleted and the transfer start anew after an interruption? > > Blake > > -----Original Message----- > From: rsync [mailto:rsync-bounces at lists.samba.org] On Behalf Of Kevin > Korb > Sent: Friday, June 24, 2016 12:03 PM > To: rsync at lists.samba.org > Subject: Re: --partial not working? > > Partial means don't delete the incomplete file. The file is being transferred because it doesn't match. But rsync does delta-transfers so it won't actually transfer the parts that match. It will just verify that they do match. > > Unless of course --whole-file. > > On 06/24/2016 11:47 AM, McDowell, Blake wrote: >> Hello, >> >> >> >> I’m running rsync -avPhi to move large video files to a remote server. >> Often we have to stop a transfer midway through to push something >> else to the server. My hope was that the -P flag would invoke >> --partial and the transfer would pick-up where it left off. This does >> not seem to be happening. Rather, the transfer starts over from the >> beginning, erasing the partially transferred file. >> >> >> When I redo the transfer my I see this: >> >>> f.st..g.... afc2010039_crhp0026_mv06_concat_Williams_Cecil_J.mov >> >> >> I understand what all of that means, but I don't know why it doesn't >> resume the partial transfer. >> >> >> Do I need to change my rsync command to allow it to resume a transfer >> of a partially transferred file? >> >> >> Thanks, >> >> Blake >> >> >> > > -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. Kevin at FutureQuest.net (work) > Orlando, Florida kmk at sanitarium.net (personal) > Web page: http://www.sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._ > ., >-- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,