Hi, Sorry if this is well known. Suposse I am transferring a large (by now) file, say 1Gbyte with --partial: rsync -e ssh -a --partial 1gbfile server: If process is interrumpted, temporary file is renamed to good filename with size samall than original, say 400Mbyte, ok. If from client I do again the same command, then, in server the 400Mbyte file is COPIED to another temporary file, but this take some time. I from client I press crtl-C when in server temporary file is not already copied, say is 200 Mbyte, temporary file overwrites the 400Mbyte file, losing a lot of data. Why do no symple rename partial file to temporry file, instead of COPYING? (Copying generates obvious additional problems where disk space is short).
On Sun, Dec 28, 2003 at 09:50:37AM +0100, Manuel Mollar wrote:> Hi, > Sorry if this is well known. > Suposse I am transferring a large (by now) file, say 1Gbyte with --partial: > rsync -e ssh -a --partial 1gbfile server: > If process is interrumpted, temporary file is renamed to good filename > with size samall than original, say 400Mbyte, ok. > If from client I do again the same command, then, in server the 400Mbyte > file is COPIED to another temporary file, but this take some time. > I from client I press crtl-C when in server temporary file is not > already copied, say is 200 Mbyte, temporary file overwrites the 400Mbyte > file, losing a lot of data. > Why do no symple rename partial file to temporry file, instead of COPYING? > (Copying generates obvious additional problems where disk space is short).Rsync has absolutely, positively no way to know it is that the destination file is an exact copy of the beginning of the source nor is it able to update-in-place. This has much less to do with --partial than with the rsync algorithm. You may want to read my "how rsync works" document. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt
Tx for your response, but I am not interested on such reading :-) My concussion is rsync must be used with more care than necessary jw schultz wrote:>On Sun, Dec 28, 2003 at 09:50:37AM +0100, Manuel Mollar wrote: > > >>Hi, >>Sorry if this is well known. >>Suposse I am transferring a large (by now) file, say 1Gbyte with --partial: >> rsync -e ssh -a --partial 1gbfile server: >>If process is interrumpted, temporary file is renamed to good filename >>with size samall than original, say 400Mbyte, ok. >>If from client I do again the same command, then, in server the 400Mbyte >>file is COPIED to another temporary file, but this take some time. >>I from client I press crtl-C when in server temporary file is not >>already copied, say is 200 Mbyte, temporary file overwrites the 400Mbyte >>file, losing a lot of data. >>Why do no symple rename partial file to temporry file, instead of COPYING? >>(Copying generates obvious additional problems where disk space is short). >> >> > >Rsync has absolutely, positively no way to know it is >that the destination file is an exact copy of the beginning >of the source nor is it able to update-in-place. > >This has much less to do with --partial than with the rsync >algorithm. You may want to read my "how rsync works" document. > > >
On Sun, Dec 28, 2003 at 09:50:37AM +0100, Manuel Mollar wrote:> If from client I do again the same command, then, in server the 400Mbyte > file is COPIED to another temporary file, but this take some time. > I from client I press crtl-C when in server temporary file is not > already copied, say is 200 Mbyte, temporary file overwrites the 400Mbyte > file, losing a lot of data.Yes, the way the --partial option handles an interruption is pretty crude. An obvious change that could be made would be to count how much data came over the link versus how much came from the local file and choose from this ratio which file to save when interrupted (which would avoid the sub-optimal scenario you cite above). Another future optimization that I've considered would be to delay creating the temporary file until the point we discover that the new file differs from the old. If one or both of the files was at EOF at that point, we could then do an optimized update (append, truncate, no update needed). If the difference occurs at a non-EOF spot, we'd just copy the data from the old file into the temp file and proceed as normal. Such a change would have lots of little pitfalls to get right (such as twiddling permissions during appending to a read-only file), but it might be do-able without getting too complicated. These are some of the things we can look into after we get 2.6.0 out the door. ..wayne..