Shushu Inbar שושו ענבר
2007-Dec-30 10:46 UTC
How to avoid loosing rsync temp/partial files
Hallo, I am using rsync to copy remote files into a local directory. I use the following flags: "--remove-sent-files --partial-dir=/tmp --partial --temp-dir=/tmp --backup --suffix=.bak". I need the partial issue since in the final destination I must have only closed and full files. I need to remove the sent files from the destination since I need to avoid transferring the same file over and over again. My problem is that when I have proper permissions in the partial dir, but no permissions in the destination dir, it means that the rsync deletes the original files, and after failing to copy the files from /tmp to the destination - it deletes the files from there as well... I tried to find solutions such as --delete-after, but in the bottom line nothing helps. Any idea for a good solution inside rsync capabilities ? Regards, Shushu -------------- next part -------------- HTML attachment scrubbed and removed
On Sun, 2007-12-30 at 12:46 +0200, Shushu Inbar ???? ???? wrote:> My problem is that when I have proper permissions in the partial dir, > but no permissions in the destination dir, it means that the rsync > deletes the original files, and after failing to copy the files > from /tmp to the destination - it deletes the files from there as > well...Yes, the premature deletion of the source files ought to be fixed. The trouble is that the receiver is sending MSG_SUCCESS as soon as the data transfer is complete, regardless of the result of the finish_transfer call (which, among other things, moves the temporary file into place). The receiver shouldn't send MSG_SUCCESS until after finish_transfer succeeds. If it fails, I'm not sure what message (if any) should be sent to indicate that the file transfer failed but shouldn't be redone. Matt
On Sun, Dec 30, 2007 at 01:49:10PM -0500, Matt McCutchen wrote:> Yes, the premature deletion of the source files ought to be fixed.This is now fixed in the latest dev version. The original file will be left intact if the rename from the --temp-dir fails, and a copy of the successful transfer will be left in the --partial-dir if that option was specified (and the rename/copy to the partial dir succeeds). The latest "nightly" tar file has this fix. ..wayne..
Hi, How about creating a temporary directory your self with write permissions? Is that possible? If it is so you can copy in the first phase to that place and still keep working with the partial flags and then copy locally to the local folder. In that case you won?t lose the files. What do you say? Shani ________________________________ From: Shushu Inbar ???? ???? [mailto:shushu.i@gmail.com] Sent: Sunday, December 30, 2007 12:46 PM To: rsync@lists.samba.org Cc: Shani Ben - Avraham; Greg Pendler; Ofer Sofer Subject: How to avoid loosing rsync temp/partial files Hallo, I am using rsync to copy remote files into a local directory. I use the following flags: "--remove-sent-files --partial-dir=/tmp --partial --temp-dir=/tmp --backup --suffix=.bak". I need the partial issue since in the final destination I must have only closed and full files. I need to remove the sent files from the destination since I need to avoid transferring the same file over and over again. My problem is that when I have proper permissions in the partial dir, but no permissions in the destination dir, it means that the rsync deletes the original files, and after failing to copy the files from /tmp to the destination - it deletes the files from there as well... I tried to find solutions such as --delete-after, but in the bottom line nothing helps. Any idea for a good solution inside rsync capabilities ? Regards, Shushu -------------- next part -------------- HTML attachment scrubbed and removed
Another possible solution is checking permissions before coping? and if the permissions on local folder are not enough? fails without trying to copy? ________________________________ From: Shushu Inbar ???? ???? [mailto:shushu.i@gmail.com] Sent: Sunday, December 30, 2007 12:46 PM To: rsync@lists.samba.org Cc: Shani Ben - Avraham; Greg Pendler; Ofer Sofer Subject: How to avoid loosing rsync temp/partial files Hallo, I am using rsync to copy remote files into a local directory. I use the following flags: "--remove-sent-files --partial-dir=/tmp --partial --temp-dir=/tmp --backup --suffix=.bak". I need the partial issue since in the final destination I must have only closed and full files. I need to remove the sent files from the destination since I need to avoid transferring the same file over and over again. My problem is that when I have proper permissions in the partial dir, but no permissions in the destination dir, it means that the rsync deletes the original files, and after failing to copy the files from /tmp to the destination - it deletes the files from there as well... I tried to find solutions such as --delete-after, but in the bottom line nothing helps. Any idea for a good solution inside rsync capabilities ? Regards, Shushu -------------- next part -------------- HTML attachment scrubbed and removed