PS. Forgot version info: Linux# rsync -v rsync version 2.6.9 protocol version 29 OSX$ rsync -v rsync version 2.6.3 protocol version 28 David Liontooth wrote:> I'm trying to rsync files to a set of symlinked directories on an XServe > running OS X. > > The directories look like this: > > tna@csx:~/2/2006$ l > total 80 > lrwxr-xr-x 1 tna tna 39 Feb 24 13:18 2006-01 -> > /Volumes/CSArchiveArray001/2006/2006-01 > lrwxr-xr-x 1 tna tna 39 Feb 24 13:18 2006-02 -> > /Volumes/CSArchiveArray001/2006/2006-02 > > When I run rsync from Linux in --dry-run mode, the results are correct: > > $ rsync -arvn --ignore-existing --exclude-from=/db/nosyncc /db/2006/ -e > ssh x:~/2/2006/ > building file list ... done > 2006-08/2006-08-12/2006-08-12_0430_KNBC_Channel_4_News.txt > 2006-08/2006-08-12/2006-08-12_0430_KTTV_A_Current_Affair.txt > 2006-08/2006-08-17/2006-08-17_0500_KABC_Eyewitness_News.txt > 2006-08/2006-08-17/2006-08-17_0500_KTTV_Morning_News_at_5.txt > 2006-08/2006-08-17/2006-08-17_0600_KTTV_Morning_News_at_6.txt > 2006-10/2006-10-29/2006-10-29_0800_KTTV-FOX_News_Sunday_with_Chris_Wallace.txt > > sent 313109 bytes received 40 bytes 208766.00 bytes/sec > total size is 445239817 speedup is 1421.81 > > When I do the actual run, the symlinks are overwritten and new > directories are created; details below. > Can I prevent this behavior and tell rsync to respect the symlinks, as > it does in --dry-run mode? > > Dave > > $ rsync -arv --ignore-existing --exclude-from=/db/nosyncc /db/2006/ -e > ssh x:~/2/2006/ > building file list ... done > ./ > 2006-01/ > 2006-01/2006-01-01/ > 2006-01/2006-01-01/2006-01-01_0430_KTTV-FOX_A_Current_Affair.txt > 2006-01/2006-01-01/2006-01-01_0500_KABC_Eyewitness_News.txt > 2006-01/2006-01-01/2006-01-01_0500_KTTV-FOX_Morning_News_at_5.txt > 2006-01/2006-01-01/2006-01-01_0800_KNBC_Meet_the_Press.txt > 2006-01/2006-01-01/2006-01-01_0800_KTTV-FOX_News_Sunday_with_Chris_Wallace.txt > 2006-01/2006-01-01/2006-01-01_1730_KABC_World_News_Tonight.txt > 2006-01/2006-01-01/2006-01-01_1900_KCBS_60_Minutes.txt > 2006-01/2006-01-01/2006-01-01_2200_KTTV-FOX_Ten_OClock_News.txt > 2006-01/2006-01-01/2006-01-01_2300_KABC_Eyewitness_News.txt > 2006-01/2006-01-02/ > 2006-01/2006-01-02/2006-01-02_0430_KNBC_Channel_4_News.txt > 2006-01/2006-01-02/2006-01-02_0430_KTTV-FOX_A_Current_Affair.txt > 2006-01/2006-01-02/2006-01-02_0500_KABC_Eyewitness_News.txt > 2006-01/2006-01-02/2006-01-02_0500_KTTV-FOX_Morning_News_at_5.txt > 2006-01/2006-01-02/2006-01-02_0530_KCET_BBC_World_News.txt > 2006-01/2006-01-02/2006-01-02_0600_KTTV-FOX_Morning_News_at_6.txt > 2006-01/2006-01-02/2006-01-02_0700_KTTV-FOX_Morning_News_at_7.txt > 2006-01/2006-01-02/2006-01-02_1700_KNBC_Channel_4_News_at_5.txt > 2006-01/2006-01-02/2006-01-02_1830_KABC_World_News_Tonight.txt > 2006-01/2006-01-02/2006-01-02_1900_KCET_Newshour_with_Jim_Lehrer.txt > 2006-01/2006-01-02/2006-01-02_2200_KTTV-FOX_Ten_OClock_News.txt > 2006-01/2006-01-02/2006-01-02_2300_KABC_Eyewitness_News.txt > 2006-01/2006-01-02/2006-01-02_2330_KABC_Nightline.txt > > > > >
I'm trying to rsync files to a set of symlinked directories on an XServe running OS X. The directories look like this: tna@csx:~/2/2006$ l total 80 lrwxr-xr-x 1 tna tna 39 Feb 24 13:18 2006-01 -> /Volumes/CSArchiveArray001/2006/2006-01 lrwxr-xr-x 1 tna tna 39 Feb 24 13:18 2006-02 -> /Volumes/CSArchiveArray001/2006/2006-02 When I run rsync from Linux in --dry-run mode, the results are correct: $ rsync -arvn --ignore-existing --exclude-from=/db/nosyncc /db/2006/ -e ssh x:~/2/2006/ building file list ... done 2006-08/2006-08-12/2006-08-12_0430_KNBC_Channel_4_News.txt 2006-08/2006-08-12/2006-08-12_0430_KTTV_A_Current_Affair.txt 2006-08/2006-08-17/2006-08-17_0500_KABC_Eyewitness_News.txt 2006-08/2006-08-17/2006-08-17_0500_KTTV_Morning_News_at_5.txt 2006-08/2006-08-17/2006-08-17_0600_KTTV_Morning_News_at_6.txt 2006-10/2006-10-29/2006-10-29_0800_KTTV-FOX_News_Sunday_with_Chris_Wallace.txt sent 313109 bytes received 40 bytes 208766.00 bytes/sec total size is 445239817 speedup is 1421.81 When I do the actual run, the symlinks are overwritten and new directories are created; details below. Can I prevent this behavior and tell rsync to respect the symlinks, as it does in --dry-run mode? Dave $ rsync -arv --ignore-existing --exclude-from=/db/nosyncc /db/2006/ -e ssh x:~/2/2006/ building file list ... done ./ 2006-01/ 2006-01/2006-01-01/ 2006-01/2006-01-01/2006-01-01_0430_KTTV-FOX_A_Current_Affair.txt 2006-01/2006-01-01/2006-01-01_0500_KABC_Eyewitness_News.txt 2006-01/2006-01-01/2006-01-01_0500_KTTV-FOX_Morning_News_at_5.txt 2006-01/2006-01-01/2006-01-01_0800_KNBC_Meet_the_Press.txt 2006-01/2006-01-01/2006-01-01_0800_KTTV-FOX_News_Sunday_with_Chris_Wallace.txt 2006-01/2006-01-01/2006-01-01_1730_KABC_World_News_Tonight.txt 2006-01/2006-01-01/2006-01-01_1900_KCBS_60_Minutes.txt 2006-01/2006-01-01/2006-01-01_2200_KTTV-FOX_Ten_OClock_News.txt 2006-01/2006-01-01/2006-01-01_2300_KABC_Eyewitness_News.txt 2006-01/2006-01-02/ 2006-01/2006-01-02/2006-01-02_0430_KNBC_Channel_4_News.txt 2006-01/2006-01-02/2006-01-02_0430_KTTV-FOX_A_Current_Affair.txt 2006-01/2006-01-02/2006-01-02_0500_KABC_Eyewitness_News.txt 2006-01/2006-01-02/2006-01-02_0500_KTTV-FOX_Morning_News_at_5.txt 2006-01/2006-01-02/2006-01-02_0530_KCET_BBC_World_News.txt 2006-01/2006-01-02/2006-01-02_0600_KTTV-FOX_Morning_News_at_6.txt 2006-01/2006-01-02/2006-01-02_0700_KTTV-FOX_Morning_News_at_7.txt 2006-01/2006-01-02/2006-01-02_1700_KNBC_Channel_4_News_at_5.txt 2006-01/2006-01-02/2006-01-02_1830_KABC_World_News_Tonight.txt 2006-01/2006-01-02/2006-01-02_1900_KCET_Newshour_with_Jim_Lehrer.txt 2006-01/2006-01-02/2006-01-02_2200_KTTV-FOX_Ten_OClock_News.txt 2006-01/2006-01-02/2006-01-02_2300_KABC_Eyewitness_News.txt 2006-01/2006-01-02/2006-01-02_2330_KABC_Nightline.txt
On 2/24/07, David Liontooth <liontooth@cogweb.net> wrote:> I'm trying to rsync files to a set of symlinked directories on an XServe > running OS X.> When I run rsync from Linux in --dry-run mode, the results are correct:> When I do the actual run, the symlinks are overwritten and new > directories are created; details below. > Can I prevent this behavior and tell rsync to respect the symlinks, as > it does in --dry-run mode?The default is supposed to be to not "respect" existing symlinks in the destination. The --keep-dirlinks option makes rsync respect them. It is a bug that rsync appears to respect the symlink on a dry run. Rsync correctly reports that it would replace the symlink with a directory; use --itemize-changes to see the cd+++++++. However, when processing files inside, it blindly follows the obsolete symlink and concludes that the files are already there, whereas the newly created directory would have no files on a real run. One possible solution is to add a variable "will_unsymlink_below" to recv_generator to hold the depth below which rsync should simply assume destination files do not exist instead statting them through an obsolete symlink. However, the *_below approach is broken in the case of --no-implied-dirs, as I mentioned in this message: http://lists.samba.org/archive/rsync/2007-January/017193.html When rsync sends files inside an implied symlink "path/to/S" with --relative, the opposite problem can occur: rsync itemizes creations of destination files that it would find already exist if it had actually created the symlink at "path/to/S". This is nasty to fix. In order to correctly process each file X under "path/to/S", rsync needs to construct a path that goes to the same place that the path "path/to/S/X" would if the symlink at "path/to/S" were already in place. Suppose the symlink's target path is "../Y". rsync can't use "path/to/../Y/X" if "path/to" doesn't exist yet on the destination. And rsync can't use "path/Y/X" if "path/to" is a symlink. There may be additional complications I haven't thought of yet. This is an unpleasant decision: do we want to bloat rsync with the necessary logic to make --dry-run correct in these corner cases or settle for a --dry-run that sometimes misleads the user? Wayne, what's your opinion? Matt
On Sat, Feb 24, 2007 at 07:59:32PM -0500, Matt McCutchen wrote:> However, when processing files inside, it blindly follows the obsolete > symlink and concludes that the files are already there, whereas the > newly created directory would have no files on a real run.Not in a modern rsync. If the 2.6.9 was the receiver and not 2.6.3, rsync would have reported that all the files would have been created inside the directory that replaced the symlink. ..wayne..
On 2/24/07, Wayne Davison <wayned@samba.org> wrote:> On Sat, Feb 24, 2007 at 07:59:32PM -0500, Matt McCutchen wrote: > > However, when processing files inside, it blindly follows the obsolete > > symlink and concludes that the files are already there, whereas the > > newly created directory would have no files on a real run.My mistake. Now I see that after pretending to delete the symlink, rsync sets statret = -1. That causes it to set missing_below three lines later. Is my other example with an implied symlink valid against the current CVS rsync? Matt