Ian! D. Allen
2007-Feb-25 05:11 UTC
moving local files, remote mirroring, and --link-dest awkwardness
I have some pictures that I unload from the camera into: dir/tmp and at night I automatically rsync all of dir/ to some remote:dir/ Then days or weeks later I do some local sorting, and I make lots of image moves of the sort where image dir/tmp/bar goes to to dir/category1/bar, possibly linked also into dir/category2/bar. Then rsync runs the next night, notices the "new" image dir/category1/bar and does the huge image upload all over again, despite the remote system already having that image in dir/tmp/bar from the previous night. I need to make rsync use the existing file. Aha! Use link-dest! But what link-dest? --link-dest=tmp doesn't work because when rsync can't find category1/bar on the remote system it looks in the remote --link-dest using tmp/category1/bar and not using tmp/bar where the file actually is on the remote system. So rsync thinks it has to copy the whole file again. My first kludge-approach is to go into the remote dir/tmp and create hundreds of "ln -s . category1; ln -s . category2; ..." links for all the dir/* categories. That allows --link-dest=tmp to work; but, it's a mess and it won't work if I move dir/tmp/bar to dir/category1/subdir/bar. Any smart people have a better way to handle remote rsync of files that move from one directory into a set of other directories? (A second kludge-approach is not to move the files at all, but simply to link the file from dir/tmp/bar to dir/category1/bar and use the -H feature to update hard links. That also works; but, then I have a mix of linked and not-linked files in dir/tmp that I have to sort out.) -- | Ian! D. Allen - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Open Source / Linux) via: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/