Brian J. Murrell
2012-Jan-20 22:27 UTC
--link-dest doesn't work if target file exists (but needs updating)
Using:
# rsync --version
rsync version 3.0.7 protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
Given two files:
# /bin/ls -li {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYD
44695517 -rw-rw---- 2 sshd messagebus 5492 2012-01-19 00:31
source/var/lib/mysql/mythconverg/jobqueue.MYD
49676928 -rw-rw---- 1 sshd messagebus 5492 2012-01-18 12:00
dest/var/lib/mysql/mythconverg/jobqueue.MYD
Their md5sums:
# md5sum {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYD
b87dafdcf59ab3e7e9907c15385175b3 source/var/lib/mysql/mythconverg/jobqueue.MYD
c03b1ee584ef1405958b695f4e55b51d dest/var/lib/mysql/mythconverg/jobqueue.MYD
Let's try to create a link from
source/var/lib/mysql/mythconverg/jobqueue.MYD to
dest/var/lib/mysql/mythconverg/jobqueue.MYD
Dry-run first to see what rsync tells us it's going to do:
# rsync -naiiAXH --link-dest=/source/var/lib/mysql/mythconverg/
source/var/lib/mysql/mythconverg/jobqueue.MYD
dest/var/lib/mysql/mythconverg/>f..t...... jobqueue.MYD
Looks like it's not going to create a link. Let's be sure:
# rsync -aiiAXH --link-dest=/source/var/lib/mysql/mythconverg/
source/var/lib/mysql/mythconverg/jobqueue.MYD
dest/var/lib/mysql/mythconverg/>f..t...... jobqueue.MYD
Yup, looks like it didn't create a link. The proof:
# /bin/ls -li {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYD
44695517 -rw-rw---- 2 sshd messagebus 5492 2012-01-19 00:31
source/var/lib/mysql/mythconverg/jobqueue.MYD
49676897 -rw-rw---- 1 sshd messagebus 5492 2012-01-19 00:31
dest/var/lib/mysql/mythconverg/jobqueue.MYD
Yet it did copy the file:
# md5sum {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYD
b87dafdcf59ab3e7e9907c15385175b3 source/var/lib/mysql/mythconverg/jobqueue.MYD
b87dafdcf59ab3e7e9907c15385175b3 dest/var/lib/mysql/mythconverg/jobqueue.MYD
Why didn't it remove the existing file and create the link like we
wanted it to with --link-dest? Let's try another file just to prove
that it will create a link if the destination file doesn't exist:
Two different files, same use case though:
# /bin/ls -li {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYI
44695518 -rw-rw---- 2 sshd messagebus 5120 2012-01-19 00:31
source/var/lib/mysql/mythconverg/jobqueue.MYI
49676956 -rw-rw---- 1 sshd messagebus 5120 2012-01-18 12:00
dest/var/lib/mysql/mythconverg/jobqueue.MYI
Different content again:
# md5sum {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYI
6a0b5bdedfe738fbca17630e6496f85c dest/var/lib/mysql/mythconverg/jobqueue.MYI
9052f8e598f8ee3bdcb2cbc510f0564c source/var/lib/mysql/mythconverg/jobqueue.MYI
See what rsync says it's going to do (before we remove the target file):
# rsync -naiiAXH --link-dest=/source/var/lib/mysql/mythconverg/
source/var/lib/mysql/mythconverg/jobqueue.MYI
dest/var/lib/mysql/mythconverg/>f..t...... jobqueue.MYI
Again, it's going to copy.
But if we remove the target file and see what rsync says it's going to do:
# rm dest/var/lib/mysql/mythconverg/jobqueue.MYI
# rsync -naiiAXH --link-dest=/source/var/lib/mysql/mythconverg/
source/var/lib/mysql/mythconverg/jobqueue.MYI
dest/var/dest/var/lib/mysql/mythconverg/
hf jobqueue.MYI
Ah ha! Say's it's going to link. Let's actually do it and see:
# rsync -aiiAXH --link-dest=/source/var/lib/mysql/mythconverg/
source/var/lib/mysql/mythconverg/jobqueue.MYI dest/var/lib/mysql/mythconverg/
hf jobqueue.MYI
And sure enough, we now have three links to the same file (as we would
expect):
# /bin/ls -li {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYI
44695518 -rw-rw---- 3 sshd messagebus 5120 2012-01-19 00:31
source/var/lib/mysql/mythconverg/jobqueue.MYI
44695518 -rw-rw---- 3 sshd messagebus 5120 2012-01-19 00:31
dest/var/lib/mysql/mythconverg/jobqueue.MYI
And the content (although we know it should report the same):
# md5sum {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYI
9052f8e598f8ee3bdcb2cbc510f0564c dest/var/lib/mysql/mythconverg/jobqueue.MYI
9052f8e598f8ee3bdcb2cbc510f0564c source/var/lib/mysql/mythconverg/jobqueue.MYI
So why doesn't the same thing work when the destination file exists?
The element of least surprise would make me think it should. That is,
even if the destination file exists, if the result of the sync is the
same as a file that a --link-dest points to, it should remove the
destination file and create the link.
Thoughts?
b.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20120120/b2ab63f3/attachment.pgp>
Kevin Korb
2012-Jan-20 22:42 UTC
--link-dest doesn't work if target file exists (but needs updating)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Am I understanding right that your source and your link-dest are actually the same path? If so what are you trying to do that wouldn't be accomplished with a simple 'cp -l'? Also, when using --link-dest the target is treated differently. Normally the target is an empty directory but if files already exist the behavior you are seeing is what is supposed to happen. Instead of deleting files and replacing them with links it assumes that they are already linked to whatever they are supposed to be linked to and updates them accordingly. On 01/20/12 17:27, Brian J. Murrell wrote:> Using: > > # rsync --version rsync version 3.0.7 protocol version 30 > Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and > others. Web site: http://rsync.samba.org/ Capabilities: 64-bit > files, 64-bit inums, 32-bit timestamps, 64-bit long ints, > socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace, > append, ACLs, xattrs, iconv, symtimes > > rsync comes with ABSOLUTELY NO WARRANTY. This is free software, > and you are welcome to redistribute it under certain conditions. > See the GNU General Public Licence for details. > > Given two files: > > # /bin/ls -li {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYD > 44695517 -rw-rw---- 2 sshd messagebus 5492 2012-01-19 00:31 > source/var/lib/mysql/mythconverg/jobqueue.MYD 49676928 -rw-rw---- 1 > sshd messagebus 5492 2012-01-18 12:00 > dest/var/lib/mysql/mythconverg/jobqueue.MYD > > Their md5sums: > > # md5sum {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYD > b87dafdcf59ab3e7e9907c15385175b3 > source/var/lib/mysql/mythconverg/jobqueue.MYD > c03b1ee584ef1405958b695f4e55b51d > dest/var/lib/mysql/mythconverg/jobqueue.MYD > > Let's try to create a link from > source/var/lib/mysql/mythconverg/jobqueue.MYD to > dest/var/lib/mysql/mythconverg/jobqueue.MYD > > Dry-run first to see what rsync tells us it's going to do: > > # rsync -naiiAXH --link-dest=/source/var/lib/mysql/mythconverg/ > source/var/lib/mysql/mythconverg/jobqueue.MYD > dest/var/lib/mysql/mythconverg/ >> f..t...... jobqueue.MYD > > Looks like it's not going to create a link. Let's be sure: > > # rsync -aiiAXH --link-dest=/source/var/lib/mysql/mythconverg/ > source/var/lib/mysql/mythconverg/jobqueue.MYD > dest/var/lib/mysql/mythconverg/ >> f..t...... jobqueue.MYD > > Yup, looks like it didn't create a link. The proof: > > # /bin/ls -li {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYD > 44695517 -rw-rw---- 2 sshd messagebus 5492 2012-01-19 00:31 > source/var/lib/mysql/mythconverg/jobqueue.MYD 49676897 -rw-rw---- 1 > sshd messagebus 5492 2012-01-19 00:31 > dest/var/lib/mysql/mythconverg/jobqueue.MYD > > Yet it did copy the file: > > # md5sum {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYD > b87dafdcf59ab3e7e9907c15385175b3 > source/var/lib/mysql/mythconverg/jobqueue.MYD > b87dafdcf59ab3e7e9907c15385175b3 > dest/var/lib/mysql/mythconverg/jobqueue.MYD > > Why didn't it remove the existing file and create the link like we > wanted it to with --link-dest? Let's try another file just to > prove that it will create a link if the destination file doesn't > exist: > > Two different files, same use case though: > > # /bin/ls -li {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYI > 44695518 -rw-rw---- 2 sshd messagebus 5120 2012-01-19 00:31 > source/var/lib/mysql/mythconverg/jobqueue.MYI 49676956 -rw-rw---- 1 > sshd messagebus 5120 2012-01-18 12:00 > dest/var/lib/mysql/mythconverg/jobqueue.MYI > > Different content again: > > # md5sum {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYI > 6a0b5bdedfe738fbca17630e6496f85c > dest/var/lib/mysql/mythconverg/jobqueue.MYI > 9052f8e598f8ee3bdcb2cbc510f0564c > source/var/lib/mysql/mythconverg/jobqueue.MYI > > See what rsync says it's going to do (before we remove the target > file): > > # rsync -naiiAXH --link-dest=/source/var/lib/mysql/mythconverg/ > source/var/lib/mysql/mythconverg/jobqueue.MYI > dest/var/lib/mysql/mythconverg/ >> f..t...... jobqueue.MYI > > Again, it's going to copy. > > But if we remove the target file and see what rsync says it's going > to do: > > # rm dest/var/lib/mysql/mythconverg/jobqueue.MYI # rsync -naiiAXH > --link-dest=/source/var/lib/mysql/mythconverg/ > source/var/lib/mysql/mythconverg/jobqueue.MYI > dest/var/dest/var/lib/mysql/mythconverg/ hf jobqueue.MYI > > Ah ha! Say's it's going to link. Let's actually do it and see: > > # rsync -aiiAXH --link-dest=/source/var/lib/mysql/mythconverg/ > source/var/lib/mysql/mythconverg/jobqueue.MYI > dest/var/lib/mysql/mythconverg/ hf jobqueue.MYI > > And sure enough, we now have three links to the same file (as we > would expect): > > # /bin/ls -li {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYI > 44695518 -rw-rw---- 3 sshd messagebus 5120 2012-01-19 00:31 > source/var/lib/mysql/mythconverg/jobqueue.MYI 44695518 -rw-rw---- 3 > sshd messagebus 5120 2012-01-19 00:31 > dest/var/lib/mysql/mythconverg/jobqueue.MYI > > And the content (although we know it should report the same): > > # md5sum {source,dest}/var/lib/mysql/mythconverg/jobqueue.MYI > 9052f8e598f8ee3bdcb2cbc510f0564c > dest/var/lib/mysql/mythconverg/jobqueue.MYI > 9052f8e598f8ee3bdcb2cbc510f0564c > source/var/lib/mysql/mythconverg/jobqueue.MYI > > So why doesn't the same thing work when the destination file > exists? The element of least surprise would make me think it > should. That is, even if the destination file exists, if the > result of the sync is the same as a file that a --link-dest points > to, it should remove the destination file and create the link. > > Thoughts? > > b. > > > > > >- -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8Z7eMACgkQVKC1jlbQAQf8XwCfYKlyLfZ4NYeP3oLQd0fn6gcT x2YAn3lxINnqyywj7dyuI45JoajM1VSj =037D -----END PGP SIGNATURE-----
Possibly Parallel Threads
- [Bug 8712] New: --link-dest doesn't work if target file exists (but needs updating)
- How to make big MySQL database more diffable/rsyncable? (aka rsyncing big files)
- Transfer hangs, both sides waiting on receive
- [Bug 1599] New: copy-unsafe-links cann't take affect
- lapply and aggregate function