-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 It will, assuming it sees both links in the same rsync run. On 01/23/2016 11:46 AM, dbonde+forum+rsync.lists.samba.org at gmail.com wrote:> On 2016-01-21 09:20, dbonde+forum+rsync.lists.samba.org at gmail.com > wrote: >> I run a rsync job transferring about 45 million >> files/approximately 1.8 TB data (a Mac OS X Time Machine backup) >> over a 100 MBit connection. >> >> I use rsync 3.1.1 from MacPorts (I first tried the built in >> rsync, version 2.6.9, since it has a Mac OS X specific cache >> parameter, but it ran out of memory) with the following >> parameters >> >> % rsync -HzvhErlptgoDW --stats --progress --out-format="%t %f >> %b" /source/ /destination/ > > Well, after some examination I found at least one problem with > this transfer (that is still running): hard links are not > preserved: > > This is how a certain file looks at the source where it is backed > up on several locations using hard links: > > source volume: > > zsh-% ls -i "/…/backups/2011-06-23-040258/Pictures/DSCF0748.JPG" > 9236871 /…/backups/2011-06-23-040258/Pictures/DSCF0748.JPG > > zsh-% ls -i "/…/backups/2010-12-18-070445/Pictures/DSCF0748.JPG" > 9236871 /…/backups/2010-12-18-070445/Pictures/DSCF0748.JPG > > > destination volume: > > zsh-% ls -i "/…/backups/2011-06-23-040258/Pictures/DSCF0748.JPG" > 20765913 /…/backups/2011-06-23-040258/Pictures/DSCF0748.JPG > > zsh-% ls -i "/…/backups/2010-12-18-070445/Pictures/DSCF0748.JPG" > 704428 /…/backups/2010-12-18-070445/Pictures/DSCF0748.JPG > > As you can see the inode number is the same on the source volume > while it is completely different on the destination volume. > > Why are my hard links not preserved? I thought the purpose with -H > was to transfer the hard links rather than the file itself. >- -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlajr1kACgkQVKC1jlbQAQdG3QCgwRt/K9u6xrxGFeZP2uoPoaoT OlcAnjE4eozRjJ1Mb9YC88YNhVTLEpP8 =p3pD -----END PGP SIGNATURE-----
dbonde+forum+rsync.lists.samba.org at gmail.com
2016-Jan-23 20:04 UTC
Why is my rsync transfer slow?
On 2016-01-23 17:50, Kevin Korb wrote:> It will, assuming it sees both links in the same rsync run.How does one handle interrupted transfers if one wants to preserve hard links? Would --partial and --append-verify work?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 As long as it still sees both links it is fine. Essentially, the way it works is that whenever rsync -H (on the source) sees a file with a link count >1 it remembers the inode#>filename pair. If it finds another instance of that inode it then links to the same file on the target. So, if you abort after it copies one but before it links the other it will still handle it correctly on the next run. It just won't handle it if you rsync tree #1 then rsync tree #2. It won't see a hard link that is common to both since it wasn't analyzing them together. On 01/23/2016 03:04 PM, dbonde+forum+rsync.lists.samba.org at gmail.com wrote:> On 2016-01-23 17:50, Kevin Korb wrote: >> It will, assuming it sees both links in the same rsync run. > > How does one handle interrupted transfers if one wants to preserve > hard links? Would --partial and --append-verify work? >- -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlaj34gACgkQVKC1jlbQAQdDPgCfQBP/mR7x0a6JVLIJZzye+6Io 0woAn3BGe9y0mjOfZbK62R0OHuzOzChl =q3hs -----END PGP SIGNATURE-----