Tom Goulet
2003-Apr-22 10:11 UTC
full upload happening even though only a timestamp has changed
Hello,
For no apparent reason Rsync is uploading the whole file even though the
the remote file is identical to the local file. Here is a transcript to
demonstrate what is going wrong:
tomg@nova:~/mylibrary$ rsync -rtlPvessh /extra/library teep:/extra/tomg
building file list ...
46349 files to consider
wrote 1358812 bytes read 20 bytes 44551.87 bytes/sec
total size is 5240249974 speedup is 3856.44
tomg@nova:~/mylibrary$ touch
/extra/library/software/Knopper,_Klaus/KNOPPIX_V3.2-2003-04-18-EN.iso
tomg@nova:~/mylibrary$ rsync -rtlPvessh /extra/library teep:/extra/tomg
building file list ...
46349 files to consider
library/software/Knopper,_Klaus/KNOPPIX_V3.2-2003-04-18-EN.iso
12288000 1% 1.05MB/s 0:11:09
It will take eleven minutes or so and upload the entire thing. What I
expected to happen was Rsync doing its checksum thing on the entire file
and then not uploading anything because the file was the same.
Here are listings of two files on the remote host to show that uploading
is really happening:
-rw------- 1 tomg users 104792064 Apr 21 18:07
.KNOPPIX_V3.2-2003-04-18-EN.iso.vNUGjU
-rw-r--r-- 1 tomg users 729716736 Apr 21 13:55
KNOPPIX_V3.2-2003-04-18-EN.iso
Local Rsync is version 2.5.6-0.1 on Debian testing. Remote Rsync is
version 2.5.4 on Slackware 8.0. If I recall correctly, the same problem
happened with a remote Rsync of version 2.5.5-0.1 on Debian testing.
Is this a bug? Am I doing something wrong? Please give me a CC as I am
not subscribed.
Thanks,
--
Tom Goulet, tomg@em.ca, D8BAD3BC, http://web.em.ca/~tomg/contact.html
I usually read no mail which is unsolicited and bulk, is not exclusively in
plain text, has advertising footers, is top-posted, or has bad line wrap.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url :
http://lists.samba.org/archive/rsync/attachments/20030422/4284c49c/attachment.bin
jw schultz
2003-Apr-22 10:23 UTC
full upload happening even though only a timestamp has changed
On Tue, Apr 22, 2003 at 12:11:10AM +0000, Tom Goulet wrote:> Hello, > > For no apparent reason Rsync is uploading the whole file even though the > the remote file is identical to the local file. Here is a transcript to > demonstrate what is going wrong: > tomg@nova:~/mylibrary$ rsync -rtlPvessh /extra/library teep:/extra/tomg > building file list ... > 46349 files to consider > wrote 1358812 bytes read 20 bytes 44551.87 bytes/sec > total size is 5240249974 speedup is 3856.44 > tomg@nova:~/mylibrary$ touch /extra/library/software/Knopper,_Klaus/KNOPPIX_V3.2-2003-04-18-EN.iso > tomg@nova:~/mylibrary$ rsync -rtlPvessh /extra/library teep:/extra/tomg > building file list ... > 46349 files to consider > library/software/Knopper,_Klaus/KNOPPIX_V3.2-2003-04-18-EN.iso > 12288000 1% 1.05MB/s 0:11:09 > > It will take eleven minutes or so and upload the entire thing. What I > expected to happen was Rsync doing its checksum thing on the entire file > and then not uploading anything because the file was the same. > > Here are listings of two files on the remote host to show that uploading > is really happening: > -rw------- 1 tomg users 104792064 Apr 21 18:07 .KNOPPIX_V3.2-2003-04-18-EN.iso.vNUGjU > -rw-r--r-- 1 tomg users 729716736 Apr 21 13:55 KNOPPIX_V3.2-2003-04-18-EN.iso > > Local Rsync is version 2.5.6-0.1 on Debian testing. Remote Rsync is > version 2.5.4 on Slackware 8.0. If I recall correctly, the same problem > happened with a remote Rsync of version 2.5.5-0.1 on Debian testing. > > Is this a bug? Am I doing something wrong? Please give me a CC as I am > not subscribed.If the mtime indicates the file has changed it will do an update. That doesn't mean that any actual file data will be transfered. If all the blocksums match it will effectively do a local copy to the temp file and then rename back. If you want it to do a whole-file checksum as part of decideing whether to update you will need to use the -c option which overrides mtime. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt
jw schultz
2003-Apr-22 12:53 UTC
full upload happening even though only a timestamp has changed
On Tue, Apr 22, 2003 at 12:11:10AM +0000, Tom Goulet wrote:> Hello, > > For no apparent reason Rsync is uploading the whole file even though the > the remote file is identical to the local file. Here is a transcript to > demonstrate what is going wrong: > tomg@nova:~/mylibrary$ rsync -rtlPvessh /extra/library teep:/extra/tomg > building file list ... > 46349 files to consider > wrote 1358812 bytes read 20 bytes 44551.87 bytes/sec > total size is 5240249974 speedup is 3856.44 > tomg@nova:~/mylibrary$ touch /extra/library/software/Knopper,_Klaus/KNOPPIX_V3.2-2003-04-18-EN.iso > tomg@nova:~/mylibrary$ rsync -rtlPvessh /extra/library teep:/extra/tomg > building file list ... > 46349 files to consider > library/software/Knopper,_Klaus/KNOPPIX_V3.2-2003-04-18-EN.iso > 12288000 1% 1.05MB/s 0:11:09 > > It will take eleven minutes or so and upload the entire thing. What I > expected to happen was Rsync doing its checksum thing on the entire file > and then not uploading anything because the file was the same. > > Here are listings of two files on the remote host to show that uploading > is really happening: > -rw------- 1 tomg users 104792064 Apr 21 18:07 .KNOPPIX_V3.2-2003-04-18-EN.iso.vNUGjU > -rw-r--r-- 1 tomg users 729716736 Apr 21 13:55 KNOPPIX_V3.2-2003-04-18-EN.iso > > Local Rsync is version 2.5.6-0.1 on Debian testing. Remote Rsync is > version 2.5.4 on Slackware 8.0. If I recall correctly, the same problem > happened with a remote Rsync of version 2.5.5-0.1 on Debian testing. > > Is this a bug? Am I doing something wrong? Please give me a CC as I am > not subscribed.I think this may be a very old bug. According to OLD_NEWS 2.5.4 has a bug where --whole-file is the default even over the network. Try it with --no-whole-file. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt