Hey all, I'm not sure if anyone has experienced this, and I have searched for it online, with no conclusive, err.. conclusions. Basically, when rsyncing two \test1(local) and \mnt\test2\ (NFS mount) it seems that when using rsync with --no-whole-file entire files (instead of just updated blocks) are sent through. I am using the following command rsync -avtz --no-whole-file \test1\ \mnt\test2\ For example, if say a particular file was 30KB, IPTRAF reports just over 31KB transferred while rsync itself reports just a few bytes (i.e. 200 or so). I am thinking that either NFS is to blame, or that rsync is trying to rebuild the file on the other side and requires additional reads and commands to do this. --in-place doesn't seem to alleviate the problem much either. Any help would be appreciated. Thanks
Hi, On Thu, 27 Oct 2005, Anban Mestry wrote:> I'm not sure if anyone has experienced this, and I have searched for > it online, with no conclusive, err.. conclusions. > > Basically, when rsyncing two \test1(local) and \mnt\test2\ (NFS mount) > it seems that when using rsync with --no-whole-file entire files > (instead of just updated blocks) are sent through. > > I am using the following command > > rsync -avtz --no-whole-file \test1\ \mnt\test2\ > > For example, if say a particular file was 30KB, IPTRAF reports just > over 31KB transferred while rsync itself reports just a few bytes > (i.e. 200 or so).One of your "local only" rsync instances has to read the entire file in order to find out that only 200 bytes are to transfer. ;-)) Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org)
On Thu, Oct 27, 2005 at 07:27:09PM +0200, Anban Mestry wrote:> rsync -avtz --no-whole-file \test1\ \mnt\test2\You don't want to do that, because --no-whole-file optimizes rsync's socket I/O at the expense of disk I/O, which means that you're making things less efficient when the "connection" between the sender and the receiver is a local pipe. The use of -z for a "local" copy is also wasteful because you're using CPU to optimize the transfer of data over a connection that is faster than the disk I/O, even when uncompressed. Your best configuration is to avoid updating via NFS and instead connect to the NFS server directly so that rsync can update the files on a local disk. That allows rsync to optimize the network traffic. rsync -avtz /test1/ remoteNFShost:/test2/ If that is not possible, the method that uses the least disk I/O for a local copy is --whole-file and (to a much smaller extent) --inplace. That still writes out the entire file over NFS for each update, though, but it does at least avoid having rsync do a full-file read followed by a full-file write (which is what occurs with --no-whole-file). ..wayne..