rsync - Sep 2007 - delta copies

I posted the other day about trying to determine why my backups seem not
to be doing this. I am trying to figure this out this weekend, I have
two files on a RAID0 FreeBSD file system using SATA drives. I do a
simple tar of /var twice and then rsync the two files only as a test, no
folders involved....trying to determine why the entire file is being
copied in its entirety and not synchronized bit-by-bit. Is it dependent
on file types that can be done this way. Stephen was helpful the other
day to reply and he is able to rsync sql backups, exchange, etc. I've
tried with uncompressed pgsql backups and now tar, neither
working...Does rsync require certain compile options perhaps that I am
missing?

esmtp# ls -la test*
test:
total 1268356
drwxr-xr-x   2 root  wheel         512 Sep 22 18:46 .
drwxr-xr-x  11 root  wheel         512 Sep 22 18:46 ..
-rw-r--r--   1 root  wheel  1298128896 Sep 22 18:47 testvar.tar

test2:
total 1268388
drwxr-xr-x   2 root  wheel         512 Sep 22 18:49 .
drwxr-xr-x  11 root  wheel         512 Sep 22 18:46 ..
-rw-r--r--   1 root  wheel  1298163200 Sep 22 18:49 testvar.tar
esmtp# rsync -az --progress --stats test/ test2/
building file list ... 
2 files to consider
./
testvar.tar
  1298128896 100%   13.00MB/s    0:01:35  (1, 100.0% of 2)

Number of files: 2
Number of files transferred: 1
Total file size: 1298128896 bytes
Total transferred file size: 1298128896 bytes
Literal data: 1298128896 bytes
Matched data: 0 bytes
File list size: 53
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 178178479
Total bytes received: 48

sent 178178479 bytes  received 48 bytes  1846409.61 bytes/sec
total size is 1298128896  speedup is 7.29

However, I have a Microsoft SQL Server (binary compressed) backup being
transferred between two Windows servers using cwRsync and this is
working fine. It only takes a couple of minutes compared to almost 30
minutes in the initial transfer. After setting that up for a customer is
when I realized this was possible after using rsync for years doing
backups on our own Unix servers. So, I'd like this to happen for me,
thanks for any help...Also, does the match data bytes show the amount
matched when making delta copies, I assume? This seems to be the case as
the above shows zero and I see almost the same amount of byes here in my
Windows transfer.

-- 
Robert

On 9/22/07, Robert Fitzpatrick <lists@webtent.net>
wrote:
> I do a
> simple tar of /var twice and then rsync the two files only as a test, no
> folders involved....trying to determine why the entire file is being
> copied in its entirety and not synchronized bit-by-bit.
> esmtp# rsync -az --progress --stats test/ test2/
> building file list ...
> 2 files to consider
> ./
> testvar.tar
>   1298128896 100%   13.00MB/s    0:01:35  (1, 100.0% of 2)
>
> Number of files: 2
> Number of files transferred: 1
> Total file size: 1298128896 bytes
> Total transferred file size: 1298128896 bytes
> Literal data: 1298128896 bytes
> Matched data: 0 bytes
> File list size: 53
> File list generation time: 0.001 seconds
> File list transfer time: 0.000 seconds
> Total bytes sent: 178178479
> Total bytes received: 48
>
> sent 178178479 bytes  received 48 bytes  1846409.61 bytes/sec
> total size is 1298128896  speedup is 7.29
Delta transfers reduce network traffic between the sending and
receiving rsync processes at the cost of some extra CPU time and disk
I/O (e.g., the receiver has to read the old destination file).  The
reduction in network traffic is only relevant if the two processes are
on different machines.  When they are on the same machine as in your
example, there is no point in doing delta transfers, so by default
rsync does not do them.  You can force rsync to do delta transfers by
passing --no-whole-file, but I can't imagine why you would want to do
this unless you are testing to see how much traffic reduction delta
transfers would achieve.
> Also, does the match data bytes show the amount
> matched when making delta copies, I assume?
Yes.  More precisely, it shows the amount that rsync found to be
matched.  Rsync's goal is to match as much data as convenient, not to
give you an exact measure of how much of the file changed.

Matt