rsync - Mar 2006 - Sent larger than literal

Hello -

When using the stats option I understand that Literal should be the amount of
data that did not match, and that Sent should be the actually amount of bytes
that were sent. I always use compress for remote transfers, so I'm used to
Sent being smaller (usually around half the size) than Literal.

However on one server, the backup from last night had the Sent figure larger
than Literal even though the compress option is used as is listed below:

Number of files: 191253
Number of files transferred: 42
Total file size: 92217159832 bytes
Total transferred file size: 692564897 bytes
Literal data: 2813738 bytes
Matched data: 689751159 bytes
File list size: 4910229
Total bytes sent: 6457999
Total bytes received: 392265

sent 6457999 bytes  received 392265 bytes  31208.49 bytes/sec
total size is 92217159832  speedup is 13461.84

Anybody have an idea why this would happen?

Thanks,
Max

On Sun, 2006-03-12 at 08:06 -0600, Max Kipness wrote:
> When using the stats option I understand that Literal should be the
> amount of data that did not match, and that Sent should be the
> actually amount of bytes that were sent. I always use compress for
> remote transfers, so I'm used to Sent being smaller (usually around
> half the size) than Literal.
> 
> However on one server, the backup from last night had the Sent figure
> larger than Literal even though the compress option is used as is
> listed below:
> 
> Number of files: 191253
> Number of files transferred: 42
> Total file size: 92217159832 bytes
> Total transferred file size: 692564897 bytes
> Literal data: 2813738 bytes
> Matched data: 689751159 bytes
> File list size: 4910229
> Total bytes sent: 6457999
> Total bytes received: 392265
You'll notice that the file list is 4910229 bytes long, presumably
because you are transferring _lots_ of files.  The file list, the
literal data, and the relatively small block-matching data are all being
sent, so it is plausible that they make up 6457999 bytes when
compressed.

-- 
Matt McCutchen
hashproduct@verizon.net
http://hashproduct.metaesthetics.net/

On Sun, Mar 12, 2006 at 08:06:35AM -0600, Max Kipness
wrote:
> Literal data:     2813738 bytes
> File list size:   4910229
                    -------
		    7723967
> Total bytes sent: 6457999
                    -------
		    1265968 bytes saved by -z

Since the file list has its own "compression" strategy, it is not
affected by the -z option.  Thus the 2813738 bytes of literal file
data was compressed to 1547770 bytes, and that made the total bytes
sent value lower than the 7723967 bytes that would have been sent
without -z.

See the 2.6.7 manpage if you want a description of what all the
values mean in the --stats output.  (It's linked from the main web
page if you don't have 2.6.7 yet.)

..wayne..