rsync - Aug 2011 - [Bug 8354] New: --skip-compress does not appear to be working

https://bugzilla.samba.org/show_bug.cgi?id=8354

           Summary: --skip-compress does not appear to be working
           Product: rsync
           Version: 3.0.7
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: justincwatt at gmail.com
         QAContact: rsync-qa at samba.org


I use rsync to backup my Ubuntu home directory to an external USB hard drive.
I've always used the flags -avz without thinking, and I've always
wondered why
my CPU shoots to 100% and my laptop gets hot enough to fry an egg. Finally it
occurred to me. The vast majority of files rsync is transferring at JPEGs,
which can't be compressed. So my laptop was spending all its time
compressing
and decompressing large files to no benefit. 

After removing the "z" flag, I noticed a tremendous speed up, not to
mention a
cooler, happier CPU. I dug deeper into the man page and discovered the
--skip-compress option, which according to Bug 4162 should have its default
file extension list enabled whenever the "z" flag is set. However
I've found
that not to be the case. I thought maybe it had to do with the fact that my
JPEG files have an uppercase .JPG extension (Bug 7512) but even when I directly
specify an uppercase extension with --skip-compress, I see no speed up.

Here are the results of my tests running rsync to backup my home directory
after repeatedly deleting a destination directory of 788,709,822 bytes of
photos and a movie.


without -z:

sent 791193666 bytes  received 8629 bytes  23617978.96 bytes/sec
total size is 205231570237  speedup is 259.39

with -z:

sent 788709822 bytes  received 8629 bytes  9677527.01 bytes/sec
total size is 205231570237  speedup is 260.21

with -z and --skip-compress=jpg/JPG/avi/AVI:

sent 788709822 bytes  received 8629 bytes  9560223.65 bytes/sec
total size is 205231570237  speedup is 260.21

Notice the order of magnitude difference in the bytes/sec rate.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8354

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #1 from Wayne Davison <wayned at samba.org> 2011-09-03
22:25:58 UTC ---
This fix was noted in 3.0.8:

- Fixed a bug in the comparing of upper-case letters in file suffixes for
--skip-compress.

The issue was that the skip-compress values are always stored in lower case,
but if the file had upper-case letters in the suffix, they wouldn't match
the
stored suffixes.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8354

--- Comment #2 from Justin Watt <justincwatt at gmail.com> 2011-09-04
03:50:40 UTC ---
Wayne, but what about when I explicitly set an uppercase file extension with
--skip-compress? Did it also treat that as lowercase?

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8354

--- Comment #3 from Wayne Davison <wayned at samba.org> 2011-09-05
06:36:52 UTC ---
The way it's written, --skip-compress always ignores case.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.