rsync - May 2004 - Would this patch be useful...

I am considering a patch that would exclude compressing a given list
of file extensions when the -z option is used.  Users may want to use
-z, but not want to try to compress .gz, .jpg, .mp3, etc. type files.
Rsyncd has this feature with it's "dont compress" option, but I
think
many users are using rsync over ssh and therefore do not have this
functionality available to them.

The way my test code works is by modifying the set_compression() function
in token.c and using the wildmatch() function, exactly like rsyncd's
dont compress option is doing now.

If this seems useful, I want to receive input on how to implement
the user-interface.  I was thinking about two command line options,
--dont-compress and --dont-compress-from.

This is what it would look like:

rsync -z --dont-compress="*.mp3 *.gz ....."

However, since this list could be very long, you would also have:

rsync -z --dont-compress-from=/etc/rsync/already_compressed.txt

where the already_compressed.txt would have a list of extensions not to
compress.


Another option, is to have --dont-compress and --dont-compress-from use
rsync's Exclude Patterns, but I have not studied this code and do not
know how much harder this would be to implement.

Any thought?

-John

On Tue, May 11, 2004 at 11:41:02AM -0400, John Taylor
wrote:
> I am considering a patch that would exclude compressing a given list
> of file extensions when the -z option is used.
Well, since the daemon code has this, it makes sense to at least
consider adding this to the normal remote-shell code.  I'm not sure how
much this option speeds things up -- have you done any measuring?

I'd also not want to let someone who is grabbing files from a server
specify a shorter list than the one that the server specified, so we'd
need to make sure that the user's list appended to the server's list
when pulling.

Though the docs don't mention this, the daemon mode only appears to use
the "dont compress" value when the daemon is sending files -- it
doesn't
appear to do anything when it is receiving files.  This is because the
list is not shared across the socket, and only the sender cares about
this value.
> I want to receive input on how to implement the user-interface.  I was
> thinking about two command line options, --dont-compress and
> --dont-compress-from.
The easiest way would be to add only the --dont-compress="foo bar ..."
option since that would allow the value to be sent to the sending side
via the existing option mechanism.  Adding a --dont-compress-from option
would mean that the protocol would have to be extended to transfer this
data in a manner similar to the current exclude list, which requires a
protocol bump.  If you go that route, I'd suggest making the options
work just like the exclude options so that each --dont-compress option
would only specify a single item (and it would be easiest to switch the
internal dont-compress code over to using the exclude-list code so that
you don't need to re-invent a bunch of identical functions).

So, I'm not convinced yet that this is worth it, but it might be.  If
you'd care to submit a patch, I'd at least put it in the patches dir,
and certainly consider it for the release version.  What do others
think?

..wayne..