rsync - Nov 2014 - [Bug 10951] New: Emtpy parameter triggers unwanted behavior, but no error message

https://bugzilla.samba.org/show_bug.cgi?id=10951

            Bug ID: 10951
           Summary: Emtpy parameter triggers unwanted behavior, but no
                    error message
           Product: rsync
           Version: 3.1.0
          Hardware: x64
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: core
          Assignee: wayned at samba.org
          Reporter: stian at wis.no
        QA Contact: rsync-qa at samba.org

If you add a parameter (ex. --exclude <path>) to the rsync command without
a
value, you get some unwanted behavior running the command. No error message
will be triggered, and no output will be shown (especially if running the
command from remote server through a SSH session).

We were running the following command for backup of a remote server:
"/usr/bin/ssh -x -l backuppc <servername>.<domain>.lan sudo
/usr/bin/rsync
--numeric-ids --perms --owner --group -D --links --hard-links --times
--block-size=2048 --recursive / --exclude --exclude /mnt --exclude
/var/log/tomcat6"

If you check after the "--recursive /" there is two
"--exclude" and the first
one without a value. If we run this command there is no output from rsync
before the ssh session outputs a "pipe is broken" message. The rsync
continues
to run on the ssh session. There is no output running the command locally
either. 

With the emtpy "--exclude", and a new "--exclude" after the
empty one, it seems
like rsync interpits the two exclude folders as destination for syncronization.
We also seem to get at loop when syncing the /mnt folder aswell, due to
syncronization of itself into itself (this is not testet since this happend on
a production server).

When we ran the command written futher up in this text, we got a copy of /
(root) in both "/mnt" and "/var/log/tomcat6". In
"/mnt" we also got many
subfolders with their own copies of the root, all folder in root down to the
"/mnt" folder. Seems to be looping.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10951

--- Comment #1 from Stian Ertv?g <stian at wis.no> ---
The "--exclude" without a value was of course a punshing error in our
system,
but we think this should trigger an error instead of syncing to the folder
defined as excluded. The lack of output may occour because of the /mnt looping.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

I didn't want to put this in Bugzilla. Hopefully the OP will see it on
this list.

I'm not sure if you have the same problem I had, but it sure looks like it:

I ran into something very similar to this around a year ago. My script
was building the rsync command and when I didn't want a dry run, the
first parameter became "". That ended up (erroneously) adding the
current directory to my sources to back up.

I posted on this list and was told essentially (much more politely) -
"It's not  a bug, it's a feature."

But, a very kind and very knowledgeable soul (01/24/2013 05:58 PM,
Volker Kuhlmann) showed me how to avoid it - and make my bash code a lot
better at the same time!

Here are the essential snips of how to avoid problems like this:

...
RSYNC_OPTIONS=(
  ${DRY_RUN}
  -avushi
  ${DELETE}
  ${EXCLUDES}
  --stats
  --progress
)
...
eval rsync "${RSYNC_OPTIONS[@]}"  ...

When you do it like this - using a bash array with eval - (changing all
the values to match your requirements), all the null strings just go away!

It makes it easy to cleanly and clearly build a complex argument list
piece by piece in your script with as much logic as needed to make sure
that things are built correctly.

The final command is built in only one place, so any changes to
parameters which are relatively constant only have to be done in that
one place, not in many similar command invocations where a common change
might be missed in one or more instances.

It also makes the code way easier to read and maintain. I am using this
technique in other applications as well.

Joe

On 11/21/2014 02:46 AM, samba-bugs at samba.org wrote:
> https://bugzilla.samba.org/show_bug.cgi?id=10951
>
>             Bug ID: 10951
>            Summary: Emtpy parameter triggers unwanted behavior, but no
>                     error message
>            Product: rsync
>            Version: 3.1.0
>           Hardware: x64
>                 OS: Linux
>             Status: NEW
>           Severity: major
>           Priority: P5
>          Component: core
>           Assignee: wayned at samba.org
>           Reporter: stian at wis.no
>         QA Contact: rsync-qa at samba.org
>
> If you add a parameter (ex. --exclude <path>) to the rsync command
without a
> value, you get some unwanted behavior running the command. No error message
> will be triggered, and no output will be shown (especially if running the
> command from remote server through a SSH session).
>
> We were running the following command for backup of a remote server:
> "/usr/bin/ssh -x -l backuppc <servername>.<domain>.lan
sudo /usr/bin/rsync
> --numeric-ids --perms --owner --group -D --links --hard-links --times
> --block-size=2048 --recursive / --exclude --exclude /mnt --exclude
> /var/log/tomcat6"
>
> If you check after the "--recursive /" there is two
"--exclude" and the first
> one without a value. If we run this command there is no output from rsync
> before the ssh session outputs a "pipe is broken" message. The
rsync continues
> to run on the ssh session. There is no output running the command locally
> either. 
>
> With the emtpy "--exclude", and a new "--exclude" after
the empty one, it seems
> like rsync interpits the two exclude folders as destination for
syncronization.
> We also seem to get at loop when syncing the /mnt folder aswell, due to
> syncronization of itself into itself (this is not testet since this happend
on
> a production server).
>
> When we ran the command written futher up in this text, we got a copy of /
> (root) in both "/mnt" and "/var/log/tomcat6". In
"/mnt" we also got many
> subfolders with their own copies of the root, all folder in root down to
the
> "/mnt" folder. Seems to be looping.
>

https://bugzilla.samba.org/show_bug.cgi?id=10951

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |INVALID

--- Comment #2 from Wayne Davison <wayned at samba.org> ---
Since the command-line you mentioned is completely valid, there's no error
for
rsync to complain about.  Your args specified "--exclude --exclude"
(so that it
won't transfer a file named "--exclude") and then specified
"/mnt" as a source
or destination arg (depending on where it appears in the list of args).

One thing you could possibly do to make the arg parsing stricter is to
"export
POSIXLY_CORRECT=1" in the environment.  That makes popt stop parsing
options at
the first non-option it finds, so something like "--exclude --exclude /mnt
--del /src/ /dest/" would at least complain about the file
"--del" not existing
while it was copying /mnt and /src/ to /dest (though it doesn't stop the
copying from happening).

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10951

--- Comment #3 from Wayne Davison <wayned at samba.org> ---
Let me also add that if the cause was a bash shell variable that ended up being
empty, then you should note that it is a bash-ism to require parameters to be
double-quoted in order for them to behave sanely (e.g. --exclude
"$arg")
whereas other shells (such as zsh) default to a saner behavior for something
like "--exclude $arg".

-- 
You are receiving this mail because:
You are the QA Contact for the bug.