openssh unix dev - Nov 2019 - scp, sftp, and special characters in filenames

I have not been able to get scp(1) to download a file with a newline
in its name.  I know that scp(1) requires that remote filenames be
escaped for the shell, but that leads to protocol errors.

I am also unable to get sftp(1) to handle such files.  It fails with
an ?unterminated quoted argument? error, no matter how I try to
quote the newline.  Furthermore, the SFTP CLI is not well-suited to
scripted transfers, as it requires creating batch files.  If sftp(1)
could accept commands as command line arguments, or in a NUL-delimited
batch file, that would be great.

While special characters (such as newlines) in filenames are probably
bad practice, I believe that tools like scp(1) and sftp(1) should
be able to deal with them.  At least the SFTP protocol is perfectly
capable of handling newlines in filenames, as I can delete a file
with a newline in its name using Nautilus over SFTP.

In case it matters, the client is a Fedora 30 Linux system, and the
server is running the latest OpenBSD snapshot.

Sincerely,

Demi





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20191103/4368a4b2/attachment.asc>

On 4/11/19 3:29 am, Demi M. Obenour wrote:
> I have not been able to get scp(1) to download a file with a newline
> in its name.  I know that scp(1) requires that remote filenames be
> escaped for the shell, but that leads to protocol errors.
I see something much worse:

    $ sudo sh
    # echo "#!/bin/sh
    touch /tmp/b-ran" > /tmp/b
    # chmod a+x /tmp/b
    # exit
    $ touch a 'a
    b'
    $ rm -f abc /tmp/b-ran
    $ scp localhost:'a
    b' abc
    a                                                 100%    0     0.0KB/s  
00:00
    $ ls /tmp/b-ran
    /tmp/b-ran

Scp can be exploited to run commands, which I think is a problem.

On Mon, 4 Nov 2019, David Newall wrote:
> On 4/11/19 3:29 am, Demi M. Obenour wrote:
> > I have not been able to get scp(1) to download a file with a newline
> > in its name.  I know that scp(1) requires that remote filenames be
> > escaped for the shell, but that leads to protocol errors.
> 
> I see something much worse:
> 
>    $ sudo sh
>    # echo "#!/bin/sh
>    touch /tmp/b-ran" > /tmp/b
>    # chmod a+x /tmp/b
>    # exit
>    $ touch a 'a
>    b'
>    $ rm -f abc /tmp/b-ran
>    $ scp localhost:'a
>    b' abc
>    a                                                 100%    0     0.0KB/s
> 00:00
>    $ ls /tmp/b-ran
>    /tmp/b-ran
> 
> Scp can be exploited to run commands, which I think is a problem.
scp isn't being exploited, you're just running commands in your own
remote shell and all interpretation happens there - scp never sees
those characters.

This "feature" dates back to rcp from 1981 (scp is rcp in a nicer pair
of shoes) and isn't fixable without breaking scp's "protocol".

-d