openssh unix dev - Sep 2015 - stderr pipe not closed when cancelling session on shared connection

Hello,

[this is a repost for an email I tried to send on Friday via
gmane and that never came through]

when investigating an issue with gitolite
(http://thread.gmane.org/gmane.comp.version-control.gitolite/4006)
I realised that when you do

# Create a master connection
ssh -MnNfS ~/.ssh/ctl localhost

# Reuse that shared connection to run a command on the host
ssh -S ~/.ssh/ctl localhost 'cat; yes >&2'

And then press Ctrl-C, sshd does close the stdin and stdout pipe
to the remote shell (so cat returns after seeing eof on stdin),
but not the stderr pipe.

That next "yes" command does fill up the stderr pipe, and
looking at strace output, at the other end of the pipe, sshd is
not reading from it.

If I kill that "yes" command, "bash" hangs again trying to
write
a "job killed" message on stderr.

If I kill "bash", I can see sshd returning from a waitpid(),
doing *one* read (of 16384 bytes) from that stderr pipe (put
doesn't do anything with the data read) and closes that last
pipe.

It seems to me that if sshd is not going to do anything with
that pipe, it should close it as soon as it becomes impossible
to send the data to the client like it does for the stdout pipe.

In the case of gitolite, gitolite was writing a character to
stderr upon EOF to force a SIGPIPE delivery when the ssh
connection is aborted. While that works for a normal ssh
connection, that does't work for one using a shared connection.

Reproduced with:
OpenSSH_6.9p1 Debian-2, OpenSSL 1.0.2d 9 Jul 2015
and
OpenSSH_7.1p1, OpenSSL 1.0.1f 6 Jan 2014


Thanks,
Stephane

On 09/29/2015 06:50 PM, Stephane Chazelas wrote:
> when investigating an issue with gitolite
> (http://thread.gmane.org/gmane.comp.version-control.gitolite/4006)
> I realised that when you do
>
> # Create a master connection
> ssh -MnNfS ~/.ssh/ctl localhost
>
> # Reuse that shared connection to run a command on the host
> ssh -S ~/.ssh/ctl localhost 'cat; yes >&2'
>
> And then press Ctrl-C, sshd does close the stdin and stdout pipe
> to the remote shell (so cat returns after seeing eof on stdin),
> but not the stderr pipe.
>
> That next "yes" command does fill up the stderr pipe, and
> looking at strace output, at the other end of the pipe, sshd is
> not reading from it.
>
> If I kill that "yes" command, "bash" hangs again trying
to write
> a "job killed" message on stderr.
>
> If I kill "bash", I can see sshd returning from a waitpid(),
> doing *one* read (of 16384 bytes) from that stderr pipe (put
> doesn't do anything with the data read) and closes that last
> pipe.
>
> It seems to me that if sshd is not going to do anything with
> that pipe, it should close it as soon as it becomes impossible
> to send the data to the client like it does for the stdout pipe.
>
> In the case of gitolite, gitolite was writing a character to
> stderr upon EOF to force a SIGPIPE delivery when the ssh
> connection is aborted. While that works for a normal ssh
> connection, that does't work for one using a shared connection.
>
> Reproduced with:
> OpenSSH_6.9p1 Debian-2, OpenSSL 1.0.2d 9 Jul 2015
> and
> OpenSSH_7.1p1, OpenSSL 1.0.1f 6 Jan 2014
The issue is wider than you describe. There is bug #52 [1] which was 
causing hang for a long time and fix for this issue brought your 
described behavior and bug #2071 [2], where is similar reproducer even 
without connection multiplexing. I was trying to figure out what is 
going on there and what could be the best solution, but so far without 
any great success.

[1] https://bugzilla.mindrot.org/show_bug.cgi?id=52
[2] https://bugzilla.mindrot.org/show_bug.cgi?id=2071

-- 
Jakub Jelen
Security Technologies
Red Hat