openssh unix dev - Dec 2000 - openssh 2.2, fbsd 4.2: incoming data hangs sshd on tty

This message is reporting 2 possible bugs:
  a possible problem with sshd from OpenSSH 2.2 on FreeBSD 4.2, and
  a possible security hole in FreeBSD's boot process (??)

I am not an expert on these kinds of issues and therefore do not have the
knowledge necessary to trace the origins of the problem to the extent that
would probably be helpful. Nevertheless, here goes:

Here is the physical environment:
 - dual Pentium 133 box running a fresh install of FreeBSD 4.2-RELEASE
 - OpenSSH 2.2 installed as a package at O/S install time
 - 2 NICs in the box; natd running to provide a LAN with Internet access
 - net connection is 256 kbps DSL

Here is the problem:

sshd, specifically the forked sshd process that is attached to a terminal
when a connection is made, tends to freeze when receiving data over the link.
The only way out is to kill -9 this process. It is easily reproducible by
pasting text into an editor.

It is reproducible using different SSH clients, and it does not matter if I
am connecting from the LAN or from the Internet, thus ruling out the
possibility of it being related to one of the network cards or the speed of
the connection (the DSL connection being much slower than the LAN).

This happens often enough that I can't do much of anything with my system
unless I sit at the console or use insecure telnet.

What can I do to diagnose this problem further?

If someone on the OpenSSH project wants access to my box, contact me
privately and that can be arranged.

...

Now for the security problem. While trying to diagnose the problem, I tried
running sshd in debug mode. To do this I put sshd_flags="-d" in
/etc/rc.conf,
and rebooted with 'shutdown -r now'.

Various debug messages appeared in the console when I SSH'd in, but the only
message of interest was "debug: tvp!=NULL kid 0 mili 10", which was
repeated
a number of times. However, neither this message nor any other appeared when
I pasted some text to induced the freeze-up.

Since it didn't look like debug mode was telling me anything useful, I
decided to reboot again and not be in debug mode. I left the child sshd hung
and telnetted in, since the main sshd, in debug mode, only allows one
connection by design.

I removed the extra line from /etc/rc.conf and issued another 'shutdown -r
now'. I got a console message saying shutdown: reboot by mike, and then ...
nothing! So from my telnet session I killed the hung sshd child.

This is the really scary part. The machine apparently thought that it had
already gone through the initial boot stages. The next messages that came up
on my console were typical of what you see at the last stages of a bootup:

.
Initial rc.i386 initialization:.
rc.i386 configuring syscons: keyrate font8x16 font8x14 font8x8 blank_time
screensaver.
additional ABI support: linux.
Local package initialization: apache dnetc.
Additional TCP options:.
Raising kernel security level
kern.securelevel: -1 -> 2
Mon Dec 25 17:05:10 MST 2000

followed by a login prompt.

...All this *without ever dropping the telnet connection*.

A few minutes later, I lost all Internet connectivity. The LAN and telnet
connection stayed up. A subsequent 'shutdown -r now' achieved a full
reboot
and the Internet was again accessible.

Is this normal?!? Did I find some kind of security hole?

What's interesting is that the console messages on the false reboot are what
you see right after the line that normally says

starting standard daemons: inetd cron sendmail sshd.

Note that the period did appear in the false reboot.

Well I hope this is of some use to someone. Please contact me for any further
info. I'm not on the openssh-unix-dev list but I am on the freebsd-questions
list.

Thanks,

   - Mike
________________________________________________________________________
 Mike Brown / Hyperreal   |   Hyperreal http://music.hyperreal.org/
 PO Box 61334             |   XML & XSL http://skew.org/xml/
 Denver CO 80206-8334 USA |    personal http://www.hyperreal.org/~mike/

I wrote:
> sshd, specifically the forked sshd process that is attached to a terminal
> when a connection is made, tends to freeze when receiving data over the
link.
> The only way out is to kill -9 this process. It is easily reproducible by
> pasting text into an editor.
With the help of someone who advised me to run ktrace on the sshd process
that freezes, I have narrowed down the circumstances under which the problem
is reproducible, and I discovered that sshd does receive a little bit more
data from the paste than I get to see coming back from the editor.

It seems to have to do with whatever the pico (yes, pico, old habits die
hard) editor is doing when it is receiving keystrokes and writing out to the
screen. General screen repaints (ctrl-L) are fine; as are individual
keystrokes. But pasting into the editor causes the ssh freeze-up. As I
mentioned, there is no way to get out of it; killing the editor or the shell
don't work; the processes stick around waiting for sshd to let them go or
something. I have to kill the sshd process.

Contrary to earlier test results, I am unable to reproduce the problem
outside of pico. Pasting the same text into an echo "..." command in
the
shell multiple times yielded no problems.

Tail end of the kdump follows. Note that it does not RET from the write() at
the end:

   407 sshd     CALL  select(0x6,0xbfbff39c,0xbfbff31c,0,0)
   407 sshd     RET   select 1
   407 sshd     CALL  read(0x4,0xbfbfb2dc,0x4000)
   407 sshd     GIO   fd 4 read 156 bytes
      
"\0\0\0>?\M^X??A?8\M^Wl\M^_<s?0?\^B?H\M^A0?\^W"AU\^\\M^]?\fP-?Cfc?\240?\
        ?\M^Am<\^P\M^Y?o??oB\M^U?\^N?Ed\^W?\^O?F0?p\0\0\0
       
|??x?\M^]????E??<-]\0\0\0>?????\aM???4?1?>?\M^M?4\^C??\M^@w?\M^I?Q\^\?\
        ?o\M^H\M^YY?A?{=l,e???\fb{?|\M^F\^T???u\M^_\M^S????V"
   407 sshd     RET   read 156/0x9c
   407 sshd     CALL  select(0x6,0xbfbff39c,0xbfbff31c,0,0)
   407 sshd     RET   select 1
   407 sshd     CALL  read(0x5,0xbfbfb2dc,0x4000)
   407 sshd     GIO   fd 5 read 8 bytes
       "\^[[19;64H"
   407 sshd     RET   read 8
   407 sshd     CALL  select(0x6,0xbfbff39c,0xbfbff31c,0,0xbfbff2d4)
   407 sshd     RET   select 1
   407 sshd     CALL  write(0x3,0x8082000,0x6ea)

Don't know if this is helpful or not.

I wrote:
> Contrary to earlier test results, I am unable to reproduce the problem
> outside of pico. Pasting the same text into an echo "..." command
in the
> shell multiple times yielded no problems.
*sigh* Again I must amend this report. I just got a classic lock-up outside
of pico, when elm 2.4ME+ PL84 (like I said, old habits..) was displaying a
list of messages in a mailbox. So the sshd problem is not just restricted to
certain text editors.

   - Mike
________________________________________________________________________
 Mike Brown / Hyperreal   |   Hyperreal http://music.hyperreal.org/
 PO Box 61334             |   XML & XSL http://skew.org/xml/
 Denver CO 80206-8334 USA |    personal http://www.hyperreal.org/~mike/