I'm not on this mailing list so please make sure that I'm listed in
any replies.
There seems to be a basic flaw in either AIX pty's or many Linux
applications and sshd falls into this category.
sshd has a routine called openpty and it looks like ssh's version
mimics the version in Linux. (I'm not sure where openpty comes from
-- I'm assuming Linux.) The key to openpty is it opens both the
slave and the master side of the pty.
sshd calls openpty before the fork of the child so the parent has
both file descriptors open. After the fork, it closes the file
descriptor to the slave. The problem is that if the child generates
output and exits before the parent can close its file descriptor to
the slave then the parent's file descriptor becomes the last file
descriptor to the slave. This implies that the close will go down to
the device driver.
On AIX, if there is data to be read on the ptc (the master side) of
the pty, then the slave device driver will sleep in the close
routine. It has been countless years since I have looked at the BSD
code but I'm pretty sure this is what the BSD code did as well. I
can't say if I have ever looked at the AT&T 5.4 code. AT&T prior to
5.4 did not have pty's (as I recall). To me, the BSD code defines
how pty's work. There are also various Posix rules but they change
so often I can't keep up with them. And, the behavior seems logical
to me. The slave side of a pty is suppose to look just like a tty
and a tty will not close until all of the data has drained out
(assuming various other things like DCD is still high, etc).
I see in at least some of the paths through the ssh code, the child
opens the slave side of the pty -- which it needs to do in order to
set up the process sessions. But what I can not tell with the short
time I've looked at the code if the code always opens the slave side
(when there is a pty involved at all). As far as I can tell, there
are three basic paths to be concerned with:
1) just a normal ssh to a host
2) an ssh to a host that includes a command
3) an ssh to a host that includes a command and also uses the -t flag
If the child always opens the slave side, then it should be possible
to close the slaves fd in the parent before the fork. Utopia would
be to never open the slave in the parent in the first place. That is
where I'm somewhat confused. If all of the Linux applications like
telnetd, rlogind, etc do the same thing, then it seems like they
would all suffer from this problem. But, I assume that is not the
case so maybe a change to the AIX pty code could be done (but I don't
see how -- which is why I'm asking).
So, my two basic questions are: 1) are the three paths through the
code I listed above a complete list that I need to be looking at and
2) can anyone comment on how the Linux pty's work?
Thank you,
Perry Smith ( pedz at easesoftware.com )
Ease Software, Inc. ( http://www.easesoftware.com )
Low cost SATA Disk Systems for IBMs p5, pSeries, and RS/6000 AIX systems
