openssh unix dev - Dec 2000 - openssh-SNAP-20001207 scp "Bad file descriptor" sort-of work-around

Everything seems to work, though I still have trouble with scp.  The
behaviour is not protocol dependent - acts the same with either.  (Using
RedHat 6.2)

When I scp <file> <host>:<dir>, bash is invoked as a login
shell on the
server side host - this doesn't seem right.  With my /etc/profile, I get

 Dec  6 15:06:20 amethyst sshd[3027]: error: select: Bad file descriptor

in the log (something doesn't like my script), and

 sourcing /etc/profile...
 Connection to <host> closed by remote host.

on the client side.  (My /etc/profile generates the "sourcing
/etc/profile..." message.)

If instead I mv /etc/profile /etc/profile.tmp, scp is successful.  strace
then gives, toward the end:

 open("/etc/passwd", O_RDONLY)           = 3
 fcntl(3, F_GETFD)                       = 0
 fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
 fstat(3, {st_mode=S_IFREG|0644, st_size=2335, ...}) = 0
 old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
 0) = 0x40015000
 read(3, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 2335
 close(3)                                = 0
 munmap(0x40015000, 4096)                = 0
 ioctl(2, TCGETS, 0xbffff734)            = -1 EINVAL (Invalid argument)
 rt_sigaction(SIGPIPE, {0x804b0bc, [], SA_RESTART|0x4000000}, {SIG_DFL}, 8)
 = 0
 pipe([3, 4])                            = 0
 pipe([5, 6])                            = 0
 pipe([7, 8])                            = 0
 close(3)                                = 0
 close(4)                                = 0
 fork()                                  = 21936
 close(5)                                = 0
 close(8)                                = 0
 read(7, bash: /etc/profile: No such file or directory
 bash: cma: command not found
 "\0", 1)                        = 1
 open("ssh-copy-id", O_RDONLY)           = 3
 fstat(3, {st_mode=S_IFREG|0644, st_size=1144, ...}) = 0
 write(6, "C0644 1144 ssh-copy-id\n", 23) = 23
 read(7, "\0", 1)                        = 1
 fstat(3, {st_mode=S_IFREG|0644, st_size=1144, ...}) = 0
 brk(0x8051000)                          = 0x8051000
 read(3, "#!/bin/sh\n\n# Shell script to ins"..., 1144) = 1144
 write(6, "#!/bin/sh\n\n# Shell script to ins"..., 1144) = 1144
 close(3)                                = 0
 write(6, "\0", 1)                       = 1
 read(7, "\0", 1)                        = 1
 _exit(0)

("cma" is a missing command from /etc/profile called from ~/.profile.
"ssh-copy-id" is the test file I'm trying to scp.)

When scp breaks, the very end looks like:

 read(7, ".", 1)                         = 1
 read(7, "\n", 1)                        = 1
 write(2, "sourcing /etc/profile...\n", 25sourcing /etc/profile...
 ) = 25
 _exit(1)                                = ?
 Connection to <host> closed by remote host.


So, is starting a bash login shell correct behavior for scp?

This login shell behaviour is a problem, as you can see.  An incompatibility
with /etc/profile or ~/.profile kills scp - uhg.

Might I have misconfigured something in ssh_config or sshd_config? or
compiling the package?  I have:

 OpenSSH configured has been configured with the following options.
                 User binaries: /usr/bin
                 User binaries: /usr/bin
               System binaries: /usr/sbin
           Configuration files: /etc/ssh
               Askpass program: /usr/libexec/ssh-askpass
                  Manual pages: /usr/man/manX
                      PID file: /var/run
      Random number collection: Device (/dev/urandom)
                Manpage format: man
                   PAM support: yes
            KerberosIV support: no
                   AFS support: no
                 S/KEY support: yes
          TCP Wrappers support: yes
          MD5 password support: no
   IP address in $DISPLAY hack: no
      Use IPv4 by default hack: no
       Translate v4 in v6 hack: yes

             Host: i686-pc-linux-gnu
         Compiler: gcc
   Compiler flags: -g -O2 -Wall -I. -I. -I/usr/include
     Linker flags:  -L/usr/lib -L/usr
        Libraries: -ldl -lnsl -lz  -lutil -lpam -lcrypto  -lskey -lwrap


Please Cc me on any response, as I'm not subscribed to the list.  Thanks.


James Feeney

On Wed, 6 Dec 2000, James wrote:
> Everything seems to work, though I still have trouble with scp.  The
> behaviour is not protocol dependent - acts the same with either.  (Using
> RedHat 6.2)
> 
> When I scp <file> <host>:<dir>, bash is invoked as a
login shell on the
> server side host - this doesn't seem right.  With my /etc/profile, I
get

Do you have something in your profile or shell rc which causes text to 
be displayed on non-interactive logins? 

What does "ssh localhost true" say?

You mustn't have junk being printed for non-interactive sessions.

-d
>  Dec  6 15:06:20 amethyst sshd[3027]: error: select: Bad file descriptor
> 
> in the log (something doesn't like my script), and
> 
>  sourcing /etc/profile...
>  Connection to <host> closed by remote host.
> 
> on the client side.  (My /etc/profile generates the "sourcing
> /etc/profile..." message.)
> 
> If instead I mv /etc/profile /etc/profile.tmp, scp is successful.  strace
> then gives, toward the end:
> 
>  open("/etc/passwd", O_RDONLY)           = 3
>  fcntl(3, F_GETFD)                       = 0
>  fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
>  fstat(3, {st_mode=S_IFREG|0644, st_size=2335, ...}) = 0
>  old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>  0) = 0x40015000
>  read(3, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 2335
>  close(3)                                = 0
>  munmap(0x40015000, 4096)                = 0
>  ioctl(2, TCGETS, 0xbffff734)            = -1 EINVAL (Invalid argument)
>  rt_sigaction(SIGPIPE, {0x804b0bc, [], SA_RESTART|0x4000000}, {SIG_DFL}, 8)
>  = 0
>  pipe([3, 4])                            = 0
>  pipe([5, 6])                            = 0
>  pipe([7, 8])                            = 0
>  close(3)                                = 0
>  close(4)                                = 0
>  fork()                                  = 21936
>  close(5)                                = 0
>  close(8)                                = 0
>  read(7, bash: /etc/profile: No such file or directory
>  bash: cma: command not found
>  "\0", 1)                        = 1
>  open("ssh-copy-id", O_RDONLY)           = 3
>  fstat(3, {st_mode=S_IFREG|0644, st_size=1144, ...}) = 0
>  write(6, "C0644 1144 ssh-copy-id\n", 23) = 23
>  read(7, "\0", 1)                        = 1
>  fstat(3, {st_mode=S_IFREG|0644, st_size=1144, ...}) = 0
>  brk(0x8051000)                          = 0x8051000
>  read(3, "#!/bin/sh\n\n# Shell script to ins"..., 1144) = 1144
>  write(6, "#!/bin/sh\n\n# Shell script to ins"..., 1144) = 1144
>  close(3)                                = 0
>  write(6, "\0", 1)                       = 1
>  read(7, "\0", 1)                        = 1
>  _exit(0)
> 
> ("cma" is a missing command from /etc/profile called from
~/.profile.
> "ssh-copy-id" is the test file I'm trying to scp.)
> 
> When scp breaks, the very end looks like:
> 
>  read(7, ".", 1)                         = 1
>  read(7, "\n", 1)                        = 1
>  write(2, "sourcing /etc/profile...\n", 25sourcing
/etc/profile...
>  ) = 25
>  _exit(1)                                = ?
>  Connection to <host> closed by remote host.
> 
> 
> So, is starting a bash login shell correct behavior for scp?
> 
> This login shell behaviour is a problem, as you can see.  An
incompatibility
> with /etc/profile or ~/.profile kills scp - uhg.
> 
> Might I have misconfigured something in ssh_config or sshd_config? or
> compiling the package?  I have:
> 
>  OpenSSH configured has been configured with the following options.
>                  User binaries: /usr/bin
>                  User binaries: /usr/bin
>                System binaries: /usr/sbin
>            Configuration files: /etc/ssh
>                Askpass program: /usr/libexec/ssh-askpass
>                   Manual pages: /usr/man/manX
>                       PID file: /var/run
>       Random number collection: Device (/dev/urandom)
>                 Manpage format: man
>                    PAM support: yes
>             KerberosIV support: no
>                    AFS support: no
>                  S/KEY support: yes
>           TCP Wrappers support: yes
>           MD5 password support: no
>    IP address in $DISPLAY hack: no
>       Use IPv4 by default hack: no
>        Translate v4 in v6 hack: yes
> 
>              Host: i686-pc-linux-gnu
>          Compiler: gcc
>    Compiler flags: -g -O2 -Wall -I. -I. -I/usr/include
>      Linker flags:  -L/usr/lib -L/usr
>         Libraries: -ldl -lnsl -lz  -lutil -lpam -lcrypto  -lskey -lwrap
> 
> 
> Please Cc me on any response, as I'm not subscribed to the list. 
Thanks.
> 
> 
> James Feeney
> 
> 
> 
-- 
| ``We've all heard that a million monkeys banging on | Damien Miller -
| a million typewriters will eventually reproduce the | <djm at
mindrot.org>
| works of Shakespeare. Now, thanks to the Internet, / 
| we know this is not true.'' - Robert Wilensky UCB /
http://www.mindrot.org

Damien wrote:
> Do you have something in your profile or shell rc which causes text to
> be displayed on non-interactive logins?
No, but there's part of where the problem is.  scp seems to read /etc/passwd
on the _target_ server machine (as well as on the client machine) to
determine the user's default shell on that target machine, and then invokes
a non-login _interactive_ shell, NOT a non-interactive shell.

I know this from changing the various scripts and the /etc/passwd login
shell for the user (root). Where man bash has

INVOCATION
       A login shell is one whose  first  character  of  argument
       zero is a -, or one started with the -login flag.

       An  interactive shell is one whose standard input and out
       put are both connected  to  terminals  (as  determined  by
       isatty(3)), or one started with the -i option.  PS1 is set
       and $- includes i if bash is interactive, allowing a shell
       script or a startup file to test this state.

       Login shells:
         On login (subject to the -noprofile option):
               if /etc/profile exists, source it.

               if ~/.bash_profile exists, source it,
                 else if ~/.bash_login exists, source it,
                   else if ~/.profile exists, source it.

         On exit:
               if ~/.bash_logout exists, source it.

       Non-login interactive shells:
         On startup (subject to the -norc and -rcfile options):
               if ~/.bashrc exists, source it.

My ~/.bashrc normally starts with "source /etc/profile", which was
what
actually generated the text, and the "sshd[3381]: error: select: Bad file
descriptor" message in the log file.  (I noticed that if I simply put echo
"a bunch of random characters asdlfkjsd" into ~/.bashrc, that will
also kill
scp, but - and only in this particular case - without the "Bad file
descriptor" error message.)

I suppose I can try to modify ~/.bashrc to check how it's being called,
but...

Could this be a problem with bash? or a problem with how sshd calls the
login shell?

 # echo $BASH_VERSION
 1.14.7(1)

> What does "ssh localhost true" say?
On the client machine:

# ssh localhost true
ssh_exchange_identification: Connection closed by remote host

with "sshd[25040]: refused connect from 127.0.0.1" in the log file.

On the server machine, the same.  Also the same if 127.0.0.1 is added to
/etc/hosts.allow for sshd.  Also the same for a non root user.

> You mustn't have junk being printed for non-interactive sessions.
What's the easiest way to invoke a for-sure non-interactive session for
testing?

Again, please Cc me on any response, as I'm not subscribed to the list.
Thanks.


James Feeney

> From: Damien Miller <djm at mindrot.org>
> To: james at nurealm.net
> Cc: openssh-unix-dev at mindrot.org
> 
> On Thu, 7 Dec 2000, James wrote:
> 
> > Damien wrote:
> > 
> > > Do you have something in your profile or shell rc which causes
text to
> > > be displayed on non-interactive logins?
> > 
> > No, but there's part of where the problem is. scp seems to read
> > /etc/passwd on the _target_ server machine (as well as on the client
> > machine) to determine the user's default shell on that target
> > machine, and then invokes a non-login _interactive_ shell, NOT a
> > non-interactive shell.
> >
> > I know this from changing the various scripts and the /etc/passwd
> > login shell for the user (root). Where man bash has
> >
> > My ~/.bashrc normally starts with "source /etc/profile",
which
> > was what actually generated the text, 
> 
> This is your problem here: profile should only be read for interactive
sessions - the shell does this automatically for you, you should not
> source it in bashrc.
> 
> Your shell initialisation for non-interactive sessions must not produce
> any output. This requirement isn't unique to scp, all the tools layered
> on top of rsh have it too.
Which is why any shell rc should check, if you have it set up to 
do silly things like that :)

Example for csh :

if ($?USER == 0 || $?prompt == 0) exit

Damien wrote:
> This is your problem here: profile should only be read for interactive
> sessions - the shell does this automatically for you, you should not
> source it in bashrc.
>
> Your shell initialisation for non-interactive sessions must not produce
> any output. This requirement isn't unique to scp, all the tools layered
on
> top of rsh have it too.
Ok, I'm having trouble with the language here.  You talk about interactive
and non-interactive sessions, but there are, in fact, for bash:

  a) Login-interactive shells
  b) Non-login-interactive shells
  c) Non-interactive shells

You say "Your shell initialisation for non-interactive sessions must not
produce any output."  Well, it does not, really, none at all, zip.  And
still, scp breaks.  scp is NOT starting a _non-interactive_ shell.  My
question was: Is scp _suppose_ to start a _non-interactive_ shell?  If so,
scp is broken, simple as that.  If not, then let's stop talking about
_non-interactive_ sessions.

I call /etc/profile from ~/.bashrc because I start Non-login-interactive
shells on my virtual terminals, and I want the same initialization as for my
single Login-interactive shell.  Copying the whole /etc/profile into
~/.bashrc seems a silly solution.  Now, if scp is NOT suppose to start a
_non-interactive_ shell, and, in fact, is supposed to start a non-login
_interactive_ shell, then we'll all have to deal with that.

Not much point in deciding what to do without knowing if scp is broken,
other than to suggest that scp ought to be smart enough to toss a little
extraneous text.

So, is scp broken, or is it _suppose_ to invoke an _interactive_ session?

Again, please Cc me, as I'm not subscribed to the list.

Thanks.


James Feeney

Richard wrote
> Under this definition, scp is indeed starting a non-interactive shell: it
> runs ssh in a subprocess, to run scp in server mode as a remote command on
> the other side. SSH does not allocate a pty on the remote side for
> commands (unless forced to with the -t option).  Your shell is run as
> "<shell> -c scp ..." to start the remote scp, and isatty(0)
will return
> false.
Ok, this helps.  All I know for sure is that bash-1.14.7(1) is calling
~/.bashrc, since any output from ~/.bashrc is returned to scp.
>From the bash man page, the only shell mode from which bash will call
~/.bashrc is from:

       Non-login interactive shells:
         On startup (subject to the -norc and -rcfile options):
               if ~/.bashrc exists, source it.
>> You say "Your shell initialisation for non-interactive sessions
must not
>> produce any output."  Well, it does not, really, none at all, zip.
> How have you tested this?
I have not tested this - I'm making a rash assumption.  I had asked earlier
"What's the easiest way to invoke a for-sure non-interactive session
for
testing?", but the distinction is moot until resolving the mode in which
bash is running.
>> And still, scp breaks.  scp is NOT starting a _non-interactive_ shell.
> What definition of "non-interactive shell" are you using?
Only indirectly, the definition quoted above (and previously) which describes
bash's "Non-login interactive shells" and, again indirectly, the
definition,
also from the man page, of bash's "Non-interactive shells".

       Non-interactive shells:
         On startup:
               if the environment variable ENV is non-null, expand
               it and source the file it names, as if the command
                       if [ "$ENV" ]; then . $ENV; fi
               had been executed, but do not use PATH to search
               for the pathname.  When not started in Posix mode, bash
               looks for BASH_ENV before ENV.

I'm assuming that a non-interactive shell is a shell that is not an
interactive shell and that the definition of "Non-interactive shells"
is
exclusive (meaning "does NOT also source ~/.bashrc"). My ENV is empty
or
unset, and I'm assuming scp does not set ENV or BASH_ENV.

If I understand you (and Damien), since the method used to invoke bash is
definitely _not_ "one whose standard input and output are both connected 
to
terminals  (as  determined  by isatty(3)), or one started with the -i
option", and thereby, also according bash(1), _not_ a method invoking an
"interactive shell", therefor, either bash-1.14.7(1) is broken or
bash(1) is
broken.

Does that seem to be a fair interpretation?

Again, please Cc me, as I'm not subscribed to the list.

Thanks.


James Feeney

Damien had asked:
> What does "ssh localhost true" say?
Before, I was getting:

# ssh localhost true
ssh_exchange_identification: Connection closed by remote host

Seems I screwed up my hosts.deny with an extra linefeed.  For the record,
the actual response should have been:

# ssh localhost true
Unable to find an authentication method

With the syslog

Dec  8 12:01:31 amethyst sshd[3985]: connect from local.host
Dec  8 12:01:39 amethyst sshd[3985]: Connection closed by 127.0.0.1
Dec  8 12:01:39 amethyst PAM_pwdb[3985]: (sshd) session closed for user james
Dec  8 12:01:39 amethyst inetd[130]: pid 3985: exit status 255

Again, please Cc me on any response, as I'm not subscribed to the list.

Thanks.


James Feeney

Ok, I'm starting to get a handle.

Damien had asked:
> What does "ssh localhost true" say?
Finally, I went back and cleaned up ~/.ssh/config and
~/.ssh/authorized_keys2.  So now that "ssh localhost" actually works,
I get:

 $ ssh localhost true
 Enter passphrase for key '/home/james/.ssh/id_dsa_jkey':
 sourcing /etc/profile...
 Running Linux 2.2.16-3.
 Fri Dec  8 16:36:21 MST 2000
 not a tty
 select: Bad file descriptor

(You can see the output of my "tty" command there.)

Now, I tried using Bash 2.03.8(1)-release from RedHat 6.2 and noted that it
acts the same as Bash 1.14.7(1).  I then downloaded Bash 2.04 and read the
man page.  Very interesting:

INVOCATION
 .
 .
 .

       Bash  attempts  to  determine  when it is being run by the
       remote shell daemon, usually rshd.  If bash determines  it
       is  being run by rshd, it reads and executes commands from
       ~/.bashrc, if that file exists and is readable.   It  will
       not  do  this  if invoked as sh.  The --norc option may be
       used to inhibit this behavior, and the --rcfile option may
       be  used  to  force another file to be read, but rshd does
       not generally invoke the shell with those options or allow
       them to be specified.

Ugh!  I'd think bash should read different startup files when invoked by a
remote shell daemon and when invoked as a non interactive shell.

Is sending random text to a remote shell daemon really such a bad thing that
ssh and scp couldn't be made to simply pass through the text?

Again, please Cc me on any response, as I'm not subscribed to the list.

Thanks.


James Feeney

On Fri, 8 Dec 2000, Richard wrote:
>> Ugh!  I'd think bash should read different startup files when
invoked by a
>> remote shell daemon and when invoked as a non interactive shell.
> How would you suggest it tell the difference?
As I understand, bash already can tell the difference - that's why BASH(1)
says:

       Bash  attempts  to  determine  when it is being run by the
       remote shell daemon, usually rshd.

That's part of the reason I had a problem with scp to start with.

bash-2.04/shell.c has:
---
/* The name of the .(shell)rc file. */
static char *bashrc_file = "~/.bashrc";
---
/* Execute ~/.bashrc for most shells.  Never execute it if
   ACT_LIKE_SH is set, or if NO_RC is set.

   If the executable file "/usr/gnu/src/bash/foo" contains:

   #!/usr/gnu/bin/bash
   echo hello

   then:

         COMMAND            EXECUTE BASHRC
         --------------------------------
         bash -c foo            NO
         bash foo               NO
         foo                    NO
         rsh machine ls         YES (for rsh, which calls `bash -c')
         rsh machine foo        YES (for shell started by rsh) NO (for foo!)
         echo ls | bash         NO
         login                  NO
         bash                   YES
*/
---
static void
run_startup_files ()
{
#if defined (JOB_CONTROL)
  int old_job_control;
#endif
  int sourced_login, run_by_ssh;

  /* get the rshd/sshd case out of the way first. */
  if (interactive_shell == 0 && no_rc == 0 && login_shell == 0
&&
      act_like_sh == 0 && local_pending_command)
    {
      run_by_ssh = find_variable ("SSH_CLIENT") != (SHELL_VAR *)0;
      run_by_ssh |= find_variable ("SSH2_CLIENT") != (SHELL_VAR *)0;

      /* If we were run by sshd or we think we were run by rshd, execute
         ~/.bashrc if we are a top-level shell. */
      if ((run_by_ssh || isnetconn (fileno (stdin))) && shell_level <
2)
        {
#ifdef SYS_BASHRC
#  if defined (__OPENNT)
          maybe_execute_file (_prefixInstallPath(SYS_BASHRC, NULL, 0), 1);
#  else
          maybe_execute_file (SYS_BASHRC, 1);
#  endif
#endif
          maybe_execute_file (bashrc_file, 1);
          return;
        }
    }

I suppose the trick is to simply replace

          maybe_execute_file (bashrc_file, 1);

with something like

          maybe_execute_file ("~/.bash_rsh", 1);

> A non-interactive session is probably one program talking to another, and
> the random text simply screws up whatever protocol they're speaking. 
It's
> more reasonable to expect a transparent, unpolluted connection, than to
> expect that every pair of programs you might want to connect together
> using SSH must try to heuristically filter out stuff that "looks like
> unwanted text" from their conversation.
I can see your point.  Still, I'm not suggesting a need for any kind of
heuristic filter.  Rather, I don't see any need for scp, in particular, to
be either the enforcer or the victim of a policy expecting a transparent,
unpolluted connection.  scp is simply not being "robust" here.  There
doesn't seem to be any _need_ to have scp fail crypticly.

Though I think bash needs to distinguish between "rshd" and
"non-interactive" startup files, I also think robust behavior for scp
would
be to simply display or trash, depending on the terminal status, any
inconsequential returned text.  (Remember that fierce "suicide squad"
at the
end of "Life of Brian"?  scp doesn't _need_ to be a member of the
"suicide
squad", does it?)

Again, please Cc me on any response, as I'm not subscribed to the list.

Thanks.


James Feeney