[Bug 2000] New: when using ssh with ControlMaster/ControlPersist, one may get zombie processes

bugzilla-daemon at bugzilla.mindrot.org
Wed, 25 Apr 2012 12:50:58 -0000

Bug #: 2000
           Summary: when using ssh with ControlMaster/ControlPersist, one
                    may get zombie processes
    Classification: Unclassified
           Product: Portable OpenSSH
           Version: 5.9p1
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: major
          Priority: P2
         Component: ssh
        AssignedTo: unassigned-bugs at mindrot.org
        ReportedBy: calestyo at scientia.net


Hi.

This is basically from:
https://dev.icinga.org/issues/2546
http://tracker.nagios.org/view.php?id=321

It was suggested there, that the actual problem may be in ssh, so I
open a bug here, too.

I basically asked before already at:
https://lists.mindrot.org/pipermail/openssh-unix-dev/2012-April/030379.html
but no one knew any advise.


What I do is using Icinga/Nagios and having checks on remote hosts
executed via ssh.
In order to dramatically speed checks up (from about 0,300 ms to 0,010
ms) I use ControlMaster = auto, which also makes the mux process
spawned on the first check.
As checks are typically sequentially scheduled I want the mux process
to persist but it should also go away automatically after some days if
not re-used (e.g. when I don't check a host anymore).
So I have something like ControlPersist 2d.

Now I stumbled across the following problem (and I'm actually not sure
whether it's a ssh issue or Icinga/Nagios):
The first time the check is done (which is when the mux process is
spawned) it times out.
The mux process keeps running and everything works on subsequent
checks.

The timeout is one enforced by Icinga/Nagios (60s), when it thinks the
command doesn't return.

I made some checks and the following turns out to happen on the FIRST
connection:
- executing the command on the remote side is actually done
- on the local side, the ssh process (or a wrapper shell script around)
becomes a zombie as soon as the remote command was executed
- after 60s, when Icinga/Nagios enforces it's timeout, the zombie goes
away
- the (local) mux process continues to run

The Zombie process is a child of icinga/nagios, while the mux process
(which is not a zombie) is a child of init.

Any ideas why this could happen? Is there perhaps something that lets
the parent processes notice that there is still a running child (i.e.
the mux process)?

I'll happily try out anything needed :)

Thanks,
Chris.