openssh unix dev - Sep 2001 - pam_limits and OpenSSH

Hi,

I have an account server with many users.  It uses pam_limits module to 
limit memory usage etc.  The problem is that sometimes SSH rejects 
connection after the password is entered.  In syslog it prints something 
like "fork: Resource temporary unavailable".  After killing some root 
processes it works perfectly.

Perhaps the daemon first sets process limits and then switches to the 
user and/or fork().  But fork() cannot succeed because there is a 
process number limit to 40 that is applied to root.  This is my 
hypothesis.  I didn't look at sources.  What you think about all this? 
Do you need more information?  I use Debian GNU/Linux potato and OpenSSH 
1.2.3-9.3.

Regards
-- 
Ognyan Kulev <ogi at fmi.uni-sofia.bg>,
"\"Programmer\""

I suggest you try OpenSSH 2.9p2 release

- Ben

On Wed, 5 Sep 2001, Ognyan Kulev wrote:
> Hi,
>
> I have an account server with many users.  It uses pam_limits module to
> limit memory usage etc.  The problem is that sometimes SSH rejects
> connection after the password is entered.  In syslog it prints something
> like "fork: Resource temporary unavailable".  After killing some
root
> processes it works perfectly.
>
> Perhaps the daemon first sets process limits and then switches to the
> user and/or fork().  But fork() cannot succeed because there is a
> process number limit to 40 that is applied to root.  This is my
> hypothesis.  I didn't look at sources.  What you think about all this?
> Do you need more information?  I use Debian GNU/Linux potato and OpenSSH
> 1.2.3-9.3.
>
> Regards
> --
> Ognyan Kulev <ogi at fmi.uni-sofia.bg>,
"\"Programmer\""
>
>

On Wed, Sep 05, 2001 at 04:53:05PM +0300, Ognyan Kulev
wrote:
> Perhaps the daemon first sets process limits and then switches to the 
> user and/or fork().  But fork() cannot succeed because there is a 
> process number limit to 40 that is applied to root.  This is my 
> hypothesis.  I didn't look at sources.  What you think about all this? 
> Do you need more information?  I use Debian GNU/Linux potato and OpenSSH 
> 1.2.3-9.3.
This is exactly the case.  The process limit is set while the server
is still running as the superuser, so it can't fork() to start the
child (which would then do a setuid() to the user's ID).

Opening the PAM session after performing the fork() and setuid() fixes
this for pam_limits, but breaks other modules which expect to be running
with superuser privileges when their pam_open_session() handlers are
called.  This was the crux of the whole pam_open_session mess from a few
months ago -- my apologies for setting it in motion.

Other process limits are going to have similar effects on sshd, and I
don't see a clean way to handle process limits within PAM in this case.

Hope this cleared things up a bit,

Nalin