CentOS - Apr 2014 - Ulimit problem - CentOS 5.10

Running across some curious stuff with ulimit on CentOS 5.10.

We have a non CentOS packaged version of Asterisk (using their packages) that we
start at boot time with a typical RC script.

Recently it started whining that it couldn't open enough file handles.

As we dug further into this, it appears that at boot time, it inherits ulimit
from init, which is pretty low: 1024.

We've set /etc/security/limits.conf and sysctl significantly higher both for
root/system wide and also for the user Asterisk runs under, to no avail.

If we log in via ssh as root (or sudo) the correct root/system-wide ulimit of
8192 is set.

Looking in /proc/[PID]/limits shows the lower ulimit only if the package is
started from init (standard rc3.d type sysV stuff...).  If we restart it via
console, remote ssh, anything else, the limit bumps to 8196.

Attempting to force the ulimit up inside the RC script has no effect, since the
package is running as a non-root user.  It fails to raise the limit.

Right now, we're not totally against just taking it out of the startup and
starting it manually anyway, since we really don't want the Asterisk
platform coming up after a crash/reboot anyway, and any other reboots there will
always be a human involved, but the way init is handling ulimit seems utterly
retarded and broken.

Some indication (different engineer found it, I haven't seen the RHEL case
number) appears to indicate that folks wanted init ulimited heavily in case of
startup DDoS type stuff, but we haven't figured out a semi-sane
unix-conventional type way AROUND this when it's needed that if we were hit
by the proverbial bus, a "normal" unix guy would find.

Perhaps we're missing something.

Thoughts?  

--
Nate Duehr
denverpilot at me.com

On Wed, Apr 23, 2014 at 5:19 PM, Nathan Duehr <denverpilot at me.com>
wrote:
> Running across some curious stuff with ulimit on CentOS 5.10.
>
> We have a non CentOS packaged version of Asterisk (using their packages)
> that we start at boot time with a typical RC script.
>
> Recently it started whining that it couldn't open enough file handles.
>
> As we dug further into this, it appears that at boot time, it inherits
> ulimit from init, which is pretty low: 1024.
>
Yep, it's rather low.

>
> We've set /etc/security/limits.conf and sysctl significantly higher
both
> for root/system wide and also for the user Asterisk runs under, to no
avail.
>
I've had success making changes in limits.conf (for MySQL in my case).
In my case I increased both the system wide and per user (mysql user).

Example:
<my_user> hard nofile 2048
<my_user> soft nofile 2048

SSH to that box (as <my_user>) ... run ulimit -Hn and -Sn ... bingo 2048

Are you able to switch users to the user your Asterisk user to run `ulimit
-Hn` and `ulimit -Sn`?
Once you've verified new sessions get the proper limits, stop and then
start your Asterisk daemon.  Hopefully from there it will stop whining
about file handles.

>
> If we log in via ssh as root (or sudo) the correct root/system-wide ulimit
> of 8192 is set.
>
> Looking in /proc/[PID]/limits shows the lower ulimit only if the package
> is started from init (standard rc3.d type sysV stuff...).  If we restart it
> via console, remote ssh, anything else, the limit bumps to 8196.
>
Interesting - I've not seen that odd behavior.  It worked across the board
(initscript and all) when I made changes via limits.conf.

>
> Attempting to force the ulimit up inside the RC script has no effect,
> since the package is running as a non-root user.  It fails to raise the
> limit.
>
> Right now, we're not totally against just taking it out of the startup
and
> starting it manually anyway, since we really don't want the Asterisk
> platform coming up after a crash/reboot anyway, and any other reboots there
> will always be a human involved, but the way init is handling ulimit seems
> utterly retarded and broken.
>
> Some indication (different engineer found it, I haven't seen the RHEL
case
> number) appears to indicate that folks wanted init ulimited heavily in case
> of startup DDoS type stuff, but we haven't figured out a semi-sane
> unix-conventional type way AROUND this when it's needed that if we were
hit
> by the proverbial bus, a "normal" unix guy would find.
>
> Perhaps we're missing something.
>
> Thoughts?
>
> --
> Nate Duehr
> denverpilot at me.com
>
>
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>


-- 
---~~.~~---
Mike
//  SilverTip257  //

Nathan Duehr wrote:
>
> Attempting to force the ulimit up inside the RC script has no effect, since
the package is running
> as a non-root user.  It fails to raise the limit.
init.d scripts run as root so you should be able to set a hard/soft limit in the
init.d script before the package is started

James Pearson