Rails - Aug 2006 - WEBrick and Mongrel die on linux unless run as root

I''ve created a straightforward Rails app for a client and it runs great
on my FreeBSD 6 box. When we deploy it on his Redhat 7.2 server, both 
WEBrick and Mongrel start and run fine, but will only stay up for a 
couple of hours. We''ll find them dead with absolutely no information in
the logs. No core dumps. Nothing in /var/log/messages. Just gone. The 
process will die even if idle, so I don''t think there''s
anything in the
actual app that''s causing the problem.

In desperation, I started Mongrel as root and the darn thing is staying 
up. I know zilch about Linux admin. Is there something I''m missing
here?
There are no obvious per-user process limits (memory, CPU, etc.) as far 
as I can tell.

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On Fri, 2006-08-25 at 03:14 +0200, Bob Showalter wrote:
> I''ve created a straightforward Rails app for a client and it runs
great
> on my FreeBSD 6 box. When we deploy it on his Redhat 7.2 server, both 
> WEBrick and Mongrel start and run fine, but will only stay up for a 
> couple of hours. We''ll find them dead with absolutely no
information in
> the logs. No core dumps. Nothing in /var/log/messages. Just gone. The 
> process will die even if idle, so I don''t think there''s
anything in the
> actual app that''s causing the problem.
> 
> In desperation, I started Mongrel as root and the darn thing is staying 
> up. I know zilch about Linux admin. Is there something I''m missing
here?
> There are no obvious per-user process limits (memory, CPU, etc.) as far 
> as I can tell.
----
I''m not gonna even speculate but RHL 7.2 has been EOL for at least 4
years. Why not use CentOS 4 ? 

You don''t say which version of ruby/rails you are using and
I''m
presuming that you are avoiding apache which is bound to be really old
too.

Craig


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Can the user that mongrel is running under access the log files? Read and
write?

Sounds permissions related to me.

> From: Bob Showalter
<rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org>
> Reply-To: <rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>
> Date: Fri, 25 Aug 2006 03:14:38 +0200
> To: <rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>
> Subject: [Rails] WEBrick and Mongrel die on linux unless run as root
> 
> 
> I''ve created a straightforward Rails app for a client and it runs
great
> on my FreeBSD 6 box. When we deploy it on his Redhat 7.2 server, both
> WEBrick and Mongrel start and run fine, but will only stay up for a
> couple of hours. We''ll find them dead with absolutely no
information in
> the logs. No core dumps. Nothing in /var/log/messages. Just gone. The
> process will die even if idle, so I don''t think there''s
anything in the
> actual app that''s causing the problem.
> 
> In desperation, I started Mongrel as root and the darn thing is staying
> up. I know zilch about Linux admin. Is there something I''m missing
here?
> There are no obvious per-user process limits (memory, CPU, etc.) as far
> as I can tell.
> 
> -- 
> Posted via http://www.ruby-forum.com/.
> 
> > 


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Craig White wrote:
> On Fri, 2006-08-25 at 03:14 +0200, Bob Showalter wrote:
>> There are no obvious per-user process limits (memory, CPU, etc.) as far
>> as I can tell.
> ----
> I''m not gonna even speculate but RHL 7.2 has been EOL for at least
4
> years. Why not use CentOS 4 ?
Because a) I''m a programmer not a sysadmin, and b) I don''t
have any idea
what RHL 7.2 and CentOS 4 even are, sadly :~) Heck, it may not even by 
RHL 7.2 for all I know. It''s what cat /etc/redhat-release says, but who
knows if that''s right?

His box seems wacky to me. Non-root users can''t run ps, or who, or top.
You get an error that /proc must be mounted (?). Is this normal for 
Linux?
> 
> You don''t say which version of ruby/rails you are using and
I''m
> presuming that you are avoiding apache which is bound to be really old
> too.
I did a fresh build of Ruby 1.8.4 and am using Rails 1.1.2 (yes, I need 
to upgrade to 1.1.6). However, this problem does not happen on FreeBSD, 
and it only happens if I start the server as a non-privileged user. 
Doesn''t happen on Win32 either.

Apache not playing a part. Just running WEBrick or mongrel_rails as the 
server, port 3000, development environment. Just vanilla stuff.

I know I''m OT for the Rails list here, sorry...

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Bob Showalter wrote:
> Because a) I''m a programmer not a sysadmin, and b) I
don''t have any idea
> what RHL 7.2 and CentOS 4 even are, sadly :~) 
Wow. I would suggest HIRING a sysadmin then.

Joe

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

HH wrote:
> Can the user that mongrel is running under access the log files? Read 
> and
> write?
Yes, the application starts up fine and writes a pid file and log files. 
You can use it with no problems, but after a couple of hours or so, it 
just goes away. You can restart it and get a couple more hours just fine 
and then poof.

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On Fri, 2006-08-25 at 04:20 +0200, Bob Showalter wrote:
> Craig White wrote:
> > On Fri, 2006-08-25 at 03:14 +0200, Bob Showalter wrote:
> >> There are no obvious per-user process limits (memory, CPU, etc.)
as far
> >> as I can tell.
> > ----
> > I''m not gonna even speculate but RHL 7.2 has been EOL for at
least 4
> > years. Why not use CentOS 4 ?
> 
> Because a) I''m a programmer not a sysadmin, and b) I
don''t have any idea
> what RHL 7.2 and CentOS 4 even are, sadly :~) Heck, it may not even by 
> RHL 7.2 for all I know. It''s what cat /etc/redhat-release says,
but who
> knows if that''s right?
----
I''d accept that it is correct.

RHL = Red Hat Linux

RHL versioned 7.2 => 7.3 => 8.0 => 9 and then became known as Fedora
(free)

Red Hat Enterprise Linux / vague comparisons
2.1 (similar to RHL 7.3) - now in maintenance mode, security updates
only
3.8 (similar to RHL 9)
4.4 (similar to Fedora Core 3)
version 5.0 scheduled for release December 2006

CentOS is freeware version of Red Hat Enterprise Linux
http://www.centos.org
CentOS has versions derived from above.

Fedora Core version 4 just went EOL and version 5 is current, version 6
is scheduled for release October 6, 2006

In essence, between Red Hat Linux and its successor, Fedora, Fedora Core
5 represents the 8th release since Red Hat Linux 7.2 - that''s like
trying to use Windows 95 (rev B)
----
> 
> His box seems wacky to me. Non-root users can''t run ps, or who, or
top.
> You get an error that /proc must be mounted (?). Is this normal for 
> Linux?
----
# ls -l /bin/ps /usr/bin/top /usr/bin/who
-r-xr-xr-x  1 root root 66416 Dec 31  2005 /bin/ps
-r-xr-xr-x  1 root root 52348 Dec 31  2005 /usr/bin/top
-rwxr-xr-x  1 root root 23440 Aug 23  2005 /usr/bin/who

that''s on a CentOS 4 system. As you can see, all users can execute
them.
No, it''s not normal for Linux. Was it normal for RHL 7.2? I
don''t think
so but I don''t have any remaining systems running anything that old to
check.
----
> 
> > 
> > You don''t say which version of ruby/rails you are using and
I''m
> > presuming that you are avoiding apache which is bound to be really old
> > too.
> 
> I did a fresh build of Ruby 1.8.4 and am using Rails 1.1.2 (yes, I need 
> to upgrade to 1.1.6). However, this problem does not happen on FreeBSD, 
> and it only happens if I start the server as a non-privileged user. 
> Doesn''t happen on Win32 either.
----
on CentOS and Fedora systems, I run as user - never an issue.
----
> 
> Apache not playing a part. Just running WEBrick or mongrel_rails as the 
> server, port 3000, development environment. Just vanilla stuff.
> 
> I know I''m OT for the Rails list here, sorry...
----
we are talking about Rails - it''s on topic

Craig


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Joe Ruby wrote:
> Bob Showalter wrote:
>> Because a) I''m a programmer not a sysadmin, and b) I
don''t have any idea
>> what RHL 7.2 and CentOS 4 even are, sadly :~) 
> 
> Wow. I would suggest HIRING a sysadmin then.
But I was hoping to get free help from you :~) I just wanted to see if 
it was an obvious Linux thing that anyone with an IQ higher than pond 
slime would know but that I was overlooking in my shameful, 
mind-boggling, blithering ignorance. It seems that''s not the case.

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On 8/24/06, Bob Showalter
<rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org>
wrote:
>
>
> Craig White wrote:
> > On Fri, 2006-08-25 at 03:14 +0200, Bob Showalter wrote:
> >> There are no obvious per-user process limits (memory, CPU, etc.)
as far
> >> as I can tell.
> > ----
> > I''m not gonna even speculate but RHL 7.2 has been EOL for at
least 4
> > years. Why not use CentOS 4 ?
>
> Because a) I''m a programmer not a sysadmin, and b) I
don''t have any idea
> what RHL 7.2 and CentOS 4 even are, sadly :~) Heck, it may not even by
> RHL 7.2 for all I know. It''s what cat /etc/redhat-release says,
but who
> knows if that''s right?
>
> His box seems wacky to me. Non-root users can''t run ps, or who, or
top.
> You get an error that /proc must be mounted (?). Is this normal for
> Linux?

Sounds like you may be hitting a system limit and your processes are being
killed.

Run ulimit -a as your user and as root to see the difference.

jeremy


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On Fri, 2006-08-25 at 04:53 +0200, Bob Showalter wrote:
> HH wrote:
> > Can the user that mongrel is running under access the log files? Read 
> > and
> > write?
> 
> Yes, the application starts up fine and writes a pid file and log files. 
> You can use it with no problems, but after a couple of hours or so, it 
> just goes away. You can restart it and get a couple more hours just fine 
> and then poof.
----
I suppose then you create a cron job that runs a script that restarts
mongrel process every hour or so - otherwise, get a system admin.

Craig


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Jeremy Kemper wrote:
> On 8/24/06, Bob Showalter
<rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote:
>> Because a) I''m a programmer not a sysadmin, and b) I
don''t have any idea
>> what RHL 7.2 and CentOS 4 even are, sadly :~) Heck, it may not even by
>> RHL 7.2 for all I know. It''s what cat /etc/redhat-release
says, but who
>> knows if that''s right?
>>
>> His box seems wacky to me. Non-root users can''t run ps, or
who, or top.
>> You get an error that /proc must be mounted (?). Is this normal for
>> Linux?
> 
> 
> Sounds like you may be hitting a system limit and your processes are 
> being
> killed.
> 
> Run ulimit -a as your user and as root to see the difference.
That''s what I thought, but here''s what I get:

[root@blah]# ulimit -a
core file size (blocks)     0
data seg size (kbytes)      unlimited
file size (blocks)          unlimited
max locked memory (kbytes)  unlimited
max memory size (kbytes)    unlimited
open files                  1024
pipe size (512 bytes)       8
stack size (kbytes)         8192
cpu time (seconds)          unlimited
max user processes          4082
virtual memory (kbytes)     unlimited
[root@nimbus prop]# su - bob
[bob@blah]$ ulimit -a
core file size (blocks)     0
data seg size (kbytes)      unlimited
file size (blocks)          unlimited
max locked memory (kbytes)  unlimited
max memory size (kbytes)    unlimited
open files                  1024
pipe size (512 bytes)       8
stack size (kbytes)         8192
cpu time (seconds)          unlimited
max user processes          4082
virtual memory (kbytes)     unlimited

I think there''s something on the box that''s killing of
processes. I did
a ps and found three daemons running that are all encrypted Perl scripts 
(the first line is "use Filter::decrtypt", and the rest is gibberish):

  monstd
  syswatchd
  healthd

These all live in /usr/prop/sbin. There''s also a bunch of cron jobs
that
run stuff in /usr/prop/sbin.

After some googling, I figured out how to:

# rpm -q --file /usr/prop/sbin/syswatchd
helix-syswatchd-ded72-1-1

Does this mean anything to anyone? I appreciate all the help.

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On Fri, 2006-08-25 at 04:20 +0200, Bob Showalter wrote:
> You get an error that /proc must be mounted (?). Is this normal for 
> Linux?
----
this might help...

$ grep proc /etc/fstab
proc      /proc          proc    defaults        0 0

add that line to fstab (obviously as root), make sure that you
have /proc directory like

$ ls -ld /proc
dr-xr-xr-x 142 root root 0 Aug 23 03:02 /proc

reboot and see if that makes happy rails/mongrel

Craig


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On Fri, 2006-08-25 at 05:25 +0200, Bob Showalter wrote:
> Jeremy Kemper wrote:
> > On 8/24/06, Bob Showalter
<rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote:
> >> Because a) I''m a programmer not a sysadmin, and b) I
don''t have any idea
> >> what RHL 7.2 and CentOS 4 even are, sadly :~) Heck, it may not
even by
> >> RHL 7.2 for all I know. It''s what cat /etc/redhat-release
says, but who
> >> knows if that''s right?
> >>
> >> His box seems wacky to me. Non-root users can''t run ps,
or who, or top.
> >> You get an error that /proc must be mounted (?). Is this normal
for
> >> Linux?
> > 
> > 
> > Sounds like you may be hitting a system limit and your processes are 
> > being
> > killed.
> > 
> > Run ulimit -a as your user and as root to see the difference.
> 
> That''s what I thought, but here''s what I get:
> 
> [root@blah]# ulimit -a
> core file size (blocks)     0
> data seg size (kbytes)      unlimited
> file size (blocks)          unlimited
> max locked memory (kbytes)  unlimited
> max memory size (kbytes)    unlimited
> open files                  1024
> pipe size (512 bytes)       8
> stack size (kbytes)         8192
> cpu time (seconds)          unlimited
> max user processes          4082
> virtual memory (kbytes)     unlimited
> [root@nimbus prop]# su - bob
> [bob@blah]$ ulimit -a
> core file size (blocks)     0
> data seg size (kbytes)      unlimited
> file size (blocks)          unlimited
> max locked memory (kbytes)  unlimited
> max memory size (kbytes)    unlimited
> open files                  1024
> pipe size (512 bytes)       8
> stack size (kbytes)         8192
> cpu time (seconds)          unlimited
> max user processes          4082
> virtual memory (kbytes)     unlimited
> 
> I think there''s something on the box that''s killing of
processes. I did
> a ps and found three daemons running that are all encrypted Perl scripts 
> (the first line is "use Filter::decrtypt", and the rest is
gibberish):
> 
>   monstd
>   syswatchd
>   healthd
> 
> These all live in /usr/prop/sbin. There''s also a bunch of cron
jobs that
> run stuff in /usr/prop/sbin.
> 
> After some googling, I figured out how to:
> 
> # rpm -q --file /usr/prop/sbin/syswatchd
> helix-syswatchd-ded72-1-1
> 
> Does this mean anything to anyone? I appreciate all the help.
----
/usr/prop # not part of Red Hat packages - 3rd party software perhaps

Craig


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Craig White wrote:
> I suppose then you create a cron job that runs a script that restarts
> mongrel process every hour or so - otherwise, get a system admin.
Thanks, looks like we''ll do the latter. I appreciate all the help.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---