mongrel_users at homerlex.mailshell.com
2006-Sep-26 12:50 UTC
[Mongrel] Mongrel Processes Dying
We are seeing mongrel processes dying in our mongrel cluster. What is the best way to troubleshoot this? We have ruby 1.8.4, Rails 1.1.0 (upgrading soon), MySql 4.1 running on Red Hat Enterprise Linux ES release 4, Apache 2.2, mongrel-0.3.13.3, mongrel_cluster-0.2.0 I saw the following messages in the mongrel.log but not sure if theay are related to the processes dying. It would be nice if the log had date/time for each entry! /usr/local/lib/ruby/1.8/net/http.rb:562: [BUG] Segmentation fault ruby 1.8.4 (2005-12-24) [i686-linux] /usr/local/lib/ruby/1.8/net/http.rb:562: [BUG] Segmentation fault ruby 1.8.4 (2005-12-24) [i686-linux] /usr/local/lib/ruby/1.8/net/http.rb:562: [BUG] Segmentation fault ruby 1.8.4 (2005-12-24) [i686-linux] Any help is greatly appreciated _______________________________________________________ The FREE service that prevents junk email mailshell.com
On Tue, 26 Sep 2006 05:50:37 -0700 mongrel_users at homerlex.mailshell.com wrote:> We are seeing mongrel processes dying in our mongrel cluster. What is the best way to > troubleshoot this? > > We have ruby 1.8.4, Rails 1.1.0 (upgrading soon), MySql 4.1 running on Red Hat Enterprise Linux > ES release 4, Apache 2.2, mongrel-0.3.13.3, mongrel_cluster-0.2.0 > > I saw the following messages in the mongrel.log but not sure if theay are related to the > processes dying. It would be nice if the log had date/time for each entry! > > /usr/local/lib/ruby/1.8/net/http.rb:562: [BUG] Segmentation fault > ruby 1.8.4 (2005-12-24) [i686-linux]These are segmentation faults inside Ruby''s http code. Not sure how you''re getting those but you''ll need to investigate how your system is apparently totally different from everyone else''s. For example, why is it in /usr/local? I''m thinking maybe you''re mixing ruby installations/versions. And, it''s really hard to change http.rb to produce timestamps during a segfault. Actually the fact that ruby is giving you a line number is pretty magical, so be thankful. Normally you''d have to gdb a core file for that. Which brings up another thing, turn on core dumps and that might help track it down. -- Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu zedshaw.com mongrel.rubyforge.org lingr.com/room/3yXhqKbfPy8 -- Come get help.
On 9/26/06, mongrel_users at homerlex.mailshell.com> We have ruby 1.8.4, Rails 1.1.0 (upgrading soon), MySql 4.1 running on Red Hat Enterprise Linux > ES release 4, Apache 2.2, mongrel-0.3.13.3, mongrel_cluster-0.2.0 > > I saw the following messages in the mongrel.log but not sure if theay are related to the > processes dying. It would be nice if the log had date/time for each entry! > > /usr/local/lib/ruby/1.8/net/http.rb:562: [BUG] Segmentation fault > ruby 1.8.4 (2005-12-24) [i686-linux]Segmentation faults are bad. That said, those segfaults are at this line: s = timeout(@open_timeout) { TCPSocket.open(conn_address(), conn_port()) } Which is curious. I don''t know where, in your version of Mongrel, that would be called. I''ve crawled all through the Mongrel source, and the only places that I see Net::HTTP usage are in the tests. Kirk Haines
Zed A. Shaw wrote:> On Tue, 26 Sep 2006 05:50:37 -0700 > mongrel_users at homerlex.mailshell.com wrote: > >> We are seeing mongrel processes dying in our mongrel cluster. What is the best way to >> troubleshoot this? >> >> We have ruby 1.8.4, Rails 1.1.0 (upgrading soon), MySql 4.1 running on Red Hat Enterprise Linux >> ES release 4, Apache 2.2, mongrel-0.3.13.3, mongrel_cluster-0.2.0 >> >> I saw the following messages in the mongrel.log but not sure if theay are related to the >> processes dying. It would be nice if the log had date/time for each entry! >> >> /usr/local/lib/ruby/1.8/net/http.rb:562: [BUG] Segmentation fault >> ruby 1.8.4 (2005-12-24) [i686-linux] > > These are segmentation faults inside Ruby''s http code. Not sure how you''re getting those but you''ll need to investigate how your system is apparently totally different from everyone else''s. For example, why is it in /usr/local? I''m thinking maybe you''re mixing ruby installations/versions. > > And, it''s really hard to change http.rb to produce timestamps during a segfault. Actually the fact that ruby is giving you a line number is pretty magical, so be thankful. Normally you''d have to gdb a core file for that. > > Which brings up another thing, turn on core dumps and that might help track it down. >I did notice that there were some fixes for http.rb in the ChangeLog for 1.8.5. I don''t know if upgrading is an option, but you may want to consider it. Mind you, given that the problem ultimately seems to be coming from TCPSocket, I''m not sure if they''ll help. Regards, Dan This communication is the property of Qwest and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
mongrel_users at homerlex.mailshell.com
2006-Sep-28 13:43 UTC
[Mongrel] Mongrel Processes Dying
I haven''t turned on core dumps yet. Things were quite for a couple days but this morning a few processes died and I see some new Segmentation faults: /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.0/lib/action_view/base.rb:410: [BUG] Segmentation fault ruby 1.8.4 (2005-12-24) [i686-linux] /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.0/lib/active_record/connection_adapters/mysql_adapter.rb:148: [BUG] /usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.3/lib/mongrel/handlers.rb:134: [BUG] /usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.3/lib/mongrel.rb:519: [BUG] Segmentation fault ruby 1.8.4 (2005-12-24) [i686-linux] /usr/local/lib/ruby/gems/1.8/gems/mongrel-0.3.13.3/lib/mongrel.rb:519: [BUG] Segmentation fault ruby 1.8.4 (2005-12-24) [i686-linux] -----Original Message----- From: mongrel-users-bounces at rubyforge.org [mailto:mongrel-users-bounces at rubyforge.org] On Behalf Of Zed A. Shaw Sent: Tuesday, September 26, 2006 10:35 AM To: mongrel-users at rubyforge.org Subject: Re: [Mongrel] Mongrel Processes Dying On Tue, 26 Sep 2006 05:50:37 -0700 mongrel_users at homerlex.mailshell.com wrote:> We are seeing mongrel processes dying in our mongrel cluster. What is > the best way to troubleshoot this? > > We have ruby 1.8.4, Rails 1.1.0 (upgrading soon), MySql 4.1 running on > Red Hat Enterprise Linux ES release 4, Apache 2.2, mongrel-0.3.13.3, > mongrel_cluster-0.2.0 > > I saw the following messages in the mongrel.log but not sure if theay > are related to the processes dying. It would be nice if the log had date/time for each entry! > > /usr/local/lib/ruby/1.8/net/http.rb:562: [BUG] Segmentation fault ruby > 1.8.4 (2005-12-24) [i686-linux]These are segmentation faults inside Ruby''s http code. Not sure how you''re getting those but you''ll need to investigate how your system is apparently totally different from everyone else''s. For example, why is it in /usr/local? I''m thinking maybe you''re mixing ruby installations/versions. And, it''s really hard to change http.rb to produce timestamps during a segfault. Actually the fact that ruby is giving you a line number is pretty magical, so be thankful. Normally you''d have to gdb a core file for that. Which brings up another thing, turn on core dumps and that might help track it down. -- Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu _______________________________________________________ The FREE service that prevents junk email mailshell.com
Reasonably Related Threads
- Does send_file act differently with Mongrel than withWebrick?
- Does send_file act differently with Mongrel than with Webrick?
- Multiple Processes Spawned from mongrel_rails start ?
- Anyway to Force a start even if the .pid file exists?
- Clustering - Avoiding "dead" processes?