We tried to give Mongrel a go running our application which has moderate traffic. We got mongrel up and running easy enough but 3 times in a short period(2hours) mongrel just died with the following error when we went live to production with it. As you can see we are running mongrel-0.3.13.3. /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.3/lib/mongrel.rb:576:in`peeraddr'': Transport endpoint is not connected - getpeername(2) (Errno::ENOTCONN) Is this a bug or a configuration problem of some sort on our end? I did some googling on the error and did not find anything. We also get a large number of not fatal messages that maybe someone can enlighten me on. What would cause this message, and what does that client that caused this warning see? Tue Jul 18 01:15:00 EDT 2006: BAD CLIENT (65.34.163.249): Invalid HTTP format, parsing fails Thanks for your time. -Dallas sportspyder.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20060718/e94a5ce5/attachment.html
I saw this error came up on the Rails list, but regarding WEBrick. Could be unrelated, but here''s that user''s solution: "map.connect '':controller/service.wsdl'', :action => ''wsdl'' One of the default routes. If I comment it out, all works fine. Left in and WEBrick crashes!" On 7/18/06, Dallas DeVries <dallas.devries at gmail.com> wrote:> > We tried to give Mongrel a go running our application which has moderate > traffic. We got mongrel up and running easy enough but 3 times in a short > period(2hours) mongrel just died with the following error when we went live > to production with it. As you can see we are running mongrel-0.3.13.3. > > /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.3/lib/mongrel.rb:576:in`peeraddr'': Transport endpoint is not connected - getpeername(2) > (Errno::ENOTCONN) > > Is this a bug or a configuration problem of some sort on our end? I did > some googling on the error and did not find anything. > > We also get a large number of not fatal messages that maybe someone can > enlighten me on. What would cause this message, and what does that client > that caused this warning see? > > Tue Jul 18 01:15:00 EDT 2006: BAD CLIENT (65.34.163.249): Invalid HTTP > format, parsing fails > > Thanks for your time. > > -Dallas > sportspyder.com > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20060718/b018aba5/attachment.html
Hey Ed, I found that rails thread and I can''t see a reference to the error I reported http://lists.rubyonrails.org/pipermail/rails/2006-March/027919.html It doesn''t seem related to me anyways. My Mongrel process does run fine for a period of time then unexpectedly dies after 10 to 50 min. I am running ruby 1.8.4 and latest version of rails on Debian Sarge. Cheers, Dallas On 7/18/06, Ed C. <defeated2k4 at gmail.com> wrote:> > I saw this error came up on the Rails list, but regarding WEBrick. Could > be unrelated, but here''s that user''s solution: > > "map.connect '':controller/service.wsdl'', :action => ''wsdl'' > One of the default routes. > If I comment it out, all works fine. > Left in and WEBrick crashes!" > > > On 7/18/06, Dallas DeVries <dallas.devries at gmail.com> wrote: > > > We tried to give Mongrel a go running our application which has moderate > traffic. We got mongrel up and running easy enough but 3 times in a short > period(2hours) mongrel just died with the following error when we went live > to production with it. As you can see we are running mongrel-0.3.13.3. > > /usr/lib/ruby/gems/1.8/gems/mongrel- 0.3.13.3/lib/mongrel.rb:576:in`peeraddr'': Transport endpoint is not connected - getpeername(2) > (Errno::ENOTCONN) > > Is this a bug or a configuration problem of some sort on our end? I did > some googling on the error and did not find anything. > > We also get a large number of not fatal messages that maybe someone can > enlighten me on. What would cause this message, and what does that client > that caused this warning see? > > Tue Jul 18 01:15:00 EDT 2006: BAD CLIENT ( 65.34.163.249): Invalid HTTP > format, parsing fails > > Thanks for your time. > > -Dallas > sportspyder.com > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > > > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20060718/7aa8542f/attachment.html
On Tue, 2006-07-18 at 12:08 -0400, Dallas DeVries wrote:> We tried to give Mongrel a go running our application which has > moderate traffic. We got mongrel up and running easy enough but 3 > times in a short period(2hours) mongrel just died with the following > error when we went live to production with it. As you can see we are > running mongrel-0.3.13.3. > > /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.3/lib/mongrel.rb:576:in > `peeraddr'': Transport endpoint is not connected - getpeername(2) > (Errno::ENOTCONN) >That don''t make any sense since that entire chunk of code is wrapped in a begin/rescue block. Actually several. Are you using anything strange for your configuration? A different load balancer or something unreliable? Putting mongrel directly on the internet? If you can do it, see if you can temporarily run with 0.3.13.4 from the pre-release: sudo gem install mongrel --source=http://mongrel.rubyforge.org/releases/ And then send me the stack trace it gives you. That release is more chatty about stuff but it has a bunch of fixes for edge cases. Maybe you''re hitting them somehow. Oh, and you get this error at that line because Mongrel doesn''t find an X-Forwarded-For header so it tries to get the client''s remote IP address. The client seems to have disconnected, so apparently that Socket#peeraddr call throws an exception in this case. WHY it throws an exception is beyond me. WHY it isn''t caught is also beyond me.> Is this a bug or a configuration problem of some sort on our end? I > did some googling on the error and did not find anything. > > We also get a large number of not fatal messages that maybe someone > can enlighten me on. What would cause this message, and what does that > client that caused this warning see? > > Tue Jul 18 01:15:00 EDT 2006: BAD CLIENT (65.34.163.249): Invalid HTTP > format, parsing failsThis happens because someone is using a client that''s most likely not written correctly. If you''re running Nagios then this would be the most likely source since Nagios is horribly written junk. Other things that cause this are people attacking your server (not kidding). Mongrel''s parser is pretty lax, but in the instances where I''ve found it blocked someone I''ve also found that Mongrel really did need to block that client for mis-behavior. I''m debating whether I should indicate what causes the error, but generally it''s better for you to use the time-stamp and a protocol sniffer to check out the traffic. Otherwise mongrel wastes a lot of time reporting the bad http content. -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/ http://www.railsmachine.com/ -- Need Mongrel support?
Hi Zed, Thanks for the response. We were not using anything strange in our configuration. No Nagios just a minimal release on debian sarge with the web server running mongrel and ssh. We had Mongrel behind a router that forwarded port 80 to the web server. Last night we installed mongrel based off this article (debian package instead of gem install) http://www.jkraemer.net/articles/2006/07/07/mongrel-apache-and-rails-on-debian-sarge We used Pen as our proxy with 3 mongrel processes. So far things are running MUCH better using this method and it has been up for 9 hours without receiving that error again and no mongrel processes have died. Also perhaps because of pen we are not receiving the BAD CLIENT message any longer either. I will see if its feasible to try to use the prerelease and repro it the old way so you can deal with the bug. Cheers, Dallas On 7/19/06, Zed Shaw <zedshaw at zedshaw.com> wrote:> > On Tue, 2006-07-18 at 12:08 -0400, Dallas DeVries wrote: > > We tried to give Mongrel a go running our application which has > > moderate traffic. We got mongrel up and running easy enough but 3 > > times in a short period(2hours) mongrel just died with the following > > error when we went live to production with it. As you can see we are > > running mongrel-0.3.13.3. > > > > /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.3/lib/mongrel.rb:576:in > > `peeraddr'': Transport endpoint is not connected - getpeername(2) > > (Errno::ENOTCONN) > > > > That don''t make any sense since that entire chunk of code is wrapped in > a begin/rescue block. Actually several. Are you using anything strange > for your configuration? A different load balancer or something > unreliable? Putting mongrel directly on the internet? > > If you can do it, see if you can temporarily run with 0.3.13.4 from the > pre-release: > > sudo gem install mongrel > --source=http://mongrel.rubyforge.org/releases/ > > And then send me the stack trace it gives you. That release is more > chatty about stuff but it has a bunch of fixes for edge cases. Maybe > you''re hitting them somehow. > > Oh, and you get this error at that line because Mongrel doesn''t find an > X-Forwarded-For header so it tries to get the client''s remote IP > address. The client seems to have disconnected, so apparently that > Socket#peeraddr call throws an exception in this case. WHY it throws an > exception is beyond me. WHY it isn''t caught is also beyond me. > > > Is this a bug or a configuration problem of some sort on our end? I > > did some googling on the error and did not find anything. > > > > We also get a large number of not fatal messages that maybe someone > > can enlighten me on. What would cause this message, and what does that > > client that caused this warning see? > > > > Tue Jul 18 01:15:00 EDT 2006: BAD CLIENT (65.34.163.249): Invalid HTTP > > format, parsing fails > > This happens because someone is using a client that''s most likely not > written correctly. If you''re running Nagios then this would be the most > likely source since Nagios is horribly written junk. Other things that > cause this are people attacking your server (not kidding). Mongrel''s > parser is pretty lax, but in the instances where I''ve found it blocked > someone I''ve also found that Mongrel really did need to block that > client for mis-behavior. > > I''m debating whether I should indicate what causes the error, but > generally it''s better for you to use the time-stamp and a protocol > sniffer to check out the traffic. Otherwise mongrel wastes a lot of > time reporting the bad http content. > > -- > Zed A. Shaw > http://www.zedshaw.com/ > http://mongrel.rubyforge.org/ > http://www.railsmachine.com/ -- Need Mongrel support? > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20060719/41c34595/attachment.html
On Tue, 2006-07-18 at 12:08 -0400, Dallas DeVries wrote:> We tried to give Mongrel a go running our application which has > moderate traffic. We got mongrel up and running easy enough but 3 > times in a short period(2hours) mongrel just died with the following > error when we went live to production with it. As you can see we are > running mongrel-0.3.13.3. >BTW, did you have ruby-sendfile installed at all? -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/ http://www.railsmachine.com/ -- Need Mongrel support?