Sometime my mongrel server threads..would simply stop responding to the requests...and in the logs i get following error: Tue Sep 05 03:56:40 IST 2006: ERROR: Transport endpoint is not connected - getpeername(2) Tue Sep 05 03:59:21 IST 2006: ERROR: Transport endpoint is not connected - getpeername(2) Tue Sep 05 03:59:21 IST 2006: ERROR: Transport endpoint is not connected - getpeername(2) Tue Sep 05 03:59:47 IST 2006: ERROR: Transport endpoint is not connected - getpeername(2) Tue Sep 05 03:59:47 IST 2006: ERROR: Transport endpoint is not connected - getpeername(2) Any clues? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20060905/051fde52/attachment.html
On Tue, 2006-09-05 at 04:49 +0530, hemant wrote:> Sometime my mongrel server threads..would simply stop responding to > the requests...and in the logs i get following error: > > Tue Sep 05 03:56:40 IST 2006: ERROR: Transport endpoint is not > connected - getpeername(2) > Tue Sep 05 03:59:21 IST 2006: ERROR: Transport endpoint is not > connected - getpeername(2) > Tue Sep 05 03:59:21 IST 2006: ERROR: Transport endpoint is not > connected - getpeername(2) > Tue Sep 05 03:59:47 IST 2006: ERROR: Transport endpoint is not > connected - getpeername(2) > Tue Sep 05 03:59:47 IST 2006: ERROR: Transport endpoint is not > connected - getpeername(2)Make sure you have the latest 0.3.13.4 pre-release, run your code under it, and after mongrel starts, turn on USR1 debugging: killall -USR1 mongrel_rails Then, look for any logging messages in mongrel.log that say there''s tons of threads waiting for one of your rails actions. Go fix that rails action and you''ll be set. -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.
On 9/5/06, Zed Shaw <zedshaw at zedshaw.com> wrote:> > On Tue, 2006-09-05 at 04:49 +0530, hemant wrote: > > Sometime my mongrel server threads..would simply stop responding to > > the requests...and in the logs i get following error: > > > > Tue Sep 05 03:56:40 IST 2006: ERROR: Transport endpoint is not > > connected - getpeername(2) > > Tue Sep 05 03:59:21 IST 2006: ERROR: Transport endpoint is not > > connected - getpeername(2) > > Tue Sep 05 03:59:21 IST 2006: ERROR: Transport endpoint is not > > connected - getpeername(2) > > Tue Sep 05 03:59:47 IST 2006: ERROR: Transport endpoint is not > > connected - getpeername(2) > > Tue Sep 05 03:59:47 IST 2006: ERROR: Transport endpoint is not > > connected - getpeername(2) > > Make sure you have the latest 0.3.13.4 pre-release, run your code under > it, and after mongrel starts, turn on USR1 debugging: > > killall -USR1 mongrel_rails > > Then, look for any logging messages in mongrel.log that say there''s tons > of threads waiting for one of your rails actions. Go fix that rails > action and you''ll be set. >I was using killall -USR1, but in your recent post..i have seen that you have updated the pre-release server...and hence i update the mongrel gem and..i am debugging it again.. Tue Sep 05 05:03:32 IST 2006: 0 threads sync_waiting for /images/tab_but_normal1.png, 4 still active in mongrel. So, what does the above line mean? 4 still active in mongrel? does it mean that 4 threads are waiting for this file? Does it mean..that for every file that is not found...in the rails application directory, mongrel threads wait? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20060905/5be20117/attachment.html
On Tue, 2006-09-05 at 05:30 +0530, hemant wrote:> On 9/5/06, Zed Shaw <zedshaw at zedshaw.com> wrote:> > I was using killall -USR1, but in your recent post..i have seen that > you have updated the pre-release server...and hence i update the > mongrel gem and..i am debugging it again.. > > Tue Sep 05 05:03:32 IST 2006: 0 threads sync_waiting > for /images/tab_but_normal1.png, 4 still active in mongrel. > > So, what does the above line mean? 4 still active in mongrel? does it > mean that 4 threads are waiting for this file? > > Does it mean..that for every file that is not found...in the rails > application directory, mongrel threads wait? >It means that a request came in, it hit rails, there were 0 waiting for rails, and there''s still 4 "in mongrel". These four are probably busy sending a response. What you don''t want to see is: Tue Sep 05 05:03:32 IST 2006: 980 threads sync_waiting for /images/tab_but_normal1.png, 975 still active in mongrel. Additionally, why in the world are you having Mongrel serve the files. Please, please, please configure your apache correctly so that apache serves the files and mongrel just does Rails. You''d be way better off just doing that. -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.
Yes yes..the message i have reported is currently from my development instance. For main instance...Apache is configured to server static pages. Do these threads add up? Currently it is: Tue Sep 05 05:31:11 IST 2006: 0 threads sync_waiting for /charts/%5ENSEI, 1 still active in mongrel. Tue Sep 05 05:31:26 IST 2006: 0 threads sync_waiting for /images/flash/video/BloomBerg.jpg, 1 still active in mongrel. Tue Sep 05 05:31:36 IST 2006: 0 threads sync_waiting for /analysts/my_avatar/kurt, 1 still active in mongrel. Tue Sep 05 05:34:18 IST 2006: 0 threads sync_waiting for /forums/home, 1 still active in mongrel. Tue Sep 05 05:34:30 IST 2006: 0 threads sync_waiting for /, 1 still active in mongrel. Tue Sep 05 05:37:39 IST 2006: 0 threads sync_waiting for /price, 1 still active in mongrel. Tue Sep 05 05:40:39 IST 2006: 0 threads sync_waiting for /price, 1 still active in mongrel. I guess it looks healthy right now. On 9/5/06, Zed Shaw <zedshaw at zedshaw.com> wrote:> > On Tue, 2006-09-05 at 05:30 +0530, hemant wrote: > > On 9/5/06, Zed Shaw <zedshaw at zedshaw.com> wrote: > > > > > I was using killall -USR1, but in your recent post..i have seen that > > you have updated the pre-release server...and hence i update the > > mongrel gem and..i am debugging it again.. > > > > Tue Sep 05 05:03:32 IST 2006: 0 threads sync_waiting > > for /images/tab_but_normal1.png, 4 still active in mongrel. > > > > So, what does the above line mean? 4 still active in mongrel? does it > > mean that 4 threads are waiting for this file? > > > > Does it mean..that for every file that is not found...in the rails > > application directory, mongrel threads wait? > > > > It means that a request came in, it hit rails, there were 0 waiting for > rails, and there''s still 4 "in mongrel". These four are probably busy > sending a response. > > What you don''t want to see is: > > Tue Sep 05 05:03:32 IST 2006: 980 threads sync_waiting > for /images/tab_but_normal1.png, 975 still active in mongrel. > > Additionally, why in the world are you having Mongrel serve the files. > Please, please, please configure your apache correctly so that apache > serves the files and mongrel just does Rails. You''d be way better off > just doing that. > > > -- > Zed A. Shaw > http://www.zedshaw.com/ > http://mongrel.rubyforge.org/ > http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help. > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users >-- nothing much to talk -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20060905/6d46ee53/attachment-0001.html
What these warnings mean: /usr/local/lib/site_ruby/1.8/transaction/simple.rb:74: warning: already initialized constant Messages /usr/local/lib/site_ruby/1.8/transaction/simple.rb:274: warning: already initialized constant TRANSACTION_SIMPLE_VERSION /usr/local/lib/site_ruby/1.8/transaction/simple.rb:635: warning: already initialized constant TRANSACTION_CHECKPOINT /usr/local/lib/site_ruby/1.8/transaction/simple.rb:636: warning: already initialized constant SKIP_TRANSACTION_VARS Earlier...i had a model called Message, so i thought I was getting these warnings because...but even after getting rid of that model..i get those errors. Still...i guess, it could be a problem at my end....but just wanted to put it up here. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20060905/6805edac/attachment.html
On 9/5/06, hemant <gethemant at gmail.com> wrote:> What these warnings mean: > > > /usr/local/lib/site_ruby/1.8/transaction/simple.rb:74: > warning: already initialized constant Messages > /usr/local/lib/site_ruby/1.8/transaction/simple.rb:274: > warning: already initialized constant TRANSACTION_SIMPLE_VERSION > /usr/local/lib/site_ruby/1.8/transaction/simple.rb:635: > warning: already initialized constant TRANSACTION_CHECKPOINT > /usr/local/lib/site_ruby/1.8/transaction/simple.rb:636: > warning: already initialized constant SKIP_TRANSACTION_VARS > > Earlier...i had a model called Message, so i thought I was getting these > warnings because...but even after getting rid of that model..i get those > errors. > Still...i guess, it could be a problem at my end....but just wanted to put > it up here.I development mode, files are reloaded upon each request. That causes redefinition of classes, and what you have seen is a side effect of the redefinition.
Well Zed, I i turned on USR1 debugging and all that stuff.....and I am *sometimes* get the above stated error. You might have seen...some of the my earlier code snippets where I depend upon the external data vending servers for data...and i timeout those requests..depending upon the request type. Now..that stuff is out bottleneck...mongrel threads often keep waiting there. Though I have never seen 900 threads waiting....but sometime i DO get 27 threads waiting and all that stuff. And once i get this, I will have to restart the mongrel. Basically I am going to further optimise my code..but I would really like to know, whats the following error: Tue Sep 05 03:56:40 IST 2006: ERROR: Transport endpoint is not connected -> getpeername(2) > Tue Sep 05 03:59:21 IST 2006: ERROR: Transport endpoint is not connected - > getpeername(2) > Tue Sep 05 03:59:21 IST 2006: ERROR: Transport endpoint is not connected - > getpeername(2) > Tue Sep 05 03:59:47 IST 2006: ERROR: Transport endpoint is not connected - > getpeername(2) > Tue Sep 05 03:59:47 IST 2006: ERROR: Transport endpoint is not connected - > getpeername(2)Just tell me...when exactly mongrel throws this error? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20060911/13a4273e/attachment-0001.html
On 9/11/06, hemant <gethemant at gmail.com> wrote:> > > Well Zed, > > I i turned on USR1 debugging and all that stuff.....and I am *sometimes* > get the above stated error. > > You might have seen...some of the my earlier code snippets where I depend > upon the external data vending servers for data...and i timeout those > requests..depending upon the request type. > > Now..that stuff is out bottleneck...mongrel threads often keep waiting > there. Though I have never seen 900 threads waiting....but sometime i DO get > 27 threads waiting and all that stuff. And once i get this, I will have to > restart the mongrel. Basically I am going to further optimise my code..but I > would really like to know, whats the following error: > > > Tue Sep 05 03:56:40 IST 2006: ERROR: Transport endpoint is not connected - > > getpeername(2) > > Tue Sep 05 03:59:21 IST 2006: ERROR: Transport endpoint is not connected > > - getpeername(2) > > Tue Sep 05 03:59:21 IST 2006: ERROR: Transport endpoint is not connected > > - getpeername(2) > > Tue Sep 05 03:59:47 IST 2006: ERROR: Transport endpoint is not connected > > - getpeername(2) > > Tue Sep 05 03:59:47 IST 2006: ERROR: Transport endpoint is not connected > > - getpeername(2) > > > Just tell me...when exactly mongrel throws this error? > > >After turning on USR1 level debugging i get following errors in my logs: Mon Sep 11 13:00:17 IST 2006: ERROR: Transport endpoint is not connected -> getpeername(2) > /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:567:in`peeraddr'' > /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:567:in`process_client'' > /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:686:in `run'' > /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:686:in `run'' > /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:673:in `run'' > /usr/lib/ruby/gems/1.8/gems/mongrel- > 0.3.13.4/lib/mongrel/configurator.rb:267:in `run'' > /usr/lib/ruby/gems/1.8/gems/mongrel- > 0.3.13.4/lib/mongrel/configurator.rb:266:in `run'' > /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/bin/mongrel_rails:127:in`run'' > /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/command.rb:203:in`run'' > /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/bin/mongrel_rails:231 > /usr/bin/mongrel_rails:18 > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20060911/bf9017d2/attachment.html
On Mon, 2006-09-11 at 13:02 +0530, hemant wrote:> > > On 9/11/06, hemant <gethemant at gmail.com> wrote: ><snip>> > After turning on USR1 level debugging i get following errors in my > logs: > > Mon Sep 11 13:00:17 IST 2006: ERROR: Transport endpoint is not > connected - getpeername(2) > /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:567:in `peeraddr''Hmmm, that is only there if there is no X_FORWARDED_FOR header from a proxy server and Mongrel has to resolve the host. It means you have a poorly setup DNS server that can''t resolve fast enough so the process is getting stuck. How are is your proxy setup? -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.
On 9/11/06, Zed Shaw <zedshaw at zedshaw.com> wrote: <snip>> > Hmmm, that is only there if there is no X_FORWARDED_FOR header from a > proxy server and Mongrel has to resolve the host. It means you have a > poorly setup DNS server that can''t resolve fast enough so the process is > getting stuck. > > How are is your proxy setup?</snip> Now this is weird...here I am talking about standalone mongrel server...which is not proxied through apache. I access the page..manually through the IP address and port (*just for development though*). Basically, all my server and rails code is there on a Ubuntu machine and we access(*use*) it through a Windows Machine. Anyway, I also have the mongrel cluster setup on the same machine, which runs in production mode and is proxied through Apache. My Proxy Configuration is, pretty default one suggested by Coda and Philip: <VirtualHost *:80> ServerAdmin info at foobar.com DocumentRoot /var/www/webfront/public ServerName foobar.com ServerAlias *.foobar.com <Directory /var/www/webfront/public> Options FollowSymLinks AllowOverride None Order allow,deny Allow from all </Directory> RewriteEngine On # Make sure people go to www.myapp.com, not myapp.com RewriteCond %{HTTP_HOST} ^foobar.com$ [NC] RewriteRule ^(.*)$ http://www.foobar.com$1 [R=301,L] # Yes, I''ve read no-www.com, but my site already has much Google-Fu on # www.blah.com. Feel free to comment this out. # Uncomment for rewrite debugging #RewriteLog logs/myapp_rewrite_log #RewriteLogLevel 9 #ProxyPass / balancer://mongrel_cluster #ProxyPassReverse / balancer://mongrel_cluster ProxyPreserveHost On # Check for maintenance file and redirect all requests RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite index to check for static RewriteRule ^/$ /index.html [QSA] # Rewrite to check for Rails cached page RewriteRule ^([^.]+)$ $1.html [QSA] # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] # Deflate AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css BrowserMatch ^Mozilla/4 gzip-only-text/html BrowserMatch ^Mozilla/4.0[678] no-gzip BrowserMatch bMSIE !no-gzip !gzip-only-text/html # Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat ''"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)'' deflate #CustomLog logs/myapp_deflate_log deflate </VirtualHost> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20060911/09b142a3/attachment.html