Rails - Oct 2008 - Long running query causes Mongrel/Rails to block completely?

Hi.

I''m troubleshooting a problem, and I would love some help
understanding what is going on.

The problem is that Mongrel/Rails appears to hang or completely block
if one of the users initiates a very slow, long-running query (via an
ad-hoc report generation tool from within the app).

For example, user A does a big query that takes minutes to complete.
User B tries to do anything, but Rails seems completely dead.

Should Rails be blocking on that query, effectively a single-user
server?  (I certainly wouldn''t expect this...)

Secondly, for what it''s worth, I''m running a mongrel cluster,
but that
isn''t helping right now.

More info:

"top" shows the host machine completely idle.
development.log shows only user A''s query SQL, but it does not show
user B''s request for a new page.
When user A''s query finally completes, then suddenly Rails comes back
to life, and user B gets a response.

This is a serious problem for me right now.  It must be a
configuration problem, because there''s no way Rails is designed to
behave this way...?

Oh also, the database is hosted on another machine, and it''s Oracle 9.

Thanks!
Michael
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

On 29 Oct 2008, at 14:31, michael_teter wrote:
>
> Hi.
>
> I''m troubleshooting a problem, and I would love some help
> understanding what is going on.
>
> The problem is that Mongrel/Rails appears to hang or completely block
> if one of the users initiates a very slow, long-running query (via an
> ad-hoc report generation tool from within the app).
>
> For example, user A does a big query that takes minutes to complete.
> User B tries to do anything, but Rails seems completely dead.
>
> Should Rails be blocking on that query, effectively a single-user
> server?  (I certainly wouldn''t expect this...)
>
Yup that''s the way it is. load balancing across several mongrels helps
a bit, but not massively because most load balancers just spread the  
load equally - they don''t prioritize mongrels that aren''t
processing a
request.
Rails 2.2 is thread safe, but MRI''s less than stellar threading means  
that won''t make a lot of difference if you are using MRI. For jruby  
things could get very interesting.

Fred
> Secondly, for what it''s worth, I''m running a mongrel
cluster, but that
> isn''t helping right now.
>
> More info:
>
> "top" shows the host machine completely idle.
> development.log shows only user A''s query SQL, but it does not
show
> user B''s request for a new page.
> When user A''s query finally completes, then suddenly Rails comes
back
> to life, and user B gets a response.
>
> This is a serious problem for me right now.  It must be a
> configuration problem, because there''s no way Rails is designed to
> behave this way...?
>
> Oh also, the database is hosted on another machine, and it''s
Oracle 9.
>
> Thanks!
> Michael
> >

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

On Oct 30, 2:00 am, Frederick Cheung
<frederick.che...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
> On 29 Oct 2008, at 14:31, michael_teter wrote:
>
>
>
>
>
> > Hi.
>
> > I''m troubleshooting a problem, and I would love some help
> > understanding what is going on.
>
> > The problem is that Mongrel/Rails appears to hang or completely block
> > if one of the users initiates a very slow, long-running query (via an
> > ad-hoc report generation tool from within the app).
>
> > For example, user A does a big query that takes minutes to complete.
> > User B tries to do anything, but Rails seems completely dead.
>
> > Should Rails be blocking on that query, effectively a single-user
> > server?  (I certainly wouldn''t expect this...)
>
> Yup that''s the way it is. load balancing across several mongrels
helps  
> a bit, but not massively because most load balancers just spread the  
> load equally - they don''t prioritize mongrels that aren''t
processing a  
> request.
> Rails 2.2 is thread safe, but MRI''s less than stellar threading
means  
> that won''t make a lot of difference if you are using MRI. For
jruby  
> things could get very interesting.
>
Wow,  Looks like I''m going to have to try out jruby/rails 2.2 what
with jruby using java/os threads.  Anyone started looking at this with
2.2 RC or whatever the latest is?

http://blog.headius.com/2008/08/qa-what-thread-safe-rails-means.html

--
Daniel Bush
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

I don''t know how practical this is, but I can envision a system
whereby each process wraps a request/response with a Start End message
to a load balancer so it will know which processes are busy and which
are not.  Further, the load balancer could keep statistics that might
be useful in optimizing the balance behavior.
> Yup that''s the way it is. load balancing across several mongrels
helps  
> a bit, but not massively because most load balancers just spread the  
> load equally - they don''t prioritize mongrels that aren''t
processing a  
> request.
> Rails 2.2 is thread safe, but MRI''s less than stellar threading
means  
> that won''t make a lot of difference if you are using MRI. For
jruby  
> things could get very interesting.
>
> Fred
>
> > Secondly, for what it''s worth, I''m running a mongrel
cluster, but that
> > isn''t helping right now.
>
> > More info:
>
> > "top" shows the host machine completely idle.
> > development.log shows only user A''s query SQL, but it does
not show
> > user B''s request for a new page.
> > When user A''s query finally completes, then suddenly Rails
comes back
> > to life, and user B gets a response.
>
> > This is a serious problem for me right now.  It must be a
> > configuration problem, because there''s no way Rails is
designed to
> > behave this way...?
>
> > Oh also, the database is hosted on another machine, and it''s
Oracle 9.
>
> > Thanks!
> > Michael
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

A slow SQL query woudn''t freeze rails 2.2 as it freezes 2.1 now as the
MRI removes threads that are waiting for IO handles to be available
from the running list, so this should not be a big issue with the new
rails, but JRuby is a definitely better option today with the latest
rails.

-
Maurício Linhares
http://alinhavado.wordpress.com/ (pt-br) | http://blog.codevader.com/ (en)
João Pessoa, PB, +55 83 8867-7208



On Wed, Oct 29, 2008 at 12:26 PM, michael_teter
<michael.teter-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
>
> I don''t know how practical this is, but I can envision a system
> whereby each process wraps a request/response with a Start End message
> to a load balancer so it will know which processes are busy and which
> are not.  Further, the load balancer could keep statistics that might
> be useful in optimizing the balance behavior.
>
>> Yup that''s the way it is. load balancing across several
mongrels helps
>> a bit, but not massively because most load balancers just spread the
>> load equally - they don''t prioritize mongrels that
aren''t processing a
>> request.
>> Rails 2.2 is thread safe, but MRI''s less than stellar
threading means
>> that won''t make a lot of difference if you are using MRI. For
jruby
>> things could get very interesting.
>>
>> Fred
>>
>> > Secondly, for what it''s worth, I''m running a
mongrel cluster, but that
>> > isn''t helping right now.
>>
>> > More info:
>>
>> > "top" shows the host machine completely idle.
>> > development.log shows only user A''s query SQL, but it
does not show
>> > user B''s request for a new page.
>> > When user A''s query finally completes, then suddenly
Rails comes back
>> > to life, and user B gets a response.
>>
>> > This is a serious problem for me right now.  It must be a
>> > configuration problem, because there''s no way Rails is
designed to
>> > behave this way...?
>>
>> > Oh also, the database is hosted on another machine, and
it''s Oracle 9.
>>
>> > Thanks!
>> > Michael
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

I was actually running in JRuby earlier on in my development process,
but "something changed" and it appeared that the Oracle driver and
JRuby stopped playing nicely together.

Perhaps I should revisit JRuby...

Thanks for the responses!

On Oct 29, 10:29 am, "Maurício Linhares"
<mauricio.linha...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
> A slow SQL query woudn''t freeze rails 2.2 as it freezes 2.1 now as
the
> MRI removes threads that are waiting for IO handles to be available
> from the running list, so this should not be a big issue with the new
> rails, but JRuby is a definitely better option today with the latest
> rails.
>
> -
> Maurício Linhareshttp://alinhavado.wordpress.com/(pt-br)
|http://blog.codevader.com/(en)
> João Pessoa, PB, +55 83 8867-7208
>
> On Wed, Oct 29, 2008 at 12:26 PM, michael_teter
<michael.te...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> > I don''t know how practical this is, but I can envision a
system
> > whereby each process wraps a request/response with a Start End message
> > to a load balancer so it will know which processes are busy and which
> > are not.  Further, the load balancer could keep statistics that might
> > be useful in optimizing the balance behavior.
>
> >> Yup that''s the way it is. load balancing across several
mongrels helps
> >> a bit, but not massively because most load balancers just spread
the
> >> load equally - they don''t prioritize mongrels that
aren''t processing a
> >> request.
> >> Rails 2.2 is thread safe, but MRI''s less than stellar
threading means
> >> that won''t make a lot of difference if you are using MRI.
For jruby
> >> things could get very interesting.
>
> >> Fred
>
> >> > Secondly, for what it''s worth, I''m running
a mongrel cluster, but that
> >> > isn''t helping right now.
>
> >> > More info:
>
> >> > "top" shows the host machine completely idle.
> >> > development.log shows only user A''s query SQL, but
it does not show
> >> > user B''s request for a new page.
> >> > When user A''s query finally completes, then suddenly
Rails comes back
> >> > to life, and user B gets a response.
>
> >> > This is a serious problem for me right now.  It must be a
> >> > configuration problem, because there''s no way Rails
is designed to
> >> > behave this way...?
>
> >> > Oh also, the database is hosted on another machine, and
it''s Oracle 9.
>
> >> > Thanks!
> >> > Michael
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

On Oct 30, 2:33 am, michael_teter
<michael.te...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
> I was actually running in JRuby earlier on in my development process,
> but "something changed" and it appeared that the Oracle driver
and
> JRuby stopped playing nicely together.
>
> Perhaps I should revisit JRuby...
>
> Thanks for the responses!
>
If it''s a report you could pass it off to a separate process. 
You''ll
have to add some extra ui so the user can go back and check or get
notified when it finishes.

http://backgroundrb.rubyforge.org/rails/

--
Daniel Bush
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

On 29 Oct 2008, at 15:29, Maurício Linhares wrote:
>
> A slow SQL query woudn''t freeze rails 2.2 as it freezes 2.1 now as
the
> MRI removes threads that are waiting for IO handles to be available
> from the running list, so this should not be a big issue with the new
It depends. C extensions can freeze the entire ruby interpreter, right  
now the native mysql driver is one of those (but see the work the  
neverblock guys have been doing). I don''t recall what other database  
drivers do.

Fred
> rails, but JRuby is a definitely better option today with the latest
> rails.
>

> -
> Maurício Linhares
> http://alinhavado.wordpress.com/ (pt-br) | http:// 
> blog.codevader.com/ (en)
> João Pessoa, PB, +55 83 8867-7208
>
>
>
> On Wed, Oct 29, 2008 at 12:26 PM, michael_teter
<michael.teter-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
> > wrote:
>>
>> I don''t know how practical this is, but I can envision a
system
>> whereby each process wraps a request/response with a Start End  
>> message
>> to a load balancer so it will know which processes are busy and which
>> are not.  Further, the load balancer could keep statistics that might
>> be useful in optimizing the balance behavior.
>>
>>> Yup that''s the way it is. load balancing across several
mongrels
>>> helps
>>> a bit, but not massively because most load balancers just spread
the
>>> load equally - they don''t prioritize mongrels that
aren''t
>>> processing a
>>> request.
>>> Rails 2.2 is thread safe, but MRI''s less than stellar
threading
>>> means
>>> that won''t make a lot of difference if you are using MRI.
For jruby
>>> things could get very interesting.
>>>
>>> Fred
>>>
>>>> Secondly, for what it''s worth, I''m running a
mongrel cluster, but
>>>> that
>>>> isn''t helping right now.
>>>
>>>> More info:
>>>
>>>> "top" shows the host machine completely idle.
>>>> development.log shows only user A''s query SQL, but it
does not show
>>>> user B''s request for a new page.
>>>> When user A''s query finally completes, then suddenly
Rails comes
>>>> back
>>>> to life, and user B gets a response.
>>>
>>>> This is a serious problem for me right now.  It must be a
>>>> configuration problem, because there''s no way Rails is
designed to
>>>> behave this way...?
>>>
>>>> Oh also, the database is hosted on another machine, and
it''s
>>>> Oracle 9.
>>>
>>>> Thanks!
>>>> Michael
>>>
>>
>
> >

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Maurício Linhares a écrit, le 10/29/2008 04:29 PM :
> A slow SQL query woudn''t freeze rails 2.2 as it freezes 2.1 now as
the
> MRI removes threads that are waiting for IO handles to be available
> from the running list, so this should not be a big issue with the new
> rails, but JRuby is a definitely better option today with the latest
> rails.
>
>   
I''m not sure if the threading in Rails 2.2 brings any benefit for JRuby
vs MRI. Last time I checked, even if JRuby used OS threads it used a
global lock to make sure only one was running at one time (maybe to
protect itself against race conditions in some Ruby code or parts of the
implementation that isn''t fully threadsafe yet). Did it change or are
there some implementation details making it more thread friendly than MRI ?

That said, JRuby could be better than MRI in many cases with Rails 2.1
already, I''m just wondering if it recently got even better :-)

Lionel

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

On Oct 29, 7:43 pm, Lionel Bouton
<lionel-subscript...-WTamNBQcZIx7tPAFqOLdPg@public.gmane.org>
wrote:
> Maurício Linhares a écrit, le 10/29/2008 04:29 PM :
>
> > A slow SQL query woudn''t freeze rails 2.2 as it freezes 2.1
now as the
> > MRI removes threads that are waiting for IO handles to be available
> > from the running list, so this should not be a big issue with the new
> > rails, but JRuby is a definitely better option today with the latest
> > rails.
>
> I''m not sure if the threading in Rails 2.2 brings any benefit for
JRuby
> vs MRI. Last time I checked, even if JRuby used OS threads it used a
> global lock to make sure only one was running at one time (maybe to
> protect itself against race conditions in some Ruby code or parts of the
> implementation that isn''t fully threadsafe yet). Did it change or
are
That''s just wrong (YARV/ruby 1.9 is a bit like that - you may have got
the idea from there). See the Charles Nutter''s blog post (http://
blog.headius.com/2008/08/qa-what-thread-safe-rails-means.html ) which
Daniel already linked to.

Fred
> there some implementation details making it more thread friendly than MRI ?
>
> That said, JRuby could be better than MRI in many cases with Rails 2.1
> already, I''m just wondering if it recently got even better :-)
>
> Lionel
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Frederick Cheung wrote:
> On Oct 29, 7:43�pm, Lionel Bouton
<lionel-subscript...-WTamNBQcZIyWDMOabwocfg@public.gmane.orge>
> wrote:
>> global lock to make sure only one was running at one time (maybe to
>> protect itself against race conditions in some Ruby code or parts of
the
>> implementation that isn''t fully threadsafe yet). Did it change
or are
> 
> That''s just wrong (YARV/ruby 1.9 is a bit like that - you may have
got
> the idea from there). See the Charles Nutter''s blog post (http://
> blog.headius.com/2008/08/qa-what-thread-safe-rails-means.html ) which
> Daniel already linked to.
> 
> Fred
Just benchmarking JRuby against Ruby 1.8 in Rails 2.2 right now. JRuby 
is slower.
-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---