thr3ads.net - Backgroundrb devel - [Backgroundrb-devel] strange issue with backgroundrb behind apache/lighttpd [Nov 2006]

If this information is useful, please help other people find it:
Share via:

Chris Hall

2006-Nov-01 13:19 UTC

[Backgroundrb-devel] strange issue with backgroundrb behind apache/lighttpd

I''m getting strange data back from my worker whenever I run our rails
app with more than one dispatcher under lighttpd.

just to give some info on what the app does:

user submits data to rails app.  rails app hands off data to worker,
which goes and does it''s thing, storing results in an array of
response objects (the array is an attribute of the worker).  the
response object includes DRbUndumped.

Back in the browser, ajax requests are made at 1 sec intervals to the
rails app to check on how the worker is progressing.  the app calls a
method on the worker to get back any finished responses.  an array of
responses is passed back to the app and the data is used to update the
browser.  the size of the array is also used to decrement a counter to
show the user how much work is left to complete until it gets to zero.

When we have 1 dispatcher running, the app works great.  as soon as we
add 2 or more dispatchers to lighttpd, we get strange results in the
counter.  the counter will start displaying negative values.

Now, what i think is happening is now that we introduced 2
dispatchers, the ajax request that is run every second is not queuing
up as it would with 1 dispatcher and we have overlap of requests
getting back the same data.  i tested this by changing the request
interval to 2 seconds and the counter now shows the correct values.

but we have another problem.  when testing with 2 or more people at
the same time, we get the following error several times in the
development log:

Processing SearchController#get_responses (for 127.0.0.1 at 2006-11-01 07:47:07)
 [POST]
  Session ID: 671f4ba93603f34e036d9bbc65919ca7
  Parameters: {"action"=>"get_responses",
"controller"=>"search"}
getting responses


RangeError (0xdb677f22 is recycled object):
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:375:in
`_id2ref''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:375:in
`to_obj''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1402:in
`to_obj''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1704:in
`to_obj''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:613:in
`recv_request''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:911:in
`recv_request''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1530:in `init_with_cl
ient''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1542:in `setup_messag
e''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1494:in
`perform''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1589:in
`main_loop''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1585:in
`loop''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1585:in
`main_loop''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1581:in
`start''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1581:in
`main_loop''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1430:in
`run''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1427:in
`start''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1427:in
`run''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1347:in
`initialize''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1627:in
`new''
    (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1627:in `start_servic
e''
    (druby://localhost:42531) /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/breakp
oint.rb:375:in `activate_drb''
    (druby://localhost:42531) /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispat
cher.rb:81:in `prepare_breakpoint''
    (druby://localhost:42531) /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispat
cher.rb:68:in `prepare_application''
    (druby://localhost:42531) /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispat
cher.rb:37:in `dispatch''
    (druby://localhost:42531) /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_h
andler.rb:150:in `process_request''
    (druby://localhost:42531) /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_h
andler.rb:54:in `process!''
    (druby://localhost:42531) /usr/lib/ruby/site_ruby/1.8/fcgi.rb:612:in `each_c
gi''
    (druby://localhost:42531) /usr/lib/ruby/site_ruby/1.8/fcgi.rb:117:in `sessio
n''
    (druby://localhost:42531) /usr/lib/ruby/site_ruby/1.8/fcgi.rb:104:in `each_r
equest''
    (druby://localhost:42531) /usr/lib/ruby/site_ruby/1.8/fcgi.rb:36:in
`each''
    (druby://localhost:42531) /usr/lib/ruby/site_ruby/1.8/fcgi.rb:609:in `each_c
gi''
    (druby://localhost:42531) /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_h
andler.rb:53:in `process!''
    (druby://localhost:42531) /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_h
andler.rb:23:in `process!''
    (druby://localhost:42531) /home/chall/radrails/workspace/chip2/public/dispat
ch.fcgi:24
    (druby://localhost:22222) /usr/lib/ruby/1.8/drb/invokemethod.rb:10:in `block
_yield''
    (druby://localhost:22222) /usr/lib/ruby/1.8/drb/invokemethod.rb:17:in `perfo
rm_with_block''
    (druby://localhost:22222) /usr/lib/ruby/1.8/drb/invokemethod.rb:14:in
`each''
    /app/controllers/search_controller.rb:123:in
`__bind_1162385227_815142''
    /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/core_ext/
object/extending.rb:44:in `[]''
    /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/core_ext/
object/extending.rb:44:in `instance_exec''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_view/helpers/protot
ype_helper.rb:378:in `initialize''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:
687:in `new''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:
687:in `render_with_no_layout''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/layout.r
b:253:in `render_without_benchmark''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmar
king.rb:53:in `render''
    /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmar
king.rb:53:in `render''
    /app/controllers/search_controller.rb:110:in `get_responses''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:
941:in `send''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:
941:in `perform_action_without_filters''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.
rb:368:in `perform_action_without_benchmark''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmar
king.rb:69:in `perform_action_without_rescue''
    /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmar
king.rb:69:in `perform_action_without_rescue''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/rescue.r
b:82:in `perform_action''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:
408:in `send''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.
rb:377:in `process_without_session_management_support''
    /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/session_
management.rb:117:in `process''
    /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispatcher.rb:38:in
`dispatch''
    /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:150:in `process_
request''
    /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:54:in
`process!''
    /usr/lib/ruby/site_ruby/1.8/fcgi.rb:612:in `each_cgi''
    /usr/lib/ruby/site_ruby/1.8/fcgi.rb:117:in `session''
    /usr/lib/ruby/site_ruby/1.8/fcgi.rb:104:in `each_request''
    /usr/lib/ruby/site_ruby/1.8/fcgi.rb:36:in `each''
    /usr/lib/ruby/site_ruby/1.8/fcgi.rb:609:in `each_cgi''
    /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:53:in
`process!''
    /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:23:in
`process!''
    /home/chall/radrails/workspace/chip2/public/dispatch.fcgi:24


Rendering /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/te
mplates/rescues/layout.rhtml (500 Internal Error)


we can''t figure out where the heck this is happening.  there are only
2 references to some controller code on lines 123 and 110 which are

110: render :update do |page|

123: done_responses.each do |r|

if anyone has any idea on what might be happening, we would truly appreciate it.

Chris

skaar

2006-Nov-01 17:52 UTC

head link

[Backgroundrb-devel] strange issue with backgroundrb behind apache/lighttpd

any chance you could try to use ''results'' for this, rather
than an
instance variable? 

is there a single worker instance serving different user sessions?

/skaar


* Chris Hall (christopher.k.hall at gmail.com) [061101
07:50]:> I''m getting strange data back from my worker whenever I run our
rails
> app with more than one dispatcher under lighttpd.
> 
> just to give some info on what the app does:
> 
> user submits data to rails app.  rails app hands off data to worker,
> which goes and does it''s thing, storing results in an array of
> response objects (the array is an attribute of the worker).  the
> response object includes DRbUndumped.
> 
> Back in the browser, ajax requests are made at 1 sec intervals to the
> rails app to check on how the worker is progressing.  the app calls a
> method on the worker to get back any finished responses.  an array of
> responses is passed back to the app and the data is used to update the
> browser.  the size of the array is also used to decrement a counter to
> show the user how much work is left to complete until it gets to zero.
> 
> When we have 1 dispatcher running, the app works great.  as soon as we
> add 2 or more dispatchers to lighttpd, we get strange results in the
> counter.  the counter will start displaying negative values.
> 
> Now, what i think is happening is now that we introduced 2
> dispatchers, the ajax request that is run every second is not queuing
> up as it would with 1 dispatcher and we have overlap of requests
> getting back the same data.  i tested this by changing the request
> interval to 2 seconds and the counter now shows the correct values.
> 
> but we have another problem.  when testing with 2 or more people at
> the same time, we get the following error several times in the
> development log:
> 
> Processing SearchController#get_responses (for 127.0.0.1 at 2006-11-01
07:47:07)
>  [POST]
>   Session ID: 671f4ba93603f34e036d9bbc65919ca7
>   Parameters: {"action"=>"get_responses",
"controller"=>"search"}
> getting responses
> 
> 
> RangeError (0xdb677f22 is recycled object):
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:375:in
`_id2ref''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:375:in
`to_obj''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1402:in
`to_obj''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1704:in
`to_obj''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:613:in
`recv_request''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:911:in
`recv_request''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1530:in
`init_with_cl
> ient''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1542:in
`setup_messag
> e''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1494:in
`perform''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1589:in
`main_loop''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1585:in
`loop''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1585:in
`main_loop''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1581:in
`start''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1581:in
`main_loop''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1430:in
`run''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1427:in
`start''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1427:in
`run''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1347:in
`initialize''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1627:in
`new''
>     (druby://localhost:42531) /usr/lib/ruby/1.8/drb/drb.rb:1627:in
`start_servic
> e''
>     (druby://localhost:42531)
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/breakp
> oint.rb:375:in `activate_drb''
>     (druby://localhost:42531)
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispat
> cher.rb:81:in `prepare_breakpoint''
>     (druby://localhost:42531)
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispat
> cher.rb:68:in `prepare_application''
>     (druby://localhost:42531)
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispat
> cher.rb:37:in `dispatch''
>     (druby://localhost:42531)
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_h
> andler.rb:150:in `process_request''
>     (druby://localhost:42531)
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_h
> andler.rb:54:in `process!''
>     (druby://localhost:42531) /usr/lib/ruby/site_ruby/1.8/fcgi.rb:612:in
`each_c
> gi''
>     (druby://localhost:42531) /usr/lib/ruby/site_ruby/1.8/fcgi.rb:117:in
`sessio
> n''
>     (druby://localhost:42531) /usr/lib/ruby/site_ruby/1.8/fcgi.rb:104:in
`each_r
> equest''
>     (druby://localhost:42531) /usr/lib/ruby/site_ruby/1.8/fcgi.rb:36:in
`each''
>     (druby://localhost:42531) /usr/lib/ruby/site_ruby/1.8/fcgi.rb:609:in
`each_c
> gi''
>     (druby://localhost:42531)
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_h
> andler.rb:53:in `process!''
>     (druby://localhost:42531)
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_h
> andler.rb:23:in `process!''
>     (druby://localhost:42531)
/home/chall/radrails/workspace/chip2/public/dispat
> ch.fcgi:24
>     (druby://localhost:22222) /usr/lib/ruby/1.8/drb/invokemethod.rb:10:in
`block
> _yield''
>     (druby://localhost:22222) /usr/lib/ruby/1.8/drb/invokemethod.rb:17:in
`perfo
> rm_with_block''
>     (druby://localhost:22222) /usr/lib/ruby/1.8/drb/invokemethod.rb:14:in
`each''
>     /app/controllers/search_controller.rb:123:in
`__bind_1162385227_815142''
>    
/usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/core_ext/
> object/extending.rb:44:in `[]''
>    
/usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/core_ext/
> object/extending.rb:44:in `instance_exec''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_view/helpers/protot
> ype_helper.rb:378:in `initialize''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:
> 687:in `new''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:
> 687:in `render_with_no_layout''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/layout.r
> b:253:in `render_without_benchmark''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmar
> king.rb:53:in `render''
>     /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmar
> king.rb:53:in `render''
>     /app/controllers/search_controller.rb:110:in `get_responses''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:
> 941:in `send''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:
> 941:in `perform_action_without_filters''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.
> rb:368:in `perform_action_without_benchmark''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmar
> king.rb:69:in `perform_action_without_rescue''
>     /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmar
> king.rb:69:in `perform_action_without_rescue''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/rescue.r
> b:82:in `perform_action''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:
> 408:in `send''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.
> rb:377:in `process_without_session_management_support''
>    
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/session_
> management.rb:117:in `process''
>     /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispatcher.rb:38:in
`dispatch''
>     /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:150:in
`process_
> request''
>     /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:54:in
`process!''
>     /usr/lib/ruby/site_ruby/1.8/fcgi.rb:612:in `each_cgi''
>     /usr/lib/ruby/site_ruby/1.8/fcgi.rb:117:in `session''
>     /usr/lib/ruby/site_ruby/1.8/fcgi.rb:104:in `each_request''
>     /usr/lib/ruby/site_ruby/1.8/fcgi.rb:36:in `each''
>     /usr/lib/ruby/site_ruby/1.8/fcgi.rb:609:in `each_cgi''
>     /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:53:in
`process!''
>     /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:23:in
`process!''
>     /home/chall/radrails/workspace/chip2/public/dispatch.fcgi:24
> 
> 
> Rendering
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/te
> mplates/rescues/layout.rhtml (500 Internal Error)
> 
> 
> we can''t figure out where the heck this is happening.  there are
only
> 2 references to some controller code on lines 123 and 110 which are
> 
> 110: render :update do |page|
> 
> 123: done_responses.each do |r|
> 
> if anyone has any idea on what might be happening, we would truly
appreciate it.
> 
> Chris
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
-- 
----------------------------------------------------------------------
|\|\             where in the       |          s_u_b_s_t_r_u_c_t_i_o_n 
| | >===========  W.A.S.T.E.        |                  genarratologies 
|/|/    (_)     is the wisdom       |                  skaar at waste.org
----------------------------------------------------------------------

Possibly Parallel Threads

Search for more seemingly similar threads

Backgroundrb devel - Nov 2006 - strange issue with backgroundrb behind apache/lighttpd

[Backgroundrb-devel] strange issue with backgroundrb behind apache/lighttpd

[Backgroundrb-devel] strange issue with backgroundrb behind apache/lighttpd

Possibly Parallel Threads