Robert Bjarnason
2007-Jan-19 11:32 UTC
[Backgroundrb-devel] Stopping DRb after using BackgrounDRb::MiddleManDRbObject.init
Hi, I''m having a problem where connections started through BackgrounDRb::MiddleManDRbObject.init are not stopped, resulting in an ever growing established connections between my Mongrels and remote backgroundrbs I added DRb.stop_service to my code executed after requests that init MiddleManDRbObjects but that does not help. Any ideas? Thanks, Robert Bjarnason
Michael D''Auria
2007-Jan-19 15:31 UTC
[Backgroundrb-devel] Stopping DRb after using BackgrounDRb::MiddleManDRbObject.init
You need to clean up your workers yourself, so toss in a self.delete inside of the do_work method to have the worker clean up after itself or you can perform a delete on the worker on the outside using it''s id. .: Michael :. On 1/19/07, Robert Bjarnason <robert.bjarnason at gmail.com> wrote:> > Hi, > > I''m having a problem where connections started through > BackgrounDRb::MiddleManDRbObject.init are not stopped, resulting in an > ever growing established connections between my Mongrels and remote > backgroundrbs > > I added DRb.stop_service to my code executed after requests that init > MiddleManDRbObjects but that does not help. > > Any ideas? > > Thanks, > Robert Bjarnason > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070119/1f89d752/attachment.html
Jacob Robbins
2007-Jan-19 16:15 UTC
[Backgroundrb-devel] Stopping DRb after using BackgrounDRb::MiddleManDRbObject.init
Robert, Is there more than one drb connection from each mongrel? I would think that each rails app would have one drb connection to each of the backgroundrb servers that it has accessed. I''m not quite understanding what the problem is, is it that you have N mongrels and M backgroundrb serves and it''s too much to have N*M drb connections? Or is it that each mongrel is constantly making more new drb connections instead of reusing the existing ones? -Jacob Michael D''Auria wrote:> You need to clean up your workers yourself, so toss in a self.delete > inside of the do_work method to have the worker clean up after itself or > you can perform a delete on the worker on the outside using it''s id. > > .: Michael :. > > > On 1/19/07, *Robert Bjarnason* <robert.bjarnason at gmail.com > <mailto:robert.bjarnason at gmail.com>> wrote: > > Hi, > > I''m having a problem where connections started through > BackgrounDRb::MiddleManDRbObject.init are not stopped, resulting in an > ever growing established connections between my Mongrels and remote > backgroundrbs > > I added DRb.stop_service to my code executed after requests that init > MiddleManDRbObjects but that does not help. > > Any ideas? > > Thanks, > Robert Bjarnason > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > <mailto:Backgroundrb-devel at rubyforge.org> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
Robert Bjarnason
2007-Jan-19 23:08 UTC
[Backgroundrb-devel] Stopping DRb after using BackgrounDRb::MiddleManDRbObject.init
Hi Jacob, Each mongrel is constantly making new drb connections instead of reusing or closing existing ones. Having N*M drb connections would not be a problem. Thanks, Robert Jacob Robbins wrote:> Robert, > Is there more than one drb connection from each mongrel? I would > think that each rails app would have one drb connection to each of the > backgroundrb servers that it has accessed. > > I''m not quite understanding what the problem is, is it that you have > N mongrels and M backgroundrb serves and it''s too much to have N*M drb > connections? Or is it that each mongrel is constantly making more new > drb connections instead of reusing the existing ones? > > -Jacob > > > > Michael D''Auria wrote: > >> You need to clean up your workers yourself, so toss in a self.delete >> inside of the do_work method to have the worker clean up after itself or >> you can perform a delete on the worker on the outside using it''s id. >> >> .: Michael :. >> >> >> On 1/19/07, *Robert Bjarnason* <robert.bjarnason at gmail.com >> <mailto:robert.bjarnason at gmail.com>> wrote: >> >> Hi, >> >> I''m having a problem where connections started through >> BackgrounDRb::MiddleManDRbObject.init are not stopped, resulting in an >> ever growing established connections between my Mongrels and remote >> backgroundrbs >> >> I added DRb.stop_service to my code executed after requests that init >> MiddleManDRbObjects but that does not help. >> >> Any ideas? >> >> Thanks, >> Robert Bjarnason >> >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> <mailto:Backgroundrb-devel at rubyforge.org> >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >> >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >> > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > >
Ezra Zygmuntowicz
2007-Jan-19 23:18 UTC
[Backgroundrb-devel] Stopping DRb after using BackgrounDRb::MiddleManDRbObject.init
Robert- Is there some reason that you are calling BackgrounDRb::MiddleManDRbObject.init directly? You should let the BackgrounDRb::MiddleManRailsProxy take care of that. If you run it the normal way then each mongrel will only make one connection to the drb server. If you keep calling BackgrounDRb::MiddleManDRbObject.init you are going to keep making more connections. -Ezra On Jan 19, 2007, at 3:08 PM, Robert Bjarnason wrote:> Hi Jacob, > > Each mongrel is constantly making new drb connections instead of > reusing > or closing existing ones. Having N*M drb connections would not be a > problem. > > Thanks, > Robert > > Jacob Robbins wrote: >> Robert, >> Is there more than one drb connection from each mongrel? I would >> think that each rails app would have one drb connection to each of >> the >> backgroundrb servers that it has accessed. >> >> I''m not quite understanding what the problem is, is it that you >> have >> N mongrels and M backgroundrb serves and it''s too much to have N*M >> drb >> connections? Or is it that each mongrel is constantly making more new >> drb connections instead of reusing the existing ones? >> >> -Jacob >> >> >> >> Michael D''Auria wrote: >> >>> You need to clean up your workers yourself, so toss in a self.delete >>> inside of the do_work method to have the worker clean up after >>> itself or >>> you can perform a delete on the worker on the outside using it''s id. >>> >>> .: Michael :. >>> >>> >>> On 1/19/07, *Robert Bjarnason* <robert.bjarnason at gmail.com >>> <mailto:robert.bjarnason at gmail.com>> wrote: >>> >>> Hi, >>> >>> I''m having a problem where connections started through >>> BackgrounDRb::MiddleManDRbObject.init are not stopped, >>> resulting in an >>> ever growing established connections between my Mongrels and >>> remote >>> backgroundrbs >>> >>> I added DRb.stop_service to my code executed after requests >>> that init >>> MiddleManDRbObjects but that does not help. >>> >>> Any ideas? >>> >>> Thanks, >>> Robert Bjarnason >>> >>> _______________________________________________ >>> Backgroundrb-devel mailing list >>> Backgroundrb-devel at rubyforge.org >>> <mailto:Backgroundrb-devel at rubyforge.org> >>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >>> >>> >>> >>> -------------------------------------------------------------------- >>> ---- >>> >>> _______________________________________________ >>> Backgroundrb-devel mailing list >>> Backgroundrb-devel at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >>> >> >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >> >> > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel-- Ezra Zygmuntowicz -- Lead Rails Evangelist -- ez at engineyard.com -- Engine Yard, Serious Rails Hosting -- (866) 518-YARD (9273)
Robert Bjarnason
2007-Jan-19 23:37 UTC
[Backgroundrb-devel] Stopping DRb after using BackgrounDRb::MiddleManDRbObject.init
Hi Ezra, I would like to control which uri is used to start the DRb connection for Rails based load-balancing purposes from multiple mongrels connecting to multiple backgroundrbs running on several boxes. I don''t see how I could BackgrounDRb::MiddleManRailsProxy for this purpose. Any ideas? Thanks, Robert Ezra Zygmuntowicz wrote:> > Robert- > > Is there some reason that you are calling > BackgrounDRb::MiddleManDRbObject.init directly? You should let the > BackgrounDRb::MiddleManRailsProxy take care of that. If you run it the > normal way then each mongrel will only make one connection to the drb > server. If you keep calling BackgrounDRb::MiddleManDRbObject.init > you are going to keep making more connections. > > > -Ezra > > On Jan 19, 2007, at 3:08 PM, Robert Bjarnason wrote: > >> Hi Jacob, >> >> Each mongrel is constantly making new drb connections instead of reusing >> or closing existing ones. Having N*M drb connections would not be a >> problem. >> >> Thanks, >> Robert >> >> Jacob Robbins wrote: >>> Robert, >>> Is there more than one drb connection from each mongrel? I would >>> think that each rails app would have one drb connection to each of the >>> backgroundrb servers that it has accessed. >>> >>> I''m not quite understanding what the problem is, is it that you have >>> N mongrels and M backgroundrb serves and it''s too much to have N*M drb >>> connections? Or is it that each mongrel is constantly making more new >>> drb connections instead of reusing the existing ones? >>> >>> -Jacob >>> >>> >>> >>> Michael D''Auria wrote: >>> >>>> You need to clean up your workers yourself, so toss in a self.delete >>>> inside of the do_work method to have the worker clean up after >>>> itself or >>>> you can perform a delete on the worker on the outside using it''s id. >>>> >>>> .: Michael :. >>>> >>>> >>>> On 1/19/07, *Robert Bjarnason* <robert.bjarnason at gmail.com >>>> <mailto:robert.bjarnason at gmail.com>> wrote: >>>> >>>> Hi, >>>> >>>> I''m having a problem where connections started through >>>> BackgrounDRb::MiddleManDRbObject.init are not stopped, >>>> resulting in an >>>> ever growing established connections between my Mongrels and >>>> remote >>>> backgroundrbs >>>> >>>> I added DRb.stop_service to my code executed after requests >>>> that init >>>> MiddleManDRbObjects but that does not help. >>>> >>>> Any ideas? >>>> >>>> Thanks, >>>> Robert Bjarnason >>>> >>>> _______________________________________________ >>>> Backgroundrb-devel mailing list >>>> Backgroundrb-devel at rubyforge.org >>>> <mailto:Backgroundrb-devel at rubyforge.org> >>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> _______________________________________________ >>>> Backgroundrb-devel mailing list >>>> Backgroundrb-devel at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >>>> >>> >>> _______________________________________________ >>> Backgroundrb-devel mailing list >>> Backgroundrb-devel at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >>> >>> >> >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > -- Ezra Zygmuntowicz-- Lead Rails Evangelist > -- ez at engineyard.com > -- Engine Yard, Serious Rails Hosting > -- (866) 518-YARD (9273) > > >
Ezra Zygmuntowicz
2007-Jan-20 00:01 UTC
[Backgroundrb-devel] Stopping DRb after using BackgrounDRb::MiddleManDRbObject.init
On Jan 19, 2007, at 3:37 PM, Robert Bjarnason wrote:> Hi Ezra, > > I would like to control which uri is used to start the DRb > connection for Rails based load-balancing purposes from multiple > mongrels connecting to multiple backgroundrbs running on several > boxes. I don''t see how I could BackgrounDRb::MiddleManRailsProxy > for this purpose. Any ideas? > > Thanks, > RobertRobert- Ahh ok I see. Well maybe something like this(untested!) # start ---- require ''pathname'' BACKGROUNDRB_ROOT = Pathname.new(RAILS_ROOT).realpath.to_s require ''backgroundrb'' # Set up a MiddleMan for the BackgrounDRb server configured for this # particular instance of Rails. class BackgrounDRb::MiddleManProxyBalancer def self.init(server_pool={}) config = { :config => "#{BACKGROUNDRB_ROOT}/config/ backgroundrb.yml" } BackgrounDRb::Config.setup(config) @@proxy_members = {} server_pool.each {|num, drburi| @@proxy_members[num] = BackgrounDRb::MiddleManDRbObject.init (:uri => drburi) } return self end def self.[](key) @@proxy_members[key] end end server_pool = { 1 => "druby://localhost:22221", 2 => "druby://localhost:22222", 3 => "druby://10.0.0.123:22221", 4 => "druby://10.0.0.123:22221" } MiddleManPool = BackgrounDRb::MiddleManProxyBalancer.init(server_pool) # end ---- Then you would use the MiddleManPool in your rails apps instead of the normal rails proxy. You would have to key the middleman you want my the number in the server_pool. So it would look like this to use: session[:jobkey] = MiddleManPool[1].new_worker( :class => :someshit ) session[:otherjobkey] = MiddleManPool[2].new_worker( :class => :someshit ) worker = MiddleManPool[1].worker(session[:jobkey]) worker.results.to_hash worker = MiddleManPool[2].worker(session[: otherjobkey]) worker.results.to_hash This is just a sketch but I think you could take this and run with it. You would use the above to replace the BackgrounDRb::MiddleManRailsProxy in the lib dir of the plugin so that rails would load your new proxy instead of the old connection stuff. So this would make sure that every mongrel had a connection to every server in the server_pool. ANd you woudl just need to always use the number of which drb server you want from the pool. But it caches the connection so that each mongrel only gets one connection to every drb server. No new connection should be made after the thing boots because the connections are cached. Make sense? Please give this or something like this a shot and let us know how it goes for you. I am interested in adding something like this to the project. Cheers- -- Ezra Zygmuntowicz -- Lead Rails Evangelist -- ez at engineyard.com -- Engine Yard, Serious Rails Hosting -- (866) 518-YARD (9273)