Backgroundrb devel - Jan 2008 - Cannot connect when spawning new workers on demand

Hi!

I''m using the latest&greatest backgroundrb with rails 2.0.2 on
ubuntu
dapper.

I''ve made an app that lets you run a query on several servers at once,
which are selected at runtime. Therefore, for each server that is
selected, I spawn a new worker and assign its work.
After everything is completed, the workers are deleted.

I often get this error:
BackgrounDRb::BdrbConnError in Multi com/indexController#uitvoeren 
Not able to connect

Here''s the code that spawns the workers: (sorry for the linewraps)

  def uitvoeren
    # put selected servers in a session
    session[:selectedservers] = params[:servers]
    session[:running_cmd] = 1
    @selected_servers = System.find_all_by_id(params[:servers])
    @selected_servers.each do |svr|
      # spawn new process
      MiddleMan.new_worker(:worker => :remote_command_worker, :job_key
=> svr.name)
      MiddleMan.ask_work(:worker => :remote_command_worker, :job_key =>
svr.name, :worker_method => :do_work, :data => {:svr =>
svr.name, :command =>
MultiComCommand.find_by_id(params[:commando]).command})
    end
  end

When I select only a few servers, it works just fine! There is no
definable threshold, I''ve had failures with only 4 servers selected and
success with 12 servers selected.

After such a connection error, the worker processes are still running:
$ ps -ef | grep worker
09:08 ?        00:00:00 ruby log_worker               
09:08 ?        00:00:00 ruby remote_command_worker    
09:08 ?        00:00:00 ruby remote_command_worker    
09:08 ?        00:00:00 ruby remote_command_worker    
09:08 ?        00:00:00 ruby remote_command_worker    
09:08 ?        00:00:00 ruby remote_command_worker 

Your help is VERY MUCH appreciated!

Thanks,
Ger.

Hi,

On Jan 22, 2008 1:59 PM, Ger Apeldoorn <gapeldoorn at wehkamp.nl>
wrote:
> Hi!
>
> I''m using the latest&greatest backgroundrb with rails 2.0.2 on
ubuntu
> dapper.
>
> I''ve made an app that lets you run a query on several servers at
once,
> which are selected at runtime. Therefore, for each server that is
> selected, I spawn a new worker and assign its work.
> After everything is completed, the workers are deleted.
>
> I often get this error:
> BackgrounDRb::BdrbConnError in Multi com/indexController#uitvoeren
> Not able to connect
>
> Here''s the code that spawns the workers: (sorry for the linewraps)
>
>   def uitvoeren
>     # put selected servers in a session
>     session[:selectedservers] = params[:servers]
>     session[:running_cmd] = 1
>     @selected_servers = System.find_all_by_id(params[:servers])
>     @selected_servers.each do |svr|
>       # spawn new process
>       MiddleMan.new_worker(:worker => :remote_command_worker, :job_key
> => svr.name)
>       MiddleMan.ask_work(:worker => :remote_command_worker, :job_key
=>
> svr.name, :worker_method => :do_work, :data => {:svr =>
> svr.name, :command =>
> MultiComCommand.find_by_id(params[:commando]).command})
>     end
>   end
>
> When I select only a few servers, it works just fine! There is no
> definable threshold, I''ve had failures with only 4 servers
selected and
> success with 12 servers selected.
>
> After such a connection error, the worker processes are still running:
> $ ps -ef | grep worker
> 09:08 ?        00:00:00 ruby log_worker
> 09:08 ?        00:00:00 ruby remote_command_worker
> 09:08 ?        00:00:00 ruby remote_command_worker
> 09:08 ?        00:00:00 ruby remote_command_worker
> 09:08 ?        00:00:00 ruby remote_command_worker
> 09:08 ?        00:00:00 ruby remote_command_worker
>
> Your help is VERY MUCH appreciated!
Did you check backgroundrb_debug.log ?
You can turn on debugging to foreground with:

:log: foreground

option to further debug this. Something is killing BackgrounDRb master
process. Can you sync with trunk and enable foreground logging and
check on this?

Also, if possible, can we see your worker code?

Hi!

Thanks a lot for your reply!
> Did you check backgroundrb_debug.log ?
> You can turn on debugging to foreground with:
> 
> :log: foreground
Where should I put :log: foreground?

Anyway, this comes from log/backgroundrb_debug.log:

--------------------%<--------------------

$ tail -f log/backgroundrb_debug.log 
000000073:      type:start_worker:
                                   worker:remote_command_worker:

job_key"

server06
{:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server06"}
000000115{
:       type:
             do_work:
                     worker:remote_command_worker:     data{:
                                                              command"
megacommandheresvr"
server06:
        job_key:worker_method;
{:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server06",
:command=>"megacommandhere"}, :worker=>:remote_command_worker,
:job_key=>"server06"}
000000073:      type:start_worker:
                                   worker:remote_command_worker:

job_key"

server12
{:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server12"}
000000115{
:       type:
             do_work:
                     worker:remote_command_worker:     data{:
                                                              command"
megacommandheresvr"
server12:
        job_key:worker_method;
{:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server12",
:command=>"megacommandhere"}, :worker=>:remote_command_worker,
:job_key=>"server12"}

--------------------%<--------------------
> 
> option to further debug this. Something is killing BackgrounDRb master
> process. Can you sync with trunk and enable foreground logging and
> check on this?
Is there an easy way to update a plugin after it''s been installed
besides removing it and checking it out again?
> Also, if possible, can we see your worker code?
Here''s my worker:

class RemoteCommandWorker < BackgrounDRb::MetaWorker
  set_worker_name :remote_command_worker
  set_no_auto_load :true

  def create(args = nil)
    register_status("processing")
  end

  def do_work(args = nil)
    result = String.new
    result = "Result for #{args[:command]}:\n #{%
x[running_a_shell_script_here]}"
    register_status(result)
  end
end

Thanks,
Ger.

Hi again,

Can I perhaps provide more information to help solve this problem? If
so, I''d be happy to give it!

Thanks,
Ger.

On Tue, 2008-01-22 at 11:03 +0100, Ger Apeldoorn wrote:
> Hi!
> 
> Thanks a lot for your reply!
> 
> > Did you check backgroundrb_debug.log ?
> > You can turn on debugging to foreground with:
> > 
> > :log: foreground
> 
> Where should I put :log: foreground?
> 
> Anyway, this comes from log/backgroundrb_debug.log:
> 
> --------------------%<--------------------
> 
> $ tail -f log/backgroundrb_debug.log 
> 000000073:      type:start_worker:
>                                    worker:remote_command_worker:
> 
> job_key"
> 
> server06
> {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server06"}
> 000000115{
> :       type:
>              do_work:
>                      worker:remote_command_worker:     data{:
>                                                               command"
> megacommandheresvr"
> server06:
>         job_key:worker_method;
> {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server06",
:command=>"megacommandhere"}, :worker=>:remote_command_worker,
:job_key=>"server06"}
> 000000073:      type:start_worker:
>                                    worker:remote_command_worker:
> 
> job_key"
> 
> server12
> {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server12"}
> 000000115{
> :       type:
>              do_work:
>                      worker:remote_command_worker:     data{:
>                                                               command"
> megacommandheresvr"
> server12:
>         job_key:worker_method;
> {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server12",
:command=>"megacommandhere"}, :worker=>:remote_command_worker,
:job_key=>"server12"}
> 
> --------------------%<--------------------
> 
> > 
> > option to further debug this. Something is killing BackgrounDRb master
> > process. Can you sync with trunk and enable foreground logging and
> > check on this?
> 
> Is there an easy way to update a plugin after it''s been installed
> besides removing it and checking it out again?
> 
> > Also, if possible, can we see your worker code?
> 
> Here''s my worker:
> 
> class RemoteCommandWorker < BackgrounDRb::MetaWorker
>   set_worker_name :remote_command_worker
>   set_no_auto_load :true
> 
>   def create(args = nil)
>     register_status("processing")
>   end
> 
>   def do_work(args = nil)
>     result = String.new
>     result = "Result for #{args[:command]}:\n #{%
> x[running_a_shell_script_here]}"
>     register_status(result)
>   end
> end
> 
> Thanks,
> Ger.
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

On Jan 23, 2008 7:40 PM, Ger Apeldoorn <gapeldoorn at wehkamp.nl>
wrote:
> Hi again,
>
> Can I perhaps provide more information to help solve this problem? If
> so, I''d be happy to give it!
You should put that in backgroundrb.yml file. Its described in README file.


>
> Thanks,
> Ger.
>
>
> On Tue, 2008-01-22 at 11:03 +0100, Ger Apeldoorn wrote:
> > Hi!
> >
> > Thanks a lot for your reply!
> >
> > > Did you check backgroundrb_debug.log ?
> > > You can turn on debugging to foreground with:
> > >
> > > :log: foreground
> >
> > Where should I put :log: foreground?
> >
> > Anyway, this comes from log/backgroundrb_debug.log:
> >
> > --------------------%<--------------------
> >
> > $ tail -f log/backgroundrb_debug.log
> > 000000073:      type: start_worker:
> >                                    worker: remote_command_worker:
> >
> > job_key"
> >
> > server06
> > {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server06"}
> > 000000115{
> > :       type:
> >              do_work:
> >                      worker: remote_command_worker:     data{:
> >                                                              
command"
> > megacommandheresvr"
> > server06:
> >         job_key: worker_method;
> > {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server06",
:command=>"megacommandhere"}, :worker=>:remote_command_worker,
:job_key=>"server06"}
> > 000000073:      type: start_worker:
> >                                    worker: remote_command_worker:
> >
> > job_key"
> >
> > server12
> > {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server12"}
> > 000000115{
> > :       type:
> >              do_work:
> >                      worker: remote_command_worker:     data{:
> >                                                              
command"
> > megacommandheresvr"
> > server12:
> >         job_key: worker_method;
> > {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server12",
:command=>"megacommandhere"}, :worker=>:remote_command_worker,
:job_key=>"server12"}
> >
> > --------------------%<--------------------
> >
> > >
> > > option to further debug this. Something is killing BackgrounDRb
master
> > > process. Can you sync with trunk and enable foreground logging
and
> > > check on this?
> >
> > Is there an easy way to update a plugin after it''s been
installed
> > besides removing it and checking it out again?
> >
> > > Also, if possible, can we see your worker code?
> >
> > Here''s my worker:
> >
> > class RemoteCommandWorker < BackgrounDRb::MetaWorker
> >   set_worker_name :remote_command_worker
> >   set_no_auto_load :true
> >
> >   def create(args = nil)
> >     register_status("processing")
> >   end
> >
> >   def do_work(args = nil)
> >     result = String.new
> >     result = "Result for #{args[:command]}:\n #{%
> > x[running_a_shell_script_here]}"
> >     register_status(result)
> >   end
> > end
> >
> > Thanks,
> > Ger.
>
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

Hi,

Here''s output from the debug log: 
Btw; There were a lot of control characters in there which I''ve
removed.

-------------------%<--------------------

{:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server06"}
000000123{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server06:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server06",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server06"}
000000075{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server06
{:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server06"}
000000123{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server06:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server06",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server06"}
000000073{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server08
{:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server08"}
000000121{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server08:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server08",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server08"}
000000074{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server06
{:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server06"}
000000122{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server06:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server06",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server06"}
000000074{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server04
{:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server04"}
000000122{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server04:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server04",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server04"}
000000073{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server04
{:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server04"}
000000121{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server04:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server04",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server04"}
000000074{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server07
{:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server07"}
000000122{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server07:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server07",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server07"}
000000073{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server07
{:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server07"}
000000121{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server07:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server07",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server07"}
000000075{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server05
{:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server05"}
000000123{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server05:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server05",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server05"}
000000075{:
type:^Qstart_worker:worker:remote_command_worker:job_key"server07
{:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server07"}
000000123{
:       type:do_work:worker:remote_command_worker:
data{:command"magiccommand:svr"server07:worker_method;:job_key@
{:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server07",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server07"}

--------------------%<--------------------

Any help very much appreciated!!

Ger.

Hi Ger,

On Jan 24, 2008 4:17 PM, Ger Apeldoorn <gapeldoorn at wehkamp.nl>
wrote:
> Hi,
>
> Here''s output from the debug log:
> Btw; There were a lot of control characters in there which I''ve
removed.
>
> -------------------%<--------------------
>
> {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server06"}
> 000000123{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server06:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server06",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server06"}
> 000000075{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server06
> {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server06"}
> 000000123{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server06:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server06",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server06"}
> 000000073{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server08
> {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server08"}
> 000000121{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server08:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server08",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server08"}
> 000000074{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server06
> {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server06"}
> 000000122{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server06:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server06",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server06"}
> 000000074{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server04
> {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server04"}
> 000000122{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server04:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server04",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server04"}
> 000000073{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server04
> {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server04"}
> 000000121{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server04:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server04",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server04"}
> 000000074{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server07
> {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server07"}
> 000000122{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server07:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server07",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server07"}
> 000000073{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server07
> {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server07"}
> 000000121{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server07:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server07",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server07"}
> 000000075{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server05
> {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server05"}
> 000000123{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server05:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server05",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server05"}
> 000000075{:
> type:^Qstart_worker:worker:remote_command_worker:job_key"server07
> {:type=>:start_worker, :worker=>:remote_command_worker,
:job_key=>"server07"}
> 000000123{
> :       type:do_work:worker:remote_command_worker:
> data{:command"magiccommand:svr"server07:worker_method;:job_key@
> {:type=>:do_work, :worker_method=>:do_work,
:data=>{:svr=>"server07",
:command=>"magiccommand"}, :worker=>:remote_command_worker,
:job_key=>"server07"}
>
> --------------------%<--------------------
>
I am looking for some exceptions, when the master process dies! I
can''t see any so far, in your log output.
Thats the reason, I have been asking you to run bdrb in foreground mode using:

:log: foreground

option and start backgroundrb with:

./script/backgroundrb

This way, when master crashed message will be logged to console.


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

Hi,

My apologies, as I didnt get any output from the ./script/backgroundrb
in the console, I thought the debug log was the place to go.... (even
with the :log: foreground in place)

That said, the backgroundrb master process doesn''t crash. After I get
the ''cannot connect'' error, I can push
''back'' on my browser and try
again with less servers successfully. Even an F5 so that all POST
parameters are submitted again can succeed. (all without restarting
anything)

I hope this can shed some light.. :)

Thanks,
Ger.


> 
> I am looking for some exceptions, when the master process dies! I
> can''t see any so far, in your log output.
> Thats the reason, I have been asking you to run bdrb in foreground mode
using:
> 
> :log: foreground
> 
> option and start backgroundrb with:
> 
> ./script/backgroundrb
> 
> This way, when master crashed message will be logged to console.
> 
>

On Jan 24, 2008 5:28 PM, Ger Apeldoorn <gapeldoorn at wehkamp.nl>
wrote:
> Hi,
>
> My apologies, as I didnt get any output from the ./script/backgroundrb
> in the console, I thought the debug log was the place to go.... (even
> with the :log: foreground in place)
>
> That said, the backgroundrb master process doesn''t crash. After I
get
> the ''cannot connect'' error, I can push
''back'' on my browser and try
> again with less servers successfully. Even an F5 so that all POST
> parameters are submitted again can succeed. (all without restarting
> anything)
>
> I hope this can shed some light.. :)
Okay, I further debugged this issue. The first Question is, are you
deleting the workers? In the first and second mail that you pasted, I
can''t see any calls to either "exit" or
MiddleMan.delete_worker().

What I found that, if you have too many workers running, even if
master doesn''t die, bdrb master may refuse the connection. For me,
threshold was around 50 parallel workers on the same machine. I
couldn''t find any fault in bdrb code, since I have 1 GB memory on this
machine and 50 parallel workers mean,  50*30 = 1500MB of memory.

Assuming other tasks running, available memory is around 600 MB or
something, and swapping brings my machine to a grinding hault.
So, its very important that you delete your worker. In worst case, if
you still can''t solve the issue, just archive the your app and send me
( assuming its not too confidential, and you have reasonable amount of
trust on me).





-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

Hi,
> Okay, I further debugged this issue. The first Question is, are you
> deleting the workers? In the first and second mail that you pasted, I
> can''t see any calls to either "exit" or
MiddleMan.delete_worker().
No, I am deleting the workers in another function when all the work is
done.

I have access to another machine with rails on it and more memory, I''ll
try if it is a problem there. If I cannot figure it out, I''ll send you
the application.

Thanks a lot for your time and effort!
Ger.
> What I found that, if you have too many workers running, even if
> master doesn''t die, bdrb master may refuse the connection. For me,
> threshold was around 50 parallel workers on the same machine. I
> couldn''t find any fault in bdrb code, since I have 1 GB memory on
this
> machine and 50 parallel workers mean,  50*30 = 1500MB of memory.
> 
> Assuming other tasks running, available memory is around 600 MB or
> something, and swapping brings my machine to a grinding hault.
> So, its very important that you delete your worker. In worst case, if
> you still can''t solve the issue, just archive the your app and
send me
> ( assuming its not too confidential, and you have reasonable amount of
> trust on me).
> 
> 
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080124/042efecd/attachment-0001.html