Backgroundrb devel - Dec 2007 - 1) Error 2) sending request to a specific worker

Hello !

1) I have this error logged by bdrb:
undefined method `send_request'' for nil:NilClass
framework/packet_master.rb:58:in `ask_worker''
backgroundrb/server/master_worker.rb:59:in `process_work''
backgroundrb/server/master_worker.rb:16:in `receive_data''
backgroundrb/framework/bin_parser.rb:29:in `call''
backgroundrb/framework/bin_parser.rb:29:in `extract''
backgroundrb/server/master_worker.rb:12:in `receive_data''
backgroundrb/framework/core.rb:191:in `read_external_socket''
backgroundrb/framework/core.rb:183:in `handle_external_messages''
backgroundrb/framework/core.rb:156:in `start_reactor''
backgroundrb/framework/core.rb:152:in `each''
backgroundrb/framework/core.rb:152:in `start_reactor''
backgroundrb/framework/core.rb:144:in `loop''
backgroundrb/framework/core.rb:144:in `start_reactor''
backgroundrb/framework/packet_master.rb:20:in `run''
backgroundrb/server/master_worker.rb:115:in `initialize''
./script/backgroundrb:39:in `new''
./script/backgroundrb:39

my code is :

class MyController < ApplicationController

  def index
  end

  def login
    @worker = MiddleMan.new_worker(:worker => :xmpp_worker, :job_key =>
:my_key)
    MiddleMan.ask_work(:worker => :xmpp_worker, :worker_method =>
:login, :data => "test")
    render(:partial=>''window'')
  end

end

class XmppWorker < BackgrounDRb::MetaWorker
  set_worker_name :xmpp_worker
  set_no_auto_load true

  def create(args = nil)
  end

  def login
  end

end

Can someone helps me ?

2) I am creating a worker with a key : MiddleMan.new_worker(:worker =>
:xmpp_worker, :job_key => :my_key)
Is it possible to use this key in a request to send work to a specific worker ?

Thanks for your help,
Mickael.

On Dec 16, 2007 3:38 AM, Mickael Faivre-Macon <faivrem at gmail.com>
wrote:
> Hello !
>
> 1) I have this error logged by bdrb:
> undefined method `send_request'' for nil:NilClass
> framework/packet_master.rb:58:in `ask_worker''
> backgroundrb/server/master_worker.rb:59:in `process_work''
> backgroundrb/server/master_worker.rb:16:in `receive_data''
> backgroundrb/framework/bin_parser.rb:29:in `call''
> backgroundrb/framework/bin_parser.rb:29:in `extract''
> backgroundrb/server/master_worker.rb:12:in `receive_data''
> backgroundrb/framework/core.rb:191:in `read_external_socket''
> backgroundrb/framework/core.rb:183:in `handle_external_messages''
> backgroundrb/framework/core.rb:156:in `start_reactor''
> backgroundrb/framework/core.rb:152:in `each''
> backgroundrb/framework/core.rb:152:in `start_reactor''
> backgroundrb/framework/core.rb:144:in `loop''
> backgroundrb/framework/core.rb:144:in `start_reactor''
> backgroundrb/framework/packet_master.rb:20:in `run''
> backgroundrb/server/master_worker.rb:115:in `initialize''
> ./script/backgroundrb:39:in `new''
> ./script/backgroundrb:39
>
> my code is :
>
> class MyController < ApplicationController
>
>   def index
>   end
>
>   def login
>     @worker = MiddleMan.new_worker(:worker => :xmpp_worker, :job_key
=> :my_key)
>     MiddleMan.ask_work(:worker => :xmpp_worker, :worker_method =>
> :login, :data => "test")
>     render(:partial=>''window'')
>   end
>
> end
>
> class XmppWorker < BackgrounDRb::MetaWorker
>   set_worker_name :xmpp_worker
>   set_no_auto_load true
>
>   def create(args = nil)
>   end
>
>   def login
>   end
>
> end
>
> Can someone helps me ?
>
> 2) I am creating a worker with a key : MiddleMan.new_worker(:worker =>
> :xmpp_worker, :job_key => :my_key)
> Is it possible to use this key in a request to send work to a specific
worker ?
>
I suggest you remove set_no_auto_load(true). Why exactly, you don''t
want worker to be automatically loaded when bdrb starts?

The problem as i see may be because, since new_worker forks new worker
in separate process, but the worker is not available immediately and
hence you get the error.

If you insist on using dynamically starting workers, I will be see
what can be done to prevent error. There are couple of options.


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

Hi hemant,

Thanks for your reply.

The reason why I don''t see the advantage to let the worker start wiht
bdrb may be related to my understanding of how it works, and
specifically to my question about keys and specific worker instances.

I am using bdrb to keeps connections to a XMPP (Jabber) server, while
Rails renders the UI (web based chat application). So the way I see
it, is that there is only one :xmpp_worker class but several instances
of that class running, each with a specific login and connection to
the Jabber server.

That''s why I am launching a new :xmpp_worker (for each chat user).
And that''s why I need to know how to send_request (or ask_work) to a
specific worker instance. I thought it could be done with the concept
of keys.

Is that the right way of doing things ?
How would you do it ?

Thanks,
Mickael.


On Dec 15, 2007 11:43 PM, hemant <gethemant at gmail.com>
wrote:
>
> On Dec 16, 2007 3:38 AM, Mickael Faivre-Macon <faivrem at gmail.com>
wrote:
> > Hello !
> >
> > 1) I have this error logged by bdrb:
> > undefined method `send_request'' for nil:NilClass
> > framework/packet_master.rb:58:in `ask_worker''
> > backgroundrb/server/master_worker.rb:59:in `process_work''
> > backgroundrb/server/master_worker.rb:16:in `receive_data''
> > backgroundrb/framework/bin_parser.rb:29:in `call''
> > backgroundrb/framework/bin_parser.rb:29:in `extract''
> > backgroundrb/server/master_worker.rb:12:in `receive_data''
> > backgroundrb/framework/core.rb:191:in `read_external_socket''
> > backgroundrb/framework/core.rb:183:in
`handle_external_messages''
> > backgroundrb/framework/core.rb:156:in `start_reactor''
> > backgroundrb/framework/core.rb:152:in `each''
> > backgroundrb/framework/core.rb:152:in `start_reactor''
> > backgroundrb/framework/core.rb:144:in `loop''
> > backgroundrb/framework/core.rb:144:in `start_reactor''
> > backgroundrb/framework/packet_master.rb:20:in `run''
> > backgroundrb/server/master_worker.rb:115:in `initialize''
> > ./script/backgroundrb:39:in `new''
> > ./script/backgroundrb:39
> >
> > my code is :
> >
> > class MyController < ApplicationController
> >
> >   def index
> >   end
> >
> >   def login
> >     @worker = MiddleMan.new_worker(:worker => :xmpp_worker,
:job_key => :my_key)
> >     MiddleMan.ask_work(:worker => :xmpp_worker, :worker_method
=>
> > :login, :data => "test")
> >     render(:partial=>''window'')
> >   end
> >
> > end
> >
> > class XmppWorker < BackgrounDRb::MetaWorker
> >   set_worker_name :xmpp_worker
> >   set_no_auto_load true
> >
> >   def create(args = nil)
> >   end
> >
> >   def login
> >   end
> >
> > end
> >
> > Can someone helps me ?
> >
> > 2) I am creating a worker with a key : MiddleMan.new_worker(:worker
=>
> > :xmpp_worker, :job_key => :my_key)
> > Is it possible to use this key in a request to send work to a specific
worker ?
> >
>
> I suggest you remove set_no_auto_load(true). Why exactly, you
don''t
> want worker to be automatically loaded when bdrb starts?
>
> The problem as i see may be because, since new_worker forks new worker
> in separate process, but the worker is not available immediately and
> hence you get the error.
>
> If you insist on using dynamically starting workers, I will be see
> what can be done to prevent error. There are couple of options.

On Dec 16, 2007 4:42 AM, Mickael Faivre-Macon <faivrem at gmail.com>
wrote:
> Hi hemant,
>
> Thanks for your reply.
>
> The reason why I don''t see the advantage to let the worker start
wiht
> bdrb may be related to my understanding of how it works, and
> specifically to my question about keys and specific worker instances.
>
> I am using bdrb to keeps connections to a XMPP (Jabber) server, while
> Rails renders the UI (web based chat application). So the way I see
> it, is that there is only one :xmpp_worker class but several instances
> of that class running, each with a specific login and connection to
> the Jabber server.
>
> That''s why I am launching a new :xmpp_worker (for each chat user).
> And that''s why I need to know how to send_request (or ask_work) to
a
> specific worker instance. I thought it could be done with the concept
> of keys.
yes you can use job keys for that. However I am curious about the fact
that, if you have 100 concurrent users, you are going to have 100
parallel workers running for each user? Is that what you want?
>
> Is that the right way of doing things ?
> How would you do it ?
I am cooking a smallish worker with detailed explanation on how to
handle multiple XMPP clients from one instance of backgroundrb.

-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

On Dec 16, 2007 6:30 AM, hemant <gethemant at gmail.com>
wrote:
> On Dec 16, 2007 4:42 AM, Mickael Faivre-Macon <faivrem at gmail.com>
wrote:
> > Hi hemant,
> >
> > Thanks for your reply.
> >
> > The reason why I don''t see the advantage to let the worker
start wiht
> > bdrb may be related to my understanding of how it works, and
> > specifically to my question about keys and specific worker instances.
> >
> > I am using bdrb to keeps connections to a XMPP (Jabber) server, while
> > Rails renders the UI (web based chat application). So the way I see
> > it, is that there is only one :xmpp_worker class but several instances
> > of that class running, each with a specific login and connection to
> > the Jabber server.
> >
> > That''s why I am launching a new :xmpp_worker (for each chat
user).
> > And that''s why I need to know how to send_request (or
ask_work) to a
> > specific worker instance. I thought it could be done with the concept
> > of keys.
>
> yes you can use job keys for that. However I am curious about the fact
> that, if you have 100 concurrent users, you are going to have 100
> parallel workers running for each user? Is that what you want?
>
Well, isn''t it the same than to have only one worker with 100 threads
? (Well at least on the end, you would have 100 threads, not 100 ruby
processes, that''s right)
I don''t know what''s better.
> >
> > Is that the right way of doing things ?
> > How would you do it ?
>
> I am cooking a smallish worker with detailed explanation on how to
> handle multiple XMPP clients from one instance of backgroundrb.
You''re spoiling the fun ;-)
But it will greatly helps me on understanding how bdrb works and how
to use it, so I will not complain :-)

I am looking forward to see your solution.

Mickael.

Hi Hemant, all:

Good work to-date on the new bdrb.

On Dec 15, 2007, at 9:30 PM, hemant wrote:
> On Dec 16, 2007 4:42 AM, Mickael Faivre-Macon <faivrem at gmail.com>
> wrote:
>>
>>
>> Is that the right way of doing things ?
>> How would you do it ?
>
> I am cooking a smallish worker with detailed explanation on how to
> handle multiple XMPP clients from one instance of backgroundrb.
>
I''ve been developing a chat middle-ware solution for a client  
(bunnybot.com). This is a the approach I''ve taken. We make heavy use  
of XMPP/Jabber, run OpenFire on the back-end, and need to be able to  
maintain user presence information. I''ve created a light-weight client
worker that manages pools of user proxy objects inspired by xmpp4r- 
simple. Using this approach, a single worker can scale to maintain a  
pretty good size collection of connections (I''ve tested with
100''s,
don''t see why it couldn''t be more pretty much limited to OS  
resources). The worker and the app communicate in the standard way,  
with simple keys that identify the instance of the proxy object, and  
serialize key data (e.g. rosters, presence updates, messages, etc).  
I''ve got a few more things to do on the implementation, being paused  
while finishing other deployments, but am happy to discuss.

I had the first version of this running under the older bdrb code-base  
using a self-managed set of threads. I''m moving it to the HEAD and  
looking at the new tcp connections in the event loop Hemant mentions.

If others are doing similar things and can share knowledge, gotchas,  
or even code, that would be cool. A few of us have exchanged some  
messages over at RailsForum and RubyOnRails-Talk, so check those  
places for other leads.

-chris

--
Christopher Haupt
chaupt at mobirobo.com

Hi Chris,

> I''ve been developing a chat middle-ware solution for a client
> (bunnybot.com). This is a the approach I''ve taken. We make heavy
use
> of XMPP/Jabber, run OpenFire on the back-end, and need to be able to
> maintain user presence information. I''ve created a light-weight
client
> worker that manages pools of user proxy objects inspired by xmpp4r-
> simple. Using this approach, a single worker can scale to maintain a
> pretty good size collection of connections (I''ve tested with
100''s,
> don''t see why it couldn''t be more pretty much limited to
OS
> resources). The worker and the app communicate in the standard way,
> with simple keys that identify the instance of the proxy object, and
> serialize key data (e.g. rosters, presence updates, messages, etc).
> I''ve got a few more things to do on the implementation, being
paused
> while finishing other deployments, but am happy to discuss.
>
> I had the first version of this running under the older bdrb code-base
> using a self-managed set of threads. I''m moving it to the HEAD and
> looking at the new tcp connections in the event loop Hemant mentions.
>
> If others are doing similar things and can share knowledge, gotchas,
> or even code, that would be cool. A few of us have exchanged some
> messages over at RailsForum and RubyOnRails-Talk, so check those
> places for other leads.
>
My idea of managing everything from single worker is pretty simple.

Keep all sever connections in hash, something like in create method of worker:

def create
  @open_connections = {}
  @open_connection_mutex = Mutex.new
end

def login(details)
  thread_pool.defer(details) do |details|
     login_resp = login_using_details
     @open_connection_mutex.sychronize do
        @open_connections[user_id] = connection_info
     end
  end
end

def send_message(message)
  thread_pool.defer(message) do |message|
    user_info = message.user_id
    connection = nil
    @open_connection_mutex.sychronize do
       @connection = @open_connections[user_info]
   end
   connection.send_message(message.text)
end

Where threa_pool is a ThreadPool that is available right inside
backgroundrb workers.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

On Dec 16, 2007 3:38 AM, Mickael Faivre-Macon <faivrem at gmail.com>
wrote:
> Hello !
>
> 1) I have this error logged by bdrb:
> undefined method `send_request'' for nil:NilClass
> framework/packet_master.rb:58:in `ask_worker''
> backgroundrb/server/master_worker.rb:59:in `process_work''
> backgroundrb/server/master_worker.rb:16:in `receive_data''
> backgroundrb/framework/bin_parser.rb:29:in `call''
> backgroundrb/framework/bin_parser.rb:29:in `extract''
> backgroundrb/server/master_worker.rb:12:in `receive_data''
> backgroundrb/framework/core.rb:191:in `read_external_socket''
> backgroundrb/framework/core.rb:183:in `handle_external_messages''
> backgroundrb/framework/core.rb:156:in `start_reactor''
> backgroundrb/framework/core.rb:152:in `each''
> backgroundrb/framework/core.rb:152:in `start_reactor''
> backgroundrb/framework/core.rb:144:in `loop''
> backgroundrb/framework/core.rb:144:in `start_reactor''
> backgroundrb/framework/packet_master.rb:20:in `run''
> backgroundrb/server/master_worker.rb:115:in `initialize''
> ./script/backgroundrb:39:in `new''
> ./script/backgroundrb:39
>
> my code is :
>
> class MyController < ApplicationController
>
>   def index
>   end
>
>   def login
>     @worker = MiddleMan.new_worker(:worker => :xmpp_worker, :job_key
=> :my_key)
>     MiddleMan.ask_work(:worker => :xmpp_worker, :worker_method =>
> :login, :data => "test")
>     render(:partial=>''window'')
>   end
>
> end
>
> class XmppWorker < BackgrounDRb::MetaWorker
>   set_worker_name :xmpp_worker
>   set_no_auto_load true
>
>   def create(args = nil)
>   end
>
>   def login
>   end
>
> end
>
> Can someone helps me ?
>
> 2) I am creating a worker with a key : MiddleMan.new_worker(:worker =>
> :xmpp_worker, :job_key => :my_key)
> Is it possible to use this key in a request to send work to a specific
worker ?
>
Hi Michale,

There was a genuine bug with backgroundrb in job_key handling. If you
don''t want to use threads and would rather want to have processes, you
can try your code now.

It should be fixed and working.
-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

Thanks Hemant, it works.

How can I change the default port 11009 ?
It is taken on my shared server.

I have changed it in the bdrb config file, but how I tell my workers
to connect to this new port ?

Mickael.



On Dec 17, 2007 10:15 AM, hemant <gethemant at gmail.com>
wrote:
>
> On Dec 16, 2007 3:38 AM, Mickael Faivre-Macon <faivrem at gmail.com>
wrote:
> > Hello !
> >
> > 1) I have this error logged by bdrb:
> Hi Michale,
>
> There was a genuine bug with backgroundrb in job_key handling. If you
> don''t want to use threads and would rather want to have processes,
you
> can try your code now.
>
> It should be fixed and working.

Hi
On Sat, 2007-12-22 at 19:26 +0100, Mickael Faivre-Macon
wrote:
> Thanks Hemant, it works.
> 
> How can I change the default port 11009 ?
> It is taken on my shared server.
> 
> I have changed it in the bdrb config file, but how I tell my workers
> to connect to this new port ?
Restarting the backgroundrb should fix it and all the workers should be
automatically aware of new port, there is nothing much to do.

Let me know, if it doesn''t work. Run the server in foreground mode as
described in docs for getting debugging information.

Oh yes, I had to restart my Rails app also, of course.
Thanks !

On Dec 22, 2007 8:48 PM, hemant kumar <gethemant at gmail.com>
wrote:
> Hi
> On Sat, 2007-12-22 at 19:26 +0100, Mickael Faivre-Macon wrote:
> > Thanks Hemant, it works.
> >
> > How can I change the default port 11009 ?
> > It is taken on my shared server.
> >
> > I have changed it in the bdrb config file, but how I tell my workers
> > to connect to this new port ?
>
> Restarting the backgroundrb should fix it and all the workers should be
> automatically aware of new port, there is nothing much to do.
>
> Let me know, if it doesn''t work. Run the server in foreground mode
as
> described in docs for getting debugging information.
>
>
>


-- 
Mickael.
Coding an AI ! http://faivrem.googlepages.com/antbattle