Backgroundrb devel - Jul 2008 - Questions about 1.04/ask_result and getting status in general

Hemant et al -

        I have been using BackgrounDRb 1.03 and am in the process of 
upgrading to 1.04.  I have been using the register_status/ask_status 
technique for getting info from my workers.  BTW my code worked fine in 
the recent edge version until the released 1.04 did away with 
register_status.  So I have a few questions about the new technique of 
passing cached values, e.g., status, between my workers & Rails (1.2.5).

(1)  With 1.03 I was able to determine whether a worker had been created 
by trying ask_status().  If the worker was not there, I got a nil back, 
otherwise I got the actual status as set by register_status() in the 
worker.  When I try this with ask_result(), it throws 
(bdrb_connection.rb/ask_result(), last line, bdrb_response[:data]) trying 
to evaluate nil.[].

        I could catch this exception in calling code, but I was wondering 
if it might not work to change ask_result() to return nil if 
read_from_bdrb() returns nil like send_request() does.

                Line 142:       def ask_result(p_data)
                        ...
                Line 149:         @mutex.synchronize { bdrb_response = 
read_from_bdrb() }
                Line 150:         close_connection
                Line 151:         bdrb_response? bdrb_response[:data]    : 
nil     <-- only line changed

        That would work for my purposes, but I don''t know if it will
break
something else.  My quick perusal of bdrb_connection.rb says it''s ok,
but
someone might be expecting ask_result() to throw and with this change it 
would return a nil instead. 

        If that is not the right way to determine whether a given worker 
is instantiated, what is?

(2)  In the more general area of the cache replacement for 1.03''s 
status...  (I am using a single bdrb server, not a cluster.) 

        Is the cached shared 
                (a) ... between all instances of all workers?  OR 
                (b) ... all instances of a given worker?  OR 
                (c)  ... does a single instance of a worker have its own 
cache?

        If the cache is shared, does that mean workers can communicate 
with each other.

(3)  One of the problems I had with 1.03 was that the register_status way 
of doing things did not seem to work unless I set no_auto_load(true) and 
started the worker myself from Rails.  Without doing that (believe it or 
not) I was getting the status from a different flavor worker, but
that''s
history.  Whether that was me or a bug in bdrb, I don''t know. 

        So the question for 1.04 is whether I will have the same problem 
with the cache of an auto_loaded worker.  If not, great.  I have yet to 
try it.

(4)  I could use some examples of passing cached values back and forth. 
The examples in git are still using register_status().  In particular, I 
need to talk to a particular instance of a worker. 


        Thanks for bdrb and the help, alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080716/b0310285/attachment.html>

On Wed, Jul 16, 2008 at 10:15 PM,  <Alan.Hayes at onevision.com>
wrote:
>
> Hemant et al -
>
>         I have been using BackgrounDRb 1.03 and am in the process of
> upgrading to 1.04.  I have been using the register_status/ask_status
> technique for getting info from my workers.  BTW my code worked fine in the
> recent edge version until the released 1.04 did away with register_status.
>  So I have a few questions about the new technique of passing cached
values,
> e.g., status, between my workers & Rails (1.2.5).
>
> (1)  With 1.03 I was able to determine whether a worker had been created by
> trying ask_status().  If the worker was not there, I got a nil back,
> otherwise I got the actual status as set by register_status() in the
worker.
>  When I try this with ask_result(), it throws
> (bdrb_connection.rb/ask_result(), last line, bdrb_response[:data]) trying
to
> evaluate nil.[].
>
>         I could catch this exception in calling code, but I was wondering
if
> it might not work to change ask_result() to return nil if read_from_bdrb()
> returns nil like send_request() does.
>
>                 Line 142:        def ask_result(p_data)
>                         ...
>                 Line 149:          @mutex.synchronize { bdrb_response >
read_from_bdrb() }
>                 Line 150:          close_connection
>                 Line 151:          bdrb_response? bdrb_response[:data]
>   : nil        <-- only line changed
>
>         That would work for my purposes, but I don''t know if it
will break
> something else.  My quick perusal of bdrb_connection.rb says it''s
ok, but
> someone might be expecting ask_result() to throw and with this change it
> would return a nil instead.
>
>         If that is not the right way to determine whether a given worker is
> instantiated, what is?
That looks perfectly alright. I have added the change and put testcase
too, so should never happen again.

>
> (2)  In the more general area of the cache replacement for 1.03''s
status...
>  (I am using a single bdrb server, not a cluster.)
>
>         Is the cached shared
>                 (a) ... between all instances of all workers?  OR
>                 (b) ... all instances of a given worker?  OR
>                 (c)  ... does a single instance of a worker have its own
> cache?
>
>         If the cache is shared, does that mean workers can communicate with
> each other.

If you are not using memcache based cache, then each instance of
worker has its own cache and when you invoke ask_result, in a cluster
environment, its invoked on all servers and if more than one response
come with same cache key, you get an array(unlikely to happen),
otherwise you get the result from whichever server/worker combination
which had the result. On the other hand, if you use memcache(highly
recommended if planning to store cache of result objects), then each
cache key is mixed with worker key and worker name and result is
fetched straight away from memcache in client itself.
>
> (3)  One of the problems I had with 1.03 was that the register_status way
of
> doing things did not seem to work unless I set no_auto_load(true) and
> started the worker myself from Rails.  Without doing that (believe it or
> not) I was getting the status from a different flavor worker, but
that''s
> history.  Whether that was me or a bug in bdrb, I don''t know.
>
>         So the question for 1.04 is whether I will have the same problem
> with the cache of an auto_loaded worker.  If not, great.  I have yet to try
Should be pretty solid. Give it a try, and let me know about any troubles.
> it.
>
> (4)  I could use some examples of passing cached values back and forth. 
The
> examples in git are still using register_status().  In particular, I need
to
> talk to a particular instance of a worker.
>
Yeah, examples directory was stale(I updated it). Actually passing
cached results back and forth is pretty simple:

In your worker, after a long running task:

class LongWorker < BackgrounDRb::MetaWorker
  set_worker_name :long_worker
  def create(args = nil)
    # this method is called, when worker is loaded for the first time
  end

  def some_long_task arg
    #.. do some long task ..
    result = some_long_longer_loner_task(arg)
    cache[some_key] = result
  end
end


Then in Controller:

MiddleMan.worker(:long_worker).ask_result(some_key)

Thats all.