Backgroundrb devel - Aug 2008 - 1.0.4, memcache, ask_result, and reload_on_schedule

Hi,

 

Am finally getting around to upgrading our backgroundrb from 1.0.1 to
the current SVN release (1.0.4?).  Most things are working quite well.

 

But ...  I was looking forward to using the reload_on_schedule option on
several memory hungry workers that don''t run that often, but am running
into a problem: even using memcached for result storage, I''m unable to
see cached results from workers started with ''reload_on_schedule
true''.

 

I created two nearly identical workers:

 

class HarvesterWorker < BackgrounDRb::MetaWorker

  set_worker_name :harvester_worker

  reload_on_schedule true

 

  def create(args = nil)

    @counter = 0

    logger.info "Harvester created (pid #{Process.pid})"

    cache[''result''] = @counter

  end

 

  def run(args = nil)

    @counter += 1

    logger.info "Harvester run called: #{@counter}"

    cache[''result''] = @counter

  end

end

 

and

 

class Harvester2Worker < BackgrounDRb::MetaWorker

  set_worker_name :harvester2_worker

 

  def create(args = nil)

    @counter = 0

    logger.info "Harvester2 created (pid #{Process.pid})"

    cache[''result''] = @counter

  end

 

  def run(args = nil)

    @counter += 1

    logger.info "Harvester2 run called: #{@counter}"

    cache[''result''] = @counter

  end

end

 

The only real difference between the workers is that HarvesterWorker
uses the reload_on_schedule option, and Harvester2Worker does not.

I scheduled each worker to exec the run method every 15 seconds, and
configured backgroundrb to use memcached for result storage.   The log
file shows the workers are running correctly.  However:

 

MiddleMan.worker(:harvester_worker).ask_result(''result'')
always returns
nil, while

 

MiddleMan.worker(:harvester2_worker).ask_result(''result'')
returns a
correct result.

 

I.e., the worker running with ''reload_on_schedule true''
doesn''t seem to
be caching the result.

 

Any thoughts?  Thanks!

 

Tom Wood

thomas.a.wood at uconn.edu <mailto:thomas.a.wood at uconn.edu> 

ITS Applications Developer

University of Connecticut Libraries

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080829/1b0e721a/attachment-0001.html>

Well this is because, when a worker is restarted on schedule, its
normally started with a unique worker key and what gets used in
memcache as key is a combination of worker name, worker key(if it
exists). So for workers with reload_on_schedule workers, you don''t
really know the key and hence you can''t query results, just using
worker name.

Its quite possible to change this behaviour. For example, you can
modify master_proxy.rb so as it doesn''t use any unique key for workers
started on schedule (see load_and_invoke method).

But this will be obtrusive, hence, I will try to think of a mechanism
so as it co-exists cleanly.


On Fri, Aug 29, 2008 at 10:33 PM, Tom Wood <thomas.a.wood at uconn.edu>
wrote:
> Hi,
>
>
>
> Am finally getting around to upgrading our backgroundrb from 1.0.1 to the
> current SVN release (1.0.4?).  Most things are working quite well.
>
>
>
> But ?  I was looking forward to using the reload_on_schedule option on
> several memory hungry workers that don''t run that often, but am
running into
> a problem: even using memcached for result storage, I''m unable to
see cached
> results from workers started with ''reload_on_schedule
true''.
>
>
>
> I created two nearly identical workers:
>
>
>
> class HarvesterWorker < BackgrounDRb::MetaWorker
>
>   set_worker_name :harvester_worker
>
>   reload_on_schedule true
>
>
>
>   def create(args = nil)
>
>     @counter = 0
>
>     logger.info "Harvester created (pid #{Process.pid})"
>
>     cache[''result''] = @counter
>
>   end
>
>
>
>   def run(args = nil)
>
>     @counter += 1
>
>     logger.info "Harvester run called: #{@counter}"
>
>     cache[''result''] = @counter
>
>   end
>
> end
>
>
>
> and
>
>
>
> class Harvester2Worker < BackgrounDRb::MetaWorker
>
>   set_worker_name :harvester2_worker
>
>
>
>   def create(args = nil)
>
>     @counter = 0
>
>     logger.info "Harvester2 created (pid #{Process.pid})"
>
>     cache[''result''] = @counter
>
>   end
>
>
>
>   def run(args = nil)
>
>     @counter += 1
>
>     logger.info "Harvester2 run called: #{@counter}"
>
>     cache[''result''] = @counter
>
>   end
>
> end
>
>
>
> The only real difference between the workers is that HarvesterWorker uses
> the reload_on_schedule option, and Harvester2Worker does not.
>
> I scheduled each worker to exec the run method every 15 seconds, and
> configured backgroundrb to use memcached for result storage.   The log file
> shows the workers are running correctly.  However:
>
>
>
> MiddleMan.worker(:harvester_worker).ask_result(''result'')
always returns nil,
> while
>
>
>
> MiddleMan.worker(:harvester2_worker).ask_result(''result'')
returns a correct
> result.
>
>
>
> I.e., the worker running with ''reload_on_schedule true''
doesn''t seem to be
> caching the result.
>
>
>
> Any thoughts?  Thanks!
>
>
>
> Tom Wood
>
> thomas.a.wood at uconn.edu
>
> ITS Applications Developer
>
> University of Connecticut Libraries
>
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org