Backgroundrb devel - Nov 2008 - How to obtain task progress for long-running methods

We''ve used previous versions of backgroundrb to handle long-running
tasks, like ingesting large XML uploads into the database. In the
previous API, we could set accessors on workers and use them to provide
status and progress indicators as the main worker method churned away.

With the recent API changes, I can''t see how this sort of thing would
be
done. It seems that calls to "ask_results" while a method is running
always return nil, rather than whatever the current contents of the
cache are.

Is there some mechanism I''m missing for doing this sort of thing? I can
see the usefulness of the new API and support for clustering, etc, but
if I can''t give the user feedback on an active worker''s
progress, I''m
going to have to go back to the previous incarnation of backgroundrb.

Hello Jim,

We had the same problem when we upgraded Backgroundrb. The solution provided
by hemant was to add these line in your backgroundrb.yml config

:backgroundrb:
  :result_storage: memcache

:memcache: "127.0.0.1:11211"


With that in place, ask_worker should start working.

Regards
Kieran


On Fri, Nov 7, 2008 at 5:13 AM, Jim Nicholson <jnicholson at
skarven.net>wrote:
> We''ve used previous versions of backgroundrb to handle
long-running
> tasks, like ingesting large XML uploads into the database. In the
> previous API, we could set accessors on workers and use them to provide
> status and progress indicators as the main worker method churned away.
>
> With the recent API changes, I can''t see how this sort of thing
would be
> done. It seems that calls to "ask_results" while a method is
running
> always return nil, rather than whatever the current contents of the
> cache are.
>
> Is there some mechanism I''m missing for doing this sort of thing?
I can
> see the usefulness of the new API and support for clustering, etc, but
> if I can''t give the user feedback on an active worker''s
progress, I''m
> going to have to go back to the previous incarnation of backgroundrb.
>
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20081107/549b87c7/attachment-0001.html>

Hmm... I''m using the git version of backgroundrb, and currently up-to-
date with the repository; I don''t have an "ask_worker"
method; I have
"ask_result." Maybe I should be asking: where can I obtain the version
of Backgroundrb that''s covered in "The Rails Way," because it
seems the
API has changed significantly.

Anyway, I''m creating new workers via something like

MiddleMan.new_worker(:worker => :foobar_worker, :worker_key =>
''some key
'', :data => {:arguments => values})

The worker is coded to such that the process logic is called inside the
create method, and it posts progress via cache[worker_key]

  def create(args = nil)
	run_my_process(args)
  end

  def run_my_process(args)
    cache[worker_key] = {}
    ... do stuff
    cache[worker_key][:progress] = 25
    ... do more stuff
    cache[worker_key][:progress] = 50
    ... etc
  end

While this runs, I can see the worker chugging away (because it logs as
it runs,) but if I call

MiddleMan.worker(:foobar_worker).ask_result(''some key'') 

while it is running, I get nil.

Prior to this, I had a prior version of the worker that stayed in
menory. To use that one, I invoked the process via

MiddleMan.worker(:foobar_worker).async_run_my_process(''some
key'')

The only difference in the coding of the worker for that version was
that the cache key was job_key rather than worker_key. Calling
ask_results(''some key'') produced nil until the task was
completed, at
which point it contained my end-state results.





On Fri, 2008-11-07 at 12:57 +1300, Kieran P wrote:
> Hello Jim,
> 
> We had the same problem when we upgraded Backgroundrb. The solution
> provided by hemant was to add these line in your backgroundrb.yml
> config
> 
> :backgroundrb:
>   :result_storage: memcache
> 
> :memcache: "127.0.0.1:11211"
> 
> 
> With that in place, ask_worker should start working.
> 
> Regards
> Kieran
> 
> 
> On Fri, Nov 7, 2008 at 5:13 AM, Jim Nicholson <jnicholson at
skarven.net>
> wrote:
>         We''ve used previous versions of backgroundrb to handle
long-
>         running
>         tasks, like ingesting large XML uploads into the database. In
>         the
>         previous API, we could set accessors on workers and use them
>         to provide
>         status and progress indicators as the main worker method
>         churned away.
>         
>         With the recent API changes, I can''t see how this sort of
>         thing would be
>         done. It seems that calls to "ask_results" while a method
is
>         running
>         always return nil, rather than whatever the current contents
>         of the
>         cache are.
>         
>         Is there some mechanism I''m missing for doing this sort of
>         thing? I can
>         see the usefulness of the new API and support for clustering,
>         etc, but
>         if I can''t give the user feedback on an active
worker''s
>         progress, I''m
>         going to have to go back to the previous incarnation of
>         backgroundrb.
>         
>         
>         
>         _______________________________________________
>         Backgroundrb-devel mailing list
>         Backgroundrb-devel at rubyforge.org
>         http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>

On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at skarven.net>
wrote:
> Hmm... I''m using the git version of backgroundrb, and currently
up-to-
> date with the repository; I don''t have an "ask_worker"
method; I have
> "ask_result." Maybe I should be asking: where can I obtain the
version
> of Backgroundrb that''s covered in "The Rails Way,"
because it seems the
> API has changed significantly.
>
> Anyway, I''m creating new workers via something like
>
> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key =>
''some key
> '', :data => {:arguments => values})
>
> The worker is coded to such that the process logic is called inside the
> create method, and it posts progress via cache[worker_key]
>
>  def create(args = nil)
>        run_my_process(args)
>  end
>
>  def run_my_process(args)
>    cache[worker_key] = {}
>    ... do stuff
>    cache[worker_key][:progress] = 25
>    ... do more stuff
>    cache[worker_key][:progress] = 50
>    ... etc
>  end
>
> While this runs, I can see the worker chugging away (because it logs as
> it runs,) but if I call
>
> MiddleMan.worker(:foobar_worker).ask_result(''some key'')
>
> while it is running, I get nil.
>
> Prior to this, I had a prior version of the worker that stayed in
> menory. To use that one, I invoked the process via
>
> MiddleMan.worker(:foobar_worker).async_run_my_process(''some
key'')
>
> The only difference in the coding of the worker for that version was
> that the cache key was job_key rather than worker_key. Calling
> ask_results(''some key'') produced nil until the task was
completed, at
> which point it contained my end-state results.
We sort of grew very conservative about memory usage in 1.1 release
and decided to handover this task to memcache. Earlier, bdrb master
process used to cache all these results and hence they were readily
available. But unless I maintain an LRU based access, I will have hard
time keeping memory usage of master process under check.

Hence this responsibility was shelved off to worker. But hey, while a
worker is processing a task, it can''t server cache result, and thus
use Memcache, which is out of process. Now even if worker is
processing task, you still get your cache result requests served
alright.

More details can be found here:

http://backgroundrb.rubyforge.org/workers/#result_caching

Lastly, compared to any book, that page is usually more up to date about stuff.

It''d be nice if you had a comprehensive tutorial covering the latest
"up to date" way to handle these things.

I''m pulling my fucking hair out trying to figure out how to upgrade
from 0.2.1 and the doc site isn''t doing much but confusing me more.

Used to be - you''d create one worker for each task you were running.
All work was run from my "do_work" methods, and I''d grab
progress by
calling methods on the worker from Rails. Simple enough....

I''m trying to transition to the latest BDRB right now and I''m
utterly
confused. Perhaps something has changed drastically and I''m missing a
huge concept shift. Right now I''ve moved my "do_work" method
to the
"create" method, which seems to run fine - but I can''t
reference any
of my workers or get any progress back.

Can anyone point to a full tutorial or code example of running an
async process and grabbing multiple results WITHOUT memcached? Or is
memcached required now?

Has the paradigm shifted? Do you only create one worker and call jobs
against that worker and grab results using the cache now?

Someone please point me in the right direction. My emails to the list
and questions on the irc channel have gone unanswered for the last
couple of days.

--------------------
seth - subimage llc
-----
http://sublog.subimage.com
-----
Cashboard - Estimates, invoices, and time tracking software - for free!
http://www.getcashboard.com
-----
Substruct - Open source RoR e-commerce software.
http://code.google.com/p/substruct/



On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at gmail.com>
wrote:
> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at
skarven.net> wrote:
>> Hmm... I''m using the git version of backgroundrb, and
currently up-to-
>> date with the repository; I don''t have an
"ask_worker" method; I have
>> "ask_result." Maybe I should be asking: where can I obtain
the version
>> of Backgroundrb that''s covered in "The Rails Way,"
because it seems the
>> API has changed significantly.
>>
>> Anyway, I''m creating new workers via something like
>>
>> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key =>
''some key
>> '', :data => {:arguments => values})
>>
>> The worker is coded to such that the process logic is called inside the
>> create method, and it posts progress via cache[worker_key]
>>
>>  def create(args = nil)
>>        run_my_process(args)
>>  end
>>
>>  def run_my_process(args)
>>    cache[worker_key] = {}
>>    ... do stuff
>>    cache[worker_key][:progress] = 25
>>    ... do more stuff
>>    cache[worker_key][:progress] = 50
>>    ... etc
>>  end
>>
>> While this runs, I can see the worker chugging away (because it logs as
>> it runs,) but if I call
>>
>> MiddleMan.worker(:foobar_worker).ask_result(''some
key'')
>>
>> while it is running, I get nil.
>>
>> Prior to this, I had a prior version of the worker that stayed in
>> menory. To use that one, I invoked the process via
>>
>> MiddleMan.worker(:foobar_worker).async_run_my_process(''some
key'')
>>
>> The only difference in the coding of the worker for that version was
>> that the cache key was job_key rather than worker_key. Calling
>> ask_results(''some key'') produced nil until the task
was completed, at
>> which point it contained my end-state results.
>
> We sort of grew very conservative about memory usage in 1.1 release
> and decided to handover this task to memcache. Earlier, bdrb master
> process used to cache all these results and hence they were readily
> available. But unless I maintain an LRU based access, I will have hard
> time keeping memory usage of master process under check.
>
> Hence this responsibility was shelved off to worker. But hey, while a
> worker is processing a task, it can''t server cache result, and
thus
> use Memcache, which is out of process. Now even if worker is
> processing task, you still get your cache result requests served
> alright.
>
> More details can be found here:
>
> http://backgroundrb.rubyforge.org/workers/#result_caching
>
> Lastly, compared to any book, that page is usually more up to date about
stuff.
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>

You need memcache, if you want to use result storing and retrieval to
work, when worker is processing a task.

http://backgroundrb.rubyforge.org/workers/#result_caching

AFAIK, thats all, thats needed to get started with result caching.
All documentation of how/what/why is on backgroundrb.rubyforge.org ,
so read whats there on the website carefully.

At last, Do not swear on mailing lists. You can do that on your blog,
irc (some channels allow, others don''t). But NEVER on mailing list, i
hate that.



On Thu, Nov 20, 2008 at 7:27 AM, seth b <subimage at gmail.com>
wrote:
> It''d be nice if you had a comprehensive tutorial covering the
latest
> "up to date" way to handle these things.
>
> I''m pulling my fucking hair out trying to figure out how to
upgrade
> from 0.2.1 and the doc site isn''t doing much but confusing me
more.
>
> Used to be - you''d create one worker for each task you were
running.
> All work was run from my "do_work" methods, and I''d grab
progress by
> calling methods on the worker from Rails. Simple enough....
>
> I''m trying to transition to the latest BDRB right now and
I''m utterly
> confused. Perhaps something has changed drastically and I''m
missing a
> huge concept shift. Right now I''ve moved my "do_work"
method to the
> "create" method, which seems to run fine - but I can''t
reference any
> of my workers or get any progress back.
>
> Can anyone point to a full tutorial or code example of running an
> async process and grabbing multiple results WITHOUT memcached? Or is
> memcached required now?
>
> Has the paradigm shifted? Do you only create one worker and call jobs
> against that worker and grab results using the cache now?
>
> Someone please point me in the right direction. My emails to the list
> and questions on the irc channel have gone unanswered for the last
> couple of days.
>
> --------------------
> seth - subimage llc
> -----
> http://sublog.subimage.com
> -----
> Cashboard - Estimates, invoices, and time tracking software - for free!
> http://www.getcashboard.com
> -----
> Substruct - Open source RoR e-commerce software.
> http://code.google.com/p/substruct/
>
>
>
> On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at gmail.com>
wrote:
>> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at
skarven.net> wrote:
>>> Hmm... I''m using the git version of backgroundrb, and
currently up-to-
>>> date with the repository; I don''t have an
"ask_worker" method; I have
>>> "ask_result." Maybe I should be asking: where can I
obtain the version
>>> of Backgroundrb that''s covered in "The Rails
Way," because it seems the
>>> API has changed significantly.
>>>
>>> Anyway, I''m creating new workers via something like
>>>
>>> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key
=> ''some key
>>> '', :data => {:arguments => values})
>>>
>>> The worker is coded to such that the process logic is called inside
the
>>> create method, and it posts progress via cache[worker_key]
>>>
>>>  def create(args = nil)
>>>        run_my_process(args)
>>>  end
>>>
>>>  def run_my_process(args)
>>>    cache[worker_key] = {}
>>>    ... do stuff
>>>    cache[worker_key][:progress] = 25
>>>    ... do more stuff
>>>    cache[worker_key][:progress] = 50
>>>    ... etc
>>>  end
>>>
>>> While this runs, I can see the worker chugging away (because it
logs as
>>> it runs,) but if I call
>>>
>>> MiddleMan.worker(:foobar_worker).ask_result(''some
key'')
>>>
>>> while it is running, I get nil.
>>>
>>> Prior to this, I had a prior version of the worker that stayed in
>>> menory. To use that one, I invoked the process via
>>>
>>>
MiddleMan.worker(:foobar_worker).async_run_my_process(''some
key'')
>>>
>>> The only difference in the coding of the worker for that version
was
>>> that the cache key was job_key rather than worker_key. Calling
>>> ask_results(''some key'') produced nil until the
task was completed, at
>>> which point it contained my end-state results.
>>
>> We sort of grew very conservative about memory usage in 1.1 release
>> and decided to handover this task to memcache. Earlier, bdrb master
>> process used to cache all these results and hence they were readily
>> available. But unless I maintain an LRU based access, I will have hard
>> time keeping memory usage of master process under check.
>>
>> Hence this responsibility was shelved off to worker. But hey, while a
>> worker is processing a task, it can''t server cache result, and
thus
>> use Memcache, which is out of process. Now even if worker is
>> processing task, you still get your cache result requests served
>> alright.
>>
>> More details can be found here:
>>
>> http://backgroundrb.rubyforge.org/workers/#result_caching
>>
>> Lastly, compared to any book, that page is usually more up to date
about stuff.
>> _______________________________________________
>> Backgroundrb-devel mailing list
>> Backgroundrb-devel at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

Give me a fucking break. What are you the internet language police?

No wonder everyone I know is switching to workling / starling. Not
only does the BDRB documentation suck, the support on the mailing list
sucks even worse.

I''ll go with Ezra''s recommendation and move along. Thanks for
the
non-help, dickhead.

--------------------
seth - subimage llc
-----
http://sublog.subimage.com
-----
Cashboard - Estimates, invoices, and time tracking software - for free!
http://www.getcashboard.com
-----
Substruct - Open source RoR e-commerce software.
http://code.google.com/p/substruct/



On Wed, Nov 19, 2008 at 7:25 PM, hemant <gethemant at gmail.com>
wrote:
> You need memcache, if you want to use result storing and retrieval to
> work, when worker is processing a task.
>
> http://backgroundrb.rubyforge.org/workers/#result_caching
>
> AFAIK, thats all, thats needed to get started with result caching.
> All documentation of how/what/why is on backgroundrb.rubyforge.org ,
> so read whats there on the website carefully.
>
> At last, Do not swear on mailing lists. You can do that on your blog,
> irc (some channels allow, others don''t). But NEVER on mailing
list, i
> hate that.
>
>
>
> On Thu, Nov 20, 2008 at 7:27 AM, seth b <subimage at gmail.com>
wrote:
>> It''d be nice if you had a comprehensive tutorial covering the
latest
>> "up to date" way to handle these things.
>>
>> I''m pulling my fucking hair out trying to figure out how to
upgrade
>> from 0.2.1 and the doc site isn''t doing much but confusing me
more.
>>
>> Used to be - you''d create one worker for each task you were
running.
>> All work was run from my "do_work" methods, and I''d
grab progress by
>> calling methods on the worker from Rails. Simple enough....
>>
>> I''m trying to transition to the latest BDRB right now and
I''m utterly
>> confused. Perhaps something has changed drastically and I''m
missing a
>> huge concept shift. Right now I''ve moved my
"do_work" method to the
>> "create" method, which seems to run fine - but I
can''t reference any
>> of my workers or get any progress back.
>>
>> Can anyone point to a full tutorial or code example of running an
>> async process and grabbing multiple results WITHOUT memcached? Or is
>> memcached required now?
>>
>> Has the paradigm shifted? Do you only create one worker and call jobs
>> against that worker and grab results using the cache now?
>>
>> Someone please point me in the right direction. My emails to the list
>> and questions on the irc channel have gone unanswered for the last
>> couple of days.
>>
>> --------------------
>> seth - subimage llc
>> -----
>> http://sublog.subimage.com
>> -----
>> Cashboard - Estimates, invoices, and time tracking software - for free!
>> http://www.getcashboard.com
>> -----
>> Substruct - Open source RoR e-commerce software.
>> http://code.google.com/p/substruct/
>>
>>
>>
>> On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at gmail.com>
wrote:
>>> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at
skarven.net> wrote:
>>>> Hmm... I''m using the git version of backgroundrb, and
currently up-to-
>>>> date with the repository; I don''t have an
"ask_worker" method; I have
>>>> "ask_result." Maybe I should be asking: where can I
obtain the version
>>>> of Backgroundrb that''s covered in "The Rails
Way," because it seems the
>>>> API has changed significantly.
>>>>
>>>> Anyway, I''m creating new workers via something like
>>>>
>>>> MiddleMan.new_worker(:worker => :foobar_worker, :worker_key
=> ''some key
>>>> '', :data => {:arguments => values})
>>>>
>>>> The worker is coded to such that the process logic is called
inside the
>>>> create method, and it posts progress via cache[worker_key]
>>>>
>>>>  def create(args = nil)
>>>>        run_my_process(args)
>>>>  end
>>>>
>>>>  def run_my_process(args)
>>>>    cache[worker_key] = {}
>>>>    ... do stuff
>>>>    cache[worker_key][:progress] = 25
>>>>    ... do more stuff
>>>>    cache[worker_key][:progress] = 50
>>>>    ... etc
>>>>  end
>>>>
>>>> While this runs, I can see the worker chugging away (because it
logs as
>>>> it runs,) but if I call
>>>>
>>>> MiddleMan.worker(:foobar_worker).ask_result(''some
key'')
>>>>
>>>> while it is running, I get nil.
>>>>
>>>> Prior to this, I had a prior version of the worker that stayed
in
>>>> menory. To use that one, I invoked the process via
>>>>
>>>>
MiddleMan.worker(:foobar_worker).async_run_my_process(''some
key'')
>>>>
>>>> The only difference in the coding of the worker for that
version was
>>>> that the cache key was job_key rather than worker_key. Calling
>>>> ask_results(''some key'') produced nil until
the task was completed, at
>>>> which point it contained my end-state results.
>>>
>>> We sort of grew very conservative about memory usage in 1.1 release
>>> and decided to handover this task to memcache. Earlier, bdrb master
>>> process used to cache all these results and hence they were readily
>>> available. But unless I maintain an LRU based access, I will have
hard
>>> time keeping memory usage of master process under check.
>>>
>>> Hence this responsibility was shelved off to worker. But hey, while
a
>>> worker is processing a task, it can''t server cache result,
and thus
>>> use Memcache, which is out of process. Now even if worker is
>>> processing task, you still get your cache result requests served
>>> alright.
>>>
>>> More details can be found here:
>>>
>>> http://backgroundrb.rubyforge.org/workers/#result_caching
>>>
>>> Lastly, compared to any book, that page is usually more up to date
about stuff.
>>> _______________________________________________
>>> Backgroundrb-devel mailing list
>>> Backgroundrb-devel at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>>
>> _______________________________________________
>> Backgroundrb-devel mailing list
>> Backgroundrb-devel at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>

There is no language policy.  Just curtosy.  We are all  
hackers( reverse engineers ).  It''s not always easy but we''re
persistent.  If you can help with documentation or code please do it''s
how apps evolve.

On Nov 19, 2008, at 7:45 PM, "seth b" <subimage at gmail.com>
wrote:
> Give me a fucking break. What are you the internet language police?
>
> No wonder everyone I know is switching to workling / starling. Not
> only does the BDRB documentation suck, the support on the mailing list
> sucks even worse.
>
> I''ll go with Ezra''s recommendation and move along. Thanks
for the
> non-help, dickhead.
>
> --------------------
> seth - subimage llc
> -----
> http://sublog.subimage.com
> -----
> Cashboard - Estimates, invoices, and time tracking software - for  
> free!
> http://www.getcashboard.com
> -----
> Substruct - Open source RoR e-commerce software.
> http://code.google.com/p/substruct/
>
>
>
> On Wed, Nov 19, 2008 at 7:25 PM, hemant <gethemant at gmail.com>
wrote:
>> You need memcache, if you want to use result storing and retrieval to
>> work, when worker is processing a task.
>>
>> http://backgroundrb.rubyforge.org/workers/#result_caching
>>
>> AFAIK, thats all, thats needed to get started with result caching.
>> All documentation of how/what/why is on backgroundrb.rubyforge.org ,
>> so read whats there on the website carefully.
>>
>> At last, Do not swear on mailing lists. You can do that on your blog,
>> irc (some channels allow, others don''t). But NEVER on mailing
list, i
>> hate that.
>>
>>
>>
>> On Thu, Nov 20, 2008 at 7:27 AM, seth b <subimage at gmail.com>
wrote:
>>> It''d be nice if you had a comprehensive tutorial covering
the latest
>>> "up to date" way to handle these things.
>>>
>>> I''m pulling my fucking hair out trying to figure out how
to upgrade
>>> from 0.2.1 and the doc site isn''t doing much but confusing
me more.
>>>
>>> Used to be - you''d create one worker for each task you
were running.
>>> All work was run from my "do_work" methods, and
I''d grab progress by
>>> calling methods on the worker from Rails. Simple enough....
>>>
>>> I''m trying to transition to the latest BDRB right now and
I''m
>>> utterly
>>> confused. Perhaps something has changed drastically and
I''m
>>> missing a
>>> huge concept shift. Right now I''ve moved my
"do_work" method to the
>>> "create" method, which seems to run fine - but I
can''t reference any
>>> of my workers or get any progress back.
>>>
>>> Can anyone point to a full tutorial or code example of running an
>>> async process and grabbing multiple results WITHOUT memcached? Or
is
>>> memcached required now?
>>>
>>> Has the paradigm shifted? Do you only create one worker and call  
>>> jobs
>>> against that worker and grab results using the cache now?
>>>
>>> Someone please point me in the right direction. My emails to the  
>>> list
>>> and questions on the irc channel have gone unanswered for the last
>>> couple of days.
>>>
>>> --------------------
>>> seth - subimage llc
>>> -----
>>> http://sublog.subimage.com
>>> -----
>>> Cashboard - Estimates, invoices, and time tracking software - for  
>>> free!
>>> http://www.getcashboard.com
>>> -----
>>> Substruct - Open source RoR e-commerce software.
>>> http://code.google.com/p/substruct/
>>>
>>>
>>>
>>> On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at
gmail.com> wrote:
>>>> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson at
skarven.net
>>>> > wrote:
>>>>> Hmm... I''m using the git version of backgroundrb,
and currently
>>>>> up-to-
>>>>> date with the repository; I don''t have an
"ask_worker" method; I
>>>>> have
>>>>> "ask_result." Maybe I should be asking: where can
I obtain the
>>>>> version
>>>>> of Backgroundrb that''s covered in "The Rails
Way," because it
>>>>> seems the
>>>>> API has changed significantly.
>>>>>
>>>>> Anyway, I''m creating new workers via something
like
>>>>>
>>>>> MiddleMan.new_worker(:worker => :foobar_worker,
:worker_key =>
>>>>> ''some key
>>>>> '', :data => {:arguments => values})
>>>>>
>>>>> The worker is coded to such that the process logic is
called
>>>>> inside the
>>>>> create method, and it posts progress via cache[worker_key]
>>>>>
>>>>> def create(args = nil)
>>>>>       run_my_process(args)
>>>>> end
>>>>>
>>>>> def run_my_process(args)
>>>>>   cache[worker_key] = {}
>>>>>   ... do stuff
>>>>>   cache[worker_key][:progress] = 25
>>>>>   ... do more stuff
>>>>>   cache[worker_key][:progress] = 50
>>>>>   ... etc
>>>>> end
>>>>>
>>>>> While this runs, I can see the worker chugging away
(because it
>>>>> logs as
>>>>> it runs,) but if I call
>>>>>
>>>>> MiddleMan.worker(:foobar_worker).ask_result(''some
key'')
>>>>>
>>>>> while it is running, I get nil.
>>>>>
>>>>> Prior to this, I had a prior version of the worker that
stayed in
>>>>> menory. To use that one, I invoked the process via
>>>>>
>>>>>
MiddleMan.worker(:foobar_worker).async_run_my_process(''some
key'')
>>>>>
>>>>> The only difference in the coding of the worker for that
version
>>>>> was
>>>>> that the cache key was job_key rather than worker_key.
Calling
>>>>> ask_results(''some key'') produced nil
until the task was
>>>>> completed, at
>>>>> which point it contained my end-state results.
>>>>
>>>> We sort of grew very conservative about memory usage in 1.1
release
>>>> and decided to handover this task to memcache. Earlier, bdrb
master
>>>> process used to cache all these results and hence they were
readily
>>>> available. But unless I maintain an LRU based access, I will
have
>>>> hard
>>>> time keeping memory usage of master process under check.
>>>>
>>>> Hence this responsibility was shelved off to worker. But hey,  
>>>> while a
>>>> worker is processing a task, it can''t server cache
result, and thus
>>>> use Memcache, which is out of process. Now even if worker is
>>>> processing task, you still get your cache result requests
served
>>>> alright.
>>>>
>>>> More details can be found here:
>>>>
>>>> http://backgroundrb.rubyforge.org/workers/#result_caching
>>>>
>>>> Lastly, compared to any book, that page is usually more up to  
>>>> date about stuff.
>>>> _______________________________________________
>>>> Backgroundrb-devel mailing list
>>>> Backgroundrb-devel at rubyforge.org
>>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>>>
>>> _______________________________________________
>>> Backgroundrb-devel mailing list
>>> Backgroundrb-devel at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>>
>>
>>
>>
>> --
>> Let them talk of their oriental summer climes of everlasting
>> conservatories; give me the privilege of making my own summer with my
>> own coals.
>>
>> http://gnufied.org
>>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

While I do agree that the documentation for BDRB can be quite confusing,
especially with major changes happening so frequently, it is quite
unfair to say that there is no support on the mailing list. Hemant and
others often write long and detailed replies to posts here and on many
other public forums but I guess it depends how you ask. 
Your use of abusive language is completely unacceptable and it throws a
rather bad light on your company (BTW: quite a lot of advertisement in
your posts)! If I was the admin on this mailing list, I''d ban you from
the list.

Frank


On Wed, 2008-11-19 at 19:45 -0800, seth b wrote:
> Give me a fucking break. What are you the internet language police?
> 
> No wonder everyone I know is switching to workling / starling. Not
> only does the BDRB documentation suck, the support on the mailing list
> sucks even worse.
> 
> I''ll go with Ezra''s recommendation and move along. Thanks
for the
> non-help, dickhead.
> 
> --------------------
> seth - subimage llc
> -----
> http://sublog.subimage.com
> -----
> Cashboard - Estimates, invoices, and time tracking software - for free!
> http://www.getcashboard.com
> -----
> Substruct - Open source RoR e-commerce software.
> http://code.google.com/p/substruct/
> 
> 
> 
> On Wed, Nov 19, 2008 at 7:25 PM, hemant <gethemant at gmail.com>
wrote:
> > You need memcache, if you want to use result storing and retrieval to
> > work, when worker is processing a task.
> >
> > http://backgroundrb.rubyforge.org/workers/#result_caching
> >
> > AFAIK, thats all, thats needed to get started with result caching.
> > All documentation of how/what/why is on backgroundrb.rubyforge.org ,
> > so read whats there on the website carefully.
> >
> > At last, Do not swear on mailing lists. You can do that on your blog,
> > irc (some channels allow, others don''t). But NEVER on mailing
list, i
> > hate that.
> >
> >
> >
> > On Thu, Nov 20, 2008 at 7:27 AM, seth b <subimage at gmail.com>
wrote:
> >> It''d be nice if you had a comprehensive tutorial covering
the latest
> >> "up to date" way to handle these things.
> >>
> >> I''m pulling my fucking hair out trying to figure out how
to upgrade
> >> from 0.2.1 and the doc site isn''t doing much but
confusing me more.
> >>
> >> Used to be - you''d create one worker for each task you
were running.
> >> All work was run from my "do_work" methods, and
I''d grab progress by
> >> calling methods on the worker from Rails. Simple enough....
> >>
> >> I''m trying to transition to the latest BDRB right now and
I''m utterly
> >> confused. Perhaps something has changed drastically and
I''m missing a
> >> huge concept shift. Right now I''ve moved my
"do_work" method to the
> >> "create" method, which seems to run fine - but I
can''t reference any
> >> of my workers or get any progress back.
> >>
> >> Can anyone point to a full tutorial or code example of running an
> >> async process and grabbing multiple results WITHOUT memcached? Or
is
> >> memcached required now?
> >>
> >> Has the paradigm shifted? Do you only create one worker and call
jobs
> >> against that worker and grab results using the cache now?
> >>
> >> Someone please point me in the right direction. My emails to the
list
> >> and questions on the irc channel have gone unanswered for the last
> >> couple of days.
> >>
> >> --------------------
> >> seth - subimage llc
> >> -----
> >> http://sublog.subimage.com
> >> -----
> >> Cashboard - Estimates, invoices, and time tracking software - for
free!
> >> http://www.getcashboard.com
> >> -----
> >> Substruct - Open source RoR e-commerce software.
> >> http://code.google.com/p/substruct/
> >>
> >>
> >>
> >> On Fri, Nov 7, 2008 at 10:04 AM, hemant <gethemant at
gmail.com> wrote:
> >>> On Fri, Nov 7, 2008 at 9:34 PM, Jim Nicholson <jnicholson
at skarven.net> wrote:
> >>>> Hmm... I''m using the git version of backgroundrb,
and currently up-to-
> >>>> date with the repository; I don''t have an
"ask_worker" method; I have
> >>>> "ask_result." Maybe I should be asking: where
can I obtain the version
> >>>> of Backgroundrb that''s covered in "The Rails
Way," because it seems the
> >>>> API has changed significantly.
> >>>>
> >>>> Anyway, I''m creating new workers via something
like
> >>>>
> >>>> MiddleMan.new_worker(:worker => :foobar_worker,
:worker_key => ''some key
> >>>> '', :data => {:arguments => values})
> >>>>
> >>>> The worker is coded to such that the process logic is
called inside the
> >>>> create method, and it posts progress via cache[worker_key]
> >>>>
> >>>>  def create(args = nil)
> >>>>        run_my_process(args)
> >>>>  end
> >>>>
> >>>>  def run_my_process(args)
> >>>>    cache[worker_key] = {}
> >>>>    ... do stuff
> >>>>    cache[worker_key][:progress] = 25
> >>>>    ... do more stuff
> >>>>    cache[worker_key][:progress] = 50
> >>>>    ... etc
> >>>>  end
> >>>>
> >>>> While this runs, I can see the worker chugging away
(because it logs as
> >>>> it runs,) but if I call
> >>>>
> >>>> MiddleMan.worker(:foobar_worker).ask_result(''some
key'')
> >>>>
> >>>> while it is running, I get nil.
> >>>>
> >>>> Prior to this, I had a prior version of the worker that
stayed in
> >>>> menory. To use that one, I invoked the process via
> >>>>
> >>>>
MiddleMan.worker(:foobar_worker).async_run_my_process(''some
key'')
> >>>>
> >>>> The only difference in the coding of the worker for that
version was
> >>>> that the cache key was job_key rather than worker_key.
Calling
> >>>> ask_results(''some key'') produced nil
until the task was completed, at
> >>>> which point it contained my end-state results.
> >>>
> >>> We sort of grew very conservative about memory usage in 1.1
release
> >>> and decided to handover this task to memcache. Earlier, bdrb
master
> >>> process used to cache all these results and hence they were
readily
> >>> available. But unless I maintain an LRU based access, I will
have hard
> >>> time keeping memory usage of master process under check.
> >>>
> >>> Hence this responsibility was shelved off to worker. But hey,
while a
> >>> worker is processing a task, it can''t server cache
result, and thus
> >>> use Memcache, which is out of process. Now even if worker is
> >>> processing task, you still get your cache result requests
served
> >>> alright.
> >>>
> >>> More details can be found here:
> >>>
> >>> http://backgroundrb.rubyforge.org/workers/#result_caching
> >>>
> >>> Lastly, compared to any book, that page is usually more up to
date about stuff.
> >>> _______________________________________________
> >>> Backgroundrb-devel mailing list
> >>> Backgroundrb-devel at rubyforge.org
> >>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >>>
> >> _______________________________________________
> >> Backgroundrb-devel mailing list
> >> Backgroundrb-devel at rubyforge.org
> >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >>
> >
> >
> >
> > --
> > Let them talk of their oriental summer climes of everlasting
> > conservatories; give me the privilege of making my own summer with my
> > own coals.
> >
> > http://gnufied.org
> >
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel