Backgroundrb devel - May 2008 - Couple questions on BDRb and concurrent processing

My Rails site uses BackgroundDRb and I have a couple of questions:

1. Can someone point me to some sample code/examples for how to use
thread_pool? The website doc says to add a line

=====pool_size 10
=====in my Worker class which seems straightforward.

I wasn''t able to understand this part though:
========thread_pool.defer(wiki_scrap_url) { |wiki_url| scrap_wikipedia(wiki_url)
}
========
Can someone explain this code please? And where does this code go? In my
Rails method that''s actually kicking off the b/g process? Right now my
method looks likke this

=====....
            worker = MiddleMan.worker(:netflix_worker)
            result = worker.parse_netflix_feeds(passed_netflix_rss_code)
....
=====where parse_netflix_feeds is the method in my
/lib/workers/netflix_worker.rb

2. My background process is initiated in two ways - one via a live user on
the web and another as a batch process on a scheduled basis. How do I
separate the two so that they don''t wait for each other to complete?
The b/g
process itself calls out to an RSS feed and inserts/updates data into a
MySQL DB. I don''t want a live user''s b/g job to wait for a
batch process
that might be going through dozens of feeds.

Thanks!

Raghu
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080520/8984ab37/attachment-0001.html>

On Wed, May 21, 2008 at 12:20 AM, Raghu Srinivasan
<raghu.srinivasan at gmail.com> wrote:
> My Rails site uses BackgroundDRb and I have a couple of questions:
>
> 1. Can someone point me to some sample code/examples for how to use
> thread_pool? The website doc says to add a line
>
> =====> pool_size 10
> =====> in my Worker class which seems straightforward.
>
> I wasn''t able to understand this part though:
> ========> thread_pool.defer(wiki_scrap_url) { |wiki_url|
scrap_wikipedia(wiki_url) }
> ========>
> Can someone explain this code please? And where does this code go? In my
> Rails method that''s actually kicking off the b/g process? Right
now my
> method looks likke this
Above code goes in your worker, if you don''t want worker to be tied up
while doing some processing.
>
> =====> ....
>             worker = MiddleMan.worker(:netflix_worker)
>             result = worker.parse_netflix_feeds(passed_netflix_rss_code)
> ....
> =====> where parse_netflix_feeds is the method in my
/lib/workers/netflix_worker.rb
>
> 2. My background process is initiated in two ways - one via a live user on
> the web and another as a batch process on a scheduled basis. How do I
> separate the two so that they don''t wait for each other to
complete? The b/g
> process itself calls out to an RSS feed and inserts/updates data into a
> MySQL DB. I don''t want a live user''s b/g job to wait for
a batch process
> that might be going through dozens of feeds.
>
One solution is to use one worker for running scheduled task and
another worker for running direct requests. Another solution is to
use, thread_pool for direct requests.


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org