Displaying 2 results from an estimated 2 matches for "scrap_wikipedia".
2007 Dec 17
1
Some more updates, enhancements and fixes
...:result_storage:
:memcache: "10.10.10.2.11211,10.10.10.6:11211"
3. Check for Ruby version.
4. Introduced Thread pool now, all the workers have access to
"thread_pool" object, which can be used to run concurrent tasks in
ruby threads:
An example,
def scrap_wikipedia(text)
thread_pool.defer(text) do |text|
scrap_and_store_text(text)
end
end
Please consult backgroundrb documentation for more information about
this. By default the thread pool is of size 20.
5. Wrote examples on, how to use "connect" and "s...
2008 May 20
1
Couple questions on BDRb and concurrent processing
...n someone point me to some sample code/examples for how to use
thread_pool? The website doc says to add a line
======
pool_size 10
======
in my Worker class which seems straightforward.
I wasn''t able to understand this part though:
=========
thread_pool.defer(wiki_scrap_url) { |wiki_url| scrap_wikipedia(wiki_url) }
=========
Can someone explain this code please? And where does this code go? In my
Rails method that''s actually kicking off the b/g process? Right now my
method looks likke this
======
....
worker = MiddleMan.worker(:netflix_worker)
result = worker.par...