Displaying 1 result from an estimated 1 matches for "scrapedpage".
2008 Jun 10
4
adding results from threads to a collection and returning it
...urned by the backgroundrb worker. Here is an example of what
I''m trying to do in a worker method:
pages = Array.new
pages_to_scrape.each do |url|
thread_pool.defer(url) do |url|
begin
# model object performs the scraping
page = ScrapedPage.new(page.url)
pages << page
rescue
logger.info "page scrape failed"
end
end
end
end
return pages
>From monitoring the backgroundrb logs, it appears that all of the pages are
completed successfully i...