similar to: Sequence of multiple background process submissions

I use BDRb to process RSS feeds for users on my site (http://feedflix.com). I have a batch job that queries the DB for records that haven''t been updated in the last so many hours and kicks off a background job for each of them. If N records are returned by the DB, N background jobs gets queued and get done serially. As long as N is 255 or under, everything works like a charm.

My Rails site uses BackgroundDRb and I have a couple of questions: 1. Can someone point me to some sample code/examples for how to use thread_pool? The website doc says to add a line ====== pool_size 10 ====== in my Worker class which seems straightforward. I wasn''t able to understand this part though: ========= thread_pool.defer(wiki_scrap_url) { |wiki_url| scrap_wikipedia(wiki_url) }

After some initial stumblings, I think I''ve got the hang of backgroundrb. It''s great! I''d been thinking for many many months how cool something like this would be! I''m trying to make a "job queue". That is, a pool of worker threads monitor a queue. When a job appears, one of the workers grabs it and executes the task. When complete, the worker

Hey All, I am looking for an example of the proper config settings for background rb to run my rails app and my backgroundrb instance on separate machines (or more precisely, to have a rails cluster with one backgroundrb machine). Does anyone have such examples? Thanks! -Noah -------------- next part -------------- An HTML attachment was scrubbed... URL:

I''m trying to run a single worker that could perform a periodic task for a given user. >From a controller, I imagine something like: def start_job MiddleMan.ask_work(:worker => :foo_worker, :worker_method => :perform_task, :data => { :user_id = current_user.id }) end def check_job @status = MiddleMan.ask_status(:worker => :foo_worker)[current_user.id] end

Hi everyone, I''ve a class that try to spawn two workers, every 5 seconds. The code is something like: while true do MiddleMan.new_worker(:class => :researches_worker,:job_key => :researches) MiddleMan.new_worker(:class => :products_worker,:job_key => :products) sleep(5) end I''m using ":job_key", so if a worker hasn''t already ended its work

Hi, i''m going to implement a syndication-service, which will get lists in xml with some meta-data an enclosed video files, which will get encoded at the end. The syndication run will be startet every five minutes of a full hour. So i thought to build 4 Worker. One for checking which feeds to syndicate (syndication_worker) at a specific time, one for processing the list

Hello all, I get this intermittent error saying OpenURI::HTTPError. At the bottom of the email is what is reported in my backgroundrb_server_<port>.log file. The process_netflix_reviews method is parsing (in the background) Netflix''s RSS feeds. This error also pretty much brings down the BDRb process and to recover, I am forced to do a ./script/backgroundrb stop

I recently switched over to v1.0, and things are rolling along pretty well. However, one thing that has always been a little confusing to me is knowing when to use thread_pool. Since most of my bgrb workers are called from my web app to process rather than being scheduled, I''m using the thread_pool for every call. Unfortunately, that means that I have to split up workers by how many

We are almost there to a final release. The uninstall issues have been fixed, and a few packages have been upgraded to more recent versions. Many thanks to Ryan Leavengood and Shahank Date who stepped in to help finish off this release! *** Only "show-stopper" problems will be fixed *** We are extreme short of manpower at the moment. So as much as I would like to address each and every

Hi, Newbie here. I''ve got a worker (for generating PDF reports) that uses the "thread_pool" to allow processing multiple reports simultaneously and queue up any requests that exceed the thread pool (pool_size = 10 currently). def process_pdf(user) thread_pool.defer(user) do |user| makepdf(user) end end My question is: I use a mutex to handle

I have two workers (ie two files in my RAILS_ROOT/lib/workers directory) on my Rails app. When I run a top on my box, I see *three* processes called packer_worker_r. Shouldn''t I just be having two? What is the third process for? Here is an extract of the output from my top command. The PIDs are 7084, 7085 and 7083. ============================================ PID USER PR NI VIRT

Within a threaded worker, I would like to have multiple methods. Will the following code work? def method_1(args) thread_pool.defer(args) do |args| #work end end def method_2(args) thread_pool.defer(args) do |args| #work end end . . . or, should I have one "meta"-method and pass which sub-method to perform via args? def meta_method(args)

I have a previous question regarding long tasks and the thread_pool (sorry for the dup, I didn''t see the first go through). To try and track things down, I made a change based on a suggestion found in the archives. I moved my import contacts worker to its own file and set the pool_size to 1. class ImportContactsWorker < BackgrounDRb::MetaWorker set_worker_name

We have 50 complex applications written in Microsoft Visual Basic 6.0 providing banking solutions. We are seriously thinking of porting our applications to Linux under WINE. Is it possible to do so? Nature of our VB applications: ------------------------------ 1. Extensively used all controls provided in VB 2. Total size of our applications is around 2 million lines 3. Applications are having

Hi Folks, I am glad to announce 1.0 release of BackgrounDRb. This would be a major release since 0.2 release of BackgrounDRb. Here is a brief summary of changes: - BackgrounDRb is DRb no longer. It makes use of EventDriven network programming library packet ( http://packet.googlecode.com ). - Since we moved to packet, many nasty thread issues, result hash corruption issues are totally

I have a job that I am starting from a Rails controller in production mode using MiddleMan: MiddleMan.new_worker(:class => :import_ccg_category_worker, :args => { :category_id => @category.id, :remote_category_id => params[:remote_category_id], :description => "Importing CCG products for #{@ category.name}" }) Both of the category ID variables are just basic integers. I

Hi Folks, We are glad to announce shiny new release of BackgrounDRb, which will soon become 1.0. A quick summary of changes: - BackgrounDRb is no londer DRb, its based on event driven network programming library packet ( http://www.packet.googlecode.com ) . - Since we moved to packet, many nasty thread issues, result hash corruption issues are totally gone. Lots of work has went in making

Greetings all, Unit tests are passing, but I''m having mucho problems getting brb to run without exceptions for more than 6 hours. I''m seeing a mixture of the following 2 exceptions: 20070605-11:26:56 (21497) failed to find slave socket - (RuntimeError) 20070605-11:26:56 (21497) /usr/local/lib/ruby/gems/1.8/gems/ slave-1.2.1/lib/slave.rb:435:in `initialize''

Forgive me if this has been addressed somewhere, but I have searched and can''t come up with anything. I am basically trying to distribute several web page scraping tasks among different threads, and have the results from each added to an Array which is ultimately returned by the backgroundrb worker. Here is an example of what I''m trying to do in a worker method: pages =