similar to: Queuing in backgroundrb, is it possible?

I use BDRb to process RSS feeds for users on my site (http://feedflix.com). I have a batch job that queries the DB for records that haven''t been updated in the last so many hours and kicks off a background job for each of them. If N records are returned by the DB, N background jobs gets queued and get done serially. As long as N is 255 or under, everything works like a charm.

Hi Folks, I am glad to announce 1.0 release of BackgrounDRb. This would be a major release since 0.2 release of BackgrounDRb. Here is a brief summary of changes: - BackgrounDRb is DRb no longer. It makes use of EventDriven network programming library packet ( http://packet.googlecode.com ). - Since we moved to packet, many nasty thread issues, result hash corruption issues are totally

Hey Guys, In the readme for Backgroundrb it says that windows support has been depcreated for this version, but then it goes on to mention how to use it in Windows. So I''m not sure if it should be running on windows or not, so I''ll ask anyway. Also, I''m new to ruby and I''m also new to server administration, so I apologize if my questions are pretty

I have a job that I am starting from a Rails controller in production mode using MiddleMan: MiddleMan.new_worker(:class => :import_ccg_category_worker, :args => { :category_id => @category.id, :remote_category_id => params[:remote_category_id], :description => "Importing CCG products for #{@ category.name}" }) Both of the category ID variables are just basic integers. I

Hi everyone, I''ve a class that try to spawn two workers, every 5 seconds. The code is something like: while true do MiddleMan.new_worker(:class => :researches_worker,:job_key => :researches) MiddleMan.new_worker(:class => :products_worker,:job_key => :products) sleep(5) end I''m using ":job_key", so if a worker hasn''t already ended its work

Hi, I am using backgroundrb to process audio files from a rails controller. Currently a new worker gets created every time the method is called on the worker, using this code: @job_key = MiddleMan.new_worker(:worker => :audio_file_worker, :job_key => Time.now.to_i) MiddleMan.worker(:audio_file_worker, @job_key).make_new_audio_file(params[:release_id]) I need to create the new

I''m struggling to keep backgroundrb v2 running for more than 24 hours. It appears to be running fine for a while, then tries to fire off a worker (exactly the same worker it''s been running for the past few hours), and it suddenly dies. I haven''t found any pattern for the cause of this. In backgroundrb_server.log, the final entry is this : 20070206-10:41:45

I have a scheduled worker that runs every minute, checking for new eBay auctions to post. If it finds an auction, then it posts it. If there are no auctions in the database that are queued up, it simply does nothing. I got the worker to finally run as a scheduled worker, every minute. However, after about 30-45 scheduled runs, it simply stops running. My backgroundrb_schedules.yml file: ------

I have a previous question regarding long tasks and the thread_pool (sorry for the dup, I didn''t see the first go through). To try and track things down, I made a change based on a suggestion found in the archives. I moved my import contacts worker to its own file and set the pool_size to 1. class ImportContactsWorker < BackgrounDRb::MetaWorker set_worker_name

I recently switched over to v1.0, and things are rolling along pretty well. However, one thing that has always been a little confusing to me is knowing when to use thread_pool. Since most of my bgrb workers are called from my web app to process rather than being scheduled, I''m using the thread_pool for every call. Unfortunately, that means that I have to split up workers by how many

Greetings all, Unit tests are passing, but I''m having mucho problems getting brb to run without exceptions for more than 6 hours. I''m seeing a mixture of the following 2 exceptions: 20070605-11:26:56 (21497) failed to find slave socket - (RuntimeError) 20070605-11:26:56 (21497) /usr/local/lib/ruby/gems/1.8/gems/ slave-1.2.1/lib/slave.rb:435:in `initialize''

Hi All, I''m using Backgroundrb as a general purpose long-running-task back-end (upload processing, email sending, etc), and it''s been a great solution. However I''ve recently run into some some intermittent connection issues that have me baffled. I''m running on apache2/fcgid and the problem occurs in both devlopment and production mode. The problem seems to

Hello, I am trying to make DRb calls to a external server from a backgrounDRb worker, but I''m getting always the next exception when I make the remote call. > ERROR: > drbunix:///tmp/backgroundrb.11175/experimenter_worker_3e899de945f0803a82cd257cfac316d0_0_0.0738904928422051 > - #<Errno::ENOENT: No such file or directory - >

Hi Folks, We are glad to announce shiny new release of BackgrounDRb, which will soon become 1.0. A quick summary of changes: - BackgrounDRb is no londer DRb, its based on event driven network programming library packet ( http://www.packet.googlecode.com ) . - Since we moved to packet, many nasty thread issues, result hash corruption issues are totally gone. Lots of work has went in making

Forgive me if this has been addressed somewhere, but I have searched and can''t come up with anything. I am basically trying to distribute several web page scraping tasks among different threads, and have the results from each added to an Array which is ultimately returned by the backgroundrb worker. Here is an example of what I''m trying to do in a worker method: pages =

Hi all I''m experiencing something strange with a :job_key, it took me a very long time to find out a worker wasn''t doing it''s job because of the name of my job_key. The first one I tried was :make_kohier, and with that name (and lot''s of other ones) I can''t get the worker to work. I do get my key in return, but the job doesn''t start

Hi, i''m going to implement a syndication-service, which will get lists in xml with some meta-data an enclosed video files, which will get encoded at the end. The syndication run will be startet every five minutes of a full hour. So i thought to build 4 Worker. One for checking which feeds to syndicate (syndication_worker) at a specific time, one for processing the list

I''m trying to run a single worker that could perform a periodic task for a given user. >From a controller, I imagine something like: def start_job MiddleMan.ask_work(:worker => :foo_worker, :worker_method => :perform_task, :data => { :user_id = current_user.id }) end def check_job @status = MiddleMan.ask_status(:worker => :foo_worker)[current_user.id] end

Hi Folks, Well, Yesterday I pushed some more updates that I was cooking since last couple of days. Here is a brief list: 1. Results from worker stays even after a worker dies. 2. You can ask backgroundrb to store status objects in memcache cluster now. by default its stored in master process memory, but I have introduced an option to allow it to be stored in a memcache cluster:

Hi, Newbie here. I''ve got a worker (for generating PDF reports) that uses the "thread_pool" to allow processing multiple reports simultaneously and queue up any requests that exceed the thread pool (pool_size = 10 currently). def process_pdf(user) thread_pool.defer(user) do |user| makepdf(user) end end My question is: I use a mutex to handle