similar to: Defer multiple methods within the same worker to the thread pool?

Displaying 20 results from an estimated 4000 matches similar to: "Defer multiple methods within the same worker to the thread pool?"

2015 May 31
4
[LLVMdev] Hash Table Virtual Calls with Conflict Resolution Stubs
Hi everyone, I'm the developer of the Loci programming language ( http://loci-lang.org ), for which the compiler is a front-end for LLVM. I would like to say that LLVM has been extremely useful for the development of the compiler and so thank you everyone for building this amazing system. ---- Virtual Method Calls ---- While most aspects of the language map well onto LLVM IR, it seems
2009 Mar 17
11
Refactoring module
Dear all Please see the following module, In module SX3 SX4 and SX5 have similar class Tasklist, but inherit from different class. I will use the following code to connect to different data sources RemoteSX3Model.establish_connection sx3_hash RemoteSX4Model.establish_connection sx4_hash RemoteSX5Model.establish_connection sx5_hash How can I refactor my code in module to look simpler? Thank you
2007 Aug 08
3
Cannot start workers in production mode
I have a job that I am starting from a Rails controller in production mode using MiddleMan: MiddleMan.new_worker(:class => :import_ccg_category_worker, :args => { :category_id => @category.id, :remote_category_id => params[:remote_category_id], :description => "Importing CCG products for #{@ category.name}" }) Both of the category ID variables are just basic integers. I
2014 Jan 16
3
[LLVMdev] Do all user-written passes have to be run through a PassManager object (called from outside the LLVM infrastructure)?
I have written several passes that have no pre-requisites for any previous LLVM native passes prior to my own. For those passes, I have verified that at least the following two approaches are equivalent in terms of executing those self-written passes and getting the correct results: #if METHOD_1 PassManager PM;
2008 May 09
1
register_status for excess thread_pool?
Hi, Newbie here. I''ve got a worker (for generating PDF reports) that uses the "thread_pool" to allow processing multiple reports simultaneously and queue up any requests that exceed the thread pool (pool_size = 10 currently). def process_pdf(user) thread_pool.defer(user) do |user| makepdf(user) end end My question is: I use a mutex to handle
2008 Jan 03
1
Thread_pool bug?
I have a previous question regarding long tasks and the thread_pool (sorry for the dup, I didn''t see the first go through). To try and track things down, I made a change based on a suggestion found in the archives. I moved my import contacts worker to its own file and set the pool_size to 1. class ImportContactsWorker < BackgrounDRb::MetaWorker set_worker_name
2007 Mar 13
3
Scheduled worker dies after about 30-45 runs
I have a scheduled worker that runs every minute, checking for new eBay auctions to post. If it finds an auction, then it posts it. If there are no auctions in the database that are queued up, it simply does nothing. I got the worker to finally run as a scheduled worker, every minute. However, after about 30-45 scheduled runs, it simply stops running. My backgroundrb_schedules.yml file: ------
2007 Feb 22
1
failed to find slave socket - (RuntimeError)
Hi everyone, I''ve a class that try to spawn two workers, every 5 seconds. The code is something like: while true do MiddleMan.new_worker(:class => :researches_worker,:job_key => :researches) MiddleMan.new_worker(:class => :products_worker,:job_key => :products) sleep(5) end I''m using ":job_key", so if a worker hasn''t already ended its work
2007 Dec 19
6
thread_pooling sleeping
I''m trying to run a single worker that could perform a periodic task for a given user. >From a controller, I imagine something like: def start_job MiddleMan.ask_work(:worker => :foo_worker, :worker_method => :perform_task, :data => { :user_id = current_user.id }) end def check_job @status = MiddleMan.ask_status(:worker => :foo_worker)[current_user.id] end
2008 Jun 10
4
adding results from threads to a collection and returning it
Forgive me if this has been addressed somewhere, but I have searched and can''t come up with anything. I am basically trying to distribute several web page scraping tasks among different threads, and have the results from each added to an Array which is ultimately returned by the backgroundrb worker. Here is an example of what I''m trying to do in a worker method: pages =
2008 Jan 03
1
Memory leak and long process problem
I use backgroundrb for many long tasks in my system, but I''m having issues with one in particular. Two large tasks for me are importing people and updating companies. def import_contacts(args = nil) thread_pool.defer(args) do |job_id| begin job = ImportJob.find(job_id) job.process_job rescue => err logger.error "MscWorker#import_contacts
2008 May 20
1
Couple questions on BDRb and concurrent processing
My Rails site uses BackgroundDRb and I have a couple of questions: 1. Can someone point me to some sample code/examples for how to use thread_pool? The website doc says to add a line ====== pool_size 10 ====== in my Worker class which seems straightforward. I wasn''t able to understand this part though: ========= thread_pool.defer(wiki_scrap_url) { |wiki_url| scrap_wikipedia(wiki_url) }
2007 Jan 23
0
Error while creating one worker from another
for some strange reason this error occours when i create one worker from another BUT only when i derive worker class from Worker::Base instead or Worker::RailsBase - when change it back it works fine, child worker can have empty do_work, but this error still ocours Bad file descriptor (Errno::EBADF) /var/lib/gems/1.8/gems/slave-1.2.0/lib/slave.rb:214:in `for_fd''
2006 Dec 03
0
worker is not executed
Hey, I''ve got a problem where my worker is not being properly executed.... it used to work in an earlier version of brb, but i realized my vendor/plugins was out of whack, upgraded to the newer schedule- oriented release (0.2.1). I can start brb, and run the new_worker middleman, but ntohing happens. When putting brb in ''run'' mode, i see this: undefined method
2007 Feb 07
1
Repeatedly dying with "failed to find slave socket"
I''m struggling to keep backgroundrb v2 running for more than 24 hours. It appears to be running fine for a while, then tries to fire off a worker (exactly the same worker it''s been running for the past few hours), and it suddenly dies. I haven''t found any pattern for the cause of this. In backgroundrb_server.log, the final entry is this : 20070206-10:41:45
2007 Dec 17
1
Some more updates, enhancements and fixes
Hi Folks, Well, Yesterday I pushed some more updates that I was cooking since last couple of days. Here is a brief list: 1. Results from worker stays even after a worker dies. 2. You can ask backgroundrb to store status objects in memcache cluster now. by default its stored in master process memory, but I have introduced an option to allow it to be stored in a memcache cluster:
2007 Apr 09
1
Drb Connection error on multiple dispatch.fcgi ''s
Hi All, I''m using Backgroundrb as a general purpose long-running-task back-end (upload processing, email sending, etc), and it''s been a great solution. However I''ve recently run into some some intermittent connection issues that have me baffled. I''m running on apache2/fcgid and the problem occurs in both devlopment and production mode. The problem seems to
2012 Sep 25
3
[PATCH] Btrfs: limit thread pool size when remounting
For some asynchronous threads, such as submit worker and cache worker, we limit their thread pool size when mounting. So we also need to do such things when remounting. Signed-off-by: Liu Bo <bo.li.liu@oracle.com> --- fs/btrfs/super.c | 13 ++++++++----- 1 files changed, 8 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 83d6f9f..a58e834 100644 ---
2008 Jan 22
2
Threadpool and queuing of tasks
I recently switched over to v1.0, and things are rolling along pretty well. However, one thing that has always been a little confusing to me is knowing when to use thread_pool. Since most of my bgrb workers are called from my web app to process rather than being scheduled, I''m using the thread_pool for every call. Unfortunately, that means that I have to split up workers by how many
2006 Dec 01
0
Worker won''t start work
(Sorry my last email had the wrong subject) Hi, I''ve just installed background rb 0.2.1. When I try start a worker in my application controller I receive the following error: wrong number of arguments (2 for 1) - (ArgumentError) /home/chris/projects/call_manager/trunk/call_manager_app/vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:211:in `initialize''