similar to: Background Processing for CSV Exports of data

Hi BackgrounDRber''s, I''m helping Obie Fernandez write a chapter on Background Processing in his upcoming Pro Rails book, and wanted to see if I could run it past the group. Any input is appreciated. See attached (word doc, owch)! p.s. thanks Ezra skaar, and File.read(CONTRIBUTORS).collect Cheers, -- Charles Brian Quinn self-promotion: www.seebq.com highgroove studios:

Hello all, i just started using backgroundrb and i really like it a lot so far. I finally succeeded in setting up version 0.21 and the examples from the readme run fine. Now i do not want to create and delete my worker anytime i want it to do something but call repeatedly a method with arguments. There is a arg_method defined in the readme so i guess this is a feasible way to procede.

I have bdrb successfully set up and running in a local project. It''s doing two things: 1) processing large csv files and using ActiveRecord to insert 2) sending large batches of emails using ActionMailer In development both work well. I see my ActionMailer emails in the log (though it''s the development.log -- odd?) On the production server csv processing works well, but

Is it possible to have 2 seperate instances of backgroundrb running on the same machine? I''m having to develop on the same box as production and I can only get backgroundrb to work for one or the other. Thanks in advance, Chris.

Hi there I have some long running queries against a nice litte database (20 million new records per day) that I have offloaded to backgroundRB. The trouble is, that backgroundrb seems to be blockin on Model.find calls. Here''s the code --- worker.rb -- def do_work(args) @rows = [] @percent = 0 @host = Host.find( args[:id]) ips = @host.ipaddresses step

Hi all, I''ve been having trouble for a long time with backgroundrb processes that suddenly vanish without a trace. What happens is that at some point I discover that all the backgroundrb processes are suddenly gone. Nothing special is seen in any of the log files. This has happened intermittently for a long time, and I was hoping that upgrading to 1.0.4 would somehow help me out, but

Hi, I am fairly new to ruby/rails, but I thought I did a fair job of setting up my background worker, but after some time, maybe 4-5 hours of running, it just stops running, without any messages. Here is the yml file: backgroundrb.yml --- :port: 2000 :rails_env: production :host: localhost here is my schedules file: alerts_task: :class: :alerts_worker :job_key: :alerts_key

Hello - I would like to figure out a way to have backgroundrb start/stop with the same UID that dispatch.fcgi uses, at the same point in time that apache is started/stopped. I see a technique that seems plausable in this archive message: http://rubyforge.org/pipermail/backgroundrb-devel/2007-February/000723.html which addresses the starting aspect, in a lazy fashion. That seems fine,

<backgroundrb-devel at rubyforge.org>I am trying to send email as a background process from within a Rails app. The goal is that when I go to http://mysite.com/foo/bar, the background job kicks in and sends an email out. The relevant code snippets are below. This errors out with a wrong number of arguments passed (2 for 1) message. I think I am making a mess of passing parameters to the

I have a question on how multiple b/g processes that are kicked off get handled in terms of sequence/schedule. On my site, a user enters an RSS feed to be processed and since this takes about 5-10 secs, I pass the process off to a background job and meanwhile do some Ajaxy spinners and tap-dancing until the job completes and then redirect_to the appropriate page. This works great. I also need to

Hi. We have a linux box serving samba to about 10 workstations, mostly running maya. The server is also used by a 20 machine render farm. We recently switched these 30 machines from NT4 to Windows 2000, and have noticed a significant reduction in performance as a result - particularly with reading large sequences of image files into the workstations. The load generated by firing off a render

Hi, I have a question about exporting interactive 3D plots. I use the following code to plot a contour of a trivariate normal distribution: library(mvtnorm) library(rgl) library(misc3d) n=25 x=seq(-3,3,length=n) X=cbind(rep(x,each=n**2),rep(rep(x,each=n),n),rep(x,n**2)) p=array(dmvnorm(X,sigma=diag(3)*0.5+0.5),c(n,n,n)) contour3d(p,x,x,x,level=mean(p)) lim=c(-3,3)

Looks like my first question a couple of days ago wasn?t very clever asked, at least very sucessfull.... So, anyway...i will try it again: I have to prefix that I am far away from fully understanding the overall concept yet, and there?s no way to get some hints in the www so far, at least my resaerches dont lead to any helpfull stuff. image_controller.rb: def create MiddleMan.new_worker

Hello backgroundrb-devel my name is Teeranit Ruangdet. I''m fairly new to Ruby on Rails so please bear with me here. I''m currently on a project where I need to import a whole bunch of records from spreadsheet(.csv) and save them in the database. My problem is that while I''m importing (huge file...say 5000 employees), no one else can access the website (it basically block

Hello, Bravo Ezra for Backgroundrb! What an elegant piece of engineering. I envision many folks like myself looking at our CPU hogging executables and finally seeing a clean path to the Web. I was wondering if any folks have a good strategy for calling out to a C++ (or any other compiled) executable from a worker. Is the only option to call system() from a worker? That feels a bit unsatisfying.

Hi, I was trying to read & filter records from a large file using read.csv.sql. I was successfully able to do that with a ~1 GB file. However I get the following error with a >2 GB file which has exactly the same structure as the first file. *Error in try({ : * * RS-DBI driver: (RS_sqlite_import: cannot open file <filename> )* -I have checked the file permission and the file

When i call a worker method my browsers hangs until the task is finished. The same code works nicely in the background when inside the do_work method. Does this mean i can only start a task in background via the do_work method ? Or am i doing something wrong here ? regards tom -------------- next part -------------- An HTML attachment was scrubbed... URL:

I have an archiving application that handles large file uploads. I already use backgroundrb to run a behind the scenes bulk import of records. All good. Thank you Ezra. Now I would like to add long running file uploads to the things that backgroundrb does for me. That''s right that ole chestnut. I''ve seen a lot of mention of this practice around the web, but have yet

I'm relatively new to R, and have looked through the documentation and FAQ, and have not been able to find out how I can accomplish something. If someone can point me in the right direction, I would greatly appreciate it. I am doing system performance tests, and end up with large volumes of data that consists of transaction timings, coupled with system timings. The analysis process is

Hello, I''ve been working for a few days on a problem which, after tracking it down, seems to be related to unicorn. I''m hoping that somebody might be able to help with information about what''s going on. In the Rails 3 application that I support, we sometimes fork during a web request in order to do background processing behind the scenes without the user (and the