similar to: Rails, background threads, ActiveRecord

I''m using backgroundrb to periodically download a large file via ftp from a remote location and then import it into the database. To perform the import, I was using the following: ActiveRecord::Base.connection.execute(%{load data infile ...;}) although on a file with 2.5 million records, this can take 5 minutes, which seems to tie up my rails application while this executes (even

I have a question concerning where to use the synchronize function. The point of it is to put all of the code blacks passed to this function in a queue. This make sure that code gets ran one at a time. This is required when doing any kind of database activity in backgrondrb and multiple threads. Otherwise I lose the connection to my database. So here is my question: Do I need to do: @lock =

First of all, I''d like to do what I''ve seen several others here do: thank Ezra and any other contributors for producing a really great plugin. Even though I''m having some minor problems, this is all so approachable and easy to work with and dig into... I hate to think of the hoops you''d need to jump through (both as creator of such a system and as a user) to

Hi, There is an issue on socket connection with backgroundrb (rev 47) and mysql backend: after a certain number of job you reach the mysql connection limit as socket are never closed and nobody can connect to mysql anymore till you stop the backgroundrb daemon. Already reported on: http://rubyforge.org/pipermail/backgroundrb-devel/2006-July/000066.html,

My app consists of receiving emails with image attachments from its users so I have been employing the MMS2R gem and Fetcher. Every 60 seconds the app goes to the email server to fetch emails. Under Rails 2.1.1 things worked fine - when I received an email, the email was fetched, parsed and saved to the database. However I needed to upgrade to Rails 2.2.2 for unrelated reasons and now I get the

I have concluded that it is impossible to get this background process stable. I am out of ideas and could really use some help. Here is what I have: 50 workers, just running an infinite loop that constantly calls save! on a model. Keep in mind this is just for testing purposes. I catch all exceptions and put them in the log. I am tried setting allow_concurrency to true and false. Neither

I lost my "connection lost" error.. but now I''ve got a "Mysql::Error: MySQL server has gone away" error.. It comes from: /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.6/lib/active_record/connection_adapters/abstract_adapter.rb:128:in ''log''. I''m not sure if Rails / BackgroundRb uses it .. but i''ve got the mysql (2.7) gem

if CONFIG[''load_rails''] ActiveRecord::Base.allow_concurrency = true ActiveRecord::Base.establish_connection(YAML.load(ERB.new(IO.read ("#{RAILS_ROOT}/#{CONFIG[''database_yml'']}")).result)[CONFIG[''environment'']]) end if i read the code right, this option doesnt really load rails. wouldnt it be better (for those who rely on the

Hi all, Both the rails book (1st Ed) and the docs on the rails site (*and* the Mongrel FAQ) tell me that rails (& ActionPack /ActiveRecord) is not threadsafe. Since I need lots of worker threads in the app I''m working on (details at the end of this mail) this poses quite a problem for me. Is the lack of thread safety a conscious design decision (kind of understandable for rails

I have bdrb successfully set up and running in a local project. It''s doing two things: 1) processing large csv files and using ActiveRecord to insert 2) sending large batches of emails using ActionMailer In development both work well. I see my ActionMailer emails in the log (though it''s the development.log -- odd?) On the production server csv processing works well, but

First off, amazing plugin! Being a rails noob, it helped me do some stuff I would''ve thought way over my head. Which brings me to my problem. :) I''m using backgroundrb to handle some processes that take so long that I don''t want them tied to a page refresh (several xml-rpc calls and saving data in text files on the server). I got it working with backgroundrb fairly

Hi, in a production environment we are running into "MySQL server has gone away" errors (see below) when we call rebuild_index in aaf. This happens sporadically. Any suggestions for what might be wrong or workarounds? I guess we try a reconnect before we call rebuild_index? Matthew ActiveRecord::StatementInvalid (Mysql::Error: MySQL server has gone away: SELECT count(*) AS

Hi All, I have this small doubt. Does backgroundrb have anything to do with dispatch.fcgi? some dispatch.fcgi are taking memory as much as 186 MB .... this is monstrous man... Also, if i put load rails environment with backgroundrb.yml file to be false.What will happen? i just loses access to db through ActiveRecord right? I can still use normal mysql gem to access db and modify the

In my continued efforts to port my rails 1.x app to Rails 2.1.2, I keep running into what appear to be ActiveRecord bugs. I am using an :include on a :has_many definition: class Request < ActiveRecord::Base has_many :service_types, :order=>''service_types.id ASC'', :include=>:service_response [...] There''s no reason this wouldn''t be supported in

Hi there, I''m trying to create a worker to do some asynchronous web client work and update the data model with its results. It seems, however, that every call I make to an ActiveRecord subclass causes the whole thing to block. At least I think it''s blocked. Right now I''m just using debug log output. Is there a decent way to debug/test workers I can try? Is there any

Has anyone actually gotten the activerecord-sybase-adapter to work in Rails2? I can''t seem to, and can''t figure out why. I am on RedHat linux. I have the sybase client libraries installed, I have the adapter installed. After installing the patch at https://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/2455-update-for-sybase_adapter#ticket-2455-4 to get a slightly

I''ve been trying unsuccessfully for the better part of today to write a worker that can use by ActiveRecord model classes. The bizarre thing is that worker just stops at the point accessing the database connection. An exception isn''t raising, it just dies. My worker class is below, I''ve interspersed log messages throughout to try to determine where it is dying. =====

Hi Folks, I''ve got a tiny little problem using MySQL-backed ActiveRecord in my worker class. Sometimes it just blocks infinitely when I try to access it. I don''t know whether it is in AR itself or in the SQL driver. It just stops doing anything, logging nothing, no exceptions are raised. I don''t know if it matters that I call the worker''s methods via

Hello, I want to scan many URLs simultaneously with a new scanner worker for each URL, but I keep getting Mysql::Error: Lost connection to MySQL. Yes, allow_concurrency=false. Each scanner thread needs access to my ActiveRecord models to store discovered feeds and whatnot. There is a DB table called page_scans that acts as a big queue holding URLs waiting to be scanned. It would be

So I''ve got an ActiveRecord model pointing to a MySQL db, with an auto-serialized column ("serialize :columnName"). Thing is, MySQL, depending on how it''s configured (like, by default), has a bad habit of just truncating your data if it''s too wide for the column, with no error raised. Yeah, I can probably reconfigure MySQL and/or my AR connection to it. But