I have concluded that it is impossible to get this background process stable. I am out of ideas and could really use some help. Here is what I have: 50 workers, just running an infinite loop that constantly calls save! on a model. Keep in mind this is just for testing purposes. I catch all exceptions and put them in the log. I am tried setting allow_concurrency to true and false. Neither of them is stable. For both true and false I have got the following errors: #<ActiveRecord::StatementInvalid: Mysql::Error: Lock wait timeout exceeded; try restarting transaction: #<ActiveRecord::StatementInvalid: Mysql::Error: MySQL server has gone away: What have you found to be most stable? Setting allow concurrency to false and just putting a Mutex.synchronize object around all database calls? Or setting it true? Thank You, Ben Johnson E: bjohnson at contuitive.com O: 800-341-6826 M: 817-229-4258 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060818/d4bef568/attachment.html
On Aug 18, 2006, at 6:20 PM, Ben Johnson wrote:> I have concluded that it is impossible to get this background > process stable. I am out of ideas and could really use some help. > > Here is what I have: > > 50 workers, just running an infinite loop that constantly calls > save! on a model. Keep in mind this is just for testing purposes. I > catch all exceptions and put them in the log. > > I am tried setting allow_concurrency to true and false. Neither of > them is stable. For both true and false I have got the following > errors: > > #<ActiveRecord::StatementInvalid: Mysql::Error: Lock wait timeout > exceeded; try restarting transaction: > > #<ActiveRecord::StatementInvalid: Mysql::Error: MySQL server has > gone away: > > What have you found to be most stable? Setting allow concurrency to > false and just putting a Mutex.synchronize object around all > database calls? Or setting it true? > > Thank You, > Ben Johnson > E: bjohnson at contuitive.com > O: 800-341-6826 > M: 817-229-4258Hey Ben- I feel your pain. I have come to the conclusion that ActiveRecord is just not thread safe no matter what you do. My best suggestion is to use either dbi or mysql bindings directly in your workers. These are thread safe and while you would have to write some sql yourself, it would solve your issue. I get the feeling that railscore is not very interested in making rails thread safe at this point and pushing as many db connections in as many threads as you are doing is more the ActiveRecord can handle. I know its not what you want to hear but its the situation we have to deal with. -Ezra -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060819/d73ba4a7/attachment-0001.html
Hi Ezra, Thanks a lot for your help. Can you clarify what dbi is or mysql bindings? I am not sure what you are talking about. Thanks again for your help. Thank You, Ben Johnson E: bjohnson at contuitive.com O: 800-341-6826 M: 817-229-4258 On Aug 19, 2006, at 11:39 PM, Ezra Zygmuntowicz wrote:> > On Aug 18, 2006, at 6:20 PM, Ben Johnson wrote: > >> I have concluded that it is impossible to get this background >> process stable. I am out of ideas and could really use some help. >> >> Here is what I have: >> >> 50 workers, just running an infinite loop that constantly calls >> save! on a model. Keep in mind this is just for testing purposes. >> I catch all exceptions and put them in the log. >> >> I am tried setting allow_concurrency to true and false. Neither of >> them is stable. For both true and false I have got the following >> errors: >> >> #<ActiveRecord::StatementInvalid: Mysql::Error: Lock wait timeout >> exceeded; try restarting transaction: >> >> #<ActiveRecord::StatementInvalid: Mysql::Error: MySQL server has >> gone away: >> >> What have you found to be most stable? Setting allow concurrency >> to false and just putting a Mutex.synchronize object around all >> database calls? Or setting it true? >> >> Thank You, >> Ben Johnson >> E: bjohnson at contuitive.com >> O: 800-341-6826 >> M: 817-229-4258 > > > Hey Ben- > > I feel your pain. I have come to the conclusion that ActiveRecord > is just not thread safe no matter what you do. My best suggestion > is to use either dbi or mysql bindings directly in your workers. > These are thread safe and while you would have to write some sql > yourself, it would solve your issue. I get the feeling that > railscore is not very interested in making rails thread safe at > this point and pushing as many db connections in as many threads as > you are doing is more the ActiveRecord can handle. > > I know its not what you want to hear but its the situation we have > to deal with. > > -Ezra-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060820/9251d721/attachment.html
dbi and mysql is the basic layer of database access. its just a way to connect to a db server and send a query. no modelous abstractions, no stuff around, just tailor your sql like back in (good) old php days. thats surely not the rails way and uncomfortable but it should give no errors (except that your db only can handle a certain amount of connections) 2006/8/20, Ben Johnson <bjohnson at contuitive.com>:> > Hi Ezra, > Thanks a lot for your help. Can you clarify what dbi is or mysql bindings? > I am not sure what you are talking about. > > Thanks again for your help. > > Thank You, > Ben Johnson > E: bjohnson at contuitive.com > O: 800-341-6826 > M: 817-229-4258 > > > > On Aug 19, 2006, at 11:39 PM, Ezra Zygmuntowicz wrote: > > > On Aug 18, 2006, at 6:20 PM, Ben Johnson wrote: > > I have concluded that it is impossible to get this background process > stable. I am out of ideas and could really use some help. > Here is what I have: > > 50 workers, just running an infinite loop that constantly calls save! on a > model. Keep in mind this is just for testing purposes. I catch all > exceptions and put them in the log. > > I am tried setting allow_concurrency to true and false. Neither of them is > stable. For both true and false I have got the following errors: > > #<ActiveRecord::StatementInvalid: Mysql::Error: Lock wait timeout > exceeded; try restarting transaction: > > #<ActiveRecord::StatementInvalid: Mysql::Error: MySQL server has gone > away: > > What have you found to be most stable? Setting allow concurrency to false > and just putting a Mutex.synchronize object around all database calls? Or > setting it true? > > Thank You, > Ben Johnson > E: bjohnson at contuitive.com > O: 800-341-6826 > M: 817-229-4258 > > > > Hey Ben- > > I feel your pain. I have come to the conclusion that ActiveRecord is just > not thread safe no matter what you do. My best suggestion is to use either > dbi or mysql bindings directly in your workers. These are thread safe and > while you would have to write some sql yourself, it would solve your issue. > I get the feeling that railscore is not very interested in making rails > thread safe at this point and pushing as many db connections in as many > threads as you are doing is more the ActiveRecord can handle. > > I know its not what you want to hear but its the situation we have to deal > with. > > -Ezra > > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > >-- Michael Siebert <info at siebert-wd.de> www.siebert-wd.de - Gedanken lesen www.stellar-legends.de - Weltraum-Browsergame im Alpha-Stadium -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060820/ed5cf82a/attachment.html
I''m planning on using Backgroundrb to run background processes for a fairly large all-Rails application that is launching in two weeks. I''ve been getting a bit paranoid about how ActiveRecord would react with possibly as many as twenty workers chugging away at the same time on the same tables. I put together a dead-simple "test" and the results were, I think, very positive. (The OS is OSX 10.4.7) Setup: One worker class that queries a MySQL database (4.1.18) n times and then calls ActiveRecord::Base.connection.disconnect! and kill. The worker executes the "SHOW PROCESSLIST" query, which returns a list of active MySQL processes, and logs the number of active connections along with the to_s of the connection and the thread. The test script uses MiddleMan to instantiate 100 workers passing in the worker''s number as the number (10, 11, 12...110) of times the worker should loop and query the database. I took this approach so that the workers would be closing connections while other threads had connections open. Goals: 1) verify that each Thread has his own connection. 2) verify that each worker can call disconnect! to close their own connection to free it up for other workers. Here''s a snippet of the log file. worker number, ActiveRecord::Base.object_id, ActiveRecord::Base.connection.to_s, Thread.current.to_s, # of active MySQL connections 25, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: 0x277ba04>, #<Thread:0x277c4b8>, 12 21, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: 0x277a730>, #<Thread:0x277b0f4>, 12 18, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: 0x26c86e8>, #<Thread:0x26c8bfc>, 12 17, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: 0x26d61d0>, #<Thread:0x26d7620>, 12 16, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: 0x26e06d0>, #<Thread:0x26e0a18>, 12 15, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: 0x26e7098>, #<Thread:0x26e8218>, 12 23, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: 0x2784f8c>, #<Thread:0x27855a4>, 11 In this rather unscientific way I have positively confirmed (I think) that on Mac OSX 10.4.7, MySQL 4.1.18 and AR 1.14.4, each worker gets his own thread with his own connection, and can freely close his connection without adversely affecting any other thread. Any thoughts on this approach? I made one small change to swap out the SHOW PROCESS list query with MyModel.find :first and the results were the same -- each worker had his own thread with his own connection that he can close without affecting any other threads. Finally, I altered the start script to set allow concurrency to false, ran the test and not surprisingly all heck broke loose: 39, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: 0x26fdca8>, #<Thread:0x2752aa0>, 1 25, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: 0x26fdca8>, #<Thread:0x2779a88>, 1 86, 3705192,#<ActiveRecord::ConnectionAdapters::MysqlAdapter: 0x26fdca8>,#<Thread:0x2614cc4>, 54, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: 0x26fdca8>, #<Thread:0x271be9c>, 1 87, 3705192,#<ActiveRecord::ConnectionAdapters::MysqlAdapter: 0x26fdca8>,#<Thread:0x26de9fc>, Mysql::Error: MySQL server has gone away: SHOW PROCESSLIST; - (ActiveRecord::StatementInvalid) /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/ active_record/connection_adapters/abstract_adapter.rb:120:in `log'' /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/ active_record/connection_adapters/mysql_adapter.rb:184:in `execute'' Different threads but the same connection and the first worker that closes his connection ruins the whole party. Are results like the above consistent with the intention of Backgroundrb? Is my approach was at least somewhat valid? Erik On Aug 20, 2006, at 12:39 AM, Ezra Zygmuntowicz wrote:> > On Aug 18, 2006, at 6:20 PM, Ben Johnson wrote: > >> I have concluded that it is impossible to get this background >> process stable. I am out of ideas and could really use some help. >> >> Here is what I have: >> >> 50 workers, just running an infinite loop that constantly calls >> save! on a model. Keep in mind this is just for testing purposes. >> I catch all exceptions and put them in the log. >> >> I am tried setting allow_concurrency to true and false. Neither of >> them is stable. For both true and false I have got the following >> errors: >> >> #<ActiveRecord::StatementInvalid: Mysql::Error: Lock wait timeout >> exceeded; try restarting transaction: >> >> #<ActiveRecord::StatementInvalid: Mysql::Error: MySQL server has >> gone away: >> >> What have you found to be most stable? Setting allow concurrency >> to false and just putting a Mutex.synchronize object around all >> database calls? Or setting it true? >> >> Thank You, >> Ben Johnson >> E: bjohnson at contuitive.com >> O: 800-341-6826 >> M: 817-229-4258 > > > Hey Ben- > > I feel your pain. I have come to the conclusion that ActiveRecord > is just not thread safe no matter what you do. My best suggestion > is to use either dbi or mysql bindings directly in your workers. > These are thread safe and while you would have to write some sql > yourself, it would solve your issue. I get the feeling that > railscore is not very interested in making rails thread safe at > this point and pushing as many db connections in as many threads as > you are doing is more the ActiveRecord can handle. > > I know its not what you want to hear but its the situation we have > to deal with. > > -Ezra > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
Hey Eric- Yeah I think your approach is valid. Would you mind sending me the code you used for this experiment? I would like to run some tests myself. I will add some unit tests to try and see what the upper limits of threaded AR usage is. Thanks- -Ezra On Aug 20, 2006, at 5:20 AM, Erik Morton wrote:> I''m planning on using Backgroundrb to run background processes for a > fairly large all-Rails application that is launching in two weeks. > I''ve been getting a bit paranoid about how ActiveRecord would react > with possibly as many as twenty workers chugging away at the same > time on the same tables. I put together a dead-simple "test" and the > results were, I think, very positive. (The OS is OSX 10.4.7) > > Setup: One worker class that queries a MySQL database (4.1.18) n > times and then calls ActiveRecord::Base.connection.disconnect! and > kill. The worker executes the "SHOW PROCESSLIST" query, which returns > a list of active MySQL processes, and logs the number of active > connections along with the to_s of the connection and the thread. > > The test script uses MiddleMan to instantiate 100 workers passing in > the worker''s number as the number (10, 11, 12...110) of times the > worker should loop and query the database. I took this approach so > that the workers would be closing connections while other threads had > connections open. > > Goals: 1) verify that each Thread has his own connection. 2) verify > that each worker can call disconnect! to close their own connection > to free it up for other workers. > > Here''s a snippet of the log file. > > worker number, ActiveRecord::Base.object_id, > ActiveRecord::Base.connection.to_s, Thread.current.to_s, # of active > MySQL connections > 25, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: > 0x277ba04>, #<Thread:0x277c4b8>, 12 > 21, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: > 0x277a730>, #<Thread:0x277b0f4>, 12 > 18, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: > 0x26c86e8>, #<Thread:0x26c8bfc>, 12 > 17, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: > 0x26d61d0>, #<Thread:0x26d7620>, 12 > 16, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: > 0x26e06d0>, #<Thread:0x26e0a18>, 12 > 15, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: > 0x26e7098>, #<Thread:0x26e8218>, 12 > 23, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: > 0x2784f8c>, #<Thread:0x27855a4>, 11 > > In this rather unscientific way I have positively confirmed (I think) > that on Mac OSX 10.4.7, MySQL 4.1.18 and AR 1.14.4, each worker gets > his own thread with his own connection, and can freely close his > connection without adversely affecting any other thread. Any thoughts > on this approach? > > I made one small change to swap out the SHOW PROCESS list query with > MyModel.find :first and the results were the same -- each worker had > his own thread with his own connection that he can close without > affecting any other threads. > > Finally, I altered the start script to set allow concurrency to > false, ran the test and not surprisingly all heck broke loose: > > 39, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: > 0x26fdca8>, #<Thread:0x2752aa0>, 1 > 25, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: > 0x26fdca8>, #<Thread:0x2779a88>, 1 > 86, 3705192,#<ActiveRecord::ConnectionAdapters::MysqlAdapter: > 0x26fdca8>,#<Thread:0x2614cc4>, > 54, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: > 0x26fdca8>, #<Thread:0x271be9c>, 1 > 87, 3705192,#<ActiveRecord::ConnectionAdapters::MysqlAdapter: > 0x26fdca8>,#<Thread:0x26de9fc>, > Mysql::Error: MySQL server has gone away: SHOW PROCESSLIST; - > (ActiveRecord::StatementInvalid) > /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/ > active_record/connection_adapters/abstract_adapter.rb:120:in `log'' > /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/ > active_record/connection_adapters/mysql_adapter.rb:184:in `execute'' > > Different threads but the same connection and the first worker that > closes his connection ruins the whole party. > > Are results like the above consistent with the intention of > Backgroundrb? Is my approach was at least somewhat valid? > > Erik > > On Aug 20, 2006, at 12:39 AM, Ezra Zygmuntowicz wrote: > >> >> On Aug 18, 2006, at 6:20 PM, Ben Johnson wrote: >> >>> I have concluded that it is impossible to get this background >>> process stable. I am out of ideas and could really use some help. >>> >>> Here is what I have: >>> >>> 50 workers, just running an infinite loop that constantly calls >>> save! on a model. Keep in mind this is just for testing purposes. >>> I catch all exceptions and put them in the log. >>> >>> I am tried setting allow_concurrency to true and false. Neither of >>> them is stable. For both true and false I have got the following >>> errors: >>> >>> #<ActiveRecord::StatementInvalid: Mysql::Error: Lock wait timeout >>> exceeded; try restarting transaction: >>> >>> #<ActiveRecord::StatementInvalid: Mysql::Error: MySQL server has >>> gone away: >>> >>> What have you found to be most stable? Setting allow concurrency >>> to false and just putting a Mutex.synchronize object around all >>> database calls? Or setting it true? >>> >>> Thank You, >>> Ben Johnson >>> E: bjohnson at contuitive.com >>> O: 800-341-6826 >>> M: 817-229-4258 >> >> >> Hey Ben- >> >> I feel your pain. I have come to the conclusion that ActiveRecord >> is just not thread safe no matter what you do. My best suggestion >> is to use either dbi or mysql bindings directly in your workers. >> These are thread safe and while you would have to write some sql >> yourself, it would solve your issue. I get the feeling that >> railscore is not very interested in making rails thread safe at >> this point and pushing as many db connections in as many threads as >> you are doing is more the ActiveRecord can handle. >> >> I know its not what you want to hear but its the situation we have >> to deal with. >> >> -Ezra >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel >
Hey Eric, Would you mind sharing this code with me? I am experiencing the same problems... Abdul Ezra Zygmuntowicz wrote:> Hey Eric- > > Yeah I think your approach is valid. Would you mind sending me the > code you used for this experiment? I would like to run some tests > myself. I will add some unit tests to try and see what the upper > limits of threaded AR usage is. > > Thanks- > -Ezra > > On Aug 20, 2006, at 5:20 AM, Erik Morton wrote: > > >> I''m planning on using Backgroundrb to run background processes for a >> fairly large all-Rails application that is launching in two weeks. >> I''ve been getting a bit paranoid about how ActiveRecord would react >> with possibly as many as twenty workers chugging away at the same >> time on the same tables. I put together a dead-simple "test" and the >> results were, I think, very positive. (The OS is OSX 10.4.7) >> >> Setup: One worker class that queries a MySQL database (4.1.18) n >> times and then calls ActiveRecord::Base.connection.disconnect! and >> kill. The worker executes the "SHOW PROCESSLIST" query, which returns >> a list of active MySQL processes, and logs the number of active >> connections along with the to_s of the connection and the thread. >> >> The test script uses MiddleMan to instantiate 100 workers passing in >> the worker''s number as the number (10, 11, 12...110) of times the >> worker should loop and query the database. I took this approach so >> that the workers would be closing connections while other threads had >> connections open. >> >> Goals: 1) verify that each Thread has his own connection. 2) verify >> that each worker can call disconnect! to close their own connection >> to free it up for other workers. >> >> Here''s a snippet of the log file. >> >> worker number, ActiveRecord::Base.object_id, >> ActiveRecord::Base.connection.to_s, Thread.current.to_s, # of active >> MySQL connections >> 25, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >> 0x277ba04>, #<Thread:0x277c4b8>, 12 >> 21, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >> 0x277a730>, #<Thread:0x277b0f4>, 12 >> 18, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >> 0x26c86e8>, #<Thread:0x26c8bfc>, 12 >> 17, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >> 0x26d61d0>, #<Thread:0x26d7620>, 12 >> 16, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >> 0x26e06d0>, #<Thread:0x26e0a18>, 12 >> 15, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >> 0x26e7098>, #<Thread:0x26e8218>, 12 >> 23, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >> 0x2784f8c>, #<Thread:0x27855a4>, 11 >> >> In this rather unscientific way I have positively confirmed (I think) >> that on Mac OSX 10.4.7, MySQL 4.1.18 and AR 1.14.4, each worker gets >> his own thread with his own connection, and can freely close his >> connection without adversely affecting any other thread. Any thoughts >> on this approach? >> >> I made one small change to swap out the SHOW PROCESS list query with >> MyModel.find :first and the results were the same -- each worker had >> his own thread with his own connection that he can close without >> affecting any other threads. >> >> Finally, I altered the start script to set allow concurrency to >> false, ran the test and not surprisingly all heck broke loose: >> >> 39, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >> 0x26fdca8>, #<Thread:0x2752aa0>, 1 >> 25, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >> 0x26fdca8>, #<Thread:0x2779a88>, 1 >> 86, 3705192,#<ActiveRecord::ConnectionAdapters::MysqlAdapter: >> 0x26fdca8>,#<Thread:0x2614cc4>, >> 54, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >> 0x26fdca8>, #<Thread:0x271be9c>, 1 >> 87, 3705192,#<ActiveRecord::ConnectionAdapters::MysqlAdapter: >> 0x26fdca8>,#<Thread:0x26de9fc>, >> Mysql::Error: MySQL server has gone away: SHOW PROCESSLIST; - >> (ActiveRecord::StatementInvalid) >> /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/ >> active_record/connection_adapters/abstract_adapter.rb:120:in `log'' >> /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/ >> active_record/connection_adapters/mysql_adapter.rb:184:in `execute'' >> >> Different threads but the same connection and the first worker that >> closes his connection ruins the whole party. >> >> Are results like the above consistent with the intention of >> Backgroundrb? Is my approach was at least somewhat valid? >> >> Erik >> >> On Aug 20, 2006, at 12:39 AM, Ezra Zygmuntowicz wrote: >> >> >>> On Aug 18, 2006, at 6:20 PM, Ben Johnson wrote: >>> >>> >>>> I have concluded that it is impossible to get this background >>>> process stable. I am out of ideas and could really use some help. >>>> >>>> Here is what I have: >>>> >>>> 50 workers, just running an infinite loop that constantly calls >>>> save! on a model. Keep in mind this is just for testing purposes. >>>> I catch all exceptions and put them in the log. >>>> >>>> I am tried setting allow_concurrency to true and false. Neither of >>>> them is stable. For both true and false I have got the following >>>> errors: >>>> >>>> #<ActiveRecord::StatementInvalid: Mysql::Error: Lock wait timeout >>>> exceeded; try restarting transaction: >>>> >>>> #<ActiveRecord::StatementInvalid: Mysql::Error: MySQL server has >>>> gone away: >>>> >>>> What have you found to be most stable? Setting allow concurrency >>>> to false and just putting a Mutex.synchronize object around all >>>> database calls? Or setting it true? >>>> >>>> Thank You, >>>> Ben Johnson >>>> E: bjohnson at contuitive.com >>>> O: 800-341-6826 >>>> M: 817-229-4258 >>>> >>> Hey Ben- >>> >>> I feel your pain. I have come to the conclusion that ActiveRecord >>> is just not thread safe no matter what you do. My best suggestion >>> is to use either dbi or mysql bindings directly in your workers. >>> These are thread safe and while you would have to write some sql >>> yourself, it would solve your issue. I get the feeling that >>> railscore is not very interested in making rails thread safe at >>> this point and pushing as many db connections in as many threads as >>> you are doing is more the ActiveRecord can handle. >>> >>> I know its not what you want to hear but its the situation we have >>> to deal with. >>> >>> -Ezra >>> _______________________________________________ >>> Backgroundrb-devel mailing list >>> Backgroundrb-devel at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >>> >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >> >> > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > >
Hey Abdur- Here is the code that he sent me when I requested it. -------------- next part -------------- A non-text attachment was scrubbed... Name: backgroundrbtests.tar.gz Type: application/x-gzip Size: 1350 bytes Desc: not available Url : http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060907/0e6e75bd/attachment-0001.gz -------------- next part -------------- -Ezra On Sep 7, 2006, at 10:18 AM, Abdur-Rahman Advany wrote:> Hey Eric, > > Would you mind sharing this code with me? I am experiencing the same > problems... > > Abdul > > Ezra Zygmuntowicz wrote: >> Hey Eric- >> >> Yeah I think your approach is valid. Would you mind sending me the >> code you used for this experiment? I would like to run some tests >> myself. I will add some unit tests to try and see what the upper >> limits of threaded AR usage is. >> >> Thanks- >> -Ezra >> >> On Aug 20, 2006, at 5:20 AM, Erik Morton wrote: >> >> >>> I''m planning on using Backgroundrb to run background processes for a >>> fairly large all-Rails application that is launching in two weeks. >>> I''ve been getting a bit paranoid about how ActiveRecord would react >>> with possibly as many as twenty workers chugging away at the same >>> time on the same tables. I put together a dead-simple "test" and the >>> results were, I think, very positive. (The OS is OSX 10.4.7) >>> >>> Setup: One worker class that queries a MySQL database (4.1.18) n >>> times and then calls ActiveRecord::Base.connection.disconnect! and >>> kill. The worker executes the "SHOW PROCESSLIST" query, which >>> returns >>> a list of active MySQL processes, and logs the number of active >>> connections along with the to_s of the connection and the thread. >>> >>> The test script uses MiddleMan to instantiate 100 workers passing in >>> the worker''s number as the number (10, 11, 12...110) of times the >>> worker should loop and query the database. I took this approach so >>> that the workers would be closing connections while other threads >>> had >>> connections open. >>> >>> Goals: 1) verify that each Thread has his own connection. 2) verify >>> that each worker can call disconnect! to close their own connection >>> to free it up for other workers. >>> >>> Here''s a snippet of the log file. >>> >>> worker number, ActiveRecord::Base.object_id, >>> ActiveRecord::Base.connection.to_s, Thread.current.to_s, # of active >>> MySQL connections >>> 25, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >>> 0x277ba04>, #<Thread:0x277c4b8>, 12 >>> 21, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >>> 0x277a730>, #<Thread:0x277b0f4>, 12 >>> 18, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >>> 0x26c86e8>, #<Thread:0x26c8bfc>, 12 >>> 17, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >>> 0x26d61d0>, #<Thread:0x26d7620>, 12 >>> 16, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >>> 0x26e06d0>, #<Thread:0x26e0a18>, 12 >>> 15, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >>> 0x26e7098>, #<Thread:0x26e8218>, 12 >>> 23, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >>> 0x2784f8c>, #<Thread:0x27855a4>, 11 >>> >>> In this rather unscientific way I have positively confirmed (I >>> think) >>> that on Mac OSX 10.4.7, MySQL 4.1.18 and AR 1.14.4, each worker >>> gets >>> his own thread with his own connection, and can freely close his >>> connection without adversely affecting any other thread. Any >>> thoughts >>> on this approach? >>> >>> I made one small change to swap out the SHOW PROCESS list query with >>> MyModel.find :first and the results were the same -- each worker had >>> his own thread with his own connection that he can close without >>> affecting any other threads. >>> >>> Finally, I altered the start script to set allow concurrency to >>> false, ran the test and not surprisingly all heck broke loose: >>> >>> 39, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >>> 0x26fdca8>, #<Thread:0x2752aa0>, 1 >>> 25, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >>> 0x26fdca8>, #<Thread:0x2779a88>, 1 >>> 86, 3705192,#<ActiveRecord::ConnectionAdapters::MysqlAdapter: >>> 0x26fdca8>,#<Thread:0x2614cc4>, >>> 54, 3705192, #<ActiveRecord::ConnectionAdapters::MysqlAdapter: >>> 0x26fdca8>, #<Thread:0x271be9c>, 1 >>> 87, 3705192,#<ActiveRecord::ConnectionAdapters::MysqlAdapter: >>> 0x26fdca8>,#<Thread:0x26de9fc>, >>> Mysql::Error: MySQL server has gone away: SHOW PROCESSLIST; - >>> (ActiveRecord::StatementInvalid) >>> /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/ >>> active_record/connection_adapters/abstract_adapter.rb:120:in `log'' >>> /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/ >>> active_record/connection_adapters/mysql_adapter.rb:184:in `execute'' >>> >>> Different threads but the same connection and the first worker that >>> closes his connection ruins the whole party. >>> >>> Are results like the above consistent with the intention of >>> Backgroundrb? Is my approach was at least somewhat valid? >>> >>> Erik >>> >>> On Aug 20, 2006, at 12:39 AM, Ezra Zygmuntowicz wrote: >>> >>> >>>> On Aug 18, 2006, at 6:20 PM, Ben Johnson wrote: >>>> >>>> >>>>> I have concluded that it is impossible to get this background >>>>> process stable. I am out of ideas and could really use some help. >>>>> >>>>> Here is what I have: >>>>> >>>>> 50 workers, just running an infinite loop that constantly calls >>>>> save! on a model. Keep in mind this is just for testing purposes. >>>>> I catch all exceptions and put them in the log. >>>>> >>>>> I am tried setting allow_concurrency to true and false. Neither of >>>>> them is stable. For both true and false I have got the following >>>>> errors: >>>>> >>>>> #<ActiveRecord::StatementInvalid: Mysql::Error: Lock wait timeout >>>>> exceeded; try restarting transaction: >>>>> >>>>> #<ActiveRecord::StatementInvalid: Mysql::Error: MySQL server has >>>>> gone away: >>>>> >>>>> What have you found to be most stable? Setting allow concurrency >>>>> to false and just putting a Mutex.synchronize object around all >>>>> database calls? Or setting it true? >>>>> >>>>> Thank You, >>>>> Ben Johnson >>>>> E: bjohnson at contuitive.com >>>>> O: 800-341-6826 >>>>> M: 817-229-4258 >>>>> >>>> Hey Ben- >>>> >>>> I feel your pain. I have come to the conclusion that ActiveRecord >>>> is just not thread safe no matter what you do. My best suggestion >>>> is to use either dbi or mysql bindings directly in your workers. >>>> These are thread safe and while you would have to write some sql >>>> yourself, it would solve your issue. I get the feeling that >>>> railscore is not very interested in making rails thread safe at >>>> this point and pushing as many db connections in as many threads as >>>> you are doing is more the ActiveRecord can handle. >>>> >>>> I know its not what you want to hear but its the situation we have >>>> to deal with. >>>> >>>> -Ezra >>>> _______________________________________________ >>>> Backgroundrb-devel mailing list >>>> Backgroundrb-devel at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >>>> >>> _______________________________________________ >>> Backgroundrb-devel mailing list >>> Backgroundrb-devel at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >>> >>> >> >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >> >> > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel