Ken Egervari
2011-May-25 19:00 UTC
[rspec-users] For those using Rails with RSpec... what do you guys use for creating Factories?
I am using factory_girl, and I have discovered that it is chiefly responsible for making my tests run slow. I have posted a question about this on Stack Overflow: http://stackoverflow.com/questions/6128476/how-can-i-get-factory-girl-to-never-hit-the-database-if-i-am-calling-factory-buil Anyway, I was curious what you guys use to create Factories? 1. Do you put up with Factory_girl? 2. Did you configure Factory_girl differently to make it run faster? 3. Do you use something else? May I ask what? Thanks! Ken -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20110525/b27ed8a8/attachment.html>
David Chelimsky
2011-May-25 21:34 UTC
[rspec-users] For those using Rails with RSpec... what do you guys use for creating Factories?
On May 25, 2011, at 2:00 PM, Ken Egervari wrote:> I am using factory_girl, and I have discovered that it is chiefly responsible for making my tests run slow. > > I have posted a question about this on Stack Overflow: > > http://stackoverflow.com/questions/6128476/how-can-i-get-factory-girl-to-never-hit-the-database-if-i-am-calling-factory-buil > > Anyway, I was curious what you guys use to create Factories? > > 1. Do you put up with Factory_girl? > 2. Did you configure Factory_girl differently to make it run faster? > 3. Do you use something else? May I ask what? > > Thanks! > > KenThe factory girl docs on http://rubydoc.info/gems/factory_girl/1.3.3/frames offer a few options. Look for the :default_strategy option and the :factory option on association. HTH, David -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20110525/f6555cf2/attachment.html>
Justin Ko
2011-May-25 21:45 UTC
[rspec-users] For those using Rails with RSpec... what do you guys use for creating Factories?
On Wed, May 25, 2011 at 1:00 PM, Ken Egervari <ken.egervari at gmail.com>wrote:> I am using factory_girl, and I have discovered that it is chiefly > responsible for making my tests run slow. > > I have posted a question about this on Stack Overflow: > > > http://stackoverflow.com/questions/6128476/how-can-i-get-factory-girl-to-never-hit-the-database-if-i-am-calling-factory-buil > > Anyway, I was curious what you guys use to create Factories? > > 1. Do you put up with Factory_girl? > 2. Did you configure Factory_girl differently to make it run faster? > 3. Do you use something else? May I ask what? > > Thanks! > > Ken > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >In Rails land, tests are "slow" because of the database. Sorry, but there is just no way around it. Sure, you could mock everything out, but we all know the implications of doing that... rspec-core has 691 examples and takes 2 seconds to run. That would be amazing if we could have that for our Rails apps. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20110525/1c31e9df/attachment-0001.html>
Sidu Ponnappa
2011-May-25 21:53 UTC
[rspec-users] For those using Rails with RSpec... what do you guys use for creating Factories?
Given the limitation Justin just mentioned, an easy performance win is to switch your testing environment to use an in-memory db. You could also parallelize your specs to use all cores on your machine. Best, Sidu. http://c42.in http://about.me/ponnappa On 26 May 2011 00:30, Ken Egervari <ken.egervari at gmail.com> wrote:> I am using factory_girl, and I have discovered that it is chiefly > responsible for making my tests run slow. > > I have posted a question about this on Stack Overflow: > > http://stackoverflow.com/questions/6128476/how-can-i-get-factory-girl-to-never-hit-the-database-if-i-am-calling-factory-buil > > Anyway, I was curious what you guys use to create Factories? > > 1. Do you put up with Factory_girl? > 2. Did you configure Factory_girl differently to make it run faster? > 3. Do you use something else? May I ask what? > > Thanks! > > Ken > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
Ken Egervari
2011-May-25 22:05 UTC
[rspec-users] For those using Rails with RSpec... what do you guys use for creating Factories?
Well, I''ve been getting rid of my Factory Girl for my controllers at least and I run 280 tests in about 1.5 seconds. To me, this is worth it as before it took 40-50 seconds. This is too much of a speed improvement to ignore I think. I am at a loss as to why Factory_girl takes so many cpu cycles to create a few objects per test. I have benchmarked a simple call to Factory.create(:user), and it takes .14 seconds to run. Even if you use Factory.build() though and it doesn''t hit the database, it is STILL slow. I am at a loss as to how factory_girl cannot come close to the performance of manually instantiating the object yourself. Why is it so much slower? I''d have to look at their source code to know for sure. Honestly though, I have removed factory_girl completely from my controller tests and I am so thrilled. I have not really lost anything in terms of expressiveness. Sure, I have to add a pieces of data to make render_views comply, but it''s not much. It is a *very small* price to pay for this much performance gain. Ken On Wed, May 25, 2011 at 5:45 PM, Justin Ko <jko170 at gmail.com> wrote:> > > On Wed, May 25, 2011 at 1:00 PM, Ken Egervari <ken.egervari at gmail.com>wrote: > >> I am using factory_girl, and I have discovered that it is chiefly >> responsible for making my tests run slow. >> >> I have posted a question about this on Stack Overflow: >> >> >> http://stackoverflow.com/questions/6128476/how-can-i-get-factory-girl-to-never-hit-the-database-if-i-am-calling-factory-buil >> >> Anyway, I was curious what you guys use to create Factories? >> >> 1. Do you put up with Factory_girl? >> 2. Did you configure Factory_girl differently to make it run faster? >> 3. Do you use something else? May I ask what? >> >> Thanks! >> >> Ken >> >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users >> > > > In Rails land, tests are "slow" because of the database. Sorry, but there > is just no way around it. Sure, you could mock everything out, but we all > know the implications of doing that... > > rspec-core has 691 examples and takes 2 seconds to run. That would be > amazing if we could have that for our Rails apps. > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20110525/106e9261/attachment.html>
Ken Egervari
2011-May-25 22:07 UTC
[rspec-users] For those using Rails with RSpec... what do you guys use for creating Factories?
Unfortunately, an in-memory database is both dangerous and doesn''t fix the problem. Factory_girl spits out 50-100 lines of sql junk in the RSpec log PER test. I tried switching to sqlite3 using the ":memory:" option, and it only saved me about 15-20%. That''s not much of an improvement. By getting rid of Factory Girl entirely, I have increased my test performance by 2000%. Ken On Wed, May 25, 2011 at 5:53 PM, Sidu Ponnappa <ckponnappa at gmail.com> wrote:> Given the limitation Justin just mentioned, an easy performance win is > to switch your testing environment to use an in-memory db. You could > also parallelize your specs to use all cores on your machine. > > Best, > Sidu. > http://c42.in > http://about.me/ponnappa > > On 26 May 2011 00:30, Ken Egervari <ken.egervari at gmail.com> wrote: > > I am using factory_girl, and I have discovered that it is chiefly > > responsible for making my tests run slow. > > > > I have posted a question about this on Stack Overflow: > > > > > http://stackoverflow.com/questions/6128476/how-can-i-get-factory-girl-to-never-hit-the-database-if-i-am-calling-factory-buil > > > > Anyway, I was curious what you guys use to create Factories? > > > > 1. Do you put up with Factory_girl? > > 2. Did you configure Factory_girl differently to make it run faster? > > 3. Do you use something else? May I ask what? > > > > Thanks! > > > > Ken > > > > _______________________________________________ > > rspec-users mailing list > > rspec-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/rspec-users > > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20110525/554615ba/attachment.html>
Justin Ko
2011-May-25 22:37 UTC
[rspec-users] For those using Rails with RSpec... what do you guys use for creating Factories?
On Wed, May 25, 2011 at 4:05 PM, Ken Egervari <ken.egervari at gmail.com>wrote:> Well, I''ve been getting rid of my Factory Girl for my controllers at least > and I run 280 tests in about 1.5 seconds. To me, this is worth it as before > it took 40-50 seconds. This is too much of a speed improvement to ignore I > think. > > I am at a loss as to why Factory_girl takes so many cpu cycles to create a > few objects per test. I have benchmarked a simple call to > Factory.create(:user), and it takes .14 seconds to run. > > Even if you use Factory.build() though and it doesn''t hit the database, it > is STILL slow. I am at a loss as to how factory_girl cannot come close to > the performance of manually instantiating the object yourself. Why is it so > much slower? I''d have to look at their source code to know for sure. > > Honestly though, I have removed factory_girl completely from my controller > tests and I am so thrilled. I have not really lost anything in terms of > expressiveness. > > Sure, I have to add a pieces of data to make render_views comply, but it''s > not much. It is a *very small* price to pay for this much performance gain. > > Ken > > > > On Wed, May 25, 2011 at 5:45 PM, Justin Ko <jko170 at gmail.com> wrote: > >> >> >> On Wed, May 25, 2011 at 1:00 PM, Ken Egervari <ken.egervari at gmail.com>wrote: >> >>> I am using factory_girl, and I have discovered that it is chiefly >>> responsible for making my tests run slow. >>> >>> I have posted a question about this on Stack Overflow: >>> >>> >>> http://stackoverflow.com/questions/6128476/how-can-i-get-factory-girl-to-never-hit-the-database-if-i-am-calling-factory-buil >>> >>> Anyway, I was curious what you guys use to create Factories? >>> >>> 1. Do you put up with Factory_girl? >>> 2. Did you configure Factory_girl differently to make it run faster? >>> 3. Do you use something else? May I ask what? >>> >>> Thanks! >>> >>> Ken >>> >>> _______________________________________________ >>> rspec-users mailing list >>> rspec-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/rspec-users >>> >> >> >> In Rails land, tests are "slow" because of the database. Sorry, but there >> is just no way around it. Sure, you could mock everything out, but we all >> know the implications of doing that... >> >> rspec-core has 691 examples and takes 2 seconds to run. That would be >> amazing if we could have that for our Rails apps. >> >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users >> > > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-usersCan you please bottom post? It is easier to follow the conversation. So, you''re right. User.new vs. Factory.build(:user) should not be much difference in speed. Factory.build(:user) will instaniate the user (and its attributes) AND its associations, but should not be hitting the database. I would bring this up with the factory_girl folks. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20110525/8d64cb96/attachment.html>
Ken Egervari
2011-May-26 00:11 UTC
[rspec-users] For those using Rails with RSpec... what do you guys use for creating Factories?
On Wed, May 25, 2011 at 5:34 PM, David Chelimsky <dchelimsky at gmail.com>wrote:> On May 25, 2011, at 2:00 PM, Ken Egervari wrote: > > I am using factory_girl, and I have discovered that it is chiefly > responsible for making my tests run slow. > > I have posted a question about this on Stack Overflow: > > > http://stackoverflow.com/questions/6128476/how-can-i-get-factory-girl-to-never-hit-the-database-if-i-am-calling-factory-buil > > Anyway, I was curious what you guys use to create Factories? > > 1. Do you put up with Factory_girl? > 2. Did you configure Factory_girl differently to make it run faster? > 3. Do you use something else? May I ask what? > > Thanks! > > Ken > > > The factory girl docs on > http://rubydoc.info/gems/factory_girl/1.3.3/frames offer a few options. > Look for the :default_strategy option and the :factory option on > association. > > HTH, > David >Yeah, :default_strategy helps somewhat... but not really. I have done a lot of tests/profiling and even if I resorted to Factory.build() for everything, it ends up being much slower than just making the objects myself. I won''t get rid of factory_girl outright - it''s a good little tool to have when you need to create a mini-database, which happens quite a bit in practice (testing scopes, and so on). But I think a good rule of thumb is not use it like a basic defacto object-instantiation tool. Maybe that''s my bad, but articles, screencasts, etc. actually tell people to do this - and given it''s performance implications, I believe that should NOT be a best practice. For me, I will use normal Rails/Ruby objects until they become too cumbersome to build in order to satisfy validations or when I need to put something in the database because that is part of the test. Otherwise, it''s better to work with pure, transient objects that you create yourself. I have literally got my entire test suite to run in 25 seconds when it used to run in over 90 before. That''s quite a big jump - several times faster than swapping to an in-memory database. I only wish I knew this when I started my Rails app and just following what I believe to be the "true" best practice from the start. I guess I gotta live and learn. I hope others don''t fall into the same pitfall like I did though. Ken -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20110525/9ead0fe6/attachment.html>
Justin Ko
2011-May-26 04:10 UTC
[rspec-users] For those using Rails with RSpec... what do you guys use for creating Factories?
On Wed, May 25, 2011 at 6:11 PM, Ken Egervari <ken.egervari at gmail.com>wrote:> On Wed, May 25, 2011 at 5:34 PM, David Chelimsky <dchelimsky at gmail.com>wrote: > >> On May 25, 2011, at 2:00 PM, Ken Egervari wrote: >> >> I am using factory_girl, and I have discovered that it is chiefly >> responsible for making my tests run slow. >> >> I have posted a question about this on Stack Overflow: >> >> >> http://stackoverflow.com/questions/6128476/how-can-i-get-factory-girl-to-never-hit-the-database-if-i-am-calling-factory-buil >> >> Anyway, I was curious what you guys use to create Factories? >> >> 1. Do you put up with Factory_girl? >> 2. Did you configure Factory_girl differently to make it run faster? >> 3. Do you use something else? May I ask what? >> >> Thanks! >> >> Ken >> >> >> The factory girl docs on >> http://rubydoc.info/gems/factory_girl/1.3.3/frames offer a few options. >> Look for the :default_strategy option and the :factory option on >> association. >> >> HTH, >> David >> > > Yeah, :default_strategy helps somewhat... but not really. I have done a lot > of tests/profiling and even if I resorted to Factory.build() for everything, > it ends up being much slower than just making the objects myself. > > I won''t get rid of factory_girl outright - it''s a good little tool to have > when you need to create a mini-database, which happens quite a bit in > practice (testing scopes, and so on). > > But I think a good rule of thumb is not use it like a basic defacto > object-instantiation tool. Maybe that''s my bad, but articles, screencasts, > etc. actually tell people to do this - and given it''s performance > implications, I believe that should NOT be a best practice. > > For me, I will use normal Rails/Ruby objects until they become too > cumbersome to build in order to satisfy validations or when I need to put > something in the database because that is part of the test. Otherwise, it''s > better to work with pure, transient objects that you create yourself. > > I have literally got my entire test suite to run in 25 seconds when it used > to run in over 90 before. That''s quite a big jump - several times faster > than swapping to an in-memory database. > > I only wish I knew this when I started my Rails app and just following what > I believe to be the "true" best practice from the start. I guess I gotta > live and learn. I hope others don''t fall into the same pitfall like I did > though. > > Ken > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >Again, please bring it up with the factory_girl authors. I''m sure they would like to know there might be some performance issues. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20110525/5d5dd575/attachment.html>
Ken Egervari
2011-May-26 04:32 UTC
[rspec-users] For those using Rails with RSpec... what do you guys use for creating Factories?
On Thu, May 26, 2011 at 12:10 AM, Justin Ko <jko170 at gmail.com> wrote:> > > On Wed, May 25, 2011 at 6:11 PM, Ken Egervari <ken.egervari at gmail.com>wrote: > >> On Wed, May 25, 2011 at 5:34 PM, David Chelimsky <dchelimsky at gmail.com>wrote: >> >>> On May 25, 2011, at 2:00 PM, Ken Egervari wrote: >>> >>> I am using factory_girl, and I have discovered that it is chiefly >>> responsible for making my tests run slow. >>> >>> I have posted a question about this on Stack Overflow: >>> >>> >>> http://stackoverflow.com/questions/6128476/how-can-i-get-factory-girl-to-never-hit-the-database-if-i-am-calling-factory-buil >>> >>> Anyway, I was curious what you guys use to create Factories? >>> >>> 1. Do you put up with Factory_girl? >>> 2. Did you configure Factory_girl differently to make it run faster? >>> 3. Do you use something else? May I ask what? >>> >>> Thanks! >>> >>> Ken >>> >>> >>> The factory girl docs on >>> http://rubydoc.info/gems/factory_girl/1.3.3/frames offer a few options. >>> Look for the :default_strategy option and the :factory option on >>> association. >>> >>> HTH, >>> David >>> >> >> Yeah, :default_strategy helps somewhat... but not really. I have done a >> lot of tests/profiling and even if I resorted to Factory.build() for >> everything, it ends up being much slower than just making the objects >> myself. >> >> I won''t get rid of factory_girl outright - it''s a good little tool to have >> when you need to create a mini-database, which happens quite a bit in >> practice (testing scopes, and so on). >> >> But I think a good rule of thumb is not use it like a basic defacto >> object-instantiation tool. Maybe that''s my bad, but articles, screencasts, >> etc. actually tell people to do this - and given it''s performance >> implications, I believe that should NOT be a best practice. >> >> For me, I will use normal Rails/Ruby objects until they become too >> cumbersome to build in order to satisfy validations or when I need to put >> something in the database because that is part of the test. Otherwise, it''s >> better to work with pure, transient objects that you create yourself. >> >> I have literally got my entire test suite to run in 25 seconds when it >> used to run in over 90 before. That''s quite a big jump - several times >> faster than swapping to an in-memory database. >> >> I only wish I knew this when I started my Rails app and just following >> what I believe to be the "true" best practice from the start. I guess I >> gotta live and learn. I hope others don''t fall into the same pitfall like I >> did though. >> >> Ken >> >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users >> > > Again, please bring it up with the factory_girl authors. I''m sure they > would like to know there might be some performance issues. Thanks. > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >Actually, I did bring it to their attention and just hear a response. It turns out someone pointed this out back in June of 2010. Not a good sign :( They already knew though. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20110526/7c57886d/attachment.html>