So I''ve just started working on a rails project which currently has something like 7500 LOC. All of the tests are written in Test::Unit, although the test coverage is pretty poor: rcov says that 25% of the code is covered, while rake stats shows the code to test ratio as 1:0.1 (800 lines of test code). I guess I''m wondering what would generally be advisable here. Is it worth it to work on (or use) a test:unit => spec translator? Should I just start using rspec when I need to start writing code, and perform regressions when need be? Or is the project too big to even consider using rspec? Thanks in advance for any experienced advice, Scott Taylor
Scott, There was a thread about this earlier... You might be interested in this plugin I wrote. Kinda basic but worked for me. http://www.davidjrice.co.uk/articles/2007/8/12/ruby-on-rails-plugin- test-unit-to-rspec-converter Best, Dave On 18 Aug 2007, at 09:38, Scott Taylor wrote:> > So I''ve just started working on a rails project which currently has > something like 7500 LOC. All of the tests are written in Test::Unit, > although the test coverage is pretty poor: rcov says that 25% of the > code is covered, while rake stats shows the code to test ratio as > 1:0.1 (800 lines of test code). > > I guess I''m wondering what would generally be advisable here. Is it > worth it to work on (or use) a test:unit => spec translator? Should > I just start using rspec when I need to start writing code, and > perform regressions when need be? Or is the project too big to even > consider using rspec? > > Thanks in advance for any experienced advice, > > Scott Taylor > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users--- David Rice http://www.davidjrice.co.uk -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/rspec-users/attachments/20070823/62b7c90e/attachment.html
Hey all, I think I saw a thread about this earlier (maybe it was on the devlopers list) but I am wondering if anyone has stubbed out AR so that the unit test do not hit the database. The reason I am curious is because there was a lightning talk given at the Ruby Hoedown (watch it here: http://rubyhoedown2007.confreaks.com/session06.html) about separating unit tests from the DB entirely and making it a lot faster. I''m not just talking about not using fixtures but this would not even query the DB for columns or do any inserts or updates either. Dan Manges was the presenter and he talks about it on his blog and his newly released gem: http://www.dcmanges.com/blog/rails-unit-record-test-without-the-database I haven''t experimented with the gem yet but I''m sure it could be changed to work with rspec. I''m curious about people''s opinions about this practice and see if anyone has done something similar with rspec already. I think the speed is very attractive but if you were to do this you would need to to have another type of test that would actually hit the DB. Has anyone done this with rspec that could shed a little more light on the advantages/disadvantages? Thanks, Ben
On Aug 23, 2007, at 12:18 PM, David Rice wrote:> Scott, > > There was a thread about this earlier... You might be interested in > this plugin I wrote. Kinda basic but worked for me. > > http://www.davidjrice.co.uk/articles/2007/8/12/ruby-on-rails-plugin- > test-unit-to-rspec-converterYeah - actually I saw it, and tried it out. Although I can''t lie - it didn''t work well. I can''t imagine any translator really working well. How would you add mocks + stubs + clear specifications + 1 test-per-spec in a translator, and do it intelligently? But, for what it''s worth - it did what I expected it to do, and would be a good, but crude way, to translate a lot test::unit code to rspec. Thanks for the tool (and have you looked into the old rspec one - test2spec, I believe it was called)? Scott> > Best, > Dave > > > On 18 Aug 2007, at 09:38, Scott Taylor wrote: > >> >> So I''ve just started working on a rails project which currently has >> something like 7500 LOC. All of the tests are written in Test::Unit, >> although the test coverage is pretty poor: rcov says that 25% of the >> code is covered, while rake stats shows the code to test ratio as >> 1:0.1 (800 lines of test code). >> >> I guess I''m wondering what would generally be advisable here. Is it >> worth it to work on (or use) a test:unit => spec translator? Should >> I just start using rspec when I need to start writing code, and >> perform regressions when need be? Or is the project too big to even >> consider using rspec? >> >> Thanks in advance for any experienced advice, >> >> Scott Taylor >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users > > --- > David Rice > http://www.davidjrice.co.uk > > > > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users
On 23 Aug 2007, at 20:23, Scott Taylor wrote:> > On Aug 23, 2007, at 12:18 PM, David Rice wrote: > >> Scott, >> >> There was a thread about this earlier... You might be interested in >> this plugin I wrote. Kinda basic but worked for me. >> >> http://www.davidjrice.co.uk/articles/2007/8/12/ruby-on-rails-plugin- >> test-unit-to-rspec-converter > > Yeah - actually I saw it, and tried it out. Although I can''t lie - > it didn''t work well. I can''t imagine any translator really working > well. How would you add mocks + stubs + clear specifications + 1 > test-per-spec in a translator, and do it intelligently? >Totally :) Yeah there''s so many different syntax variants that it becomes hard across testing styles. Especially now that all of my test suites are converted over to RSpec... I really don''t think the complexity is worth it because for the people that use the tool it instantly becomes useless after they are "converted".> But, for what it''s worth - it did what I expected it to do, and would > be a good, but crude way, to translate a lot test::unit code to rspec. > > Thanks for the tool (and have you looked into the old rspec one - > test2spec, I believe it was called)? >No problem! Yeah the original was very interesting to look at once I realised it did exist and wasn''t a dream ( I went on a fruitless hunt before rolling my own ).> Scott > > >> >> Best, >> Dave >> >> >> On 18 Aug 2007, at 09:38, Scott Taylor wrote: >> >>> >>> So I''ve just started working on a rails project which currently has >>> something like 7500 LOC. All of the tests are written in >>> Test::Unit, >>> although the test coverage is pretty poor: rcov says that 25% of the >>> code is covered, while rake stats shows the code to test ratio as >>> 1:0.1 (800 lines of test code). >>> >>> I guess I''m wondering what would generally be advisable here. Is it >>> worth it to work on (or use) a test:unit => spec translator? Should >>> I just start using rspec when I need to start writing code, and >>> perform regressions when need be? Or is the project too big to even >>> consider using rspec? >>> >>> Thanks in advance for any experienced advice, >>> >>> Scott Taylor >>> _______________________________________________ >>> rspec-users mailing list >>> rspec-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/rspec-users >> >> --- >> David Rice >> http://www.davidjrice.co.uk >> >> >> >> >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users--- David Rice +44 (0)78 708 12996 http://www.davidjrice.co.uk -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/rspec-users/attachments/20070823/f907eef5/attachment-0001.html
David Chelimsky
2007-Aug-23 21:43 UTC
[rspec-users] Keeping unit tests from hitting the DB
On 8/23/07, Ben Mabey <ben at benmabey.com> wrote:> Hey all, > I think I saw a thread about this earlier (maybe it was on the devlopers > list) but I am wondering if anyone has stubbed out AR so that the unit > test do not hit the database. The reason I am curious is because there > was a lightning talk given at the Ruby Hoedown (watch it here: > http://rubyhoedown2007.confreaks.com/session06.html) about separating > unit tests from the DB entirely and making it a lot faster. I''m not > just talking about not using fixtures but this would not even query the > DB for columns or do any inserts or updates either. Dan Manges was the > presenter and he talks about it on his blog and his newly released gem: > http://www.dcmanges.com/blog/rails-unit-record-test-without-the-database > > I haven''t experimented with the gem yet but I''m sure it could be changed > to work with rspec. I''m curious about people''s opinions about this > practice and see if anyone has done something similar with rspec > already.I''ve heard of ppl experimenting w/ this but haven''t seen it in action. I would LOVE to see this capability but I''d like to be able to control it from behaviour to behaviour. For example, I tend to mock all models in view, controller and helper specs, but not in model specs, where I tend to let them interact w/ the DB. I''m not recommending that approach over mocking models in model specs, just saying that it''s my personal preference. There are very good arguments to do it differently. That said, I''d want to be able to do this in spec_helper.rb: Spec::Runner.configure do |config| config.deny_db_access :view, :controller, :helper end or _something_ like that. Then if a controller spec tries to hit the db you should get an error. Feel like contributing that?> I think the speed is very attractive but if you were to do > this you would need to to have another type of test that would actually > hit the DB. Has anyone done this with rspec that could shed a little > more light on the advantages/disadvantages? > > Thanks, > Ben > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
David Chelimsky wrote:> On 8/23/07, Ben Mabey <ben at benmabey.com> wrote: > >> Hey all, >> I think I saw a thread about this earlier (maybe it was on the devlopers >> list) but I am wondering if anyone has stubbed out AR so that the unit >> test do not hit the database. The reason I am curious is because there >> was a lightning talk given at the Ruby Hoedown (watch it here: >> http://rubyhoedown2007.confreaks.com/session06.html) about separating >> unit tests from the DB entirely and making it a lot faster. I''m not >> just talking about not using fixtures but this would not even query the >> DB for columns or do any inserts or updates either. Dan Manges was the >> presenter and he talks about it on his blog and his newly released gem: >> http://www.dcmanges.com/blog/rails-unit-record-test-without-the-database >> >> I haven''t experimented with the gem yet but I''m sure it could be changed >> to work with rspec. I''m curious about people''s opinions about this >> practice and see if anyone has done something similar with rspec >> already. >> > > I''ve heard of ppl experimenting w/ this but haven''t seen it in action. > I would LOVE to see this capability but I''d like to be able to control > it from behaviour to behaviour. For example, I tend to mock all models > in view, controller and helper specs, but not in model specs, where I > tend to let them interact w/ the DB. I''m not recommending that > approach over mocking models in model specs, just saying that it''s my > personal preference. There are very good arguments to do it > differently. > > That said, I''d want to be able to do this in spec_helper.rb: > > Spec::Runner.configure do |config| > config.deny_db_access :view, :controller, :helper > end > > or _something_ like that. Then if a controller spec tries to hit the > db you should get an error. > > Feel like contributing that? > >So would the "config.deny_db_access :view, :controller, :helper" line just throw an exception upon any DB access thereby acting as an alert saying that your using the DB instead of mocking appropriately? Is that right? Or should that line automatically stub the AR methods like new, save, create, etc to prevent any DB activity but give the illusion of talking to the DB? The latter is is what I was thinking for the model specs, although the alert exception would also be a good idea- especially for the controllers, views, and helpers. I have yet to dive into the rspec internals but I would definitely like to contribute. Let me know if the alert idea is what you meant. Thanks, Ben
On 8/23/07, Ben Mabey <ben at benmabey.com> wrote:> David Chelimsky wrote: > > On 8/23/07, Ben Mabey <ben at benmabey.com> wrote: > > > >> Hey all, > >> I think I saw a thread about this earlier (maybe it was on the devlopers > >> list) but I am wondering if anyone has stubbed out AR so that the unit > >> test do not hit the database. The reason I am curious is because there > >> was a lightning talk given at the Ruby Hoedown (watch it here: > >> http://rubyhoedown2007.confreaks.com/session06.html) about separating > >> unit tests from the DB entirely and making it a lot faster. I''m not > >> just talking about not using fixtures but this would not even query the > >> DB for columns or do any inserts or updates either. Dan Manges was the > >> presenter and he talks about it on his blog and his newly released gem: > >> http://www.dcmanges.com/blog/rails-unit-record-test-without-the-database > >> > >> I haven''t experimented with the gem yet but I''m sure it could be changed > >> to work with rspec. I''m curious about people''s opinions about this > >> practice and see if anyone has done something similar with rspec > >> already. > >> > > > > I''ve heard of ppl experimenting w/ this but haven''t seen it in action. > > I would LOVE to see this capability but I''d like to be able to control > > it from behaviour to behaviour. For example, I tend to mock all models > > in view, controller and helper specs, but not in model specs, where I > > tend to let them interact w/ the DB. I''m not recommending that > > approach over mocking models in model specs, just saying that it''s my > > personal preference. There are very good arguments to do it > > differently. > > > > That said, I''d want to be able to do this in spec_helper.rb: > > > > Spec::Runner.configure do |config| > > config.deny_db_access :view, :controller, :helper > > end > > > > or _something_ like that. Then if a controller spec tries to hit the > > db you should get an error. > > > > Feel like contributing that? > > > > > So would the "config.deny_db_access :view, :controller, :helper" line just throw an exception upon any DB access thereby acting as an alert saying that your using the DB instead of mocking appropriately? Is that right? Or should that line automatically stub the AR methods like new, save, create, etc to prevent any DB activity but give the illusion of talking to the DB? The latter is is what I was thinking for the model specs, although the alert exception would also be a good idea- especially for the controllers, views, and helpers. I have yet to dive into the rspec internals but I would definitely like to contribute. Let me know if the alert idea is what you meant. Thanks, > Ben > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >It should, imo, throw an exception when the DB is touched. Jay Fields posted about something like this a while ago: http://blog.jayfields.com/2006/06/ruby-on-rails-unit-tests.html As far as model specs go...I''ve been toying with the idea of splitting up non-db-reliant behavior specs from the specs that do require the database. When you want to specify behavior you can just stub out the columns (because AR does try to hit the db to find the column info). Obviously for things where you need to hit the db, such as custom finders and such, you should use a real db. Kent Beck proposed writing a mock db interface that expects SQL queries but I thoroughly hate that idea. Pat
>>>> > As far as model specs go...I''ve been toying with the idea of splitting > up non-db-reliant behavior specs from the specs that do require the > database. When you want to specify behavior you can just stub out the > columns (because AR does try to hit the db to find the column info). > Obviously for things where you need to hit the db, such as custom > finders and such, you should use a real db. Kent Beck proposed > writing a mock db interface that expects SQL queries but I thoroughly > hate that idea. >I''m wondering why custom finders need to hit the database? Mocking out the columns method (as suggested by the articles) should work just as well for custom finders as it would for AR finders. Am I missing something? Jim> Pat > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users
On 8/23/07, Jim Deville <james.deville at gmail.com> wrote:> >>>> > > As far as model specs go...I''ve been toying with the idea of splitting > > up non-db-reliant behavior specs from the specs that do require the > > database. When you want to specify behavior you can just stub out the > > columns (because AR does try to hit the db to find the column info). > > Obviously for things where you need to hit the db, such as custom > > finders and such, you should use a real db. Kent Beck proposed > > writing a mock db interface that expects SQL queries but I thoroughly > > hate that idea. > > > > I''m wondering why custom finders need to hit the database? Mocking > out the columns method (as suggested by the articles) should work > just as well for custom finders as it would for AR finders. Am I > missing something? > > JimIf I''ve got something like def find_recent(limit = 5) find :all, :order => "created_at DESC", :limit => limit end I think it''s better to just create 6 records and make sure it returns the proper 5. Then write another spec that only gets 3. I suppose you could use a mock db that ensures that the SQL is valid...but that seems brittle. Also at some point you have to ensure that the generated SQL actually works (by running it). Might as well do that in the spec itself. Also that''s a better way because the behavior you want in this case is for it to find the proper records, not to generate some particular SQL string. Pat
On Aug 23, 2007, at 9:48 PM, Pat Maddox wrote:> On 8/23/07, Jim Deville <james.deville at gmail.com> wrote: >>>>>> >>> As far as model specs go...I''ve been toying with the idea of >>> splitting >>> up non-db-reliant behavior specs from the specs that do require the >>> database. When you want to specify behavior you can just stub >>> out the >>> columns (because AR does try to hit the db to find the column info). >>> Obviously for things where you need to hit the db, such as custom >>> finders and such, you should use a real db. Kent Beck proposed >>> writing a mock db interface that expects SQL queries but I >>> thoroughly >>> hate that idea. >>> >> >> I''m wondering why custom finders need to hit the database? Mocking >> out the columns method (as suggested by the articles) should work >> just as well for custom finders as it would for AR finders. Am I >> missing something? >> >> Jim > > If I''ve got something like > > def find_recent(limit = 5) > find :all, :order => "created_at DESC", :limit => limit > end > > I think it''s better to just create 6 records and make sure it returns > the proper 5. Then write another spec that only gets 3. >I understand this, but overriding the Columns method, as suggested by Fields and Manges, should take care of this without hitting the DB. It''s not checking the validity of the SQL, its using the columns method to return your result from fixtures without hitting the DB.> I suppose you could use a mock db that ensures that the SQL is > valid...but that seems brittle. Also at some point you have to ensure > that the generated SQL actually works (by running it). Might as well > do that in the spec itself. Also that''s a better way because theAgreed, in some cases. In this case, testing the proper SQL feels like testing AR''s implementation. If this was a find_by_sql method, I would completely agree with you here.> behavior you want in this case is for it to find the proper records, > not to generate some particular SQL string. > > Pat > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-usersJim
David Chelimsky
2007-Aug-24 16:58 UTC
[rspec-users] Keeping unit tests from hitting the DB
On 8/24/07, Jim Deville <james.deville at gmail.com> wrote:> > On Aug 23, 2007, at 9:48 PM, Pat Maddox wrote: > > > On 8/23/07, Jim Deville <james.deville at gmail.com> wrote: > >>>>>> > >>> As far as model specs go...I''ve been toying with the idea of > >>> splitting > >>> up non-db-reliant behavior specs from the specs that do require the > >>> database. When you want to specify behavior you can just stub > >>> out the > >>> columns (because AR does try to hit the db to find the column info). > >>> Obviously for things where you need to hit the db, such as custom > >>> finders and such, you should use a real db. Kent Beck proposed > >>> writing a mock db interface that expects SQL queries but I > >>> thoroughly > >>> hate that idea. > >>> > >> > >> I''m wondering why custom finders need to hit the database? Mocking > >> out the columns method (as suggested by the articles) should work > >> just as well for custom finders as it would for AR finders. Am I > >> missing something? > >> > >> Jim > > > > If I''ve got something like > > > > def find_recent(limit = 5) > > find :all, :order => "created_at DESC", :limit => limit > > end > > > > I think it''s better to just create 6 records and make sure it returns > > the proper 5. Then write another spec that only gets 3. > > > > I understand this, but overriding the Columns method, as suggested by > Fields and Manges, should take care of this without hitting the DB. > It''s not checking the validity of the SQL, its using the columns > method to return your result from fixtures without hitting the DB. > > > > I suppose you could use a mock db that ensures that the SQL is > > valid...but that seems brittle. Also at some point you have to ensure > > that the generated SQL actually works (by running it). Might as well > > do that in the spec itself. Also that''s a better way because the > > Agreed, in some cases. In this case, testing the proper SQL feels > like testing AR''s implementation. If this was a find_by_sql method, I > would completely agree with you here. > > > behavior you want in this case is for it to find the proper records, > > not to generate some particular SQL string.Well - there are two schools of thought on this. At a system test level, there is no argument. However, at the object level, since this object is interacting with the database, that''s its behaviour. If you accept that, then generating the correct SQL string is no different than any other mock expectation. FYI - I tried using the unit_record gem and there are some changes required in rspec to make it work, but they are trivial and it works great. The only trick is that the prevention of DB access is global per process, so you''d have to separate examples that hit the DB from those that don''t into two separate suites. I''ll explore this possibility and follow up. Cheers, David> > > > Pat > > _______________________________________________ > > rspec-users mailing list > > rspec-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/rspec-users > > Jim > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
On 8/24/07, Jim Deville <james.deville at gmail.com> wrote:> > On Aug 23, 2007, at 9:48 PM, Pat Maddox wrote: > > > On 8/23/07, Jim Deville <james.deville at gmail.com> wrote: > >>>>>> > >>> As far as model specs go...I''ve been toying with the idea of > >>> splitting > >>> up non-db-reliant behavior specs from the specs that do require the > >>> database. When you want to specify behavior you can just stub > >>> out the > >>> columns (because AR does try to hit the db to find the column info). > >>> Obviously for things where you need to hit the db, such as custom > >>> finders and such, you should use a real db. Kent Beck proposed > >>> writing a mock db interface that expects SQL queries but I > >>> thoroughly > >>> hate that idea. > >>> > >> > >> I''m wondering why custom finders need to hit the database? Mocking > >> out the columns method (as suggested by the articles) should work > >> just as well for custom finders as it would for AR finders. Am I > >> missing something? > >> > >> Jim > > > > If I''ve got something like > > > > def find_recent(limit = 5) > > find :all, :order => "created_at DESC", :limit => limit > > end > > > > I think it''s better to just create 6 records and make sure it returns > > the proper 5. Then write another spec that only gets 3. > > > > I understand this, but overriding the Columns method, as suggested by > Fields and Manges, should take care of this without hitting the DB. > It''s not checking the validity of the SQL, its using the columns > method to return your result from fixtures without hitting the DB.I don''t follow. How does it (and I''m not sure what it is, tbh) know how to return the proper fixtures in the proper order? You need an SQL engine to make it happen. Pat