As I work with Rails TestUnit tests I am reconsidering how to use cucumber features. It seems to me that it might be best to have a coherent view of how to arrange my test suites before I get much further into this. Now, so far I have considered three possibilities: 1. Use features exclusively. Create a feature file for each model, one for each controller and possibly an additional one for those views that need separate tests. Name then with the nomenclature x_model.feature, x_model_controller.feature and where desired, x_view.feature 2. Use features exclusively. Create one feature file for each non-decomposable piece of system functionality. Test models, controllers and views within each feature file. 3. Use features and RSpec. Well, is that not what I am doing with features alone? Or, are their cases when testing with RSpec spec files are a better choice than a features scenario? 4. Use features and TestUnit. Naaahhh... 5. Some other way of which I have not considered. In the interest of enlightenment and from a desire to avoid unnecessary back tracking as my project develops, what do experienced practitioners suggest as the favoured way of arranging test suites. Yes, I have done the goggle thing but I really have not found anything terribly useful about organizing tests specifically for Rails. -- Posted via http://www.ruby-forum.com/.
James Byrne wrote:> into this. Now, so far I have considered three possibilities:Ok, five... -- Posted via http://www.ruby-forum.com/.
Its tempting to see features as a kind of extended test tool, I certainly looked at them in that way initially. However I think this isn''t really the most profitable way to look at them, nor how they were designed. It would probably be worthwhile if you looked up some articles on BDD, and perhaps pondered a little further on the ideas of customer developer communication, and of features driving development. Also have a look at the declarative vs imperative feature stuff that has appreared in this list and in Ben Mabey''s articles. Once you do this you might agree that features are not an appropriate tool for doing low-level testing, like testing individual models and controllers. They certainly can reduce the need to do some tests in the normal rails/rspec testing stack e.g. view testing but there much more a design/communication tool than a testing tool In particular I don''t see that driving unit tests with features as being particularly productive because 1) Customers don''t need to know the details of unit tests (they''re is to much detail for them to comprehend). In fact writing unit tests as features actually pollutes the feature space meaning that you end up having to try and seperate one kind of feature from another 2) Programmers should be able to understand unit tests (they don''t need the story format) 3) Unit tests are by nature very detailed and the step matching step writing mechanism is pretty inefficient at expressing this, even with FIT tables 4) Features run slowly in comparison to well written test suites, so running autospec with them gets unproductive fairly quickly. This will be made worse if you create lots of extra features. To summarize see features as things that drive development and bridge the customer developer divide, then once you start developing something use unit tests to ensure the intimate details of your models and to help with regression etc.. Then have a ponder about what you need in the controller, view testing area. This should work fairly well in Rails if you use ''fat model skinny controller''. There''s definitely the need for a really insightful blog article(s) into the detail of moving through this process of feature to implementation via unit test etc.. I get the feeling there''s quite alot of personal style involved and certainly haven''t worked out how to do it myself yet Anyhow HTH Andrew 2008/11/21 James Byrne <lists at ruby-forum.com>:> James Byrne wrote: > >> into this. Now, so far I have considered three possibilities: > > Ok, five... > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
On 21 Nov 2008, at 18:15, James Byrne wrote:> As I work with Rails TestUnit tests I am reconsidering how to use > cucumber features. It seems to me that it might be best to have a > coherent view of how to arrange my test suites before I get much > further > into this. Now, so far I have considered three possibilities: > > 1. Use features exclusively. Create a feature file for each model, one > for each controller and possibly an additional one for those views > that > need separate tests. Name then with the nomenclature x_model.feature, > x_model_controller.feature and where desired, x_view.feature > > 2. Use features exclusively. Create one feature file for each > non-decomposable piece of system functionality. Test models, > controllers and views within each feature file. > > 3. Use features and RSpec. Well, is that not what I am doing with > features alone? Or, are their cases when testing with RSpec spec > files > are a better choice than a features scenario? > > 4. Use features and TestUnit. Naaahhh... > > 5. Some other way of which I have not considered. > > In the interest of enlightenment and from a desire to avoid > unnecessary > back tracking as my project develops, what do experienced > practitioners > suggest as the favoured way of arranging test suites. > > Yes, I have done the goggle thing but I really have not found anything > terribly useful about organizing tests specifically for Rails.No doubt it''s (3) for me. It''s all about working from the outside in. Cucumber is for Acceptance Testing, and those acceptance tests should not be cluttered up with edge cases. They should ideally be readable by your stakeholders / customers. Equally, when you hit something complicated in a class that''s likely to be pulled in different directions by different features, it''s responsible to start writing unit tests for it. Whether you choose to use RSpec or Test::Unit for those unit tests is really up to you. Those unit tests can then be used by the next programmer who comes along to change that class to satisfy another feature as well as yours. I am absolutely loving doing BDD from the outside in. Driving early changes from Cucumber acceptance tests means you have so much more space within which to refactor (not being hemmed in my unit tests / mocks) as the design emerges during those first few iterations. It''s been nothing short of a revelation for me. cheers, Matt
Matt Wynne <matt at mattwynne.net> writes:> On 21 Nov 2008, at 18:15, James Byrne wrote: > >> As I work with Rails TestUnit tests I am reconsidering how to use >> cucumber features. It seems to me that it might be best to have a >> coherent view of how to arrange my test suites before I get much >> further >> into this. Now, so far I have considered three possibilities: >> >> 1. Use features exclusively. Create a feature file for each model, one >> for each controller and possibly an additional one for those views >> that >> need separate tests. Name then with the nomenclature x_model.feature, >> x_model_controller.feature and where desired, x_view.feature >> >> 2. Use features exclusively. Create one feature file for each >> non-decomposable piece of system functionality. Test models, >> controllers and views within each feature file. >> >> 3. Use features and RSpec. Well, is that not what I am doing with >> features alone? Or, are their cases when testing with RSpec spec >> files >> are a better choice than a features scenario? >> >> 4. Use features and TestUnit. Naaahhh... >> >> 5. Some other way of which I have not considered. >> >> In the interest of enlightenment and from a desire to avoid >> unnecessary >> back tracking as my project develops, what do experienced >> practitioners >> suggest as the favoured way of arranging test suites. >> >> Yes, I have done the goggle thing but I really have not found anything >> terribly useful about organizing tests specifically for Rails. > > No doubt it''s (3) for me. It''s all about working from the outside in.Agreed.> Cucumber is for Acceptance Testing, and those acceptance tests should > not be cluttered up with edge cases. They should ideally be readable > by your stakeholders / customers.I disagree with the part about edge cases. Acceptance Tests are about defining and verifying business value, and edge cases are supremely valuable to businesses. What happens when an ATM user tries to withdraw $1 more than he has available in his account? Lately I''ve been putting more and more stuff into ATs. I''m finding it valuable to keep tests for domain logic separate from plain ol unit tests...meaning that my Account object may be tested mostly with Cucumber, but helper objects such as a stats aggregator will be spec''d with code-level examples. Pat
On 24 Nov 2008, at 18:29, Pat Maddox wrote:> Matt Wynne <matt at mattwynne.net> writes: > >> On 21 Nov 2008, at 18:15, James Byrne wrote: >> >>> As I work with Rails TestUnit tests I am reconsidering how to use >>> cucumber features. It seems to me that it might be best to have a >>> coherent view of how to arrange my test suites before I get much >>> further >>> into this. Now, so far I have considered three possibilities: >>> >>> 1. Use features exclusively. Create a feature file for each model, >>> one >>> for each controller and possibly an additional one for those views >>> that >>> need separate tests. Name then with the nomenclature >>> x_model.feature, >>> x_model_controller.feature and where desired, x_view.feature >>> >>> 2. Use features exclusively. Create one feature file for each >>> non-decomposable piece of system functionality. Test models, >>> controllers and views within each feature file. >>> >>> 3. Use features and RSpec. Well, is that not what I am doing with >>> features alone? Or, are their cases when testing with RSpec spec >>> files >>> are a better choice than a features scenario? >>> >>> 4. Use features and TestUnit. Naaahhh... >>> >>> 5. Some other way of which I have not considered. >>> >>> In the interest of enlightenment and from a desire to avoid >>> unnecessary >>> back tracking as my project develops, what do experienced >>> practitioners >>> suggest as the favoured way of arranging test suites. >>> >>> Yes, I have done the goggle thing but I really have not found >>> anything >>> terribly useful about organizing tests specifically for Rails. >> >> No doubt it''s (3) for me. It''s all about working from the outside in. > > Agreed. > > >> Cucumber is for Acceptance Testing, and those acceptance tests should >> not be cluttered up with edge cases. They should ideally be readable >> by your stakeholders / customers. > > I disagree with the part about edge cases. Acceptance Tests are about > defining and verifying business value, and edge cases are supremely > valuable to businesses. What happens when an ATM user tries to > withdraw > $1 more than he has available in his account?> > Lately I''ve been putting more and more stuff into ATs. I''m finding it > valuable to keep tests for domain logic separate from plain ol unit > tests...meaning that my Account object may be tested mostly with > Cucumber, but helper objects such as a stats aggregator will be spec''d > with code-level examples.Yep. Agreed, you put it better than I could. I am freaking people out at work because I seem to have practically stopped writing unit tests, but I''m still certain that I don''t lift a finger without some automated test to force me into it. I''m just finding that ATs give me a much bigger lever in terms of how much code they drive out with a single test. I have a feeling that people who''ve developed with FIT / JUnit must have been through this loop before. Any war stories out there? cheers, Matt
I''m working on a Rails project with someone else, and getting different results on specs. After a lot of poking around for differences, the only thing I can find is autospec in /opt/local/bin/ autospec vs. /usr/bin/autospec Seems like a big clue, but who actually installs this - rspec, ZenTest? I can''t seem to google up any clues as to what actually creates this, or how it might be different on different machines. We both have the same version of rspec, rspec-rails, zen-test, etc. and both on the same version of OS X, both on the same project cloned out of git, same ruby out of port, etc.. So it''s not obvious why our specs aren''t running the same. This is the only difference I can find - any theories of how it could be different or what that might imply are welcome. Thanks, Steve
On Mon, Nov 24, 2008 at 10:29 AM, Pat Maddox <pergesu at gmail.com> wrote:> > I disagree with the part about edge cases. Acceptance Tests are about > defining and verifying business value, and edge cases are supremely > valuable to businesses. What happens when an ATM user tries to withdraw > $1 more than he has available in his account? >Withdrawing $1 more than available is an edge case, vulnerable to an off-by-one error, and doesn''t need to be shown to business, in my opinion. Testing that is a means of detecting errors - it''s not a specification. What happens when an ATM user tries to withdraw $100 more than available is not an edge case, and should be shown to business. I realize it''s a fine point - I''m just responding to whether the business needs to see what we call "edge cases." ///ark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20081124/4b24fd86/attachment.html>
Hi guys, I''m having trouble figuring out where the line between writing a spec or a feature is. Since I started with rspec stories, I have the idea that stories where just the evolution of specs. My main reason for this was the re-usability of steps throughout stories, which I think is great. On contraire, at our company, we have lots of very similar specs, that check very similar things over and over, which doesn''t'' seem to bother anyone too much, apart from myself. They are simply not DRY. I have the idea that using steps and resources we can come up with a very coherent test framework, that will simplify the most common tests, and allow us to concentrate on the really specific things of our project. I can vouch this idea with a very helpful number of steps (which I''m still polishing with the help of my colleagues and would love to share and extend) Which either check things in the database or in the views, in a highly reusable manner, thus with 2 simple steps and a non-complex regexp, you can basically substitute hundreds of lines of specs that do the same job. Now that I''m at it, let me point out controller and view specs, which if I''m still up to date with the list, have become less important to test with specs, either because we are writing more restful applications, or because Then steps and view specs cover the same areas of testing. Now the ones that are left to discuss further are the model specs, or unit tests, which is where I see the most repeated test code, and believe it could be dryed up by using the concept behinds steps. Last but not least, I don''t see why we should follow the MVC pattern of rails to structure our test code. It feels wrong to have to move all those things around when you refactor (I know this is not a problem in static languages, but still, there is a clear coupling between implementation code and test code which I''m concerned with) Looking forward to your comments and suggestions. Kind Regards, Raimond Garcia -- Posted via http://www.ruby-forum.com/.
On Mon, Nov 24, 2008 at 2:18 PM, Mark Wilden <mark at mwilden.com> wrote:> On Mon, Nov 24, 2008 at 10:29 AM, Pat Maddox <pergesu at gmail.com> wrote: >> >> I disagree with the part about edge cases. Acceptance Tests are about >> defining and verifying business value, and edge cases are supremely >> valuable to businesses. What happens when an ATM user tries to withdraw >> $1 more than he has available in his account? > > Withdrawing $1 more than available is an edge case, vulnerable to an > off-by-one error, and doesn''t need to be shown to business, in my opinion. > Testing that is a means of detecting errors - it''s not a specification. > > What happens when an ATM user tries to withdraw $100 more than available is > not an edge case, and should be shown to business. > > I realize it''s a fine point - I''m just responding to whether the business > needs to see what we call "edge cases."If the customer feels the scenario is important then it will be a scenario. I would not rely on that scenario for exhaustive coverage that withdrawing money from the account worked in all cases. Nor would I go write more scenarios to cover withdrawing $1, $2, $100, etc. T This is where the value of object-level code examples come into play. Features/scenarios are intended to provide clear communication for the behaviour of the system and the value it provides. They are not intended to be exhaustive. Object level code examples (the specs) are intended to be exhaustive. This is where we affect the design and maintainability of the objects in our code. This is where we as developers are disciplined to make sure that we have good examples and regression against withdrawing too much money from an account, and not worrying about other objects in the system unrelated to the behaviour of an Account. Object level code examples and scenarios combined give us a huge amount of confidence in refactoring the system. One without the other drastically reduces this confidence (at least for me). They work best together. I don''t see it as a either-or proposition. Well, the only time I see it as either-or is in the case of things that end up like controllers. On a thread not to long ago Pat Maddox mentioned that he stopped writing a lot of controller specs, and he let the scenarios cover them. I agree with Pat, and I have since started doing this where my controller doesn''t have logic that is not tested outside of a scenario. This requires discipline to not let things creep into your controller actions or filters, and to ensure your controller is only used to wire together incoming requests with outgoing responses. In this case, writing object level code examples is largely unneeded IMO. However, as soon as some kind of logic is introduced that isn''t adequately covered by a scenario, I''m still writing controller specs. my 2 cents, -- Zach Dennis http://www.continuousthinking.com http://www.mutuallyhuman.com
On Nov 24, 2008, at 2:03 PM, Steven Rogers wrote:> > I''m working on a Rails project with someone else, and getting > different results on specs. After a lot of poking around for > differences, the only thing I can find is autospec in /opt/local/bin/ > autospec vs. /usr/bin/autospec Seems like a big clue, but who > actually installs this - rspec, ZenTest? I can''t seem to google up > any clues as to what actually creates this, or how it might be > different on different machines. We both have the same version of > rspec, rspec-rails, zen-test, etc. and both on the same version of > OS X, both on the same project cloned out of git, same ruby out of > port, etc.. So it''s not obvious why our specs aren''t running the > same. This is the only difference I can find - any theories of how > it could be different or what that might imply are welcome.What versions of rspec & zentest are the two of you running? Does one of you have --reverse in spec.opts? Scott
On Nov 24, 2008, at 2:05 PM, Scott Taylor wrote:> On Nov 24, 2008, at 2:03 PM, Steven Rogers wrote: > >> I''m working on a Rails project with someone else, and getting >> different results on specs. After a lot of poking around for >> differences, the only thing I can find is autospec in /opt/local/ >> bin/autospec vs. /usr/bin/autospec . . . . This is the only >> difference I can find - any theories of how it could be different >> or what that might imply are welcome. > > What versions of rspec & zentest are the two of you running? > > Does one of you have --reverse in spec.opts?rspec/rspec-rails 1.1.11, ZenTest 3.11.0 we both have --reverse in spec.opts SR
On Mon, Nov 24, 2008 at 6:18 PM, Steven Rogers <srogers1 at gmail.com> wrote:> > On Nov 24, 2008, at 2:05 PM, Scott Taylor wrote: > >> On Nov 24, 2008, at 2:03 PM, Steven Rogers wrote: >> >>> I''m working on a Rails project with someone else, and getting different >>> results on specs. After a lot of poking around for differences, the only >>> thing I can find is autospec in /opt/local/bin/autospec vs. >>> /usr/bin/autospec . . . . This is the only difference I can find - any >>> theories of how it could be different or what that might imply are welcome. >> >> What versions of rspec & zentest are the two of you running? >> >> Does one of you have --reverse in spec.opts? > > rspec/rspec-rails 1.1.11, ZenTest 3.11.0 > > we both have --reverse in spec.opts >both have mtime as loadby order? Also, will be nice to know what kind of differences you''re getting. Just output or errors? -- Luis Lavena AREA 17 - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams
"Mark Wilden" <mark at mwilden.com> writes:> On Mon, Nov 24, 2008 at 10:29 AM, Pat Maddox <pergesu at gmail.com> wrote: > > I disagree with the part about edge cases. Acceptance Tests are about > defining and verifying business value, and edge cases are supremely > valuable to businesses. What happens when an ATM user tries to withdraw > $1 more than he has available in his account? > > Withdrawing $1 more than available is an edge case, vulnerable to an off-by-one error, and doesn''t need to be shown to business, in my > opinion. Testing that is a means of detecting errors - it''s not a specification.I respectfully disagree :) In my hypothetical example, the specification is what to do when someone withdraws more than they have in their account. Then when you write scenarios, the simplest thing you can do to show that is the edge case itself, which is in fact valuable. $1 over is (probably) the same as $100 over, and I think that $100 raises more questions than it answers. Why is it $100? Is there some business rule that charges a penalty if they go $100 over the limit, but not less? Pat
On Nov 24, 2008, at 2:21 PM, Luis Lavena wrote:> On Mon, Nov 24, 2008 at 6:18 PM, Steven Rogers <srogers1 at gmail.com> > wrote: >> >> On Nov 24, 2008, at 2:05 PM, Scott Taylor wrote: >> >>> On Nov 24, 2008, at 2:03 PM, Steven Rogers wrote: >>> >>>> I''m working on a Rails project with someone else, and getting >>>> different >>>> results on specs. After a lot of poking around for differences, >>>> the only >>>> thing I can find is autospec in /opt/local/bin/autospec vs. >>>> /usr/bin/autospec . . . . This is the only difference I can find >>>> - any >>>> theories of how it could be different or what that might imply >>>> are welcome. >>> >>> What versions of rspec & zentest are the two of you running? >>> >>> Does one of you have --reverse in spec.opts? >> >> rspec/rspec-rails 1.1.11, ZenTest 3.11.0 >> >> we both have --reverse in spec.opts >> > > both have mtime as loadby order?Yes - they both have the loadby order because they''re running in unmodified git clones of the same project - so whatever the differences are, they''re in the machine environment, not in flags and files within the Rails project.> Also, will be nice to know what kind of differences you''re getting. > Just output or errors?The differences are errors. Still would like to know where /opt/local/ bin/autospec comes from if anyone knows . . . SR
On Mon, Nov 24, 2008 at 7:23 PM, Steven Rogers <srogers1 at gmail.com> wrote:> ... > > Yes - they both have the loadby order because they''re running in unmodified > git clones of the same project - so whatever the differences are, they''re in > the machine environment, not in flags and files within the Rails project. > >> Also, will be nice to know what kind of differences you''re getting. Just >> output or errors? > > The differences are errors. Still would like to know where > /opt/local/bin/autospec comes from if anyone knows . . . >Is weird that you have autospec installed in two different places, unless there was some forced GEM_PATH and GEM_HOME for it. Both installations are using the bundled ruby version that came with OSX? If you post the errors will be much helpful, is hard to figure out things and assist you without the proper details. -- Luis Lavena AREA 17 - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams
On Mon, Nov 24, 2008 at 1:07 PM, Pat Maddox <pergesu at gmail.com> wrote:> In my hypothetical example, the > specification is what to do when someone withdraws more than they have > in their account. Then when you write scenarios, the simplest thing you > can do to show that is the edge case itself, which is in fact valuable. > $1 over is (probably) the same as $100 over, and I think that $100 > raises more questions than it answers. Why is it $100? Is there some > business rule that charges a penalty if they go $100 over the limit, but > not less? >I was addressing the general idea of presenting what we call "edge cases" to business users. Put it this way: if edge cases were necessary in general, then in addition to the scenario with $1, we''d need one with $0.01 and one with $0. But to respectfully agree with you, if the edge case is actually a business edge case and not just a test for, e.g., an off-by-one error, then I''ll go along with Zach and you and say that $1 is a good scenario. ///ark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20081124/e36db508/attachment.html>
On 25/11/2008, at 7:29 AM, Pat Maddox wrote:> > Lately I''ve been putting more and more stuff into ATs. I''m finding it > valuable to keep tests for domain logic separate from plain ol unit > tests...meaning that my Account object may be tested mostly with > Cucumber, but helper objects such as a stats aggregator will be spec''d > with code-level examples. > > Pat > _______________________________________________Just wondering a bit on this ... the bit about your Account object being tested mostly through Cucumber vs unit tests. In my mind (as I think about this) cucumber is more about testing than rspec is, meaning Acceptance Tests are more about testing where unit tests when using TDD is more about design than testing. Do I make sense? Email is not the best medium for conversations :- ) Now from what I have read of your writings I am assuming that you are using TDD .... and so just from what I read above I am wondering if that statement is true because TDD is driving the implementation of Account, and so you will have a 100% coverage of the Account class behaviour, but when it comes to testing, the AT is doing more of that (meaning testing). Is that it? IIRC, in Kent Beck''s TDD book there is the example of TDD''ing a triangle (or something), but it was something that showed that to TDD the triangle you ended up with less tests than if you were testing the triangle. How''s that sound? Cheers Shane -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2423 bytes Desc: not available URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20081125/a4b5d0d2/attachment-0001.bin>
On Nov 24, 2008, at 3:26 PM, Luis Lavena wrote:> Is weird that you have autospec installed in two different places, > unless there was some forced GEM_PATH and GEM_HOME for it.Yes, exactly - that''s why I''m trying to figure out where among all this code and gems and install procedures that autospec gets created - because I suspect that''s a clue.> Both installations are using the bundled ruby version that came with > OSX?Both are using the same mac ports ruby, but one of them might have been using bundled ruby when ZenTest was installed if that matters.> If you post the errors will be much helpful, is hard to figure out > things and assist you without the proper details.I don''t really expect anyone else to slog through that - I would really just like the one little tidbit "where does autospec come from?" Thanks, SR
Shane Mingins <smingins at elctech.com> writes:> On 25/11/2008, at 7:29 AM, Pat Maddox wrote: >> >> Lately I''ve been putting more and more stuff into ATs. I''m finding it >> valuable to keep tests for domain logic separate from plain ol unit >> tests...meaning that my Account object may be tested mostly with >> Cucumber, but helper objects such as a stats aggregator will be spec''d >> with code-level examples. >> >> Pat >> _______________________________________________ > > Just wondering a bit on this ... the bit about your Account object > being tested mostly through Cucumber vs unit tests. > > In my mind (as I think about this) cucumber is more about testing than > rspec is, meaning Acceptance Tests are more about testing where unit > tests when using TDD is more about design than testing. Do I make > sense? Email is not the best medium for conversations :- ) > > Now from what I have read of your writings I am assuming that you are > using TDD .... and so just from what I read above I am wondering if > that statement is true because TDD is driving the implementation of > Account, and so you will have a 100% coverage of the Account class > behaviour, but when it comes to testing, the AT is doing more of that > (meaning testing). Is that it? IIRC, in Kent Beck''s TDD book there > is the example of TDD''ing a triangle (or something), but it was > something that showed that to TDD the triangle you ended up with less > tests than if you were testing the triangle. > > How''s that sound? > > Cheers > Shane > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-usersHere''s my latest Theory of Testing, in a nutshell: Logic in a system falls into one of two categories, business logic and supporting (infrastructural?) logic. Business logic is often then further subdivided into domain logic and application logic. Domain logic is the logic essential to the domain being used and would remain consistent between applications, whereas app logic is unique to each application. Most domain models I''ve seen used in Rails apps are used *only* in that Rails app, and tend to be simple enough, that this distinction doesn''t matter a whole deal. Well, actually I think Rails devs just don''t think in those terms, but whatever. It''s this business logic that is important to capture in acceptance tests, in my opinion. We can write ATs for domain logic, as with the example I gave for the ATM withdrawal. And we can also write ATs for application logic, which is stuff like showing a success message on a page, or sending off an email, or whatever. I think some of the confusion comes from "edge case" having a pretty particular meaning to developers, but in the example I gave there''s a lot of overlap between an important business scenario and a programmer''s edge case. I''m finding that when that occurs, and I can be confident in my code after writing acceptance tests to cover the business scenarios, I don''t need to write object examples. Then there are a bunch of objects that provide neither domain nor application logic - they''re basically just the result of refactoring to keep your design clean. These are the objects I write examples for. They don''t matter to the customer, but it is important to me that they work and are well-designed. And actually, I''m starting to relax on those a bit as well sometimes...Yehuda advocates a style that is basically JUST acceptance testing (though at various levels, because sometimes the "customer" is a business user and sometimes it''s another programmer using your API). While I don''t go as far as he does, I am less strict about directly testing every single method of every single object than I was just a few months ago. Wow, if that''s it in a nutshell... :) Pat
Steven Rogers <srogers1 at gmail.com> writes:> I don''t really expect anyone else to slog through that - I would > really just like the one little tidbit "where does autospec come > from?"autospec is installed as part of the rspec gem. Pat
On Mon, Nov 24, 2008 at 2:40 PM, Pat Maddox <pergesu at gmail.com> wrote:> Wow, if that''s it in a nutshell... :) >Nice nut. ///ark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20081124/5c62e418/attachment.html>
On Nov 24, 2008, at 4:44 PM, Pat Maddox wrote:> Steven Rogers <srogers1 at gmail.com> writes: > >> I don''t really expect anyone else to slog through that - I would >> really just like the one little tidbit "where does autospec come >> from?" > > autospec is installed as part of the rspec gem.Cool - thanks SR
> Wow, if that''s it in a nutshell... :) > > PatThanks Pat, great summary. I have to admit that I''m as crazy as Yehuda, and believe that all we need are just acceptance tests, at different layers of abstraction, for clients and developers. I also see the benefits of speccing out single object''s behaviors, with the aim of a good design. However, the drawbacks of doing this out-weight the benefits, in my opinion. Testing how every method of an object is going to behave, implies that after refactoring, that spec will no longer be useful, eventhough the business and application logic stay the same. I believe that being able to come up with a good design, is not only dependent on writing tests before your implementation, but also on knowing how to write a good implementation. This can be gained through experience, reading books, blogs, pair-programming, using tools to tell you about the complexity of your code, and a constant process of refactoring as we complete requirements, and then fully understand what the best design could be. Therefore in my opinion, by writing tests that guarantee the correct functioning of the system, we have a robust suite of tests. Let the refactoring come storming in and change the whole implementation, but the tests should not be affected at all, as I''m not testing my implementation nor design, only the correct functioning of the system, and relying on other tools on top of tests to maintain my code nice, clean and understandable by anyone that comes along. Kind Regards, Rai -- Posted via http://www.ruby-forum.com/.
Pat Maddox wrote:> Here''s my latest Theory of Testing, in a nutshell:I really understand what you are getting at. However, as I less experienced developer (my degree is actually in business) I have found that having more unit tests (for models and controllers) helps ensure that I write better code. I can''t think of a single case in which the code I write where every public method is tested is not better than the code I write where I don''t do that. Am I unique in this? Or is strict TDD for every public method a good practice for someone who is still learning how to design code well? Thanks, Paul -- Posted via http://www.ruby-forum.com/.
Pau Cor <lists at ruby-forum.com> writes:> Pat Maddox wrote: >> Here''s my latest Theory of Testing, in a nutshell: > > I really understand what you are getting at. However, as I less > experienced developer (my degree is actually in business) I have found > that having more unit tests (for models and controllers) helps ensure > that I write better code. I can''t think of a single case in which the > code I write where every public method is tested is not better than the > code I write where I don''t do that. > > Am I unique in this? Or is strict TDD for every public method a good > practice for someone who is still learning how to design code well?No, I don''t think you''re unique in that. And yes, I think it''s good practice. In my opinion, you have to extensively practice disciplined TDD before you get a feel for when you can ease up. This is also why I''m more strict with my own code when I''m working with newer people (although the ideal situation would be pairing!) Pat
I came across this idea of dropping unit tests for acceptance tests in the java world. It didn''t like it there and I don''t like it here, but maybe thats because I''m an old fuddy duddy or something :). I do think that every public method of an object should be specifically unit tested, and yes that means that if you refactor your object you should refactor your unit tests. This isn''t really that much of a burden if you design your objects to have simple and minimal public api''s in the first place. What is that makes you think you can refactor code run acceptance tests and be save without unit tests? Writing tests "that guarantee the correct functioning of the system" isn''t something you can just do. Best you can hope for with acceptance tests is that part of the system functions correctly most of the time in some circumstances. Perhaps its the BDD ideal that your only writing the code you need to make your acceptance tests pass, that make you think your acceptance tests cover all your code. However just because you''ve written minimal code to make an acceptance test pass doesn''t mean that you can''t use this code in a multitude of different ways Do you really think that BDD created code is just some black box that you can tinker around with restructure and still be sure it works just because your black box tests still work? I just don''t believe you can get the coverage you need for an application using acceptance testing / features alone. If you do actually write enough features to do this you''ll end up doing much more work than writing unit tests combined with features. All best Andrew 2008/11/25 Raimond Garcia <lists at ruby-forum.com>:>> Wow, if that''s it in a nutshell... :) >> >> Pat > > Thanks Pat, great summary. > > I have to admit that I''m as crazy as Yehuda, > and believe that all we need are just acceptance tests, > at different layers of abstraction, for clients and developers. > > I also see the benefits of speccing out single object''s behaviors, with > the aim of a good design. > However, the drawbacks of doing this out-weight the benefits, in my > opinion. > > Testing how every method of an object is going to behave, > implies that after refactoring, that spec will no longer be useful, > eventhough the business and application logic stay the same. > > I believe that being able to come up with a good design, > is not only dependent on writing tests before your implementation, > but also on knowing how to write a good implementation. > > This can be gained through experience, > reading books, blogs, pair-programming, > using tools to tell you about the complexity of your code, > and a constant process of refactoring as we complete requirements, > and then fully understand what the best design could be. > > Therefore in my opinion, by writing tests that guarantee > the correct functioning of the system, we have a robust suite of tests. > Let the refactoring come storming in and change the whole > implementation, > but the tests should not be affected at all, > as I''m not testing my implementation nor design, > only the correct functioning of the system, > and relying on other tools on top of tests to maintain my code > nice, clean and understandable by anyone that comes along. > > Kind Regards, > > Rai > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
On Mon, Nov 24, 2008 at 9:41 PM, Pau Cor <lists at ruby-forum.com> wrote:> I really understand what you are getting at. However, as I less > experienced developer (my degree is actually in business) I have found > that having more unit tests (for models and controllers) helps ensure > that I write better code. I can''t think of a single case in which the > code I write where every public method is tested is not better than the > code I write where I don''t do that. > > Am I unique in this? Or is strict TDD for every public method a good > practice for someone who is still learning how to design code well?I feel the same way. Unit testing for me is as much about writing better code as about avoiding errors. As a rule, if i can write clear specs, I''m writing clean code. For instance, the Law of Demeter can seem silly at times--Why can''t I just call foo.bar.baz.bax? But try spec''ing a method which calls foo.bar.baz.bax and you''ll see. You have to tie yourself in knots of mock objects to "decouple" from other object''s implementations. Now you''ve tightly couples your spec to the implementation of the method, as well as some other methods. That coupling is actually a reflection of a hidden coupling in your code. The complexity of the tests reveals a hidden complexity in that train wreck you put in your method. It''s easy to write bad code without noticing. It''s harder to write bad specs. So I write the specs, and I write them first. Peter
On Mon, Nov 24, 2008 at 1:18 PM, Mark Wilden <mark at mwilden.com> wrote:> On Mon, Nov 24, 2008 at 10:29 AM, Pat Maddox <pergesu at gmail.com> wrote: >> >> I disagree with the part about edge cases. Acceptance Tests are about >> defining and verifying business value, and edge cases are supremely >> valuable to businesses. What happens when an ATM user tries to withdraw >> $1 more than he has available in his account? > > Withdrawing $1 more than available is an edge case, vulnerable to an > off-by-one error, and doesn''t need to be shown to business, > in my opinion. > Testing that is a means of detecting errors - it''s not a specification. > > What happens when an ATM user tries to withdraw $100 more than available is > not an edge case, and should be shown to business. > > I realize it''s a fine point - I''m just responding to whether the business > needs to see what we call "edge cases." > > ///arkI think this is going to vary from customer to customer, but in the end, I think it''s up to the business to make this decision, not the developers. 2 more cents. Don''t spend ''em all in one place :) David
Andrew Premdas wrote:> I came across this idea of dropping unit tests for acceptance tests in > the java world. It didn''t like it there and I don''t like it here, but > maybe thats because I''m an old fuddy duddy or something :). I do think > that every public method of an object should be specifically unit > tested, and yes that means that if you refactor your object you should > refactor your unit tests. This isn''t really that much of a burden if > you design your objects to have simple and minimal public api''s in the > first place. >+1> What is that makes you think you can refactor code run acceptance > tests and be save without unit tests? Writing tests "that guarantee > the correct functioning of the system" isn''t something you can just > do. Best you can hope for with acceptance tests is that part of the > system functions correctly most of the time in some circumstances. > > Perhaps its the BDD ideal that your only writing the code you need to > make your acceptance tests pass, that make you think your acceptance > tests cover all your code. However just because you''ve written minimal > code to make an acceptance test pass doesn''t mean that you can''t use > this code in a multitude of different ways > > Do you really think that BDD created code is just some black box that > you can tinker around with restructure and still be sure it works just > because your black box tests still work? > > I just don''t believe you can get the coverage you need for an > application using acceptance testing / features alone. If you do > actually write enough features to do this you''ll end up doing much > more work than writing unit tests combined with features. >+1 again.> All best > > Andrew >Here is how I look at the two sets of tests... Features at the application level (acceptance tests) instill more confidence in me about the correctness of the system''s behavior. Object level code examples (unit tests) instill more confidence in me about the design of the system. With acceptance tests passing we have no guarantee about the state of the design. Remember, TDD/BDD naturally produces easy to test objects and by skipping object level examples you run the risk of creating dependent laden, highly coupled objects that are hard to test. (Just think, you can make all of your features, for a web app, pass by writing the app in PHP4 with no objects at all :p .) I also think that acceptance tests are too slow to be used in all refactorings and they are not fine grained enough so you''ll end up doing more debugging than you would otherwise with good object level coverage. I generally try to keep each individual unit test faster than a tenth of a second, as suggested in ''Working Effectively With Legacy Code''. What results is an extremely fast suite that can be used to quickly do refactorings. I have experienced the pain of using just Cucumber features first hand-- finding bugs on this level is just not as fast object level examples. If you skip object level examples you are incurring a technical debt that you will feel down the road, IMO. Someone at the start of this thread had wondered what people had learned when they went through this process of balancing FIT tests with unit tests. While I know some people on this list could provide some first hand experience, I think this post by Bob Martin should provide some good insight: http://blog.objectmentor.com/articles/2007/10/17/tdd-with-acceptance-tests-and-unit-tests - Ben Mabey> 2008/11/25 Raimond Garcia <lists at ruby-forum.com>: > >>> Wow, if that''s it in a nutshell... :) >>> >>> Pat >>> >> Thanks Pat, great summary. >> >> I have to admit that I''m as crazy as Yehuda, >> and believe that all we need are just acceptance tests, >> at different layers of abstraction, for clients and developers. >> >> I also see the benefits of speccing out single object''s behaviors, with >> the aim of a good design. >> However, the drawbacks of doing this out-weight the benefits, in my >> opinion. >> >> Testing how every method of an object is going to behave, >> implies that after refactoring, that spec will no longer be useful, >> eventhough the business and application logic stay the same. >> >> I believe that being able to come up with a good design, >> is not only dependent on writing tests before your implementation, >> but also on knowing how to write a good implementation. >> >> This can be gained through experience, >> reading books, blogs, pair-programming, >> using tools to tell you about the complexity of your code, >> and a constant process of refactoring as we complete requirements, >> and then fully understand what the best design could be. >> >> Therefore in my opinion, by writing tests that guarantee >> the correct functioning of the system, we have a robust suite of tests. >> Let the refactoring come storming in and change the whole >> implementation, >> but the tests should not be affected at all, >> as I''m not testing my implementation nor design, >> only the correct functioning of the system, >> and relying on other tools on top of tests to maintain my code >> nice, clean and understandable by anyone that comes along. >> >> Kind Regards, >> >> Rai >> -- >> Posted via http://www.ruby-forum.com/. >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users >> >> > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
On Tue, Nov 25, 2008 at 12:52 AM, Ben Mabey <ben at benmabey.com> wrote:> Andrew Premdas wrote: >> >> I came across this idea of dropping unit tests for acceptance tests in >> the java world. It didn''t like it there and I don''t like it here, but >> maybe thats because I''m an old fuddy duddy or something :). I do think >> that every public method of an object should be specifically unit >> tested, and yes that means that if you refactor your object you should >> refactor your unit tests. This isn''t really that much of a burden if >> you design your objects to have simple and minimal public api''s in the >> first place. >> > > +1 >> >> What is that makes you think you can refactor code run acceptance >> tests and be save without unit tests? Writing tests "that guarantee >> the correct functioning of the system" isn''t something you can just >> do. Best you can hope for with acceptance tests is that part of the >> system functions correctly most of the time in some circumstances. >> >> Perhaps its the BDD ideal that your only writing the code you need to >> make your acceptance tests pass, that make you think your acceptance >> tests cover all your code. However just because you''ve written minimal >> code to make an acceptance test pass doesn''t mean that you can''t use >> this code in a multitude of different ways >> >> Do you really think that BDD created code is just some black box that >> you can tinker around with restructure and still be sure it works just >> because your black box tests still work? >> >> I just don''t believe you can get the coverage you need for an >> application using acceptance testing / features alone. If you do >> actually write enough features to do this you''ll end up doing much >> more work than writing unit tests combined with features. >> > > +1 again. >> >> All best >> >> Andrew >> > > Here is how I look at the two sets of tests... > > Features at the application level (acceptance tests) instill more confidenceCONFIDENCE! That and, as Kent Beck describes today, responsible software, are why we do testing at all.> in me about the correctness of the system''s behavior. Object level code > examples (unit tests) instill more confidence in me about the design of the > system. > With acceptance tests passing we have no guarantee about the state of the > design. Remember, TDD/BDD naturally produces easy to test objects and by > skipping object level examples you run the risk of creating dependent laden, > highly coupled objects that are hard to test. (Just think, you can make all > of your features, for a web app, pass by writing the app in PHP4 with no > objects at all :p .)Which is not an inherently bad deal, if that''s your comfort zone, and if that''s the comfort zone of *everybody* on your team.> I also think that acceptance tests are too slow to be used in all > refactorings and they are not fine grained enough so you''ll end up doing > more debugging than you would otherwise with good object level coverage. I > generally try to keep each individual unit test faster than a tenth of a > second, as suggested in ''Working Effectively With Legacy Code''. What > results is an extremely fast suite that can be used to quickly do > refactorings. I have experienced the pain of using just Cucumber features > first hand-- finding bugs on this level is just not as fast object level > examples. If you skip object level examples you are incurring a technical > debt that you will feel down the road, IMO. > > Someone at the start of this thread had wondered what people had learned > when they went through this process of balancing FIT tests with unit tests.I can speak to this a bit. Maybe more than a bit. When I was working with .NET FitNesse and NUnit, we had very high levels of coverage in NUnit. Early on one project I told Micah Martin (who co-created FitNesse with Bob Martin) that I was concerned about the duplication between our FitNesse tests and NUnit tests and questioned the value of keeping it. Micah pointed out reasons that made absolute 100% perfect sense in the context of the project we were working on. The customers were encouraged to own the FitNesse tests. They were stored on a file system, backed up in zip files, while the NUnit tests were stored in subversion with the code. The FitNesse fixtures were stored with the application code, distant from the FitNesse tests. In order to foster confidence in the code amongst the developers, having a high level of coverage in NUnit made sense, in spite of the duplication with some of the FitNesse tests. That duplication, by the way, was only in terms of method calls at the highest levels of the system. When a FitNesse test made an API call, that message went all the way to the database and back. When an NUnit test made the same call, that message typically got no further than the object in the test, using stubs and mocks to keep it isolated. Now fast forward to our current discussion about Cucumber and RSpec. As things stand today, we tend to store .feature files right in the app alongside the step_definitions and the application code. The implications here are different from having a completely decoupled acceptance testing system. I''m not saying that abandoning RSpec or Test::Unit or whatever is the right thing to do. But I certainly feel less concerned about removing granular code examples, especially on rails/merb controllers and views, when I''ve got excellent coverage of them from Cucumber with Webrat. Thus far I have seen a case where I couldn''t quickly understand a failure in a view or controller based on the feedback I get from Cucumber with Webrat. But this is mostly because that combination of tools does a very good job of pointing me to the right place. This is not always the case with high level examples. If you''re considering relaxing a requirement for granular examples, you should really consider each case separately and include the level of granularity of feedback you''re going to get from your toolset when you make that decision. Now this is how *I* see things. For anybody who is brand new to all this, my feeling is that whatever pain there is from duplication between the two levels of examples and having to change granular examples to refactor is eclipsed by the pain of debugging from high level examples. Also, as I alluded to earlier, every team is different. If you are working solo, the implications of taking risks by working predominantly at higher levels is different from when you are on a team. The point of testing is not to follow a specific process. The point is to instill confidence so you can continue to work without migraines, and deliver quality software. Cheers, David> While I know some people on this list could provide some first hand > experience, I think this post by Bob Martin should provide some good > insight: > > http://blog.objectmentor.com/articles/2007/10/17/tdd-with-acceptance-tests-and-unit-tests > > - Ben Mabey > >> 2008/11/25 Raimond Garcia <lists at ruby-forum.com>: >> >>>> >>>> Wow, if that''s it in a nutshell... :) >>>> >>>> Pat >>>> >>> >>> Thanks Pat, great summary. >>> >>> I have to admit that I''m as crazy as Yehuda, >>> and believe that all we need are just acceptance tests, >>> at different layers of abstraction, for clients and developers. >>> >>> I also see the benefits of speccing out single object''s behaviors, with >>> the aim of a good design. >>> However, the drawbacks of doing this out-weight the benefits, in my >>> opinion. >>> >>> Testing how every method of an object is going to behave, >>> implies that after refactoring, that spec will no longer be useful, >>> eventhough the business and application logic stay the same. >>> >>> I believe that being able to come up with a good design, >>> is not only dependent on writing tests before your implementation, >>> but also on knowing how to write a good implementation. >>> >>> This can be gained through experience, >>> reading books, blogs, pair-programming, >>> using tools to tell you about the complexity of your code, >>> and a constant process of refactoring as we complete requirements, >>> and then fully understand what the best design could be. >>> >>> Therefore in my opinion, by writing tests that guarantee >>> the correct functioning of the system, we have a robust suite of tests. >>> Let the refactoring come storming in and change the whole >>> implementation, >>> but the tests should not be affected at all, >>> as I''m not testing my implementation nor design, >>> only the correct functioning of the system, >>> and relying on other tools on top of tests to maintain my code >>> nice, clean and understandable by anyone that comes along. >>> >>> Kind Regards, >>> >>> Rai >>> -- >>> Posted via http://www.ruby-forum.com/. >>> _______________________________________________ >>> rspec-users mailing list >>> rspec-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/rspec-users >>> >>> >> >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users >> > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
James Byrne wrote:> As I work with Rails TestUnit tests I am reconsidering how to useI discover that in Ruby 1.9 TestUnit is out and minitest is in. I wonder what effect, if any, this will have on future releases of Rails. http://www.ruby-forum.com/topic/171625 -- Posted via http://www.ruby-forum.com/.
On Mon, Nov 24, 2008 at 8:34 PM, Raimond Garcia <lists at ruby-forum.com> wrote:>> Wow, if that''s it in a nutshell... :) >> >> Pat > > Thanks Pat, great summary. > > I have to admit that I''m as crazy as Yehuda, > and believe that all we need are just acceptance tests, > at different layers of abstraction, for clients and developers.There are different ways to ensure code works at different layers of abstraction. One might even call these ATs for clients, and specs for developers. ;) That''s not to say that you couldn''t simply write Cucumber scenarios for both of these levels. But it''s important to point out that the end goal is the same, the means and tools used to get there may use different approaches. I am glad that in this community there are such diverse opinions on how-to approach delivering quality software. It pushes the envelopes, challenges what we know now, and makes us each better at our craft. Zach> > I also see the benefits of speccing out single object''s behaviors, with > the aim of a good design. > However, the drawbacks of doing this out-weight the benefits, in my > opinion. > > Testing how every method of an object is going to behave, > implies that after refactoring, that spec will no longer be useful, > eventhough the business and application logic stay the same. > > I believe that being able to come up with a good design, > is not only dependent on writing tests before your implementation, > but also on knowing how to write a good implementation. > > This can be gained through experience, > reading books, blogs, pair-programming, > using tools to tell you about the complexity of your code, > and a constant process of refactoring as we complete requirements, > and then fully understand what the best design could be. > > Therefore in my opinion, by writing tests that guarantee > the correct functioning of the system, we have a robust suite of tests. > Let the refactoring come storming in and change the whole > implementation, > but the tests should not be affected at all, > as I''m not testing my implementation nor design, > only the correct functioning of the system, > and relying on other tools on top of tests to maintain my code > nice, clean and understandable by anyone that comes along. > > Kind Regards, > > Rai > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-- Zach Dennis http://www.continuousthinking.com http://www.mutuallyhuman.com
David Chelimsky wrote:> On Tue, Nov 25, 2008 at 12:52 AM, Ben Mabey <ben at benmabey.com> wrote: > >> Andrew Premdas wrote: >> >>> I came across this idea of dropping unit tests for acceptance tests in >>> the java world. It didn''t like it there and I don''t like it here, but >>> maybe thats because I''m an old fuddy duddy or something :). I do think >>> that every public method of an object should be specifically unit >>> tested, and yes that means that if you refactor your object you should >>> refactor your unit tests. This isn''t really that much of a burden if >>> you design your objects to have simple and minimal public api''s in the >>> first place. >>> >>> >> +1 >> >>> What is that makes you think you can refactor code run acceptance >>> tests and be save without unit tests? Writing tests "that guarantee >>> the correct functioning of the system" isn''t something you can just >>> do. Best you can hope for with acceptance tests is that part of the >>> system functions correctly most of the time in some circumstances. >>> >>> Perhaps its the BDD ideal that your only writing the code you need to >>> make your acceptance tests pass, that make you think your acceptance >>> tests cover all your code. However just because you''ve written minimal >>> code to make an acceptance test pass doesn''t mean that you can''t use >>> this code in a multitude of different ways >>> >>> Do you really think that BDD created code is just some black box that >>> you can tinker around with restructure and still be sure it works just >>> because your black box tests still work? >>> >>> I just don''t believe you can get the coverage you need for an >>> application using acceptance testing / features alone. If you do >>> actually write enough features to do this you''ll end up doing much >>> more work than writing unit tests combined with features. >>> >>> >> +1 again. >> >>> All best >>> >>> Andrew >>> >>> >> Here is how I look at the two sets of tests... >> >> Features at the application level (acceptance tests) instill more confidence >> > > CONFIDENCE! > > That and, as Kent Beck describes today, responsible software, are why > we do testing at all. > > >> in me about the correctness of the system''s behavior. Object level code >> examples (unit tests) instill more confidence in me about the design of the >> system. >> With acceptance tests passing we have no guarantee about the state of the >> design. Remember, TDD/BDD naturally produces easy to test objects and by >> skipping object level examples you run the risk of creating dependent laden, >> highly coupled objects that are hard to test. (Just think, you can make all >> of your features, for a web app, pass by writing the app in PHP4 with no >> objects at all :p .) >> > > Which is not an inherently bad deal, if that''s your comfort zone, and > if that''s the comfort zone of *everybody* on your team. > > >> I also think that acceptance tests are too slow to be used in all >> refactorings and they are not fine grained enough so you''ll end up doing >> more debugging than you would otherwise with good object level coverage. I >> generally try to keep each individual unit test faster than a tenth of a >> second, as suggested in ''Working Effectively With Legacy Code''. What >> results is an extremely fast suite that can be used to quickly do >> refactorings. I have experienced the pain of using just Cucumber features >> first hand-- finding bugs on this level is just not as fast object level >> examples. If you skip object level examples you are incurring a technical >> debt that you will feel down the road, IMO. >> >> Someone at the start of this thread had wondered what people had learned >> when they went through this process of balancing FIT tests with unit tests. >> > > I can speak to this a bit. Maybe more than a bit. > > When I was working with .NET FitNesse and NUnit, we had very high > levels of coverage in NUnit. Early on one project I told Micah Martin > (who co-created FitNesse with Bob Martin) that I was concerned about > the duplication between our FitNesse tests and NUnit tests and > questioned the value of keeping it. > > Micah pointed out reasons that made absolute 100% perfect sense in the > context of the project we were working on. The customers were > encouraged to own the FitNesse tests. They were stored on a file > system, backed up in zip files, while the NUnit tests were stored in > subversion with the code. The FitNesse fixtures were stored with the > application code, distant from the FitNesse tests. > > In order to foster confidence in the code amongst the developers, > having a high level of coverage in NUnit made sense, in spite of the > duplication with some of the FitNesse tests. > > That duplication, by the way, was only in terms of method calls at the > highest levels of the system. When a FitNesse test made an API call, > that message went all the way to the database and back. > > When an NUnit test made the same call, that message typically got no > further than the object in the test, using stubs and mocks to keep it > isolated. > > Now fast forward to our current discussion about Cucumber and RSpec. > As things stand today, we tend to store .feature files right in the > app alongside the step_definitions and the application code. > > The implications here are different from having a completely decoupled > acceptance testing system. I''m not saying that abandoning RSpec or > Test::Unit or whatever is the right thing to do. But I certainly feel > less concerned about removing granular code examples, especially on > rails/merb controllers and views, when I''ve got excellent coverage of > them from Cucumber with Webrat. Thus far I have seen a case where I > couldn''t quickly understand a failure in a view or controller based on > the feedback I get from Cucumber with Webrat. > > But this is mostly because that combination of tools does a very good > job of pointing me to the right place. This is not always the case > with high level examples. If you''re considering relaxing a requirement > for granular examples, you should really consider each case separately > and include the level of granularity of feedback you''re going to get > from your toolset when you make that decision. > > Now this is how *I* see things. > > For anybody who is brand new to all this, my feeling is that whatever > pain there is from duplication between the two levels of examples and > having to change granular examples to refactor is eclipsed by the pain > of debugging from high level examples. > > Also, as I alluded to earlier, every team is different. If you are > working solo, the implications of taking risks by working > predominantly at higher levels is different from when you are on a > team. The point of testing is not to follow a specific process. The > point is to instill confidence so you can continue to work without > migraines, and deliver quality software. > > Cheers, > David >Thanks for sharing your experience and insight! Having never used FitNesse I didn''t see that distinction at all. What you said makes a lot of sense. -Ben> >> While I know some people on this list could provide some first hand >> experience, I think this post by Bob Martin should provide some good >> insight: >> >> http://blog.objectmentor.com/articles/2007/10/17/tdd-with-acceptance-tests-and-unit-tests >> >> - Ben Mabey >> >> >>> 2008/11/25 Raimond Garcia <lists at ruby-forum.com>: >>> >>> >>>>> Wow, if that''s it in a nutshell... :) >>>>> >>>>> Pat >>>>> >>>>> >>>> Thanks Pat, great summary. >>>> >>>> I have to admit that I''m as crazy as Yehuda, >>>> and believe that all we need are just acceptance tests, >>>> at different layers of abstraction, for clients and developers. >>>> >>>> I also see the benefits of speccing out single object''s behaviors, with >>>> the aim of a good design. >>>> However, the drawbacks of doing this out-weight the benefits, in my >>>> opinion. >>>> >>>> Testing how every method of an object is going to behave, >>>> implies that after refactoring, that spec will no longer be useful, >>>> eventhough the business and application logic stay the same. >>>> >>>> I believe that being able to come up with a good design, >>>> is not only dependent on writing tests before your implementation, >>>> but also on knowing how to write a good implementation. >>>> >>>> This can be gained through experience, >>>> reading books, blogs, pair-programming, >>>> using tools to tell you about the complexity of your code, >>>> and a constant process of refactoring as we complete requirements, >>>> and then fully understand what the best design could be. >>>> >>>> Therefore in my opinion, by writing tests that guarantee >>>> the correct functioning of the system, we have a robust suite of tests. >>>> Let the refactoring come storming in and change the whole >>>> implementation, >>>> but the tests should not be affected at all, >>>> as I''m not testing my implementation nor design, >>>> only the correct functioning of the system, >>>> and relying on other tools on top of tests to maintain my code >>>> nice, clean and understandable by anyone that comes along. >>>> >>>> Kind Regards, >>>> >>>> Rai >>>> -- >>>> Posted via http://www.ruby-forum.com/. >>>> _______________________________________________ >>>> rspec-users mailing list >>>> rspec-users at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/rspec-users >>>> >>>> >>>> >>> _______________________________________________ >>> rspec-users mailing list >>> rspec-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/rspec-users >>> >>> >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users >> >> > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
On 25 Nov 2008, at 17:26, Ben Mabey wrote:> David Chelimsky wrote: >> On Tue, Nov 25, 2008 at 12:52 AM, Ben Mabey <ben at benmabey.com> wrote: >> >>> Andrew Premdas wrote: >>> >>>> I came across this idea of dropping unit tests for acceptance >>>> tests in >>>> the java world. It didn''t like it there and I don''t like it here, >>>> but >>>> maybe thats because I''m an old fuddy duddy or something :). I do >>>> think >>>> that every public method of an object should be specifically unit >>>> tested, and yes that means that if you refactor your object you >>>> should >>>> refactor your unit tests. This isn''t really that much of a burden >>>> if >>>> you design your objects to have simple and minimal public api''s >>>> in the >>>> first place. >>>> >>>> >>> +1 >>> >>>> What is that makes you think you can refactor code run acceptance >>>> tests and be save without unit tests? Writing tests "that guarantee >>>> the correct functioning of the system" isn''t something you can just >>>> do. Best you can hope for with acceptance tests is that part of the >>>> system functions correctly most of the time in some circumstances. >>>> >>>> Perhaps its the BDD ideal that your only writing the code you >>>> need to >>>> make your acceptance tests pass, that make you think your >>>> acceptance >>>> tests cover all your code. However just because you''ve written >>>> minimal >>>> code to make an acceptance test pass doesn''t mean that you can''t >>>> use >>>> this code in a multitude of different ways >>>> >>>> Do you really think that BDD created code is just some black box >>>> that >>>> you can tinker around with restructure and still be sure it works >>>> just >>>> because your black box tests still work? >>>> >>>> I just don''t believe you can get the coverage you need for an >>>> application using acceptance testing / features alone. If you do >>>> actually write enough features to do this you''ll end up doing much >>>> more work than writing unit tests combined with features. >>>> >>>> >>> +1 again. >>> >>>> All best >>>> >>>> Andrew >>>> >>>> >>> Here is how I look at the two sets of tests... >>> >>> Features at the application level (acceptance tests) instill more >>> confidence >>> >> >> CONFIDENCE! >> >> That and, as Kent Beck describes today, responsible software, are why >> we do testing at all. >> >> >>> in me about the correctness of the system''s behavior. Object level >>> code >>> examples (unit tests) instill more confidence in me about the >>> design of the >>> system. >>> With acceptance tests passing we have no guarantee about the state >>> of the >>> design. Remember, TDD/BDD naturally produces easy to test objects >>> and by >>> skipping object level examples you run the risk of creating >>> dependent laden, >>> highly coupled objects that are hard to test. (Just think, you >>> can make all >>> of your features, for a web app, pass by writing the app in PHP4 >>> with no >>> objects at all :p .) >>> >> >> Which is not an inherently bad deal, if that''s your comfort zone, and >> if that''s the comfort zone of *everybody* on your team. >> >> >>> I also think that acceptance tests are too slow to be used in all >>> refactorings and they are not fine grained enough so you''ll end up >>> doing >>> more debugging than you would otherwise with good object level >>> coverage. I >>> generally try to keep each individual unit test faster than a >>> tenth of a >>> second, as suggested in ''Working Effectively With Legacy Code''. >>> What >>> results is an extremely fast suite that can be used to quickly do >>> refactorings. I have experienced the pain of using just Cucumber >>> features >>> first hand-- finding bugs on this level is just not as fast object >>> level >>> examples. If you skip object level examples you are incurring a >>> technical >>> debt that you will feel down the road, IMO. >>> >>> Someone at the start of this thread had wondered what people had >>> learned >>> when they went through this process of balancing FIT tests with >>> unit tests. >>> >> >> I can speak to this a bit. Maybe more than a bit. >> >> When I was working with .NET FitNesse and NUnit, we had very high >> levels of coverage in NUnit. Early on one project I told Micah Martin >> (who co-created FitNesse with Bob Martin) that I was concerned about >> the duplication between our FitNesse tests and NUnit tests and >> questioned the value of keeping it. >> >> Micah pointed out reasons that made absolute 100% perfect sense in >> the >> context of the project we were working on. The customers were >> encouraged to own the FitNesse tests. They were stored on a file >> system, backed up in zip files, while the NUnit tests were stored in >> subversion with the code. The FitNesse fixtures were stored with the >> application code, distant from the FitNesse tests. >> >> In order to foster confidence in the code amongst the developers, >> having a high level of coverage in NUnit made sense, in spite of the >> duplication with some of the FitNesse tests. >> >> That duplication, by the way, was only in terms of method calls at >> the >> highest levels of the system. When a FitNesse test made an API call, >> that message went all the way to the database and back. >> >> When an NUnit test made the same call, that message typically got no >> further than the object in the test, using stubs and mocks to keep it >> isolated. >> >> Now fast forward to our current discussion about Cucumber and RSpec. >> As things stand today, we tend to store .feature files right in the >> app alongside the step_definitions and the application code. >> >> The implications here are different from having a completely >> decoupled >> acceptance testing system. I''m not saying that abandoning RSpec or >> Test::Unit or whatever is the right thing to do. But I certainly feel >> less concerned about removing granular code examples, especially on >> rails/merb controllers and views, when I''ve got excellent coverage of >> them from Cucumber with Webrat. Thus far I have seen a case where I >> couldn''t quickly understand a failure in a view or controller based >> on >> the feedback I get from Cucumber with Webrat. >> >> But this is mostly because that combination of tools does a very good >> job of pointing me to the right place. This is not always the case >> with high level examples. If you''re considering relaxing a >> requirement >> for granular examples, you should really consider each case >> separately >> and include the level of granularity of feedback you''re going to get >> from your toolset when you make that decision. >> >> Now this is how *I* see things. >> >> For anybody who is brand new to all this, my feeling is that whatever >> pain there is from duplication between the two levels of examples and >> having to change granular examples to refactor is eclipsed by the >> pain >> of debugging from high level examples. >> >> Also, as I alluded to earlier, every team is different. If you are >> working solo, the implications of taking risks by working >> predominantly at higher levels is different from when you are on a >> team. The point of testing is not to follow a specific process. The >> point is to instill confidence so you can continue to work without >> migraines, and deliver quality software. >> >> Cheers, >> David >> > > Thanks for sharing your experience and insight! Having never used > FitNesse I didn''t see that distinction at all. What you said makes a > lot of sense. > -BenAmen to that. Thanks guys, it''s been a fascinating and enlightening discussion. I am looking forward to the next chance I get to talk about this with someone (who''s interested!) over a beer. I don''t suppose any of you are going to XP Day, London, this year? cheers, Matt
Question: In Cucumber when you''re writing code to satisfy steps and accessing the model objects directly, what support for asserts, responses, etc. do people use. (the equivalent of ActionController::TestCase and ActiveSupport::TestCase), Fixtures, etc. Thanks, T On Tue, Nov 25, 2008 at 12:16 PM, Matt Wynne <matt at mattwynne.net> wrote:> > On 25 Nov 2008, at 17:26, Ben Mabey wrote: > >> David Chelimsky wrote: >>> >>> On Tue, Nov 25, 2008 at 12:52 AM, Ben Mabey <ben at benmabey.com> wrote: >>> >>>> Andrew Premdas wrote: >>>> >>>>> I came across this idea of dropping unit tests for acceptance tests in >>>>> the java world. It didn''t like it there and I don''t like it here, but >>>>> maybe thats because I''m an old fuddy duddy or something :). I do think >>>>> that every public method of an object should be specifically unit >>>>> tested, and yes that means that if you refactor your object you should >>>>> refactor your unit tests. This isn''t really that much of a burden if >>>>> you design your objects to have simple and minimal public api''s in the >>>>> first place. >>>>> >>>>> >>>> +1 >>>> >>>>> What is that makes you think you can refactor code run acceptance >>>>> tests and be save without unit tests? Writing tests "that guarantee >>>>> the correct functioning of the system" isn''t something you can just >>>>> do. Best you can hope for with acceptance tests is that part of the >>>>> system functions correctly most of the time in some circumstances. >>>>> >>>>> Perhaps its the BDD ideal that your only writing the code you need to >>>>> make your acceptance tests pass, that make you think your acceptance >>>>> tests cover all your code. However just because you''ve written minimal >>>>> code to make an acceptance test pass doesn''t mean that you can''t use >>>>> this code in a multitude of different ways >>>>> >>>>> Do you really think that BDD created code is just some black box that >>>>> you can tinker around with restructure and still be sure it works just >>>>> because your black box tests still work? >>>>> >>>>> I just don''t believe you can get the coverage you need for an >>>>> application using acceptance testing / features alone. If you do >>>>> actually write enough features to do this you''ll end up doing much >>>>> more work than writing unit tests combined with features. >>>>> >>>>> >>>> +1 again. >>>> >>>>> All best >>>>> >>>>> Andrew >>>>> >>>>> >>>> Here is how I look at the two sets of tests... >>>> >>>> Features at the application level (acceptance tests) instill more >>>> confidence >>>> >>> >>> CONFIDENCE! >>> >>> That and, as Kent Beck describes today, responsible software, are why >>> we do testing at all. >>> >>> >>>> in me about the correctness of the system''s behavior. Object level code >>>> examples (unit tests) instill more confidence in me about the design of >>>> the >>>> system. >>>> With acceptance tests passing we have no guarantee about the state of >>>> the >>>> design. Remember, TDD/BDD naturally produces easy to test objects and >>>> by >>>> skipping object level examples you run the risk of creating dependent >>>> laden, >>>> highly coupled objects that are hard to test. (Just think, you can make >>>> all >>>> of your features, for a web app, pass by writing the app in PHP4 with no >>>> objects at all :p .) >>>> >>> >>> Which is not an inherently bad deal, if that''s your comfort zone, and >>> if that''s the comfort zone of *everybody* on your team. >>> >>> >>>> I also think that acceptance tests are too slow to be used in all >>>> refactorings and they are not fine grained enough so you''ll end up doing >>>> more debugging than you would otherwise with good object level coverage. >>>> I >>>> generally try to keep each individual unit test faster than a tenth of a >>>> second, as suggested in ''Working Effectively With Legacy Code''. What >>>> results is an extremely fast suite that can be used to quickly do >>>> refactorings. I have experienced the pain of using just Cucumber >>>> features >>>> first hand-- finding bugs on this level is just not as fast object level >>>> examples. If you skip object level examples you are incurring a >>>> technical >>>> debt that you will feel down the road, IMO. >>>> >>>> Someone at the start of this thread had wondered what people had learned >>>> when they went through this process of balancing FIT tests with unit >>>> tests. >>>> >>> >>> I can speak to this a bit. Maybe more than a bit. >>> >>> When I was working with .NET FitNesse and NUnit, we had very high >>> levels of coverage in NUnit. Early on one project I told Micah Martin >>> (who co-created FitNesse with Bob Martin) that I was concerned about >>> the duplication between our FitNesse tests and NUnit tests and >>> questioned the value of keeping it. >>> >>> Micah pointed out reasons that made absolute 100% perfect sense in the >>> context of the project we were working on. The customers were >>> encouraged to own the FitNesse tests. They were stored on a file >>> system, backed up in zip files, while the NUnit tests were stored in >>> subversion with the code. The FitNesse fixtures were stored with the >>> application code, distant from the FitNesse tests. >>> >>> In order to foster confidence in the code amongst the developers, >>> having a high level of coverage in NUnit made sense, in spite of the >>> duplication with some of the FitNesse tests. >>> >>> That duplication, by the way, was only in terms of method calls at the >>> highest levels of the system. When a FitNesse test made an API call, >>> that message went all the way to the database and back. >>> >>> When an NUnit test made the same call, that message typically got no >>> further than the object in the test, using stubs and mocks to keep it >>> isolated. >>> >>> Now fast forward to our current discussion about Cucumber and RSpec. >>> As things stand today, we tend to store .feature files right in the >>> app alongside the step_definitions and the application code. >>> >>> The implications here are different from having a completely decoupled >>> acceptance testing system. I''m not saying that abandoning RSpec or >>> Test::Unit or whatever is the right thing to do. But I certainly feel >>> less concerned about removing granular code examples, especially on >>> rails/merb controllers and views, when I''ve got excellent coverage of >>> them from Cucumber with Webrat. Thus far I have seen a case where I >>> couldn''t quickly understand a failure in a view or controller based on >>> the feedback I get from Cucumber with Webrat. >>> >>> But this is mostly because that combination of tools does a very good >>> job of pointing me to the right place. This is not always the case >>> with high level examples. If you''re considering relaxing a requirement >>> for granular examples, you should really consider each case separately >>> and include the level of granularity of feedback you''re going to get >>> from your toolset when you make that decision. >>> >>> Now this is how *I* see things. >>> >>> For anybody who is brand new to all this, my feeling is that whatever >>> pain there is from duplication between the two levels of examples and >>> having to change granular examples to refactor is eclipsed by the pain >>> of debugging from high level examples. >>> >>> Also, as I alluded to earlier, every team is different. If you are >>> working solo, the implications of taking risks by working >>> predominantly at higher levels is different from when you are on a >>> team. The point of testing is not to follow a specific process. The >>> point is to instill confidence so you can continue to work without >>> migraines, and deliver quality software. >>> >>> Cheers, >>> David >>> >> >> Thanks for sharing your experience and insight! Having never used >> FitNesse I didn''t see that distinction at all. What you said makes a lot of >> sense. >> -Ben > > Amen to that. Thanks guys, it''s been a fascinating and enlightening > discussion. > > I am looking forward to the next chance I get to talk about this with > someone (who''s interested!) over a beer. > > I don''t suppose any of you are going to XP Day, London, this year? > > cheers, > Matt > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
Tim Walker wrote:> Question: In Cucumber when you''re writing code to satisfy steps and > accessing the model objects directly, what support for asserts, responses, etc. > do people use. (the equivalent of ActionController::TestCase and > ActiveSupport::TestCase), Fixtures, etc.Cucumber depends upon RSpec. Try here: http://rspec.info/documentation/rails/writing/ -- Posted via http://www.ruby-forum.com/.
> Tim Walker wrote: >> Question: In Cucumber when you''re writing code to satisfy steps and >> accessing the model objects directly, what support for asserts, >> responses, etc. >> do people use. (the equivalent of ActionController::TestCase and >> ActiveSupport::TestCase), Fixtures, etc. > > Cucumber depends upon RSpec.No it doesn''t Aslak> Try here: > > http://rspec.info/documentation/rails/writing/ > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users
Aslak Helles?y wrote:>> >> Cucumber depends upon RSpec. > > No it doesn''t > > AslakForgive my misapprehension. -- Posted via http://www.ruby-forum.com/.
James Byrne wrote:> Aslak Helles?y wrote: > >>> >>> Cucumber depends upon RSpec. >> >> No it doesn''t >> >> Aslak > > Forgive my misapprehension.So, where does one find a comprehensive list of expectations for cucumber step matchers? Things like: response.body.should +~ \pattern\ In my ignorance I have been using RSpec as a guide I am looking in the cucumber rdocs but I do not recognize anything as an expectation. -- Posted via http://www.ruby-forum.com/.
On Tue, Nov 25, 2008 at 4:06 PM, James Byrne <lists at ruby-forum.com> wrote:> James Byrne wrote: >> Aslak Helles?y wrote: >> >>>> >>>> Cucumber depends upon RSpec. >>> >>> No it doesn''t >>> >>> Aslak >> >> Forgive my misapprehension. > > So, where does one find a comprehensive list of expectations for > cucumber step matchers? Things like: > > response.body.should +~ \pattern\ >Cucumber doesn''t supply those. You can use whatever tool you want to do the job. RSpec, test/unit, assert 2.0, etc.. could all be used. You may have to include something into "World" to get the methods available from your tool of choice, but I use RSpec and haven''t had to do that (just guessing)... -- Zach Dennis http://www.continuousthinking.com http://www.mutuallyhuman.com
Zach Dennis wrote:> On Tue, Nov 25, 2008 at 4:06 PM, James Byrne <lists at ruby-forum.com> wrote: > >> James Byrne wrote: >> >>> Aslak Helles?y wrote: >>> >>> >>>>> Cucumber depends upon RSpec. >>>>> >>>> No it doesn''t >>>> >>>> Aslak >>>> >>> Forgive my misapprehension. >>> >> So, where does one find a comprehensive list of expectations for >> cucumber step matchers? Things like: >> >> response.body.should +~ \pattern\ >> >> > > Cucumber doesn''t supply those. You can use whatever tool you want to > do the job. RSpec, test/unit, assert 2.0, etc.. could all be used. You > may have to include something into "World" to get the methods > available from your tool of choice, but I use RSpec and haven''t had to > do that (just guessing)... > >Cucumber has some examples on how to use other testing frameworks: http://github.com/aslakhellesoy/cucumber/tree/89c56b13f09d5293b738c3f3feb0ed08af8a43ce/examples%2Ftest_unit%2Ffeatures%2Fstep_definitions%2Ftest_unit_steps.rb The previous gem releases of cucumber required the rspec gem but as of a few commits ago that dependency is only there for developing cucumber. -Ben
James Byrne wrote:> Aslak Helles?y wrote: > >>> >>> Cucumber depends upon RSpec. >> >> No it doesn''t >> >> Aslak > > Forgive my misapprehension.However, this is what rdoc says: cucumber 0.1.9 [rdoc] [www] - depends on diff-lcs, hoe, rspec, term-ansicolor, treetop. -- Posted via http://www.ruby-forum.com/.
Ben Mabey wrote:> > The previous gem releases of cucumber required the rspec gem but as of a > few commits ago that dependency is only there for developing cucumber. > > -BenI see. So, if I understand correctly, rspec is the "default" testing framework? But, if one wished to incorporate minitest say, then one would extend the cucumber world (a concept that I have only the vaguest conception of) in a manner similar to this? require ''minitest/unit/assertions'' World do |o| o.extend(MiniTest::Unit::Assertions) o end -- Posted via http://www.ruby-forum.com/.
James Byrne wrote:> But, if one wished to incorporate minitest say, then one > would extend the cucumber worldWhere does one put this? A the begining of each step_definitions file? In support/env.rb? -- Posted via http://www.ruby-forum.com/.
James Byrne wrote:> Ben Mabey wrote: > > >> The previous gem releases of cucumber required the rspec gem but as of a >> few commits ago that dependency is only there for developing cucumber. >> >> -Ben >> > > I see. So, if I understand correctly, rspec is the "default" testing > framework? But, if one wished to incorporate minitest say, then one > would extend the cucumber world (a concept that I have only the vaguest > conception of) in a manner similar to this? > > require ''minitest/unit/assertions'' > World do |o| > o.extend(MiniTest::Unit::Assertions) > o > end > >Right. Although, I''m unsure if rspec is even the default framework outside of the rails generators. -Ben
James Byrne wrote:> James Byrne wrote: > >> But, if one wished to incorporate minitest say, then one >> would extend the cucumber world >> > > Where does one put this? A the begining of each step_definitions file? > In support/env.rb? > >You only need it once- so the env.rb file is fine and natural place for it.
Ben Mabey wrote:> Right. Although, I''m unsure if rspec is even the default framework > outside of the rails generators. > -BenWhere can one get a handy quick reference of what syntax is acceptable to cucumber by default? -- Posted via http://www.ruby-forum.com/.
James Byrne wrote:> Ben Mabey wrote: > > >> Right. Although, I''m unsure if rspec is even the default framework >> outside of the rails generators. >> -Ben >> > > Where can one get a handy quick reference of what syntax is acceptable > to cucumber by default? >Hmm.. I''m not sure what you mean but the cucumber wiki is the best place: http://github.com/aslakhellesoy/cucumber/wikis -Ben
On Tue, Nov 25, 2008 at 9:54 PM, James Byrne <lists at ruby-forum.com> wrote:> Aslak Helles?y wrote: > > >> > >> Cucumber depends upon RSpec. > > > > No it doesn''t > > > > Aslak > > Forgive my misapprehension.Sorry - I should never email from my iPhone. What I meant is that Cucumber itself does not have any dependencies on RSpec. However, your own step definitions *can* use RSpec''s Rails extensions if you require cucumber/rails/rspec. Most Cucumber/Rails users seem to be doing this although there are no hard dependencies. If you only require cucumber/rails/world you can just use regular Rails integration testing APIs in your step definitions. I hope this clarifies a little more. Cheers, Aslak> > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20081125/ebc3f68e/attachment.html>
James Byrne <lists at ruby-forum.com> writes:> Ben Mabey wrote: > >> Right. Although, I''m unsure if rspec is even the default framework >> outside of the rails generators. >> -Ben > > Where can one get a handy quick reference of what syntax is acceptable > to cucumber by default?<cheeky>Ruby syntax is acceptable</cheeky> Pat
Since Cucumber is about BDD and defining the "acceptable and desired behavior" of the software through plain english (executable requirements if you will) it is not always clear what "level" the steps will implement. In the case of Rails testing out of the box this maps roughly to: 1) unit tests - models 2) functional tests - controllers 3) integration tests - multiple controllers/models In Cucumber we''re not really drawing those lines so clearly and tests will draw on some or more of each of these levels. Is this accurate? Thanks, Tim On Tue, Nov 25, 2008 at 3:37 PM, Pat Maddox <pergesu at gmail.com> wrote:> James Byrne <lists at ruby-forum.com> writes: > >> Ben Mabey wrote: >> >>> Right. Although, I''m unsure if rspec is even the default framework >>> outside of the rails generators. >>> -Ben >> >> Where can one get a handy quick reference of what syntax is acceptable >> to cucumber by default? > > <cheeky>Ruby syntax is acceptable</cheeky> > > Pat > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
On Tue, Nov 25, 2008 at 6:09 PM, Tim Walker <walketim at gmail.com> wrote:> Since Cucumber is about BDD and defining the "acceptable and desired > behavior" of the software through plain english (executable > requirements if you will) it is not always clear what "level" the > steps will implement. > > In the case of Rails testing out of the box this maps roughly to: > 1) unit tests - models > 2) functional tests - controllers > 3) integration tests - multiple controllers/models > > In Cucumber we''re not really drawing those lines so clearly and tests > will draw on some or more of each of these levels. > > Is this accurate?The general recommendation is that Cucumber replaces rails integration tests and RSpec replaces rails functional and unit tests. HTH, David> > Thanks, > > Tim > > On Tue, Nov 25, 2008 at 3:37 PM, Pat Maddox <pergesu at gmail.com> wrote: >> James Byrne <lists at ruby-forum.com> writes: >> >>> Ben Mabey wrote: >>> >>>> Right. Although, I''m unsure if rspec is even the default framework >>>> outside of the rails generators. >>>> -Ben >>> >>> Where can one get a handy quick reference of what syntax is acceptable >>> to cucumber by default? >> >> <cheeky>Ruby syntax is acceptable</cheeky> >> >> Pat >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users >> > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
Tim Walker wrote:> Question: In Cucumber when you''re writing code to satisfy steps and > accessing the model objects directly, what support for asserts, > responses, etc. > do people use. (the equivalent of ActionController::TestCase and > ActiveSupport::TestCase), Fixtures, etc. > > Thanks, > > TThis question prompted a really interesting journey through cucumber for me. I now have a much firmer, if still very limited, grasp of what is happening. When one uses "ruby script/generate cucumber" at the rails project root then the script generates (among other things) a features/support/env.rb file. This file contains (in part): require ''cucumber/rails/world'' require ''cucumber/rails/rspec'' The cucumber gem location lib/cucumber/rails/rspec contains rspec.rb world.rb and rspec.rb has this: require ''spec'' require ''spec/rails'' Next, world.rb makes a conditional reference (almost universally met in a Rails project - Do you use ActiveRecord?) to testunit via /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/test_help.rb. Now that file contains: require ''test/unit'' require ''active_support/test_case'' require ''active_record/fixtures'' require ''action_controller/test_case'' require ''action_controller/integration'' require ''action_mailer/test_case'' if defined?(ActionMailer) So, it appears that when you generate the cucumber infrastructure via the rails generator then you get rspec ''should'' ''should_not'' and rails testunit assert_* support. As previously discussed in this thread, adding other testing harnesses is a fairly straight forward procedure best done in the aforementioned support/env.rb. For example, adding watir gem support is done via this: require ''webrat'' if !defined?(Webrat) I gather from allusions made elsewhere that if watir is installed as a plugin within the rail project then it gets picked up automatically and the default watir.steps generated by the rails generator work without further modification to the env.rb file. I was unable to discover how to employ fixtures. I found and read this thread: http://www.ruby-forum.com/topic/167716 but the final recommendation "Fixtures.create_fixtures("spec/fixtures", "entities")" did not work for me. It did not raise an error but it did not load the fixture either. The TestUnit syntax of "fixtures :model" throws an undefined method error which is passing strange given test_help.rb''s require ''active_record/fixtures''. Nonetheless, TestUnit syntax like assert_something(argument,...argument,message) works fine. Anyway, a most enlightening code crawl. -- Posted via http://www.ruby-forum.com/.
Great post James. Very, helpful. Perhaps should be on the cucumber Wiki? I hope someone follows up on the load fixtures question. Lots to go play with now!!! Tim On Wed, Nov 26, 2008 at 11:04 AM, James Byrne <lists at ruby-forum.com> wrote:> Tim Walker wrote: >> Question: In Cucumber when you''re writing code to satisfy steps and >> accessing the model objects directly, what support for asserts, >> responses, etc. >> do people use. (the equivalent of ActionController::TestCase and >> ActiveSupport::TestCase), Fixtures, etc. >> >> Thanks, >> >> T > > This question prompted a really interesting journey through cucumber for > me. I now have a much firmer, if still very limited, grasp of what is > happening. > > When one uses "ruby script/generate cucumber" at the rails project root > then the script generates (among other things) a features/support/env.rb > file. This file contains (in part): > > require ''cucumber/rails/world'' > require ''cucumber/rails/rspec'' > > The cucumber gem location lib/cucumber/rails/rspec contains > > rspec.rb > world.rb > > and rspec.rb has this: > > require ''spec'' > require ''spec/rails'' > > Next, world.rb makes a conditional reference (almost universally met in > a Rails project - Do you use ActiveRecord?) to testunit via > /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/test_help.rb. Now that file > contains: > > require ''test/unit'' > require ''active_support/test_case'' > require ''active_record/fixtures'' > require ''action_controller/test_case'' > require ''action_controller/integration'' > require ''action_mailer/test_case'' if defined?(ActionMailer) > > So, it appears that when you generate the cucumber infrastructure via > the rails generator then you get rspec ''should'' ''should_not'' and rails > testunit assert_* support. As previously discussed in this thread, > adding other testing harnesses is a fairly straight forward procedure > best done in the aforementioned support/env.rb. For example, adding > watir gem support is done via this: > > require ''webrat'' if !defined?(Webrat) > > I gather from allusions made elsewhere that if watir is installed as a > plugin within the rail project then it gets picked up automatically and > the default watir.steps generated by the rails generator work without > further modification to the env.rb file. > > I was unable to discover how to employ fixtures. I found and read this > thread: http://www.ruby-forum.com/topic/167716 but the final > recommendation "Fixtures.create_fixtures("spec/fixtures", "entities")" > did not work for me. It did not raise an error but it did not load the > fixture either. The TestUnit syntax of "fixtures :model" throws an > undefined method error which is passing strange given test_help.rb''s > require ''active_record/fixtures''. Nonetheless, TestUnit syntax like > assert_something(argument,...argument,message) works fine. > > Anyway, a most enlightening code crawl. > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
Fixtures = yuk!! Try object_daddy or maybe factory_girl instead :) Andrew 2008/11/26 Tim Walker <walketim at gmail.com>:> Great post James. Very, helpful. Perhaps should be on the cucumber > Wiki? I hope someone follows up on the load fixtures question. Lots to > go play with now!!! Tim > > On Wed, Nov 26, 2008 at 11:04 AM, James Byrne <lists at ruby-forum.com> wrote: >> Tim Walker wrote: >>> Question: In Cucumber when you''re writing code to satisfy steps and >>> accessing the model objects directly, what support for asserts, >>> responses, etc. >>> do people use. (the equivalent of ActionController::TestCase and >>> ActiveSupport::TestCase), Fixtures, etc. >>> >>> Thanks, >>> >>> T >> >> This question prompted a really interesting journey through cucumber for >> me. I now have a much firmer, if still very limited, grasp of what is >> happening. >> >> When one uses "ruby script/generate cucumber" at the rails project root >> then the script generates (among other things) a features/support/env.rb >> file. This file contains (in part): >> >> require ''cucumber/rails/world'' >> require ''cucumber/rails/rspec'' >> >> The cucumber gem location lib/cucumber/rails/rspec contains >> >> rspec.rb >> world.rb >> >> and rspec.rb has this: >> >> require ''spec'' >> require ''spec/rails'' >> >> Next, world.rb makes a conditional reference (almost universally met in >> a Rails project - Do you use ActiveRecord?) to testunit via >> /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/test_help.rb. Now that file >> contains: >> >> require ''test/unit'' >> require ''active_support/test_case'' >> require ''active_record/fixtures'' >> require ''action_controller/test_case'' >> require ''action_controller/integration'' >> require ''action_mailer/test_case'' if defined?(ActionMailer) >> >> So, it appears that when you generate the cucumber infrastructure via >> the rails generator then you get rspec ''should'' ''should_not'' and rails >> testunit assert_* support. As previously discussed in this thread, >> adding other testing harnesses is a fairly straight forward procedure >> best done in the aforementioned support/env.rb. For example, adding >> watir gem support is done via this: >> >> require ''webrat'' if !defined?(Webrat) >> >> I gather from allusions made elsewhere that if watir is installed as a >> plugin within the rail project then it gets picked up automatically and >> the default watir.steps generated by the rails generator work without >> further modification to the env.rb file. >> >> I was unable to discover how to employ fixtures. I found and read this >> thread: http://www.ruby-forum.com/topic/167716 but the final >> recommendation "Fixtures.create_fixtures("spec/fixtures", "entities")" >> did not work for me. It did not raise an error but it did not load the >> fixture either. The TestUnit syntax of "fixtures :model" throws an >> undefined method error which is passing strange given test_help.rb''s >> require ''active_record/fixtures''. Nonetheless, TestUnit syntax like >> assert_something(argument,...argument,message) works fine. >> >> Anyway, a most enlightening code crawl. >> -- >> Posted via http://www.ruby-forum.com/. >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users >> > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
No arguments there! Just curious why it didn''t work...
FWIW - I just did it and it seemed OK...
/features/steps/holiday_steps.rb
...
Then /^there should be 2 nodes in the control group$/ do
Fixtures.create_fixtures("/../../test/fixtures",
"holiday_schedules")
end
...
/test/fixtures/holiday_schedules.yml
one:
for_year: 2020
No moaning from Cucumber though it complained as I was getting the
path and table name right. YMMV.
Thanks again, very much.
Tim
On Wed, Nov 26, 2008 at 3:32 PM, Andrew Premdas <apremdas at gmail.com>
wrote:> Fixtures = yuk!!
>
> Try object_daddy or maybe factory_girl instead :)
>
> Andrew
>
> 2008/11/26 Tim Walker <walketim at gmail.com>:
>> Great post James. Very, helpful. Perhaps should be on the cucumber
>> Wiki? I hope someone follows up on the load fixtures question. Lots to
>> go play with now!!! Tim
>>
>> On Wed, Nov 26, 2008 at 11:04 AM, James Byrne <lists at
ruby-forum.com> wrote:
>>> Tim Walker wrote:
>>>> Question: In Cucumber when you''re writing code to
satisfy steps and
>>>> accessing the model objects directly, what support for asserts,
>>>> responses, etc.
>>>> do people use. (the equivalent of ActionController::TestCase
and
>>>> ActiveSupport::TestCase), Fixtures, etc.
>>>>
>>>> Thanks,
>>>>
>>>> T
>>>
>>> This question prompted a really interesting journey through
cucumber for
>>> me. I now have a much firmer, if still very limited, grasp of what
is
>>> happening.
>>>
>>> When one uses "ruby script/generate cucumber" at the
rails project root
>>> then the script generates (among other things) a
features/support/env.rb
>>> file. This file contains (in part):
>>>
>>> require ''cucumber/rails/world''
>>> require ''cucumber/rails/rspec''
>>>
>>> The cucumber gem location lib/cucumber/rails/rspec contains
>>>
>>> rspec.rb
>>> world.rb
>>>
>>> and rspec.rb has this:
>>>
>>> require ''spec''
>>> require ''spec/rails''
>>>
>>> Next, world.rb makes a conditional reference (almost universally
met in
>>> a Rails project - Do you use ActiveRecord?) to testunit via
>>> /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/test_help.rb. Now that
file
>>> contains:
>>>
>>> require ''test/unit''
>>> require ''active_support/test_case''
>>> require ''active_record/fixtures''
>>> require ''action_controller/test_case''
>>> require ''action_controller/integration''
>>> require ''action_mailer/test_case'' if
defined?(ActionMailer)
>>>
>>> So, it appears that when you generate the cucumber infrastructure
via
>>> the rails generator then you get rspec ''should''
''should_not'' and rails
>>> testunit assert_* support. As previously discussed in this thread,
>>> adding other testing harnesses is a fairly straight forward
procedure
>>> best done in the aforementioned support/env.rb. For example,
adding
>>> watir gem support is done via this:
>>>
>>> require ''webrat'' if !defined?(Webrat)
>>>
>>> I gather from allusions made elsewhere that if watir is installed
as a
>>> plugin within the rail project then it gets picked up automatically
and
>>> the default watir.steps generated by the rails generator work
without
>>> further modification to the env.rb file.
>>>
>>> I was unable to discover how to employ fixtures. I found and read
this
>>> thread: http://www.ruby-forum.com/topic/167716 but the final
>>> recommendation
"Fixtures.create_fixtures("spec/fixtures",
"entities")"
>>> did not work for me. It did not raise an error but it did not load
the
>>> fixture either. The TestUnit syntax of "fixtures :model"
throws an
>>> undefined method error which is passing strange given
test_help.rb''s
>>> require ''active_record/fixtures''. Nonetheless,
TestUnit syntax like
>>> assert_something(argument,...argument,message) works fine.
>>>
>>> Anyway, a most enlightening code crawl.
>>> --
>>> Posted via http://www.ruby-forum.com/.
>>> _______________________________________________
>>> rspec-users mailing list
>>> rspec-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/rspec-users
>>>
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
>>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>
Tim Walker wrote:> FWIW - I just did it and it seemed OK... > > /features/steps/holiday_steps.rb > ... > Then /^there should be 2 nodes in the control group$/ do > Fixtures.create_fixtures("/../../test/fixtures", "holiday_schedules") > end > ... > > /test/fixtures/holiday_schedules.yml > one: > for_year: 2020 > > No moaning from Cucumber though it complained as I was getting the > path and table name right. YMMV.That is probably my problem. I did not pass it the correct path and file name so the reason the fixture was not loaded was because it did not exist. -- Posted via http://www.ruby-forum.com/.