A couple months ago I asked how stories and specs might impact each other. [1] If you look at Dan North''s example of what''s in a story [2], and you imagine using the spec framework to drive the design, you can probably imagine a significant bit of overlap in the two. Is that a bad thing? I''m not sure. It has made me a bit uncomfortable though, and I''ve kept that question in the back of my mind these past few months. (please forgive any mistakes in the code, I just typed it all into this email) The traditional way of writing specs has been to organize them around context, or the fixture used. This leads to specs that look like describe Stack, "empty" do it "should be empty" do Stack.new.should be_empty end it "should blow up when sent pop" do lambda { Stack.new.pop }.should raise_error(StackEmptyError) end end describe Stack, "with one item" do before(:each) do @stack = Stack.new @stack.push :foo end it "should not be empty" do @stack.should_not be_empty end it "should return the item when sent pop" do @stack.pop.should == :foo end end One of the guys at work showed me a different style of writing specs. He organizes them mostly around methods. So his spec would look more like: describe Stack do describe "#empty?" do it "should be true with no items" do Stack.new.should be_empty end it "should be false with one item" do s = Stack.new s.push :foo s.should_not be_empty end end describe "#pop" do it "should blow up with no items" do lambda { Stack.new.pop }.should raise_error(StackEmptyError) end it "return the last item added" do s = Stack.new s.push :foo @stack.pop.should == :foo end end end At first I didn''t like it so much. It goes against the way I''ve used for so long, which is to organize the specs around fixtures. Second, I''m concerned that in using this style, one might focus more on the behavior of a method, rather than the object as a whole. I''ve slowly been coming around to it though. I think the method-focused approach has some significant benefits for the developer. It serves as excellent documentation. If I want to use #pop, I can just look at the spec and see exactly how it''s going to behave. If I make any modifications, I know exactly where it goes. Contrast this with the context approach, where I might have to hunt through a couple example groups to find all the examples that use the method. I think writing specs with more focus on methods ends up being a lot more maintainable in the long run. My issues with that approach were that it doesn''t follow tradition, and the nebulous "what if I forget how to BDD well?" The benefits are far more concrete and practical. I can use stories to express stuff at a high level, in terms of the domain, and write these method-focused examples to really help me out as a developer. Which is how the benefits are supposed to be derived in the first place. Interestingly, if you take a look at Dan''s example, you''ll see that it is based around one method. That sort of throws me for a loop. However at a high level, it''s actually based around one feature that the customer wants. The fact that it''s all the same #withdraw method under the hood is mostly coincidental. I kind of lost steam towards the end here :) I''d love to know what you guys think. Pat [1] http://rubyforge.org/pipermail/rspec-users/2007-November/004455.html [2] http://dannorth.net/whats-in-a-story
On Jan 11, 2008 3:16 PM, Pat Maddox <pergesu at gmail.com> wrote:> A couple months ago I asked how stories and specs might impact each > other. [1] If you look at Dan North''s example of what''s in a story > [2], and you imagine using the spec framework to drive the design, you > can probably imagine a significant bit of overlap in the two. Is that > a bad thing? I''m not sure. It has made me a bit uncomfortable > though, and I''ve kept that question in the back of my mind these past > few months. > > (please forgive any mistakes in the code, I just typed it all into this email) > > The traditional way of writing specs has been to organize them around > context, or the fixture used. This leads to specs that look like > > describe Stack, "empty" do > it "should be empty" do > Stack.new.should be_empty > end > > it "should blow up when sent pop" do > lambda { Stack.new.pop }.should raise_error(StackEmptyError) > end > end > > describe Stack, "with one item" do > before(:each) do > @stack = Stack.new > @stack.push :foo > end > > it "should not be empty" do > @stack.should_not be_empty > end > > it "should return the item when sent pop" do > @stack.pop.should == :foo > end > end > > One of the guys at work showed me a different style of writing specs. > He organizes them mostly around methods. So his spec would look more > like: > > describe Stack do > describe "#empty?" do > it "should be true with no items" do > Stack.new.should be_empty > end > > it "should be false with one item" do > s = Stack.new > s.push :foo > s.should_not be_empty > end > end > > describe "#pop" do > it "should blow up with no items" do > lambda { Stack.new.pop }.should raise_error(StackEmptyError) > end > > it "return the last item added" do > s = Stack.new > s.push :foo > @stack.pop.should == :foo > end > end > end > > At first I didn''t like it so much. It goes against the way I''ve used > for so long, which is to organize the specs around fixtures. Second, > I''m concerned that in using this style, one might focus more on the > behavior of a method, rather than the object as a whole. > > I''ve slowly been coming around to it though. I think the > method-focused approach has some significant benefits for the > developer. It serves as excellent documentation. If I want to use > #pop, I can just look at the spec and see exactly how it''s going to > behave. If I make any modifications, I know exactly where it goes. > Contrast this with the context approach, where I might have to hunt > through a couple example groups to find all the examples that use the > method. I think writing specs with more focus on methods ends up > being a lot more maintainable in the long run. > > My issues with that approach were that it doesn''t follow tradition, > and the nebulous "what if I forget how to BDD well?" The benefits are > far more concrete and practical. I can use stories to express stuff > at a high level, in terms of the domain, and write these > method-focused examples to really help me out as a developer. Which > is how the benefits are supposed to be derived in the first place. > > Interestingly, if you take a look at Dan''s example, you''ll see that it > is based around one method. That sort of throws me for a loop. > However at a high level, it''s actually based around one feature that > the customer wants. The fact that it''s all the same #withdraw method > under the hood is mostly coincidental. > > I kind of lost steam towards the end here :) I''d love to know what > you guys think.Personally, I think whatever works for you is fine as long as a) it''s focused on behaviour in the exampes and b) ... well ... it works for you! As for the overlap between stories/specs - you need to understand what Acceptance Test Driven Development and ATDPlanning are all about to appreciate that there will, and *should* be overlap. The two tools serve completely different purposes. At least that''s their target. If you use them differently, you won''t lose your license :) But the target of stories are system level descriptions of behaviour. This will inevitably appear to have some overlap with the specs for the outermost layers of the system. But when you start refactoring, those object specifications are going to change - the system specs (stories) should NOT. At least not as a result of refactoring. Conversely, once your application has some legs to it, as stories change due to changing requirements, you''ll find that some stories change without ever needing to change any of the object specs. That''s all for now. FWIW, David> > Pat > > > [1] http://rubyforge.org/pipermail/rspec-users/2007-November/004455.html > [2] http://dannorth.net/whats-in-a-story > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
I''m going to hijack this a bit :) On Jan 11, 2008 1:25 PM, David Chelimsky <dchelimsky at gmail.com> wrote:> > But the target of stories are system level descriptions of behaviour. > This will inevitably appear to have some overlap with the specs for > the outermost layers of the system. But when you start refactoring, > those object specifications are going to change - the system specs > (stories) should NOT. At least not as a result of refactoring.In the first thread I linked to, I said something along the lines of "maybe stories are better for refactoring, since you don''t have to change them." Examples, on the other hand, sometimes have to be changed when refactoring, particularly if you use mocks. However, stories are probably too slow for refactoring, and some of the problems with refactoring with examples are simply a matter of lacking tool support, which should eventually be fixed. I don''t have any problem with that. I do things that way, and I get my work done just fine. However, I''m having a tough time clarifying my position when talking to people who believe that unit tests should not have to change when refactoring either. I don''t think they''re wrong, actually. It''s just a different approach and I''d like to know how to bridge that communication gap better. Pat
On Jan 11, 2008 3:43 PM, Pat Maddox <pergesu at gmail.com> wrote:> I''m going to hijack this a bit :) > > On Jan 11, 2008 1:25 PM, David Chelimsky <dchelimsky at gmail.com> wrote: > > > > But the target of stories are system level descriptions of behaviour. > > This will inevitably appear to have some overlap with the specs for > > the outermost layers of the system. But when you start refactoring, > > those object specifications are going to change - the system specs > > (stories) should NOT. At least not as a result of refactoring. > > In the first thread I linked to, I said something along the lines of > "maybe stories are better for refactoring, since you don''t have to > change them."Keep in mind that in Java, where I think refactoring really grew in its formality as part of the process, the tools make the changes in your tests too. When you change a method signature the tool changes it everywhere. I don''t recall, ever, seeing anyone complaining about having to change tests when refactoring in java. So this is a price we pay for the benefits of early adoption in our language of choice.> Examples, on the other hand, sometimes have to be > changed when refactoring, particularly if you use mocks. > > However, stories are probably too slow for refactoring, and some of > the problems with refactoring with examples are simply a matter of > lacking tool support, which should eventually be fixed.Right - missed this on the first read - that''s what I''m talking about in the paragraph above.> > I don''t have any problem with that. I do things that way, and I get > my work done just fine. However, I''m having a tough time clarifying > my position when talking to people who believe that unit tests should > not have to change when refactoring either. I don''t think they''re > wrong, actually.I do. If you read the refactoring book, every refactoring has a "now change all the clients" step - tests are clients that have to be changed. It''s just part of the deal. The thing is that, ideally, you don''t want to have to make changes to the tests for object A when you''re refactoring B. WDYT?> It''s just a different approach and I''d like to know > how to bridge that communication gap better. > > Pat > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
On Jan 11, 2008 1:48 PM, David Chelimsky <dchelimsky at gmail.com> wrote:> On Jan 11, 2008 3:43 PM, Pat Maddox <pergesu at gmail.com> wrote: > > I don''t have any problem with that. I do things that way, and I get > > my work done just fine. However, I''m having a tough time clarifying > > my position when talking to people who believe that unit tests should > > not have to change when refactoring either. I don''t think they''re > > wrong, actually. > > I do. If you read the refactoring book, every refactoring has a "now > change all the clients" step - tests are clients that have to be > changed. It''s just part of the deal.That makes sense. I''ve never really thought of it that way.> The thing is that, ideally, you don''t want to have to make changes to > the tests for object A when you''re refactoring B. > > WDYT?Yeah, I buy that. Not everyone does though. Or at least not everyone feels that it''s a particularly important goal. Pat
>> The thing is that, ideally, you don''t want to have to make changes to >> the tests for object A when you''re refactoring B. >> >> WDYT?> Yeah, I buy that. Not everyone does though. Or at least not everyone > feels that it''s a particularly important goal.I think the fear many of us classicists have is that if A uses B (and Bs interface changes) then A will pass but it shouldn''t. We also have to go around updating all the mocks for tests that use B. With a state based approach we wouldn''t have to change any tests assuming we''ve fixed the outer methods already. If we haven''t then our tests will fail and tell us we need to. I know, our integration tests should catch this. I''m curious what you guys mean by "integration tests" in this case. For example, in a rails app are the ''integration tests'' always or usually testing the full rails stack by sending web requests and verifying against the response? I.e. simulating a user interaction? ____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs
On Jan 11, 2008 5:11 PM, Jay Donnell <jaydonnell at yahoo.com> wrote:> >> The thing is that, ideally, you don''t want to have to make changes to > >> the tests for object A when you''re refactoring B. > >> > >> WDYT? > > > Yeah, I buy that. Not everyone does though. Or at least not everyone > > feels that it''s a particularly important goal. > > I think the fear many of us classicists have is that if A uses B (and Bs interface changes) then A will pass but it shouldn''t. We also have to go around updating all the mocks for tests that use B. With a state based approach we wouldn''t have to change any tests assuming we''ve fixed the outer methods already. If we haven''t then our tests will fail and tell us we need to.As David has mentioned before, that''s a matter of tool support. Consider how this works if you''re using Java. You might have some interface that looks like: public interface Account { void deposit(int amount); void withdraw(int amount); } You''ve got a BankAccount class that implements it, and in tests that depend on the Account role, you use mock objects. If you were to rename deposit to credit, and withdraw to debit, the first thing that would happen is that the compiler would complain that BankAccount doesn''t implement the Account interface anymore. So you can lean on the compiler to fix it. Even better, you would use a refactoring tool, and that''d look for any classes that implement Account and change their method names. Conceptually, the same exact thing is possible and desirable in Ruby. There are some extra hoops to jump through because we can''t lean on the compiler like we can in Java. Otoh, we write Ruby a lot faster than Java, and we don''t want to stab ourselves. So at this point it''s still a worthwhile tradeoff (to me). One problem is that in a language with interfaces, roles are made explicit. In Ruby, any object can play any role, as long as it implements the necessary protocol. Maybe we could do something at the framework level to make the roles more explicit. Something like describe AccountService, " transferring between two accounts" do before(:each) do @source = mock_role(:account) @target = mock_role(:account) @service = AccountService.new end it "should debit the source account" do @source.should_receive(:debit).with(50) @service.transfer 50, @source, @target end it "should credit the target account" do @target.should_receive(:credit).with(50) @service.transfer 50, @source, @target end end Then you have a spec somewhere that verifies that the Account class fulfills the account role. It''s an interesting idea, and I remember it being bounced around a while back, as something like "mock integration mode." There are plenty of obstacles though...you can''t just use respond_to? on an instance of a class, because objects could use method_missing, be decorated with behavior at runtime, etc. It also doesn''t seem practical to try to define instances manually and then swap them in to tests. You probably have to create so many objects for different edge cases that it makes more sense just to use state-based verification in the first place. This latter set of issues is just conjecture though, as I''ve never given it a shot myself. At this point, we can just use integration tests to catch the things we miss, and be thankful that even large Rails projects seldom pass 25k lines of code :) If demand increases for solutions to the mock maintainability problem, one will arise.> I know, our integration tests should catch this. I''m curious what you guys mean by "integration tests" in this case. For example, in a rails app are the ''integration tests'' always or usually testing the full rails stack by sending web requests and verifying against the response? I.e. simulating a user interaction?In Rails apps, yes, a lot of the stories would go through the full Rails stack. They don''t have to though. User stories represent some facet of behavior which, once completed, provides value to the customer. In short, working software. By definition, then, stories must use real production implementations of objects instead of mocks. This means they work quite nicely as integration tests as well. Pat
> If you were to rename deposit to credit, and withdraw to debitI''m more concerned with adding a parameter or changing the default value of a parameter. My fear is that given enough time and size there will be a fair number of inconsistent mocks or mocks that aren''t doing any real testing. It feels a bit like forced exception handling in java.> User stories represent some facet of behavior which, once completed, > provides value to the customer. In short, working software. By > definition, then, stories must use real production implementations of > objects instead of mocks. This means they work quite nicely as > integration tests as well.Does anyone know of any open source ruby apps (preferably rails) that use good BDD so I can see how it''s done right rather than ask a bunch of questions? ____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
Jay Donnell wrote:> > Does anyone know of any open source ruby apps (preferably rails) that use good BDD so I can see how it''s done right rather than ask a bunch of questions? > > > >Hmm.. I don''t know of any rails apps that are open source that you could look at.. But I know datamapper (http://www.datamapper.org/) is using rspec. Also, be sure to look at rspec''s code if you haven''t already. I have learned a great deal by looking over its use of itself. :) -Ben
On Jan 12, 2008, at 3:33 PM, Ben Mabey wrote:> Jay Donnell wrote: >> >> Does anyone know of any open source ruby apps (preferably rails) >> that use good BDD so I can see how it''s done right rather than ask >> a bunch of questions? >> >> >> >> > > Hmm.. I don''t know of any rails apps that are open source that you > could > look at.. But I know datamapper (http://www.datamapper.org/) is using > rspec. Also, be sure to look at rspec''s code if you haven''t > already. I > have learned a great deal by looking over its use of itself. :)Yep. Merb, too, as well as Rubinius, which uses "mspec" - a scaled down version of rspec. Plus - the specs should be pretty easy to understand, being that they are of ruby itself. I know Rick Olson has started using rspec - you might want to look at his plugins. I also have a few projects out there using rspec: DatedBackup, and FixtureReplacement, both on rubyforge. Hope that helps, Scott
http://synthesis.rubyforge.org/ *just* showed up in my feedreader (well, the links at the bottom did) On Jan 11, 2008 9:28 PM, Jay Donnell <jaydonnell at yahoo.com> wrote:> > > If you were to rename deposit to credit, and withdraw to debit > > I''m more concerned with adding a parameter or changing the default value of a parameter. My fear is that given enough time and size there will be a fair number of inconsistent mocks or mocks that aren''t doing any real testing. It feels a bit like forced exception handling in java.Yeah. I played with Synthesis a bit, and it doesn''t handle this case. It''s very early on though. I''m not sure that you''ll end up with a number of inconsistent mocks...because if you have integration tests covering everything, then you''ll know pretty quickly when your tests aren''t up to par. But it''s not like you''d have a bunch of passing tests and broken production code, and deploy that out. Using mocks without acceptance tests is like using Ruby without any tests. It''s just stupid.> Does anyone know of any open source ruby apps (preferably rails) that use good BDD so I can see how it''s done right rather than ask a bunch of questions?Not really. Rick Olsen has a Rails example app that uses specs. There are a bunch of published plugins that use specs. I''m working on a bliki thing in my spare time (which means never) which will be open source. Pat
>because if you have integration tests covering everything, >then you''ll know pretty quickly when your tests aren''t up to par.If we have integration tests that cover everything then why have another set of tests? At that point wouldn''t it simply be duplication and wouldn''t it create more work for refactorings and such? ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping
On Jan 12, 2008 8:35 PM, Jay Donnell <jaydonnell at yahoo.com> wrote:> >because if you have integration tests covering everything, > >then you''ll know pretty quickly when your tests aren''t up to par. > > If we have integration tests that cover everything then why have another set of tests? At that point wouldn''t it simply be duplication and wouldn''t it create more work for refactorings and such?Jay - are you familiar with Acceptance Test Driven Development/Planning and/or Test Driven Development? Cheers, David
> Jay - are you familiar with Acceptance Test Driven > Development/Planning and/or Test Driven Development?I am with TDD and Acceptance testing (not sure if this is the same as Acceptance Test Driven Development) . I use a more classic approach to use Martin Fowler''s terminology which is why I asked earlier if there are any open source rails apps that use rspec the way it''s intended so I can explore it that way rather than asking a million questions. ____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs