Hey all, Here''s a question for everyone: how do you keep your mocks and stubs consistent with your real objects? I''m dipping my toes into the RSpec world, and it''s absolutely delightful (running dozens of tests in less than a second is nice as well), but I''m concerned about my ability to keep the mocks in my specs consistent with the behavior of their real-world equivalents. I''m writing specs for a controller in a Rails application, and I''ve got stuff like this: ComplicatedObject.should_receive(:execute).and_return(800.00) If I change ComplicatedObject.execute down the line to return something besides a float, this spec will still pass, and the controller will only fail in real life. So I could have full test coverage, green lights across the board, and massive amounts of bugs in the way my classes interact. Is there any mechanism in RSpec to keep this from happening? Any methodology beyond "be careful"? Should I even be mocking my own objects? How do you manage this? -- Coda Hale http://blog.codahale.com
On 12/18/06, Coda Hale <coda.hale at gmail.com> wrote:> Hey all, > > Here''s a question for everyone: how do you keep your mocks and stubs > consistent with your real objects? > > I''m dipping my toes into the RSpec world, and it''s absolutely > delightful (running dozens of tests in less than a second is nice as > well), but I''m concerned about my ability to keep the mocks in my > specs consistent with the behavior of their real-world equivalents. > > I''m writing specs for a controller in a Rails application, and I''ve > got stuff like this: > > ComplicatedObject.should_receive(:execute).and_return(800.00) > > If I change ComplicatedObject.execute down the line to return > something besides a float, this spec will still pass, and the > controller will only fail in real life. So I could have full test > coverage, green lights across the board, and massive amounts of bugs > in the way my classes interact. > > Is there any mechanism in RSpec to keep this from happening? Any > methodology beyond "be careful"? Should I even be mocking my own > objects? > > How do you manage this?The isolated unit level testing that you''re getting from using mocks should (MUST) be coupled with some level of integration testing. Right now I''m using either rails integration testing and/or selenium. David> > -- > Coda Hale > http://blog.codahale.com > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
"The isolated unit level testing that you''re getting from using mocks should (MUST) be coupled with some level of integration testing. Right now I''m using either rails integration testing and/or selenium." I have similar concerns and this doesn''t quite address them. He asked how to manage changes that affect mocks that could be in many places. It seems like something that would be very easy to forget or miss. Also, catching the errors at the integration level is forcing yourself to test the same functionality twice. I''m just starting with rspec and my standard tests in rails would test functionality once at the rails unit test level. At the integration level I would assume the functionality worked and mainly test that the controllers and views handled different cases (i.e. errors) properly. I know this isn''t ideal, but I''m wondering if the duplication of work, both in testing and in mock building, is worth the effort on medium sized projects done by small teams. -- View this message in context: http://www.nabble.com/How-to-keep-mocks-consistent--tf2842806.html#a7996667 Sent from the rspec-users mailing list archive at Nabble.com.
On 12/20/06, jshen <jaydonnell at yahoo.com> wrote:> > "The isolated unit level testing that you''re getting from using mocks > should (MUST) be coupled with some level of integration testing. > Right now I''m using either rails integration testing and/or selenium." > > I have similar concerns and this doesn''t quite address them. He asked how to > manage changes that affect mocks that could be in many places.I wasn''t clear about this. I think that you just have to absorb that risk if you chose to take this approach. Because I combine very granular and very coarse testing in the same project, I''m comfortable with the idea that what isn''t caught by one will be caught by the other. For the most part. We are, after all, talking about automated tests. You still need exploratory testing, but that''s another matter.> It seems like > something that would be very easy to forget or miss. Also, catching the > errors at the integration level is forcing yourself to test the same > functionality twice.The coverage you get from end to end testing should touch everything in the system. If everything isn''t touched, then why is it in your system? That said, a world filled with nothing but end to end tests makes it very difficult to understand where problems lie. It also doesn''t provide the granular design assistance that you get from writing specs before code. As soon as you enter a world in which you have both granular specs and end to end tests, you''re going to be testing everything twice. But you do so in very different ways which help each other to gain a better sense of coverage (and therefore security that your code works). So yes, you do cover things twice. But I view that as a GOOD thing because it does so in different ways.> > I''m just starting with rspec and my standard tests in rails would test > functionality once at the rails unit test level. At the integration level I > would assume the functionality worked and mainly test that the controllers > and views handled different cases (i.e. errors) properly. I know this isn''t > ideal, but I''m wondering if the duplication of work, both in testing and in > mock building, is worth the effort on medium sized projects done by small > teams.The benefit you get from using mocks and stubs to isolate the small parts of the system from other parts in a test is that your tests/specs run FAST! That means you run them more often. That means you get rapid feedback when you introduce bugs. That means you spend less time trying to track them down. Some bugs won''t be caught until you run your integration suite, so this is not perfect. But in my experience, in the end the cost of the duplication you''re talking about is seriously outweighed by the ability to keep moving, adjust to changes, and confidently deliver robust software without spending your life deciding whether to step over or into.> -- > View this message in context: http://www.nabble.com/How-to-keep-mocks-consistent--tf2842806.html#a7996667 > Sent from the rspec-users mailing list archive at Nabble.com. > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
The benefit you get from using mocks and stubs to isolate the small parts of the system from other parts in a test is that your tests/specs run FAST! This is the big draw for me. One of my current rails projects is taking around 20 minutes to run the non integration tests. This is way to long for the reasons you point out. I also love the rspec syntax and mock/stub features. I''ve only been playing with rspec for the past day and a half so I''m probably hitting all the usual beginner hiccups. p.s. - the rspec site/documentation is excellent. You guys did an amazing job on that. the mocks/stubs could use some complete examples (i.e. use in an actual spec) :) -- View this message in context: http://www.nabble.com/How-to-keep-mocks-consistent--tf2842806.html#a8003101 Sent from the rspec-users mailing list archive at Nabble.com.