rspec users - Aug 2008 - Four Question From an RSpec Baby - Give me something to chew

The project has been written - around 10,000 lines of code, but
certainly less than that to add tests to. There has been no official
testing put into place, and I''d very much like to implement RSpec into
the project for a few reasons, but mostly to setup solid examples for
the behavior of the application.

It was suggested to me to begin with Unit Tests. The User Model is
almost 1000 lines with plenty of methods and no coverage, so I figured I
would start there. After installing Rcov to track my progress, I wrote
about 10 passing examples that were, I guess, pretty trivial - the
methods I tested all dealt directly with an "instantiated user
object"(is that the correct terminology?).

I have been writing the user examples in the order that a real user
might usually take (A User who: is activating his account, is logging
in, forgets his password and resets it, wants to use a new email
address, wants to change his username.)

Now, I''m not sure of the next step in the process, I''m stuck!
The method
(upgrade) takes a Payment object parameter. Inside the upgrade method,
some payment object attributes are set (user_id, payment_type) and then
saved. Then, the user object that is being upgraded has some attributes
that are set (payment_id, member, member_since) and finally the user is
saved, ending the method. And to top it all off, this method takes place
in a transaction.

For a visual guide:
  def upgrade(payment)
    transaction do
      payment.user_id = self.id
      payment.payment_type = Payment::SUBSCRIPTION_PAYMENT_TYPE
      return false unless (payment.save and payment.external_id)
      self.subscription_id = payment.external_id
      self.payment_id = payment.id
      self.member = true
      self.member_since = AppLib.today_utc
      self.save
      return true
    end
  end

Now that you have sufficient back story (I hope), here are my questions:

1.) Do I need to use any mocking/stubbing in this example (or in Unit
Tests)
2.) Is it wrong to access multiple objects (user and payment in my
example) in a Unit Test example?
3.) Would you mind showing me an example of how you might implement a
spec for this method.
4.) Could you PLEASE PLEASE PLEASE guide me to a resource that helped
you the most with figuring out Unit Testing with RSpec. Not limited to
books or blog posts... good source code examples might be helpful.

Hopefully you can see my sincere want to know more and I wish that a
simple "want" will blossom into a realization of a "need" to
test test
test.
Thanks for any help you can provide me on my journey - I have a long way
ahead of me.

Lake
-- 
Posted via http://www.ruby-forum.com/.

On 2008-08-27, at 13:20, Lake Denman wrote:
> For a visual guide:
>  def upgrade(payment)
>    transaction do
>      payment.user_id = self.id
>      payment.payment_type = Payment::SUBSCRIPTION_PAYMENT_TYPE
>      return false unless (payment.save and payment.external_id)
>      self.subscription_id = payment.external_id
>      self.payment_id = payment.id
>      self.member = true
>      self.member_since = AppLib.today_utc
>      self.save
>      return true
>    end
>  end

Hi Lake.
> 1.) Do I need to use any mocking/stubbing in this example (or in Unit
> Tests)
You need to use whatever you think is most suitable for your  
application. I''m sure many people on here would use mocks and stubs  
extensively throughout that method. For example, they might mock the  
entire "payment" object, and stub out AppLib.today_utc .
> 2.) Is it wrong to access multiple objects (user and payment in my
> example) in a Unit Test example?
I don''t think so, but I''m not a unit test expert.
> 3.) Would you mind showing me an example of how you might implement a
> spec for this method.
User upgrade
- can succeed if the payment amount is correct
- fails if the payment amount is incorrect
- can succeed if the payment type is valid
- fails if the payment type if invalid
etc..

I''m new to RSpec though, so there''s probably a
"better"/clearer/more
efficient way of doing it.
> 4.) Could you PLEASE PLEASE PLEASE guide me to a resource that helped
> you the most with figuring out Unit Testing with RSpec. Not limited to
> books or blog posts... good source code examples might be helpful.
Why not use BDD?  =)
http://blog.davidchelimsky.net/articles/2007/05/14/an-introduction-to-rspec-part-i
http://blog.davidchelimsky.net/articles/2006/11/06/view-spec-tutorial
http://blog.withoutincident.com/2007/5/29/blogification-part-ii-rspec-bdd-and-authors
> Hopefully you can see my sincere want to know more and I wish that a
> simple "want" will blossom into a realization of a
"need" to test test
> test.
Just search, read, try, and ask.
> Thanks for any help you can provide me on my journey - I have a long  
> way
> ahead of me.
That doesn''t answer all of your questions directly, but hopefully  
it''ll be helpful in one way or another. Cheers,
Nick

I have another, more general tip - read this book:
http://www.amazon.com/Working-Effectively-Legacy-Robert-Martin/dp/ 
0131177052

Michael''s definition of Legacy Code is simply ''code that
isn''t
covered by tests''. So... err... that would be your whole app!

The book suggests chipping away at the lack of test coverage by  
writing tests (or specs) when you
  - change code to add a feature
  - need to fix a bug (write a test / spec that fails because of the  
bug... fix the bug... high five)

So rather than trying to tackle all 1000 lines of you user model at  
once, you have some practical motivation. This is also a great habit  
to get into for future work.

Working with legacy code is particularly tricky as it may be hard to  
test, since it wasn''t written with testing in mind. The book has some  
great guidance in the techniques you can use to introduce
''seams''
between parts of the code that you''ll need to test in isolation.

cheers,
Matt
----
http://blog.mattwynne.net
http://songkick.com

In case you wondered: The opinions expressed in this email are my own  
and do not necessarily reflect the views of any former, current or  
future employers of mine.



On 27 Aug 2008, at 19:25, Nick Hoffman wrote:
> On 2008-08-27, at 13:20, Lake Denman wrote:
>> For a visual guide:
>>  def upgrade(payment)
>>    transaction do
>>      payment.user_id = self.id
>>      payment.payment_type = Payment::SUBSCRIPTION_PAYMENT_TYPE
>>      return false unless (payment.save and payment.external_id)
>>      self.subscription_id = payment.external_id
>>      self.payment_id = payment.id
>>      self.member = true
>>      self.member_since = AppLib.today_utc
>>      self.save
>>      return true
>>    end
>>  end
>
>
> Hi Lake.
>
>> 1.) Do I need to use any mocking/stubbing in this example (or in Unit
>> Tests)
>
> You need to use whatever you think is most suitable for your  
> application. I''m sure many people on here would use mocks and
stubs
> extensively throughout that method. For example, they might mock  
> the entire "payment" object, and stub out AppLib.today_utc .
>
>> 2.) Is it wrong to access multiple objects (user and payment in my
>> example) in a Unit Test example?
>
> I don''t think so, but I''m not a unit test expert.
>
>> 3.) Would you mind showing me an example of how you might implement a
>> spec for this method.
>
> User upgrade
> - can succeed if the payment amount is correct
> - fails if the payment amount is incorrect
> - can succeed if the payment type is valid
> - fails if the payment type if invalid
> etc..
>
> I''m new to RSpec though, so there''s probably a
"better"/clearer/
> more efficient way of doing it.
>
>> 4.) Could you PLEASE PLEASE PLEASE guide me to a resource that helped
>> you the most with figuring out Unit Testing with RSpec. Not  
>> limited to
>> books or blog posts... good source code examples might be helpful.
>
> Why not use BDD?  =)
> http://blog.davidchelimsky.net/articles/2007/05/14/an-introduction- 
> to-rspec-part-i
> http://blog.davidchelimsky.net/articles/2006/11/06/view-spec-tutorial
> http://blog.withoutincident.com/2007/5/29/blogification-part-ii- 
> rspec-bdd-and-authors
>
>> Hopefully you can see my sincere want to know more and I wish that a
>> simple "want" will blossom into a realization of a
"need" to test
>> test
>> test.
>
> Just search, read, try, and ask.
>
>> Thanks for any help you can provide me on my journey - I have a  
>> long way
>> ahead of me.
>
> That doesn''t answer all of your questions directly, but hopefully
> it''ll be helpful in one way or another. Cheers,
> Nick
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users

On Wed, Aug 27, 2008 at 1:20 PM, Lake Denman <lists at ruby-forum.com>
wrote:
> The project has been written - around 10,000 lines of code, but
> certainly less than that to add tests to. There has been no official
> testing put into place, and I''d very much like to implement RSpec
into
> the project for a few reasons, but mostly to setup solid examples for
> the behavior of the application.
>
> It was suggested to me to begin with Unit Tests. The User Model is
> almost 1000 lines with plenty of methods and no coverage, so I figured I
> would start there. After installing Rcov to track my progress, I wrote
> about 10 passing examples that were, I guess, pretty trivial - the
> methods I tested all dealt directly with an "instantiated user
> object"(is that the correct terminology?).
When trying to get a large, existing code base under test, I think
it''s more valuable to begin with some very high-level tests that cover
a lot of ground.  Basically, you''re going for "something
broke" rather
than "this specific thing broke" with these.  As time goes on and you
develop high functional coverage over the code, you can write more
focused tests that will alert you to specific breakages.  Also, keep
in mind that one of the primary benefits of unit testing is as a
design tool.  When you''re retrofitting tests, you don''t get
that
benefit (except for the frequent cases where you go "gosh, this design
sucks!")

However, taking a couple weeks off to write exhaustive tests for your
code base is basically never feasible.  So you have to make tradeoffs.
 The simplest approach is to write tests to cover any code that you''re
changing.  You''ll need to spend some extra time and mental energy
analyzing the various pathways, because the code you''re changing has
dependencies, and there are other dependencies on the code you''re
change, all of which you have to discover and account for.  "Working
Effectively with Legacy Code" by Michael Feathers has great
information on this.

So start off with high-level tests to maximize your value.  At this
point, that means being alerted to regressions.  The lowest level you
should probably go is controller tests...set up some state, hit the
action, verify that the leftover state is what you expect.  Writing
even higher level tests with Cucumber would be an even better idea, I
think.  It covers more of the stack and lets you actually take
pathways through the application.  Actually now that I think about it,
a combo of acceptance tests and controller specs would probably be
best.  Acceptance tests for happy paths, controller specs for testing
other paths at a slightly lower level.

> I have been writing the user examples in the order that a real user
> might usually take (A User who: is activating his account, is logging
> in, forgets his password and resets it, wants to use a new email
> address, wants to change his username.)
>
> Now, I''m not sure of the next step in the process, I''m
stuck! The method
> (upgrade) takes a Payment object parameter. Inside the upgrade method,
> some payment object attributes are set (user_id, payment_type) and then
> saved. Then, the user object that is being upgraded has some attributes
> that are set (payment_id, member, member_since) and finally the user is
> saved, ending the method. And to top it all off, this method takes place
> in a transaction.
>
> For a visual guide:
>  def upgrade(payment)
>    transaction do
>      payment.user_id = self.id
>      payment.payment_type = Payment::SUBSCRIPTION_PAYMENT_TYPE
>      return false unless (payment.save and payment.external_id)
>      self.subscription_id = payment.external_id
>      self.payment_id = payment.id
>      self.member = true
>      self.member_since = AppLib.today_utc
>      self.save
>      return true
>    end
>  end
>
> Now that you have sufficient back story (I hope), here are my questions:
>
> 1.) Do I need to use any mocking/stubbing in this example (or in Unit
> Tests)
Sure, you can.  As I suggested above, you''ll want to get some
high-level functional coverage over the code you''re changing.  But
then you can zoom in and write some unit tests for it...and using mock
objects here will probably point out bad dependencies.
> 2.) Is it wrong to access multiple objects (user and payment in my
> example) in a Unit Test example?
No

> 3.) Would you mind showing me an example of how you might implement a
> spec for this method.
The ideal way is for the object to behave differently once changes
have been made.  But, in Rails, model objects often just shuffle data
around and don''t do anything particularly interesting.  So the
interesting visible behavior is actually in the UI, or at a lower
level, the attributes on the models themselves.  So for this spec
you''d probably just want to run it and make sure that the attributes
are what you expect them to be.  Pretty easy, so I won''t write an
example :)

> 4.) Could you PLEASE PLEASE PLEASE guide me to a resource that helped
> you the most with figuring out Unit Testing with RSpec. Not limited to
> books or blog posts... good source code examples might be helpful.
http://blog.davidchelimsky.net/ is a good starting point.  Read his
articles, and then the blogs listed on the right side of the page.
Check out the Webrat and Merb projects, all their code is RSpec''d

Cheers,
Pat

On Wed, Aug 27, 2008 at 10:20 AM, Lake Denman <lists at ruby-forum.com>
wrote:

> def upgrade(payment)
>    transaction do
>      payment.user_id = self.id
>      payment.payment_type = Payment::SUBSCRIPTION_PAYMENT_TYPE
>      return false unless (payment.save and payment.external_id)
>      self.subscription_id = payment.external_id
>      self.payment_id = payment.id
>      self.member = true
>      self.member_since = AppLib.today_utc
>      self.save
>      return true
>    end
>  end
>
> 2.) Is it wrong to access multiple objects (user and payment in my
> example) in a Unit Test example?
>
I''ll disagree with Pat and say yes (in general). A unit test/spec
should
exercise a unit, which I usually consider to end at the class boundary.

For example, this code relies on Payment#save being correct. If it''s
buggy
(or doesn''t exist), a spec of this User method could very well fail,
even if
there is nothing wrong with the payment method. That''s not a unit test,
in
my view. I would mock out the payment in a spec for this method to ensure
that save and external_id are called, but not rely on them being correct (or
even existing). In a spec for Payment#save, I would make sure that it sets
external_id correctly (if that''s what it''s supposed to do).
The combination
of the two specs would be enough (in my view) to document and test the
behavior of these classes.

I won''t rattle on about the benefits of unit tests, other than to say
that
by testing only one unit at a time, you reduce the odds of that horrible
moment when you change one line of code and 36 tests in 18 other spec files
fail. All that said, pragmatism trumps purity, and if you''re willing to
spec
two "units" in one "unit test," and it makes your life
easier to do so, then
go for it. :)

The other thing I would say is that mocking and stubbing are powerful tools
that you should add to your arsenal as soon as possible. I''ve had
several
coworkers who resisted using them, only to finally achieve that "aha!"
moment later. Your tests get easier to write, and they''re less brittle
to
change.

I might add that I learned much of this from working on a prior codebase of
Pat''s. That doesn''t mean I couldn''t have got it all
wrong, of course. :)

///ark
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20080827/53e0b8b7/attachment.html>

I''ll just second the other suggestions of a) getting some high-level
integration tests over the app; b) doing the detailed specs
just-in-time when you need to make a change; and c) reading Michael
Feathers'' book.  I''ve been in the same position, and the above
three
points helped a lot.

-- 
Avdi

Home: http://avdi.org
Developer Blog: http://avdi.org/devblog/
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

On Aug 27, 2008, at 2:40 PM, Matt Wynne wrote:
> I have another, more general tip - read this book:
>
http://www.amazon.com/Working-Effectively-Legacy-Robert-Martin/dp/0131177052
>
> Michael''s definition of Legacy Code is simply ''code that
isn''t
> covered by tests''. So... err... that would be your whole app!
Well - I think it was zenspider who said legacy code is code either:  
1) you didn''t write, or 2) you didn''t *just* write 20 minutes
ago.
There is something to it.
>
>
> The book suggests chipping away at the lack of test coverage by  
> writing tests (or specs) when you
> - change code to add a feature
> - need to fix a bug (write a test / spec that fails because of the  
> bug... fix the bug... high five)
+1

With my last job, I inherited an 8000 line rails code base with 30  
Test::Unit tests, 25 of which were failing.  I decided to delete  
those, and start a fresh with test suite (under rspec, of course).

Over a few months, without ever going out of my way to write specs for  
legacy code, I had over over 80% coverage (if I remember correctly).   
Just chip away as you would normally, and sooner or later the crap  
code gets replaced.  Don''t get overwhelmed by rcov.  Just write specs  
for everything *you* write, and write specs for anything else you  
touch.  Especially regressions.  Usually you can get in a few extra  
specs when writing a regression that has nothing to do with the bug  
itself (it''s a sort of testing after the fact - almost like proving  
theorems of an existing system).

Honestly, you won''t have time to write new specs - it''s
frustrating /
painful to write specs *after* code, and so it doesn''t usually get  
done (since it''s not any fun).  Just chip away gradually and
you''ll
get there eventually.  If you''re really eager to go off and write  
extra specs, just write really clear specs and then show the HTML  
specdocs to your boss - he''ll probably appreciate it, especially if  
formal specs hadn''t been drafted up previously.

Scott

On 28 Aug 2008, at 04:46, Scott Taylor wrote:
> Especially regressions.  Usually you can get in a few extra specs  
> when writing a regression that has nothing to do with the bug  
> itself (it''s a sort of testing after the fact - almost like
proving
> theorems of an existing system).
The Feathers book talks about writing tests in this situation almost  
the way a scientist might perform experiments on a mysterious  
chemical to try and determine its properties - writing tests to  
validate the assumptions you''re making about the behaviour of the  
class as you read the code and try to understand it. Writing  
executable specs as you do this obviously leaves your team a lot  
better off in the long run than if you just dived in there, figured  
it out, made your change and scarpered.

So, that was a great response to my questions. Thanks a lot, guys.

Nick,
Thanks for your help and your suggestions and your time.
The articles you linked to were helpful, no doubt.

Pat,
I realize now that you''re the guy I first heard about story runner from
with your screencast on it.
I think I saw that about a year ago and I''m just now getting into 
testing.... shame shame shame.
Your insights are warmly welcomed and greatly appreciated. You are right 
when you say that it is never feasible to take weeks off to write code.
Well, it has been one week now and I''ve taken your advice to start 
writing tests for code that I am changing or fixing.
I promptly downloaded Cucumber and Webrat and set up an
''rspec'' branch
for my app in order to cherry pick commits with the master.
All of a sudden, though, I didn''t know anything about Cucumber, Webrat,
or writing "features".
Not only that, I could only find scarce, miniature examples of Cucumber. 
No examples I found had much to do with a rails app, anyhow.
So, I wrote one Feature and that worked out fine, I guess. I just 
haven''t figured out how to re-use a Step, so that''s tripping
me up.
Controller Specs are a different story... I''ve written three controller
specs! Are they good controller specs, Lake? Well, maybe... I''m not 
quite sure.
I guess the "question" turned from "How do I become confident in
my
code? Write Specs!" to "How do I become confident in my Specs?!"

On the positive side, I feel like I am getting stronger with speccing. 
I''ve been using mocking and stubbing like I never have before. I think 
some things are becoming more natural, easier to grasp. I think the 
reason my brain wouldn''t have it was because I was not USING IT, I just
passively read about it... but I think I''ve hit that AHA moment with 
mocking and stubbing - or, at least, I''m steadily on my way to the 
climax of the moment.

Thanks for your suggestions and help! I haven''t read all of Chelimsky, 
or his friend''s blogs, but I will.
I did download the merb-core, though. :)

Matt Wynne,
"Working Effective With Legacy Code" is on the way.
Thanks for taking time out of your day to help out.
Hopefully, I''ll be high fiving much more in the future.
Oh, your latest article on your blog struck a note with me... also it 
led me to Michael Feathers blog, Thanks!

Mark Wilden,
First, your suggestion about mocking and stubbing really encouraged me 
to take up the task of learning about and using them. I feel like I 
actually experienced my AHA moment. I think you are right about testing 
units and keeping a unit, a unit. I''ll certainly use Mocks and Stubs 
when I end up unit testing and then I''ll fondly remember your post.

Avdi Grimm,
Thank you much for your added input. Especially about the book!

Scott Taylor,
Your post encouraged me to relax. The idea of "chipping away" really
hit
home with me especially since I have a... don''t say it... a pickaxe!
You are correct about not having enough time.  I''ll definitely generate
the specdoc for the boss.

----

Hopfully, I didn''t leave anything important out... but now on to the 
mashed peas:

Taking Pat''s suggestion, I downloaded Cucumber - if you
haven''t used it,
try it out. I haven''t used story runner, so there!
Anyways, I downloaded Cucumber and started writing a feature and the 
steps. Here is what I came up with:

http://pastie.org/266264
That feature passes with flying covers. Am I in the right direction with 
it, though?
Unfortunately, my feature fun ran out when I tried to re-use a step name 
in a different step file... Apparantly you can''t duplicate steps?

So, I moved on to a controller spec for a controller called 
"admin/issues". Here is what I came up with... Don''t cry,
it''s ugly:
http://pastie.org/266272

That should be fairly readable, but I don''t really know if I''m
doing the
right stuff.
Any pointers would be very helpful.

Those are the two files that I came up with in the last week. Well, I 
came up with a couple more, but those are the more completed ones. I''m 
really grateful that this stuff is falling in line, or at least I hope 
it is.
Thanks for your time guys.

Lake


-- 
Posted via http://www.ruby-forum.com/.

On 2008-08-27, at 15:25, Mark Wilden wrote:
> The other thing I would say is that mocking and stubbing are  
> powerful tools that you should add to your arsenal as soon as  
> possible. I''ve had several coworkers who resisted using them, only
> to finally achieve that "aha!" moment later. Your tests get
easier
> to write, and they''re less brittle to change.
G''day Mark. I was re-reading this thread and noticed this paragraph of
yours. I''ve been using RSpec and BDD for about 2 months now, and love  
it.

However, I''m not a fan of mocking and stubbing, primarily for two  
reasons:
1) I believe that specs should test behaviour, rather than a  
behaviour''s implementation.
2) Using mocks and stubs causes your specs and implementation to be  
tightly coupled, which often forces you to modify your specs if  
changes occur in the implementation.

However, #2 contradicts what you said about "tests ... [are] less  
brittle to change" when using mocks and stubs. Considering that
I''m
still very new to mocks and stubs, I''m probably missing something  
here. When you have a minute, would you mind countering me?

Thanks!
Nick

On Thu, Sep 4, 2008 at 4:14 PM, Nick Hoffman <nick at deadorange.com>
wrote:

> However, I''m not a fan of mocking and stubbing, primarily for two
reasons:
> 1) I believe that specs should test behaviour, rather than a
behaviour''s
> implementation.

"Behavior" (to me) means "what it does." "What it
does" means how it
interacts with other objects. If you make sure it interacts with other
objects properly (and you make sure those other objects also behave
correctly) you have a confidence-inspiring  spec.

The alternative is to test state, which could be termed "what it is."
I
think you may be saying you prefer that approach, which is completely valid.
Martin Fowler wrote the canonical article on the difference between these
two approaches.

2) Using mocks and stubs causes your specs and implementation to be
tightly
> coupled, which often forces you to modify your specs if changes occur in
the
> implementation.
>
That''s completely true. However, you are shielded from -other-
classes''
implementation. With state-based testing, everything has to be correct all
the way down. With behavior-based testing, you''re only testing one
thing at
a time.

I haven''t been working with mock objects all that long myself, so
anything
I''ve said here is subject to contradiction by someone who really knows
what
they''re talking about. :)

///ark
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20080904/016e912d/attachment-0001.html>

On Thu, Sep 4, 2008 at 7:14 PM, Nick Hoffman <nick at deadorange.com>
wrote:
> On 2008-08-27, at 15:25, Mark Wilden wrote:
>>
>> The other thing I would say is that mocking and stubbing are powerful
>> tools that you should add to your arsenal as soon as possible.
I''ve had
>> several coworkers who resisted using them, only to finally achieve that
>> "aha!" moment later. Your tests get easier to write, and
they''re less
>> brittle to change.
>
> G''day Mark. I was re-reading this thread and noticed this
paragraph of
> yours. I''ve been using RSpec and BDD for about 2 months now, and
love it.
>
> However, I''m not a fan of mocking and stubbing, primarily for two
reasons:
> 1) I believe that specs should test behaviour, rather than a
behaviour''s
> implementation.
This is a misleading statement. Testing the behavior of an object
involves its input and output collaborators. When used appropriately
mocks and stubs act as an agent for design and discovery. They also
allow objects under test to be isolated from other objects that it
depends on (which may not even exist yet).

Collaborators are what should be used to set up with stubs and mock
expectations. This way we can test the behavior of the object under
test given different values returned by its collaborators.


For example:

   class FundsTransfer
      def transfer(amount, source_account, target_account)
         if source_account.balance < amount
            raise "not enough money"
         else
            source_account.deposit amount
            target_account.withdraw amount
      end
   end

Without using mocks or stubs to test the above code I wouldn''t be able
to test the case where the source account doesn''t have enough money or
does have enough money -- unless I already have implemented the
Account class and its withdraw, deposit and balance methods. Taking
the approach of writing the Account class first is an approach that a
lot of developers take. But you build what you think you need before
you actually need it. I prefer to develop from the other direction,
only building what I discover I need to make it work.  Either way
though behavior is being tested, and it is not testing against the
internal state of an object.
> 2) Using mocks and stubs causes your specs and implementation to be tightly
> coupled, which often forces you to modify your specs if changes occur in
the
> implementation.
>
> However, #2 contradicts what you said about "tests ... [are] less
brittle to
> change" when using mocks and stubs. Considering that I''m
still very new to
> mocks and stubs, I''m probably missing something here. When you
have a
> minute, would you mind countering me?
You can write bad tests with mocks/stubs and without mocks/stubs.
Using mocks/stubs doesn''t immediately tightly couple your specs to
your implementation. It does however tie your specs to the external
interface of the object under test. If you change that, then you will
need update your specs.

Consider our FundsTransfer example again:

   class FundsTransfer
      def transfer(amount, source_account, target_account)
         if source_account.balance < amount
            raise "not enough money"
         else
            source_account.deposit amount
            target_account.withdraw amount
      end
   end

If I supply a source_account that has a stubbed out balance of less
than the amount requesting to be transferred am I tightly coupling the
spec to the implementation? No more so then creating a source account
fixture with an amount less than the amount that I am requesting to
transfer. I say this because at some point you need to test the
scenario where funds are requested to be transferred from a source
account that doesn''t have enough money.

Do you really see one of the below examples more tightly coupling the
test to the implementation?

    account = mock("Account", :balance => 0)
    account = Account.new :balance => 0

The difference to me is that using the mock allows me to discover
earlier what interface I want to work with on the Account class. If I
go the non-mock route then I need to make sure I build the Account
class with the interfaces I need first.

I much prefer to work in small steps. Focus on one behaviour, and then
when it works as expected, make sure any collaborators that need to be
implemented are. Then use my integration tests to make sure everything
is wired up correctly together.

That''s just me though,

-- 
Zach Dennis
http://www.continuousthinking.com
http://www.mutuallyhuman.com

Mark and Zach, you both posed great perspectives and gave great  
explanations. Thanks for taking the time to write all of that. I''m  
going to re-read your emails several times over the next couple of  
days and see how the info percolates into my brain. I may even rewrite  
my current specs using mocks and stubs to see how it feels.

Much appreciated, guys!
-Nick

Nick Hoffman wrote:
> On 2008-08-27, at 15:25, Mark Wilden wrote:
>> The other thing I would say is that mocking and stubbing are powerful
>> tools that you should add to your arsenal as soon as possible.
I''ve
>> had several coworkers who resisted using them, only to finally
>> achieve that "aha!" moment later. Your tests get easier to
write, and
>> they''re less brittle to change.
>
> G''day Mark. I was re-reading this thread and noticed this
paragraph of
> yours. I''ve been using RSpec and BDD for about 2 months now, and
love it.
>
> However, I''m not a fan of mocking and stubbing, primarily for two
> reasons:
> 1) I believe that specs should test behaviour, rather than a
> behaviour''s implementation.
> 2) Using mocks and stubs causes your specs and implementation to be
> tightly coupled, which often forces you to modify your specs if
> changes occur in the implementation.
I''m completely with Zach on this subject, so I won''t repeat
what he has
stated.  I will however point out that many people who avoid using mocks
simply based on the "test the behaviour not the implementation"
argument
fail to realize that the same argument applies just as much to
state-based testing as it does to interaction-based testing (mocking.) 
Coupling a test to the implementation is much more subtle than using
mocks to predefine collaborator interactions.

Consider this glaringly stupid example in which an ambitious first-time
BDDer writes the first spec for a Stack class:

describe Stack do

  describe "#push" do
   
    it "should add an item" do
      stack = Stack.new
      stack.push(5)
      stack.items.should == [5]
    end
   
  end
end

To get this spec to pass they quickly churn out the following code, and
all is green:

class Stack
  attr_accessor :items
 
  def initialize
    @items = []
  end
 
  def push(object)
    @items.push object
  end
   
end

The encapsulation of the Stack is clearly broken by adding the
''attr_accessor :items'' call.  Not just that, but the spec is
now tied to
how the stack implements it''s behaviour. (In fact no  relevant
behaviour
was even being speced in the first place, it was all internal
structure!) If the Stack decides to use some other container or method
to implement the behavior then all the specs would need to be changed.

This was a painfully simple example, but the best I could come up with
off the top of my head.  I''m certainly not suggesting that using mocks
here would be better, I''m just illustrating what I believe is meant by
"test the behaviour not the implementation". 

As programs grow in size these sort of issues tend to creep up more
subtlety and such coupling isn''t so obvious.  I have noticed that with
a
purely state-based approach the creation and testing of objects at the
unit level seems to increase in difficultly and require more setup as
the complexity of a system increases.  Mocks not only allow you to
discover your interface but also help in breaking dependencies and
keeping your unit level specs focused on just that object''s
responsibility. 

I should point out however that with every spec I have that uses mocking
I always have an application level test (a story) that executes the same
code with the full stack in motion.  I''m also constantly trying to find
a good balance between state and interaction based testing in my
projects.  I think that balance is different from project to project,
team to team, and developer to developer.  With all that said, I think
mocking is a fantastic tool to have in your tool kit that can help
alleviate a lot of the pain associated with testing and, as Zach
explained, is a great tool to discover your design.

Wow, sorry for the long-winded reply... that was not my intention when I
began to write. :)

As Mark pointed out a good article on the matter is Martin Fowler''s:
http://www.martinfowler.com/articles/mocksArentStubs.html

If you read that article would recommend that you follow it up with this
one to get another perspective:
http://nat.truemesh.com/archives/000342.html

-Ben
http://www.benmabey.com

On Thu, Sep 4, 2008 at 4:14 PM, Nick Hoffman <nick at deadorange.com>
wrote:
> On 2008-08-27, at 15:25, Mark Wilden wrote:
>>
>> The other thing I would say is that mocking and stubbing are powerful
>> tools that you should add to your arsenal as soon as possible.
I''ve had
>> several coworkers who resisted using them, only to finally achieve that
>> "aha!" moment later. Your tests get easier to write, and
they''re less
>> brittle to change.
>
> G''day Mark. I was re-reading this thread and noticed this
paragraph of
> yours. I''ve been using RSpec and BDD for about 2 months now, and
love it.
>
> However, I''m not a fan of mocking and stubbing, primarily for two
reasons:
> 1) I believe that specs should test behaviour, rather than a
behaviour''s
> implementation.
> 2) Using mocks and stubs causes your specs and implementation to be tightly
> coupled, which often forces you to modify your specs if changes occur in
the
> implementation.
>
> However, #2 contradicts what you said about "tests ... [are] less
brittle to
> change" when using mocks and stubs. Considering that I''m
still very new to
> mocks and stubs, I''m probably missing something here. When you
have a
> minute, would you mind countering me?

Hey Nick,

I''ve talked with many people that echo your concern that mocks couple
specs to an object''s implementation.  It''s a legitimate
concern, of
course, however, what people tend to fail to recognize is that at some
level, specs are *always* coupled to implementation.  Let''s consider
for a moment what a specification is: a statement describing expected
behavior.  In programming terms, what an object *does*.  Well, what
does an object do, anyway?

There are three basic things an object can do:

* It can respond to a message
* It can return a result as the response to a message
* It can interact with collaborators

Let''s look at a super basic spec example and its associated object
(just typing it up, please excuse typos):

describe BankService, "#debit" do
  before(:each) do
    @account = Account.new(100)
    @service = BankService.new
  end

  it "should debit the account" do
    @service.debit @account, 25
    @account.balance.should == 75
  end
end

class BankService
  def debit(account, amount)
    account.debit amount
  end
end


Now this example is totally contrived - there''s nothing going on,
there''s just a middleman delegating a call to another object.  But
let''s take a look at it anyway.  What are the changes that could cause
this spec to fail?  I can think of several:

* BankService.new changes signature
* BankService#debit gets renamed or changes signature
* Account#debit gets renamed or changes signature
* Account.new changes signature
* Account.debit changes implementation (e.g. #debit also applies some
kind of charge, resulting in #balance returning a different result)

That, to me, represents a serious problem.  Out of five ways in which
this spec could break, only TWO of them are related to the Unit Under
Test (and this doesn''t include a name/signature change to
Account#balance)

But what happens if we use a mock object instead?

describe BankService, "#debit" do
  before(:each) do
    @mock_account = mock("account")
    @service = BankService.new
  end

  it "should debit the account" do
    @mock_account.should_receive(:debit).with(25)
    @service.debit @account, 25
  end
end

* BankService.new changes signature (spec fails)
* BankService#debit gets renamed or changes signature (spec fails)
* Account#debit gets renamed or changes signature (spec still passes)
* Account.new changes signature (spec still passes)
* Account.debit changes implementation (spec still passes)

By using a mock object, we''ve reduced the number of potential failure
causes from 5 to 2.  Now, I will grant you that #3 (Account#debit gets
renamed or changes signature) may result in a false positive, which is
a Bad Thing.  It''s a false positive in the sense that the *system as a
whole* doesn''t work though, not that there''s something wrong
with the
BankService object itself.  This is why we need integration tests.
But basically, as long as the BankService''s logic stays correct, the
existing specs pass.  And this is what happens when there''s basically
no logic - it''s all delegation - so imagine what happens when we have
real logic and multiple collaborators!

So, any time you write a spec for an object that has one or more
collaborators, you must ask yourself the following question:  "Do I
want my specs to be coupled to this object, or do I want my specs to
be coupled to this object''s collaborators?"  The problem with
choosing
the second option is that whenever you''re coupled to an
object''s
collaborators, you''re also coupled to the collaborators''
collaborators!

When you use mock objects, your specs ensure that you''re coupled only
to collaborators'' interfaces and not to their interfaces AND
implementations.  That''s nice, because collaborators (= dependencies)
have their OWN collaborators (=dependencies), so using *real*
collaborating objects in specs means that you''ve introduced a
dependency, and all its dependencies, and all its dependencies''
dependencies, ad infinitum.  So which spec is more brittle in reality?
 The one that breaks whenever the UUT changes, or the one that breaks
whenever one of X nested dependent objects changed?

So there we go, a lengthy (but hopefully useful) explanation of why
mock objects are good for object-level specs aka unit tests.  In
short, an interaction-based spec is coupled to its collaborators''
interfaces, and a purely state-based spec is coupled to its
collaborators'' interfaces and implementations, *recursively until
there are no more collaborators*.

That''s that.  And there''s another piece to this whole
"mocks couple
your specs to implementation" thing, which is that whenever I notice
developers being slowed down by mocks, it''s usually because
they''re
missing an abstraction.  There''s a great paper by Steve Freeman and
Nat Pryce called "mock roles not objects" [1] that gets into this.
Basically, you''ll only have success mocking useful abstractions, not
low-level stuff.  That is to say, mocking File.read might eliminate
your test''s dependency on the filesystem, but it doesn''t
change the
fact that your object is dealing with a relatively low-level operation
- and ultimately, we care about the design and effectiveness about the
production code itself rather than the tests.  So instead of mocking
out File.read, you spec out a ConfigReader class (for example) that
actually reads and parses some file, and once that''s complete you use
mock objects in any spec that needs a ConfigReader instance.
Unfortunately it''s late and I''ve run completely out of steam
and can''t
write anymore / create examples.

Bullet points
* Your tests are *always* coupled to an implementation at some level
* Mocks reduce the number of potential failure causes by eliminating
dependencies
* Pain when mocking usually points to potential design improvements

I encourage you to voice any other comments or concerns you''ve got,
and to point out the holes in my thinking.

Pat



[1] (PDF) http://www.jmock.org/oopsla2004.pdf

> * Pain when mocking usually points to potential design improvements
+1

It''s all about Behaviour Driven *Design*.

Ben and Pat, your descriptions, explanations, and examples were  
awesome. I''m beginning to see why and how mocks and stubs are useful,  
and when they should be used. As I mentioned to Mark and Zach, I''m  
going to take a couple of days to re-read all of your emails and have  
this info settle, try out some mocking and stubbing, and see what  
happens from there.

Thanks again for your insight guys! I didn''t expect such an in-depth  
discussion, but I definitely appreciate it.
-Nick

I think that''s one of the nicest descriptions of the value of
outside-in
I''ve seen.

Thanks Zach.

2008/9/5 Zach Dennis <zach.dennis at gmail.com>
> On Thu, Sep 4, 2008 at 7:14 PM, Nick Hoffman <nick at deadorange.com>
wrote:
> > On 2008-08-27, at 15:25, Mark Wilden wrote:
> >>
> >> The other thing I would say is that mocking and stubbing are
powerful
> >> tools that you should add to your arsenal as soon as possible.
I''ve had
> >> several coworkers who resisted using them, only to finally achieve
that
> >> "aha!" moment later. Your tests get easier to write, and
they''re less
> >> brittle to change.
> >
> > G''day Mark. I was re-reading this thread and noticed this
paragraph of
> > yours. I''ve been using RSpec and BDD for about 2 months now,
and love it.
> >
> > However, I''m not a fan of mocking and stubbing, primarily for
two
> reasons:
> > 1) I believe that specs should test behaviour, rather than a
behaviour''s
> > implementation.
>
> This is a misleading statement. Testing the behavior of an object
> involves its input and output collaborators. When used appropriately
> mocks and stubs act as an agent for design and discovery. They also
> allow objects under test to be isolated from other objects that it
> depends on (which may not even exist yet).
>
> Collaborators are what should be used to set up with stubs and mock
> expectations. This way we can test the behavior of the object under
> test given different values returned by its collaborators.
>
>
> For example:
>
>   class FundsTransfer
>      def transfer(amount, source_account, target_account)
>         if source_account.balance < amount
>            raise "not enough money"
>         else
>            source_account.deposit amount
>            target_account.withdraw amount
>      end
>   end
>
> Without using mocks or stubs to test the above code I wouldn''t be
able
> to test the case where the source account doesn''t have enough
money or
> does have enough money -- unless I already have implemented the
> Account class and its withdraw, deposit and balance methods. Taking
> the approach of writing the Account class first is an approach that a
> lot of developers take. But you build what you think you need before
> you actually need it. I prefer to develop from the other direction,
> only building what I discover I need to make it work.  Either way
> though behavior is being tested, and it is not testing against the
> internal state of an object.
>
> > 2) Using mocks and stubs causes your specs and implementation to be
> tightly
> > coupled, which often forces you to modify your specs if changes occur
in
> the
> > implementation.
> >
> > However, #2 contradicts what you said about "tests ... [are] less
brittle
> to
> > change" when using mocks and stubs. Considering that I''m
still very new
> to
> > mocks and stubs, I''m probably missing something here. When
you have a
> > minute, would you mind countering me?
>
> You can write bad tests with mocks/stubs and without mocks/stubs.
> Using mocks/stubs doesn''t immediately tightly couple your specs to
> your implementation. It does however tie your specs to the external
> interface of the object under test. If you change that, then you will
> need update your specs.
>
> Consider our FundsTransfer example again:
>
>   class FundsTransfer
>      def transfer(amount, source_account, target_account)
>         if source_account.balance < amount
>            raise "not enough money"
>         else
>            source_account.deposit amount
>            target_account.withdraw amount
>      end
>   end
>
> If I supply a source_account that has a stubbed out balance of less
> than the amount requesting to be transferred am I tightly coupling the
> spec to the implementation? No more so then creating a source account
> fixture with an amount less than the amount that I am requesting to
> transfer. I say this because at some point you need to test the
> scenario where funds are requested to be transferred from a source
> account that doesn''t have enough money.
>
> Do you really see one of the below examples more tightly coupling the
> test to the implementation?
>
>    account = mock("Account", :balance => 0)
>    account = Account.new :balance => 0
>
> The difference to me is that using the mock allows me to discover
> earlier what interface I want to work with on the Account class. If I
> go the non-mock route then I need to make sure I build the Account
> class with the interfaces I need first.
>
> I much prefer to work in small steps. Focus on one behaviour, and then
> when it works as expected, make sure any collaborators that need to be
> implemented are. Then use my integration tests to make sure everything
> is wired up correctly together.
>
> That''s just me though,
>
> --
> Zach Dennis
> http://www.continuousthinking.com
> http://www.mutuallyhuman.com
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20080905/5847f13d/attachment-0001.html>