mocha developer - Feb 2007 - how does Mocha compare in terms of classical vs mock-based testing, and stubbing???

Hi guys,

I''ve just been reading Martin Fowler''s article re mock versus
stubbing<http://martinfowler.com/articles/mocksArentStubs.html>where
he compares traditional TDD testing techniques with mock based
testing. I''d be interested in comments from a ruby on rails perspective
in
terms of this and Mocha?  For example:

a) Do you see Mocha as a robust way to test Ruby on Rails based apps?

b) Why would one use Mocha over traditional non-mock object based testing
for Ruby on Rails?

c) Is Mocha a good choice for stubbing out other components that the rails
app may interact with, for the purpose of testing?

Tks
Greg

On 2/4/07, Greg Hauptmann <greg.hauptmann.ruby at gmail.com>
wrote:
> Hi guys,
>
> I''ve just been reading Martin Fowler''s article re mock
versus
> stubbing<http://martinfowler.com/articles/mocksArentStubs.html>where
> he compares traditional TDD testing techniques with mock based
> testing. I''d be interested in comments from a ruby on rails
perspective in
> terms of this and Mocha?  For example:
>
> a) Do you see Mocha as a robust way to test Ruby on Rails based apps?
Yes - though I think of mocking as one tool among many in a robust
testing-toolkit.
>
> b) Why would one use Mocha over traditional non-mock object based testing
> for Ruby on Rails?
Interface Discovery!!!!!! Read http://jmock.org/oopsla2004.pdf for
more info on that.
>
> c) Is Mocha a good choice for stubbing out other components that the rails
> app may interact with, for the purpose of testing?
Short version: Yes.

Longer version: A guideline that I like to follow is to wrap 3rd party
APIs with thin adapters that provide context-appropriate method names
and only those that your app needs. Then you mock your own API, not
the 3rd party API. You still need to test your adapter, but that test
is isolated from the tests that are about your application and what it
does.

Cheers,
David
>
> Tks
> Greg
> _______________________________________________
> mocha-developer mailing list
> mocha-developer at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mocha-developer
>

On 05/02/07, Greg Hauptmann <greg.hauptmann.ruby at gmail.com>
wrote:
>
> I''ve just been reading Martin Fowler''s article re mock
versus
> stubbing<http://martinfowler.com/articles/mocksArentStubs.html>where
> he compares traditional TDD testing techniques with mock based
> testing. I''d be interested in comments from a ruby on rails
perspective in
> terms of this and Mocha?  For example:
>
> a) Do you see Mocha as a robust way to test Ruby on Rails based apps?
>
> b) Why would one use Mocha over traditional non-mock object based testing
> for Ruby on Rails?
>
> c) Is Mocha a good choice for stubbing out other components that the rails
> app may interact with, for the purpose of testing?

Hi Greg,

Some great questions!

The team at reevoo.com has been using Mocha and its earlier incarnations to
help us test drive Rails applications over the last 18 months or so. We find
it useful to be able to do interaction-based testing as well as traditional
state-based testing. But you could use any mocking library to achieve the
same effect. In general terms, I think doing interaction-based TDD can help
you write more cohesive, less coupled code, although of course it''s not
a
silver bullet. You will still need higher level functional/integration tests
which check that the real objects are all wired up correctly. I would say
you will see more benefits of using an interaction-based approach, the
bigger your application gets - mainly due to the reduction in coupling and
hence the reduction in brittleness of tests.

There are some great resources on the JMock & MockObjects websites (see
http://www.jmock.org/docs.html and http://www.mockobjects.com/) which
although they give examples in Java, are just as applicable in Ruby. In
particular the "Mock Roles, Not Objects" paper (
http://www.jmock.org/oopsla2004.pdf) is well worth a read.

I''m not sure I''ve completely answered your questions, but
please feel free
to ask more questions like this on the mailing list. I find it''s often
easier to talk about specific examples rather than generalities, so why not
have a go with Mocha in a Rails app and drop us a line if you have any
questions.

-- 
James.
http://blog.floehopper.org

On 2/5/07, James Mead <jamesmead44 at gmail.com>
wrote:
> On 05/02/07, Greg Hauptmann <greg.hauptmann.ruby at gmail.com> wrote:
> >
> > I''ve just been reading Martin Fowler''s article re
mock versus
> >
stubbing<http://martinfowler.com/articles/mocksArentStubs.html>where
> > he compares traditional TDD testing techniques with mock based
> > testing. I''d be interested in comments from a ruby on rails
perspective in
> > terms of this and Mocha?  For example:
> >
> > a) Do you see Mocha as a robust way to test Ruby on Rails based apps?
> >
> > b) Why would one use Mocha over traditional non-mock object based
testing
> > for Ruby on Rails?
> >
> > c) Is Mocha a good choice for stubbing out other components that the
rails
> > app may interact with, for the purpose of testing?
>
>
> Hi Greg,
>
> Some great questions!
>
> The team at reevoo.com has been using Mocha and its earlier incarnations to
> help us test drive Rails applications over the last 18 months or so. We
find
> it useful to be able to do interaction-based testing as well as traditional
> state-based testing. But you could use any mocking library to achieve the
> same effect. In general terms, I think doing interaction-based TDD can help
> you write more cohesive, less coupled code, although of course
it''s not a
> silver bullet. You will still need higher level functional/integration
tests
> which check that the real objects are all wired up correctly. I would say
> you will see more benefits of using an interaction-based approach, the
> bigger your application gets - mainly due to the reduction in coupling and
> hence the reduction in brittleness of tests.
Very well put!
>
> There are some great resources on the JMock & MockObjects websites (see
> http://www.jmock.org/docs.html and http://www.mockobjects.com/) which
> although they give examples in Java, are just as applicable in Ruby. In
> particular the "Mock Roles, Not Objects" paper (
> http://www.jmock.org/oopsla2004.pdf) is well worth a read.
>
> I''m not sure I''ve completely answered your questions, but
please feel free
> to ask more questions like this on the mailing list. I find it''s
often
> easier to talk about specific examples rather than generalities, so why not
> have a go with Mocha in a Rails app and drop us a line if you have any
> questions.
>
> --
> James.
> http://blog.floehopper.org
> _______________________________________________
> mocha-developer mailing list
> mocha-developer at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mocha-developer
>

Thanks James/David,

I''ll read through the references.  I''m getting that same
feeling I had when
I came across blocks for the 1st time in ruby actually :) [I still wonder if
I truly understand the full power of blocks and whether I''m missing
something]   I think perhaps I need to write/use the framework to have it
sink in.

In the meantime if you had a few minutes for a couple of followup
questions.  Actually I''ll just focus around my stubing-out-paypal
requirement:

So the suggestion is to have (a) a thin paypal adaptor to stub out the
interface, (b) use of mocha for testing the interface.

Q1 - Was there a specific ruby on rails pattern/approach you had in mind
when you mentioned "adaptor"?

Q2 - Is the purpose of the components suggested per the below, i.e. do I
have it right.  Use cases:
Testing the application
(a) ability to manually test app - uses paypal adaptor
(b) ability to run state based unit tests - uses paypal adaptor
(c) ability to run mocha tests within the unit testing framework - My
understanding here is that Mocha wouldn''t actually use the adaptor? 
i.e.
looking at the examples it seems with Mocha you would treat the paypal
adaptor as a collaborator and effectively build the understanding of how it
behaves manually yourself into your Mocha test code?  (not sure if this is
DRY - maybe I''m missing something)
Testing the Paypal interface
(d) well I guess in terms of testing the Paypal interface this isn''t
what
we''re focusing on here.   However it is an interesting question.  It
would
have to be a test cases that run only in the non-development environment
when the system is deployed to production (i.e. so Paypal can call back to
my app)

Q3 - What method would you recommend in a situation where some test cases (
i.e. the final Paypal interface focused unit tests) should only run in one
particular environment.  I did see an article in the past from memory which
highlighted a ruby on rails approach for loading specific code based on
which environment one was in.  Perhaps I should dig this up?

Q4 - This was buried in Q2 probably, but just confirming: The Mocha based
tests don''t actually run code within the collaborator do they?  i.e.
rather
they seem to build up the expected responses from the collaborator that the
SUT (system under test) will see once the test is initiated?

Thanks again
Greg










development mode code insertion







On 2/5/07, David Chelimsky <dchelimsky at gmail.com>
wrote:
>
> On 2/4/07, Greg Hauptmann <greg.hauptmann.ruby at gmail.com> wrote:
> > Hi guys,
> >
> > I''ve just been reading Martin Fowler''s article re
mock versus
> >
stubbing<http://martinfowler.com/articles/mocksArentStubs.html>where
> > he compares traditional TDD testing techniques with mock based
> > testing. I''d be interested in comments from a ruby on rails
perspective
> in
> > terms of this and Mocha?  For example:
> >
> > a) Do you see Mocha as a robust way to test Ruby on Rails based apps?
>
> Yes - though I think of mocking as one tool among many in a robust
> testing-toolkit.
>
> >
> > b) Why would one use Mocha over traditional non-mock object based
> testing
> > for Ruby on Rails?
>
> Interface Discovery!!!!!! Read http://jmock.org/oopsla2004.pdf for
> more info on that.
>
> >
> > c) Is Mocha a good choice for stubbing out other components that the
> rails
> > app may interact with, for the purpose of testing?
>
> Short version: Yes.
>
> Longer version: A guideline that I like to follow is to wrap 3rd party
> APIs with thin adapters that provide context-appropriate method names
> and only those that your app needs. Then you mock your own API, not
> the 3rd party API. You still need to test your adapter, but that test
> is isolated from the tests that are about your application and what it
> does.
>
> Cheers,
> David
>
> >
> > Tks
> > Greg
> > _______________________________________________
> > mocha-developer mailing list
> > mocha-developer at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mocha-developer
> >
> _______________________________________________
> mocha-developer mailing list
> mocha-developer at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mocha-developer
>

On 05/02/07, Greg Hauptmann <greg.hauptmann.ruby at gmail.com>
wrote:
>
> I''ll read through the references.  I''m getting that same
feeling I had
> when
> I came across blocks for the 1st time in ruby actually :) [I still wonder
> if
> I truly understand the full power of blocks and whether I''m
missing
> something]   I think perhaps I need to write/use the framework to have it
> sink in.

I think that''s very true. I''m still learning how to improve my
TDD and
mocking techniques. I find a mixture of reading and trying things out is the
best way to learn. Don''t be daunted by all the articles about mocking
and
interaction based testing - there''s never a right and wrong way - just
better and worse ways ;-)

In the meantime if you had a few minutes for a couple of
followup
> questions.  Actually I''ll just focus around my stubing-out-paypal
> requirement:
>
> So the suggestion is to have (a) a thin paypal adaptor to stub out the
> interface, (b) use of mocha for testing the interface.
>
> Q1 - Was there a specific ruby on rails pattern/approach you had in mind
> when you mentioned "adaptor"?

Not really - I was thinking of an "adapter" in fairly general terms.
An
adapter could be useful to simplify the interface to the external service
e.g. you may not need all the flexibility of a 3rd party library. An adapter
can also be useful to decouple your application from a particular 3rd party
library - it''s much better to mock/stub your own code (i.e. the
adapter) and
not 3rd party code which may be subject to change. So an adapter is not
essential, but gives these advantages.

Q2 - Is the purpose of the components suggested per the below, i.e. do
I
> have it right.  Use cases:
> Testing the application
> (a) ability to manually test app - uses paypal adaptor
> (b) ability to run state based unit tests - uses paypal adaptor
> (c) ability to run mocha tests within the unit testing framework - My
> understanding here is that Mocha wouldn''t actually use the
adaptor?  i.e.
> looking at the examples it seems with Mocha you would treat the paypal
> adaptor as a collaborator and effectively build the understanding of how
> it
> behaves manually yourself into your Mocha test code?  (not sure if this is
> DRY - maybe I''m missing something)
> Testing the Paypal interface
> (d) well I guess in terms of testing the Paypal interface this
isn''t what
> we''re focusing on here.   However it is an interesting question. 
It would
> have to be a test cases that run only in the non-development environment
> when the system is deployed to production (i.e. so Paypal can call back to
> my app)

Hmm - I''m a bit confused by all the above. In tests for the application
I
would replace the adapter with a mock object and either expect or stub
method calls.

Q3 - What method would you recommend in a situation where some test cases
(
> i.e. the final Paypal interface focused unit tests) should only run in one
> particular environment.  I did see an article in the past from memory
> which
> highlighted a ruby on rails approach for loading specific code based on
> which environment one was in.  Perhaps I should dig this up?

I''m not sure what you''re getting at here. I think you might be
talking about
the normal "Rails mocks"  approach i.e. alternative Fake Object
definitions
of particular classes which are loaded in preference to the real definitions
in a particular environment. These are the classes you can put in the
test/mocks directory.

Q4 - This was buried in Q2 probably, but just confirming: The Mocha
based
> tests don''t actually run code within the collaborator do they? 
i.e.
> rather
> they seem to build up the expected responses from the collaborator that
> the
> SUT (system under test) will see once the test is initiated?
>
Again, I''m not 100% sure I understand you correctly, but I think the
answer
is yes!

At the risk of sounding like a broken record, I really recommend diving in
and having a go - conversations like this one tend to suffer from ambiguity
of language on both sides and so often go around in circles. Concrete
examples are worth a lot of words! If you get stuck or are unhappy about a
particular test, post it here and we''ll try to help you.

-- 
James.
http://blog.floehopper.org

Hi David - I''ve checked at the Mocha source code and starting having a
look.  I''m interested in understanding your comment on interface
discovery.   If you (or someone else) is across the Mocha codebase you could
provide a pointer to where this interface discovery takes please?

Tks
greg

On 2/5/07, David Chelimsky <dchelimsky at gmail.com>
wrote:
>
> On 2/4/07, Greg Hauptmann <greg.hauptmann.ruby at gmail.com> wrote:
> > Hi guys,
> >
> > I''ve just been reading Martin Fowler''s article re
mock versus
> >
stubbing<http://martinfowler.com/articles/mocksArentStubs.html>where
> > he compares traditional TDD testing techniques with mock based
> > testing. I''d be interested in comments from a ruby on rails
perspective
> in
> > terms of this and Mocha?  For example:
> >
> > a) Do you see Mocha as a robust way to test Ruby on Rails based apps?
>
> Yes - though I think of mocking as one tool among many in a robust
> testing-toolkit.
>
> >
> > b) Why would one use Mocha over traditional non-mock object based
> testing
> > for Ruby on Rails?
>
> Interface Discovery!!!!!! Read http://jmock.org/oopsla2004.pdf for
> more info on that.
>
> >
> > c) Is Mocha a good choice for stubbing out other components that the
> rails
> > app may interact with, for the purpose of testing?
>
> Short version: Yes.
>
> Longer version: A guideline that I like to follow is to wrap 3rd party
> APIs with thin adapters that provide context-appropriate method names
> and only those that your app needs. Then you mock your own API, not
> the 3rd party API. You still need to test your adapter, but that test
> is isolated from the tests that are about your application and what it
> does.
>
> Cheers,
> David
>
> >
> > Tks
> > Greg
> > _______________________________________________
> > mocha-developer mailing list
> > mocha-developer at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mocha-developer
> >
> _______________________________________________
> mocha-developer mailing list
> mocha-developer at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mocha-developer
>

On 2/7/07, Greg Hauptmann <greg.hauptmann.ruby at gmail.com>
wrote:
> Hi David - I''ve checked at the Mocha source code and starting
having a
> look.  I''m interested in understanding your comment on interface
> discovery.   If you (or someone else) is across the Mocha codebase you
could
> provide a pointer to where this interface discovery takes please?
Hi Greg - did you read http://jmock.org/oopsla2004.pdf? It explains it
better than I could.

Read that first and if you have any questions about it feel free to ask.

Cheers,
David
>
> Tks
> greg
>
> On 2/5/07, David Chelimsky <dchelimsky at gmail.com> wrote:
> >
> > On 2/4/07, Greg Hauptmann <greg.hauptmann.ruby at gmail.com>
wrote:
> > > Hi guys,
> > >
> > > I''ve just been reading Martin Fowler''s article
re mock versus
> > >
stubbing<http://martinfowler.com/articles/mocksArentStubs.html>where
> > > he compares traditional TDD testing techniques with mock based
> > > testing. I''d be interested in comments from a ruby on
rails perspective
> > in
> > > terms of this and Mocha?  For example:
> > >
> > > a) Do you see Mocha as a robust way to test Ruby on Rails based
apps?
> >
> > Yes - though I think of mocking as one tool among many in a robust
> > testing-toolkit.
> >
> > >
> > > b) Why would one use Mocha over traditional non-mock object based
> > testing
> > > for Ruby on Rails?
> >
> > Interface Discovery!!!!!! Read http://jmock.org/oopsla2004.pdf for
> > more info on that.
> >
> > >
> > > c) Is Mocha a good choice for stubbing out other components that
the
> > rails
> > > app may interact with, for the purpose of testing?
> >
> > Short version: Yes.
> >
> > Longer version: A guideline that I like to follow is to wrap 3rd party
> > APIs with thin adapters that provide context-appropriate method names
> > and only those that your app needs. Then you mock your own API, not
> > the 3rd party API. You still need to test your adapter, but that test
> > is isolated from the tests that are about your application and what it
> > does.
> >
> > Cheers,
> > David
> >
> > >
> > > Tks
> > > Greg
> > > _______________________________________________
> > > mocha-developer mailing list
> > > mocha-developer at rubyforge.org
> > > http://rubyforge.org/mailman/listinfo/mocha-developer
> > >
> > _______________________________________________
> > mocha-developer mailing list
> > mocha-developer at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mocha-developer
> >
> _______________________________________________
> mocha-developer mailing list
> mocha-developer at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mocha-developer
>

Hi David, (all)

I did read this, but re-read briefly again.....arrr....by interface
discovery you were talking about the development concept behind the mock
object TDD concept.   I was on the lookout for some aspect of Mocha that was
going to do a "reflection" style dynamic interface discovery and then
build
the mock object from this.  Got it now, so with the Mock object approach the
concept is that you haven''t written the code for the collaborators yet.
Let me know if I''m wrong  :)

The only negative I see re potential use of Mocha to solve my "need to stub
out paypal for local testing" is that Mocha seems to extend test cases no?
i.e. this would allow me to unit test my components with paypal stubbed out
in the form of mock object, HOWEVER if I just wanted to do ad hoc
testing/use of my web application (i.e. not driven by a test case) then
I''ll
still need a separate "test paypal layer" (or stub out paypal in other
words).   If I''ve still missed something I''d be keen to know.


Tks
Greg



On 2/8/07, David Chelimsky <dchelimsky at gmail.com>
wrote:
>
> On 2/7/07, Greg Hauptmann <greg.hauptmann.ruby at gmail.com> wrote:
> > Hi David - I''ve checked at the Mocha source code and starting
having a
> > look.  I''m interested in understanding your comment on
interface
> > discovery.   If you (or someone else) is across the Mocha codebase you
> could
> > provide a pointer to where this interface discovery takes please?
>
> Hi Greg - did you read http://jmock.org/oopsla2004.pdf? It explains it
> better than I could.
>
> Read that first and if you have any questions about it feel free to ask.
>
> Cheers,
> David
>
> >
> > Tks
> > greg
> >
> > On 2/5/07, David Chelimsky <dchelimsky at gmail.com> wrote:
> > >
> > > On 2/4/07, Greg Hauptmann <greg.hauptmann.ruby at
gmail.com> wrote:
> > > > Hi guys,
> > > >
> > > > I''ve just been reading Martin Fowler''s
article re mock versus
> > > >
stubbing<http://martinfowler.com/articles/mocksArentStubs.html>where
> > > > he compares traditional TDD testing techniques with mock
based
> > > > testing. I''d be interested in comments from a ruby
on rails
> perspective
> > > in
> > > > terms of this and Mocha?  For example:
> > > >
> > > > a) Do you see Mocha as a robust way to test Ruby on Rails
based
> apps?
> > >
> > > Yes - though I think of mocking as one tool among many in a
robust
> > > testing-toolkit.
> > >
> > > >
> > > > b) Why would one use Mocha over traditional non-mock object
based
> > > testing
> > > > for Ruby on Rails?
> > >
> > > Interface Discovery!!!!!! Read http://jmock.org/oopsla2004.pdf
for
> > > more info on that.
> > >
> > > >
> > > > c) Is Mocha a good choice for stubbing out other components
that the
> > > rails
> > > > app may interact with, for the purpose of testing?
> > >
> > > Short version: Yes.
> > >
> > > Longer version: A guideline that I like to follow is to wrap 3rd
party
> > > APIs with thin adapters that provide context-appropriate method
names
> > > and only those that your app needs. Then you mock your own API,
not
> > > the 3rd party API. You still need to test your adapter, but that
test
> > > is isolated from the tests that are about your application and
what it
> > > does.
> > >
> > > Cheers,
> > > David
> > >
> > > >
> > > > Tks
> > > > Greg
> > > > _______________________________________________
> > > > mocha-developer mailing list
> > > > mocha-developer at rubyforge.org
> > > > http://rubyforge.org/mailman/listinfo/mocha-developer
> > > >
> > > _______________________________________________
> > > mocha-developer mailing list
> > > mocha-developer at rubyforge.org
> > > http://rubyforge.org/mailman/listinfo/mocha-developer
> > >
> > _______________________________________________
> > mocha-developer mailing list
> > mocha-developer at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mocha-developer
> >
> _______________________________________________
> mocha-developer mailing list
> mocha-developer at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mocha-developer
>

On 08/02/07, Greg Hauptmann <greg.hauptmann.ruby at gmail.com>
wrote:
>
> I did read this, but re-read briefly again.....arrr....by interface
> discovery you were talking about the development concept behind the mock
> object TDD concept.   I was on the lookout for some aspect of Mocha that
> was
> going to do a "reflection" style dynamic interface discovery and
then
> build
> the mock object from this.  Got it now, so with the Mock object approach
> the
> concept is that you haven''t written the code for the collaborators
yet.
> Let me know if I''m wrong  :)

Yes. Using TDD with mock objects makes you think about what behaviour does
the object under test need from its collaborator(s). This drives out the
public method signatures for the collaborator(s) (i.e. interface discovery),
but allows you to focus on the object under test and not immediately worry
about how to implement the behaviour of the collaborators.

The only negative I see re potential use of Mocha to solve my "need to
stub
> out paypal for local testing" is that Mocha seems to extend test cases
no?
> i.e. this would allow me to unit test my components with paypal stubbed
> out
> in the form of mock object, HOWEVER if I just wanted to do ad hoc
> testing/use of my web application (i.e. not driven by a test case) then
> I''ll
> still need a separate "test paypal layer" (or stub out paypal in
other
> words).   If I''ve still missed something I''d be keen to
know.
>
The latest version of Mocha is not tightly coupled to Test::Unit. In theory
you could use Mocha to stub out the PayPal service in your development
environment without needing a TestCase. However, I think you''d be
better off
using something more like a standard Rails "mock" in the test/mocks
directory (what Martin Fowler calls a Fake Object - see
http://www.martinfowler.com/articles/mocksArentStubs.html#TheDifferenceBetweenMocksAndStubs).
i.e. have an alternative implementation of the PayPal service which replaces
the real implementation and returns canned results.

-- 
James.
http://blog.floehopper.org

This might be worth a look -
http://blog.tammersaleh.com/articles/2007/02/13/mocking-a-network-resource-with-camping

-- 
James.
http://blog.floehopper.org