rspec users - Oct 2008 - Mocks and Refactoring - doing it wrong?

Hello RSpec folks,
I''ve only been introduced to the world of mock objects since Wednesday
of last week, so go easy on me if I come off as ignorant. :P

So, I''m a big fan of testing, especially since it has really helped to
do refactoring in the past. But, I''m running into an issue that
I''m
hoping could be cleared up. Let''s say that I have some code like the
following:

mock_engine = mock("Engine")
mock_engine.should_receive(:ignite)
car = Car.new(mock_engine)
car.start

Internally, the car class calls "ignite" on the engine that was passed
to the constructor. We run the spec, everything is green. So far, so
good.

Now, here''s what I don''t understand: suppose I refactor the
Engine class
and rename the "ignite" method to "turn_on". If I re-run the
example
from above, the test is still green - but it shouldn''t be, since the
"ignite" method doesn''t exist on the Engine class anymore.

So I''m wondering...what am I doing wrong? I feel like my tests should
help catch these sorts of things when I do a refactoring - my hunch is
that I''m missing a step here.

All comments are welcomed and appreciated! :)
-- 
Posted via http://www.ruby-forum.com/.

On Oct 30, 2008, at 1:02 AM, Sebastian W. wrote:
> Hello RSpec folks,
> I''ve only been introduced to the world of mock objects since
Wednesday
> of last week, so go easy on me if I come off as ignorant. :P
>
> So, I''m a big fan of testing, especially since it has really
helped to
> do refactoring in the past. But, I''m running into an issue that
I''m
> hoping could be cleared up. Let''s say that I have some code like
the
> following:
>
> mock_engine = mock("Engine")
> mock_engine.should_receive(:ignite)
> car = Car.new(mock_engine)
> car.start
>
> Internally, the car class calls "ignite" on the engine that was
passed
> to the constructor. We run the spec, everything is green. So far, so
> good.
>
> Now, here''s what I don''t understand: suppose I refactor
the Engine
> class
> and rename the "ignite" method to "turn_on". If I
re-run the example
> from above, the test is still green - but it shouldn''t be, since
the
> "ignite" method doesn''t exist on the Engine class
anymore.
>
> So I''m wondering...what am I doing wrong? I feel like my tests
should
> help catch these sorts of things when I do a refactoring - my hunch is
> that I''m missing a step here.
The step your missing is that using mocks is different then classical,  
black box testing.

When you use a mock object, you''re making a tradeoff.   One of the  
things you gain is isolation.  For instance, if you were testing a  
webservice or a database, you may not care about it being around *for  
this test*.  Another thing you''re gaining is pure speed (of the  
running test case) - you simply won''t have to go through the full  
stack.  On the other hand, that''s also what your loosing - meaning  
that if that API changes, your screwed.

This is why anyone who knows anything about testing will recommend  
several different layers of tests - some very close to the metal which  
will use mocks (which can be repeatedly run quickly), and others which  
are closer on the integration side of the spectrum.

Scott

Hi Scott,
Cool - I see what you''re saying here. The only thing that I''m
a bit
confused still is that it seems like, at least if your system is 
starting to get larger, you''d really *want* your fast unit test to help
you catch API changes like this to help you make updates faster.

Having to run a suite of more expensive integration tests just to catch 
API changes seems a little funny. But I guess I''m also hoping that 
there''s some way for the mocks to help with that sort of thing -
it''s my
understanding that some other frameworks out there help you with that 
sort of stuff. One example mentioned to me was JMock -- granted, that''s
Java, but still - if it''s possible in Java, Ruby should be able to do
it
too. :P
-- 
Posted via http://www.ruby-forum.com/.

On Oct 30, 2008, at 2:01 AM, Sebastian W. wrote:
> Hi Scott,
> Cool - I see what you''re saying here. The only thing that
I''m a bit
> confused still is that it seems like, at least if your system is
> starting to get larger, you''d really *want* your fast unit test to
> help
> you catch API changes like this to help you make updates faster.
>
> Having to run a suite of more expensive integration tests just to  
> catch
> API changes seems a little funny. But I guess I''m also hoping that
> there''s some way for the mocks to help with that sort of thing -  
> it''s my
> understanding that some other frameworks out there help you with that
> sort of stuff. One example mentioned to me was JMock -- granted,  
> that''s
> Java, but still - if it''s possible in Java, Ruby should be able to
> do it
> too. :P
Most certainly.

David, et. all:

Why don''t we have a partial mock which will raise an error (or at  
least a warning) when stubbing an object who''s class doesn''t  
respond_to? the method given?  I feel like this sort of simple  
dependency has been brought up 1000 times on the list before, but  
never been explicitly stated.

WDYT?

Scott
>
> -- 
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users

On Thu, Oct 30, 2008 at 1:14 AM, Scott Taylor
<mailing_lists at railsnewbie.com> wrote:
>
> On Oct 30, 2008, at 2:01 AM, Sebastian W. wrote:
>
>> Hi Scott,
>> Cool - I see what you''re saying here. The only thing that
I''m a bit
>> confused still is that it seems like, at least if your system is
>> starting to get larger, you''d really *want* your fast unit
test to help
>> you catch API changes like this to help you make updates faster.
>>
>> Having to run a suite of more expensive integration tests just to catch
>> API changes seems a little funny. But I guess I''m also hoping
that
>> there''s some way for the mocks to help with that sort of thing
- it''s my
>> understanding that some other frameworks out there help you with that
>> sort of stuff. One example mentioned to me was JMock -- granted,
that''s
>> Java, but still - if it''s possible in Java, Ruby should be
able to do it
>> too. :P
>
> Most certainly.
>
> David, et. all:
>
> Why don''t we have a partial mock which will raise an error (or at
least a
> warning) when stubbing an object who''s class doesn''t
respond_to? the method
> given?  I feel like this sort of simple dependency has been brought up 1000
> times on the list before, but never been explicitly stated.
>
> WDYT?
This has come up before.

The problem is that we''re dealing with a dynamic language and the
class definition may or may not include the definition of that method
at the point that the mock framework would do such an evaluation,
especially if thing are getting mocked that might otherwise be loading
modules and extending behaviour.

I''d rather have it be a known problem, but a consistent problem, then
a partially solved problem that will inevitably cause more pain that
it does today :)

Cheers,
David
>
> Scott
>
>>
>> --
>> Posted via http://www.ruby-forum.com/.
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

On Oct 30, 2008, at 2:14 AM, Scott Taylor wrote:
>
> On Oct 30, 2008, at 2:01 AM, Sebastian W. wrote:
>
>> Hi Scott,
>> Cool - I see what you''re saying here. The only thing that
I''m a bit
>> confused still is that it seems like, at least if your system is
>> starting to get larger, you''d really *want* your fast unit
test to
>> help
>> you catch API changes like this to help you make updates faster.
>>
>> Having to run a suite of more expensive integration tests just to  
>> catch
>> API changes seems a little funny. But I guess I''m also hoping
that
>> there''s some way for the mocks to help with that sort of thing
-
>> it''s my
>> understanding that some other frameworks out there help you with that
>> sort of stuff. One example mentioned to me was JMock -- granted,  
>> that''s
>> Java, but still - if it''s possible in Java, Ruby should be
able to
>> do it
>> too. :P
>
> Most certainly.
>
> David, et. all:
>
> Why don''t we have a partial mock which will raise an error (or at
> least a warning) when stubbing an object who''s class
doesn''t
> respond_to? the method given?  I feel like this sort of simple  
> dependency has been brought up 1000 times on the list before, but  
> never been explicitly stated.
Also, I''d be game for implementing this, and think it would be a good  
default behavior for the #mock method when passed a class name, but  
not a literal string.

Scott

>
>
> WDYT?
>
> Scott
>
>>
>> -- 
>> Posted via http://www.ruby-forum.com/.
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users

> Why don''t we have a partial mock which will raise an error (or at
> least a warning) when stubbing an object who''s class
doesn''t
> respond_to? the method given?  I feel like this sort of simple
> dependency has been brought up 1000 times on the list before, but
> never been explicitly stated.
> 
> WDYT?
> 
> Scott
Hey cool! I doubt you''d want me poking around in your codebase, but if 
there''s somewhere you can point me to help, I''d love to take a
look.

But I doubt I''d be of much help. :D Can''t hurt to look though.

Thanks so much for your responses Scott!
-- 
Posted via http://www.ruby-forum.com/.

On Oct 30, 2008, at 2:17 AM, David Chelimsky wrote:
> On Thu, Oct 30, 2008 at 1:14 AM, Scott Taylor
> <mailing_lists at railsnewbie.com> wrote:
>>
>> On Oct 30, 2008, at 2:01 AM, Sebastian W. wrote:
>>
>>> Hi Scott,
>>> Cool - I see what you''re saying here. The only thing that
I''m a bit
>>> confused still is that it seems like, at least if your system is
>>> starting to get larger, you''d really *want* your fast unit
test to
>>> help
>>> you catch API changes like this to help you make updates faster.
>>>
>>> Having to run a suite of more expensive integration tests just to  
>>> catch
>>> API changes seems a little funny. But I guess I''m also
hoping that
>>> there''s some way for the mocks to help with that sort of
thing -
>>> it''s my
>>> understanding that some other frameworks out there help you with  
>>> that
>>> sort of stuff. One example mentioned to me was JMock -- granted,  
>>> that''s
>>> Java, but still - if it''s possible in Java, Ruby should be
able to
>>> do it
>>> too. :P
>>
>> Most certainly.
>>
>> David, et. all:
>>
>> Why don''t we have a partial mock which will raise an error (or
at
>> least a
>> warning) when stubbing an object who''s class doesn''t
respond_to?
>> the method
>> given?  I feel like this sort of simple dependency has been brought  
>> up 1000
>> times on the list before, but never been explicitly stated.
>>
>> WDYT?
>
> This has come up before.
>
> The problem is that we''re dealing with a dynamic language and the
> class definition may or may not include the definition of that method
> at the point that the mock framework would do such an evaluation,
> especially if thing are getting mocked that might otherwise be loading
> modules and extending behaviour.
>
> I''d rather have it be a known problem, but a consistent problem,
then
> a partially solved problem that will inevitably cause more pain that
> it does today :)
Yeah, that''s a good point.

I wonder if possibly passing some sort of flag to the stub, which is  
off by default (or on) might solve this problem.

Scott

On Oct 30, 2008, at 2:17 AM, David Chelimsky wrote:
> On Thu, Oct 30, 2008 at 1:14 AM, Scott Taylor
> <mailing_lists at railsnewbie.com> wrote:
>>
>> On Oct 30, 2008, at 2:01 AM, Sebastian W. wrote:
>>
>>> Hi Scott,
>>> Cool - I see what you''re saying here. The only thing that
I''m a bit
>>> confused still is that it seems like, at least if your system is
>>> starting to get larger, you''d really *want* your fast unit
test to
>>> help
>>> you catch API changes like this to help you make updates faster.
>>>
>>> Having to run a suite of more expensive integration tests just to  
>>> catch
>>> API changes seems a little funny. But I guess I''m also
hoping that
>>> there''s some way for the mocks to help with that sort of
thing -
>>> it''s my
>>> understanding that some other frameworks out there help you with  
>>> that
>>> sort of stuff. One example mentioned to me was JMock -- granted,  
>>> that''s
>>> Java, but still - if it''s possible in Java, Ruby should be
able to
>>> do it
>>> too. :P
>>
>> Most certainly.
>>
>> David, et. all:
>>
>> Why don''t we have a partial mock which will raise an error (or
at
>> least a
>> warning) when stubbing an object who''s class doesn''t
respond_to?
>> the method
>> given?  I feel like this sort of simple dependency has been brought  
>> up 1000
>> times on the list before, but never been explicitly stated.
>>
>> WDYT?
>
> This has come up before.
>
> The problem is that we''re dealing with a dynamic language and the
> class definition may or may not include the definition of that method
> at the point that the mock framework would do such an evaluation,
> especially if thing are getting mocked that might otherwise be loading
> modules and extending behaviour.
>
> I''d rather have it be a known problem, but a consistent problem,
then
> a partially solved problem that will inevitably cause more pain that
> it does today :)
Actually, I take back my previous comment.  Where would you stub a  
method on an object where the method *isn''t* even loaded by further  
(or previous) stubbing?

In Rails, at least, it''s those damn class-macro type things which do  
all the method loading, but those are never stubbed out (they can''t  
be, since the code needs to be loaded first).

Any concrete examples come to mind?

Scott

>
>
> Cheers,
> David
>
>>
>> Scott
>>
>>>
>>> --
>>> Posted via http://www.ruby-forum.com/.
>>> _______________________________________________
>>> rspec-users mailing list
>>> rspec-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/rspec-users
>>
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
>>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users

> I''d rather have it be a known problem, but a consistent problem,
then
> a partially solved problem that will inevitably cause more pain that
> it does today :)
+1

This is one of the down-sides of working in a dynamic language. We  
have to suck it up, IMO.
> Now, here''s what I don''t understand: suppose I refactor
the Engine
> class
> and rename the "ignite" method to "turn_on". If I
re-run the example
> from above, the test is still green - but it shouldn''t be, since
the
> "ignite" method doesn''t exist on the Engine class
anymore.
>
> So I''m wondering...what am I doing wrong? I feel like my tests
should
> help catch these sorts of things when I do a refactoring - my hunch is
> that I''m missing a step here.

I asked Steve Freeman (author of the original JMock) about this very  
question the other week. His reply was straightfowrard - "that''s
just
crap testing!". In other words, as Scott said, if you''re using
mocks
for your unit tests, you have to also use acceptance tests to prove  
that the whole stack integrates.

Thanks goodness for cucumber. I have already found it a massive  
blessing when refactoring.

Scott Taylor <mailing_lists at railsnewbie.com> writes:
> Actually, I take back my previous comment.  Where would you stub a
> method on an object where the method *isn''t* even loaded by
further
> (or previous) stubbing?
>
> Any concrete examples come to mind?
How about if you haven''t implemented that object yet?

Pat

On Thu, Oct 30, 2008 at 3:05 PM, Pat Maddox <pergesu at gmail.com>
wrote:
> Scott Taylor <mailing_lists at railsnewbie.com> writes:
>
>> Actually, I take back my previous comment.  Where would you stub a
>> method on an object where the method *isn''t* even loaded by
further
>> (or previous) stubbing?
>>
>> Any concrete examples come to mind?
>
> How about if you haven''t implemented that object yet?
+1000
>
> Pat
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

On Oct 30, 2008, at 4:05 PM, Pat Maddox wrote:
> Scott Taylor <mailing_lists at railsnewbie.com> writes:
>
>> Actually, I take back my previous comment.  Where would you stub a
>> method on an object where the method *isn''t* even loaded by
further
>> (or previous) stubbing?
>>
>> Any concrete examples come to mind?
>
> How about if you haven''t implemented that object yet?
It would only work if given a concrete class name (i.e. mock(Foo) as  
opposed to mock(''foo''))

No idea why you couldn''t pass a flag for regression testing:

mock(Foo, :check_methods => true)

Scott

On Oct 30, 2008, at 4:05 PM, Pat Maddox wrote:
> Scott Taylor <mailing_lists at railsnewbie.com> writes:
>
>> Actually, I take back my previous comment.  Where would you stub a
>> method on an object where the method *isn''t* even loaded by
further
>> (or previous) stubbing?
>>
>> Any concrete examples come to mind?
>
> How about if you haven''t implemented that object yet?
Of course, this also depends on what context.

Often in Rails I find myself using mocks when I have no need to - the  
dependent object is already built, and I''m using a mock just so that I
don''t hit the database at all.

Scott

> No idea why you couldn''t pass a flag for regression testing:
> 
> mock(Foo, :check_methods => true)
> 
> Scott
It seems to me this way, too - though I''m obviously biased. :)

It''s true that a lot of the dynamic stuff could be problematic - what
if
the flag was only for checking methods directly implemented on that 
class? Though I guess that''d lead down a potential dark path where
you''d
then also want the "mock" method to potentially check all mixins and 
validate, etc. etc.

On the other hand, maybe it could serve as a gentle remonstrance to 
folks to not overdo it with the "i am an 3l337 ninj4 haxx0r" 
metaprogramming? Just a thought.
-- 
Posted via http://www.ruby-forum.com/.

On Oct 30, 2008, at 7:52 PM, Sebastian W. wrote:
>> No idea why you couldn''t pass a flag for regression testing:
>>
>> mock(Foo, :check_methods => true)
>>
>> Scott
>
> It seems to me this way, too - though I''m obviously biased. :)
>
> It''s true that a lot of the dynamic stuff could be problematic -  
> what if
> the flag was only for checking methods directly implemented on that
> class? Though I guess that''d lead down a potential dark path where
> you''d
> then also want the "mock" method to potentially check all mixins
and
> validate, etc. etc.
>
> On the other hand, maybe it could serve as a gentle remonstrance to
> folks to not overdo it with the "i am an 3l337 ninj4 haxx0r"
> metaprogramming? Just a thought.
+1

Tests *are* supposed to influence the way we write production code.

Scott

> 
> +1
> 
> Tests *are* supposed to influence the way we write production code.
> 
> Scott
So maybe we could try it and see how it works out? If it makes things 
unhappy, it can always be labeled "a worthwhile effort and
experiment".
: )
-- 
Posted via http://www.ruby-forum.com/.