rspec users - Sep 2008 - Adding Test Spy into Rspec mocking framework

Thanks to a pointer from lizkeogh.com site I''ve been discovering the
joy
of the Mockito (http://mockito.org/) mocking framework for Java. It
verifies behaviour after the action. It struck me how this matched the
format I use in my Ruby cucumber features. Within Rspec if we could use
post behaviour verifications I could unify the format of my specs with
my features. Specs could look like:

---
it "should make verfication of behaviour after action"
  #Given
  setup some mock/stubs

  #When
  perform action

  #Then
  Check whether the behaviour was as expected.
end
---

While we can do this with output checks in Rspec we cannot do such a
thing with ''should_receive''.

Having a consistent format for specs and stories sounds like a great
idea to me.
Also the separation of the verification from the Given set-up is nice.

So I''m proposing the idea of adding
''should_have_received'' and test_spy
in Rspec:

---
x = test_spy(''like a mock but more into espionage'')

#Action which does not fail on unexpected method but just records calls.
some.action(x)

x.should_have_received(:method).and_returned_with(''some
string'')
---

What I''m still looking at is if the idea of post validation is possible
in the Rspec framework mocking framework. I know RR
(http://github.com/btakita/rr/tree/master) is looking at implementing
the Spy pattern which will enable this sort of testing.

Do people think this would be a good addition to Rspec?

((I''m really excited with the idea and I have started playing around
with some code for a prototype. If its not for Rspec I can always look
at it for RR.))

I''ve added a issue in lighthouse (though its behaving very strangely at
the moment not picking my ticket up in searches/tags)
http://rspec.lighthouseapp.com/projects/5645/tickets/527-adding-spy-to-rspecs-mocking-framework

--
Joseph Wilk
http://www.joesniff.co.uk
-- 
Posted via http://www.ruby-forum.com/.

On Mon, Sep 15, 2008 at 5:09 AM, Joseph Wilk <lists at ruby-forum.com>
wrote:
> Thanks to a pointer from lizkeogh.com site I''ve been discovering
the joy
> of the Mockito (http://mockito.org/) mocking framework for Java. It
> verifies behaviour after the action. It struck me how this matched the
> format I use in my Ruby cucumber features. Within Rspec if we could use
> post behaviour verifications I could unify the format of my specs with
> my features. Specs could look like:
>
> ---
> it "should make verfication of behaviour after action"
>  #Given
>  setup some mock/stubs
>
>  #When
>  perform action
>
>  #Then
>  Check whether the behaviour was as expected.
> end
> ---
>
> While we can do this with output checks in Rspec we cannot do such a
> thing with ''should_receive''.
>
> Having a consistent format for specs and stories sounds like a great
> idea to me.
> Also the separation of the verification from the Given set-up is nice.
>
> So I''m proposing the idea of adding
''should_have_received'' and test_spy
> in Rspec:
>
> ---
> x = test_spy(''like a mock but more into espionage'')
>
> #Action which does not fail on unexpected method but just records calls.
> some.action(x)
>
> x.should_have_received(:method).and_returned_with(''some
string'')
> ---
>
> What I''m still looking at is if the idea of post validation is
possible
> in the Rspec framework mocking framework. I know RR
> (http://github.com/btakita/rr/tree/master) is looking at implementing
> the Spy pattern which will enable this sort of testing.
The main technical problem with this is that RSpec''s mocks are
designed to fail fast, and that would have to be turned off for
objects intended to simply record calls. It might be that what you''re
talking about should really be a completely separate implementation
which, I might add, would probably be quite trivial compared to the
complexities of failing fast in a variety of contexts.
> Do people think this would be a good addition to Rspec?
My initial reaction is "no" based on a couple of things:

1. Mocks are intended for the indirect discovery of new objects and
their behaviours while driving out their consumers with code examples.
If our stories start using them for this purpose, we''re heading
towards what I view is too granular a view from the story level.

2. Mocks can get separated from the real implementations, meaning that
all the code examples can pass yet things can fall apart in production
if the mocks aren''t maintained in alignment with the real
objects/methods. The risk of using mocks in code examples is
significantly mitigated with the use of scenarios as integration
tests. As soon as we start using mocks in stories as well, we''re
reintroducing that risk and likely increasing it.

Now with all of that said, I think this is worth exploring. I''m just
not sure that exploration should occur in rspec proper. What I''d
encourage you to do is write either a standalone library with hooks
that you can exploit from rspec examples and/or rspec/cucumber
scenarios or write an extension library for rspec''s mocking framework.

I''d also recommend that everyone interested in this thread read
Dan''s
article about endotesting/mockito:
http://dannorth.net/2008/09/the-end-of-endotesting.
> ((I''m really excited with the idea and I have started playing
around
> with some code for a prototype. If its not for Rspec I can always look
> at it for RR.))
Doesn''t RR already support test spy?
> I''ve added a issue in lighthouse (though its behaving very
strangely at
> the moment not picking my ticket up in searches/tags)
>
http://rspec.lighthouseapp.com/projects/5645/tickets/527-adding-spy-to-rspecs-mocking-framework
Thanks for getting this conversation going.

Cheers,
David
> --
> Joseph Wilk
> http://www.joesniff.co.uk

> So I''m proposing the idea of adding
''should_have_received'' and test_spy
> in Rspec:
>
> ---
> x = test_spy(''like a mock but more into espionage'')
>
> #Action which does not fail on unexpected method but just records calls.
> some.action(x)
>
> x.should_have_received(:method).and_returned_with(''some
string'')
I actually started implementing this the day before Dan posted that.
It works, but I don''t have all the nice error messages and stuff yet.
Will take me a couple days to get around to finishing it probably.

Pat

On Mon, Sep 15, 2008 at 7:39 AM, Pat Maddox <pergesu at gmail.com>
wrote:
>> So I''m proposing the idea of adding
''should_have_received'' and test_spy
>> in Rspec:
>>
>> ---
>> x = test_spy(''like a mock but more into espionage'')
>>
>> #Action which does not fail on unexpected method but just records
calls.
>> some.action(x)
>>
>> x.should_have_received(:method).and_returned_with(''some
string'')
>
> I actually started implementing this the day before Dan posted that.
> It works, but I don''t have all the nice error messages and stuff
yet.
> Will take me a couple days to get around to finishing it probably.
Have you implemented this within the mock framework or as a separate concept?

David
>
> Pat
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

On Mon, Sep 15, 2008 at 8:43 AM, David Chelimsky <dchelimsky at gmail.com>
wrote:
> On Mon, Sep 15, 2008 at 7:39 AM, Pat Maddox <pergesu at gmail.com>
wrote:
>>> So I''m proposing the idea of adding
''should_have_received'' and test_spy
>>> in Rspec:
>>>
>>> ---
>>> x = test_spy(''like a mock but more into
espionage'')
>>>
>>> #Action which does not fail on unexpected method but just records
calls.
>>> some.action(x)
>>>
>>> x.should_have_received(:method).and_returned_with(''some
string'')
>>
>> I actually started implementing this the day before Dan posted that.
>> It works, but I don''t have all the nice error messages and
stuff yet.
>> Will take me a couple days to get around to finishing it probably.
>
> Have you implemented this within the mock framework or as a separate
concept?
It''s just a new matcher.  I did mess with the mock framework a little
in that I record calls every time it receives a message, rather than
only when it hits method_missing.

I will be restructuring the mock internals to better support this, I
think.  But of course it won''t change the public API, and mocks will
still have the fail-fast behavior by default if that''s what you want.

Basic example of what I''m doing...

o = stub("stub", :foo => true)
o.foo
o.should have_received(:foo)

It''s not a true spy in the sense that it accepts any message (you can
use :null_object for that).  My only desire here was to unify the
format of interaction-based tests with the arrange/act/assert format
typical of state-based tests.

Pat

On Mon, Sep 15, 2008 at 7:18 AM, David Chelimsky <dchelimsky at gmail.com>
wrote:
> On Mon, Sep 15, 2008 at 5:09 AM, Joseph Wilk <lists at
ruby-forum.com> wrote:
>> Thanks to a pointer from lizkeogh.com site I''ve been
discovering the joy
>> of the Mockito (http://mockito.org/) mocking framework for Java. It
>> verifies behaviour after the action. It struck me how this matched the
>> format I use in my Ruby cucumber features. Within Rspec if we could use
>> post behaviour verifications I could unify the format of my specs with
>> my features. Specs could look like:
>>
>> ---
>> it "should make verfication of behaviour after action"
>>  #Given
>>  setup some mock/stubs
>>
>>  #When
>>  perform action
>>
>>  #Then
>>  Check whether the behaviour was as expected.
>> end
>> ---
>>
>> While we can do this with output checks in Rspec we cannot do such a
>> thing with ''should_receive''.
>>
>> Having a consistent format for specs and stories sounds like a great
>> idea to me.
>> Also the separation of the verification from the Given set-up is nice.
>>
>> So I''m proposing the idea of adding
''should_have_received'' and test_spy
>> in Rspec:
>>
>> ---
>> x = test_spy(''like a mock but more into espionage'')
>>
>> #Action which does not fail on unexpected method but just records
calls.
>> some.action(x)
>>
>> x.should_have_received(:method).and_returned_with(''some
string'')
>> ---
>>
>> What I''m still looking at is if the idea of post validation is
possible
>> in the Rspec framework mocking framework. I know RR
>> (http://github.com/btakita/rr/tree/master) is looking at implementing
>> the Spy pattern which will enable this sort of testing.
>
> The main technical problem with this is that RSpec''s mocks are
> designed to fail fast, and that would have to be turned off for
> objects intended to simply record calls. It might be that what
you''re
> talking about should really be a completely separate implementation
> which, I might add, would probably be quite trivial compared to the
> complexities of failing fast in a variety of contexts.
>
>> Do people think this would be a good addition to Rspec?
>
> My initial reaction is "no" based on a couple of things:
>
> 1. Mocks are intended for the indirect discovery of new objects and
> their behaviours while driving out their consumers with code examples.
> If our stories start using them for this purpose, we''re heading
> towards what I view is too granular a view from the story level.
>
> 2. Mocks can get separated from the real implementations, meaning that
> all the code examples can pass yet things can fall apart in production
> if the mocks aren''t maintained in alignment with the real
> objects/methods. The risk of using mocks in code examples is
> significantly mitigated with the use of scenarios as integration
> tests. As soon as we start using mocks in stories as well, we''re
> reintroducing that risk and likely increasing it.
Just re-read your initial post Joseph - realize that you were not
proposing adding this to stories, so please disregard those concerns.

Cheers,
David
>
> Now with all of that said, I think this is worth exploring. I''m
just
> not sure that exploration should occur in rspec proper. What I''d
> encourage you to do is write either a standalone library with hooks
> that you can exploit from rspec examples and/or rspec/cucumber
> scenarios or write an extension library for rspec''s mocking
framework.
>
> I''d also recommend that everyone interested in this thread read
Dan''s
> article about endotesting/mockito:
> http://dannorth.net/2008/09/the-end-of-endotesting.
>
>> ((I''m really excited with the idea and I have started playing
around
>> with some code for a prototype. If its not for Rspec I can always look
>> at it for RR.))
>
> Doesn''t RR already support test spy?
>
>> I''ve added a issue in lighthouse (though its behaving very
strangely at
>> the moment not picking my ticket up in searches/tags)
>>
http://rspec.lighthouseapp.com/projects/5645/tickets/527-adding-spy-to-rspecs-mocking-framework
>
> Thanks for getting this conversation going.
>
> Cheers,
> David
>
>> --
>> Joseph Wilk
>> http://www.joesniff.co.uk
>

>Doesn''t RR already support test spy?
I don''t believe so. There is nothing on the github page or anything I 
can spot in the current code in git. The github docs do mention it as 
something they are aiming to do.
> o = stub("stub", :foo => true)
> o.foo
> o.should have_received(:foo)
That sounds really interesting Pat. Is is something that''s living in 
github? I would be interested to see what you are coming up with when 
you get a chance to let it out into the wild.
>Now with all of that said, I think this is worth exploring. I''m
just
>not sure that exploration should occur in Rspec proper. What I''d
>encourage you to do is write either a standalone library with hooks
>that you can exploit from Rspec examples and/or rspec/cucumber
>scenarios or write an extension library for Rspec''s mocking
framework.
Thanks for the feedback. I''ll take a look at making it something
outside
of Rspec as you suggested. I''ll notify the mailing list of any progress
I make.

To follow on from David''s link you can see the slides from agile 2008 
where Mockito makes an introduction 
http://szczepiq.files.wordpress.com/2008/08/dont-give-up-on-mocking.pdf

And from the creator of mockito 
http://monkeyisland.pl/2008/01/14/mockito/
(whose URL brings back good old memories.)

Have a good evening/morning/afternoon,
--
Joseph Wilk
http://www.joesniff.co.uk

-- 
Posted via http://www.ruby-forum.com/.

On Mon, Sep 15, 2008 at 12:54 PM, Joseph Wilk <lists at ruby-forum.com>
wrote:
>>Doesn''t RR already support test spy?
> I don''t believe so. There is nothing on the github page or
anything I
> can spot in the current code in git. The github docs do mention it as
> something they are aiming to do.
>
>> o = stub("stub", :foo => true)
>> o.foo
>> o.should have_received(:foo)
>
> That sounds really interesting Pat. Is is something that''s living
in
> github? I would be interested to see what you are coming up with when
> you get a chance to let it out into the wild.
It''s not up there yet, but I can push it when I get home.  Granted,
it''s far from complete, but the basics are in place.  I need to pull
out some of the existing expectation code so we can easily reuse it in
the post hoc verification style.

Pat

On Mon, Sep 15, 2008 at 2:28 PM, Pat Maddox <pergesu at gmail.com>
wrote:
> On Mon, Sep 15, 2008 at 12:54 PM, Joseph Wilk <lists at
ruby-forum.com> wrote:
>>>Doesn''t RR already support test spy?
>> I don''t believe so. There is nothing on the github page or
anything I
>> can spot in the current code in git. The github docs do mention it as
>> something they are aiming to do.
>>
>>> o = stub("stub", :foo => true)
>>> o.foo
>>> o.should have_received(:foo)
>>
>> That sounds really interesting Pat. Is is something that''s
living in
>> github? I would be interested to see what you are coming up with when
>> you get a chance to let it out into the wild.
>
> It''s not up there yet, but I can push it when I get home. 
Granted,
> it''s far from complete, but the basics are in place.  I need to
pull
> out some of the existing expectation code so we can easily reuse it in
> the post hoc verification style.
You have been a busy man recently Pat! Kudos to all the work you''ve
been doing providing goodness for the community,

-- 
Zach Dennis
http://www.continuousthinking.com
http://www.mutuallyhuman.com

On Mon, Sep 15, 2008 at 9:54 AM, Joseph Wilk <lists at ruby-forum.com>
wrote:
>>Doesn''t RR already support test spy?
> I don''t believe so. There is nothing on the github page or
anything I
> can spot in the current code in git. The github docs do mention it as
> something they are aiming to do.
No, spies have not been implemented yet on RR.
>
>> o = stub("stub", :foo => true)
>> o.foo
>> o.should have_received(:foo)
Thats pretty nice.
>
> That sounds really interesting Pat. Is is something that''s living
in
> github? I would be interested to see what you are coming up with when
> you get a chance to let it out into the wild.
>
>>Now with all of that said, I think this is worth exploring. I''m
just
>>not sure that exploration should occur in Rspec proper. What
I''d
>>encourage you to do is write either a standalone library with hooks
>>that you can exploit from Rspec examples and/or rspec/cucumber
>>scenarios or write an extension library for Rspec''s mocking
framework.
>
> Thanks for the feedback. I''ll take a look at making it something
outside
> of Rspec as you suggested. I''ll notify the mailing list of any
progress
> I make.
>
> To follow on from David''s link you can see the slides from agile
2008
> where Mockito makes an introduction
> http://szczepiq.files.wordpress.com/2008/08/dont-give-up-on-mocking.pdf
>
> And from the creator of mockito
> http://monkeyisland.pl/2008/01/14/mockito/
> (whose URL brings back good old memories.)
>
> Have a good evening/morning/afternoon,
> --
> Joseph Wilk
> http://www.joesniff.co.uk
Brian
>
> --
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

I''ve been doing further work on adding the Test Spy to the RR mocking 
framework.

During my background research I came across a Spy like mocking framework 
called ''Not A Mock'' by Pete Yandell.

This comes with Rspec hooks. It uses a similar syntax to what we have 
discussed here and as Pat mentioned does some of this through a Rspec 
Matcher:
>object.should have_received(:length).without_args.and_returned(42)
>object.should have_received(:name).twice
It works on real objects or stub objects and you have to register the 
methods to track:
>object.track_methods(:name, :length)
http://notahat.com/not_a_mock
http://github.com/notahat/not_a_mock/tree/master

Thought this might be of interest to you Pat.

--
Joseph Wilk
http://www.joesniff.co.uk


Pat Maddox wrote:
> On Mon, Sep 15, 2008 at 8:43 AM, David Chelimsky <dchelimsky at
gmail.com>
> wrote:
>>>>
x.should_have_received(:method).and_returned_with(''some
string'')
>>>
>>> I actually started implementing this the day before Dan posted
that.
>>> It works, but I don''t have all the nice error messages and
stuff yet.
>>> Will take me a couple days to get around to finishing it probably.
>>
>> Have you implemented this within the mock framework or as a separate
concept?
> 
> It''s just a new matcher.  I did mess with the mock framework a
little
> in that I record calls every time it receives a message, rather than
> only when it hits method_missing.
> 
> I will be restructuring the mock internals to better support this, I
> think.  But of course it won''t change the public API, and mocks
will
> still have the fail-fast behavior by default if that''s what you
want.
> 
> Basic example of what I''m doing...
> 
> o = stub("stub", :foo => true)
> o.foo
> o.should have_received(:foo)
> 
> It''s not a true spy in the sense that it accepts any message (you
can
> use :null_object for that).  My only desire here was to unify the
> format of interaction-based tests with the arrange/act/assert format
> typical of state-based tests.
> 
> Pat
-- 
Posted via http://www.ruby-forum.com/.

Joseph Wilk <lists at ruby-forum.com> writes:
> http://notahat.com/not_a_mock
> http://github.com/notahat/not_a_mock/tree/master
>
> Thought this might be of interest to you Pat.
This is awesome.  Thanks, Joseph.

On Sun, Sep 21, 2008 at 11:17 AM, Joseph Wilk <lists at ruby-forum.com>
wrote:
> I''ve been doing further work on adding the Test Spy to the RR
mocking
> framework.
Awesome! Do you have a clone available?
>
> During my background research I came across a Spy like mocking framework
> called ''Not A Mock'' by Pete Yandell.
>
> This comes with Rspec hooks. It uses a similar syntax to what we have
> discussed here and as Pat mentioned does some of this through a Rspec
> Matcher:
>
>>object.should have_received(:length).without_args.and_returned(42)
>>object.should have_received(:name).twice
>
> It works on real objects or stub objects and you have to register the
> methods to track:
>
>>object.track_methods(:name, :length)
>
> http://notahat.com/not_a_mock
> http://github.com/notahat/not_a_mock/tree/master
>
> Thought this might be of interest to you Pat.
>
> --
> Joseph Wilk
> http://www.joesniff.co.uk
>
>
> Pat Maddox wrote:
>> On Mon, Sep 15, 2008 at 8:43 AM, David Chelimsky <dchelimsky at
gmail.com>
>> wrote:
>>>>>
x.should_have_received(:method).and_returned_with(''some
string'')
>>>>
>>>> I actually started implementing this the day before Dan posted
that.
>>>> It works, but I don''t have all the nice error messages
and stuff yet.
>>>> Will take me a couple days to get around to finishing it
probably.
>>>
>>> Have you implemented this within the mock framework or as a
separate concept?
>>
>> It''s just a new matcher.  I did mess with the mock framework a
little
>> in that I record calls every time it receives a message, rather than
>> only when it hits method_missing.
>>
>> I will be restructuring the mock internals to better support this, I
>> think.  But of course it won''t change the public API, and
mocks will
>> still have the fail-fast behavior by default if that''s what
you want.
>>
>> Basic example of what I''m doing...
>>
>> o = stub("stub", :foo => true)
>> o.foo
>> o.should have_received(:foo)
>>
>> It''s not a true spy in the sense that it accepts any message
(you can
>> use :null_object for that).  My only desire here was to unify the
>> format of interaction-based tests with the arrange/act/assert format
>> typical of state-based tests.
>>
>> Pat
>
> --
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

Brian Takita wrote:
> On Sun, Sep 21, 2008 at 11:17 AM, Joseph Wilk <lists at
ruby-forum.com> wrote:
>   
>> I''ve been doing further work on adding the Test Spy to the RR
mocking
>> framework.
>>     
> Awesome! Do you have a clone available?
I''m not quite ready to push my work up to Github. I''ve been
trying lots
of spikes experimenting with what a spy means in RR. I''ll let you know 
as soon as I push my work up to github.

I''ll share my thoughts so far on what I think RR''s
interpretation of a
Spy could be. In the traditional sense a spy is a stub object that 
records method invocations.

But RR is all about taking real objects and using double injection. It 
feels out of place to introduce a stub object as a spy. So the direction 
I''ve been leaning towards is a spy should be a real object that logs
its
calls. Those calls follow the real implementation. You can still stub 
the spy.

This feels like the most natural solution for RR, though its much harder 
to implement than just a stub object :)

@@@
class Example
  def shout(words)
    "#{words}!"
  end
end

example = Example.new
spy(example)

example.shout("monkeys")
 >> "monkeys!"

verify(example).shout("monkeys")
@@@

Whats your opinion Brian? Anyone else?

Oh, I''ve really enjoyed going through RR''s source and
I''ve learnt a lot
from it, thanks!
--
Joseph Wilk
http://www.joesniff.co.uk