rspec users - Jun 2009 - Test doubles: expect "x" and don't care about anything else

I''ve had one of my recurring doubts about test doubles come up again.

The full post is here but I''ll abbreviate the content in this message  
in any case:

   https://wincent.com/blog/thinking-about-switching-to-rr

Basically, in one of my controller specs I wanted to verify that the  
following line was being called and doing the right thing:
   @comment = Comment.find params[:id]
I had a mock for this set up, but it broke when unrelated code in the  
model was modified (a complex callback which itself called  
Comment.find).

The problems were as follows:

   - A mock was more than I really needed, as I didn''t want to go  
through the complication of returning a substitute object.

   - The expectation set on the mock was too strict, because the other  
message send to Comment.find, the one I didn''t care about, was  
triggering a failure.

   - A proxy would suffice, because all I really wanted to confirm was  
that the "find" message was sent, without actually interfering with  
the returned object.

   - I basically wanted to set an expectation "that this class will  
receive this message with these params", but the frameworks didn''t
allow me to do that because in reality you can only assert "that this  
class will receive this message with these params _and not receive  
that message with any other params at any time_"

In my blog post I detailed the possible options for avoiding the  
problem, and the easiest ended up being: forget mocks and proxies  
entirely and instead test the side-effect (that the expected object  
ends up getting assigned to the "@comment" instance variable).

So the workaround worked, but RSpec''s own mock framework, and from  
what I can tell, the alternatives such as Mocha, RR et al, wouldn''t  
really let me make the kind of assertion that I wanted to make: ie.  
"confirm this message gets sent at some point, but don''t modify
the
behaviour at all, and don''t interfere with or worry about any other  
messages that get sent to the object, including messages sent to the  
method that I''m setting the expectation on".

In my ideal test-double framework, I''d like to really assert two  
things about the line of code in question:

   1. That Comment.find gets called with a specific param at some  
point in time.
   2. That the @comment instance variable gets the expected value  
assigned to it.

I literally don''t care about what other messages get sent to Comment,  
nor about other places in the code where Comment.find might get called  
with other parameters, and in any case I don''t want to actually modify
the behaviour or substitute my own return values. But it seems I can''t
do this with existing test double frameworks, and it makes it hard to  
write minimal specs with one expectation and as few test doubles as  
possible (ideally zero or one) per "it" block.

Ideally I''d want to write something like:

   it ''should find the comment'' do
     proxy(Comment).find(@comment.id.to_s)
     do_put
   end

   it ''should assign to the @comment instance variable'' do
     assigns[:comment].should == @comment
     do_put
   end

Note that with the "proxy" syntax above I''m trying to say:

   - I expect this message and these params to be sent at some point

   - I don''t care if other messages are sent

   - I don''t even care if the "find" method is also called
with
different params

   - I don''t care about the order of the messages

   - I don''t want to interfere with or substitute the return value

I don''t know whether the syntax is adequate, or whether some keyword  
other than "proxy" would be required. Another alternative I thought of
was:

   it ''should find the comment'' do
     spy(Comment)
     do_put
     Comment.should have_received.find(@comment.id.to_s)
   end

Or similar... Basically saying that I want the double framework to spy  
(proxy _and_ record) all messages to the specified receiver, and that  
afterwards I''m going to retrospectively check that among the recorded  
messages is the one I''m looking for.

What do other people think?

   - is what I''m wanting to do a reasonable approach?

   - are there any test double frameworks out there which would allow  
me to work in this way?

Cheers,
Wincent

On 28 Jun 2009, at 13:07, Wincent Colaiuta wrote:
> I''ve had one of my recurring doubts about test doubles come up
again.
>
> The full post is here but I''ll abbreviate the content in this  
> message in any case:
>
>  https://wincent.com/blog/thinking-about-switching-to-rr
>
> Basically, in one of my controller specs I wanted to verify that the  
> following line was being called and doing the right thing:
>  @comment = Comment.find params[:id]
> I had a mock for this set up, but it broke when unrelated code in  
> the model was modified (a complex callback which itself called  
> Comment.find).
I''d like to know more about how this happened. How did the model  
object''s behaviour leak into the controller spec?
> The problems were as follows:
>
>  - A mock was more than I really needed, as I didn''t want to go  
> through the complication of returning a substitute object.
>
>  - The expectation set on the mock was too strict, because the other  
> message send to Comment.find, the one I didn''t care about, was  
> triggering a failure.
>
>  - A proxy would suffice, because all I really wanted to confirm was  
> that the "find" message was sent, without actually interfering
with
> the returned object.
>
>  - I basically wanted to set an expectation "that this class will  
> receive this message with these params", but the frameworks
didn''t
> allow me to do that because in reality you can only assert "that  
> this class will receive this message with these params _and not  
> receive that message with any other params at any time_"
>
> In my blog post I detailed the possible options for avoiding the  
> problem, and the easiest ended up being: forget mocks and proxies  
> entirely and instead test the side-effect (that the expected object  
> ends up getting assigned to the "@comment" instance variable).
>
> So the workaround worked, but RSpec''s own mock framework, and from
> what I can tell, the alternatives such as Mocha, RR et al,
wouldn''t
> really let me make the kind of assertion that I wanted to make: ie.  
> "confirm this message gets sent at some point, but don''t
modify the
> behaviour at all, and don''t interfere with or worry about any
other
> messages that get sent to the object, including messages sent to the  
> method that I''m setting the expectation on".
>
> In my ideal test-double framework, I''d like to really assert two  
> things about the line of code in question:
>
>  1. That Comment.find gets called with a specific param at some  
> point in time.
>  2. That the @comment instance variable gets the expected value  
> assigned to it.
So why not use

Comment.stub!(:find).with(123).and_return(mock(Comment))
> I literally don''t care about what other messages get sent to  
> Comment, nor about other places in the code where Comment.find might  
> get called with other parameters, and in any case I don''t want to
> actually modify the behaviour or substitute my own return values.  
> But it seems I can''t do this with existing test double frameworks,
> and it makes it hard to write minimal specs with one expectation and  
> as few test doubles as possible (ideally zero or one) per "it"
block.
>
> Ideally I''d want to write something like:
>
>  it ''should find the comment'' do
>    proxy(Comment).find(@comment.id.to_s)
>    do_put
>  end
>
>  it ''should assign to the @comment instance variable'' do
>    assigns[:comment].should == @comment
>    do_put
>  end
>
> Note that with the "proxy" syntax above I''m trying to
say:
>
>  - I expect this message and these params to be sent at some point
>
>  - I don''t care if other messages are sent
>
>  - I don''t even care if the "find" method is also called
with
> different params
>
>  - I don''t care about the order of the messages
>
>  - I don''t want to interfere with or substitute the return value
>
> I don''t know whether the syntax is adequate, or whether some
keyword
> other than "proxy" would be required. Another alternative I
thought
> of was:
>
>  it ''should find the comment'' do
>    spy(Comment)
>    do_put
>    Comment.should have_received.find(@comment.id.to_s)
>  end
>
> Or similar... Basically saying that I want the double framework to  
> spy (proxy _and_ record) all messages to the specified receiver, and  
> that afterwards I''m going to retrospectively check that among the
> recorded messages is the one I''m looking for.
>
> What do other people think?
>
>  - is what I''m wanting to do a reasonable approach?
>
>  - are there any test double frameworks out there which would allow  
> me to work in this way?
>
> Cheers,
> Wincent
cheers,
Matt Wynne

http://mattwynne.net
+447974 430184

El 28/6/2009, a las 23:04, Matt Wynne escribi?:
> On 28 Jun 2009, at 13:07, Wincent Colaiuta wrote:
>
>> I''ve had one of my recurring doubts about test doubles come up
again.
>>
>> The full post is here but I''ll abbreviate the content in this
>> message in any case:
>>
>> https://wincent.com/blog/thinking-about-switching-to-rr
>>
>> Basically, in one of my controller specs I wanted to verify that  
>> the following line was being called and doing the right thing:
>> @comment = Comment.find params[:id]
>> I had a mock for this set up, but it broke when unrelated code in  
>> the model was modified (a complex callback which itself called  
>> Comment.find).
>
> I''d like to know more about how this happened. How did the model  
> object''s behaviour leak into the controller spec?
This was a spec for the controller''s "update" action, which
does a
"save" on the record. At one point a change was made to the model to  
do some complex updates in the after_save callback, and these involved  
doing another Comment.find call, but with different parameters.
>> In my ideal test-double framework, I''d like to really assert
two
>> things about the line of code in question:
>>
>> 1. That Comment.find gets called with a specific param at some  
>> point in time.
>> 2. That the @comment instance variable gets the expected value  
>> assigned to it.
>
> So why not use
>
> Comment.stub!(:find).with(123).and_return(mock(Comment))
Because there are actually two "find" calls here:

- the one I actually care about
- the other one in the after_save callback which is irrelevant to the  
controller

I original used "should_receive", not "stub", so RSpec
complained
about getting "find" with the unexpected parameters. If I change to  
"stub" then I''m losing my assertion (no longer checking that
the
message gets sent), injecting a different return value (adding  
complexity), for no visible benefit (may as well just throw away the  
expectation).

Cheers,
Wincent

On 28 Jun 2009, at 23:02, Wincent Colaiuta wrote:
> El 28/6/2009, a las 23:04, Matt Wynne escribi?:
>
>> On 28 Jun 2009, at 13:07, Wincent Colaiuta wrote:
>>
>>> I''ve had one of my recurring doubts about test doubles
come up
>>> again.
>>>
>>> The full post is here but I''ll abbreviate the content in
this
>>> message in any case:
>>>
>>> https://wincent.com/blog/thinking-about-switching-to-rr
>>>
>>> Basically, in one of my controller specs I wanted to verify that  
>>> the following line was being called and doing the right thing:
>>> @comment = Comment.find params[:id]
>>> I had a mock for this set up, but it broke when unrelated code in  
>>> the model was modified (a complex callback which itself called  
>>> Comment.find).
>>
>> I''d like to know more about how this happened. How did the
model
>> object''s behaviour leak into the controller spec?
>
> This was a spec for the controller''s "update" action,
which does a
> "save" on the record. At one point a change was made to the model
to
> do some complex updates in the after_save callback, and these  
> involved doing another Comment.find call, but with different  
> parameters.
If I understand this correctly, there was only one call from  
Controller -> Comment that you wanted to test; the other one was a  
call from Comment -> Comment that happened as a side-effect.

So I''m wondering: if you''d returned a fake (mock, stub,
whatever)
comment from your stubbed Comment.find, would that have solved the  
problem?
>
>
>>> In my ideal test-double framework, I''d like to really
assert two
>>> things about the line of code in question:
>>>
>>> 1. That Comment.find gets called with a specific param at some  
>>> point in time.
>>> 2. That the @comment instance variable gets the expected value  
>>> assigned to it.
>>
>> So why not use
>>
>> Comment.stub!(:find).with(123).and_return(mock(Comment))
>
> Because there are actually two "find" calls here:
>
> - the one I actually care about
> - the other one in the after_save callback which is irrelevant to  
> the controller
>
> I original used "should_receive", not "stub", so RSpec
complained
> about getting "find" with the unexpected parameters. If I change
to
> "stub" then I''m losing my assertion (no longer checking
that the
> message gets sent), injecting a different return value (adding  
> complexity), for no visible benefit (may as well just throw away the  
> expectation).
What I often do is put a stub in first, which will work in all the  
examples, then put a should_receive in one of the examples if (as  
seems to be the case here) it''s important to me to test the  
collaboration between the objects. So it would look like this:

describe "#update" do
   before(:each)
     @comment = mock(Comment)
     Comment.stub!(:find).and_return(@comment)
   end

   it "should call the model to try and find the comment"
     Comment.should_receive(:find).with(123).and_return(@comment)
     do_request
   end

   it "should assign the comment to the view"
     do_request
     assigns[:comment].should == @comment
   end

So the stub works in the background, then when you want to actually  
assert for the collaboration, you can override it with a  
should_receive. I find this pattern works really well for me.

cheers,
Matt Wynne

http://mattwynne.net
+447974 430184

El 29/6/2009, a las 9:33, Matt Wynne escribi?:
> On 28 Jun 2009, at 23:02, Wincent Colaiuta wrote:
>
>> El 28/6/2009, a las 23:04, Matt Wynne escribi?:
>>
>>> On 28 Jun 2009, at 13:07, Wincent Colaiuta wrote:
>>>
>>>> I''ve had one of my recurring doubts about test doubles
come up
>>>> again.
>>>>
>>>> The full post is here but I''ll abbreviate the content
in this
>>>> message in any case:
>>>>
>>>> https://wincent.com/blog/thinking-about-switching-to-rr
>>>>
>>>> Basically, in one of my controller specs I wanted to verify
that
>>>> the following line was being called and doing the right thing:
>>>> @comment = Comment.find params[:id]
>>>> I had a mock for this set up, but it broke when unrelated code
in
>>>> the model was modified (a complex callback which itself called
>>>> Comment.find).
>>>
>>> I''d like to know more about how this happened. How did the
model
>>> object''s behaviour leak into the controller spec?
>>
>> This was a spec for the controller''s "update"
action, which does a
>> "save" on the record. At one point a change was made to the
model
>> to do some complex updates in the after_save callback, and these  
>> involved doing another Comment.find call, but with different  
>> parameters.
>
> If I understand this correctly, there was only one call from  
> Controller -> Comment that you wanted to test; the other one was a  
> call from Comment -> Comment that happened as a side-effect.
Exactly.
> So I''m wondering: if you''d returned a fake (mock, stub,
whatever)
> comment from your stubbed Comment.find, would that have solved the  
> problem?
Yes, but there''s something about that that felt somehow harder than it
should be.

I mean, I had a working, dead-simple controller spec. I made an  
unrelated change in the model, and the controller spec broke.  
Returning a fake would certainly fix the breakage, but the spec would  
no longer be dead-simple and it somehow feels like one step foward,  
two steps back, that due to innocuous model changes I have to increase  
the complexity of my controller tests.

So what I''m really pining for is the ability to say "expect this  
message, but don''t return fakes -- just proxy the message through and  
return the real return value -- and don''t worry about any other  
messages".

Cheers,
Wincent

Matt Wynne wrote:
>
> On 28 Jun 2009, at 23:02, Wincent Colaiuta wrote:
>
>> El 28/6/2009, a las 23:04, Matt Wynne escribi?:
>>
>>> On 28 Jun 2009, at 13:07, Wincent Colaiuta wrote:
>>>
>>>> I''ve had one of my recurring doubts about test doubles
come up again.
>>>>
>>>> The full post is here but I''ll abbreviate the content
in this
>>>> message in any case:
>>>>
>>>> https://wincent.com/blog/thinking-about-switching-to-rr
>>>>
>>>> Basically, in one of my controller specs I wanted to verify
that
>>>> the following line was being called and doing the right thing:
>>>> @comment = Comment.find params[:id]
>>>> I had a mock for this set up, but it broke when unrelated code
in
>>>> the model was modified (a complex callback which itself called 
>>>> Comment.find).
>>>
>>> I''d like to know more about how this happened. How did the
model
>>> object''s behaviour leak into the controller spec?
>>
>> This was a spec for the controller''s "update"
action, which does a
>> "save" on the record. At one point a change was made to the
model to
>> do some complex updates in the after_save callback, and these 
>> involved doing another Comment.find call, but with different
parameters.
>
> If I understand this correctly, there was only one call from 
> Controller -> Comment that you wanted to test; the other one was a 
> call from Comment -> Comment that happened as a side-effect.
>
> So I''m wondering: if you''d returned a fake (mock, stub,
whatever)
> comment from your stubbed Comment.find, would that have solved the 
> problem?
>
>>
>>
>>>> In my ideal test-double framework, I''d like to really
assert two
>>>> things about the line of code in question:
>>>>
>>>> 1. That Comment.find gets called with a specific param at some 
>>>> point in time.
>>>> 2. That the @comment instance variable gets the expected value 
>>>> assigned to it.
>>>
>>> So why not use
>>>
>>> Comment.stub!(:find).with(123).and_return(mock(Comment))
>>
>> Because there are actually two "find" calls here:
>>
>> - the one I actually care about
>> - the other one in the after_save callback which is irrelevant to the 
>> controller
>>
>> I original used "should_receive", not "stub", so
RSpec complained
>> about getting "find" with the unexpected parameters. If I
change to
>> "stub" then I''m losing my assertion (no longer
checking that the
>> message gets sent), injecting a different return value (adding 
>> complexity), for no visible benefit (may as well just throw away the 
>> expectation).
>
> What I often do is put a stub in first, which will work in all the 
> examples, then put a should_receive in one of the examples if (as 
> seems to be the case here) it''s important to me to test the 
> collaboration between the objects. So it would look like this:
>
> describe "#update" do
>   before(:each)
>     @comment = mock(Comment)
>     Comment.stub!(:find).and_return(@comment)
>   end
>
>   it "should call the model to try and find the comment"
>     Comment.should_receive(:find).with(123).and_return(@comment)
>     do_request
>   end
You probably know this, but for the benefit of others... Pat made a 
change a while back that makes it so the stubbed return value will still 
be returned even if an expectation is added.  Meaning, assuming the stub 
is in the before block, you can change the expectation to:
    Comment.should_receive(:find).with(123)  # this will still return 
@comment

-Ben

El 29/6/2009, a las 16:26, Ben Mabey escribi?:
> You probably know this, but for the benefit of others... Pat made a  
> change a while back that makes it so the stubbed return value will  
> still be returned even if an expectation is added.  Meaning,  
> assuming the stub is in the before block, you can change the  
> expectation to:
>   Comment.should_receive(:find).with(123)  # this will still return  
> @comment
I didn''t know that, but it''s pretty awesome. Basically means
that
RSpec mocks can double as proxies now. That''s pretty neat.

Cheers,
Wincent

El 29/6/2009, a las 17:15, Wincent Colaiuta escribi?:
> El 29/6/2009, a las 16:26, Ben Mabey escribi?:
>
>> You probably know this, but for the benefit of others... Pat made a  
>> change a while back that makes it so the stubbed return value will  
>> still be returned even if an expectation is added.  Meaning,  
>> assuming the stub is in the before block, you can change the  
>> expectation to:
>>  Comment.should_receive(:find).with(123)  # this will still return  
>> @comment
>
> I didn''t know that, but it''s pretty awesome. Basically
means that
> RSpec mocks can double as proxies now. That''s pretty neat.
Er, I stand corrected. I went back looking for the change and found it  
(commit 72facc08), and then I re-read your description. Forget what I  
said about proxying.

Cheers,
Wincent