rspec users - Apr 2011 - Mocking/Stubbing behavior question

I''ve run into some strange behavior in porting my specs from rspec to
rspec2.
I am wondering if I am doing something wrong, or if I''ve misunderstood
something, or if this is some kind of bug.

Look at the specs below: a
Both examples will pass if run singly. 
The second one will fail if I run both examples.

Can someone tell me why the second one fails the way it does? 

It has to do with the conditional assignment of @my_foo, but I''m not
sure why @stupid_mock is getting disconnected.
My thinking is that the class Bar is returning the same object in both tests,
and that the stub call in the second test is being sent to a different object.

Still, it did not behave this way in the last version, so I don''t know
what I''m missing.

require ''spec_helper''

class Foo
end

class Bar
  def self.my_foo
    @my_foo ||= Foo.new
  end
  def self.perform
    my_foo.do_something
  end
end

describe Foo do

  before(:each) do
    @stupid_mock = double(''wtf'')
    Foo.stub(:new => @stupid_mock)
  end
  
  it "passes here" do
   
@stupid_mock.should_receive(:do_something).and_return(''value'')
    Bar.perform
  end
  
  it "fails here" do
    @stupid_mock.stub(:do_something => ''value'')
    Bar.perform
    # double "wtf" received unexpected message :do_something with (no
args)
  end
  
end
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20110425/dded785d/attachment.html>

On Apr 25, 2011, at 1:36 PM, Matthew Van Horn wrote:
> I''ve run into some strange behavior in porting my specs from rspec
to rspec2.
> I am wondering if I am doing something wrong, or if I''ve
misunderstood something, or if this is some kind of bug.
> 
> Look at the specs below: a
> Both examples will pass if run singly. 
> The second one will fail if I run both examples.
> 
> Can someone tell me why the second one fails the way it does? 
> 
> It has to do with the conditional assignment of @my_foo, but I''m
not sure why @stupid_mock is getting disconnected.
> My thinking is that the class Bar is returning the same object in both
tests, and that the stub call in the second test is being sent to a different
object.
> 
> Still, it did not behave this way in the last version, so I don''t
know what I''m missing.
> 
> require ''spec_helper''
> 
> class Foo
> end
> 
> class Bar
>   def self.my_foo
>     @my_foo ||= Foo.new
>   end
>   def self.perform
>     my_foo.do_something
>   end
> end
> 
> describe Foo do
> 
>   before(:each) do
>     @stupid_mock = double(''wtf'')
>     Foo.stub(:new => @stupid_mock)
>   end
>   
>   it "passes here" do
>    
@stupid_mock.should_receive(:do_something).and_return(''value'')
>     Bar.perform
>   end
>   
>   it "fails here" do
>     @stupid_mock.stub(:do_something => ''value'')
>     Bar.perform
>     # double "wtf" received unexpected message :do_something with
(no args)
>   end
>   
> end
btw - I just realized, that should be: "describe Bar do"
and, I can solve the problem by doing: Bar.stub(:my_foo => @stupid_mock)
instead of Foo.stub(:new => @stupid_mock) but I really don''t like
stubbing methods on the class I am testing.

-- matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20110425/e199b0b3/attachment.html>

Em 25-04-2011 14:58, Matthew Van Horn escreveu:
>
> On Apr 25, 2011, at 1:36 PM, Matthew Van Horn wrote:
>
>> I''ve run into some strange behavior in porting my specs from
rspec to
>> rspec2.
>> I am wondering if I am doing something wrong, or if I''ve 
>> misunderstood something, or if this is some kind of bug.
>>
>> Look at the specs below: a
>> Both examples will pass if run singly.
>> The second one will fail if I run both examples.
>>
>> Can someone tell me why the second one fails the way it does?
>>
>> It has to do with the conditional assignment of @my_foo, but
I''m not
>> sure why @stupid_mock is getting disconnected.
>> My thinking is that the class Bar is returning the same object in 
>> both tests, and that the stub call in the second test is being sent 
>> to a different object.
>>
>> Still, it did not behave this way in the last version, so I
don''t
>> know what I''m missing.
>>
>> require ''spec_helper''
>>
>> class Foo
>> end
>>
>> class Bar
>>   def self.my_foo
>>     @my_foo ||= Foo.new
>>   end
>>   def self.perform
>>     my_foo.do_something
>>   end
>> end
>>
>> describe Foo do
>>
>>   before(:each) do
>>     @stupid_mock = double(''wtf'')
>>     Foo.stub(:new => @stupid_mock)
>>   end
>>   it "passes here" do
>>    
@stupid_mock.should_receive(:do_something).and_return(''value'')
>>     Bar.perform
>>   end
>>   it "fails here" do
>>     @stupid_mock.stub(:do_something => ''value'')
>>     Bar.perform
>>     # double "wtf" received unexpected message :do_something
with (no
>> args)
>>   end
>> end
>
> btw - I just realized, that should be: "describe *Bar* do"
> and, I can solve the problem by doing: Bar.stub(:my_foo => 
> @stupid_mock) instead of Foo.stub(:new => @stupid_mock) but I really 
> don''t like stubbing methods on the class I am testing.
>
The problem is that "new" is not a regular method on Ruby. I
don''t mind
mocking some methods of the class being tested in some cases.

For instance, I have a class that does lots of calculation based on 
numerous input data. It would be impossible to provide all possible 
combinations of input data for testing the calculation of some methods.

Consider the following class:

class InterestingCalculator
     include Singleton
     def a
         @a ||= hard_calculation_based_on(@several_inputs)
     end

     def b
         return 0 if a < 0
         calculate_using a
     end
end

Then I would write something like:

describe InterestingCalculator do
   example "b should be 0 when a < 0" do
     InterestingCalculator.instance.stub!(:a).and_return(-10)
     InterestingCalculator.instance.b.should == 0
   end

   example "b should do something great when a >= 0" do
     InterestingCalculator.instance.stub!(:a).and_return(10)
     InterestingCalculator.instance.b.should == 786
   end
end

At least, this makes sense to me...

Best regards,

Rodrigo.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20110426/715bdb0b/attachment-0001.html>

On Apr 25, 2011, at 11:30 PM, Rodrigo Rosenfeld Rosas wrote:
> Em 25-04-2011 14:58, Matthew Van Horn escreveu:
>> 
>> 
>> On Apr 25, 2011, at 1:36 PM, Matthew Van Horn wrote:
>> 
>>> I''ve run into some strange behavior in porting my specs
from rspec to rspec2.
>>> I am wondering if I am doing something wrong, or if I''ve
misunderstood something, or if this is some kind of bug.
>>> 
>>> Look at the specs below: a
>>> Both examples will pass if run singly. 
>>> The second one will fail if I run both examples.
>>> 
>>> Can someone tell me why the second one fails the way it does? 
>>> 
>>> It has to do with the conditional assignment of @my_foo, but
I''m not sure why @stupid_mock is getting disconnected.
>>> My thinking is that the class Bar is returning the same object in
both tests, and that the stub call in the second test is being sent to a
different object.
>>> 
>>> Still, it did not behave this way in the last version, so I
don''t know what I''m missing.
>>> 
>>> require ''spec_helper''
>>> 
>>> class Foo
>>> end
>>> 
>>> class Bar
>>>   def self.my_foo
>>>     @my_foo ||= Foo.new
>>>   end
>>>   def self.perform
>>>     my_foo.do_something
>>>   end
>>> end
>>> 
>>> describe Foo do
>>> 
>>>   before(:each) do
>>>     @stupid_mock = double(''wtf'')
>>>     Foo.stub(:new => @stupid_mock)
>>>   end
>>>   
>>>   it "passes here" do
>>>    
@stupid_mock.should_receive(:do_something).and_return(''value'')
>>>     Bar.perform
>>>   end
>>>   
>>>   it "fails here" do
>>>     @stupid_mock.stub(:do_something =>
''value'')
>>>     Bar.perform
>>>     # double "wtf" received unexpected message
:do_something with (no args)
>>>   end
>>>   
>>> end
>> 
>> btw - I just realized, that should be: "describe Bar do"
>> and, I can solve the problem by doing: Bar.stub(:my_foo =>
@stupid_mock) instead of Foo.stub(:new => @stupid_mock) but I really
don''t like stubbing methods on the class I am testing.
>> 
> 
> The problem is that "new" is not a regular method on Ruby. I
don''t mind mocking some methods of the class being tested in some
cases.
It doesn''t have to do with the "new" method - if I did:
class Bar  def self.my_foo
    @my_foo ||= Foo.make_one_of_these
  end
  ...
end

and

Foo.stub(:make_one_of_these => @stupid_mock)

It still would fail.

It is more of a pass-by-value/pass-by-reference issue. Because of the
conditional assignment, the variable @stupid_mock in the failing example seems
to be pointing to a different object than the one in the passing example, even
though under the old rspec, they seemed to be the same.


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20110426/6d3985c2/attachment.html>

Ok, now I understand what is your issue.
class Fooend
class Bar??def self.my_foo?? ?@my_foo ||= Foo.new??end??def self.perform??
?my_foo.do_something??endend
describe Foo do
??before(:each) do?? ?@stupid_mock = double(''wtf'')??
?Foo.stub(:new => @stupid_mock)??end????it "passes here" do??
?@stupid_mock.should_receive(:do_something).and_return(''value'')??
?Bar.perform??end????it "fails here" do??
?@stupid_mock.stub(:do_something => ''value'')??
?Bar.perform?? ?# double "wtf" received unexpected message
:do_something with (no args)??end??end

The first time Bar.foo is being called, it caches Foo.new. Since you stubbed
Foo.new to return a mock before running Bar.foo for the first time, this mock is
cached inside Bar, so any subsequent call will return the first created mock
object. This obviously won''t happen if you run only the other spec and
that is why it passes in this case.
This is not about passing by reference, but it is about caching Bar.my_foo.
The only way I can think of RSpec not being affected by this is to reloading all
classes before each example, which would be both complicate and costly.
You''ll probably have to rethink your testing strategies, avoiding
caching class methods or not mocking something that could affect them, or
initializing them first before running the specs...
Best regards,
Rodrigo.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20110426/e534ae0e/attachment.html>

On Apr 26, 2011, at 4:02 PM, Rodrigo Rosenfeld Rosas wrote:
> Ok, now I understand what is your issue.
> 
>>>> class Foo
>>>> end
>>>> 
>>>> class Bar
>>>>   def self.my_foo
>>>>     @my_foo ||= Foo.new
>>>>   end
>>>>   def self.perform
>>>>     my_foo.do_something
>>>>   end
>>>> end
>>>> 
>>>> describe Foo do
>>>> 
>>>>   before(:each) do
>>>>     @stupid_mock = double(''wtf'')
>>>>     Foo.stub(:new => @stupid_mock)
>>>>   end
>>>>   
>>>>   it "passes here" do
>>>>    
@stupid_mock.should_receive(:do_something).and_return(''value'')
>>>>     Bar.perform
>>>>   end
>>>>   
>>>>   it "fails here" do
>>>>     @stupid_mock.stub(:do_something =>
''value'')
>>>>     Bar.perform
>>>>     # double "wtf" received unexpected message
:do_something with (no args)
>>>>   end
>>>>   
>>>> end
>>> 
> 
>> 
> 
> The first time Bar.foo is being called, it caches Foo.new. Since you
stubbed Foo.new to return a mock before running Bar.foo for the first time, this
mock is cached inside Bar, so any subsequent call will return the first created
mock object. This obviously won''t happen if you run only the other spec
and that is why it passes in this case.
> 
> This is not about passing by reference, but it is about caching Bar.my_foo.
> 
> The only way I can think of RSpec not being affected by this is to
reloading all classes before each example, which would be both complicate and
costly.
> 
> You''ll probably have to rethink your testing strategies, avoiding
caching class methods or not mocking something that could affect them, or
initializing them first before running the specs...
> 
> Best regards,
> 
> Rodrigo.

I see now the crux of the issue - mocks are implicitly cleared out before each
example, so even though the same object is being returned, RSpec is sneakily
substituting a doppelganger for @stupid_mock. This apparently wasn''t
the case in the last version, where the instance variable would continue to be
connected to the same object.

This might be intended behavior, but it feels like a bug because that memoizing
behavior is so common.

Because I can''t set up the double in a before(:all) block due to the
above behavior, the solution is to add:
    Bar.instance_variable_set(:@my_foo, nil) # Forcibly clear Bar''s
cached instance
to the before(:each) block.

-- matt


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20110426/71a2906a/attachment-0001.html>

On Apr 26, 2011, at 5:39 PM, Matthew Van Horn wrote:
> 
> On Apr 26, 2011, at 4:02 PM, Rodrigo Rosenfeld Rosas wrote:
> 
>> Ok, now I understand what is your issue.
>> 
>>>>> class Foo
>>>>> end
>>>>> 
>>>>> class Bar
>>>>>   def self.my_foo
>>>>>     @my_foo ||= Foo.new
>>>>>   end
>>>>>   def self.perform
>>>>>     my_foo.do_something
>>>>>   end
>>>>> end
>>>>> 
>>>>> describe Foo do
>>>>> 
>>>>>   before(:each) do
>>>>>     @stupid_mock = double(''wtf'')
>>>>>     Foo.stub(:new => @stupid_mock)
>>>>>   end
>>>>>   
>>>>>   it "passes here" do
>>>>>    
@stupid_mock.should_receive(:do_something).and_return(''value'')
>>>>>     Bar.perform
>>>>>   end
>>>>>   
>>>>>   it "fails here" do
>>>>>     @stupid_mock.stub(:do_something =>
''value'')
>>>>>     Bar.perform
>>>>>     # double "wtf" received unexpected message
:do_something with (no args)
>>>>>   end
>>>>>   
>>>>> end
>>>> 
>> 
>>> 
>> 
>> The first time Bar.foo is being called, it caches Foo.new. Since you
stubbed Foo.new to return a mock before running Bar.foo for the first time, this
mock is cached inside Bar, so any subsequent call will return the first created
mock object. This obviously won''t happen if you run only the other spec
and that is why it passes in this case.
>> 
>> This is not about passing by reference, but it is about caching
Bar.my_foo.
>> 
>> The only way I can think of RSpec not being affected by this is to
reloading all classes before each example, which would be both complicate and
costly.
>> 
>> You''ll probably have to rethink your testing strategies,
avoiding caching class methods or not mocking something that could affect them,
or initializing them first before running the specs...
>> 
>> Best regards,
>> 
>> Rodrigo.
The example at the top of this thread fails the same way with rspec-1.3.1,
1.3.2, and 2.5.0. I''m guessing that this was not the real example that
you saw passing in rspec-1 and failing in rspec-2, in which case this example
probably doesn''t trigger whatever differences you were seeing between
rspec-1 and rspec-2. Are you able to show us the real code?
> I see now the crux of the issue - mocks are implicitly cleared out before
each example, so even though the same object is being returned, RSpec is
sneakily substituting a doppelganger for @stupid_mock.
RSpec is doing nothing sneaky. The code in the before hook assigns a new double
to the @stupid_mock instance variable for each example, but the my_foo method
memoizes the @stupid_mock from the first example at the class level, so it
returns that instance for all subsequent requests in the same runtime.
> This apparently wasn''t the case in the last version, where the
instance variable would continue to be connected to the same object.
before(:each) blocks have been run before each example since they were first
introduced, so this is not accurate. Whatever behavior you were seeing was not
due to RSpec keeping instance variable assignments across examples.
> This might be intended behavior, but it feels like a bug because that
memoizing behavior is so common.
Agreed it is common, but the common approach to stubbing things that are
memoized is to stub the method (in this case my_foo).
> Because I can''t set up the double in a before(:all) block due to
the above behavior, the solution is to add:
>     Bar.instance_variable_set(:@my_foo, nil) # Forcibly clear
Bar''s cached instance
> to the before(:each) block.
I''d recommend you just stub the my_foo method instead. You said you
prefer to avoid doing that earlier this thread, but I think it''s a
cleaner solution as it is isolated to the current example. Keep in mind that if
the double ends up the memoized value for my_foo, it will remain so for the rest
of the suite, whereas if you just stub my_foo for the examples you need the
double, it will be limited to those.

HTH,
David



-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20110427/e70546aa/attachment.html>

On Apr 27, 2011, at 6:40 AM, David Chelimsky wrote:
> 
> On Apr 26, 2011, at 5:39 PM, Matthew Van Horn wrote:
> 
>> 
>> On Apr 26, 2011, at 4:02 PM, Rodrigo Rosenfeld Rosas wrote:
>> 
>>> Ok, now I understand what is your issue.
>>> 
>>>>>> class Foo
>>>>>> end
>>>>>> 
>>>>>> class Bar
>>>>>>   def self.my_foo
>>>>>>     @my_foo ||= Foo.new
>>>>>>   end
>>>>>>   def self.perform
>>>>>>     my_foo.do_something
>>>>>>   end
>>>>>> end
>>>>>> 
>>>>>> describe Foo do
>>>>>> 
>>>>>>   before(:each) do
>>>>>>     @stupid_mock = double(''wtf'')
>>>>>>     Foo.stub(:new => @stupid_mock)
>>>>>>   end
>>>>>>   
>>>>>>   it "passes here" do
>>>>>>    
@stupid_mock.should_receive(:do_something).and_return(''value'')
>>>>>>     Bar.perform
>>>>>>   end
>>>>>>   
>>>>>>   it "fails here" do
>>>>>>     @stupid_mock.stub(:do_something =>
''value'')
>>>>>>     Bar.perform
>>>>>>     # double "wtf" received unexpected
message :do_something with (no args)
>>>>>>   end
>>>>>>   
>>>>>> end
>>>>> 
>>> 
>>>> 
>>> 
>>> The first time Bar.foo is being called, it caches Foo.new. Since
you stubbed Foo.new to return a mock before running Bar.foo for the first time,
this mock is cached inside Bar, so any subsequent call will return the first
created mock object. This obviously won''t happen if you run only the
other spec and that is why it passes in this case.
>>> 
>>> This is not about passing by reference, but it is about caching
Bar.my_foo.
>>> 
>>> The only way I can think of RSpec not being affected by this is to
reloading all classes before each example, which would be both complicate and
costly.
>>> 
>>> You''ll probably have to rethink your testing strategies,
avoiding caching class methods or not mocking something that could affect them,
or initializing them first before running the specs...
>>> 
>>> Best regards,
>>> 
>>> Rodrigo.
> 
> The example at the top of this thread fails the same way with rspec-1.3.1,
1.3.2, and 2.5.0. I''m guessing that this was not the real example that
you saw passing in rspec-1 and failing in rspec-2, in which case this example
probably doesn''t trigger whatever differences you were seeing between
rspec-1 and rspec-2. Are you able to show us the real code?
I''ll have to see if I can dig it up later and verify. I''ve
refactored quite a bit along the way of this upgrade, so the tests are not all
in the same place. I am reasonably sure my suite was passing, but I could be
wrong. Going from memory, in the original version, I stubbed the :do_something
method on the double before it got memoized, because I wanted that as the
default behavior except for the one example where I set the expectation.
Perhaps that is what worked in 1.3.1 and not now.
>> I see now the crux of the issue - mocks are implicitly cleared out
before each example, so even though the same object is being returned, RSpec is
sneakily substituting a doppelganger for @stupid_mock.
> RSpec is doing nothing sneaky. The code in the before hook assigns a new
double to the @stupid_mock instance variable for each example, but the my_foo
method memoizes the @stupid_mock from the first example at the class level, so
it returns that instance for all subsequent requests in the same runtime.
Email is bad for tone - ''sneakily'' is meant tongue-in-cheek.
Although, it is what came to mind when I tried to fix the problem by doing:
before(:each) do
    @stupid_mock ||= double(''wtf'')

and it still failed. I''m still kind of unclear as to which object is
holding that instance variable (and when), so it seemed weird to me that it was
getting reassigned.

>> Because I can''t set up the double in a before(:all) block due
to the above behavior, the solution is to add:
>>     Bar.instance_variable_set(:@my_foo, nil) # Forcibly clear
Bar''s cached instance
>> to the before(:each) block.
> 
> I''d recommend you just stub the my_foo method instead. You said
you prefer to avoid doing that earlier this thread, but I think it''s a
cleaner solution as it is isolated to the current example. Keep in mind that if
the double ends up the memoized value for my_foo, it will remain so for the rest
of the suite, whereas if you just stub my_foo for the examples you need the
double, it will be limited to those.
Hmm. I guess, I''ll re-think my opinions on unit testing. The isolation
is a good feature, and helps attenuate my fears of stubbing methods of the
object under test leading to ineffective tests down the road.

-- matt

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20110427/04bd2f7e/attachment-0001.html>