I''m not sure if this is by design, but I''ve stumbled across
this a
few times trying to debug my own tests. If an assertion fails the
test, and a mock expectation was to blame, the test''s failure / error
messages don''t give enough info. I''ve had to resort to
def teardown
mocha_verify
end
which just gets tedious.
I''ve created a patch against trunk v164 and included it below.
Example:
def test_something
mock = mock("some mock")
mock.expects(:message)
flunk # or fail
end
I''m expecting to get both the flunk / fail message AND the Mocha
expectation failure.
It''s a very minor change, but I''m interested in what you guys
think,
-Zach
(Sorry if anyone receives duplicates, I''m not sure if the first one
was successfully sent)
I''m not sure if this is by design, but I''ve stumbled across
this a
few times trying to debug my own tests. If an assertion fails the
test, and a missing mock expectation was to blame, the test''s
failure / error messages don''t give enough info. I''ve had to
resort to
def teardown
mocha_verify
end
which just gets tedious.
I''ve created a patch against trunk v164 and included it below.
Example:
def test_something
mock = mock("some mock")
mock.expects(:message)
flunk # or fail
end
I''m expecting to get both the flunk / fail message AND the Mocha
expectation failure.
It''s a very minor change, but I''m interested in what you guys
think,
-Zach
-------------- next part --------------
Hi Zach, On 26/07/07, Zach Moazeni <zach.lists at gmail.com> wrote:> > I''m not sure if this is by design, but I''ve stumbled across this a > few times trying to debug my own tests. If an assertion fails the > test, and a missing mock expectation was to blame, the test''s > failure / error messages don''t give enough info.This is by design. In my mind a verification failure is analogous to an assertion failure. Test::Unit does not report multiple assertion failures per test, but reports the first failure. Failing fast seems like the right thing to do. At that point all bets are off and there is no point in continuing. I wonder if the pain you are feeling is because you have too many assertions and/or expectations in each test. You might find the following article interesting... http://blog.jayfields.com/2007/06/testing-one-assertion-per-test.html Thanks for taking the time to put a patch together, but at this stage I''m not convinced it''s the way to go. -- James. http://blog.floehopper.org
James,
That''s an interesting article. I haven''t come across that, but
I do
something very similar when testing validations in ActiveRecord. The
problems I encounter more often are testing an object in isolation
where I''ll have a scenario similar to the following.
class Car
def intialize(engine)
@parts = []
@parts << engine\
end
def start
# run though a ton of part and return whether the car started
# typically there would be error handling, or you would just AND
all the parts.start
started = false
@parts.each do | part |
started = started && part.start
end
started
end
end
def test_start_the_car
engine_mock = mock("engine")
car = Car.new(engine_mock)
engine_mock.expects(:start).returns(true)
assert car.start
end
This is a trivial example I just made up of the top of my head, but
you can imagine if the code at Car#start did not call Engine#start,
and that resulted in the assertion failure. The test does not give as
much feedback as it could to help the developer determine what went
wrong.
In my opinion its quicker to debug if both errors / failures are
reported and you have a little more to go off on rather than having
to dig into the code / tests immediately to try and locate where the
point of failure occurred.
If you''d still rather the designed behavior, would it be possible to
propose an alternate patch that looks similar to the following
class VerboseFailuresTest < Test::Unit::TestCase
mocha_force_verifications = true
def test_foo
...
end
end
which has the default behavior of false (to accommodate the original
design)?
-Zach
On Jul 26, 2007, at 5:12 AM, James Mead wrote:
> Hi Zach,
>
> On 26/07/07, Zach Moazeni <zach.lists at gmail.com> wrote:
>>
>> I''m not sure if this is by design, but I''ve stumbled
across this a
>> few times trying to debug my own tests. If an assertion fails the
>> test, and a missing mock expectation was to blame, the test''s
>> failure / error messages don''t give enough info.
>
>
> This is by design. In my mind a verification failure is analogous
> to an
> assertion failure. Test::Unit does not report multiple assertion
> failures
> per test, but reports the first failure. Failing fast seems like
> the right
> thing to do. At that point all bets are off and there is no point in
> continuing.
>
> I wonder if the pain you are feeling is because you have too many
> assertions
> and/or expectations in each test. You might find the following article
> interesting...
>
> http://blog.jayfields.com/2007/06/testing-one-assertion-per-test.html
>
> Thanks for taking the time to put a patch together, but at this
> stage I''m
> not convinced it''s the way to go.
>
> --
> James.
> http://blog.floehopper.org
> _______________________________________________
> mocha-developer mailing list
> mocha-developer at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mocha-developer
James,
That''s an interesting article. I haven''t come across that, but
I do
something very similar when testing validations in ActiveRecord. The
problems I encounter more often are testing an object in isolation
where I''ll have a scenario similar to the following.
class Car
def intialize(engine)
@parts = []
@parts << engine\
end
def start
# run though a ton of part and return whether the car started
# typically there would be error handling, or you would just AND
all the parts.start
started = false
@parts.each do | part |
started = started && part.start
end
started
end
end
def test_start_the_car
engine_mock = mock("engine")
car = Car.new(engine_mock)
engine_mock.expects(:start).returns(true)
assert car.start
end
This is a trivial example I just made up of the top of my head, but
you can imagine if the code at Car#start did not call Engine#start,
and that resulted in the assertion failure. The test does not give as
much feedback as it could to help the developer determine what went
wrong.
In my opinion its quicker to debug if both errors / failures are
reported and you have a little more to go off on rather than having
to dig into the code / tests immediately to try and locate where the
point of failure occurred.
If you''d still rather the designed behavior, would it be possible to
propose an alternate patch that looks similar to the following
class VerboseFailuresTest < Test::Unit::TestCase
mocha_force_verifications = true
def test_foo
...
end
end
which has the default behavior of false (to accommodate the original
design)?
-Zach
On Jul 26, 2007, at 5:12 AM, James Mead wrote:
> Hi Zach,
>
> On 26/07/07, Zach Moazeni <zach.lists at gmail.com> wrote:
>>
>> I''m not sure if this is by design, but I''ve stumbled
across this a
>> few times trying to debug my own tests. If an assertion fails the
>> test, and a missing mock expectation was to blame, the test''s
>> failure / error messages don''t give enough info.
>
>
> This is by design. In my mind a verification failure is analogous
> to an
> assertion failure. Test::Unit does not report multiple assertion
> failures
> per test, but reports the first failure. Failing fast seems like
> the right
> thing to do. At that point all bets are off and there is no point in
> continuing.
>
> I wonder if the pain you are feeling is because you have too many
> assertions
> and/or expectations in each test. You might find the following article
> interesting...
>
> http://blog.jayfields.com/2007/06/testing-one-assertion-per-test.html
>
> Thanks for taking the time to put a patch together, but at this
> stage I''m
> not convinced it''s the way to go.
>
> --
> James.
> http://blog.floehopper.org
> _______________________________________________
> mocha-developer mailing list
> mocha-developer at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mocha-developer
If anyone else feels these pains, I''ve written a monkey patch (ripped off from Hardmock) that will ensure both the verification and the user''s teardown at http://www.simplechatter.com/2007/8/3/mocha-and- forcing-verification Mostly posting for archiving purposes. Thanks for the great mocking library, guys. -Zach On Jul 26, 2007, at 5:12 AM, James Mead wrote:> Hi Zach, > > On 26/07/07, Zach Moazeni <zach.lists at gmail.com> wrote: >> >> I''m not sure if this is by design, but I''ve stumbled across this a >> few times trying to debug my own tests. If an assertion fails the >> test, and a missing mock expectation was to blame, the test''s >> failure / error messages don''t give enough info. > > > This is by design. In my mind a verification failure is analogous > to an > assertion failure. Test::Unit does not report multiple assertion > failures > per test, but reports the first failure. Failing fast seems like > the right > thing to do. At that point all bets are off and there is no point in > continuing. > > I wonder if the pain you are feeling is because you have too many > assertions > and/or expectations in each test. You might find the following article > interesting... > > http://blog.jayfields.com/2007/06/testing-one-assertion-per-test.html > > Thanks for taking the time to put a patch together, but at this > stage I''m > not convinced it''s the way to go. > > -- > James. > http://blog.floehopper.org > _______________________________________________ > mocha-developer mailing list > mocha-developer at rubyforge.org > http://rubyforge.org/mailman/listinfo/mocha-developer