[rspec-users] Sufficiently tested?

David Chelimsky
2012-03-09T14:15:59Z

On Fri, Mar 9, 2012 at 7:06 AM, U. M. <lists at ruby-forum.com>
wrote:
> Hi everybody,
>
> I have a general question about how many different cases have to be
> covered in one Example.
>
> Considering the famous Fizzbuzz program, you should test if your output
> is either "Fizz", "buzz", "Fizzbuzz" or just
the number (between 1 and
> 100). Pretty simple. However, I often see solutions of testing multiples
> of 3 only by looking at some relevant values like 6, 9 etc.
>
> it "should return ''fizz'' when number is divisible by
3" do
> ? ?@fizzbuzz.calculate(3).should == ''fizz''
> ? ?@fizzbuzz.calculate(6).should == ''fizz''
> ? ?@fizzbuzz.calculate(9).should == ''fizz''
> end
>
> So far, so good. But what if I claim that the test will fail in case of
> 12? It''s actually not covered and we are just assuming, that all
further
> multiples will work as well, because these do. My idea was to check all
> multiples right away in a loop.
>
> it "should return Fizz if number is divisible only by 3" do
> ?(1..100).each do |number|
> ? ?@fizzbuzz.calculate(number).should == "Fizz" if number % 3 ==
0 &&
> ? ?number % 5 != 0
> ?end
> end
>
> Given that the calculate method works fine, both ways will pass. But my
> question is: Does my solution more than it actually needs to or is the
> first one in fact a little too lazy?
>
> Thanks for your thoughts.
> Ulfmann

TDD (and testing, in general) is, at its core, a risk-reduction
technique. Because every example is code that you have to maintain,
there is a tradeoff between completeness and maintainability. The
first example is very simple and easy to understand at first glance.
The second example is less so, which increases the long term cost of
maintainability, thereby increasing risk (by increasing cost).

Additionally, TDD is a gray-box sport: it''s not white-box because
you''re operating at the surface of the object under test; it''s
not
black-box because you can see the implementation! For me, the
questions are:

* given the relative complexity of the problem and the implementation,
do the examples provide me sufficient confidence that:
  * the implementation is sound?
  * the examples are likely to catch regressions?

As soon as the answer to both of those questions is "yes", I move on.

There is, of course, a ton of context that needs to be considered. Are
you working on a team of developers that you trust to not to
ridiculous shit? If so, then you don''t need to protect against idiocy.
If not, you might need more examples (and you might consider finding a
new job).

HTH,
David