Hi everybody,
I have a general question about how many different cases have to be
covered in one Example.
Considering the famous Fizzbuzz program, you should test if your output
is either "Fizz", "buzz", "Fizzbuzz" or just the
number (between 1 and
100). Pretty simple. However, I often see solutions of testing multiples
of 3 only by looking at some relevant values like 6, 9 etc.
it "should return ''fizz'' when number is divisible by
3" do
@fizzbuzz.calculate(3).should == ''fizz''
@fizzbuzz.calculate(6).should == ''fizz''
@fizzbuzz.calculate(9).should == ''fizz''
end
So far, so good. But what if I claim that the test will fail in case of
12? It''s actually not covered and we are just assuming, that all
further
multiples will work as well, because these do. My idea was to check all
multiples right away in a loop.
it "should return Fizz if number is divisible only by 3" do
(1..100).each do |number|
@fizzbuzz.calculate(number).should == "Fizz" if number % 3 == 0
&&
number % 5 != 0
end
end
Given that the calculate method works fine, both ways will pass. But my
question is: Does my solution more than it actually needs to or is the
first one in fact a little too lazy?
Thanks for your thoughts.
Ulfmann
--
Posted via http://www.ruby-forum.com/.
On Fri, Mar 9, 2012 at 7:06 AM, U. M. <lists at ruby-forum.com> wrote:> Hi everybody, > > I have a general question about how many different cases have to be > covered in one Example. > > Considering the famous Fizzbuzz program, you should test if your output > is either "Fizz", "buzz", "Fizzbuzz" or just the number (between 1 and > 100). Pretty simple. However, I often see solutions of testing multiples > of 3 only by looking at some relevant values like 6, 9 etc. > > it "should return ''fizz'' when number is divisible by 3" do > ? ?@fizzbuzz.calculate(3).should == ''fizz'' > ? ?@fizzbuzz.calculate(6).should == ''fizz'' > ? ?@fizzbuzz.calculate(9).should == ''fizz'' > end > > So far, so good. But what if I claim that the test will fail in case of > 12? It''s actually not covered and we are just assuming, that all further > multiples will work as well, because these do. My idea was to check all > multiples right away in a loop. > > it "should return Fizz if number is divisible only by 3" do > ?(1..100).each do |number| > ? ?@fizzbuzz.calculate(number).should == "Fizz" if number % 3 == 0 && > ? ?number % 5 != 0 > ?end > end > > Given that the calculate method works fine, both ways will pass. But my > question is: Does my solution more than it actually needs to or is the > first one in fact a little too lazy? > > Thanks for your thoughts. > UlfmannTDD (and testing, in general) is, at its core, a risk-reduction technique. Because every example is code that you have to maintain, there is a tradeoff between completeness and maintainability. The first example is very simple and easy to understand at first glance. The second example is less so, which increases the long term cost of maintainability, thereby increasing risk (by increasing cost). Additionally, TDD is a gray-box sport: it''s not white-box because you''re operating at the surface of the object under test; it''s not black-box because you can see the implementation! For me, the questions are: * given the relative complexity of the problem and the implementation, do the examples provide me sufficient confidence that: * the implementation is sound? * the examples are likely to catch regressions? As soon as the answer to both of those questions is "yes", I move on. There is, of course, a ton of context that needs to be considered. Are you working on a team of developers that you trust to not to ridiculous shit? If so, then you don''t need to protect against idiocy. If not, you might need more examples (and you might consider finding a new job). HTH, David
On Fri, Mar 9, 2012 at 8:06 AM, U. M. <lists at ruby-forum.com> wrote:> Hi everybody,Hello!> > I have a general question about how many different cases have to be > covered in one Example. > > Considering the famous Fizzbuzz program, you should test if your output > is either "Fizz", "buzz", "Fizzbuzz" or just the number (between 1 and > 100). Pretty simple. However, I often see solutions of testing multiples > of 3 only by looking at some relevant values like 6, 9 etc. > > it "should return ''fizz'' when number is divisible by 3" do > ? ?@fizzbuzz.calculate(3).should == ''fizz'' > ? ?@fizzbuzz.calculate(6).should == ''fizz'' > ? ?@fizzbuzz.calculate(9).should == ''fizz'' > end > > So far, so good. But what if I claim that the test will fail in case of > 12?Then add an example for this and make that change. I might update the spec to look like this: describe "#calculate" do it "returns ''fizz'' when number is divisible by 3" context "(exceptions to the divisible by 3 rule)" do it "returns nil when number is 12" end end> It''s actually not covered and we are just assuming, that all further > multiples will work as well, because these do. My idea was to check all > multiples right away in a loop. > > it "should return Fizz if number is divisible only by 3" do > ?(1..100).each do |number| > ? ?@fizzbuzz.calculate(number).should == "Fizz" if number % 3 == 0 && > ? ?number % 5 != 0 > ?end > endThis is more complicated then it needs to be and suffers the same problem that you are trying to avoid. Here you are testing a larger data set, but numbers are infinite... so while 100 numbers is more than 3, it''s still a far cry from covering a large amount of the available set of numbers (which is infinite). Similar to David, I try to write examples that show the implementation working, but once I am confident that the examples I have are proving that implementation is working then I move on.> > Given that the calculate method works fine, both ways will pass. But my > question is: Does my solution more than it actually needs to or is the > first one in fact a little too lazy?What do you think? Zach
Dear Ulf, in your fizzbuzz example a kind of simple algorithm is used (% 3 and % 5) which can be easily tested in a loop. If this loop is clearly predictable, a test with only three levels like 3, 6, 9 is enough in my opinion. To extend the test from 1 to 100 in this case is a bit to much. Assuming there are exceptions in fizzbuzz like 60 should not be "Fizzbuzz" and 89 should not be "Fizz" your loop would fail. My point is not that your loop fails. My point is You have to create a new algorithm for your loop, taking care of the exceptions for 60 and 89. Keeping the thought of a loop including a special algorithm, one might be tempted to extend the loop from 100 to lets say 1000 or even more. I think loops an algorithms in tests can be used if they are aware of expectations and still valid, if they can be extended. Special scenarios like days in a month have no values > 31, so there is no reason, to extend a loop for days of a month to more than 31. Days in a month is commonly known thing. But other types of information can of course be mor exceptional, like 0 kelvin = -273.15 degrees Celsius and an algorithm testing a colder temperature would be some how out of scope. So when an algorithm is with out exceptions valid and easily predictable, testing of three values should be fine. Cheers, Andreas -- Posted via http://www.ruby-forum.com/.