rspec users - Dec 2008 - Do people write specs for the code in their steps? How?

> The approach I like to take is to follow the guidance of the system
> (cucumber + rspec) every step of the way. Right now there are no
> passing steps, so I''d write the code for the 1st pending step, run
the
> features and find the first thing that is missing or failing.
This is one thing I don''t get.  I just started implementing steps, but
I feel like THAT code is all completely untested.  I don''t know if my
regular expression is correct, I don''t know if it does what I think it
should do.  What I really want to do is write something like this:

describe "steps for withdrawing money"

  describe "Given user has a balance"

    before :each
      cucumber "Given user has $50 dollars"
    end

    it "should match a particular step"

    it "should create an account"

    it "should set the account balance to $50"

  end

end


But is there such a "cucumber" method?  And how do you check that your
regular expressions are going to the right place?

Maybe the best thing to do is write your cucumber steps like this:

  user_has_a_balance = /user has $(.*) dollars/
  Given user_has_a_balance do |balance|
    ...
  end

And then in your spec you can do:

  user_has_a_balance.match("user has $20 dollars").should_not == nil

How do people write specs for their cucumber steps?  And if you don''t
write specs, how do you live with the uncertainty?

Erik

Hi Erik --

Interesting ideas.

We''re using cucumber for a medium-sized application (total steps run is
about 550), but I hadn''t really considered testing the step
definitions.

I have not yet noticed bugs originating from not testing step definitions.

I think the following issues drive this / me being OK with it:

--

Our step definitions are short, and we do not have that many.
When you think about it, there are not a whole lot of things you can do to a
webpage, compared to the things you can do in Ruby.

Webrat, email-spec and a few other steps cover most of the things that I
need to do / check on a webpage. Therefore, many of my step definitions end
up only involving  a) setup, b) a bunch of webrat steps all at once.

a) Setup: Since we use Machinist, there is very little setup to do. Most of
these steps just go something like User.make, and then do a login or
something.

b) Most other steps are just things like "When i fill in the new user
form"
with a bunch of fill_in steps.


W/r/t your example, I might write something like:

Given "I have a "(.*)" dollar balance" do |dough|
  User.make
  User.set_balance!(dough)
  # User.balance.should == dough
end

Where set_balance! is whatever method you would use.

Since the defn above is so simple, I wouldn''t worry about not testing
it.
You could  even add a .should into the step itself, as I have above, however
imo step defns above 3 lines (that are not all just fill_in) are candidates
for refactoring.

Your unit test for set_balance! would then cover that step, basically.

WDYT?

M







On Mon, Dec 29, 2008 at 8:17 PM, Erik Pukinskis <erikpukinskis at
gmail.com>wrote:
> > The approach I like to take is to follow the guidance of the system
> > (cucumber + rspec) every step of the way. Right now there are no
> > passing steps, so I''d write the code for the 1st pending
step, run the
> > features and find the first thing that is missing or failing.
>
> This is one thing I don''t get.  I just started implementing steps,
but
> I feel like THAT code is all completely untested.  I don''t know if
my
> regular expression is correct, I don''t know if it does what I
think it
> should do.  What I really want to do is write something like this:
>
> describe "steps for withdrawing money"
>
>  describe "Given user has a balance"
>
>    before :each
>      cucumber "Given user has $50 dollars"
>    end
>
>    it "should match a particular step"
>
>    it "should create an account"
>
>    it "should set the account balance to $50"
>
>  end
>
> end
>
>
> But is there such a "cucumber" method?  And how do you check that
your
> regular expressions are going to the right place?
>
> Maybe the best thing to do is write your cucumber steps like this:
>
>  user_has_a_balance = /user has $(.*) dollars/
>  Given user_has_a_balance do |balance|
>    ...
>  end
>
> And then in your spec you can do:
>
>  user_has_a_balance.match("user has $20 dollars").should_not ==
nil
>
> How do people write specs for their cucumber steps?  And if you
don''t
> write specs, how do you live with the uncertainty?
>
> Erik
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20081230/cfaab8a4/attachment-0001.html>

Actually,

I would probably even change it to:

Given a user
And the user has "50" dollars

And split it into 2 steps, and wait until it is really in need of drying up
(I have those two steps at the top of like 5 scenarios or something) to
combine it into 1.




On Tue, Dec 30, 2008 at 12:30 AM, Mischa Fierer <f.mischa at gmail.com>
wrote:
> Hi Erik --
>
> Interesting ideas.
>
> We''re using cucumber for a medium-sized application (total steps
run is
> about 550), but I hadn''t really considered testing the step
definitions.
>
> I have not yet noticed bugs originating from not testing step definitions.
>
> I think the following issues drive this / me being OK with it:
>
> --
>
> Our step definitions are short, and we do not have that many.
> When you think about it, there are not a whole lot of things you can do to
> a webpage, compared to the things you can do in Ruby.
>
> Webrat, email-spec and a few other steps cover most of the things that I
> need to do / check on a webpage. Therefore, many of my step definitions end
> up only involving  a) setup, b) a bunch of webrat steps all at once.
>
> a) Setup: Since we use Machinist, there is very little setup to do. Most of
> these steps just go something like User.make, and then do a login or
> something.
>
> b) Most other steps are just things like "When i fill in the new user
form"
> with a bunch of fill_in steps.
>
>
> W/r/t your example, I might write something like:
>
> Given "I have a "(.*)" dollar balance" do |dough|
>   User.make
>   User.set_balance!(dough)
>   # User.balance.should == dough
> end
>
> Where set_balance! is whatever method you would use.
>
> Since the defn above is so simple, I wouldn''t worry about not
testing it.
> You could  even add a .should into the step itself, as I have above,
however
> imo step defns above 3 lines (that are not all just fill_in) are candidates
> for refactoring.
>
> Your unit test for set_balance! would then cover that step, basically.
>
> WDYT?
>
> M
>
>
>
>
>
>
>
>
> On Mon, Dec 29, 2008 at 8:17 PM, Erik Pukinskis <erikpukinskis at
gmail.com>wrote:
>
>> > The approach I like to take is to follow the guidance of the
system
>> > (cucumber + rspec) every step of the way. Right now there are no
>> > passing steps, so I''d write the code for the 1st pending
step, run the
>> > features and find the first thing that is missing or failing.
>>
>> This is one thing I don''t get.  I just started implementing
steps, but
>> I feel like THAT code is all completely untested.  I don''t
know if my
>> regular expression is correct, I don''t know if it does what I
think it
>> should do.  What I really want to do is write something like this:
>>
>> describe "steps for withdrawing money"
>>
>>  describe "Given user has a balance"
>>
>>    before :each
>>      cucumber "Given user has $50 dollars"
>>    end
>>
>>    it "should match a particular step"
>>
>>    it "should create an account"
>>
>>    it "should set the account balance to $50"
>>
>>  end
>>
>> end
>>
>>
>> But is there such a "cucumber" method?  And how do you check
that your
>> regular expressions are going to the right place?
>>
>> Maybe the best thing to do is write your cucumber steps like this:
>>
>>  user_has_a_balance = /user has $(.*) dollars/
>>  Given user_has_a_balance do |balance|
>>    ...
>>  end
>>
>> And then in your spec you can do:
>>
>>  user_has_a_balance.match("user has $20 dollars").should_not
== nil
>>
>> How do people write specs for their cucumber steps?  And if you
don''t
>> write specs, how do you live with the uncertainty?
>>
>> Erik
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20081230/2e22a5d3/attachment.html>

On Mon, Dec 29, 2008 at 11:17 PM, Erik Pukinskis
<erikpukinskis at gmail.com> wrote:
>
> This is one thing I don''t get.  I just started implementing steps,
but
> I feel like THAT code is all completely untested.  I don''t know if
my
> regular expression is correct, I don''t know if it does what I
think it
> should do.
The tests are tested by your working application.  If you have
application code that clearly does the right thing in actual use, but
your tests for that specific use case fail, your tests are broken.  If
your application has bugs but your tests for that use case pass, your
tests are broken.  If your application works and your tests pass, then
practically speaking the tests are valid.  If your tests fail and your
application fails for that use case, the tests are valid.

Is that the same as formal proof?  No.  From a formal logic
perspective it''s not even in the ballpark.  And it''s not
necessarily
sufficient, either.  You could have entirely valid tests but still not
be testing everything you should.  There''s always some edge case you
didn''t think of.  TDD/BDD isn''t formal, it''s agile. 
It''s supposed to
make it easy to get to "good enough" and then improve incrementally
from there, raising the bar for "good enough" with every iteration.

As for writing tests for your tests...  Sure, you could potentially
have turtles all the way down.  But as I see it, I''m not being paid to
write tests.  Or when I''m doing projects for personal goals, the test
code isn''t the goal.  Eventually common sense has to kick in, and you
have to trust yourself somewhere.  If you can''t trust yourself, if
your regexes are so complicated that you can''t understand them and
determine their validity with a couple of passing/failing application
cases, maybe you need to simplify things with clearer or more granular
steps.



-- 
Have Fun,
   Steve Eley (sfeley at gmail.com)
   ESCAPE POD - The Science Fiction Podcast Magazine
   http://www.escapepod.org

On 30 Dec 2008, at 04:17, Erik Pukinskis wrote:
>> The approach I like to take is to follow the guidance of the system
>> (cucumber + rspec) every step of the way. Right now there are no
>> passing steps, so I''d write the code for the 1st pending step,
run
>> the
>> features and find the first thing that is missing or failing.
>
> This is one thing I don''t get.  I just started implementing steps,
but
> I feel like THAT code is all completely untested.  I don''t know if
my
> regular expression is correct, I don''t know if it does what I
think it
> should do.  What I really want to do is write something like this:
>
> describe "steps for withdrawing money"
>
>  describe "Given user has a balance"
>
>    before :each
>      cucumber "Given user has $50 dollars"
>    end
>
>    it "should match a particular step"
>
>    it "should create an account"
>
>    it "should set the account balance to $50"
>
>  end
>
> end
>
>
> But is there such a "cucumber" method?  And how do you check that
your
> regular expressions are going to the right place?
>
> Maybe the best thing to do is write your cucumber steps like this:
>
>  user_has_a_balance = /user has $(.*) dollars/
>  Given user_has_a_balance do |balance|
>    ...
>  end
>
> And then in your spec you can do:
>
>  user_has_a_balance.match("user has $20 dollars").should_not ==
nil
>
> How do people write specs for their cucumber steps?  And if you
don''t
> write specs, how do you live with the uncertainty?
>
> Erik
It seems like at least part of the uncertainty you''re talking about is
that the step matcher in your step definition ruby code won''t match  
all the steps defined in your features files that you intended it to.  
I have also worried about this, particularly after refactoring steps  
in a codebase with a large number of features.

A while back, I suggested a feature for cucumber, which would allow  
you to run it in a ''strict'' mode, where the build would fail
if any of
the steps in features were not matched to a ruby step definition. I''m  
not sure whether this ticket has been implemented or not - I had a  
feeling it was, but I can''t see anything in the help for 0.1.13, and  
the ticket hasn''t been updated. Plus it''s Christmas and my
brain is
pickled. Anyway, take a look at the ticket:

http://rspec.lighthouseapp.com/projects/16211-cucumber/tickets/52-unmatched-steps-should-less-tolerable-than-pending-steps#ticket-52-23

cheers,

Matt Wynne
http://blog.mattwynne.net
http://www.songkick.com

A year back or so i wrote a helper_method "step_eval(stepname, 
stepgroupname)" for the old storyrunnerframework that still works with 
the current rspec gem.
The method has just 9 loc and  think it would be even less with 
cucumber.

Here is an executable script:

http://pastie.org/363852

Matthias


-- 
Posted via http://www.ruby-forum.com/.