rspec users - Mar 2009 - Cucumber step definitions vs. RSpec examples

Hi,

So, I''m confused. I''ve been trying to use Cucumber and RSpec,
but step
definitions and RSpec examples just seem to be overlapping. What should go
in either one of these?

For instance, let''s say I''m testing if a form exists on a
page. What goes in
Cucumber and what goes in RSpec?

What I did for now is put Webrat methods in the step definitions, like
fill_in, select, etc, without submitting since I''m not testing
processing
yet; and then put selector matchers in the RSpec examples.

But I have no idea if that''s right. How do you separate these and keep
them
from duplicating one another?

Thanks,
Brandon

On Sat, Mar 28, 2009 at 9:49 AM, Brandon Olivares
<programmer2188 at gmail.com> wrote:
>
> So, I''m confused. I''ve been trying to use Cucumber and
RSpec, but step
> definitions and RSpec examples just seem to be overlapping. What should go
> in either one of these?
This is a subjective judgment call that doesn''t have an absolute
''right'' answer.  Ask a dozen programmers how they test and
you''re
likely to get at least three dozen answers.

I believe the purists would say that the duplication is a feature.
The gist of this point of view is that RSpec''s main strength is on the
unit testing level: you want to test that model code works separate
from your controller code, and your controller code separate from your
view code.  Then if one fails, you know exactly what class went wrong
and what to fix.

Cucumber''s main strength is on the acceptance or integration level:
you want to make sure it all comes together correctly and creates the
right application behavior, which is very hard to prove with unit
testing alone.  But if you _only_ did Cucumber tests, your test to see
if a form exists on a page (to use your example) would be too broad.
If it failed, where do you look first?  Is the view broken?  The
controller code?  Do you even have a model for the view to operate on?
 Etc.

So you need to have bot levels.  You specify your overall behavior
with Cucumber, then you specify the isolated behaviors of the pieces
to make that work, and then you write your code.  That''s the
full-round-trip, complete by-the-RSpec-book BDD answer.

My own view...  Well, my view is that I''ve tried that and been driven
nuts by it.  Writing specs for controllers and views can take me many
times longer than writing the code.  Mocking out the models for
controllers is frustrating and fragile to me, and view specs are just
obvious and tedious.  I don''t have any _fun_ writing those specs.  And
if I''m not having fun, I find myself not motivated to program.

So I stopped writing those specs, and now I mostly just write model
specs and Cucumber features.  It usually doesn''t take that long for me
to look at a failing Cucumber test and figure out what went wrong --
from the failure text, a stacktrace, or sometimes just manually
running the action and eyeballing it.  Sometimes, if the controller
action is unusual, I will write a spec for it, but only if my own
instincts tell me it''s warranted.  (Or if something failed in an odd
way and I want to make sure that doesn''t happen again.)

That''s my current answer based on past experience.  Next month I might
have a different way of doing things.  I''m not going to make the case
here that I''m right and you should never write controller/view specs
for ordinary actions.  (If nothing else, doing it a bunch of times --
and getting some things wrong, then fixing them -- is a great way to
learn how controllers and views really work.)

I think the real answer is to try it a few different ways, decide what
works within your comfort level *and* leads to good code, and let
yourself experiment.  And then, if you develop an effective working
pattern down the road that you think makes good general sense, share
it with the community as one more way of doing things.   >8->

-- 
Have Fun,
   Steve Eley (sfeley at gmail.com)
   ESCAPE POD - The Science Fiction Podcast Magazine
   http://www.escapepod.org

On Sat, Mar 28, 2009 at 1:49 PM, Brandon Olivares
<programmer2188 at gmail.com> wrote:
> Hi,
>
> So, I''m confused. I''ve been trying to use Cucumber and
RSpec, but step
> definitions and RSpec examples just seem to be overlapping. What should go
> in either one of these?
>
> For instance, let''s say I''m testing if a form exists on a
page. What goes in
> Cucumber and what goes in RSpec?
>
> What I did for now is put Webrat methods in the step definitions, like
> fill_in, select, etc, without submitting since I''m not testing
processing
> yet; and then put selector matchers in the RSpec examples.
>
> But I have no idea if that''s right. How do you separate these and
keep them
> from duplicating one another?
>
> Thanks,
> Brandon
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>
So far I''ve not found a situation where I want to just test a
form''s
existence at the acceptance level - I always just submit the form.
Then I might write a couple RSpec examples to verify the
presence/absence of that form in certain situations.

Occasionally there is duplication between your acceptance and unit
tests - I think that''s okay.  Acceptance tests verify the behavior at
the application level, and unit tests generally do a better job of
defect localization.  I try to minimize that duplication though, and
frequently I will sacrifice the defect localization and only test that
behavior through Cucumber.  Then I go back and write the unit test the
first time I experience a problem and it takes me longer to track down
than it should have :)

Pat

> -----Original Message-----
> From: rspec-users-bounces at rubyforge.org [mailto:rspec-users-
> bounces at rubyforge.org] On Behalf Of Stephen Eley
> Sent: Sunday, March 29, 2009 3:47 PM
> To: rspec-users
> Subject: Re: [rspec-users] Cucumber step definitions vs. RSpec examples
> 
> This is a subjective judgment call that doesn''t have an absolute
> ''right'' answer.  Ask a dozen programmers how they test
and you''re
> likely to get at least three dozen answers.
> 
> I believe the purists would say that the duplication is a feature.
> The gist of this point of view is that RSpec''s main strength is on
the
> unit testing level: you want to test that model code works separate
> from your controller code, and your controller code separate from your
> view code.  Then if one fails, you know exactly what class went wrong
> and what to fix.
> 
> Cucumber''s main strength is on the acceptance or integration
level:
> you want to make sure it all comes together correctly and creates the
> right application behavior, which is very hard to prove with unit
> testing alone.  But if you _only_ did Cucumber tests, your test to see
> if a form exists on a page (to use your example) would be too broad.
> If it failed, where do you look first?  Is the view broken?  The
> controller code?  Do you even have a model for the view to operate on?
>  Etc.
> 
Thank you very much. That is much more understandable. This made me realize
I forgot the controller testing for the feature I''m talking about.

> My own view...  Well, my view is that I''ve tried that and been
driven
> nuts by it.  Writing specs for controllers and views can take me many
> times longer than writing the code.  Mocking out the models for
> controllers is frustrating and fragile to me, and view specs are just
> obvious and tedious.  I don''t have any _fun_ writing those specs. 
And
> if I''m not having fun, I find myself not motivated to program.
> 
> So I stopped writing those specs, and now I mostly just write model
> specs and Cucumber features.  It usually doesn''t take that long
for me
> to look at a failing Cucumber test and figure out what went wrong --
> from the failure text, a stacktrace, or sometimes just manually
> running the action and eyeballing it.  Sometimes, if the controller
> action is unusual, I will write a spec for it, but only if my own
> instincts tell me it''s warranted.  (Or if something failed in an
odd
> way and I want to make sure that doesn''t happen again.)
> 
> That''s my current answer based on past experience.  Next month I
might
> have a different way of doing things.  I''m not going to make the
case
> here that I''m right and you should never write controller/view
specs
> for ordinary actions.  (If nothing else, doing it a bunch of times --
> and getting some things wrong, then fixing them -- is a great way to
> learn how controllers and views really work.)
> 
Yeah, that''s what I''m worried about. I wrote probably 5
examples just to
test the existence of one field and ensure it had the options I wanted.
I''m
worried about it taking way too long. But I do like the security provided by
extensive unit tests.

Thanks,
Brandon

On Sat, Mar 28, 2009 at 9:49 AM, Brandon Olivares
<programmer2188 at gmail.com> wrote:
> Hi,
>
> So, I''m confused. I''ve been trying to use Cucumber and
RSpec, but step
> definitions and RSpec examples just seem to be overlapping. What should go
> in either one of these?
Cucumber is intended to drive out the behaviour of the application and
RSpec is used to drive out the design and behaviour for the components
and objects that make up that application. Cucumber and RSpec are
bound to overlap. They''re providing you insights at different levels
about how the application is expected to be working.
>
> For instance, let''s say I''m testing if a form exists on a
page. What goes in
> Cucumber and what goes in RSpec?
Does the scenario successfully drive the implementation of the form
and the page its on? If so, I would probably stick with the scenario.
However, if it doesn''t then I drop down to a view spec and spec
whatever else the page needs that the scenario doesn''t cover.

Scenarios usually don''t communicate well the expectations against a
page for showing and hiding information, etc based on permissions or
roles. RSpec works great for this. It provides a wonderful way to
simply and concisely make each of those expectations clear rather than
some strange side-effect of a scenario which is focused on something
else.
>
> What I did for now is put Webrat methods in the step definitions, like
> fill_in, select, etc, without submitting since I''m not testing
processing
> yet; and then put selector matchers in the RSpec examples.
>
> But I have no idea if that''s right. How do you separate these and
keep them
> from duplicating one another?
At some level duplication will happen, especially in a web app since
the way you verify the application is working is by
POSTing/GETting/etc and verifying output on the page itself AND it''s
also what you have to do when you creating the individual views and
the controllers that make up those pages and respond to those
requests.

You may find that a scenario drives you to make a view with a form and
a controller with a #create action to process it. This may be adequate
and you may be able to confidently move forward without writing a view
spec or a controller spec. I know I have found this it the case many
times.

On the flip side, when something additional or conditional occurs that
is usually a good time to drive that with an example in a spec.

I use my scenarios to drive my app from the outside, and then specs to
drive the implementation for each of the underlying pieces. The more
I''ve practiced the easier and more natural it has become to know when
to do or not to do something. Also, the only way to not have something
feel like it''s taking too long is to know your tools.  These things
won''t just come because you want it to. It takes practice.
Remember to practice.

http://www.vimeo.com/3756344

-- 
Zach Dennis
http://www.continuousthinking.com
http://www.mutuallyhuman.com

On Sun, Mar 29, 2009 at 3:47 PM, Stephen Eley <sfeley at gmail.com>
wrote:
> On Sat, Mar 28, 2009 at 9:49 AM, Brandon Olivares
> <programmer2188 at gmail.com> wrote:
>>
>> So, I''m confused. I''ve been trying to use Cucumber
and RSpec, but step
>> definitions and RSpec examples just seem to be overlapping. What should
go
>> in either one of these?
>
> This is a subjective judgment call that doesn''t have an absolute
> ''right'' answer.  Ask a dozen programmers how they test
and you''re
> likely to get at least three dozen answers.
>
> I believe the purists would say that the duplication is a feature.
> The gist of this point of view is that RSpec''s main strength is on
the
> unit testing level: you want to test that model code works separate
> from your controller code, and your controller code separate from your
> view code.  Then if one fails, you know exactly what class went wrong
> and what to fix.
>
> Cucumber''s main strength is on the acceptance or integration
level:
> you want to make sure it all comes together correctly and creates the
> right application behavior, which is very hard to prove with unit
> testing alone.  But if you _only_ did Cucumber tests, your test to see
> if a form exists on a page (to use your example) would be too broad.
> If it failed, where do you look first?  Is the view broken?  The
> controller code?  Do you even have a model for the view to operate on?
>  Etc.
>
> So you need to have bot levels.  You specify your overall behavior
> with Cucumber, then you specify the isolated behaviors of the pieces
> to make that work, and then you write your code.  That''s the
> full-round-trip, complete by-the-RSpec-book BDD answer.
>
> My own view...  Well, my view is that I''ve tried that and been
driven
> nuts by it.  Writing specs for controllers and views can take me many
> times longer than writing the code.  Mocking out the models for
> controllers is frustrating and fragile to me, and view specs are just
> obvious and tedious.  I don''t have any _fun_ writing those specs. 
And
> if I''m not having fun, I find myself not motivated to program.
>
> So I stopped writing those specs, and now I mostly just write model
> specs and Cucumber features.  It usually doesn''t take that long
for me
> to look at a failing Cucumber test and figure out what went wrong --
> from the failure text, a stacktrace, or sometimes just manually
> running the action and eyeballing it.  Sometimes, if the controller
> action is unusual, I will write a spec for it, but only if my own
> instincts tell me it''s warranted.  (Or if something failed in an
odd
> way and I want to make sure that doesn''t happen again.)
>
> That''s my current answer based on past experience.  Next month I
might
> have a different way of doing things.  I''m not going to make the
case
> here that I''m right and you should never write controller/view
specs
> for ordinary actions.  (If nothing else, doing it a bunch of times --
> and getting some things wrong, then fixing them -- is a great way to
> learn how controllers and views really work.)
>
> I think the real answer is to try it a few different ways, decide what
> works within your comfort level *and* leads to good code, and let
> yourself experiment.  And then, if you develop an effective working
> pattern down the road that you think makes good general sense, share
> it with the community as one more way of doing things.   >8->
>
I like a lot of what Steve is saying. To summarize and communicate in
my own way...

It is good to know how and when to do something. Like writing view
specs and controller specs. If you don''t know how then you''ll
never
know when. Sometimes learning to take small steps seems ridiculous,
but it''s by being able to work in small steps that you can be
confident about making larger ones.

Here are three questions I ask myself when I get frustrated:

 1 - Am I doing it wrong?
 2 - Should I be doing this in the first place?
 3 - Am I experiencing a skills deficiency?

Usually I think it''s #1 and then I quickly jump to #2 when I
don''t get
it resolved in 5 minutes of googling. And then I find out a day later
it was because of #3 and I needed to go home and practice a little
more so I better understood what I was doing, why I was doing it, and
how I could leverage the tools to make it easier.

We all learn to make good decisions by first making bad ones, and
practice lets us do that w/o putting giant turds in an application''s
production codebase.

-- 
Zach Dennis
http://www.continuousthinking.com
http://www.mutuallyhuman.com

> -----Original Message-----
> From: rspec-users-bounces at rubyforge.org [mailto:rspec-users-
> bounces at rubyforge.org] On Behalf Of Zach Dennis
> Sent: Sunday, March 29, 2009 11:10 PM
> To: rspec-users
> Subject: Re: [rspec-users] Cucumber step definitions vs. RSpec examples
> > I like a lot of what Steve is saying. To summarize and communicate in
> my own way...
> 
> It is good to know how and when to do something. Like writing view
> specs and controller specs. If you don''t know how then
you''ll never
> know when. Sometimes learning to take small steps seems ridiculous,
> but it''s by being able to work in small steps that you can be
> confident about making larger ones.
> 
> Here are three questions I ask myself when I get frustrated:
> 
>  1 - Am I doing it wrong?
>  2 - Should I be doing this in the first place?
>  3 - Am I experiencing a skills deficiency?
> 
> Usually I think it''s #1 and then I quickly jump to #2 when I
don''t get
> it resolved in 5 minutes of googling. And then I find out a day later
> it was because of #3 and I needed to go home and practice a little
> more so I better understood what I was doing, why I was doing it, and
> how I could leverage the tools to make it easier.
> 
> We all learn to make good decisions by first making bad ones, and
> practice lets us do that w/o putting giant turds in an
application''s
> production codebase.
> 
Thanks for that explanation.

For me, it''s either TDD or BDD, honestly. I''ve experienced the
assurance
having tests for nearly every line of code affords me, and don''t want
to
forgo that. But I like the semantics of BDD a lot better, after reading the
RSpec book.

Thanks,
Brandon

On Mon, Mar 30, 2009 at 7:08 AM, Brandon Olivares
<programmer2188 at gmail.com> wrote:
>
> For me, it''s either TDD or BDD, honestly. I''ve
experienced the assurance
> having tests for nearly every line of code affords me, and don''t
want to
> forgo that. But I like the semantics of BDD a lot better, after reading the
> RSpec book.
Interesting.  I don''t think what you''re saying is invalid, but
I do
think it''s an _unusual_ perspective to consider TDD and BDD as an
either/or.  Depending on whom you ask, BDD is most often seen either
as an evolution/elaboration on TDD, or else the differences are viewed
as purely semantic.

Either way, though, by practicing BDD you''re generally doing
everything you''d do in TDD.  You''re still specifying
success-vs.-failure conditions before you write the code, and then
writing only the code necessary to achieve success.  Whether you call
those conditions ''behaviors'' or ''tests''
doesn''t change that
fundamental pattern.


-- 
Have Fun,
   Steve Eley (sfeley at gmail.com)
   ESCAPE POD - The Science Fiction Podcast Magazine
   http://www.escapepod.org

> -----Original Message-----
> From: rspec-users-bounces at rubyforge.org [mailto:rspec-users-
> bounces at rubyforge.org] On Behalf Of Stephen Eley
> Sent: Monday, March 30, 2009 10:15 AM
> To: rspec-users
> Subject: Re: [rspec-users] Cucumber step definitions vs. RSpec examples
> 
> Interesting.  I don''t think what you''re saying is
invalid, but I do
> think it''s an _unusual_ perspective to consider TDD and BDD as an
> either/or.  Depending on whom you ask, BDD is most often seen either
> as an evolution/elaboration on TDD, or else the differences are viewed
> as purely semantic.
> 
> Either way, though, by practicing BDD you''re generally doing
> everything you''d do in TDD.  You''re still specifying
> success-vs.-failure conditions before you write the code, and then
> writing only the code necessary to achieve success.  Whether you call
> those conditions ''behaviors'' or ''tests''
doesn''t change that
> fundamental pattern.
> 
> 
Yeah I guess I didn''t express that quite right, and I see what you
mean.
Either way, I mean I have to do some kind of test driven development, in
whatever form it is.

Brandon