rspec users - Jul 2012 - A recently observed anti pattern: commented out tests

Hey,

in a bunch of the rescues i''ve recently done, I see a pretty big anti
pattern: tests don''t work, and so rather than making them work, the
dev team just comments them out till ''later''.

Does anyone think it''d be useful/interesting to get a flag for rspec
which would compare lines vs lines-commented, and if the percentage
was higher than xx, it''d issue some kind of warning?

Best,
james

I must admit that I''m guilting of commenting out tests. I often do this
when I know that the code works, and I just upgraded some gem, and now one
of my tests no longer passes. I look at the test and realize that the
effort it would take to get the test to work outweighs the value that the
test provides me. (This is a general frustration for me that testing takes
more time than coding.)

I just did a quick scan of my main rails project (one that''s been
around
for four years) and has ~700 tests. I found two kinds of commented out
tests. About 12 of these commented tests look real and I should spend some
time and see why they are there. About 10 of these commented tests look
like I was brainstorming pending tests, I implemented them another way, but
never bothered cleaning them up.

In the spirit of clean code, I''ll take a moment to clean house. Thanks
James!

Todd Sedano
Director of Software Engineering
Carnegie Mellon University
Silicon Valley Campus
Developing Software Leaders (TM)
T: 650-335-2812



On Mon, Jul 23, 2012 at 9:19 AM, James Cox <james at imaj.es> wrote:
> Hey,
>
> in a bunch of the rescues i''ve recently done, I see a pretty big
anti
> pattern: tests don''t work, and so rather than making them work,
the
> dev team just comments them out till ''later''.
>
> Does anyone think it''d be useful/interesting to get a flag for
rspec
> which would compare lines vs lines-commented, and if the percentage
> was higher than xx, it''d issue some kind of warning?
>
> Best,
> james
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20120723/0fed7368/attachment.html>

On Mon, Jul 23, 2012 at 11:19 AM, James Cox <james at imaj.es>
wrote:
> Hey,
>
> in a bunch of the rescues i''ve recently done, I see a pretty big
anti
> pattern: tests don''t work, and so rather than making them work,
the
> dev team just comments them out till ''later''.
>
> Does anyone think it''d be useful/interesting to get a flag for
rspec
> which would compare lines vs lines-commented, and if the percentage
> was higher than xx, it''d issue some kind of warning?
The pending feature is designed to help with this problem by allowing
you to disable an example while still keeping it visible.

If we were to do what you propose, we''d need to offer an opt-out
and/or the ability to configure the percentage. Consider a suite that
uses a lot of comments to annotate the specs. The problem with making
it configurable is that the folks who''s priorities lead them to
comment out examples instead of fixing them will likely just disable
this feature.

I''d say, let''s encourage people to use
''pending'' correctly. WDYT?

Cheers,
David

I haven''t posted in a while, but I want to say that as someone who
spends a
significant portion of his time teaching (T/B)DD I am totally in love with
pending specs. There are analogous concepts in nearly every xUnit/xSpec,
but pending is by far the best. Kudos.
 On Jul 23, 2012 9:57 PM, "David Chelimsky" <dchelimsky at
gmail.com> wrote:
> On Mon, Jul 23, 2012 at 11:19 AM, James Cox <james at imaj.es> wrote:
> > Hey,
> >
> > in a bunch of the rescues i''ve recently done, I see a pretty
big anti
> > pattern: tests don''t work, and so rather than making them
work, the
> > dev team just comments them out till ''later''.
> >
> > Does anyone think it''d be useful/interesting to get a flag
for rspec
> > which would compare lines vs lines-commented, and if the percentage
> > was higher than xx, it''d issue some kind of warning?
>
> The pending feature is designed to help with this problem by allowing
> you to disable an example while still keeping it visible.
>
> If we were to do what you propose, we''d need to offer an opt-out
> and/or the ability to configure the percentage. Consider a suite that
> uses a lot of comments to annotate the specs. The problem with making
> it configurable is that the folks who''s priorities lead them to
> comment out examples instead of fixing them will likely just disable
> this feature.
>
> I''d say, let''s encourage people to use
''pending'' correctly. WDYT?
>
> Cheers,
> David
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20120723/a2e86e29/attachment.html>

Yeah, I love pending too. but it doesn''t help me get a sense of the
state of a suite before I start. now it''s part of my practice to go in
and find out how much is commented out.

David,

three concerns with pending as an option:

a. it won''t help the people who think it''s ok to comment out
whole
tests. If you make that choice it''s not a good thing (?). I
don''t
think there''s enough evangelism in the world to change them.

b. how do you distinguish between a pending and a broken-but-fixing
test? one means, "i''ve got no coverage here, and i
haven''t thought
about it'' whereas the other says, ''i used to have coverage,
and i need
to fix it''. I know it''s semantics, but that''s
important here: i need
to know where no effort for testing has been made vs where testing
existed (which may imply some domain knowledge which was at one point
true).

c. if you see # it ''should ?'' or similar, that''s a
commented test, not
a test comment. This metric is always going to be loose? but it may
give an indication, a sniff test. some kind of idea of what the state
of a test is. It''s the same as running rake stats - you and I know
it''s a bullshit metric (it can''t possibly tell me how good the
tests
are), but it tells me at least if any effort to test has happened.
Then, I run coverage and figure out how exercised the code is..
somewhere along that line, it''d be good to know if there used to be
tests but they are commented out.


an anecdote? i experienced this recently with a project, and a
significant majority of the tests were just commented out. They used
to work, and a lot of it modeled the domain reasonably well - but
either due to a breaking gem upgrade or a refactor or something - the
original dev just didn''t move/fix the test. So, on the face of it, the
test scenario looked horrible, but in the end, for the key components,
fixing the tests wasn''t that painful. Regardless, I got a pretty fast
sense of how much water he was treading at the time (or, how much he
was under it :/)

so yes, pending is ok, but a second keyword "broken" might be nicer,
which would act the same but output different info.

-james

On Mon, Jul 23, 2012 at 10:58 PM, Adam Sroka <adam.sroka at gmail.com>
wrote:
> I haven''t posted in a while, but I want to say that as someone who
spends a
> significant portion of his time teaching (T/B)DD I am totally in love with
> pending specs. There are analogous concepts in nearly every xUnit/xSpec,
but
> pending is by far the best. Kudos.
>
> On Jul 23, 2012 9:57 PM, "David Chelimsky" <dchelimsky at
gmail.com> wrote:
>>
>> On Mon, Jul 23, 2012 at 11:19 AM, James Cox <james at imaj.es>
wrote:
>> > Hey,
>> >
>> > in a bunch of the rescues i''ve recently done, I see a
pretty big anti
>> > pattern: tests don''t work, and so rather than making them
work, the
>> > dev team just comments them out till ''later''.
>> >
>> > Does anyone think it''d be useful/interesting to get a
flag for rspec
>> > which would compare lines vs lines-commented, and if the
percentage
>> > was higher than xx, it''d issue some kind of warning?
>>
>> The pending feature is designed to help with this problem by allowing
>> you to disable an example while still keeping it visible.
>>
>> If we were to do what you propose, we''d need to offer an
opt-out
>> and/or the ability to configure the percentage. Consider a suite that
>> uses a lot of comments to annotate the specs. The problem with making
>> it configurable is that the folks who''s priorities lead them
to
>> comment out examples instead of fixing them will likely just disable
>> this feature.
>>
>> I''d say, let''s encourage people to use
''pending'' correctly. WDYT?
>>
>> Cheers,
>> David
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
>
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users


-- 
James Cox,
Consultant, Raconteur, Photographer, Entrepreneur
t: +1 347 433 0567  e: james at imaj.es w: http://imaj.es/
talk: http://twitter.com/imajes photos: http://500px.com/imajes

On Tue, Jul 24, 2012 at 1:55 PM, James Cox <james at imaj.es> wrote:
> so yes, pending is ok, but a second keyword "broken" might be
nicer,
> which would act the same but output different info.--
>
There is a block form of pending.  It actually executes the contents of the
block, but outputs as a pending test -- unless the test passes, in which
case it fails with a differing message:

  it "is a broken test that I need to fix sometime" do
    pending("broke on nonesuch upgrade") {
      domain.do_failing_thing
    }
  end

So,  that shows up in test output just like every other pending note.
Whenever someone gets around to fixing whatever failed, the contents of the
block start passing -- and the test will fail.  The failure is a signal to
remove the pending block from around the test.

This may not be as helpful as you like, if it''s drowning in a sea of
true
"pending" tests ... but I feel that leaving unimplemented, pending
tests in
the suite for a long time is a Bad Thing -- precisely because you loose
useful information like this.

:wq
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20120724/2b4af5e3/attachment.html>

AFAIK, there is no framework or tool that can prevent people from doing
stupid things.

I actually only use pending for one thing: in the morning it reminds me
where I was heading the prior evening.
 On Jul 24, 2012 1:57 PM, "James Cox" <james at imaj.es> wrote:
> Yeah, I love pending too. but it doesn''t help me get a sense of
the
> state of a suite before I start. now it''s part of my practice to
go in
> and find out how much is commented out.
>
> David,
>
> three concerns with pending as an option:
>
> a. it won''t help the people who think it''s ok to comment
out whole
> tests. If you make that choice it''s not a good thing (?). I
don''t
> think there''s enough evangelism in the world to change them.
>
> b. how do you distinguish between a pending and a broken-but-fixing
> test? one means, "i''ve got no coverage here, and i
haven''t thought
> about it'' whereas the other says, ''i used to have
coverage, and i need
> to fix it''. I know it''s semantics, but that''s
important here: i need
> to know where no effort for testing has been made vs where testing
> existed (which may imply some domain knowledge which was at one point
> true).
>
> c. if you see # it ''should ?'' or similar, that''s
a commented test, not
> a test comment. This metric is always going to be loose? but it may
> give an indication, a sniff test. some kind of idea of what the state
> of a test is. It''s the same as running rake stats - you and I know
> it''s a bullshit metric (it can''t possibly tell me how
good the tests
> are), but it tells me at least if any effort to test has happened.
> Then, I run coverage and figure out how exercised the code is..
> somewhere along that line, it''d be good to know if there used to
be
> tests but they are commented out.
>
>
> an anecdote? i experienced this recently with a project, and a
> significant majority of the tests were just commented out. They used
> to work, and a lot of it modeled the domain reasonably well - but
> either due to a breaking gem upgrade or a refactor or something - the
> original dev just didn''t move/fix the test. So, on the face of it,
the
> test scenario looked horrible, but in the end, for the key components,
> fixing the tests wasn''t that painful. Regardless, I got a pretty
fast
> sense of how much water he was treading at the time (or, how much he
> was under it :/)
>
> so yes, pending is ok, but a second keyword "broken" might be
nicer,
> which would act the same but output different info.
>
> -james
>
> On Mon, Jul 23, 2012 at 10:58 PM, Adam Sroka <adam.sroka at
gmail.com> wrote:
> > I haven''t posted in a while, but I want to say that as
someone who
> spends a
> > significant portion of his time teaching (T/B)DD I am totally in love
> with
> > pending specs. There are analogous concepts in nearly every
xUnit/xSpec,
> but
> > pending is by far the best. Kudos.
> >
> > On Jul 23, 2012 9:57 PM, "David Chelimsky" <dchelimsky at
gmail.com> wrote:
> >>
> >> On Mon, Jul 23, 2012 at 11:19 AM, James Cox <james at
imaj.es> wrote:
> >> > Hey,
> >> >
> >> > in a bunch of the rescues i''ve recently done, I see
a pretty big anti
> >> > pattern: tests don''t work, and so rather than making
them work, the
> >> > dev team just comments them out till
''later''.
> >> >
> >> > Does anyone think it''d be useful/interesting to get
a flag for rspec
> >> > which would compare lines vs lines-commented, and if the
percentage
> >> > was higher than xx, it''d issue some kind of warning?
> >>
> >> The pending feature is designed to help with this problem by
allowing
> >> you to disable an example while still keeping it visible.
> >>
> >> If we were to do what you propose, we''d need to offer an
opt-out
> >> and/or the ability to configure the percentage. Consider a suite
that
> >> uses a lot of comments to annotate the specs. The problem with
making
> >> it configurable is that the folks who''s priorities lead
them to
> >> comment out examples instead of fixing them will likely just
disable
> >> this feature.
> >>
> >> I''d say, let''s encourage people to use
''pending'' correctly. WDYT?
> >>
> >> Cheers,
> >> David
> >> _______________________________________________
> >> rspec-users mailing list
> >> rspec-users at rubyforge.org
> >> http://rubyforge.org/mailman/listinfo/rspec-users
> >
> >
> > _______________________________________________
> > rspec-users mailing list
> > rspec-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/rspec-users
>
>
>
> --
> James Cox,
> Consultant, Raconteur, Photographer, Entrepreneur
> t: +1 347 433 0567  e: james at imaj.es w: http://imaj.es/
> talk: http://twitter.com/imajes photos: http://500px.com/imajes
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20120724/cf4e2fc2/attachment.html>

I''d suggest adding a coverage ratchet to your build. It''s the
most
effective (if occasionally annoying) tool when dealing with such
situations. Some assumptions:

* You need a CI server and everyone''s using CCMenu/Buildnotify so the
team knows as soon as the build breaks
* You don''t have a brittle build that''s red a non-trivial
portion of
the time (not unusual when you inherit a test-less codebase and start
adding Cukes to introduce sanity)
* Your team is now used to the "drop everything if the build is red
and fix it" rule

I should really extract the ratchet I''ve hand-rolled for RSpec on
Goldberg into something reusable.

Best,
Sidu.
http://c42.in
http://sidu.in


On 25 July 2012 09:01, Adam Sroka <adam.sroka at gmail.com>
wrote:
> AFAIK, there is no framework or tool that can prevent people from doing
> stupid things.
>
> I actually only use pending for one thing: in the morning it reminds me
> where I was heading the prior evening.
>
> On Jul 24, 2012 1:57 PM, "James Cox" <james at imaj.es>
wrote:
>>
>> Yeah, I love pending too. but it doesn''t help me get a sense
of the
>> state of a suite before I start. now it''s part of my practice
to go in
>> and find out how much is commented out.
>>
>> David,
>>
>> three concerns with pending as an option:
>>
>> a. it won''t help the people who think it''s ok to
comment out whole
>> tests. If you make that choice it''s not a good thing (?). I
don''t
>> think there''s enough evangelism in the world to change them.
>>
>> b. how do you distinguish between a pending and a broken-but-fixing
>> test? one means, "i''ve got no coverage here, and i
haven''t thought
>> about it'' whereas the other says, ''i used to have
coverage, and i need
>> to fix it''. I know it''s semantics, but
that''s important here: i need
>> to know where no effort for testing has been made vs where testing
>> existed (which may imply some domain knowledge which was at one point
>> true).
>>
>> c. if you see # it ''should ?'' or similar,
that''s a commented test, not
>> a test comment. This metric is always going to be loose? but it may
>> give an indication, a sniff test. some kind of idea of what the state
>> of a test is. It''s the same as running rake stats - you and I
know
>> it''s a bullshit metric (it can''t possibly tell me how
good the tests
>> are), but it tells me at least if any effort to test has happened.
>> Then, I run coverage and figure out how exercised the code is..
>> somewhere along that line, it''d be good to know if there used
to be
>> tests but they are commented out.
>>
>>
>> an anecdote? i experienced this recently with a project, and a
>> significant majority of the tests were just commented out. They used
>> to work, and a lot of it modeled the domain reasonably well - but
>> either due to a breaking gem upgrade or a refactor or something - the
>> original dev just didn''t move/fix the test. So, on the face of
it, the
>> test scenario looked horrible, but in the end, for the key components,
>> fixing the tests wasn''t that painful. Regardless, I got a
pretty fast
>> sense of how much water he was treading at the time (or, how much he
>> was under it :/)
>>
>> so yes, pending is ok, but a second keyword "broken" might be
nicer,
>> which would act the same but output different info.
>>
>> -james
>>
>> On Mon, Jul 23, 2012 at 10:58 PM, Adam Sroka <adam.sroka at
gmail.com> wrote:
>> > I haven''t posted in a while, but I want to say that as
someone who
>> > spends a
>> > significant portion of his time teaching (T/B)DD I am totally in
love
>> > with
>> > pending specs. There are analogous concepts in nearly every
xUnit/xSpec,
>> > but
>> > pending is by far the best. Kudos.
>> >
>> > On Jul 23, 2012 9:57 PM, "David Chelimsky"
<dchelimsky at gmail.com> wrote:
>> >>
>> >> On Mon, Jul 23, 2012 at 11:19 AM, James Cox <james at
imaj.es> wrote:
>> >> > Hey,
>> >> >
>> >> > in a bunch of the rescues i''ve recently done, I
see a pretty big anti
>> >> > pattern: tests don''t work, and so rather than
making them work, the
>> >> > dev team just comments them out till
''later''.
>> >> >
>> >> > Does anyone think it''d be useful/interesting to
get a flag for rspec
>> >> > which would compare lines vs lines-commented, and if the
percentage
>> >> > was higher than xx, it''d issue some kind of
warning?
>> >>
>> >> The pending feature is designed to help with this problem by
allowing
>> >> you to disable an example while still keeping it visible.
>> >>
>> >> If we were to do what you propose, we''d need to offer
an opt-out
>> >> and/or the ability to configure the percentage. Consider a
suite that
>> >> uses a lot of comments to annotate the specs. The problem with
making
>> >> it configurable is that the folks who''s priorities
lead them to
>> >> comment out examples instead of fixing them will likely just
disable
>> >> this feature.
>> >>
>> >> I''d say, let''s encourage people to use
''pending'' correctly. WDYT?
>> >>
>> >> Cheers,
>> >> David
>> >> _______________________________________________
>> >> rspec-users mailing list
>> >> rspec-users at rubyforge.org
>> >> http://rubyforge.org/mailman/listinfo/rspec-users
>> >
>> >
>> > _______________________________________________
>> > rspec-users mailing list
>> > rspec-users at rubyforge.org
>> > http://rubyforge.org/mailman/listinfo/rspec-users
>>
>>
>>
>> --
>> James Cox,
>> Consultant, Raconteur, Photographer, Entrepreneur
>> t: +1 347 433 0567  e: james at imaj.es w: http://imaj.es/
>> talk: http://twitter.com/imajes photos: http://500px.com/imajes
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
>
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users

On Tue, Jul 24, 2012 at 1:55 PM, James Cox <james at imaj.es>
wrote:
> Yeah, I love pending too. but it doesn''t help me get a sense of
the
> state of a suite before I start. now it''s part of my practice to
go in
> and find out how much is commented out.
>
> David,
>
> three concerns with pending as an option:
>
> a. it won''t help the people who think it''s ok to comment
out whole
> tests. If you make that choice it''s not a good thing (?). I
don''t
> think there''s enough evangelism in the world to change them.
Those same people will work around whatever we make available to help them.
> b. how do you distinguish between a pending and a broken-but-fixing
> test? one means, "i''ve got no coverage here, and i
haven''t thought
> about it'' whereas the other says, ''i used to have
coverage, and i need
> to fix it''. I know it''s semantics, but that''s
important here: i need
> to know where no effort for testing has been made vs where testing
> existed (which may imply some domain knowledge which was at one point
> true).
What Chris Flipse said.
> c. if you see # it ''should ?'' or similar, that''s
a commented test, not
> a test comment. This metric is always going to be loose? but it may
> give an indication, a sniff test. some kind of idea of what the state
> of a test is. It''s the same as running rake stats - you and I know
> it''s a bullshit metric (it can''t possibly tell me how
good the tests
> are), but it tells me at least if any effort to test has happened.
> Then, I run coverage and figure out how exercised the code is..
> somewhere along that line, it''d be good to know if there used to
be
> tests but they are commented out.
"it" is an alias for example, specify, and pending (among others), so
we''d have to parse for all of those.
> an anecdote? i experienced this recently with a project, and a
> significant majority of the tests were just commented out. They used
> to work, and a lot of it modeled the domain reasonably well - but
> either due to a breaking gem upgrade or a refactor or something - the
> original dev just didn''t move/fix the test. So, on the face of it,
the
> test scenario looked horrible, but in the end, for the key components,
> fixing the tests wasn''t that painful. Regardless, I got a pretty
fast
> sense of how much water he was treading at the time (or, how much he
> was under it :/)
>
> so yes, pending is ok, but a second keyword "broken" might be
nicer,
> which would act the same but output different info.
You can actually do that!

RSpec.configuration do |c|
  c.alias_example_to :broken, :pending => "Broken"
end

See http://rubydoc.info/gems/rspec-core/RSpec/Core/ExampleGroup.alias_example_to
and
https://github.com/rspec/rspec-core/blob/master/lib/rspec/core/example_group.rb#L69-104

HTH,
David
>
> -james
>
> On Mon, Jul 23, 2012 at 10:58 PM, Adam Sroka <adam.sroka at
gmail.com> wrote:
>> I haven''t posted in a while, but I want to say that as someone
who spends a
>> significant portion of his time teaching (T/B)DD I am totally in love
with
>> pending specs. There are analogous concepts in nearly every
xUnit/xSpec, but
>> pending is by far the best. Kudos.
>>
>> On Jul 23, 2012 9:57 PM, "David Chelimsky" <dchelimsky at
gmail.com> wrote:
>>>
>>> On Mon, Jul 23, 2012 at 11:19 AM, James Cox <james at
imaj.es> wrote:
>>> > Hey,
>>> >
>>> > in a bunch of the rescues i''ve recently done, I see a
pretty big anti
>>> > pattern: tests don''t work, and so rather than making
them work, the
>>> > dev team just comments them out till
''later''.
>>> >
>>> > Does anyone think it''d be useful/interesting to get a
flag for rspec
>>> > which would compare lines vs lines-commented, and if the
percentage
>>> > was higher than xx, it''d issue some kind of warning?
>>>
>>> The pending feature is designed to help with this problem by
allowing
>>> you to disable an example while still keeping it visible.
>>>
>>> If we were to do what you propose, we''d need to offer an
opt-out
>>> and/or the ability to configure the percentage. Consider a suite
that
>>> uses a lot of comments to annotate the specs. The problem with
making
>>> it configurable is that the folks who''s priorities lead
them to
>>> comment out examples instead of fixing them will likely just
disable
>>> this feature.
>>>
>>> I''d say, let''s encourage people to use
''pending'' correctly. WDYT?
>>>
>>> Cheers,
>>> David
>>> _______________________________________________
>>> rspec-users mailing list
>>> rspec-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/rspec-users
>>
>>
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
>
>
>
> --
> James Cox,
> Consultant, Raconteur, Photographer, Entrepreneur
> t: +1 347 433 0567  e: james at imaj.es w: http://imaj.es/
> talk: http://twitter.com/imajes photos: http://500px.com/imajes
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users

>
> > so yes, pending is ok, but a second keyword "broken" might
be nicer,
> > which would act the same but output different info.
>
> You can actually do that!
>
> RSpec.configuration do |c|
>   c.alias_example_to :broken, :pending => "Broken"
> end
>
> See
>
http://rubydoc.info/gems/rspec-core/RSpec/Core/ExampleGroup.alias_example_to
> and
>
https://github.com/rspec/rspec-core/blob/master/lib/rspec/core/example_group.rb#L69-104
>
>
David, i''m going to give that a go - see how it shakes out in
real-world
use. It might work to visually make it distinguishable enough to process.
:) thanks.

-james
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20120726/c7288f81/attachment.html>

Chris,

On Tue, Jul 24, 2012 at 4:54 PM, Chris Flipse <cflipse at gmail.com>
wrote:
>
>
> On Tue, Jul 24, 2012 at 1:55 PM, James Cox <james at imaj.es> wrote:
>
>>  so yes, pending is ok, but a second keyword "broken" might
be nicer,
>> which would act the same but output different info.--
>>
>
> There is a block form of pending.  It actually executes the contents of
> the block, but outputs as a pending test -- unless the test passes, in
> which case it fails with a differing message:
>
>   it "is a broken test that I need to fix sometime" do
>     pending("broke on nonesuch upgrade") {
>       domain.do_failing_thing
>     }
>   end
>
> So,  that shows up in test output just like every other pending note.
> Whenever someone gets around to fixing whatever failed, the contents of the
> block start passing -- and the test will fail.  The failure is a signal to
> remove the pending block from around the test.
>
> This may not be as helpful as you like, if it''s drowning in a sea
of true
> "pending" tests ... but I feel that leaving unimplemented,
pending tests in
> the suite for a long time is a Bad Thing -- precisely because you loose
> useful information like this.
>
One of the examples i''ve found for commented out tests is when the code
plainly doesn''t work. Wouldn''t it also fail (and output a
backtrace) in
this instance, therefore thoroughly confusing whoever is observing the test
output? I do like the idea of this though: pend it until it passes, and
then fail because you should check it, but ? i dont'' imagine that
it''d
really work that way.

-james
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20120726/377b9a90/attachment.html>

On Thu, Jul 26, 2012 at 4:06 PM, James Cox <james at imaj.es>
wrote:
>> > so yes, pending is ok, but a second keyword "broken"
might be nicer,
>> > which would act the same but output different info.
>>
>> You can actually do that!
>>
>> RSpec.configuration do |c|
>>   c.alias_example_to :broken, :pending => "Broken"
>> end
>>
>> See
>>
http://rubydoc.info/gems/rspec-core/RSpec/Core/ExampleGroup.alias_example_to
>> and
>>
https://github.com/rspec/rspec-core/blob/master/lib/rspec/core/example_group.rb#L69-104
>>
>
> David, i''m going to give that a go - see how it shakes out in
real-world
> use. It might work to visually make it distinguishable enough to process.
:)
> thanks.
Cool. Please report back if you learn anything interesting.

Cheers,
David

On Thu, Jul 26, 2012 at 4:08 PM, James Cox <james at imaj.es>
wrote:
> Chris,
>
> On Tue, Jul 24, 2012 at 4:54 PM, Chris Flipse <cflipse at gmail.com>
wrote:
>>
>>
>>
>> On Tue, Jul 24, 2012 at 1:55 PM, James Cox <james at imaj.es>
wrote:
>>>
>>> so yes, pending is ok, but a second keyword "broken"
might be nicer,
>>> which would act the same but output different info.--
>>
>>
>> There is a block form of pending.  It actually executes the contents of
>> the block, but outputs as a pending test -- unless the test passes, in
which
>> case it fails with a differing message:
>>
>>   it "is a broken test that I need to fix sometime" do
>>     pending("broke on nonesuch upgrade") {
>>       domain.do_failing_thing
>>     }
>>   end
>>
>> So,  that shows up in test output just like every other pending note.
>> Whenever someone gets around to fixing whatever failed, the contents of
the
>> block start passing -- and the test will fail.  The failure is a signal
to
>> remove the pending block from around the test.
>>
>> This may not be as helpful as you like, if it''s drowning in a
sea of true
>> "pending" tests ... but I feel that leaving unimplemented,
pending tests in
>> the suite for a long time is a Bad Thing -- precisely because you loose
>> useful information like this.
>
>
> One of the examples i''ve found for commented out tests is when the
code
> plainly doesn''t work. Wouldn''t it also fail (and output a
backtrace) in this
> instance, therefore thoroughly confusing whoever is observing the test
> output? I do like the idea of this though: pend it until it passes, and
then
> fail because you should check it, but ? i dont'' imagine that
it''d really
> work that way.
No - this form passes if there is either an error or a failed
expectation (which is also an error, but internal to rspec). It will
only fail when the code works without event. This means that if the
code doesn''t actually do anything it all, it will fail, but that never
happens in practice in my experience.

On Mon, Jul 23, 2012 at 10:54 PM, David Chelimsky <dchelimsky at
gmail.com> wrote:
> I''d say, let''s encourage people to use
''pending'' correctly. WDYT?
+1.
-- 
J. B. (Joe) Rainsberger :: http://www.jbrains.ca ::
http://blog.thecodewhisperer.com
Author, JUnit Recipes
Free Your Mind to Do Great Work :: http://www.freeyourmind-dogreatwork.com