rspec users - Jun 2007 - Something like rcov but more explicit?

Hey list,

Does anything exist that can show me which lines are explicitly tested? RCov
is great, but it doesn''t really give a sense of _actual_ coverage,
indeed,
there is no guarantee that functions executed were in fact called as the
result of a line being tested.

I know nothing of the rspec internals, but would a simple list of executed
lines be a relatively simple feature to add?

Cheers
Ian
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/rspec-users/attachments/20070618/5567e433/attachment.html

On 6/18/07, Ian Leitch <port001 at gmail.com>
wrote:
> Hey list,
>
> Does anything exist that can show me which lines are explicitly tested?
RCov
> is great, but it doesn''t really give a sense of _actual_ coverage,
indeed,
> there is no guarantee that functions executed were in fact called as the
> result of a line being tested.
>
Why do you care whether a line was executed directly (spec->code) or
indirectly (spec->code->code->code)?
> I know nothing of the rspec internals, but would a simple list of executed
> lines be a relatively simple feature to add?
>
That would be hard I think. Feel free to investigate it. I would
rather recommend you use RSpec''s --heckle switch if you want to know
how good your examples are. Coverage is overrated.

See http://seattlerb.rubyforge.org/heckle/ for more info.

Aslak
> Cheers
> Ian
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

On 18/06/07, aslak hellesoy <aslak.hellesoy at gmail.com>
wrote:
>
> On 6/18/07, Ian Leitch <port001 at gmail.com> wrote:
> > Hey list,
> >
> > Does anything exist that can show me which lines are explicitly
tested?
> RCov
> > is great, but it doesn''t really give a sense of _actual_
coverage,
> indeed,
> > there is no guarantee that functions executed were in fact called as
the
> > result of a line being tested.
> >
>
> Why do you care whether a line was executed directly (spec->code) or
> indirectly (spec->code->code->code)?


Just because a spec executed a function coincidentally and it generates some
kind of error (most likely semantic), there is no guarantee that the code
that made the call will also trip up. Therefore, rcov doesn''t really
tell me
to what extent my code base is explicitly tested.

> I know nothing of the rspec internals, but would a simple list of executed
> > lines be a relatively simple feature to add?
> >
>
> That would be hard I think. Feel free to investigate it. I would
> rather recommend you use RSpec''s --heckle switch if you want to
know
> how good your examples are. Coverage is overrated.
>
> See http://seattlerb.rubyforge.org/heckle/ for more info.

Thanks, I''ll give Heckle a look.


Aslak
>
> > Cheers
> > Ian
> >
> > _______________________________________________
> > rspec-users mailing list
> > rspec-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/rspec-users
> >
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/rspec-users/attachments/20070620/f0857b3f/attachment.html

On 6/20/07, Ian Leitch <port001 at gmail.com>
wrote:
> On 18/06/07, aslak hellesoy <aslak.hellesoy at gmail.com> wrote:
> > On 6/18/07, Ian Leitch <port001 at gmail.com> wrote:
> > > Hey list,
> > >
> > > Does anything exist that can show me which lines are explicitly
tested?
> RCov
> > > is great, but it doesn''t really give a sense of _actual_
coverage,
> indeed,
> > > there is no guarantee that functions executed were in fact called
as the
> > > result of a line being tested.
> > >
> >
> > Why do you care whether a line was executed directly (spec->code)
or
> > indirectly (spec->code->code->code)?
>
>
> Just because a spec executed a function coincidentally and it generates
some
> kind of error (most likely semantic), there is no guarantee that the code
> that made the call will also trip up.
Are you saying that an ''explicit'' call to that method would
give you
confidence about how the code that made the call will behave? In the
scenario you''re describing this ''intermediate'' code
isn''t even
involved.

Maybe I''m misunderstanding what you''re trying to do. Some
sample code
would help.

I''m also sceptical about having a goal along the lines of "all
methods
in our code must be invoked explicitly by specs". It''s a very low
level (too low level IMO) way of approaching the overall behaviour of
your app.

Aslak
> Therefore, rcov doesn''t really tell me
> to what extent my code base is explicitly tested.
>
> > > I know nothing of the rspec internals, but would a simple list of
> executed
> > > lines be a relatively simple feature to add?
> > >
> >
> > That would be hard I think. Feel free to investigate it. I would
> > rather recommend you use RSpec''s --heckle switch if you want
to know
> > how good your examples are. Coverage is overrated.
> >
> > See http://seattlerb.rubyforge.org/heckle/ for more info.
>
> Thanks, I''ll give Heckle a look.
>
> > Aslak
> >
> > > Cheers
> > > Ian
> > >
> > > _______________________________________________
> > > rspec-users mailing list
> > > rspec-users at rubyforge.org
> > > http://rubyforge.org/mailman/listinfo/rspec-users
> > >
> > _______________________________________________
> > rspec-users mailing list
> > rspec-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/rspec-users
> >
>
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

>
> I''m also sceptical about having a goal along the lines of
"all methods in
> our code must be invoked explicitly by specs". It''s a very
low level (too
> low level IMO) way of approaching the overall behaviour of your app.
>
> Aslak

I would also add that it violates some principals of encapsulation.  If you
expose an object to perform an action, then an rspec test should interact
with that object to test those actions.  And only those actions exposed by
the object.  Unit tests should be the vehicle for testing any methods relied
upon by the "outer object" to accomplish the action.

I am very new to rspec (and rails), but my interpretation of BDD is that it
should honor principles of encapsulation, and unit testing should be used
for low-level testing.  If this is not the spirit of rspec, I would love to
know how other people approach it - then I can fix my misperceptions.

my two cents
Scott Sehlhorst
http://tynerblain.com/blog
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/rspec-users/attachments/20070620/3f2eb2e4/attachment.html

I appreciate that testing every single line off code isn''t the approach
of
BDD, and indeed that isn''t my goal, although at the same time
I''d like to be
able to "see" with coverage reports just to what extent the code base
is
explicitly tested.

Say I had ViewA that uses HelperA and ViewB that uses HelperB, ViewA and
HelperA both have specs, as does ViewB, however HelperB does not. With an
rcov coverage report this isn''t visible, what I''m after is
something that
can tell me "HelperB was executed during the running of ViewB, but its
behavior was not explicitly specified."

It''d like to look into seeing if this is possible with rspec,
I''ll have to
try convince my boss to allow me to devote some time to it.

Cheers
Ian

On 20/06/07, Scott Sehlhorst <scott at tynerblain.com>
wrote:
>
> I''m also sceptical about having a goal along the lines of
"all methods in
> > our code must be invoked explicitly by specs". It''s a
very low level (too
> > low level IMO) way of approaching the overall behaviour of your app.
> >
> > Aslak
>
>
> I would also add that it violates some principals of encapsulation.  If
> you expose an object to perform an action, then an rspec test should
> interact with that object to test those actions.  And only those actions
> exposed by the object.  Unit tests should be the vehicle for testing any
> methods relied upon by the "outer object" to accomplish the
action.
>
> I am very new to rspec (and rails), but my interpretation of BDD is that
> it should honor principles of encapsulation, and unit testing should be
used
> for low-level testing.  If this is not the spirit of rspec, I would love to
> know how other people approach it - then I can fix my misperceptions.
>
> my two cents
> Scott Sehlhorst
> http://tynerblain.com/blog
>
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/rspec-users/attachments/20070620/c3f6e8a0/attachment.html

On 6/20/07, Ian Leitch <port001 at gmail.com>
wrote:
> I appreciate that testing every single line off code isn''t the
approach of
> BDD, and indeed that isn''t my goal, although at the same time
I''d like to be
> able to "see" with coverage reports just to what extent the code
base is
> explicitly tested.
>
> Say I had ViewA that uses HelperA and ViewB that uses HelperB, ViewA and
> HelperA both have specs, as does ViewB, however HelperB does not. With an
> rcov coverage report this isn''t visible, what I''m after
is something that
> can tell me "HelperB was executed during the running of ViewB, but its
> behavior was not explicitly specified."
>
If you want to verify how well tested your code is you should look
into branch coverage (RCov first, then Heckle).

It''s irrelevant where a method invocation originates from.

Aslak
> It''d like to look into seeing if this is possible with rspec,
I''ll have to
> try convince my boss to allow me to devote some time to it.
>
> Cheers
> Ian
>
>
> On 20/06/07, Scott Sehlhorst <scott at tynerblain.com> wrote:
> >
> >
> > > I''m also sceptical about having a goal along the lines
of "all methods
> in our code must be invoked explicitly by specs". It''s a very
low level (too
> low level IMO) way of approaching the overall behaviour of your app.
> > >
> > > Aslak
> >
> >
> > I would also add that it violates some principals of encapsulation. 
If
> you expose an object to perform an action, then an rspec test should
> interact with that object to test those actions.  And only those actions
> exposed by the object.  Unit tests should be the vehicle for testing any
> methods relied upon by the "outer object" to accomplish the
action.
> >
> > I am very new to rspec (and rails), but my interpretation of BDD is
that
> it should honor principles of encapsulation, and unit testing should be
used
> for low-level testing.  If this is not the spirit of rspec, I would love to
> know how other people approach it - then I can fix my misperceptions.
> >
> > my two cents
> > Scott Sehlhorst
> > http://tynerblain.com/blog
> >
> >
> > _______________________________________________
> > rspec-users mailing list
> > rspec-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/rspec-users
> >
>
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>