rspec users - Jun 2007 - Autotest bug with rerunning passing tests?

Am I correct in remembering that autotest (with Test::Unit) would  
rerun all the tests if a subset of the tests passed?  If so,  
shouldn''t this behaviour also be present in rspec''s autotest
library?

Scott

On 6/23/07, Scott Taylor <mailing_lists at railsnewbie.com>
wrote:
>
> Am I correct in remembering that autotest (with Test::Unit) would
> rerun all the tests if a subset of the tests passed?  If so,
> shouldn''t this behaviour also be present in rspec''s
autotest library?
Yes, but it depends on where you make the change. Let''s say you get a
failure and you go look at your example and realize there''s something
wrong and change that. Then autotest will only rerun that example
(well, that whole file).

If you make the change to subject code, then it should re-run
everything once you get the failing example to pass.

If you''re experiencing something different, please follow up with a
bit of detail about what files you''re changing and what examples are
getting run.

Also, if you want to take a look at one source of any potential
problems, look at @test_mappings in
vendor/plugins/rspec_on_rails/lib/autotest/rails_rspec.rb.

Cheers,
David
>
> Scott
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

On Jun 23, 2007, at 4:14 PM, David Chelimsky wrote:
> On 6/23/07, Scott Taylor <mailing_lists at railsnewbie.com> wrote:
>>
>> Am I correct in remembering that autotest (with Test::Unit) would
>> rerun all the tests if a subset of the tests passed?  If so,
>> shouldn''t this behaviour also be present in rspec''s
autotest library?
>
> Yes, but it depends on where you make the change. Let''s say you
get a
> failure and you go look at your example and realize there''s
something
> wrong and change that. Then autotest will only rerun that example
> (well, that whole file).
>
> If you make the change to subject code, then it should re-run
> everything once you get the failing example to pass.
>
> If you''re experiencing something different, please follow up with
a
> bit of detail about what files you''re changing and what examples
are
> getting run.
Well actually I was in the refactoring step.  Here is the behaviour:

1. All of the specs are passing.
2. I refactor.
3. The subset of specs that apply runs, and passed
4. Autotest now waits for the next file change or Interrupt.  Nothing  
else happens (although, now, all of the specs should run)

Scott

P.S. You mentioned that the whole spec file runs with autotest, not  
just the ones that are failing.  Is this a bug in the code, or is it  
presumed that we should be using the --failures option in spec.opts?


>
> Also, if you want to take a look at one source of any potential
> problems, look at @test_mappings in
> vendor/plugins/rspec_on_rails/lib/autotest/rails_rspec.rb.
>
> Cheers,
> David
>>
>> Scott
>>
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
>>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users

On 6/23/07, Scott Taylor <mailing_lists at railsnewbie.com>
wrote:
>
> On Jun 23, 2007, at 4:14 PM, David Chelimsky wrote:
>
> > On 6/23/07, Scott Taylor <mailing_lists at railsnewbie.com>
wrote:
> >>
> >> Am I correct in remembering that autotest (with Test::Unit) would
> >> rerun all the tests if a subset of the tests passed?  If so,
> >> shouldn''t this behaviour also be present in
rspec''s autotest library?
> >
> > Yes, but it depends on where you make the change. Let''s say
you get a
> > failure and you go look at your example and realize there''s
something
> > wrong and change that. Then autotest will only rerun that example
> > (well, that whole file).
> >
> > If you make the change to subject code, then it should re-run
> > everything once you get the failing example to pass.
> >
> > If you''re experiencing something different, please follow up
with a
> > bit of detail about what files you''re changing and what
examples are
> > getting run.
>
> Well actually I was in the refactoring step.  Here is the behaviour:
>
> 1. All of the specs are passing.
> 2. I refactor.
> 3. The subset of specs that apply runs, and passed
> 4. Autotest now waits for the next file change or Interrupt.  Nothing
> else happens (although, now, all of the specs should run)
That''s the way autotest works AFAIK. It''s not going to rerun
everything until you get a failing example to pass. Otherwise it just
runs the files that map to those that you change.
>
> Scott
>
> P.S. You mentioned that the whole spec file runs with autotest, not
> just the ones that are failing.  Is this a bug in the code, or is it
> presumed that we should be using the --failures option in spec.opts?
>
>
>
> >
> > Also, if you want to take a look at one source of any potential
> > problems, look at @test_mappings in
> > vendor/plugins/rspec_on_rails/lib/autotest/rails_rspec.rb.
> >
> > Cheers,
> > David
> >>
> >> Scott
> >>
> >> _______________________________________________
> >> rspec-users mailing list
> >> rspec-users at rubyforge.org
> >> http://rubyforge.org/mailman/listinfo/rspec-users
> >>
> > _______________________________________________
> > rspec-users mailing list
> > rspec-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/rspec-users
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

On 6/23/07, Scott Taylor <mailing_lists at railsnewbie.com>
wrote:
> P.S. You mentioned that the whole spec file runs with autotest, not
> just the ones that are failing.  Is this a bug in the code, or is it
> presumed that we should be using the --failures option in spec.opts?
Neither. It''s just the way we set up the mappings. My experience is
that the performance savings of targeting just one example as opposed
to the whole file is not that noticeable but the feedback from running
all the examples in a given file is much more useful than the feedback
you get from running just one example until it passes.

If you want to get it to run one at a time, it''ll take a bit of work
on rails_rspec.rb. Feel free to contribute a patch if you like, but if
you do, please set it up so that you can flip a switch of some sort to
get it to work either way.

Cheers,
David

On 6/23/07, David Chelimsky <dchelimsky at gmail.com>
wrote:
> Neither. It''s just the way we set up the mappings. My experience
is
> that the performance savings of targeting just one example as opposed
> to the whole file is not that noticeable but the feedback from running
> all the examples in a given file is much more useful than the feedback
> you get from running just one example until it passes.
>
I''m a little off-trunk right now, so maybe it''s not an issue
anymore -
but when I have a spec that has say, 4 failing examples, autotest puts
that file into the next run 4 times. It''s a little frustrating when
the number is much larger than four. I often get the spec completely
passing, and then sit and wait for many more identical passing test
runs before autotest finishes the last command-line batch, which in
the worst case can look like this (actually the real command line was
at least 20 times longer):

script/spec -O spec/spec.opts spec/models/track_collision_spec.rb;
script/spec -O spec/spec.opts
spec/controllers/playlists_controller_spec.rb; script/spec -O
spec/spec.opts  spec/controllers/stars_controller_spec.rb; script/spec
-O spec/spec.opts  spec/controllers/stars_controller_spec.rb;
script/spec -O spec/spec.opts
spec/controllers/stars_controller_spec.rb; script/spec -O
spec/spec.opts  spec/controllers/stars_controller_spec.rb; script/spec
-O spec/spec.opts  spec/controllers/stars_controller_spec.rb;
script/spec -O spec/spec.opts
spec/controllers/stars_controller_spec.rb; script/spec -O
spec/spec.opts  spec/controllers/stars_controller_spec.rb; script/spec
-O spec/spec.opts  spec/controllers/stars_controller_spec.rb;
script/spec -O spec/spec.opts
spec/controllers/stars_controller_spec.rb; script/spec -O
spec/spec.opts  spec/controllers/stars_controller_spec.rb; script/spec
-O spec/spec.opts  spec/controllers/stars_controller_spec.rb;
script/spec -O spec/spec.opts
spec/controllers/stars_controller_spec.rb; script/spec -O
spec/spec.opts  spec/controllers/stars_controller_spec.rb; script/spec
-O spec/spec.opts  spec/controllers/stars_controller_spec.rb;
script/spec -O spec/spec.opts
spec/controllers/stars_controller_spec.rb; script/spec -O
spec/spec.opts  spec/controllers/grabs_controller_spec.rb

If nothing else it''s bad for my battery-life. :) The autotest - rspec
interface seems like black-magic to me right now, so if someone is
gonna jump in there and clean things up, I''d be really happy if this
one goes away too.

-- 
Chris Anderson
http://jchris.mfdz.com

On 6/23/07, Chris Anderson <jchris at mfdz.com>
wrote:
> On 6/23/07, David Chelimsky <dchelimsky at gmail.com> wrote:
> > Neither. It''s just the way we set up the mappings. My
experience is
> > that the performance savings of targeting just one example as opposed
> > to the whole file is not that noticeable but the feedback from running
> > all the examples in a given file is much more useful than the feedback
> > you get from running just one example until it passes.
> >
>
> I''m a little off-trunk right now, so maybe it''s not an
issue anymore -
> but when I have a spec that has say, 4 failing examples, autotest puts
> that file into the next run 4 times. It''s a little frustrating
when
> the number is much larger than four. I often get the spec completely
> passing, and then sit and wait for many more identical passing test
> runs before autotest finishes the last command-line batch, which in
> the worst case can look like this (actually the real command line was
> at least 20 times longer):
>
> script/spec -O spec/spec.opts spec/models/track_collision_spec.rb;
> script/spec -O spec/spec.opts
> spec/controllers/playlists_controller_spec.rb; script/spec -O
> spec/spec.opts  spec/controllers/stars_controller_spec.rb; script/spec
> -O spec/spec.opts  spec/controllers/stars_controller_spec.rb;
> script/spec -O spec/spec.opts
> spec/controllers/stars_controller_spec.rb; script/spec -O
> spec/spec.opts  spec/controllers/stars_controller_spec.rb; script/spec
> -O spec/spec.opts  spec/controllers/stars_controller_spec.rb;
> script/spec -O spec/spec.opts
> spec/controllers/stars_controller_spec.rb; script/spec -O
> spec/spec.opts  spec/controllers/stars_controller_spec.rb; script/spec
> -O spec/spec.opts  spec/controllers/stars_controller_spec.rb;
> script/spec -O spec/spec.opts
> spec/controllers/stars_controller_spec.rb; script/spec -O
> spec/spec.opts  spec/controllers/stars_controller_spec.rb; script/spec
> -O spec/spec.opts  spec/controllers/stars_controller_spec.rb;
> script/spec -O spec/spec.opts
> spec/controllers/stars_controller_spec.rb; script/spec -O
> spec/spec.opts  spec/controllers/stars_controller_spec.rb; script/spec
> -O spec/spec.opts  spec/controllers/stars_controller_spec.rb;
> script/spec -O spec/spec.opts
> spec/controllers/stars_controller_spec.rb; script/spec -O
> spec/spec.opts  spec/controllers/grabs_controller_spec.rb
>
> If nothing else it''s bad for my battery-life. :) The autotest -
rspec
> interface seems like black-magic to me right now, so if someone is
> gonna jump in there and clean things up, I''d be really happy if
this
> one goes away too.
There have been a few bugs fixed related to this that have been
released already - are you on 1.0.5?

Also, give this a read - it might be the source of your problem:
http://blog.davidchelimsky.net/articles/2007/05/29/rspec-and-autotest
>
> --
> Chris Anderson
> http://jchris.mfdz.com
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>