rspec users - May 2009 - autospec runs failing specs forever in endless loop

I wanted to start using autospec on my little (non-Rails) project. I''m
on Windows XP and have used it some time ago successfully (on the same
PC), although I''ve updated many gems in the meantime (RSpec for
example among others).

In short, everything is fine when all specs pass, but if some fails,
then autospec won''t stop running failing specs again even if no files
are changed. I expect it to run specs again only if some file gets
modified. It''s kinda irritating when it doesn''t stop and is by
no
means helping any development :/

Also, when I fix the failing spec then it will run it only once and
then run whole suit once and stops as expected.

Any ideas what might cause this distracting behaviour?

BR,
Jarmo

On Sun, May 31, 2009 at 3:59 PM, Jarmo Pertman <Jarmo.P at gmail.com>
wrote:
> I wanted to start using autospec on my little (non-Rails) project.
I''m
> on Windows XP and have used it some time ago successfully (on the same
> PC), although I''ve updated many gems in the meantime (RSpec for
> example among others).
>
> In short, everything is fine when all specs pass, but if some fails,
> then autospec won''t stop running failing specs again even if no
files
> are changed. I expect it to run specs again only if some file gets
> modified. It''s kinda irritating when it doesn''t stop and
is by no
> means helping any development :/
>
> Also, when I fix the failing spec then it will run it only once and
> then run whole suit once and stops as expected.
>
> Any ideas what might cause this distracting behaviour?
Do you have "--format failing_examples" in spec/spec.opts? If so,
rspec saves a file with a list of the failing examples in it, which
would trigger autotest to run again unless that file is explicitly
ignored.

HTH,
David
>
> BR,
> Jarmo

No, I didn''t have that one, but had one similar problem where one of
my specs edited one non-Ruby file, which caused autotest to reload. I
solved that problem by explicitly ignoring it in autotest. I was
surprised that autotest cares about non-mapped files :)

But now I have another problem - it seems that autotest doesn''t
understand always when something really failed and it seems to be due
to the fact that autotest.rb has this line:
self.failed_results_re = /^\s+\d+\) (?:Failure|Error):\n(.*?)\((.*?)
\)/

It doesn''t match correctly with most of the failing specs so Snarl
(like Growl for Windows) shows that tests are passing when they''re
actually not. So, I digged around a little more to see if RSpec
modifies this regular expression somewhere and I found it at lib/
autotest/rspec.rb and there is this class Autotest::Rspec < Autotest,
which overwrites failed_results_re as: self.failed_results_re = /^\d+\)
\n(?:\e\[\d*m)?(?:.*?in )?''([^\n]*)''(?:
FAILED)?(?:\e\[\d*m)?\n(.*?)\n
\n/m

This autotest thing seems to be quite strange, because if I added some
puts statements into discover.rb and rspec.rb then they didn''t seem to
get executed, BUT if I putsed this failed_results_re in autotest.rb
itself, then it was set correctly as written above.

My .autotest file is as following:
require ''autotest/menu''
require ''autotest/snarl''

Autotest.add_hook :initialize do |at|
	at.add_exception(/\.sqlite/)
    at.add_exception(/^\.git/)
end

And spec.opts is like this:
-r spec/env.rb
-c

And I''m running autospec and seeing this output:
1)
''correct year'' FAILED

expected "2008"
     got "2009"

(compared using eql?)

./spec/year_spec.rb:38:

Finished in 5.918 seconds

11 examples, 1 failure

And when I added to autotest.rb line 410 this: p self.files_to_test
Then the resulted hash was empty, e.g. this line sets result to
"green": color = completed &&
self.files_to_test.empty? ? :green : :red

When I break some other test with different error message like this,
then Snarl shows me failing tests:
1)
NoMethodError in ''DatabaseOperations adds options''
undefined method `[]'' for nil:NilClass
./spec/database_operations_spec.rb:50:

The difference is that failing reason is shown on one line, instead of
two lines. Also, p elf.files_to_test results in "normal" result:
{"spec/database_operations_spec.rb"=>["DatabaseOperations adds
options", "DatabaseOperations adds new default_options if some are
missing"]}

- Is it possible that this regular expression is little outdated and
can''t handle the situation where failing reason is shown on 2 lines?
- Why won''t anything puts''ed from autotest/discover.rb or
autotest/
rspec.rb like they aren''t loaded?
- Any other reasons why autotest behaviour might be broken?

Jarmo

On Jun 1, 12:56?am, David Chelimsky <dchelim... at gmail.com>
wrote:
> On Sun, May 31, 2009 at 3:59 PM, Jarmo Pertman <Jarm... at gmail.com>
wrote:
> > I wanted to start using autospec on my little (non-Rails) project.
I''m
> > on Windows XP and have used it some time ago successfully (on the same
> > PC), although I''ve updated many gems in the meantime (RSpec
for
> > example among others).
>
> > In short, everything is fine when all specs pass, but if some fails,
> > then autospec won''t stop running failing specs again even if
no files
> > are changed. I expect it to run specs again only if some file gets
> > modified. It''s kinda irritating when it doesn''t stop
and is by no
> > means helping any development :/
>
> > Also, when I fix the failing spec then it will run it only once and
> > then run whole suit once and stops as expected.
>
> > Any ideas what might cause this distracting behaviour?
>
> Do you have "--format failing_examples" in spec/spec.opts? If so,
> rspec saves a file with a list of the failing examples in it, which
> would trigger autotest to run again unless that file is explicitly
> ignored.
>
> HTH,
> David
>
>
>
> > BR,
> > Jarmo
>
> _______________________________________________
> rspec-users mailing list
> rspec-us... at
rubyforge.orghttp://rubyforge.org/mailman/listinfo/rspec-users

Ok. I''ve created failing example.

Just create project directory and spec directory into it and add there
one spec file:

describe "autospec" do

	it "fails" do
		"hi".should eql("hello")
	end

end

It fails, but for autotest it is still green (thus making growl and
snarl to report as everything is passing) and now I''m pretty sure that
it has to do with the fact that "should eql" outputs
"expected" and
"got" result on different lines. As soon as you replace the matcher
to: "hi".eql?("hello").should be_true - then output is on
single line
and autotest detects it as failing.

Still, problem is with the regular expression provided for autotest?

Jarmo

On Tue, Jun 2, 2009 at 11:07 AM, Jarmo Pertman <Jarmo.P at gmail.com>
wrote:
> Ok. I''ve created failing example.
>
> Just create project directory and spec directory into it and add there
> one spec file:
>
> describe "autospec" do
>
> ? ? ? ?it "fails" do
> ? ? ? ? ? ? ? ?"hi".should eql("hello")
> ? ? ? ?end
>
> end
>
> It fails, but for autotest it is still green (thus making growl and
> snarl to report as everything is passing) and now I''m pretty sure
that
> it has to do with the fact that "should eql" outputs
"expected" and
> "got" result on different lines. As soon as you replace the
matcher
> to: "hi".eql?("hello").should be_true - then output is
on single line
> and autotest detects it as failing.
>
> Still, problem is with the regular expression provided for autotest?
Seems likely. Wanna make a proper patch w/ specs?
>
> Jarmo
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

Done at
https://rspec.lighthouseapp.com/projects/5645-rspec/tickets/832-autotest-not-detecting-failing-examples-correctly-when-matcher-outputs-multiple-lines-of-information

I started even before you''ve replied, because that bugged me too
much :)

Jarmo

On Jun 2, 7:50?pm, David Chelimsky <dchelim... at gmail.com>
wrote:
> Seems likely. Wanna make a proper patch w/ specs?