Rails - Jul 2009 - how to make ZenTest autotest run whenever my tests change

Railsters:

Despite Rails being the only Web platform designed for TDD, a lot of its test 
infrastructure is still "cargo cult" - imitating other test rigs
instead of
understanding their principles.

Most importantly, tests should run instantly. There''s no excuse for
breaking
this rule, and if you invent a platform that can''t obey it then you are
doing
something wrong. And I have yet to meet a combination of editor and test script 
which does this. Even Visual Studio and C++ are faster and easier to work with.

The Rails problem is apparently rake db:test:prepare, which clones your database
instead of detecting whether it needs a clone.

I have heard the fix is ZenTest''s autotest. This allegedly loads the
Ruby "VM"
once, awaits file changes, and responds correctly to each one.

So I have a vanilla Rails 2.3.2 application, and I install autotest (and 
autotest -v does not work, so I can''t see which version number I
actually get),
and the gem system claims I have version 4.1.3. And I run autotest in the 
project folder, and it says:

$ autotest
(Not running features.  To run features in autotest, set AUTOFEATURE=true.)
/usr/bin/ruby1.8 -I.:lib:test -rubygems -e "%w[test/unit 
test/test_helper.rb].each { |f| require f }" | unit_diff -u
Loaded suite -e
Started

Finished in 0.000437 seconds.

0 tests, 0 assertions, 0 failures, 0 errors

Note the system did not even run the whole test batch, first. And note it sees 
and warns that it rejects my features folder. I need it to see my test folder 
(NOT a spec folder!).

Then I make a non-trivial change to a correctly-named test file, save the file, 
and nothing triggers. (Pure Ubuntu Linux substrate, BTW.)

So how do I trigger?

If I could get that working, how can I TDD both story and unit tests AT THE SAME
TIME, without using two different test runners?

-- 
   Phlip

On Mon, Jul 6, 2009 at 10:54 AM,
Phlip<phlip2005-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
> So I have a vanilla Rails 2.3.2 application, and I install autotest (and
> autotest -v does not work, so I can''t see which version number I
actually get),
> and the gem system claims I have version 4.1.3. And I run autotest in the
> project folder, and it says:
>
> $ autotest
> (Not running features.  To run features in autotest, set AUTOFEATURE=true.)
> /usr/bin/ruby1.8 -I.:lib:test -rubygems -e "%w[test/unit
> test/test_helper.rb].each { |f| require f }" | unit_diff -u
> Loaded suite -e
> Started
>
> Finished in 0.000437 seconds.
>
> 0 tests, 0 assertions, 0 failures, 0 errors
sudo gem install autotest-rails. Autotest does not automatically
intuit project directory structures, beyond the simplest, oldest Ruby
idiom (lib/*.rb => test/test_*.rb). The autotest-* set of support gems
adds support for more complex directory structures, like Rails''.


~ j.

John Barnette wrote:
>> 0 tests, 0 assertions, 0 failures, 0 errors
> 
> sudo gem install autotest-rails
Yay thank you the good news is that runs now. (I would have gone with 
test/**/*suite.rb myself...)

The bad news is it''s still slow as f---. Yes Gherkin is partly to
blame. But
does anyone have a replacement for rake db:test:prepare that does less??

-- 
   Phlip

Phlip wrote:
> Railsters:
> 
> Despite Rails being the only Web platform designed for TDD, 
Really?  I have the impression that there are others...if nothing else, 
some of the Railsy PHP frameworks have copied this feature.

[...]
> Most importantly, tests should run instantly. 
In most cases, mine do.  The exceptions are those which use regexps or 
DOM wizardry for parsing HTML.  I suspect Oniguruma might help here, but 
I haven''t tried it.
> And I have yet to meet a combination of editor and test 
> script
> which does this.
RSpec + autotest + out-of-the-box test scripts + any editor you like. 
Done.
> 
> The Rails problem is apparently rake db:test:prepare, which clones your 
> database
> instead of detecting whether it needs a clone.
When would it *not* need a clone?  Tests are supposed to start from a 
clean environment.

I think the real time sink may be loading the Rails environment, which 
seems to typically take several seconds on an otherwise fast test run. 
But I can wait that long for my tests to run.

Jay Fields has posted some stuff on his blog about how to make Rails 
unit tests not touch the DB, but I don''t really see much need to do 
that.
> 
> I have heard the fix is ZenTest''s autotest. This allegedly loads
the
> Ruby "VM"
> once, awaits file changes, and responds correctly to each one.
Yup.  That''s what it does as far as I understand.
> 
> So I have a vanilla Rails 2.3.2 application, and I install autotest (and
> autotest -v does not work,
-v is verbose, not version.  The autotest script is part of the ZenTest 
package, and doesn''t appear to have a version number of its own.
> so I can''t see which version number I 
> actually get),
> and the gem system claims I have version 4.1.3. 
Of what?  ZenTest?  (FWIW, I''m using ZenTest 3.11.1; I''ll have
to try
the newer version.)
> And I run autotest in 
> the
> project folder, and it says:
> 
> $ autotest
> (Not running features.  To run features in autotest, set 
> AUTOFEATURE=true.)
> /usr/bin/ruby1.8 -I.:lib:test -rubygems -e "%w[test/unit
> test/test_helper.rb].each { |f| require f }" | unit_diff -u
> Loaded suite -e
> Started
> 
> Finished in 0.000437 seconds.
> 
> 0 tests, 0 assertions, 0 failures, 0 errors
> 
> Note the system did not even run the whole test batch, first. 
Bizarre.  It does for me with RSpec.  Are your tests in the directories 
that Autotest is searching (which would appear to be test/unit)?
> And note 
> it sees
> and warns that it rejects my features folder. 
Not quite.  It is warning you that it did not even look for your 
features folder, and will not look for it unless you set 
AUTOFEATURE=true.  So try that.   (Personally, I think features are too 
time-consuming to run as often as tests, because I tend to have them do 
a lot of browser simulation.)
> I need it to see my test 
> folder
> (NOT a spec folder!).
It should do that.  Check your configuration.
> 
> Then I make a non-trivial change to a correctly-named test file, save 
> the file,
> and nothing triggers. (Pure Ubuntu Linux substrate, BTW.)
> 
> So how do I trigger?
Again, this is likely a ZenTest/autotest config issue.  What you''re 
describing as desired is exactly what ZT/AT should normally do.
> 
> If I could get that working, how can I TDD both story and unit tests AT 
> THE SAME
> TIME, without using two different test runners?
Set AUTOFEATURE=true, like it''s telling you in that message.  ZT/AT
will
call rake test and cucumber as appropriate.
> 
> --
>    Phlip
Best,
--
Marnen Laibow-Koser
http://www.marnen.org
marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org
-- 
Posted via http://www.ruby-forum.com/.

Marnen Laibow-Koser wrote:
>> Despite Rails being the only Web platform designed for TDD, 
> 
> Really?  I have the impression that there are others...if nothing else, 
> some of the Railsy PHP frameworks have copied this feature.
designed for - not copied on.
>> Most importantly, tests should run instantly. 
> 
> In most cases, mine do.  The exceptions are those which use regexps or 
> DOM wizardry for parsing HTML.  I suspect Oniguruma might help here, but 
> I haven''t tried it.
Rails??
> RSpec + autotest + out-of-the-box test scripts + any editor you like. 
> Done.
When the tests fail, what button do you type to navigate instantly to the test 
failure?
>> The Rails problem is apparently rake db:test:prepare, which clones your
>> database
>> instead of detecting whether it needs a clone.
> 
> When would it *not* need a clone?  Tests are supposed to start from a 
> clean environment.
We do not build a new CPU with new libraries and Ruby installation between each 
test run. All test isolation is a trade-off of some kind.
> Jay Fields has posted some stuff on his blog about how to make Rails 
> unit tests not touch the DB, but I don''t really see much need to
do
> that.
Thanks - that''s the next thing I was going to ... rail at. Some
consultants who
teach TDD use a mind-trick to motivate their newbs to decouple their code. In a 
big-iron environment where database _migrations_ are expensive, until you pay 
that down, you can get a slight benefit to unit tests that reach into code that 
decouples from the database.

We should do that too. And when we need a fixture, to start such a test case, it
should come from the database.

-- 
   Phlip

On Mon, Jul 6, 2009 at 12:10 PM,
Phlip<phlip2005-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
> John Barnette wrote:
>> sudo gem install autotest-rails
>
> Yay thank you the good news is that runs now. (I would have gone with
> test/**/*suite.rb myself...)
Excellent, glad it worked for you. Suites were originally bad for
autotest because it attempts to do two-way mapping between test and
implementation files, and suites generally aggregate a bunch of test
files.
> The bad news is it''s still slow as f---. Yes Gherkin is partly to
blame. But
> does anyone have a replacement for rake db:test:prepare that does less??
Are you absolutely certain that it''s the DB load? Have you benchmarked
''rake db:test:prepare'' against ''rake
environment''? The latter is
basically the minimum price you pay for a Rake task that loads Rails.


~ j.

> RSpec + autotest + out-of-the-box test scripts + any editor you like. 
> Done.
FWIW, this is still not competing with using Alt+Tab to switch to a console, 
then running script/cucumber features/whatever.feature. Autotest seems to just 
get mired in unit tests that I did not recently change, or something!

-- 
   Phlip

John Barnette wrote:
> Are you absolutely certain that it''s the DB load?
Nope!

 > Have you benchmarked
> ''rake db:test:prepare'' against ''rake
environment''?
Of course not!
> The latter is
> basically the minimum price you pay for a Rake task that loads Rails.
$ time rake db:test:prepare

real    0m2.153s
user    0m1.324s
sys     0m0.324s

$ time rake environment

real    0m1.623s
user    0m1.316s
sys     0m0.280s

Rhetorical question: So if that''s the fast part, what is so f---ing
slow???

-- 
   Phlip

Phlip wrote:
> Marnen Laibow-Koser wrote:
> 
>>> Despite Rails being the only Web platform designed for TDD, 
>> 
>> Really?  I have the impression that there are others...if nothing else,
>> some of the Railsy PHP frameworks have copied this feature.
> 
> designed for - not copied on.
Copying a feature in the design phase is still designing it in.  Or were 
you going to say that Ruby wasn''t really designed for most of its cool 
features because they were copied from Smalltalk and Lisp? :)
> 
>>> Most importantly, tests should run instantly. 
>> 
>> In most cases, mine do.  The exceptions are those which use regexps or 
>> DOM wizardry for parsing HTML.  I suspect Oniguruma might help here,
but
>> I haven''t tried it.
> 
> Rails??
Huh?  Oniguruma is what I haven''t tried.
> 
>> RSpec + autotest + out-of-the-box test scripts + any editor you like. 
>> Done.
> 
> When the tests fail, what button do you type to navigate instantly to 
> the test
> failure?
Cmd-Tab.  The test failure is usually where I was last editing. :)

Seriously, you''re asking the wrong person.  I don''t really
care that
much about that level of editor integration.  If you do, I believe there 
is a TextMate bundle that will do what you want, but you will get better 
answers from others on this list.

[...]
> We do not build a new CPU with new libraries and Ruby installation 
> between each
> test run. 
No, because we can guarantee that those bits won''t be changed in a 
typical app.  We cannot guarantee that about the DB, can we?

I *would* want a new Ruby installation for each test run if I thought my 
code were likely to modify the system Ruby installation as it ran.
> All test isolation is a trade-off of some kind.
Sure.
> 
>> Jay Fields has posted some stuff on his blog about how to make Rails 
>> unit tests not touch the DB, but I don''t really see much need
to do
>> that.
> 
> Thanks - that''s the next thing I was going to ... rail at. Some 
> consultants who
> teach TDD use a mind-trick to motivate their newbs to decouple their 
> code. In a
> big-iron environment where database _migrations_ are expensive, until 
> you pay
> that down, you can get a slight benefit to unit tests that reach into 
> code that
> decouples from the database.
> 
> We should do that too. 
[...]

Do what?  You mentioned several things, and I''m not sure which one 
you''re referring to.

Best,
--
Marnen Laibow-Koser
http://www.marnen.org
marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org
-- 
Posted via http://www.ruby-forum.com/.