mocha developer - Jan 2007 - slow tests on an established project...

Hi all

Hope this isn''t too far off-topic on this list, as I think it would
largely go ignored on the main rails list.

I''m looking for the most pragmatic way to speed up our test suite.
This is on an established project, and lets just say that we used way
too many fixtures when we started =).  For example:

Finished in 128.870144 seconds.
==============392 tests, 2106 assertions, 0 failures, 0 errors

I''ve read many of the examples on many blogs and I don''t see
anything
that could be applied to legacy test cases that are testing across
many associations.  Or how to change functional tests to make it
possible to inject stubs instead of hitting the db.

Right now I''m thinking the only real options are going with test
groups based on speed, and/or trying to use mysql''s in memory db for
testing.  Rewriting our tests from the ground up is definitely not an
option.  Sqlite3 seems to finicky to handle anything beyond a very
basic data model.  Tools I''m trying to incorporate include mocha,
spec/unit, and zentest (mainly for autotest, I don''t use the view
testing stuff much).

thanks for any ideas or experiences!

- Rob
www.robsanheim.com

On 07/01/07, Rob Sanheim <rsanheim at gmail.com>
wrote:
>
> Hope this isn''t too far off-topic on this list, as I think it
would
> largely go ignored on the main rails list.
>
> I''m looking for the most pragmatic way to speed up our test suite.
> This is on an established project, and lets just say that we used way
> too many fixtures when we started =).  For example:
>
> Finished in 128.870144 seconds.
> ==============> 392 tests, 2106 assertions, 0 failures, 0 errors
>
> I''ve read many of the examples on many blogs and I don''t
see anything
> that could be applied to legacy test cases that are testing across
> many associations.  Or how to change functional tests to make it
> possible to inject stubs instead of hitting the db.
>
> Right now I''m thinking the only real options are going with test
> groups based on speed, and/or trying to use mysql''s in memory db
for
> testing.  Rewriting our tests from the ground up is definitely not an
> option.  Sqlite3 seems to finicky to handle anything beyond a very
> basic data model.  Tools I''m trying to incorporate include mocha,
> spec/unit, and zentest (mainly for autotest, I don''t use the view
> testing stuff much).
>
> thanks for any ideas or experiences!

Can you give an example of one of your "legacy test cases that are testing
across many associations"?

I find it''s always more productive to discuss a concrete example.

-- 
James.
http://blog.floehopper.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/mocha-developer/attachments/20070107/75b0d740/attachment.html

On 1/7/07, James Mead <jamesmead44 at gmail.com>
wrote:
> On 07/01/07, Rob Sanheim <rsanheim at gmail.com> wrote:
>
> > Hope this isn''t too far off-topic on this list, as I think it
would
> > largely go ignored on the main rails list.
> >
> > I''m looking for the most pragmatic way to speed up our test
suite.
> > This is on an established project, and lets just say that we used way
> > too many fixtures when we started =).  For example:
> >
> > Finished in 128.870144 seconds.
> > ==============> > 392 tests, 2106 assertions, 0 failures, 0
errors
> >
> > I''ve read many of the examples on many blogs and I
don''t see anything
> > that could be applied to legacy test cases that are testing across
> > many associations.  Or how to change functional tests to make it
> > possible to inject stubs instead of hitting the db.
> >
> > Right now I''m thinking the only real options are going with
test
> > groups based on speed, and/or trying to use mysql''s in memory
db for
> > testing.  Rewriting our tests from the ground up is definitely not an
> > option.  Sqlite3 seems to finicky to handle anything beyond a very
> > basic data model.  Tools I''m trying to incorporate include
mocha,
> > spec/unit, and zentest (mainly for autotest, I don''t use the
view
> > testing stuff much).
> >
> > thanks for any ideas or experiences!
>
>
> Can you give an example of one of your "legacy test cases that are
testing
> across many associations"?
>
> I find it''s always more productive to discuss a concrete example.
>
> --
>  James.
> http://blog.floehopper.org
Hi James

Sorry for the slow reply, didn''t have any time to get examples last
week.  Here are two typical examples.  Part of the problem that isn''t
clear from these tests is that we have a *lot* of records in
controllers that are pulled back to construct things like the sidebars
and headers.  A lot of it is data that will eventually be cached
heavily, but we aren''t to that point yet and I''m guessing
we''ll leave
memcache off in test anyways.

For certain functional tests I''ve gone through and overridden things
for the test environment, for instance specifying that sidebars
shouldn''t be loaded.  That works, but it seems a bit smelly to me.

Also, I realize this code isn''t the best - a lot of it could greatly
benefit for some heavy refactoring and "less controller, more model"
=).

(two tests within CategoryControllerTest:)

  def test_should_get_second_chunk_of_posts_for_page_two
  # posts per page has been set to 1, to make this easier to test with
a small set of test data
    get :show, :category => categories(:internet_search).slug, :page_number
=> 2
    assert_equal 1, assigns(:posts).size
    posts = internet_search.find_recent_posts(:limit => 2)
    assert_equal posts[1], assigns(:posts)[0]
  end

  def test_should_get_latest_comments_for_author
    get :show, :category => "david-smith", :type =>
"author"
    comments = assigns(:latest_comments)
    assert !comments.empty?
    comments.each { |c| assert_kind_of(Comment, c) }
    assert comments.include?(common_comments(:first_comment_on_google_vs_yahoo))
    assert_success
  end

I''ve posted the controller under test here, as it probably
wouldn''t
come through too well in email.

http://pastie.caboo.se/34843

thanks,
Rob

I''m sorry I haven''t had time to look at your examples in
detail yet.

I just read this article which I thought might be of interest to you...

http://blog.caboo.se/articles/2007/2/13/what-about-those-bloated-tests

-- 
James.
http://blog.floehopper.org