Rails - Nov 2005 - Concerns over Rails' handling of tests

Sorry if this comes across as a bunch of disjointed thoughts...I''m just
trying to put my thoughts down and I''d like to know what other
people''s
opinions on the subject are.

I''ve been working with Rails for 3 or 4 months now and I''m
constantly trying
to look at ways of improving my testing techniques, especially when it comes
to TDD. I have a few concerns about the way Rails approaches tests with
regards to controllers/models.

Rails has always seemed to apply that you unit test your models and
functional test your controllers. For me however, this distinction breaks
down in practice and leads to what are in my opinion very smelly tests.

Firstly, I don''t like the way that by default Rails assumes you want to
test
an entire controller in a testcase. I''ve found that this leads to tests
that
are far to big. I believe better breakdown would be to have a test case for
each action, or even a testcase for each action in different contexts. So,
as an example, say you had an ArticleController with view, post and create
actions. The post action simply displays the form which submits to the
create action.

Now, instead of having one large ArticleControllerTest, it would make far
more sense to break it down into ArticleViewTest, ArticlePostTest,
FailedArticleCreate test and SuccessfulArticleCreate, because each of those
represent different contexts, or fixtures.

I''m quite keen on Ben Griffiths'' testing approach outlined
here<http://www.reevoo.com/blogs/bengriffiths/2005/06/24/a-test-by-any-other-name/>and
have also been looking into Dave Astels''
RSpec <http://rspec.rubyforge.org/> and have to agree with the idea of
testcases being bound by fixtures. If you have test methods that require
common setups, the code should be extracted to the setup method. If you have
different groups of methods using two different types of setups, this is an
indication that you should be creating a new testcase for the new fixture.
Overall, I''ve found that trying to test a whole controller leads to
multiple
fixtures in one testcase and multiple violations of DRY.

This also leads me on to my other concern, and that is the practice of
testing for the presence of markup in a view using assert_tag. I can''t
really see the value in this, and again it seems to violate DRY to me. Why
would you want to assert that a tag exists in most cases? For example, given
the ArticlePostTest example above, which wouldn''t really do much in the
controller, and simply pass to the view for the rendering of the form. Why
would I want to test for the structure of the form? Ultimately my form will
be laid out using CSS but there are still several ways I could markup my
form, with all variations being "correct". By testing for one
particular
implementation in my test, I''m duplicating the markup in my test and my
view, and binding the test to my implementation of the form. Testing that a
particular HTML element is present in my view does nothing to tell me that
my view is "correct", therefore I think this makes the test worthless.

Which leads me on to saying that views such as forms are the realm for
proper functional testing using something like Watir or Selenium, not
TestUnit and tag assertions. You want to check that you can fill in the form
and submit it, not that the page contains a <form> tag.

So what would I really like to see? No separation of unit and functional
testing in terms of model and controller code. They should all be covered by
unit tests. Functional testing should be the realm of something like Watir
or Selenium, unless somebody can put together a good library that lets you
do this kind of functional testing (i.e., filling in forms, interacting with
your web application as if it were in a browser) from within your TestUnit
tests. I''d like to see less emphasis on the idea of testing your
controllers
as a whole and testing your different actions as smaller, focussed unit
tests. I think this would also aid with TDD.

Dave Astels'' has some good reading material here:

http://blog.daveastels.com/?p=53

What do you think?

Cheers
Luke


_______________________________________________
Rails mailing list
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
http://lists.rubyonrails.org/mailman/listinfo/rails

At the risk of starting a flame war, I see no difference between
testing the controller in one test or testing every element of the
controller in seperate tests.  It sounds like semantics to me.

The important thing is that there is actual testing going on. 
Everything after that is gravy. :)

On 11/16/05, Luke Redpath
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org>
wrote:
> Sorry if this comes across as a bunch of disjointed thoughts...I''m
just
> trying to put my thoughts down and I''d like to know what other
people''s
> opinions on the subject are.
>
> I''ve been working with Rails for 3 or 4 months now and
I''m constantly trying
> to look at ways of improving my testing techniques, especially when it
comes
> to TDD. I have a few concerns about the way Rails approaches tests with
> regards to controllers/models.
>
> Rails has always seemed to apply that you unit test your models and
> functional test your controllers. For me however, this distinction breaks
> down in practice and leads to what are in my opinion very smelly tests.
>
> Firstly, I don''t like the way that by default Rails assumes you
want to test
> an entire controller in a testcase. I''ve found that this leads to
tests that
> are far to big. I believe better breakdown would be to have a test case for
> each action, or even a testcase for each action in different contexts. So,
> as an example, say you had an ArticleController with view, post and create
> actions. The post action simply displays the form which submits to the
> create action.
>
> Now, instead of having one large ArticleControllerTest, it would make far
> more sense to break it down into ArticleViewTest, ArticlePostTest,
> FailedArticleCreate test and SuccessfulArticleCreate, because each of those
> represent different contexts, or fixtures.
>
> I''m quite keen on Ben Griffiths'' testing approach
outlined here and have
> also been looking into Dave Astels'' RSpec and have to agree with
the idea of
> testcases being bound by fixtures. If you have test methods that require
> common setups, the code should be extracted to the setup method. If you
have
> different groups of methods using two different types of setups, this is an
> indication that you should be creating a new testcase for the new fixture.
> Overall, I''ve found that trying to test a whole controller leads
to multiple
> fixtures in one testcase and multiple violations of DRY.
>
> This also leads me on to my other concern, and that is the practice of
> testing for the presence of markup in a view using assert_tag. I
can''t
> really see the value in this, and again it seems to violate DRY to me. Why
> would you want to assert that a tag exists in most cases? For example,
given
> the ArticlePostTest example above, which wouldn''t really do much
in the
> controller, and simply pass to the view for the rendering of the form. Why
> would I want to test for the structure of the form? Ultimately my form will
> be laid out using CSS but there are still several ways I could markup my
> form, with all variations being "correct". By testing for one
particular
> implementation in my test, I''m duplicating the markup in my test
and my
> view, and binding the test to my implementation of the form. Testing that a
> particular HTML element is present in my view does nothing to tell me that
> my view is "correct", therefore I think this makes the test
worthless.
>
> Which leads me on to saying that views such as forms are the realm for
> proper functional testing using something like Watir or Selenium, not
> TestUnit and tag assertions. You want to check that you can fill in the
form
> and submit it, not that the page contains a <form> tag.
>
> So what would I really like to see? No separation of unit and functional
> testing in terms of model and controller code. They should all be covered
by
> unit tests. Functional testing should be the realm of something like Watir
> or Selenium, unless somebody can put together a good library that lets you
> do this kind of functional testing ( i.e., filling in forms, interacting
> with your web application as if it were in a browser) from within your
> TestUnit tests. I''d like to see less emphasis on the idea of
testing your
> controllers as a whole and testing your different actions as smaller,
> focussed unit tests. I think this would also aid with TDD.
>
> Dave Astels'' has some good reading material here:
>
> http://blog.daveastels.com/?p=53
>
> What do you think?
>
> Cheers
> Luke
>
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
>
>

--
Chris Hartjes

@TheBallpark - http://www.littlehart.net/attheballpark
@TheKeyboard - http://www.littlehart.net/atthekeyboard

> At the risk of starting a flame war, I see no difference between
> testing the controller in one test or testing every element of the
> controller in seperate tests. It sounds like semantics to me.
Well to me, thats like saying "there is no difference between having
everything wrapped up in one God class or separated into smaller, more
specific fine-grained classes". In fact, why have separate test cases for
each controller, why not one test case for your whole app? ;)

I think tests should be given as much as care and consideration as your
production code. If they start getting bloated they should be refactored,
like your production code. Smelly tests become harder and harder to work
with and IMO this will start to have a detrimental effect on your testing.

Cheers
Luke

On 11/16/05, Chris Hartjes
<chartjes-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
>
> At the risk of starting a flame war, I see no difference between
> testing the controller in one test or testing every element of the
> controller in seperate tests. It sounds like semantics to me.
>
> The important thing is that there is actual testing going on.
> Everything after that is gravy. :)
>
> On 11/16/05, Luke Redpath
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org> wrote:
> > Sorry if this comes across as a bunch of disjointed
thoughts...I''m just
> > trying to put my thoughts down and I''d like to know what
other people''s
> > opinions on the subject are.
> >
> > I''ve been working with Rails for 3 or 4 months now and
I''m constantly
> trying
> > to look at ways of improving my testing techniques, especially when it
> comes
> > to TDD. I have a few concerns about the way Rails approaches tests
with
> > regards to controllers/models.
> >
> > Rails has always seemed to apply that you unit test your models and
> > functional test your controllers. For me however, this distinction
> breaks
> > down in practice and leads to what are in my opinion very smelly
tests.
> >
> > Firstly, I don''t like the way that by default Rails assumes
you want to
> test
> > an entire controller in a testcase. I''ve found that this
leads to tests
> that
> > are far to big. I believe better breakdown would be to have a test
case
> for
> > each action, or even a testcase for each action in different contexts.
> So,
> > as an example, say you had an ArticleController with view, post and
> create
> > actions. The post action simply displays the form which submits to the
> > create action.
> >
> > Now, instead of having one large ArticleControllerTest, it would make
> far
> > more sense to break it down into ArticleViewTest, ArticlePostTest,
> > FailedArticleCreate test and SuccessfulArticleCreate, because each of
> those
> > represent different contexts, or fixtures.
> >
> > I''m quite keen on Ben Griffiths'' testing approach
outlined here and have
> > also been looking into Dave Astels'' RSpec and have to agree
with the
> idea of
> > testcases being bound by fixtures. If you have test methods that
require
> > common setups, the code should be extracted to the setup method. If
you
> have
> > different groups of methods using two different types of setups, this
is
> an
> > indication that you should be creating a new testcase for the new
> fixture.
> > Overall, I''ve found that trying to test a whole controller
leads to
> multiple
> > fixtures in one testcase and multiple violations of DRY.
> >
> > This also leads me on to my other concern, and that is the practice of
> > testing for the presence of markup in a view using assert_tag. I
can''t
> > really see the value in this, and again it seems to violate DRY to me.
> Why
> > would you want to assert that a tag exists in most cases? For example,
> given
> > the ArticlePostTest example above, which wouldn''t really do
much in the
> > controller, and simply pass to the view for the rendering of the form.
> Why
> > would I want to test for the structure of the form? Ultimately my form
> will
> > be laid out using CSS but there are still several ways I could markup
my
> > form, with all variations being "correct". By testing for
one particular
> > implementation in my test, I''m duplicating the markup in my
test and my
> > view, and binding the test to my implementation of the form. Testing
> that a
> > particular HTML element is present in my view does nothing to tell me
> that
> > my view is "correct", therefore I think this makes the test
worthless.
> >
> > Which leads me on to saying that views such as forms are the realm for
> > proper functional testing using something like Watir or Selenium, not
> > TestUnit and tag assertions. You want to check that you can fill in
the
> form
> > and submit it, not that the page contains a <form> tag.
> >
> > So what would I really like to see? No separation of unit and
functional
> > testing in terms of model and controller code. They should all be
> covered by
> > unit tests. Functional testing should be the realm of something like
> Watir
> > or Selenium, unless somebody can put together a good library that lets
> you
> > do this kind of functional testing ( i.e., filling in forms,
interacting
> > with your web application as if it were in a browser) from within your
> > TestUnit tests. I''d like to see less emphasis on the idea of
testing
> your
> > controllers as a whole and testing your different actions as smaller,
> > focussed unit tests. I think this would also aid with TDD.
> >
> > Dave Astels'' has some good reading material here:
> >
> > http://blog.daveastels.com/?p=53
> >
> > What do you think?
> >
> > Cheers
> > Luke
> >
> > _______________________________________________
> > Rails mailing list
> > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> > http://lists.rubyonrails.org/mailman/listinfo/rails
> >
> >
> >
>
>
> --
> Chris Hartjes
>
> @TheBallpark - http://www.littlehart.net/attheballpark
> @TheKeyboard - http://www.littlehart.net/atthekeyboard
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>

_______________________________________________
Rails mailing list
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
http://lists.rubyonrails.org/mailman/listinfo/rails

Another difference this brings to mind is that functional tests (or
Acceptance Tests as XP calls them) should not be written test-first, but are
instead defined at the beginning of an iteration for a series of user
stories. Acceptance tests do not have to run at 100% all the time (in fact
they start at 0% and approach 100% as the iteration nears completion),
wherease Unit Tests should always be running at 100%. Acceptance tests
represent what the completed system should do when finished, whereas Unit
Tests represent what your code actually does.

Its perfectly possible to design controller actions test-first, but this is
still unit testing. Acceptance testing an action would be to load it in your
browser, fill in the form, submit it, and check it worked. This is why I
mention things like Watir or Selenium for acceptance testing over TestUnit.
FIT would be another option.

This is just more evidence to me, that Rails'' distinction of unit tests
for
models and functional tests for controllers, is wrong.

Cheers
Luke

On 11/16/05, Luke Redpath
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org>
wrote:
>
> > At the risk of starting a flame war, I see no difference between
> > testing the controller in one test or testing every element of the
> > controller in seperate tests. It sounds like semantics to me.
>
> Well to me, thats like saying "there is no difference between having
> everything wrapped up in one God class or separated into smaller, more
> specific fine-grained classes". In fact, why have separate test cases
for
> each controller, why not one test case for your whole app? ;)
>
> I think tests should be given as much as care and consideration as your
> production code. If they start getting bloated they should be refactored,
> like your production code. Smelly tests become harder and harder to work
> with and IMO this will start to have a detrimental effect on your testing.
>
> Cheers
> Luke
>
> On 11/16/05, Chris Hartjes
<chartjes-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >
> > At the risk of starting a flame war, I see no difference between
> > testing the controller in one test or testing every element of the
> > controller in seperate tests. It sounds like semantics to me.
> >
> > The important thing is that there is actual testing going on.
> > Everything after that is gravy. :)
> >
> > On 11/16/05, Luke Redpath
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org> wrote:
> > > Sorry if this comes across as a bunch of disjointed
thoughts...I''m
> > just
> > > trying to put my thoughts down and I''d like to know what
other
> > people''s
> > > opinions on the subject are.
> > >
> > > I''ve been working with Rails for 3 or 4 months now and
I''m constantly
> > trying
> > > to look at ways of improving my testing techniques, especially
when it
> > comes
> > > to TDD. I have a few concerns about the way Rails approaches
tests
> > with
> > > regards to controllers/models.
> > >
> > > Rails has always seemed to apply that you unit test your models
and
> > > functional test your controllers. For me however, this
distinction
> > breaks
> > > down in practice and leads to what are in my opinion very smelly
> > tests.
> > >
> > > Firstly, I don''t like the way that by default Rails
assumes you want
> > to test
> > > an entire controller in a testcase. I''ve found that this
leads to
> > tests that
> > > are far to big. I believe better breakdown would be to have a
test
> > case for
> > > each action, or even a testcase for each action in different
contexts.
> > So,
> > > as an example, say you had an ArticleController with view, post
and
> > create
> > > actions. The post action simply displays the form which submits
to the
> > > create action.
> > >
> > > Now, instead of having one large ArticleControllerTest, it would
make
> > far
> > > more sense to break it down into ArticleViewTest,
ArticlePostTest,
> > > FailedArticleCreate test and SuccessfulArticleCreate, because
each of
> > those
> > > represent different contexts, or fixtures.
> > >
> > > I''m quite keen on Ben Griffiths'' testing
approach outlined here and
> > have
> > > also been looking into Dave Astels'' RSpec and have to
agree with the
> > idea of
> > > testcases being bound by fixtures. If you have test methods that
> > require
> > > common setups, the code should be extracted to the setup method.
If
> > you have
> > > different groups of methods using two different types of setups,
this
> > is an
> > > indication that you should be creating a new testcase for the new
> > fixture.
> > > Overall, I''ve found that trying to test a whole
controller leads to
> > multiple
> > > fixtures in one testcase and multiple violations of DRY.
> > >
> > > This also leads me on to my other concern, and that is the
practice of
> > > testing for the presence of markup in a view using assert_tag. I
can''t
> >
> > > really see the value in this, and again it seems to violate DRY
to me.
> > Why
> > > would you want to assert that a tag exists in most cases? For
example,
> > given
> > > the ArticlePostTest example above, which wouldn''t really
do much in
> > the
> > > controller, and simply pass to the view for the rendering of the
form.
> > Why
> > > would I want to test for the structure of the form? Ultimately my
form
> > will
> > > be laid out using CSS but there are still several ways I could
markup
> > my
> > > form, with all variations being "correct". By testing
for one
> > particular
> > > implementation in my test, I''m duplicating the markup in
my test and
> > my
> > > view, and binding the test to my implementation of the form.
Testing
> > that a
> > > particular HTML element is present in my view does nothing to
tell me
> > that
> > > my view is "correct", therefore I think this makes the
test worthless.
> > >
> > > Which leads me on to saying that views such as forms are the
realm for
> >
> > > proper functional testing using something like Watir or Selenium,
not
> > > TestUnit and tag assertions. You want to check that you can fill
in
> > the form
> > > and submit it, not that the page contains a <form> tag.
> > >
> > > So what would I really like to see? No separation of unit and
> > functional
> > > testing in terms of model and controller code. They should all be
> > covered by
> > > unit tests. Functional testing should be the realm of something
like
> > Watir
> > > or Selenium, unless somebody can put together a good library that
lets
> > you
> > > do this kind of functional testing ( i.e., filling in forms,
> > interacting
> > > with your web application as if it were in a browser) from within
your
> >
> > > TestUnit tests. I''d like to see less emphasis on the
idea of testing
> > your
> > > controllers as a whole and testing your different actions as
smaller,
> > > focussed unit tests. I think this would also aid with TDD.
> > >
> > > Dave Astels'' has some good reading material here:
> > >
> > > http://blog.daveastels.com/?p=53
> > >
> > > What do you think?
> > >
> > > Cheers
> > > Luke
> > >
> > > _______________________________________________
> > > Rails mailing list
> > > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> > > http://lists.rubyonrails.org/mailman/listinfo/rails
> > >
> > >
> > >
> >
> >
> > --
> > Chris Hartjes
> >
> > @TheBallpark - http://www.littlehart.net/attheballpark
> > @TheKeyboard - http://www.littlehart.net/atthekeyboard
> > _______________________________________________
> > Rails mailing list
> > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> > http://lists.rubyonrails.org/mailman/listinfo/rails
> >
>
>

_______________________________________________
Rails mailing list
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
http://lists.rubyonrails.org/mailman/listinfo/rails

So, further to that, I think it would be great if Rails removed this
distinction as it currently stands, and introduced a means of writing proper
functional/acceptance tests, ideally using something like
FIT/Watir/Selenium, that can be defined early and made to pass through the
development of working code (covered by unit tests, written test first).

Of course, I''m very much aware of the whole Rails being opinionated. I
just
think this is an instance where Rails has got it wrong and am voicing my
opinion and trying to provoke discussion. I''m not trying to start a
flame
war. I''m well aware I can implement my own ideas in my own code and
intend
on doing so. But I do think Rails'' take on things muddies the waters a
bit.

Cheers
Luke

On 11/16/05, Luke Redpath
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org>
wrote:
>
> Another difference this brings to mind is that functional tests (or
> Acceptance Tests as XP calls them) should not be written test-first, but
are
> instead defined at the beginning of an iteration for a series of user
> stories. Acceptance tests do not have to run at 100% all the time (in fact
> they start at 0% and approach 100% as the iteration nears completion),
> wherease Unit Tests should always be running at 100%. Acceptance tests
> represent what the completed system should do when finished, whereas Unit
> Tests represent what your code actually does.
>
> Its perfectly possible to design controller actions test-first, but this
> is still unit testing. Acceptance testing an action would be to load it in
> your browser, fill in the form, submit it, and check it worked. This is why
> I mention things like Watir or Selenium for acceptance testing over
> TestUnit. FIT would be another option.
>
> This is just more evidence to me, that Rails'' distinction of unit
tests
> for models and functional tests for controllers, is wrong.
>
> Cheers
> Luke
>
> On 11/16/05, Luke Redpath
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org> wrote:
> >
> > > At the risk of starting a flame war, I see no difference between
> > > testing the controller in one test or testing every element of
the
> > > controller in seperate tests. It sounds like semantics to me.
> >
> > Well to me, thats like saying "there is no difference between
having
> > everything wrapped up in one God class or separated into smaller, more
> > specific fine-grained classes". In fact, why have separate test
cases for
> > each controller, why not one test case for your whole app? ;)
> >
> > I think tests should be given as much as care and consideration as
your
> > production code. If they start getting bloated they should be
refactored,
> > like your production code. Smelly tests become harder and harder to
work
> > with and IMO this will start to have a detrimental effect on your
testing.
> >
> > Cheers
> > Luke
> >
> > On 11/16/05, Chris Hartjes <
chartjes-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > >
> > > At the risk of starting a flame war, I see no difference between
> > > testing the controller in one test or testing every element of
the
> > > controller in seperate tests. It sounds like semantics to me.
> > >
> > > The important thing is that there is actual testing going on.
> > > Everything after that is gravy. :)
> > >
> > > On 11/16/05, Luke Redpath
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org> wrote:
> > > > Sorry if this comes across as a bunch of disjointed
thoughts...I''m
> > > just
> > > > trying to put my thoughts down and I''d like to know
what other
> > > people''s
> > > > opinions on the subject are.
> > > >
> > > > I''ve been working with Rails for 3 or 4 months now
and I''m
> > > constantly trying
> > > > to look at ways of improving my testing techniques,
especially when
> > > it comes
> > > > to TDD. I have a few concerns about the way Rails approaches
tests
> > > with
> > > > regards to controllers/models.
> > > >
> > > > Rails has always seemed to apply that you unit test your
models and
> > > > functional test your controllers. For me however, this
distinction
> > > breaks
> > > > down in practice and leads to what are in my opinion very
smelly
> > > tests.
> > > >
> > > > Firstly, I don''t like the way that by default Rails
assumes you want
> > > to test
> > > > an entire controller in a testcase. I''ve found that
this leads to
> > > tests that
> > > > are far to big. I believe better breakdown would be to have
a test
> > > case for
> > > > each action, or even a testcase for each action in different
> > > contexts. So,
> > > > as an example, say you had an ArticleController with view,
post and
> > > create
> > > > actions. The post action simply displays the form which
submits to
> > > the
> > > > create action.
> > > >
> > > > Now, instead of having one large ArticleControllerTest, it
would
> > > make far
> > > > more sense to break it down into ArticleViewTest,
ArticlePostTest,
> > > > FailedArticleCreate test and SuccessfulArticleCreate,
because each
> > > of those
> > > > represent different contexts, or fixtures.
> > > >
> > > > I''m quite keen on Ben Griffiths'' testing
approach outlined here and
> > > have
> > > > also been looking into Dave Astels'' RSpec and have
to agree with the
> > > idea of
> > > > testcases being bound by fixtures. If you have test methods
that
> > > require
> > > > common setups, the code should be extracted to the setup
method. If
> > > you have
> > > > different groups of methods using two different types of
setups,
> > > this is an
> > > > indication that you should be creating a new testcase for
the new
> > > fixture.
> > > > Overall, I''ve found that trying to test a whole
controller leads to
> > > multiple
> > > > fixtures in one testcase and multiple violations of DRY.
> > > >
> > > > This also leads me on to my other concern, and that is the
practice
> > > of
> > > > testing for the presence of markup in a view using
assert_tag. I
> > > can''t
> > > > really see the value in this, and again it seems to violate
DRY to
> > > me. Why
> > > > would you want to assert that a tag exists in most cases?
For
> > > example, given
> > > > the ArticlePostTest example above, which wouldn''t
really do much in
> > > the
> > > > controller, and simply pass to the view for the rendering of
the
> > > form. Why
> > > > would I want to test for the structure of the form?
Ultimately my
> > > form will
> > > > be laid out using CSS but there are still several ways I
could
> > > markup my
> > > > form, with all variations being "correct". By
testing for one
> > > particular
> > > > implementation in my test, I''m duplicating the
markup in my test and
> > > my
> > > > view, and binding the test to my implementation of the form.
Testing
> > > that a
> > > > particular HTML element is present in my view does nothing
to tell
> > > me that
> > > > my view is "correct", therefore I think this makes
the test
> > > worthless.
> > > >
> > > > Which leads me on to saying that views such as forms are the
realm
> > > for
> > > > proper functional testing using something like Watir or
Selenium,
> > > not
> > > > TestUnit and tag assertions. You want to check that you can
fill in
> > > the form
> > > > and submit it, not that the page contains a <form>
tag.
> > > >
> > > > So what would I really like to see? No separation of unit
and
> > > functional
> > > > testing in terms of model and controller code. They should
all be
> > > covered by
> > > > unit tests. Functional testing should be the realm of
something like
> > > Watir
> > > > or Selenium, unless somebody can put together a good library
that
> > > lets you
> > > > do this kind of functional testing ( i.e., filling in forms,
> > > interacting
> > > > with your web application as if it were in a browser) from
within
> > > your
> > > > TestUnit tests. I''d like to see less emphasis on
the idea of testing
> > > your
> > > > controllers as a whole and testing your different actions as
> > > smaller,
> > > > focussed unit tests. I think this would also aid with TDD.
> > > >
> > > > Dave Astels'' has some good reading material here:
> > > >
> > > > http://blog.daveastels.com/?p=53
> > > >
> > > > What do you think?
> > > >
> > > > Cheers
> > > > Luke
> > > >
> > > > _______________________________________________
> > > > Rails mailing list
> > > > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> > > > http://lists.rubyonrails.org/mailman/listinfo/rails
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Chris Hartjes
> > >
> > > @TheBallpark - http://www.littlehart.net/attheballpark
> > > @TheKeyboard - http://www.littlehart.net/atthekeyboard
> > > _______________________________________________
> > > Rails mailing list
> > > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> > > http://lists.rubyonrails.org/mailman/listinfo/rails
> > >
> >
> >
>

_______________________________________________
Rails mailing list
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
http://lists.rubyonrails.org/mailman/listinfo/rails

maybe testing should be broken into the same levels as the app, M,V,C.. 

tests/models/user/*_test.rb
...

test/controllers/user/view_test.rb
test/controllers/user/add_test.rb
..

test/view/user/view_test.rb
...


where view testing would be something more like what you are suggesting,
using FIT/Watir/Selenium etc... model tests would be what
''unit''t tests
currently are, and ''controller'' tests would be current
functional tests

I agree with the idea of having more, better targeted test suites as
well..

rather than having one test with a bunch of assertions for adding a
user, you could have controllers/user/add_test.rb with a bunch of tests
with single or very few, very specific, tests

just brainstorming here, I suspose it wouldn''t be hard to implement
this
(or whatever else you want) in the current rails framework.. but I agree
that the topic is worthy of discussion





On Wed, 2005-11-16 at 17:22 +0000, Luke Redpath wrote:
> So, further to that, I think it would be great if Rails removed this
> distinction as it currently stands, and introduced a means of writing
> proper functional/acceptance tests, ideally using something like
> FIT/Watir/Selenium, that can be defined early and made to pass through
> the development of working code (covered by unit tests, written test
> first). 
> 
> Of course, I''m very much aware of the whole Rails being
opinionated. I
> just think this is an instance where Rails has got it wrong and am
> voicing my opinion and trying to provoke discussion. I''m not
trying to
> start a flame war. I''m well aware I can implement my own ideas in
my
> own code and intend on doing so. But I do think Rails'' take on
things
> muddies the waters a bit. 
> 
> Cheers
> Luke
> 
> On 11/16/05, Luke Redpath
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org> wrote:
>         Another difference this brings to mind is that functional
>         tests (or Acceptance Tests as XP calls them) should not be
>         written test-first, but are instead defined at the beginning
>         of an iteration for a series of user stories. Acceptance tests
>         do not have to run at 100% all the time (in fact they start at
>         0% and approach 100% as the iteration nears completion),
>         wherease Unit Tests should always be running at 100%.
>         Acceptance tests represent what the completed system should do
>         when finished, whereas Unit Tests represent what your code
>         actually does. 
>         
>         Its perfectly possible to design controller actions
>         test-first, but this is still unit testing. Acceptance testing
>         an action would be to load it in your browser, fill in the
>         form, submit it, and check it worked. This is why I mention
>         things like Watir or Selenium for acceptance testing over
>         TestUnit. FIT would be another option. 
>         
>         This is just more evidence to me, that Rails'' distinction
of
>         unit tests for models and functional tests for controllers, is
>         wrong.
>         
>         Cheers
>         Luke
>         
>         On 11/16/05, Luke Redpath
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org> wrote:
>                 > At the risk of starting a flame war, I see no
>                 difference between
>                 > testing the controller in one test or testing every
>                 element of the
>                 > controller in seperate tests.  It sounds like
>                 semantics to me. 
>                 
>                 Well to me, thats like saying "there is no difference
>                 between having everything wrapped up in one God class
>                 or separated into smaller, more specific fine-grained
>                 classes". In fact, why have separate test cases for
>                 each controller, why not one test case for your whole
>                 app? ;) 
>                 
>                 I think tests should be given as much as care and
>                 consideration as your production code. If they start
>                 getting bloated they should be refactored, like your
>                 production code. Smelly tests become harder and harder
>                 to work with and IMO this will start to have a
>                 detrimental effect on your testing. 
>                 
>                 Cheers
>                 Luke
>                 
>                 
>                 On 11/16/05, Chris Hartjes
<chartjes-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>                         At the risk of starting a flame war, I see no
>                         difference between
>                         testing the controller in one test or testing
>                         every element of the
>                         controller in seperate tests.  It sounds like
>                         semantics to me.
>                         
>                         The important thing is that there is actual
>                         testing going on. 
>                         Everything after that is gravy. :)
>                         
>                         On 11/16/05, Luke Redpath
>                        
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org> wrote:
>                         > Sorry if this comes across as a bunch of
>                         disjointed thoughts...I''m just 
>                         > trying to put my thoughts down and
I''d like
>                         to know what other people''s
>                         > opinions on the subject are.
>                         >
>                         > I''ve been working with Rails for 3 or
4
>                         months now and I''m constantly trying
>                         > to look at ways of improving my testing
>                         techniques, especially when it comes 
>                         > to TDD. I have a few concerns about the way
>                         Rails approaches tests with
>                         > regards to controllers/models.
>                         >
>                         > Rails has always seemed to apply that you
>                         unit test your models and
>                         > functional test your controllers. For me
>                         however, this distinction breaks 
>                         > down in practice and leads to what are in my
>                         opinion very smelly tests.
>                         >
>                         > Firstly, I don''t like the way that by
>                         default Rails assumes you want to test
>                         > an entire controller in a testcase.
I''ve
>                         found that this leads to tests that 
>                         > are far to big. I believe better breakdown
>                         would be to have a test case for
>                         > each action, or even a testcase for each
>                         action in different contexts. So,
>                         > as an example, say you had an
>                         ArticleController with view, post and create 
>                         > actions. The post action simply displays the
>                         form which submits to the
>                         > create action.
>                         >
>                         > Now, instead of having one large
>                         ArticleControllerTest, it would make far
>                         > more sense to break it down into
>                         ArticleViewTest, ArticlePostTest, 
>                         > FailedArticleCreate test and
>                         SuccessfulArticleCreate, because each of those
>                         > represent different contexts, or fixtures.
>                         >
>                         > I''m quite keen on Ben
Griffiths'' testing
>                         approach outlined here and have 
>                         > also been looking into Dave Astels''
RSpec
>                         and have to agree with the idea of
>                         > testcases being bound by fixtures. If you
>                         have test methods that require
>                         > common setups, the code should be extracted
>                         to the setup method. If you have 
>                         > different groups of methods using two
>                         different types of setups, this is an
>                         > indication that you should be creating a new
>                         testcase for the new fixture.
>                         > Overall, I''ve found that trying to
test a
>                         whole controller leads to multiple 
>                         > fixtures in one testcase and multiple
>                         violations of DRY.
>                         >
>                         > This also leads me on to my other concern,
>                         and that is the practice of
>                         > testing for the presence of markup in a view
>                         using assert_tag. I can''t 
>                         > really see the value in this, and again it
>                         seems to violate DRY to me. Why
>                         > would you want to assert that a tag exists
>                         in most cases? For example, given
>                         > the ArticlePostTest example above, which
>                         wouldn''t really do much in the 
>                         > controller, and simply pass to the view for
>                         the rendering of the form. Why
>                         > would I want to test for the structure of
>                         the form? Ultimately my form will
>                         > be laid out using CSS but there are still
>                         several ways I could markup my 
>                         > form, with all variations being
"correct".
>                         By testing for one particular
>                         > implementation in my test, I''m
duplicating
>                         the markup in my test and my
>                         > view, and binding the test to my
>                         implementation of the form. Testing that a 
>                         > particular HTML element is present in my
>                         view does nothing to tell me that
>                         > my view is "correct", therefore I
think this
>                         makes the test worthless.
>                         >
>                         > Which leads me on to saying that views such
>                         as forms are the realm for 
>                         > proper functional testing using something
>                         like Watir or Selenium, not
>                         > TestUnit and tag assertions. You want to
>                         check that you can fill in the form
>                         > and submit it, not that the page contains a
>                         <form> tag. 
>                         >
>                         > So what would I really like to see? No
>                         separation of unit and functional
>                         > testing in terms of model and controller
>                         code. They should all be covered by
>                         > unit tests. Functional testing should be the
>                         realm of something like Watir 
>                         > or Selenium, unless somebody can put
>                         together a good library that lets you
>                         > do this kind of functional testing ( i.e.,
>                         filling in forms, interacting
>                         > with your web application as if it were in a
>                         browser) from within your 
>                         > TestUnit tests. I''d like to see less
>                         emphasis on the idea of testing your
>                         > controllers as a whole and testing your
>                         different actions as smaller,
>                         > focussed unit tests. I think this would also
>                         aid with TDD. 
>                         >
>                         > Dave Astels'' has some good reading
material
>                         here:
>                         >
>                         > http://blog.daveastels.com/?p=53
>                         >
>                         > What do you think?
>                         >
>                         > Cheers
>                         > Luke
>                         >
>                         >
>                         _______________________________________________
>                         > Rails mailing list
>                         >
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
>                         >
>                         http://lists.rubyonrails.org/mailman/listinfo/rails
>                         >
>                         >
>                         >
>                         
>                         
>                         --
>                         Chris Hartjes
>                         
>                         @TheBallpark -
>                         http://www.littlehart.net/attheballpark
>                         @TheKeyboard -
>                         http://www.littlehart.net/atthekeyboard
>                         _______________________________________________ 
>                         Rails mailing list
>                        
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
>                         http://lists.rubyonrails.org/mailman/listinfo/rails
>                 
>                 
>         
>         
> 
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails

> rather than having one test with a bunch of assertions for adding a
> user, you could have controllers/user/add_test.rb with a bunch of tests
> with single or very few, very specific, tests
wow, I must have been out of it.. what I meant to say is that having
individual test suites for individual functions of your controller, is
better than having one big test suite for everything to do with your
controller. As well breaking controller tests into controller and view
would be nice. 


I''m not sure how my last email came out so incoherent, I better go eat
lunch before my head explodes.

On 16-nov-2005, at 18:22, Luke Redpath wrote:
> So, further to that, I think it would be great if Rails removed  
> this distinction as it currently stands, and introduced a means of  
> writing proper functional/acceptance tests, ideally using something  
> like FIT/Watir/Selenium, that can be defined early and made to pass  
> through the development of working code (covered by unit tests,  
> written test first).
>
> Of course, I''m very much aware of the whole Rails being  
> opinionated. I just think this is an instance where Rails has got  
> it wrong and am voicing my opinion and trying to provoke  
> discussion. I''m not trying to start a flame war. I''m well
aware I
> can implement my own ideas in my own code and intend on doing so.  
> But I do think Rails'' take on things muddies the waters a bit.
I second that opinion as long as you take personal responsibility for  
getting FIT/Watir/Selenium/Whatever to drive At Least One Browser on  
my trusty old PowerBook.

As long as it is not the case (and considering many Rails developers  
are keen on their PowerBooks) your idea wouldn''t find as much support  
as you expect.

-- 
Julian "Julik" Tarkhanov

I agree with some of this.  In my current project, I have my functional
tests split out per action: test/functional/controller_action_test.rb

I''ve done some poking around in the past, and any subdirs of test/unit
and test/functional get pulled in on the test suite, so there''s nothing
stopping you from setting up a test tree that looks like this:
test/functional/[module/]controller/action_test.rb

I also think that the more esoteric uses of assert_tag that I''m ending
up using could be done much better with FIT or something, rather than
having to assert (as generically as possible) that I have a tag with a
certain ID, with X or X+1 child nodes (+1 for optional header row on a
table).  Being able to use browser-based acceptance testing would be
great, but probably a bit more difficult to do (how to handle fixtures?)
in a generic way.

As far as your other points go, I think you might be overreacting a
little.  If you want to split your unit tests into a case by case basis
(ArticleViewTest, ArticlePostTest, FailedArticleCreateTest and
SuccessfulArticleCreateTest) for ease of reading and maintenance, then
by all means do so.  Nothing here stopping you.  All the support you
need as far as the testrunner goes is there.  I don''t know that forcing
it upon everyone would be all that useful.

Merging functional tests into the units hierarchy is a bad idea, even
though as you say they really are unit tests.  Merging your model and
controller tests would be an ease-of-use nightmare.  Even moreso than us
developers writing output tests under functionals.

- Jamie



On Wed, 2005-11-16 at 17:22 +0000, Luke Redpath wrote:
> So, further to that, I think it would be great if Rails removed this
> distinction as it currently stands, and introduced a means of writing
> proper functional/acceptance tests, ideally using something like
> FIT/Watir/Selenium, that can be defined early and made to pass through
> the development of working code (covered by unit tests, written test
> first). 
> 
> Of course, I''m very much aware of the whole Rails being
opinionated. I
> just think this is an instance where Rails has got it wrong and am
> voicing my opinion and trying to provoke discussion. I''m not
trying to
> start a flame war. I''m well aware I can implement my own ideas in
my
> own code and intend on doing so. But I do think Rails'' take on
things
> muddies the waters a bit. 
> 
> Cheers
> Luke
> 
> On 11/16/05, Luke Redpath
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org> wrote:
>         Another difference this brings to mind is that functional
>         tests (or Acceptance Tests as XP calls them) should not be
>         written test-first, but are instead defined at the beginning
>         of an iteration for a series of user stories. Acceptance tests
>         do not have to run at 100% all the time (in fact they start at
>         0% and approach 100% as the iteration nears completion),
>         wherease Unit Tests should always be running at 100%.
>         Acceptance tests represent what the completed system should do
>         when finished, whereas Unit Tests represent what your code
>         actually does. 
>         
>         Its perfectly possible to design controller actions
>         test-first, but this is still unit testing. Acceptance testing
>         an action would be to load it in your browser, fill in the
>         form, submit it, and check it worked. This is why I mention
>         things like Watir or Selenium for acceptance testing over
>         TestUnit. FIT would be another option. 
>         
>         This is just more evidence to me, that Rails'' distinction
of
>         unit tests for models and functional tests for controllers, is
>         wrong.
>         
>         Cheers
>         Luke
>         
>         On 11/16/05, Luke Redpath
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org> wrote:
>                 > At the risk of starting a flame war, I see no
>                 difference between
>                 > testing the controller in one test or testing every
>                 element of the
>                 > controller in seperate tests.  It sounds like
>                 semantics to me. 
>                 
>                 Well to me, thats like saying "there is no difference
>                 between having everything wrapped up in one God class
>                 or separated into smaller, more specific fine-grained
>                 classes". In fact, why have separate test cases for
>                 each controller, why not one test case for your whole
>                 app? ;) 
>                 
>                 I think tests should be given as much as care and
>                 consideration as your production code. If they start
>                 getting bloated they should be refactored, like your
>                 production code. Smelly tests become harder and harder
>                 to work with and IMO this will start to have a
>                 detrimental effect on your testing. 
>                 
>                 Cheers
>                 Luke
>                 
>                 
>                 On 11/16/05, Chris Hartjes
<chartjes-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>                         At the risk of starting a flame war, I see no
>                         difference between
>                         testing the controller in one test or testing
>                         every element of the
>                         controller in seperate tests.  It sounds like
>                         semantics to me.
>                         
>                         The important thing is that there is actual
>                         testing going on. 
>                         Everything after that is gravy. :)
>                         
>                         On 11/16/05, Luke Redpath
>                        
<contact-5bSlq4ThgqCxcuGT1ypO+1pr/1R2p/CL@public.gmane.org> wrote:
>                         > Sorry if this comes across as a bunch of
>                         disjointed thoughts...I''m just 
>                         > trying to put my thoughts down and
I''d like
>                         to know what other people''s
>                         > opinions on the subject are.
>                         >
>                         > I''ve been working with Rails for 3 or
4
>                         months now and I''m constantly trying
>                         > to look at ways of improving my testing
>                         techniques, especially when it comes 
>                         > to TDD. I have a few concerns about the way
>                         Rails approaches tests with
>                         > regards to controllers/models.
>                         >
>                         > Rails has always seemed to apply that you
>                         unit test your models and
>                         > functional test your controllers. For me
>                         however, this distinction breaks 
>                         > down in practice and leads to what are in my
>                         opinion very smelly tests.
>                         >
>                         > Firstly, I don''t like the way that by
>                         default Rails assumes you want to test
>                         > an entire controller in a testcase.
I''ve
>                         found that this leads to tests that 
>                         > are far to big. I believe better breakdown
>                         would be to have a test case for
>                         > each action, or even a testcase for each
>                         action in different contexts. So,
>                         > as an example, say you had an
>                         ArticleController with view, post and create 
>                         > actions. The post action simply displays the
>                         form which submits to the
>                         > create action.
>                         >
>                         > Now, instead of having one large
>                         ArticleControllerTest, it would make far
>                         > more sense to break it down into
>                         ArticleViewTest, ArticlePostTest, 
>                         > FailedArticleCreate test and
>                         SuccessfulArticleCreate, because each of those
>                         > represent different contexts, or fixtures.
>                         >
>                         > I''m quite keen on Ben
Griffiths'' testing
>                         approach outlined here and have 
>                         > also been looking into Dave Astels''
RSpec
>                         and have to agree with the idea of
>                         > testcases being bound by fixtures. If you
>                         have test methods that require
>                         > common setups, the code should be extracted
>                         to the setup method. If you have 
>                         > different groups of methods using two
>                         different types of setups, this is an
>                         > indication that you should be creating a new
>                         testcase for the new fixture.
>                         > Overall, I''ve found that trying to
test a
>                         whole controller leads to multiple 
>                         > fixtures in one testcase and multiple
>                         violations of DRY.
>                         >
>                         > This also leads me on to my other concern,
>                         and that is the practice of
>                         > testing for the presence of markup in a view
>                         using assert_tag. I can''t 
>                         > really see the value in this, and again it
>                         seems to violate DRY to me. Why
>                         > would you want to assert that a tag exists
>                         in most cases? For example, given
>                         > the ArticlePostTest example above, which
>                         wouldn''t really do much in the 
>                         > controller, and simply pass to the view for
>                         the rendering of the form. Why
>                         > would I want to test for the structure of
>                         the form? Ultimately my form will
>                         > be laid out using CSS but there are still
>                         several ways I could markup my 
>                         > form, with all variations being
"correct".
>                         By testing for one particular
>                         > implementation in my test, I''m
duplicating
>                         the markup in my test and my
>                         > view, and binding the test to my
>                         implementation of the form. Testing that a 
>                         > particular HTML element is present in my
>                         view does nothing to tell me that
>                         > my view is "correct", therefore I
think this
>                         makes the test worthless.
>                         >
>                         > Which leads me on to saying that views such
>                         as forms are the realm for 
>                         > proper functional testing using something
>                         like Watir or Selenium, not
>                         > TestUnit and tag assertions. You want to
>                         check that you can fill in the form
>                         > and submit it, not that the page contains a
>                         <form> tag. 
>                         >
>                         > So what would I really like to see? No
>                         separation of unit and functional
>                         > testing in terms of model and controller
>                         code. They should all be covered by
>                         > unit tests. Functional testing should be the
>                         realm of something like Watir 
>                         > or Selenium, unless somebody can put
>                         together a good library that lets you
>                         > do this kind of functional testing ( i.e.,
>                         filling in forms, interacting
>                         > with your web application as if it were in a
>                         browser) from within your 
>                         > TestUnit tests. I''d like to see less
>                         emphasis on the idea of testing your
>                         > controllers as a whole and testing your
>                         different actions as smaller,
>                         > focussed unit tests. I think this would also
>                         aid with TDD. 
>                         >
>                         > Dave Astels'' has some good reading
material
>                         here:
>                         >
>                         > http://blog.daveastels.com/?p=53
>                         >
>                         > What do you think?
>                         >
>                         > Cheers
>                         > Luke
>                         >
>                         >
>                         _______________________________________________
>                         > Rails mailing list
>                         >
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
>                         >
>                         http://lists.rubyonrails.org/mailman/listinfo/rails
>                         >
>                         >
>                         >
>                         
>                         
>                         --
>                         Chris Hartjes
>                         
>                         @TheBallpark -
>                         http://www.littlehart.net/attheballpark
>                         @TheKeyboard -
>                         http://www.littlehart.net/atthekeyboard
>                         _______________________________________________ 
>                         Rails mailing list
>                        
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
>                         http://lists.rubyonrails.org/mailman/listinfo/rails
>                 
>                 
>         
>         
> 
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails

Well, I know Selenium works in quite a few browsers and the next version of
Watir is meant to have support for more than just IE.

Also, I think some of you may have misunderstood what I meant with regards
to removing the distinction between functional testing controllers or unit
testing models. I was suggesting that they should both be considered unit
tests (something the rake tasks test_unit and test_functional doesn''t
seem
to imply) but I wasn''t suggesting merging the file structure - that is
fine
as it is. I also think it would be a bit OTT to create a new file for every
single test case. The examples I suggested would make good test cases but
all of your test cases for a particular controller would also make a very
logical test suite and could probably still remain in the same file, if you
want.

Cheers
Luke

On 11/16/05, Julian ''Julik'' Tarkhanov
<listbox-RY+snkucC20@public.gmane.org> wrote:
>
>
> On 16-nov-2005, at 18:22, Luke Redpath wrote:
>
> > So, further to that, I think it would be great if Rails removed
> > this distinction as it currently stands, and introduced a means of
> > writing proper functional/acceptance tests, ideally using something
> > like FIT/Watir/Selenium, that can be defined early and made to pass
> > through the development of working code (covered by unit tests,
> > written test first).
> >
> > Of course, I''m very much aware of the whole Rails being
> > opinionated. I just think this is an instance where Rails has got
> > it wrong and am voicing my opinion and trying to provoke
> > discussion. I''m not trying to start a flame war. I''m
well aware I
> > can implement my own ideas in my own code and intend on doing so.
> > But I do think Rails'' take on things muddies the waters a
bit.
>
> I second that opinion as long as you take personal responsibility for
> getting FIT/Watir/Selenium/Whatever to drive At Least One Browser on
> my trusty old PowerBook.
>
> As long as it is not the case (and considering many Rails developers
> are keen on their PowerBooks) your idea wouldn''t find as much
support
> as you expect.
>
> --
> Julian "Julik" Tarkhanov
>
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>

_______________________________________________
Rails mailing list
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
http://lists.rubyonrails.org/mailman/listinfo/rails

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 16, 2005, at 3:54 PM, Luke Redpath wrote:
> Well, I know Selenium works in quite a few browsers and the next  
> version of Watir is meant to have support for more than just IE.
>
> Also, I think some of you may have misunderstood what I meant with  
> regards to removing the distinction between functional testing  
> controllers or unit testing models. I was suggesting that they  
> should both be considered unit tests (something the rake tasks  
> test_unit and test_functional doesn''t seem to imply) but I
wasn''t
> suggesting merging the file structure - that is fine as it is. I  
> also think it would be a bit OTT to create a new file for every  
> single test case. The examples I suggested would make good test  
> cases but all of your test cases for a particular controller would  
> also make a very logical test suite and could probably still remain  
> in the same file, if you want.
Luke, from what I can tell, you''re advocating a change in semantics.

The approach to testing you suggest is achievable already, but is not  
the default.

It''s a matter of patching the app generator.  Interested?

Best,
jeremy
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (Darwin)

iD8DBQFDe8juAQHALep9HFYRAjouAJwLq2jobC/QEDdRiEmUF5qtlsnQRwCdFFgp
XvoO/gZucqCelSGqUYxIfO4=ceHf
-----END PGP SIGNATURE-----

I would be interested. I still have to think about this some more though. As
I said, I''m just throwing thoughts out there for now.

A change in semantics is certainly part of what I''m advocating. In
fact, I''d
love to see support for RSpec down the line once the code base has matured
and stabilised a bit - at the moment its not that different to Test::Unit
but I believe the future plan is to evolve it into a complete DSL in its own
right.

Cheers
Luke

On 11/17/05, Jeremy Kemper
<jeremy-w7CzD/W5Ocjk1uMJSBkQmQ@public.gmane.org>
wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Nov 16, 2005, at 3:54 PM, Luke Redpath wrote:
> > Well, I know Selenium works in quite a few browsers and the next
> > version of Watir is meant to have support for more than just IE.
> >
> > Also, I think some of you may have misunderstood what I meant with
> > regards to removing the distinction between functional testing
> > controllers or unit testing models. I was suggesting that they
> > should both be considered unit tests (something the rake tasks
> > test_unit and test_functional doesn''t seem to imply) but I
wasn''t
> > suggesting merging the file structure - that is fine as it is. I
> > also think it would be a bit OTT to create a new file for every
> > single test case. The examples I suggested would make good test
> > cases but all of your test cases for a particular controller would
> > also make a very logical test suite and could probably still remain
> > in the same file, if you want.
>
> Luke, from what I can tell, you''re advocating a change in
semantics.
>
> The approach to testing you suggest is achievable already, but is not
> the default.
>
> It''s a matter of patching the app generator. Interested?
>
> Best,
> jeremy
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (Darwin)
>
> iD8DBQFDe8juAQHALep9HFYRAjouAJwLq2jobC/QEDdRiEmUF5qtlsnQRwCdFFgp
> XvoO/gZucqCelSGqUYxIfO4> =ceHf
> -----END PGP SIGNATURE-----
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>

_______________________________________________
Rails mailing list
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
http://lists.rubyonrails.org/mailman/listinfo/rails

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 16, 2005, at 4:09 PM, Luke Redpath wrote:
> I would be interested. I still have to think about this some more  
> though. As I said, I''m just throwing thoughts out there for now.
>
> A change in semantics is certainly part of what I''m advocating. In
> fact, I''d love to see support for RSpec down the line once the
code
> base has matured and stabilised a bit - at the moment its not that  
> different to Test::Unit but I believe the future plan is to evolve  
> it into a complete DSL in its own right.
Getting Rails on the leading edge with RSpec would be sweet.

It''ll take a bit of work to go through the additional assertions  
Rails provides for Test::Unit::TestCase and RSpecify them.

Do it, man :-)

jeremy
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (Darwin)

iD8DBQFDe8vGAQHALep9HFYRAvItAKCOl6YtPvrole0uCAOV6R0UDtSRDQCeNaf0
n1t6KvP21r3ZQI+eoz2Qs4w=6oS6
-----END PGP SIGNATURE-----

Hi !

2005/11/16, Jeremy Kemper <jeremy@bitsweat.net>:
> Getting Rails on the leading edge with RSpec would be sweet.
>
> It'll take a bit of work to go through the additional assertions
> Rails provides for Test::Unit::TestCase and RSpecify them.
>
> Do it, man :-)
After reading about rspec here yesterday, I played around with it. 
Works like a charm.  I'm not using it under Rails, but I liked what I
saw.  I already patched a few things in RSpec.

I am developing a plugin using RSpec instead of Test::Unit.  It's an
HABTM helper.

More info on my blog later today.

Bye !
--
François Beausoleil
http://blog.teksol.info/

_______________________________________________
Rails mailing list
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
http://lists.rubyonrails.org/mailman/listinfo/rails

Yeah right,
2005/11/17, Francois Beausoleil
<francois.beausoleil@gmail.com>:
> After reading about rspec here yesterday, I played around with it.
> Works like a charm.  I'm not using it under Rails, but I liked what I
> saw.  I already patched a few things in RSpec.
What I REALLY meant to say is that I'll probably start RSpecing the
Rails assertions.

Bye !
--
François Beausoleil
http://blog.teksol.info/

_______________________________________________
Rails mailing list
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
http://lists.rubyonrails.org/mailman/listinfo/rails

http://dev.rubyonrails.org/ticket/2923

I''ve submited a research ticket and patch regarding this.. 

The attached patch will allow for 

test/models/
test/views/
test/controllers/

as well as maintaining compatibility with the current
test/unit/
test/functional/

While it may not be neccesary since test/unit and /functional still
work, i modified the default generators to use the new m/v/c test
layout... 

but anyway, i put it as RESEARCH in the hopes that others will poke at
it, maybe come up with a better idea, or additional ideas..


I think that having a separate place for view specific tests could go a
long way in organization, as well as paving the way for a proper,
seperate, view-specific testing infrastructure such as FIT etc. 

However even just the separation itself i think is worth it.

Testing if your controller functions as intended is different than
testing what the user sees, I think we''ve all come to accept that which
is why we are using a MVC framework - it doesn''t make sense to not
follow the same metaphor for tests!


I




On Wed, 2005-11-16 at 16:16 -0800, Jeremy Kemper wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Nov 16, 2005, at 4:09 PM, Luke Redpath wrote:
> > I would be interested. I still have to think about this some more  
> > though. As I said, I''m just throwing thoughts out there for
now.
> >
> > A change in semantics is certainly part of what I''m
advocating. In
> > fact, I''d love to see support for RSpec down the line once
the code
> > base has matured and stabilised a bit - at the moment its not that  
> > different to Test::Unit but I believe the future plan is to evolve  
> > it into a complete DSL in its own right.
> 
> Getting Rails on the leading edge with RSpec would be sweet.
> 
> It''ll take a bit of work to go through the additional assertions  
> Rails provides for Test::Unit::TestCase and RSpecify them.
> 
> Do it, man :-)
> 
> jeremy
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (Darwin)
> 
> iD8DBQFDe8vGAQHALep9HFYRAvItAKCOl6YtPvrole0uCAOV6R0UDtSRDQCeNaf0
> n1t6KvP21r3ZQI+eoz2Qs4w> =6oS6
> -----END PGP SIGNATURE-----
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 17, 2005, at 3:20 PM, Jerrett Taylor wrote:
> I think that having a separate place for view specific tests could  
> go a
> long way in organization, as well as paving the way for a proper,
> seperate, view-specific testing infrastructure such as FIT etc.
>
> However even just the separation itself i think is worth it.
>
> Testing if your controller functions as intended is different than
> testing what the user sees, I think we''ve all come to accept that
> which
> is why we are using a MVC framework - it doesn''t make sense to not
> follow the same metaphor for tests!
My hunch is that splitting controller and view testing will create a  
lot of busywork without increasing the value of the tests.

Testing action bar on controller foo involves running through each  
story for that action and verifying that it gives the expected results.

Testing view bar for controller foo involves.. running through each  
story for that view and verifying that is gives the expected results.

So their usage is largely one-to-one, though there are other cases  
such as shared partials and layouts.

I think clean, expressive tests are great, but if divorcing the  
controller from its views makes testing more of a chore then I''m  
happy to lose the benefit that this restructuring provides.

Best,
jeremy
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (Darwin)

iD8DBQFDfRcVAQHALep9HFYRAjE9AJsGN0hKSmZUyKAnk+OjK5zAfTfwMACdH9EM
0HPaGtULMon1mF56z2p+JK8=Y7Cq
-----END PGP SIGNATURE-----

> My hunch is that splitting controller and view testing will create a  
> lot of busywork without increasing the value of the tests.
> 
> Testing action bar on controller foo involves running through each  
> story for that action and verifying that it gives the expected results.
> 
> Testing view bar for controller foo involves.. running through each  
> story for that view and verifying that is gives the expected results.

Currently that should probably be two separate tests anyway. If you are
testing to ensure data is created after a successful action, you don''t
want the test to fail because there was a missing tag in your view.
You''d want a different test to fail, and your test that checks for the
data to be created to pass, since the data was in fact created.

Currently this means writing two tests in the functional tests, one
dealing with the expected behaviour of the app, and one dealing with the
expected output of the app. To me, this makes sense to break them apart.
Also that means when you are working on views, you can TDD (BDD,
whatever) your views without having to mess with (and run all of) your
controller tests, by running rake test_views

If you have people on your team that deal primarily with creating the
front end, they can still be given a place to do their testing without
cluttering the controller tests.


> So their usage is largely one-to-one, though there are other cases  
> such as shared partials and layouts.
> 
> I think clean, expressive tests are great, but if divorcing the  
> controller from its views makes testing more of a chore then I''m  
> happy to lose the benefit that this restructuring provides.
I think that with this restructuring it doesn''t get in the way of
anything anyone is doing currently. test/functional and test/unit will
continue to work, and with new projects that have the new structured
test directories, somebody can still decide they would rather have fewer
tests with many assertions by doing what they do now in test/functional
in test/controller and only use test/view for things like you mentioned,
shared partials and layouts. 

In short, this change wouldn''t be forcing anything new anyone, it would
simply be providing the structure to split when the developer finds it
appropriate. For people that subscribe to the more tests, less
assertions per test mentality this could be a big help, for people who
don''t they don''t really have to do anything any differently. 


my two (4? 6? 10?) cents

Hi,

I am relatively new to rails so it may be best to take this with a
grain of salt but ...

On 11/18/05, Jeremy Kemper
<jeremy-w7CzD/W5Ocjk1uMJSBkQmQ@public.gmane.org>
wrote:
> My hunch is that splitting controller and view testing will create a
> lot of busywork without increasing the value of the tests.
>
> Testing action bar on controller foo involves running through each
> story for that action and verifying that it gives the expected results.
>
> Testing view bar for controller foo involves.. running through each
> story for that view and verifying that is gives the expected results.
This may be largely true but I guess as a realtive new-comer to rails this
is NOT how I have been intuitively testing my application. The vast majority
of my tests look something like the following

  def test_new_get
    get(:new, {:issue_id => 1}, {"user_id" => 1} )
    assert_response(:success)
    assert_template(''new'')
    assert_equal(1, assigns(:comment).issue_id)
    assert_nil(flash[:notice])
    assert_nil(flash[:alert])
  end

I test the state of the assigns, flash, session and database after the action
has been run. I have yet to test the presentation layer in any way
shape or form.

I had planned to start writing tests in the future for the view using
something like

def test_lists
  items = ....
  category = ...
  render(''lists'',:locals => {:items => items},
                    :instance => {:category => category},
                    :session => {:items => items} )
  assert_tag ....
  assert_tag ....
  assert_tag ....
end

I also assumed that if I ever wrote a web service that they would have
another set of separate tests. So essentially I would have a separate
top-level directory in my test hierarchy for every separate hierarchy in
my app directory except for helpers.

This is just what I assumed from my first touch of rails development.

--
Cheers,

Peter Donald

Great thread.

I might add that since Selenium now has Ruby bindings ( 
http://selenium.thoughtworks.com/driven.html ), and works with 
IE/Firefox/Safari (and hence, on *nix/Windows/Mac), integrating it into 
rails might be a good way to handle things.

I''d propose:

1) Include in the basic rails framework the Selenium files and ruby 
bindings.

2) Rails should apply it''s autogenerate magic to Selenium tests, too. 
Under tests/acceptance, basic stubs for Selenium based tests would 
appear.

3) The basic idiom for these tests wouldn''t be along the lines of unit 
testing, but rather use cases.  So an individual test case would walk 
the virtual user through a sequence of actions (click here, fill out 
form, hit back, change form, etc.), and verify the results.

4) Since the whole thing is being done integrated and via Ruby, we have 
access to logging, fixtures, models, etc., should we need them.  (For 
instance, to verify that an order was purchased, we could check the 
confirmation page in the virtual browser, but also check that the DB 
shows the new order.)


-- 
Posted via http://www.ruby-forum.com/.