Rails - Apr 2005 - Selenium integration into Rails, request for feedback

Selenium is an open-source functional testing tool that supports
multiple browsers and multiple operative systems.

I''m working on getting Selenium nicely integrated into Rails.

I''ve created a wiki page outlining how I feel Selenium could be
integrated into Rails:
http://wiki.rubyonrails.com/rails/show/SeleniumIntegration

Any feedback is greatly appreciated.

I don''t have any feedback for you yet but I''ve only recently
looked into
Selenium, so I haven''t actually gone around to writing an actual tests
but it definitely looks interesting, I''ll be keeping on eye on the wiki
page for updates.

Would it be a good or bad idea to scrap controller acceptance tests in
favor or just writing Selenium tests?

Jon Tirsen wrote:
>Selenium is an open-source functional testing tool that supports
>multiple browsers and multiple operative systems.
>
>I''m working on getting Selenium nicely integrated into Rails.
>
>I''ve created a wiki page outlining how I feel Selenium could be
>integrated into Rails:
>http://wiki.rubyonrails.com/rails/show/SeleniumIntegration
>
>Any feedback is greatly appreciated.
>_______________________________________________
>Rails mailing list
>Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
>http://lists.rubyonrails.org/mailman/listinfo/rails
>  
>

Francisco Hernandez wrote:
>Would it be a good or bad idea to scrap controller acceptance tests in
>favor or just writing Selenium tests?
>  
>
It would be a bad idea, Selenium is good for customer-facing tests, but 
too slow for TDD.

-- 
Best regards,

Alexey Verkhovsky

Ruby Forum: http://ruby-forum.org        (moderator)
RForum:     http://rforum.andreas-s.net  (co-author)
Instiki:    http://instiki.org           (maintainer)

On 4/29/05, Alexey Verkhovsky
<alex-vV7tgcE2N9Nhl2p70BpVqQ@public.gmane.org>
wrote:
> Francisco Hernandez wrote:
> 
> >Would it be a good or bad idea to scrap controller acceptance tests in
> >favor or just writing Selenium tests?
> >
> >
> It would be a bad idea, Selenium is good for customer-facing tests, but
> too slow for TDD.
Yes, I agree. Unit integration tests (model and controller tests) are
good for testing intricate and complex things where as system
functional tests (selenium tests) are good for testing broad sweeping
things and to ensure that the whole system hangs together. They have
different purposes and a good software development practice employs
both.

That said I think given what project you are doing it might make sense
to choose just one of these. If you have a simpler app without very
complex domain logic I think system functional tests (selenium) will
provide very good value. A system-level test provides a lot of
protection from regressions so I prefer to at least do these. A
unit-level test allows you to do TDD and are usually very fast.

Some Java projects also do unit isolation tests (based on mock
objects) but I doubt these have much value in a Ruby project. We did
it a lot early with DamageControl, but they provided very little
value.

What is it about Ruby projects (vs Java projects) that you makes you  
doubt the value of using Mocks?

Thats one of the things I noticed about how Rails apps are being  
tested, they use the DB on both unit and acceptance tests, in my Java  
apps I only use the DB on my DAO level tests and mocked out the DAOs  
when testing Services, Controllers would also use mocked Services

On Apr 28, 2005, at 5:52 PM, Jon Tirsen wrote:
> On 4/29/05, Alexey Verkhovsky
<alex-vV7tgcE2N9Nhl2p70BpVqQ@public.gmane.org> wrote:
>
>> Francisco Hernandez wrote:
>>
>>
>>> Would it be a good or bad idea to scrap controller acceptance  
>>> tests in
>>> favor or just writing Selenium tests?
>>>
>>>
>>>
>> It would be a bad idea, Selenium is good for customer-facing  
>> tests, but
>> too slow for TDD.
>>
>
> Yes, I agree. Unit integration tests (model and controller tests) are
> good for testing intricate and complex things where as system
> functional tests (selenium tests) are good for testing broad sweeping
> things and to ensure that the whole system hangs together. They have
> different purposes and a good software development practice employs
> both.
>
> That said I think given what project you are doing it might make sense
> to choose just one of these. If you have a simpler app without very
> complex domain logic I think system functional tests (selenium) will
> provide very good value. A system-level test provides a lot of
> protection from regressions so I prefer to at least do these. A
> unit-level test allows you to do TDD and are usually very fast.
>
> Some Java projects also do unit isolation tests (based on mock
> objects) but I doubt these have much value in a Ruby project. We did
> it a lot early with DamageControl, but they provided very little
> value.
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>

This is completely off-topic but I''ll make a quick response:

Java enforces strict typing compile time. That means that if you
define an interface and have a component that implements the interface
the compiler will ensure that this is the case, furthermore a
component that depends on that interface is ensured by the compiler to
use it correctly.

When you are using mock objects you are making assertions on how the
unit under test interacts with the interfaces of the components it
depends on. The compiler then enforces the implementation of those
interfaces to comply with these interfaces. It can only do so in a
statical way (it can check very little in terms of dynamics), but at
least it can do some checking.

Ruby doesn''t have compile time type checking so there is no way you
can ensure that system hangs together at all. The only way you can
ensure this by doing various forms of integration tests. Thus, unit
isolation tests (mock objects based tests) have some value but it''s
vastly diminished.

On 4/29/05, Francisco Hernandez
<lagcisco-b7MHZcQsHeJWk0Htik3J/w@public.gmane.org>
wrote:
> What is it about Ruby projects (vs Java projects) that you makes you
> doubt the value of using Mocks?
> 
> Thats one of the things I noticed about how Rails apps are being
> tested, they use the DB on both unit and acceptance tests, in my Java
> apps I only use the DB on my DAO level tests and mocked out the DAOs
> when testing Services, Controllers would also use mocked Services
> 
> On Apr 28, 2005, at 5:52 PM, Jon Tirsen wrote:
> 
> > On 4/29/05, Alexey Verkhovsky
<alex-vV7tgcE2N9Nhl2p70BpVqQ@public.gmane.org> wrote:
> >
> >> Francisco Hernandez wrote:
> >>
> >>
> >>> Would it be a good or bad idea to scrap controller acceptance
> >>> tests in
> >>> favor or just writing Selenium tests?
> >>>
> >>>
> >>>
> >> It would be a bad idea, Selenium is good for customer-facing
> >> tests, but
> >> too slow for TDD.
> >>
> >
> > Yes, I agree. Unit integration tests (model and controller tests) are
> > good for testing intricate and complex things where as system
> > functional tests (selenium tests) are good for testing broad sweeping
> > things and to ensure that the whole system hangs together. They have
> > different purposes and a good software development practice employs
> > both.
> >
> > That said I think given what project you are doing it might make sense
> > to choose just one of these. If you have a simpler app without very
> > complex domain logic I think system functional tests (selenium) will
> > provide very good value. A system-level test provides a lot of
> > protection from regressions so I prefer to at least do these. A
> > unit-level test allows you to do TDD and are usually very fast.
> >
> > Some Java projects also do unit isolation tests (based on mock
> > objects) but I doubt these have much value in a Ruby project. We did
> > it a lot early with DamageControl, but they provided very little
> > value.
> > _______________________________________________
> > Rails mailing list
> > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> > http://lists.rubyonrails.org/mailman/listinfo/rails
> >
> 
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>

Jon Tirsen <jon.tirsen-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
> Ruby doesn''t have compile time type checking so there is no way
you
> can ensure that system hangs together at all. The only way you can
> ensure this by doing various forms of integration tests. Thus, unit
> isolation tests (mock objects based tests) have some value but
it''s
> vastly diminished.
I am new to Ruby, but have done a lot of TDD in Java and a lot of work
in other dynamic languages. It seems to me that the lack of
compile-time type-checking *increases* rather than reduces the need
for unit testing and mock objects; you need additional tests to
convince yourself that the object obeys the interface.

Jake

--
I would prefer to be spared the overpowering number of particulars.