Rails - Feb 2006 - Selenium. Replacement for traditional rails functional test?

Hi list

I''ve got a bit of a philosophical question. After just having
discovered
Selenium /Selenium IDE and the rails selenium plugin I''ve realized that
it''s a whole lot easier to do functinoal testing than writing the 
traditional rails functional tests.

However it implies that the development database consists of 
standardized test data (e.g the test & dev database should have the same 
data). This could be achieved via the ar_fixture''s extension to AR.

Is this how people are using selenium?

Also, are there any advantages to the traditional functional tests that 
selenium doesn''t already have (Apart from being able to call the tests 
via rake)?

Thanks & regards,

Saimon

-- 
Posted via http://www.ruby-forum.com/.

I was thinking about that same question recently.

I decided to slim down my functional tests to only test the logic of the 
controller. If an action is supposed to update/save/add records, I will 
have a functional test to ensure that the action works correctly, and 
that records are saved/updated in the database. A Selenium test will 
check that the correct parts of the page are updated and/or the user is 
redirected appropriately.

Basic idea: no assert_tag calls in functional tests! Only test the 
business logic of each action.

BTW: I have got my own Selenium testing plugin, which lets you write 
Selenium tests in the same way you write functional/unit tests. It''s a 
bit rough, but check out 
http://blog.viney.net.nz/articles/2006/02/09/selenium-testing if you''re
interested. You end up writing tests like:

  class PersonSeleniumTest < Test::Unit::TestCase
    selenium_test
    fixtures :people, :notes

    def test_view
      p = people(:bob)
      open "/person/view/#{p.id}"
      assert_element ''name''
      p.notes.each do |note|
        assert_element "note_#{n.id}"
      end
    end
  end

I like this approach because it gives a consistent way of writing tests, 
the plugin definitely needs tidying up though.

-Jonny.

-- 
Posted via http://www.ruby-forum.com/.

Hi Jonny,

The advantage I see to selenium is that it in combination with the 
selenium ide,
functionality tests are a whole lot faster to write, than the 
traditional tests.

I click on record in the selenium ide, interact with my app to test a 
specific feature (or sequence). When I want to assert a specific element 
is present (or not) I use the xpath checker to get a specific handle to 
the dom element.

At the end of the session, my functional test is ready for me and I''ve 
done very little typing (just a few clicks). I can now modify it if 
needed.

I''d start by generating the test with script/generate selenium 
test_name, open that test in the ide and click away.

The way I see it, testing the controller mostly involves testable 
changes in the UI. Testing the business logic usually occurs in the unit 
tests for me. I use the functional testing to make sure the UI is 
responding the way I expect it to, so why use the traditional functional 
tests for that when you can use selenium.

I can imagine testing things like action mailer with the traditional 
functional tests but little more.

I imagine Selenium to be like my end user, beta testing my app.

or am I missing something? ;)

Jonathan Viney wrote:
> I was thinking about that same question recently.
> 
> I decided to slim down my functional tests to only test the logic of the 
> controller. If an action is supposed to update/save/add records, I will 
> have a functional test to ensure that the action works correctly, and 
> that records are saved/updated in the database. A Selenium test will 
> check that the correct parts of the page are updated and/or the user is 
> redirected appropriately.
> 
> Basic idea: no assert_tag calls in functional tests! Only test the 
> business logic of each action.
> 
> BTW: I have got my own Selenium testing plugin, which lets you write 
> Selenium tests in the same way you write functional/unit tests.
It''s a
> bit rough, but check out 
> http://blog.viney.net.nz/articles/2006/02/09/selenium-testing if
you''re
> interested. You end up writing tests like:
> 
>   class PersonSeleniumTest < Test::Unit::TestCase
>     selenium_test
>     fixtures :people, :notes
> 
>     def test_view
>       p = people(:bob)
>       open "/person/view/#{p.id}"
>       assert_element ''name''
>       p.notes.each do |note|
>         assert_element "note_#{n.id}"
>       end
>     end
>   end
> 
> I like this approach because it gives a consistent way of writing tests, 
> the plugin definitely needs tidying up though.
> 
> -Jonny.

-- 
Posted via http://www.ruby-forum.com/.

On Mon, 13 Feb 2006 08:07:58 +0100, Saimon Moore wrote:
> However it implies that the development database consists of 
> standardized test data (e.g the test & dev database should have the
same
> data). This could be achieved via the ar_fixture''s extension to
AR.
Can''t you just keep a second webrick (or, perhaps now, Mongrel!)
running on
the test environment, and connect to that w/Selenium?

Jay

Jay Levitt wrote:
> Can''t you just keep a second webrick (or, perhaps now, Mongrel!)
running
> on
> the test environment, and connect to that w/Selenium?
That''s a possibility but I''m beginning to wonder whether there
is really
a necessity for having the two databases (test & dev).

-- 
Posted via http://www.ruby-forum.com/.

On Feb 13, 2006, at 4:27 AM, Jonathan Viney wrote:
>   class PersonSeleniumTest < Test::Unit::TestCase
>     selenium_test
Yuck! This is what inheritance was made for.

It should be just:

class TestSeleniumPerson < Selenium::TestCase

(Test at the front is the test/unit way.)

It doesn''t look like you can test a single view or partial.  Is this  
true?

-- 
Eric Hodel - drbrain@segment7.net - http://segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

Yes, quite correct! I just found Test::Unit:TestCase to be quite 
uncooperative, and I didn''t want to spend much time with this.

You can test whatever Selenium can test using assertElement, 
assertTextPresent etc...

I also agree with Saimon''s comment about test/dev databases, I barely 
have a distinction as I almost always use my fixtures as development 
data. But I guess it doesn''t hurt to have it there in case anyone did 
feel like using it.

I think that it is still important to test the business logic of each 
action, simply using Selenium does leave you open to miss parts, unless 
you are extremely thorough. I guess if you only have very simple 
business logic then you can possibly get away without functional tests.

I just looked at the Selenium IDE. Very nice. My only reservation is 
that I can''t see a way to link it into AR fixtures, which is important 
for DRY.

-Jonny.

Eric Hodel wrote:
> On Feb 13, 2006, at 4:27 AM, Jonathan Viney wrote:
> 
>>   class PersonSeleniumTest < Test::Unit::TestCase
>>     selenium_test
> 
> Yuck! This is what inheritance was made for.
> 
> It should be just:
> 
> class TestSeleniumPerson < Selenium::TestCase
> 
> (Test at the front is the test/unit way.)
> 
> It doesn''t look like you can test a single view or partial.  Is
this
> true?
-- 
Posted via http://www.ruby-forum.com/.

Hi,

Saimon Moore wrote:
> Also, are there any advantages to the traditional functional tests that 
> selenium doesn''t already have (Apart from being able to call the
tests
> via rake)?
I''ve seen a few approaches, like Jonathan''s, to running
Selenium tests
from rake and am slowingly implementing it in Selenium on Rails as well, 
so I don''t think it needs to be a distinguishing factor.

There are a few things that are hard to do in Selenium tests: 
mocking/faking/stubbing services, making changes directly to the db 
(besides fixtures), checking the db directly etc. Functional tests are 
also bound to run quicker.

Selenium tests the whole stack (the browser is an integral part of a web 
applications). And depending on how the tests are written it should be 
possible for a non-programmer to author/record them.

Functional tests are also normally concerned only with one controller, 
whereas Selenium tests are concerned with the flow of the application.

I guess all normal black-box vs. white-box testing differences apply.

What I''m trying to say is that even if functional and Selenium tests 
overlap to some degree, both are needed and fill different needs.

/Jonas

-- 
Posted via http://www.ruby-forum.com/.

Jonas,

That''s exactly the kind of insight I was looking for.
Thanks, it''s a lot clearer for me now.

Though I''m still tending to think that I should just standardize the 
dev/test data into one db.

Regards,

Saimon

Jonas Bengtsson wrote:
> Hi,
> 
> Saimon Moore wrote:
>> Also, are there any advantages to the traditional functional tests that
>> selenium doesn''t already have (Apart from being able to call
the tests
>> via rake)?
> 
> I''ve seen a few approaches, like Jonathan''s, to running
Selenium tests
> from rake and am slowingly implementing it in Selenium on Rails as well, 
> so I don''t think it needs to be a distinguishing factor.
> 
> There are a few things that are hard to do in Selenium tests: 
> mocking/faking/stubbing services, making changes directly to the db 
> (besides fixtures), checking the db directly etc. Functional tests are 
> also bound to run quicker.
> 
> Selenium tests the whole stack (the browser is an integral part of a web 
> applications). And depending on how the tests are written it should be 
> possible for a non-programmer to author/record them.
> 
> Functional tests are also normally concerned only with one controller, 
> whereas Selenium tests are concerned with the flow of the application.
> 
> I guess all normal black-box vs. white-box testing differences apply.
> 
> What I''m trying to say is that even if functional and Selenium
tests
> overlap to some degree, both are needed and fill different needs.
> 
> /Jonas

-- 
Posted via http://www.ruby-forum.com/.

On another thread, someone commented that they had environments called
"qa" and (something else), in addition to the standard
"development",
"test" and "production".

Having done a lot of performance testing, it''s much easier to run each
stream of testing on a separate, dedicated environment.  That said,
there''s obviously a tradeoff between doing this and winding up with
either lots of excess hardware or scheduling problems.

Back to the original thread about Selenium, there''s a really nice
Selenium IDE for Firefox 1.5 that''s available for download. 
It''s a
Firefox extension, but essentially gives you a
simple-but-highly-useful record/edit/playback mechanism for creating
Selenium test scripts.  Wonderful stuff!

Regards

Dave M.

On 14/02/06, Saimon Moore <saimonmoore@gmail.com>
wrote:
> Jonas,
>
> That''s exactly the kind of insight I was looking for.
> Thanks, it''s a lot clearer for me now.
>
> Though I''m still tending to think that I should just standardize
the
> dev/test data into one db.
>
> Regards,
>
> Saimon
>
> Jonas Bengtsson wrote:
> > Hi,
> >
> > Saimon Moore wrote:
> >> Also, are there any advantages to the traditional functional tests
that
> >> selenium doesn''t already have (Apart from being able to
call the tests
> >> via rake)?
> >
> > I''ve seen a few approaches, like Jonathan''s, to
running Selenium tests
> > from rake and am slowingly implementing it in Selenium on Rails as
well,
> > so I don''t think it needs to be a distinguishing factor.
> >
> > There are a few things that are hard to do in Selenium tests:
> > mocking/faking/stubbing services, making changes directly to the db
> > (besides fixtures), checking the db directly etc. Functional tests are
> > also bound to run quicker.
> >
> > Selenium tests the whole stack (the browser is an integral part of a
web
> > applications). And depending on how the tests are written it should be
> > possible for a non-programmer to author/record them.
> >
> > Functional tests are also normally concerned only with one controller,
> > whereas Selenium tests are concerned with the flow of the application.
> >
> > I guess all normal black-box vs. white-box testing differences apply.
> >
> > What I''m trying to say is that even if functional and
Selenium tests
> > overlap to some degree, both are needed and fill different needs.
> >
> > /Jonas
>
>
> --
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>