Hey all,
I am currently working on coming up w/ an easy to use, developer-centric web
testing framework to test a J2EE app with. (I have 3 rails apps in
production, love rspec, and am currently at a java shop).
I''ve looked at selenium, and it just doesn''t seem like it is
ready for prime
time, and the target audience is developers. So, having said that, does
the rspec community want a "functional" test framework? Are you happy
w/
selenium, and is it working?
Unless someone convinces me of selenium, I''m planning on taking the
basics
and concepts of rspec, and focus it more towards testing web applications,
and am happy to contribute. My company is totally cool with contributing to
open source projects if it makes sense.
Any thoughts or suggestions? Here is a sketch of what I think I''d
like:
server "http://localhost:8080" do
test ''/'' do
page.status.should_be "200"
page.should_have :image, "logo.gif"
page.should_have :string, "Copyright 2006-Present SomeCo, Inc. All
rights reserved."
page.should_have :div, :id => ''copyright''
page.should_include :css, ''default.css''
page.should_have.no_broken_links
page.should_have.at_least.3 :div, :class => "bbxBody"
page.should_have.at_most.6 :div, :class => "someClassName"
# page.[name_of_form]_form.
page.search_form.submit :name => "California" do
result.status.should_be :success
result.should_have :image, "logo.gif"
result.url.should_contain "some_page.html"
end
page.should_have :div, :class => ''something else''
page.should_have :div, :id => ''foo'', :class =>
''bar''
page.should_have ''/div[1]/div[2]/td''
end
test ''/some/url.html?id=35'' do
page.should_redirect :to => ''/foobar.html''
end
end
--
Tim Tischler
ttischler at homeaway.com
tischler at purplecoffeecup.com
512-565-4750
AIM:tjtischler512
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/rspec-users/attachments/20070509/1656c839/attachment.html
"Tim Tischler" <tischler at purplecoffeecup.com> writes:> Hey all, > > I am currently working on coming up w/ an easy to use, developer-centric > web testing framework to test a J2EE app with. (I have 3 rails apps in > production, love rspec, and am currently at a java shop). > > I''ve looked at selenium, and it just doesn''t seem like it is ready for > prime time, and the target audience is developers. So, having said that, > does the rspec community want a "functional" test framework? Are you > happy w/ selenium, and is it working?We have re-written our functional test framework 4 times, and finally settled on Selenium. No other open-source tool gets you the cross-browser, cross-platform support. We use the Ruby bindings for Selenium-RC.> Unless someone convinces me of selenium, I''m planning on taking the basics > and concepts of rspec, and focus it more towards testing web applications, > and am happy to contribute. My company is totally cool with contributing > to open source projects if it makes sense. >Rspec sits at the core of our acceptance testing process. At the beginning of an iteration, we define our acceptance tests using rspec and product skeleton behaviors: describe "this part of the system, in these conditions" do it "should behave like this" end etc...> Any thoughts or suggestions? Here is a sketch of what I think I''d > like: > > server " http://localhost:8080" do > > test ''/'' do > page.status.should_be "200" > page.should_have :image, "logo.gif" > page.should_have :string, "Copyright 2006-Present SomeCo, Inc. All > rights reserved." > page.should_have :div, :id => ''copyright''[...] To me, this framework is "talking" at the wrong level of abstraction. The domain of its "language" has to do with the underlying technology, in this case HTML. Tests written at this level will be brittle and hard to maintain as the application changes. Our approach has been to build a framework that models our application and provides a language in the domain of that application, i.e. it uses the nouns and verbs of our application to provide an API for testing the app. It is in essence a Domain Specific Language for testing our application. For example, when the user creates "things" in our app, they use an "Editor", and abstraction of an HTML form. The editor framework allows you to write code that looks like this: user_page.click_create do |editor| editor.first_name = "John" editor.last_name = "Smith" end click_create handles the saving of the form (i.e. clicking save) for you. The Editor class might look something like this: class UserEditor element_accessor :first_name, element_locator("first_name") element_accessor :last_name, element_locator("last_name") end where ''element_writer'' and ''element_locator'' are wrappers dealing with input fields in Selenium. Then you can check the values in the form: editor.first_name.should == "John" Now your tests don''t talk about HTML. -Bob
> We have re-written our functional test framework 4 times, and finally > settled on Selenium. No other open-source tool gets you the > cross-browser, cross-platform support. > > We use the Ruby bindings for Selenium-RC.How is it working? How long did it take you from initial download of selenium to continuous integration deployment running tests and all, and how stable is it? The documentation for selenium is horrid, and we kept having our very simple tests, on multiple machines, hang indefinitely. We were primarily using the java flavor, as I hadn''t sold anyone on ruby anywhere yet. Should I look again?> > To me, this framework is "talking" at the wrong level of > abstraction. The domain of its "language" has to do with the > underlying technology, in this case HTML.I hear you. At some level, you do have assertions on the HTML, right? You abstract them away into some class, but you do need to know that the, say... "left nav menu" exists on each page, right? Tests written at this level will be brittle and hard to maintain as> the application changes.Agreed.> > Now your tests don''t talk about HTML.Cool, thanks. So, I''m taking it that you think I should look again at seleium, and that it''s ready for prime time. I guess I''ll give it a shot. Thanks, Tim -Bob> _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-- Tim Tischler ttischler at homeaway.com tischler at purplecoffeecup.com 512-565-4750 AIM:tjtischler512 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/rspec-users/attachments/20070509/cfbf4bf7/attachment-0001.html
"Tim Tischler" <tischler at purplecoffeecup.com> writes:> We have re-written our functional test framework 4 times, and finally > settled on Selenium. No other open-source tool gets you the > cross-browser, cross-platform support. > > We use the Ruby bindings for Selenium-RC. > > How is it working? How long did it take you from initial download of > selenium to continuous integration deployment running tests and all, and > how stable is it? The documentation for selenium is horrid, and we kept > having our very simple tests, on multiple machines, hang > indefinitely.I don''t know if the hanging is a result of timing issues. But they are things you have to bake into your framework. Checking to make sure that elements are present before you try to touch them.> We were primarily using the java flavor, as I hadn''t sold anyone on > ruby anywhere yet.The language features of Ruby, plus things like rspec make a compelling argument.> Should I look again?I think so.> To me, this framework is "talking" at the wrong level of > abstraction. The domain of its "language" has to do with the > underlying technology, in this case HTML. > > I hear you. At some level, you do have assertions on the HTML, right? > You abstract them away into some class, but you do need to know that the, > say... "left nav menu" exists on each page, right?Yep, hide the details in the framework classes. We have a validate method that gets called on construction, that checks interesting things on the page. Then things like ''"left nav menu" exists on each page'' gets baked into the framework. When using Ruby, you can mix-in these sorts of things.> Cool, thanks. So, I''m taking it that you think I should look again at > seleium, and that it''s ready for prime time. I guess I''ll give it a shot.We''ve had a very good experience with it. -Bob
On 5/9/07, Tim Tischler <tischler at purplecoffeecup.com> wrote:> Hey all, > > I am currently working on coming up w/ an easy to use, developer-centric web > testing framework to test a J2EE app with. (I have 3 rails apps in > production, love rspec, and am currently at a java shop). >One of the biggest projects in my company (BEKK Consulting) is using RSpec to drive Watir, and we have about 400 examples (it blocks). At the core of this we''re using Spec::Ui, which is an RSpec extension specially designed for Watir and Selenium-RC. Spec::Ui extends the standard RSpec HTML report with: * screenshot at the time of failure * HTML source code of the browser at the time of failure These detailed reports is something the teams have become completely dependent upon. It really kicks ass. Spec::Ui also has custom RSpec matchers Watir (I haven''t made any for Selenium-RC yet, but we''re accepting patches). These matchers allow you to do things like this: @ie.should have_link(:url, "http://rspec.rubyforge.org") @ie.should_not have_button(:name, "delete") etc. All of Watir''s "element lookup" methods have a corresponding have_* method. Spec::Ui has been in RSpec''s trunk for several months (trunk/spec_ui), but I have deliberately kept mostly quiet about it because it has been somewhat immature. However, it has stabilised a lot lately, and I encourage you to take a look at it. There are examples both for Watir and Selenium-RC. See separate email releasing RSpec 0.9.4.> I''ve looked at selenium, and it just doesn''t seem like it is ready for prime > time, and the target audience is developers.Do you think Selenium is more developer-targeted than Watir? Have you seen the FIT-style markup for Selenium?> So, having said that, does > the rspec community want a "functional" test framework? Are you happy w/ > selenium, and is it working? >I hope Spec::Ui is what you''re looking for. Both Watir and Selenium-RC are working great with RSpec/Spec::Ui. There are some minor caveats with Selenium-RC on OS X though (see README.txt in the selenium example).> Unless someone convinces me of selenium, I''m planning on taking the basics > and concepts of rspec, and focus it more towards testing web applications, > and am happy to contribute. My company is totally cool with contributing to > open source projects if it makes sense. > > Any thoughts or suggestions? Here is a sketch of what I think I''d like: > > server " http://localhost:8080" do > > test ''/'' do > page.status.should_be "200" > page.should_have :image, "logo.gif" > page.should_have :string, "Copyright 2006-Present SomeCo, Inc. All > rights reserved." > page.should_have :div, :id => ''copyright'' > > page.should_include :css, ''default.css'' > > page.should_have.no_broken_links > page.should_have.at_least.3 :div, :class => "bbxBody" > page.should_have.at_most.6 :div, :class => "someClassName" > > # page.[name_of_form]_form. > page.search_form.submit :name => "California" do > result.status.should_be :success > result.should_have :image, "logo.gif" > result.url.should_contain "some_page.html" > end > > page.should_have :div, :class => ''something else'' > page.should_have :div, :id => ''foo'', :class => ''bar'' > page.should_have ''/div[1]/div[2]/td'' > end > > test ''/some/url.html?id=35'' do > page.should_redirect :to => ''/foobar.html'' > end > end >This is quite similar to what Spec::Ui us doing today. The syntax is a little different for Watir and Selenium-RC, but I hope we can unify it with custom RSpec matchers with similar APIs. This should make it easier to switch transparently(ish) between Selenium-RC and Watir. I second Bob''s recommendation to not use this API extensively in your own specs though, but rather implement your own domain-specific API around it and use that instead. We have also done this at the project I mentioned above, and it makes for much more readable and maintainable specs. Aslak> -- > Tim Tischler > ttischler at homeaway.com > tischler at purplecoffeecup.com > 512-565-4750 > AIM:tjtischler512 > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >