What''s the status on a rake task for the story runner. If nothing is in progress, where could I start to try and build one? JD
James Deville wrote:> What''s the status on a rake task for the story runner. If nothing is > in progress, where could I start to try and build one? > > JD > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >If you check out rspec trunk you can see that they have created one for the rspec project. It just runs the all.rb file in the stories directory. -Ben
On Dec 19, 2007 7:01 PM, James Deville <james.deville at gmail.com> wrote:> What''s the status on a rake task for the story runner. If nothing is > in progress, where could I start to try and build one? > > JDI''ve been using: task ''stories'' do sh "ruby stories/all.rb" end task ''stories:rcov'' do sh "rcov -o story_coverage --exclude stories stories/all.rb" end Kyle
Ben Mabey wrote:> James Deville wrote: > >> What''s the status on a rake task for the story runner. If nothing is >> in progress, where could I start to try and build one? >> >> JD >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users >> >> > > If you check out rspec trunk you can see that they have created one for > the rspec project. It just runs the all.rb file in the stories directory. > > -Ben > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >This is the actual task they are using: desc "Run all stories" task :stories do html = ''story_server/prototype/rspec_stories.html'' ruby "stories/all.rb --colour --format plain --format html:#{html}" unless IO.read(html) =~ /<span class="param">/m raise ''highlighted parameters are broken in story HTML'' end end
On 12/19/07, James Deville <james.deville at gmail.com> wrote:> What''s the status on a rake task for the story runner. If nothing is > in progress, where could I start to try and build one?Until step/scenario directories are standardized you''re not going to see a rake task committed. Feel free to roll your own, I wrote a simple one that just calls all.rb. -- Josh Knowles phone: 509-979-1593 email: joshknowles at gmail.com web: http://joshknowles.com
Yeah, had a slight email conversation with David C about that in regards to bug #188. I am wondering why we don''t standardize it, ya know convention over configuration and all. JD On Dec 19, 2007, at 6:28 PM, Josh Knowles wrote:> On 12/19/07, James Deville <james.deville at gmail.com> wrote: >> What''s the status on a rake task for the story runner. If nothing is >> in progress, where could I start to try and build one? > > Until step/scenario directories are standardized you''re not going to > see a rake task committed. > > Feel free to roll your own, I wrote a simple one that just calls > all.rb. > > > -- > Josh Knowles > phone: 509-979-1593 > email: joshknowles at gmail.com > web: http://joshknowles.com > _______________________________________________ > rspec-users mailing list > rspec-use
What file is that in? I was looking for one in trunk earlier. JD On Dec 19, 2007, at 5:07 PM, Ben Mabey wrote:> Ben Mabey wrote: >> James Deville wrote: >> >>> What''s the status on a rake task for the story runner. If nothing is >>> in progress, where could I start to try and build one? >>> >>> JD >>> _______________________________________________ >>> rspec-users mailing list >>> rspec-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/rspec-users >>> >>> >> >> If you check out rspec trunk you can see that they have created one >> for >> the rspec project. It just runs the all.rb file in the stories >> directory. >> >> -Ben >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users >> > This is the actual task they are using: > > desc "Run all stories" > task :stories do > html = ''story_server/prototype/rspec_stories.html'' > ruby "stories/all.rb --colour --format plain --format html:#{html}" > unless IO.read(html) =~ /<span class="param">/m > raise ''highlighted parameters are broken in story HTML'' > end > end > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users
On Dec 19, 2007 11:34 PM, James Deville <james.deville at gmail.com> wrote:> Yeah, had a slight email conversation with David C about that in > regards to bug #188. I am wondering why we don''t standardize it, ya > know convention over configuration and all.Because I think it''s premature to call anything related to story runner a convention. I actually organize them differently from what many are calling convention, and my way is not necessarily "right" or "better." Let''s wait a while on this. We''ll get there.
On Dec 19, 2007, at 9:38 PM, David Chelimsky wrote:> On Dec 19, 2007 11:34 PM, James Deville <james.deville at gmail.com> > wrote: >> Yeah, had a slight email conversation with David C about that in >> regards to bug #188. I am wondering why we don''t standardize it, ya >> know convention over configuration and all. > > Because I think it''s premature to call anything related to story > runner a convention. I actually organize them differently from what > many are calling convention, and my way is not necessarily "right" or > "better." Let''s wait a while on this. We''ll get there. > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-usersGood enough for me. I just wanted a reason. May I ask how you set them up? James Deville
On Dec 19, 2007 11:40 PM, James Deville <james.deville at gmail.com> wrote:> > > On Dec 19, 2007, at 9:38 PM, David Chelimsky wrote: > > > On Dec 19, 2007 11:34 PM, James Deville <james.deville at gmail.com> > > wrote: > >> Yeah, had a slight email conversation with David C about that in > >> regards to bug #188. I am wondering why we don''t standardize it, ya > >> know convention over configuration and all. > > > > Because I think it''s premature to call anything related to story > > runner a convention. I actually organize them differently from what > > many are calling convention, and my way is not necessarily "right" or > > "better." Let''s wait a while on this. We''ll get there. > > _______________________________________________ > > rspec-users mailing list > > rspec-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/rspec-users > > > Good enough for me. I just wanted a reason. May I ask how you set them > up?Sure. stories/ stories/helper.rb stories/steps/ (steps go in here - that seems to be the "convention") stories/stuff_related_to_one_feature stories/stuff_related_to_another_feature stories/stuff_related_to_yet_another_feature So in this case, the only thing that would be consistent across projects would be helper.rb and the steps directory. Even that should probably be called step_definitions or something. I''m not sure. Anyhow - that''s where I''m at. How about you?
On Dec 19, 2007, at 9:44 PM, David Chelimsky wrote:> On Dec 19, 2007 11:40 PM, James Deville <james.deville at gmail.com> > wrote: >> >> >> On Dec 19, 2007, at 9:38 PM, David Chelimsky wrote: >> >>> On Dec 19, 2007 11:34 PM, James Deville <james.deville at gmail.com> >>> wrote: >>>> Yeah, had a slight email conversation with David C about that in >>>> regards to bug #188. I am wondering why we don''t standardize it, ya >>>> know convention over configuration and all. >>> >>> Because I think it''s premature to call anything related to story >>> runner a convention. I actually organize them differently from what >>> many are calling convention, and my way is not necessarily "right" >>> or >>> "better." Let''s wait a while on this. We''ll get there. >>> _______________________________________________ >>> rspec-users mailing list >>> rspec-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/rspec-users >> >> >> Good enough for me. I just wanted a reason. May I ask how you set >> them >> up? > > Sure. > > stories/ > stories/helper.rb > stories/steps/ (steps go in here - that seems to be the "convention") > stories/stuff_related_to_one_feature > stories/stuff_related_to_another_feature > stories/stuff_related_to_yet_another_feature > > So in this case, the only thing that would be consistent across > projects would be helper.rb and the steps directory. Even that should > probably be called step_definitions or something. I''m not sure. > > Anyhow - that''s where I''m at. How about you? > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-usersI''ve been doing a feature per file. So I have: stories/ stories/helper.rb stories/steps/ stories/stories/ So in stories/stories I have my stories with multiple scenarios per story. I''m definitely seeing your point, and the reason for leaving it as is. Do you use selenium or anything like that via the stories? If not, what kinds of things do you test with stories? JD
On Dec 19, 2007 11:50 PM, James Deville <james.deville at gmail.com> wrote:> > > On Dec 19, 2007, at 9:44 PM, David Chelimsky wrote: > > > On Dec 19, 2007 11:40 PM, James Deville <james.deville at gmail.com> > > wrote: > >> > >> > >> On Dec 19, 2007, at 9:38 PM, David Chelimsky wrote: > >> > >>> On Dec 19, 2007 11:34 PM, James Deville <james.deville at gmail.com> > >>> wrote: > >>>> Yeah, had a slight email conversation with David C about that in > >>>> regards to bug #188. I am wondering why we don''t standardize it, ya > >>>> know convention over configuration and all. > >>> > >>> Because I think it''s premature to call anything related to story > >>> runner a convention. I actually organize them differently from what > >>> many are calling convention, and my way is not necessarily "right" > >>> or > >>> "better." Let''s wait a while on this. We''ll get there. > >>> _______________________________________________ > >>> rspec-users mailing list > >>> rspec-users at rubyforge.org > >>> http://rubyforge.org/mailman/listinfo/rspec-users > >> > >> > >> Good enough for me. I just wanted a reason. May I ask how you set > >> them > >> up? > > > > Sure. > > > > stories/ > > stories/helper.rb > > stories/steps/ (steps go in here - that seems to be the "convention") > > stories/stuff_related_to_one_feature > > stories/stuff_related_to_another_feature > > stories/stuff_related_to_yet_another_feature > > > > So in this case, the only thing that would be consistent across > > projects would be helper.rb and the steps directory. Even that should > > probably be called step_definitions or something. I''m not sure. > > > > Anyhow - that''s where I''m at. How about you? > > _______________________________________________ > > rspec-users mailing list > > rspec-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/rspec-users > > I''ve been doing a feature per file. So I have: > > stories/ > stories/helper.rb > stories/steps/ > stories/stories/ > > So in stories/stories I have my stories with multiple scenarios per > story. > > I''m definitely seeing your point, and the reason for leaving it as is. > > Do you use selenium or anything like that via the stories? If not, > what kinds of things do you test with stories?I''ve got a bunch of selenium tests being driven by spec/ui. I haven''t converted them to stories yet. Mostly I''ve been using webrat, Bryan Helmkamp''s awesome Hpricot-wrapping goodness. It doesn''t cover javascript, but the apps I''ve been working on have been very vanilla with respect to ajax, so I''ve been satisfied with have_rjs. Anyhow, webrat does something really cool - it ties your form submissions to the html of the rendered form. So you make steps like this: ====================When "I go to log in" do visits "/session/new" end When "I enter $label: $value" do |label, value| fills_in label, :with => value end When "I submit my credentials" do clicks_button end Then "I should see $message" do |message| response.should have_text(/#{message}/) end ==================== And a scenario like this: ====================When I go to log in And I enter Username: david And I enter Password: webrat rules And I submit my credentials Then I should see Welcome david ==================== And what webrat does is grabs the html from the response in the visits method, modifies that html with your data in the fills_in method, and then yanks the data from the html and submits it with the clicks_button method. What this means is that if the form isn''t aligned with the fields you''re submitting, you''ll get a failure. How clean is that?
On Dec 19, 2007, at 10:16 PM, David Chelimsky wrote:> On Dec 19, 2007 11:50 PM, James Deville <james.deville at gmail.com> > wrote: >> >> >> On Dec 19, 2007, at 9:44 PM, David Chelimsky wrote: >> >>> On Dec 19, 2007 11:40 PM, James Deville <james.deville at gmail.com> >>> wrote: >>>> >>>> >>>> On Dec 19, 2007, at 9:38 PM, David Chelimsky wrote: >>>> >>>>> On Dec 19, 2007 11:34 PM, James Deville <james.deville at gmail.com> >>>>> wrote: >>>>>> Yeah, had a slight email conversation with David C about that in >>>>>> regards to bug #188. I am wondering why we don''t standardize >>>>>> it, ya >>>>>> know convention over configuration and all. >>>>> >>>>> Because I think it''s premature to call anything related to story >>>>> runner a convention. I actually organize them differently from >>>>> what >>>>> many are calling convention, and my way is not necessarily "right" >>>>> or >>>>> "better." Let''s wait a while on this. We''ll get there. >>>>> _______________________________________________ >>>>> rspec-users mailing list >>>>> rspec-users at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/rspec-users >>>> >>>> >>>> Good enough for me. I just wanted a reason. May I ask how you set >>>> them >>>> up? >>> >>> Sure. >>> >>> stories/ >>> stories/helper.rb >>> stories/steps/ (steps go in here - that seems to be the >>> "convention") >>> stories/stuff_related_to_one_feature >>> stories/stuff_related_to_another_feature >>> stories/stuff_related_to_yet_another_feature >>> >>> So in this case, the only thing that would be consistent across >>> projects would be helper.rb and the steps directory. Even that >>> should >>> probably be called step_definitions or something. I''m not sure. >>> >>> Anyhow - that''s where I''m at. How about you? >>> _______________________________________________ >>> rspec-users mailing list >>> rspec-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/rspec-users >> >> I''ve been doing a feature per file. So I have: >> >> stories/ >> stories/helper.rb >> stories/steps/ >> stories/stories/ >> >> So in stories/stories I have my stories with multiple scenarios per >> story. >> >> I''m definitely seeing your point, and the reason for leaving it as >> is. >> >> Do you use selenium or anything like that via the stories? If not, >> what kinds of things do you test with stories? > > I''ve got a bunch of selenium tests being driven by spec/ui. I haven''t > converted them to stories yet. > > Mostly I''ve been using webrat, Bryan Helmkamp''s awesome > Hpricot-wrapping goodness. It doesn''t cover javascript, but the apps > I''ve been working on have been very vanilla with respect to ajax, so > I''ve been satisfied with have_rjs. > > Anyhow, webrat does something really cool - it ties your form > submissions to the html of the rendered form. So you make steps like > this: > > ====================> When "I go to log in" do > visits "/session/new" > end > > When "I enter $label: $value" do |label, value| > fills_in label, :with => value > end > > When "I submit my credentials" do > clicks_button > end > > Then "I should see $message" do |message| > response.should have_text(/#{message}/) > end > ====================> > And a scenario like this: > > ====================> When I go to log in > And I enter Username: david > And I enter Password: webrat rules > And I submit my credentials > Then I should see Welcome david > ====================> > And what webrat does is grabs the html from the response in the visits > method, modifies that html with your data in the fills_in method, and > then yanks the data from the html and submits it with the > clicks_button method. What this means is that if the form isn''t > aligned with the fields you''re submitting, you''ll get a failure. > > How clean is that? > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-usersUm... Sweetness!!! How is it for speed? There is a ton of stuff we would love to move out of Selenium due to issues with sessions, flash, and database persistance. JD
On Dec 20, 2007 12:25 AM, James Deville <james.deville at gmail.com> wrote:> > > On Dec 19, 2007, at 10:16 PM, David Chelimsky wrote: > > > On Dec 19, 2007 11:50 PM, James Deville <james.deville at gmail.com> > > wrote: > >> > >> > >> On Dec 19, 2007, at 9:44 PM, David Chelimsky wrote: > >> > >>> On Dec 19, 2007 11:40 PM, James Deville <james.deville at gmail.com> > >>> wrote: > >>>> > >>>> > >>>> On Dec 19, 2007, at 9:38 PM, David Chelimsky wrote: > >>>> > >>>>> On Dec 19, 2007 11:34 PM, James Deville <james.deville at gmail.com> > >>>>> wrote: > >>>>>> Yeah, had a slight email conversation with David C about that in > >>>>>> regards to bug #188. I am wondering why we don''t standardize > >>>>>> it, ya > >>>>>> know convention over configuration and all. > >>>>> > >>>>> Because I think it''s premature to call anything related to story > >>>>> runner a convention. I actually organize them differently from > >>>>> what > >>>>> many are calling convention, and my way is not necessarily "right" > >>>>> or > >>>>> "better." Let''s wait a while on this. We''ll get there. > >>>>> _______________________________________________ > >>>>> rspec-users mailing list > >>>>> rspec-users at rubyforge.org > >>>>> http://rubyforge.org/mailman/listinfo/rspec-users > >>>> > >>>> > >>>> Good enough for me. I just wanted a reason. May I ask how you set > >>>> them > >>>> up? > >>> > >>> Sure. > >>> > >>> stories/ > >>> stories/helper.rb > >>> stories/steps/ (steps go in here - that seems to be the > >>> "convention") > >>> stories/stuff_related_to_one_feature > >>> stories/stuff_related_to_another_feature > >>> stories/stuff_related_to_yet_another_feature > >>> > >>> So in this case, the only thing that would be consistent across > >>> projects would be helper.rb and the steps directory. Even that > >>> should > >>> probably be called step_definitions or something. I''m not sure. > >>> > >>> Anyhow - that''s where I''m at. How about you? > >>> _______________________________________________ > >>> rspec-users mailing list > >>> rspec-users at rubyforge.org > >>> http://rubyforge.org/mailman/listinfo/rspec-users > >> > >> I''ve been doing a feature per file. So I have: > >> > >> stories/ > >> stories/helper.rb > >> stories/steps/ > >> stories/stories/ > >> > >> So in stories/stories I have my stories with multiple scenarios per > >> story. > >> > >> I''m definitely seeing your point, and the reason for leaving it as > >> is. > >> > >> Do you use selenium or anything like that via the stories? If not, > >> what kinds of things do you test with stories? > > > > I''ve got a bunch of selenium tests being driven by spec/ui. I haven''t > > converted them to stories yet. > > > > Mostly I''ve been using webrat, Bryan Helmkamp''s awesome > > Hpricot-wrapping goodness. It doesn''t cover javascript, but the apps > > I''ve been working on have been very vanilla with respect to ajax, so > > I''ve been satisfied with have_rjs. > > > > Anyhow, webrat does something really cool - it ties your form > > submissions to the html of the rendered form. So you make steps like > > this: > > > > ====================> > When "I go to log in" do > > visits "/session/new" > > end > > > > When "I enter $label: $value" do |label, value| > > fills_in label, :with => value > > end > > > > When "I submit my credentials" do > > clicks_button > > end > > > > Then "I should see $message" do |message| > > response.should have_text(/#{message}/) > > end > > ====================> > > > And a scenario like this: > > > > ====================> > When I go to log in > > And I enter Username: david > > And I enter Password: webrat rules > > And I submit my credentials > > Then I should see Welcome david > > ====================> > > > And what webrat does is grabs the html from the response in the visits > > method, modifies that html with your data in the fills_in method, and > > then yanks the data from the html and submits it with the > > clicks_button method. What this means is that if the form isn''t > > aligned with the fields you''re submitting, you''ll get a failure. > > > > How clean is that? > > _______________________________________________ > > rspec-users mailing list > > rspec-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/rspec-users > > > Um... Sweetness!!! How is it for speed? There is a ton of stuff we > would love to move out of Selenium due to issues with sessions, flash, > and database persistance.Obviously it''s much faster than running stuff in a browser. I have nothing else to measure it against and haven''t done any real benchmarking, but it is perceivably much faster than running selenium. What I''d recommend - where I''m headed as the opportunity presents itself - is to have in-memory and in-browser implementations of the same steps. Then you can run them in-memory most of the time and skip the javascript, but run them in-browser once in a while - like before commits - to make sure the javascript is all working too. There''s also an interesting javascript testing framework that I think shows a lot of promise called crosscheck (http://www.thefrontside.net/crosscheck). It''s basically got it''s own browser-less javascript engines and runs in-memory. Last I heard it was mostly dead-on, but that it missed some things. The guy who told me that said he was running selenium for a few things that crosscheck didn''t do right, but I don''t know what those are :) I did toy with it for a minute, and it''s blazingly fast. I just haven''t made a commitment to use it in anger yet - but I''m looking forward to the opportunity to. Cheers, David
James Deville wrote:> On Dec 19, 2007, at 10:16 PM, David Chelimsky wrote: > > >> On Dec 19, 2007 11:50 PM, James Deville <james.deville at gmail.com> >> wrote: >> >>> On Dec 19, 2007, at 9:44 PM, David Chelimsky wrote: >>> >>> >>>> On Dec 19, 2007 11:40 PM, James Deville <james.deville at gmail.com> >>>> wrote: >>>> >>>>> On Dec 19, 2007, at 9:38 PM, David Chelimsky wrote: >>>>> >>>>> >>>>>> On Dec 19, 2007 11:34 PM, James Deville <james.deville at gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Yeah, had a slight email conversation with David C about that in >>>>>>> regards to bug #188. I am wondering why we don''t standardize >>>>>>> it, ya >>>>>>> know convention over configuration and all. >>>>>>> >>>>>> Because I think it''s premature to call anything related to story >>>>>> runner a convention. I actually organize them differently from >>>>>> what >>>>>> many are calling convention, and my way is not necessarily "right" >>>>>> or >>>>>> "better." Let''s wait a while on this. We''ll get there. >>>>>> _______________________________________________ >>>>>> rspec-users mailing list >>>>>> rspec-users at rubyforge.org >>>>>> http://rubyforge.org/mailman/listinfo/rspec-users >>>>>> >>>>> Good enough for me. I just wanted a reason. May I ask how you set >>>>> them >>>>> up? >>>>> >>>> Sure. >>>> >>>> stories/ >>>> stories/helper.rb >>>> stories/steps/ (steps go in here - that seems to be the >>>> "convention") >>>> stories/stuff_related_to_one_feature >>>> stories/stuff_related_to_another_feature >>>> stories/stuff_related_to_yet_another_feature >>>> >>>> So in this case, the only thing that would be consistent across >>>> projects would be helper.rb and the steps directory. Even that >>>> should >>>> probably be called step_definitions or something. I''m not sure. >>>> >>>> Anyhow - that''s where I''m at. How about you? >>>> _______________________________________________ >>>> rspec-users mailing list >>>> rspec-users at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/rspec-users >>>> >>> I''ve been doing a feature per file. So I have: >>> >>> stories/ >>> stories/helper.rb >>> stories/steps/ >>> stories/stories/ >>> >>> So in stories/stories I have my stories with multiple scenarios per >>> story. >>> >>> I''m definitely seeing your point, and the reason for leaving it as >>> is. >>> >>> Do you use selenium or anything like that via the stories? If not, >>> what kinds of things do you test with stories? >>> >> I''ve got a bunch of selenium tests being driven by spec/ui. I haven''t >> converted them to stories yet. >> >> Mostly I''ve been using webrat, Bryan Helmkamp''s awesome >> Hpricot-wrapping goodness. It doesn''t cover javascript, but the apps >> I''ve been working on have been very vanilla with respect to ajax, so >> I''ve been satisfied with have_rjs. >> >> Anyhow, webrat does something really cool - it ties your form >> submissions to the html of the rendered form. So you make steps like >> this: >> >> ====================>> When "I go to log in" do >> visits "/session/new" >> end >> >> When "I enter $label: $value" do |label, value| >> fills_in label, :with => value >> end >> >> When "I submit my credentials" do >> clicks_button >> end >> >> Then "I should see $message" do |message| >> response.should have_text(/#{message}/) >> end >> ====================>> >> And a scenario like this: >> >> ====================>> When I go to log in >> And I enter Username: david >> And I enter Password: webrat rules >> And I submit my credentials >> Then I should see Welcome david >> ====================>> >> And what webrat does is grabs the html from the response in the visits >> method, modifies that html with your data in the fills_in method, and >> then yanks the data from the html and submits it with the >> clicks_button method. What this means is that if the form isn''t >> aligned with the fields you''re submitting, you''ll get a failure. >> >> How clean is that? >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users >> > > > Um... Sweetness!!! How is it for speed? There is a ton of stuff we > would love to move out of Selenium due to issues with sessions, flash, > and database persistance. > > JD > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >Yeah, I''m curious about speed as well. We decided not to switch over to webrat because we thought that using hpricot on all of the tests would slow things down compared to the regular assert_select and other integration helpers. Is there much of a speed difference? -Ben
On Dec 20, 2007 12:42 AM, Ben Mabey <ben at benmabey.com> wrote:> > James Deville wrote: > > On Dec 19, 2007, at 10:16 PM, David Chelimsky wrote: > > > > > >> On Dec 19, 2007 11:50 PM, James Deville <james.deville at gmail.com> > >> wrote: > >> > >>> On Dec 19, 2007, at 9:44 PM, David Chelimsky wrote: > >>> > >>> > >>>> On Dec 19, 2007 11:40 PM, James Deville <james.deville at gmail.com> > >>>> wrote: > >>>> > >>>>> On Dec 19, 2007, at 9:38 PM, David Chelimsky wrote: > >>>>> > >>>>> > >>>>>> On Dec 19, 2007 11:34 PM, James Deville <james.deville at gmail.com> > >>>>>> wrote: > >>>>>> > >>>>>>> Yeah, had a slight email conversation with David C about that in > >>>>>>> regards to bug #188. I am wondering why we don''t standardize > >>>>>>> it, ya > >>>>>>> know convention over configuration and all. > >>>>>>> > >>>>>> Because I think it''s premature to call anything related to story > >>>>>> runner a convention. I actually organize them differently from > >>>>>> what > >>>>>> many are calling convention, and my way is not necessarily "right" > >>>>>> or > >>>>>> "better." Let''s wait a while on this. We''ll get there. > >>>>>> _______________________________________________ > >>>>>> rspec-users mailing list > >>>>>> rspec-users at rubyforge.org > >>>>>> http://rubyforge.org/mailman/listinfo/rspec-users > >>>>>> > >>>>> Good enough for me. I just wanted a reason. May I ask how you set > >>>>> them > >>>>> up? > >>>>> > >>>> Sure. > >>>> > >>>> stories/ > >>>> stories/helper.rb > >>>> stories/steps/ (steps go in here - that seems to be the > >>>> "convention") > >>>> stories/stuff_related_to_one_feature > >>>> stories/stuff_related_to_another_feature > >>>> stories/stuff_related_to_yet_another_feature > >>>> > >>>> So in this case, the only thing that would be consistent across > >>>> projects would be helper.rb and the steps directory. Even that > >>>> should > >>>> probably be called step_definitions or something. I''m not sure. > >>>> > >>>> Anyhow - that''s where I''m at. How about you? > >>>> _______________________________________________ > >>>> rspec-users mailing list > >>>> rspec-users at rubyforge.org > >>>> http://rubyforge.org/mailman/listinfo/rspec-users > >>>> > >>> I''ve been doing a feature per file. So I have: > >>> > >>> stories/ > >>> stories/helper.rb > >>> stories/steps/ > >>> stories/stories/ > >>> > >>> So in stories/stories I have my stories with multiple scenarios per > >>> story. > >>> > >>> I''m definitely seeing your point, and the reason for leaving it as > >>> is. > >>> > >>> Do you use selenium or anything like that via the stories? If not, > >>> what kinds of things do you test with stories? > >>> > >> I''ve got a bunch of selenium tests being driven by spec/ui. I haven''t > >> converted them to stories yet. > >> > >> Mostly I''ve been using webrat, Bryan Helmkamp''s awesome > >> Hpricot-wrapping goodness. It doesn''t cover javascript, but the apps > >> I''ve been working on have been very vanilla with respect to ajax, so > >> I''ve been satisfied with have_rjs. > >> > >> Anyhow, webrat does something really cool - it ties your form > >> submissions to the html of the rendered form. So you make steps like > >> this: > >> > >> ====================> >> When "I go to log in" do > >> visits "/session/new" > >> end > >> > >> When "I enter $label: $value" do |label, value| > >> fills_in label, :with => value > >> end > >> > >> When "I submit my credentials" do > >> clicks_button > >> end > >> > >> Then "I should see $message" do |message| > >> response.should have_text(/#{message}/) > >> end > >> ====================> >> > >> And a scenario like this: > >> > >> ====================> >> When I go to log in > >> And I enter Username: david > >> And I enter Password: webrat rules > >> And I submit my credentials > >> Then I should see Welcome david > >> ====================> >> > >> And what webrat does is grabs the html from the response in the visits > >> method, modifies that html with your data in the fills_in method, and > >> then yanks the data from the html and submits it with the > >> clicks_button method. What this means is that if the form isn''t > >> aligned with the fields you''re submitting, you''ll get a failure. > >> > >> How clean is that? > >> _______________________________________________ > >> rspec-users mailing list > >> rspec-users at rubyforge.org > >> http://rubyforge.org/mailman/listinfo/rspec-users > >> > > > > > > Um... Sweetness!!! How is it for speed? There is a ton of stuff we > > would love to move out of Selenium due to issues with sessions, flash, > > and database persistance. > > > > JD > > _______________________________________________ > > rspec-users mailing list > > rspec-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/rspec-users > > > > Yeah, I''m curious about speed as well. We decided not to switch over to > webrat because we thought that using hpricot on all of the tests would > slow things down compared to the regular assert_select and other > integration helpers. Is there much of a speed difference?I haven''t compared directly - I would have to assume it''s slower not necessarily because of hpricot, but because of the strategy of tying the inputs to the form and building the post from the html. But that is very, very comforting - especially in integration tests. Without that, assert_select leaves you with the same problems that mocks do - you could get all your isolation tests to pass, but integration fails.
On Dec 19, 2007, at 10:42 PM, David Chelimsky wrote:> On Dec 20, 2007 12:25 AM, James Deville <james.deville at gmail.com> > wrote: >> >> >> On Dec 19, 2007, at 10:16 PM, David Chelimsky wrote: >> >>> On Dec 19, 2007 11:50 PM, James Deville <james.deville at gmail.com> >>> wrote: >>>> >>>> >>>> On Dec 19, 2007, at 9:44 PM, David Chelimsky wrote: >>>> >>>>> On Dec 19, 2007 11:40 PM, James Deville <james.deville at gmail.com> >>>>> wrote: >>>>>> >>>>>> >>>>>> On Dec 19, 2007, at 9:38 PM, David Chelimsky wrote: >>>>>> >>>>>>> On Dec 19, 2007 11:34 PM, James Deville >>>>>>> <james.deville at gmail.com> >>>>>>> wrote: >>>>>>>> Yeah, had a slight email conversation with David C about that >>>>>>>> in >>>>>>>> regards to bug #188. I am wondering why we don''t standardize >>>>>>>> it, ya >>>>>>>> know convention over configuration and all. >>>>>>> >>>>>>> Because I think it''s premature to call anything related to story >>>>>>> runner a convention. I actually organize them differently from >>>>>>> what >>>>>>> many are calling convention, and my way is not necessarily >>>>>>> "right" >>>>>>> or >>>>>>> "better." Let''s wait a while on this. We''ll get there. >>>>>>> _______________________________________________ >>>>>>> rspec-users mailing list >>>>>>> rspec-users at rubyforge.org >>>>>>> http://rubyforge.org/mailman/listinfo/rspec-users >>>>>> >>>>>> >>>>>> Good enough for me. I just wanted a reason. May I ask how you set >>>>>> them >>>>>> up? >>>>> >>>>> Sure. >>>>> >>>>> stories/ >>>>> stories/helper.rb >>>>> stories/steps/ (steps go in here - that seems to be the >>>>> "convention") >>>>> stories/stuff_related_to_one_feature >>>>> stories/stuff_related_to_another_feature >>>>> stories/stuff_related_to_yet_another_feature >>>>> >>>>> So in this case, the only thing that would be consistent across >>>>> projects would be helper.rb and the steps directory. Even that >>>>> should >>>>> probably be called step_definitions or something. I''m not sure. >>>>> >>>>> Anyhow - that''s where I''m at. How about you? >>>>> _______________________________________________ >>>>> rspec-users mailing list >>>>> rspec-users at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/rspec-users >>>> >>>> I''ve been doing a feature per file. So I have: >>>> >>>> stories/ >>>> stories/helper.rb >>>> stories/steps/ >>>> stories/stories/ >>>> >>>> So in stories/stories I have my stories with multiple scenarios per >>>> story. >>>> >>>> I''m definitely seeing your point, and the reason for leaving it as >>>> is. >>>> >>>> Do you use selenium or anything like that via the stories? If not, >>>> what kinds of things do you test with stories? >>> >>> I''ve got a bunch of selenium tests being driven by spec/ui. I >>> haven''t >>> converted them to stories yet. >>> >>> Mostly I''ve been using webrat, Bryan Helmkamp''s awesome >>> Hpricot-wrapping goodness. It doesn''t cover javascript, but the apps >>> I''ve been working on have been very vanilla with respect to ajax, so >>> I''ve been satisfied with have_rjs. >>> >>> Anyhow, webrat does something really cool - it ties your form >>> submissions to the html of the rendered form. So you make steps like >>> this: >>> >>> ====================>>> When "I go to log in" do >>> visits "/session/new" >>> end >>> >>> When "I enter $label: $value" do |label, value| >>> fills_in label, :with => value >>> end >>> >>> When "I submit my credentials" do >>> clicks_button >>> end >>> >>> Then "I should see $message" do |message| >>> response.should have_text(/#{message}/) >>> end >>> ====================>>> >>> And a scenario like this: >>> >>> ====================>>> When I go to log in >>> And I enter Username: david >>> And I enter Password: webrat rules >>> And I submit my credentials >>> Then I should see Welcome david >>> ====================>>> >>> And what webrat does is grabs the html from the response in the >>> visits >>> method, modifies that html with your data in the fills_in method, >>> and >>> then yanks the data from the html and submits it with the >>> clicks_button method. What this means is that if the form isn''t >>> aligned with the fields you''re submitting, you''ll get a failure. >>> >>> How clean is that? >>> _______________________________________________ >>> rspec-users mailing list >>> rspec-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/rspec-users >> >> >> Um... Sweetness!!! How is it for speed? There is a ton of stuff we >> would love to move out of Selenium due to issues with sessions, >> flash, >> and database persistance. > > Obviously it''s much faster than running stuff in a browser. I have > nothing else to measure it against and haven''t done any real > benchmarking, but it is perceivably much faster than running selenium. > > What I''d recommend - where I''m headed as the opportunity presents > itself - is to have in-memory and in-browser implementations of the > same steps. Then you can run them in-memory most of the time and skip > the javascript, but run them in-browser once in a while - like before > commits - to make sure the javascript is all working too. >Not a bad idea. Our tests are in terrible condition due to the length of time it takes to run selenium.> There''s also an interesting javascript testing framework that I think > shows a lot of promise called crosscheck > (http://www.thefrontside.net/crosscheck). It''s basically got it''s own > browser-less javascript engines and runs in-memory. Last I heard it > was mostly dead-on, but that it missed some things. The guy who told > me that said he was running selenium for a few things that crosscheck > didn''t do right, but I don''t know what those are :) > > I did toy with it for a minute, and it''s blazingly fast. I just > haven''t made a commitment to use it in anger yet - but I''m looking > forward to the opportunity to. >I''ll look into it. Thanks for the pointer!> Cheers, > David > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users
Thanks for the endorsement, David. Just wanted to mention that I''d be very interested in optimizing the internals of Webrat for performance down the road if necessary. We''re not tied to Hpricot by the API, so swapping it out for a faster parser would be fine if it produces a speed gain. My first goal was to nail the approach and get a good API. In the mean time, in my experience it is quite fast, and I''m personally quite picky about test speed. When I''ve got more concrete benchmarks on a larger test suite, I''ll post them. -- Bryan Helmkamp http://brynary.com -- My blog
On Dec 19, 2007, at 10:42 PM, David Chelimsky wrote:> On Dec 20, 2007 12:25 AM, James Deville <james.deville at gmail.com> > wrote: >> >> >> On Dec 19, 2007, at 10:16 PM, David Chelimsky wrote: >> >>> On Dec 19, 2007 11:50 PM, James Deville <james.deville at gmail.com> >>> wrote: >>>> >>>> >>>> On Dec 19, 2007, at 9:44 PM, David Chelimsky wrote: >>>> >>>>> On Dec 19, 2007 11:40 PM, James Deville <james.deville at gmail.com> >>>>> wrote: >>>>>> >>>>>> >>>>>> On Dec 19, 2007, at 9:38 PM, David Chelimsky wrote: >>>>>> >>>>>>> On Dec 19, 2007 11:34 PM, James Deville >>>>>>> <james.deville at gmail.com> >>>>>>> wrote: >>>>>>>> Yeah, had a slight email conversation with David C about that >>>>>>>> in >>>>>>>> regards to bug #188. I am wondering why we don''t standardize >>>>>>>> it, ya >>>>>>>> know convention over configuration and all. >>>>>>> >>>>>>> Because I think it''s premature to call anything related to story >>>>>>> runner a convention. I actually organize them differently from >>>>>>> what >>>>>>> many are calling convention, and my way is not necessarily >>>>>>> "right" >>>>>>> or >>>>>>> "better." Let''s wait a while on this. We''ll get there. >>>>>>> _______________________________________________ >>>>>>> rspec-users mailing list >>>>>>> rspec-users at rubyforge.org >>>>>>> http://rubyforge.org/mailman/listinfo/rspec-users >>>>>> >>>>>> >>>>>> Good enough for me. I just wanted a reason. May I ask how you set >>>>>> them >>>>>> up? >>>>> >>>>> Sure. >>>>> >>>>> stories/ >>>>> stories/helper.rb >>>>> stories/steps/ (steps go in here - that seems to be the >>>>> "convention") >>>>> stories/stuff_related_to_one_feature >>>>> stories/stuff_related_to_another_feature >>>>> stories/stuff_related_to_yet_another_feature >>>>> >>>>> So in this case, the only thing that would be consistent across >>>>> projects would be helper.rb and the steps directory. Even that >>>>> should >>>>> probably be called step_definitions or something. I''m not sure. >>>>> >>>>> Anyhow - that''s where I''m at. How about you? >>>>> _______________________________________________ >>>>> rspec-users mailing list >>>>> rspec-users at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/rspec-users >>>> >>>> I''ve been doing a feature per file. So I have: >>>> >>>> stories/ >>>> stories/helper.rb >>>> stories/steps/ >>>> stories/stories/ >>>> >>>> So in stories/stories I have my stories with multiple scenarios per >>>> story. >>>> >>>> I''m definitely seeing your point, and the reason for leaving it as >>>> is. >>>> >>>> Do you use selenium or anything like that via the stories? If not, >>>> what kinds of things do you test with stories? >>> >>> I''ve got a bunch of selenium tests being driven by spec/ui. I >>> haven''t >>> converted them to stories yet. >>> >>> Mostly I''ve been using webrat, Bryan Helmkamp''s awesome >>> Hpricot-wrapping goodness. It doesn''t cover javascript, but the apps >>> I''ve been working on have been very vanilla with respect to ajax, so >>> I''ve been satisfied with have_rjs. >>> >>> Anyhow, webrat does something really cool - it ties your form >>> submissions to the html of the rendered form. So you make steps like >>> this: >>> >>> ====================>>> When "I go to log in" do >>> visits "/session/new" >>> end >>> >>> When "I enter $label: $value" do |label, value| >>> fills_in label, :with => value >>> end >>> >>> When "I submit my credentials" do >>> clicks_button >>> end >>> >>> Then "I should see $message" do |message| >>> response.should have_text(/#{message}/) >>> end >>> ====================>>> >>> And a scenario like this: >>> >>> ====================>>> When I go to log in >>> And I enter Username: david >>> And I enter Password: webrat rules >>> And I submit my credentials >>> Then I should see Welcome david >>> ====================>>> >>> And what webrat does is grabs the html from the response in the >>> visits >>> method, modifies that html with your data in the fills_in method, >>> and >>> then yanks the data from the html and submits it with the >>> clicks_button method. What this means is that if the form isn''t >>> aligned with the fields you''re submitting, you''ll get a failure. >>> >>> How clean is that? >>> _______________________________________________ >>> rspec-users mailing list >>> rspec-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/rspec-users >> >> >> Um... Sweetness!!! How is it for speed? There is a ton of stuff we >> would love to move out of Selenium due to issues with sessions, >> flash, >> and database persistance. > > Obviously it''s much faster than running stuff in a browser. I have > nothing else to measure it against and haven''t done any real > benchmarking, but it is perceivably much faster than running selenium. > > What I''d recommend - where I''m headed as the opportunity presents > itself - is to have in-memory and in-browser implementations of the > same steps. Then you can run them in-memory most of the time and skip > the javascript, but run them in-browser once in a while - like before > commits - to make sure the javascript is all working too. > > There''s also an interesting javascript testing framework that I think > shows a lot of promise called crosscheck > (http://www.thefrontside.net/crosscheck). It''s basically got it''s own > browser-less javascript engines and runs in-memory. Last I heard it > was mostly dead-on, but that it missed some things. The guy who told > me that said he was running selenium for a few things that crosscheck > didn''t do right, but I don''t know what those are :) > > I did toy with it for a minute, and it''s blazingly fast. I just > haven''t made a commitment to use it in anger yet - but I''m looking > forward to the opportunity to. > > Cheers, > David > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-usersCan you show me how you set up for webrat? I can''t get the story runner to recognize webrat. I have require ''webrat'' in helper.rb, and then I''m trying to use visits, and it fails! JD
On Dec 21, 2007 6:16 PM, James Deville <james.deville at gmail.com> wrote:> Can you show me how you set up for webrat? I can''t get the story > runner to recognize webrat. I have require ''webrat'' in helper.rb, and > then I''m trying to use visits, and it fails!Actually, you shouldn''t need to add an include to helper.rb because Webrat''s init.rb loads it automatically when in the test environment. Can you email me off-list with a stacktrace? Thanks, -- Bryan Helmkamp http://brynary.com -- My blog