We will be using Cucumber for a fairly large project. Are there any areas that cucumber is lacking when there are many scenarios? I believe the best candidate for showing our client the scenario results is the html output. It strikes me that that becomes fairly cumbersome to check with many scenarios. Methinks some javascript code to hide all passing scenarios / features would be useful. Is there any work being done on this? In which areas could we help cucumber accommodate for large projects? Tom.
Tom ten Thij wrote:> We will be using Cucumber for a fairly large project. Are there any > areas that cucumber is lacking when there are many scenarios? > > I believe the best candidate for showing our client the scenario > results is the html output. It strikes me that that becomes fairly > cumbersome to check with many scenarios. Methinks some javascript code > to hide all passing scenarios / features would be useful. Is there any > work being done on this? > > In which areas could we help cucumber accommodate for large projects? > > Tom.Try the progress formatter. $ cucumber -f progress -r features features -- Posted via http://www.ruby-forum.com/.
Hi Tom, We have written an extension that counts your tests and provides a link(s) to the failure backtrace in the Cucumber HTML. There seem to be issues when Cucumber writes to the HTML over around 800 steps, so we are waiting for version 0.2. Aidy On 26/02/2009, Tom ten Thij <mail at tomtenthij.nl> wrote:> We will be using Cucumber for a fairly large project. Are there any > areas that cucumber is lacking when there are many scenarios? > > I believe the best candidate for showing our client the scenario > results is the html output. It strikes me that that becomes fairly > cumbersome to check with many scenarios. Methinks some javascript code > to hide all passing scenarios / features would be useful. Is there any > work being done on this? > > In which areas could we help cucumber accommodate for large projects? > > Tom. > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
On 26 Feb 2009, at 15:33, Tom ten Thij wrote:> We will be using Cucumber for a fairly large project. Are there any > areas that cucumber is lacking when there are many scenarios? > > I believe the best candidate for showing our client the scenario > results is the html output. It strikes me that that becomes fairly > cumbersome to check with many scenarios. Methinks some javascript code > to hide all passing scenarios / features would be useful. Is there any > work being done on this? > > In which areas could we help cucumber accommodate for large projects? > > Tom.Hi Tom, At Songkick.com, we currently have 635 scenarios in our Cucumber tests, and while we''ve had a few issues, the ROI has definitely been worth it. At the moment, IMO, the tools for feeding back the features to non- technical people are pretty immature. My colleague Dan Lucraft wrote a tool which produces a nicely-formatted PDF document from your features folder[1] which is great, but won''t work with the new version (0.2) of Cucumber when it''s released as the API against which such formatters are written has undergone some significant changes for the new version. Once this stuff stabilises into the 0.2 release I expect more effort will be put into such tools, as the changes Aslak has made will make it much easier to write them. Check out the brave vision for the new HTML formatter: http://rspec.lighthouseapp.com/projects/16211/tickets/178-design-the-html-markup-for-cucumber-02 Other than that, you''ll be likely to hit some issues just to do with managing the large number of specifications, but to be honest this has surprised me by being much easier than expected. The amount of code you actually write in the step definitions is minimal, especially if you''re using Rails and something like factory_girl, so with a bit of regular re-factoring it''s really easy to stay on top of it. We made a call early on to split our features folder into sub-folders named after the type of user, which has worked well for us: e.g. features/visitor/view_homepage.feature features/user/change_password.feature features/admin/list_users.feature That''s helped us to keep things tidy, but there are obviously other ways to slice things up too. Are you planning to test the browser too (with selenium / watir / etc) or just from the HTTP requests down? Assuming you''re building a web app of course... :) [1]http://www.daniellucraft.com/blog/2009/01/features_report/ Matt Wynne http://blog.mattwynne.net http://www.songkick.com
Speed will possibly become a big issue once you get above 1-2k steps or so. Testjour was written to solve this: http://github.com/brynary/testjour/tree/master While I haven''t been able to use it myself, since I work mostly from home, it apparently radically reduces the time it takes to run features. The only other tip I have for using cucumber in big projects is to make sure that you stay on top of small changes to code that don''t break features, but over time end up changing what is happening to the point that, while the test passes, it is no longer testing what you think it is testing. So, like Matt said, it''s a good idea to keep on top of things. However, this is a minor issue and not cucumber specific. Best, M On Mon, Mar 2, 2009 at 5:17 PM, Matt Wynne <matt at mattwynne.net> wrote:> > On 26 Feb 2009, at 15:33, Tom ten Thij wrote: > > We will be using Cucumber for a fairly large project. Are there any >> areas that cucumber is lacking when there are many scenarios? >> >> I believe the best candidate for showing our client the scenario >> results is the html output. It strikes me that that becomes fairly >> cumbersome to check with many scenarios. Methinks some javascript code >> to hide all passing scenarios / features would be useful. Is there any >> work being done on this? >> >> In which areas could we help cucumber accommodate for large projects? >> >> Tom. >> > > Hi Tom, > > At Songkick.com, we currently have 635 scenarios in our Cucumber tests, and > while we''ve had a few issues, the ROI has definitely been worth it. > > At the moment, IMO, the tools for feeding back the features to > non-technical people are pretty immature. My colleague Dan Lucraft wrote a > tool which produces a nicely-formatted PDF document from your features > folder[1] which is great, but won''t work with the new version (0.2) of > Cucumber when it''s released as the API against which such formatters are > written has undergone some significant changes for the new version. > > Once this stuff stabilises into the 0.2 release I expect more effort will > be put into such tools, as the changes Aslak has made will make it much > easier to write them. Check out the brave vision for the new HTML formatter: > > http://rspec.lighthouseapp.com/projects/16211/tickets/178-design-the-html-markup-for-cucumber-02 > > Other than that, you''ll be likely to hit some issues just to do with > managing the large number of specifications, but to be honest this has > surprised me by being much easier than expected. The amount of code you > actually write in the step definitions is minimal, especially if you''re > using Rails and something like factory_girl, so with a bit of regular > re-factoring it''s really easy to stay on top of it. We made a call early on > to split our features folder into sub-folders named after the type of user, > which has worked well for us: > > e.g. > > features/visitor/view_homepage.feature > features/user/change_password.feature > features/admin/list_users.feature > > That''s helped us to keep things tidy, but there are obviously other ways to > slice things up too. Are you planning to test the browser too (with selenium > / watir / etc) or just from the HTTP requests down? Assuming you''re building > a web app of course... :) > > [1]http://www.daniellucraft.com/blog/2009/01/features_report/ > > Matt Wynne > http://blog.mattwynne.net > http://www.songkick.com > > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20090303/921acf86/attachment-0001.html>
Hi Matt/Dan 2009/3/2 Matt Wynne <matt at mattwynne.net>:> At the moment, IMO, the tools for feeding back the features to non-technical > people are pretty immature. My colleague Dan Lucraft wrote a tool which > produces a nicely-formatted PDF document from your features folder[1] which > is great, but won''t work with the new version (0.2) of Cucumber when it''s > released as the API against which such formatters are written has undergone > some significant changes for the new version. >> [1]http://www.daniellucraft.com/blog/2009/01/features_report/To me the user\customer should be involved in the creation of the stories and the acceptance criteria and by producing a pdf later in the cycle for those stakeholders to read and understand may be argued as something that is not Story-Driven-Development and more like comprehendible automated tests. Aidy
On 3 Mar 2009, at 10:06, aidy lewis wrote:> Hi Matt/Dan > > 2009/3/2 Matt Wynne <matt at mattwynne.net>: > >> At the moment, IMO, the tools for feeding back the features to non- >> technical >> people are pretty immature. My colleague Dan Lucraft wrote a tool >> which >> produces a nicely-formatted PDF document from your features >> folder[1] which >> is great, but won''t work with the new version (0.2) of Cucumber >> when it''s >> released as the API against which such formatters are written has >> undergone >> some significant changes for the new version. >> > >> [1]http://www.daniellucraft.com/blog/2009/01/features_report/ > > To me the user\customer should be involved in the creation of the > stories and the acceptance criteria and by producing a pdf later in > the cycle for those stakeholders to read and understand may be argued > as something that is not Story-Driven-Development and more like > comprehendible automated tests.Should I expect a visit from the Extreme Programing Police Aidy? ;) I take your point, and I always try to pair with stakeholders when creating new features. I think it''s also important though that when stakeholders are considering adding new features or changing existing ones that the current set of Running Tested Features is as visible as possible to them. This is one of the advantages of something like Fitnesse, where each feature is only a URL away. Matt Wynne http://blog.mattwynne.net http://www.songkick.com
On Tue, Mar 3, 2009 at 2:06 AM, aidy lewis <aidy.lewis at googlemail.com> wrote:> Hi Matt/Dan > > 2009/3/2 Matt Wynne <matt at mattwynne.net>: > >> At the moment, IMO, the tools for feeding back the features to non-technical >> people are pretty immature. My colleague Dan Lucraft wrote a tool which >> produces a nicely-formatted PDF document from your features folder[1] which >> is great, but won''t work with the new version (0.2) of Cucumber when it''s >> released as the API against which such formatters are written has undergone >> some significant changes for the new version. >> > >> [1]http://www.daniellucraft.com/blog/2009/01/features_report/ > > To me the user\customer should be involved in the creation of the > stories and the acceptance criteria and by producing a pdf later in > the cycle for those stakeholders to read and understand may be argued > as something that is not Story-Driven-Development and more like > comprehendible automated tests.Not quite. Some customers don''t want to sit down next to you as you write the stories. In that situation, we''ll talk about features, I take notes, then I write up the features and ship them via email or wiki. They''re still involved in the creation of stories and acceptance criteria, they''re just not present when those criteria get entered into cucumber features. Pat
At 16:41 -0800 3/3/09, Pat Maddox wrote:> Not quite. Some customers don''t want to sit down next to you as you > write the stories. In that situation, we''ll talk about features, I > take notes, then I write up the features and ship them via email or > wiki. They''re still involved in the creation of stories and > acceptance criteria, they''re just not present when those criteria get > entered into cucumber features.In the Ontiki project, I am acting as my own initial customer and the primary customers (for at least a while) will be Ruby programmers. So, I''m playing with the use of a pre-processor which allows use of eRuby (for selection, repetition, and variable substitution), line folding, etc. This DRYs out my editable files (fe/*.fe), while generating readable and Gherkin-compliant feature (*.feature) files. This would not work well for a highly interactive situation, but it might work in a situation like the one you describe. It may also give us some useful feedback about useful extensions for Gherkin. -r -- http://www.cfcl.com/rdm Rich Morin http://www.cfcl.com/rdm/resume rdm at cfcl.com http://www.cfcl.com/rdm/weblog +1 650-873-7841 Technical editing and writing, programming, and web development
Let me share my experience. On my side we do produce 3 documents before starting with rspec. 1- A web designer creates a design for a screen in conjunction with the marketing team and the analyst assigned to the this feature. The marketing dep, acts as the customer. 2-The analyst produce a "word" document containing the stories and scenarios attached to this screen. This work is done again in conjunction with the web designer cause she is also an ergonomist and the Marketing dep. The analyst has to have the approval of these stories-scenarios before he can share this info to the ruby programmers. 3- The analyst produce also a little spreadsheet defining all labels, fields and any specifities of the screen to eleminates any "second guess" of the ruby dev. As soon the ruby dep got those docs they start creating specs for this. For now we are not using cucumber yet. It is already a huge challenge for me to convience legacy programmers to learn ruby, rails OOP in the BDD way. In short term I plan to experience it. For now we are not having any tool to transfer stories-scenarios from our word doc to rspec file. If any of you have something for that, let me know. R?mi Gagnon On Tue, Mar 3, 2009 at 8:51 PM, Rich Morin <rdm at cfcl.com> wrote:> At 16:41 -0800 3/3/09, Pat Maddox wrote: > > Not quite. Some customers don''t want to sit down next to you as you > > write the stories. In that situation, we''ll talk about features, I > > take notes, then I write up the features and ship them via email or > > wiki. They''re still involved in the creation of stories and > > acceptance criteria, they''re just not present when those criteria get > > entered into cucumber features. > > In the Ontiki project, I am acting as my own initial customer and the > primary customers (for at least a while) will be Ruby programmers. So, > I''m playing with the use of a pre-processor which allows use of eRuby > (for selection, repetition, and variable substitution), line folding, > etc. This DRYs out my editable files (fe/*.fe), while generating > readable and Gherkin-compliant feature (*.feature) files. > > This would not work well for a highly interactive situation, but it > might work in a situation like the one you describe. It may also give > us some useful feedback about useful extensions for Gherkin. > > -r > -- > http://www.cfcl.com/rdm Rich Morin > http://www.cfcl.com/rdm/resume rdm at cfcl.com > http://www.cfcl.com/rdm/weblog +1 650-873-7841 > > Technical editing and writing, programming, and web development > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20090303/e8cf2454/attachment.html>
On Tue, Mar 3, 2009 at 5:06 AM, aidy lewis <aidy.lewis at googlemail.com> wrote:> > To me the user\customer should be involved in the creation of the > stories and the acceptance criteria and by producing a pdf later in > the cycle for those stakeholders to read and understand may be argued > as something that is not Story-Driven-Development and more like > comprehendible automated tests.There''s more than one way to skin a ca...a cucumber. In my organization, people are definitely invested in what I intend to build, and we do discuss requirements. But if I tried to tell them, "All right, I need all of us to sit down and write scenarios in a particular text file format for all of our critical path success and failure cases," I would just get blank stares. They wouldn''t Get It. And I don''t see it as a necessary part of my job to drag everyone else into this culture, if I can just document my best understanding and then show them the results for feedback. -- Have Fun, Steve Eley (sfeley at gmail.com) ESCAPE POD - The Science Fiction Podcast Magazine http://www.escapepod.org
At 21:21 -0500 3/3/09, Remi Gagnon wrote:> > For now we are not having any tool to transfer > stories-scenarios from our word doc to rspec file. > If any of you have something for that, let me know.I don''t have any software to offer, but I can offer a suggestion regarding process. If the "word" document is a Microsoft Word document, and the format is stable and well-defined, you may be able to have Word export a document in an intermediate format (eg, HTML, XML) that can be parsed and used as input by a mechanized process. -r -- http://www.cfcl.com/rdm Rich Morin http://www.cfcl.com/rdm/resume rdm at cfcl.com http://www.cfcl.com/rdm/weblog +1 650-873-7841 Technical editing and writing, programming, and web development