So... In the recent five-hundred-and-forty-seventh iteration of the "What''s the point of mocks?" discussion on this list, David Chelimsky ended a message with:> [ . . . ] Really, I think > that''s what we should all be striving for. Not so much "should I use > mocks or not," but "when should I choose to use them?"And here... Here you made me really pause and think. David: Now that you''re a Pragmatic author -- have you read Andy''s _Pragmatic Thinking and Learning?_ Damn good book. The most valuable part of it to me was in the first couple of chapters, which laid out the Dreyfus model of skill acquisition. Having learned about it a few months ago, I''m starting to see it everywhere now. I don''t intend to rehash the whole theory, but it basically says that the development of a skill (_any_ skill) follows a predictable pattern: 1. Novice (needs clear checklists or recipes; not ready for theory or problem-solving) 2. Advanced Beginner (can solve typical problems and alter the recipe within parameters) 3. Competent (understands the theory behind the recipes and can act creatively) 4. Proficient (acts on principles rather than recipes; using the skill is nearly automatic) 5. Expert (defines and innovates the field; works on intuition rather than rules)>From reading a fair amount on TDD/BDD -- including the Rspec Book -- Ithink there''s widespread consensus that learning TDD/BDD is a totally separate skill from programming itself. One can be an expert programmer with decades of experience, but still be at zero when it comes to writing and using tests. (Or even an "anti-learner," hostile to the concept itself.) My thesis in this message is that there is a hurdle built into learning the skill of TDD/BDD with RSpec: that there''s a steep curve somewhere in the early part of stage 2 -- "Advanced Beginner" -- that makes it more difficult than it ought to be to ascend to competence. The hurdle is that some easy things are easy, but other easy things are very complicated. Not just hard to do, but hard to *learn.* Hard to understand why stuff works. The tasks where the curve begins can be identified by following the high-level trends of conversations on this list. Usually it''s on Rails. Models are easy enough; heck, models are fun. But then we try to do controllers, and... *Boom.* Brick wall. You can run the generator, but... What is it doing? How do you change it? Let''s say I''m a novice to Rails, too: what the hell *is* that routing spec? That last sentence is the sharpest way I can think to phrase the problem. "What the hell *is* that spec?" Rails is easy to start rolling with for the default use cases, but moderately hard to _understand_ when you''re ready to do more complex things. Its nuances are scattered in a lot of places, and some of the connections are invisible. RSpec on Rails, even in the default cases, exposes more of those nuances and *requires* you to understand them. ActiveRecord stands on its own, so models are fine (and views are mostly just tedious) but you can''t even begin to understand that generated controller spec until you understand Rails and all of its nuances better. And if you''re trying to learn both at once? And you don''t want to go deeper into your Rails code until your specs pass? ...Learning deadlock. The failure to understand one keeps you from making progress with the other. In the ideal case the specs would help to teach you Rails, but in practice it doesn''t work that way. They specs are too opaque without Rails knowledge. When I set out to consciously learn this stuff, I was determined to really learn it. I didn''t want to type any code I couldn''t understand. So I ran the RSpec scaffold in a ''dummy'' app and used it only as a reference to build my own by hand. The first controller spec I tried was for a nested resource with parent and child objects. (Bad choice perhaps.) It took me almost two days to write that first full spec with all the mocks and stubs, and get it to pass on every action. I was on contract and even to me that felt hard to justify. And then I had to deal with authentication and filters, and then I felt the pain *again.* I''m a stubborn ass, so I plodded through. Eventually I even figured out what I was doing. (I think.) But I believe I have seen people run into this curve and view it as a brick wall. Many give up, sometimes concluding that RSpec is beyond them -- after all, everyone says it''s "easy," so if they can''t write a controller spec quickly or see the immediate value of this stuff, they must be too dumb for it. It isn''t just Rails either. That''s the most common entry point, and controllers seem to be the most common wall. But I hit a wall again when I needed to write an adapter for a complex SOAP interface, building some abstractions on top of soap4r. "Where do I start?" was quite hard for me. I fished around for blog posts and examples to give me some precedent. Eventually I figured out a reasonably efficient way to mock out the SOAP responses without building a whole fake server, and I think I''m better for the exercise. But as soon as I got beyond "this is all my code, so I understand it," I hit that curve again. ...This is a real problem. It''s not entirely the fault of RSpec (in which I include RSpec-Rails) as a framework. Some of it''s built into the ideology: some types of systems really are hard to specify or interact with, and we don''t acknowledge that enough. Or we say "If it''s painful to specify you''re doing it wrong." Not always true, and a dangerous statement for a novice who''s only feeling the sorts of pain everyone feels when they first begin exercising. Some of it''s built into Rails and "Rails-like" MVC frameworks This is off the topic here, but I can''t shake the feeling that if DHH had had RSpec from the very start and applied some of its philosophies faithfully, Rails would be different in some fundamental ways. Instead, some things are coupled in ways that require RSpec-Rails to jump through some weird hoops. It all works, but it isn''t all _simple._ And beginners can''t grok beyond simple. Some of it''s the fault of the community. The problems I see here are subtle and certainly not malignant. This list, for instance, is excellent for its purpose -- it has probably the highest signal-to-noise ratio of any active technical list I follow, and everyone''s eager to help at every level. I''ve never seen anyone sniped at for asking beginner questions. That''s great. But the questions and the answers are both coming from such a broad range of skill levels, and sometimes that makes for a disconnect that''s tough for learning. That''s what I saw in the quote above that triggered this weird polemic from me. In the Dreyfus model, there''s no question that many people here would be classed as Expert -- almost by definition in David''s or Aslak''s case. Me, I see myself on the level of "mid-to-high Competent, struggling toward Proficient." Whether that''s accurate or not, I''d be the wrong person to say. Studies show that most incompetent people are very bad at gauging their incompetence, and I might be one of them. Studies also show that Experts are generally not good at teaching Novices, because they no longer _think_ in terms of rules and recipes. Making an Expert follow a recipe is terribly frustrating and a waste of everyone''s time. OTOH telling Novice "use your own judgment" is even more frustrating -- and may result in the loss of that Novice. The Novice doesn''t _have_ his or her own judgment yet. When we say things like, "I think that''s what we should be striving for -- not ''should I use mocks or not,'' but ''when should I choose to use them?''" ...That problem is, if we buy into that Dreyfus hierarchy, that is a harmful statement to deliver to an audience that''s broadly distributed along the skill curve. The question of "When should I use X?" is an appropriate question for someone at Competent or Proficient levels. But it would be very challenging for most Advanced Beginners to suggest an answer, and a Novice wouldn''t be able to answer it at all. (As for Experts: they probably don''t even need to think about it. They just know.) We need more awareness of this. It''s not a bad thing, and I think there are probably more benefits in having a skill-level-distributed list like this than to set up some sort of "beginner''s" or "learner''s" list or anything. (For one thing, most programmers who''d benefit would have too much ego to join it.) We just need to recognize -- and I include myself, as *I* spun off the theory tangent in that last thread -- that a Novice question needs a Novice or Advanced Beginner answer, and that saying "It''s really more complicated than that" isn''t helpful to certain classes of question. Community-wise: I also think we need better learning resources. The RSpec Book is great. Followed faithfully, I have a hunch it can bring an Advanced Beginner up to Competent with remarkable efficiency. I think it could work as well for *some* Novices. But not all. It depends on their thinking patterns, and whether they need or want to be grounded in theory before they get going with the recipes. The Mastermind puzzle is a good starting exercise, but there''s too much explanation going on before the rest of the practical reference begins. (Again, for some people. This was _my_ impression. For others it might be just right.) The RSpec.info site is...problematic. It''s quirky in its information structure, and I''ve gotten lost sometimes clicking around to find something I *knew* was there. I don''t feel comfortable pointing people to it as a start-from-zero starting point. The explanations are clear and up-to-date, but it''s hard to find a handhold. The Peepcode videos worked better for me, though they were a bit too long for my ADD-driven brain. At one point I was going to write a "Getting Started With RSpec" sort of primer. It was going to be informed by my own growth curve, and truly start with "Step by step, okay, just do this." No generators, write the code by hand, but get actual specs working quickly in actual projects. Of course, my "do this" would be somewhat different from David''s -- I''ve been vocal enough about doing a lot of my tests a different way -- so I felt a bit awkward about breaking from orthodoxy when I don''t feel like an expert myself. But I felt bold. I started it, got a little distracted and overplanned the demo app (a wiki to host the primer itself, but I wanted some new sorts of features) and it''s languishing. This thread makes me wonder if I should put a little more foreground time into it. Not the stupid wiki, but the document. I''m rambling now, and I''ve spent *way* too much of my workday on this message, so I''ll just summarize: * Some easy things are complicated in RSpec; * Incomplete understanding of RSpec and a framework being spec''ed can create learning deadlock; * Novices need different answers than competent practitioners; * The community as a whole should be aware and respectful of the growth curve difficulty. If anyone had the patience to read this far, any thoughts? Am I identifying real issues, or am I just full of hot air? -- Have Fun, Steve Eley (sfeley at gmail.com) ESCAPE POD - The Science Fiction Podcast Magazine http://www.escapepod.org
On Thu, Jun 11, 2009 at 2:48 PM, Stephen Eley<sfeley at gmail.com> wrote:> So... ?In the recent five-hundred-and-forty-seventh iteration of the > "What''s the point of mocks?" discussion on this list, David Chelimsky > ended a message with: > >> [ . . . ] ? Really, I think >> that''s what we should all be striving for. Not so much "should I use >> mocks or not," but "when should I choose to use them?" > > And here... ?Here you made me really pause and think. > > David: Now that you''re a Pragmatic author -- have you read Andy''s > _Pragmatic Thinking and Learning?_ ?Damn good book. ?The most valuable > part of it to me was in the first couple of chapters, which laid out > the Dreyfus model of skill acquisition. ?Having learned about it a few > months ago, I''m starting to see it everywhere now. ?I don''t intend to > rehash the whole theory, but it basically says that the development of > a skill (_any_ skill) follows a predictable pattern: > > 1. Novice (needs clear checklists or recipes; not ready for theory or > problem-solving) > 2. Advanced Beginner (can solve typical problems and alter the recipe > within parameters) > 3. Competent (understands the theory behind the recipes and can act creatively) > 4. Proficient (acts on principles rather than recipes; using the skill > is nearly automatic) > 5. Expert (defines and innovates the field; works on intuition rather > than rules)And look at the chart in the section "Is a Ruby Code-base Hard to Understand?" in the article which Martin Fowler just published today http://martinfowler.com/articles/rubyAtThoughtWorks.html It shows a curve of the usage of metaprogramming in Ruby as the experience of the programmer progresses, along with annotations about what the thoughts about the technique during the leaning progression. The progression goes: Attitude Usage Scary and bad low Scary and good at the high end of the sensible range Obvious and good high in the "over usage" range Obvious and bad at the high end of the sensible range The real key to mastery is gaining experience by making mistakes and learning not only techniques, but the trade-offs in when and when not to use them. -- Rick DeNatale Blog: http://talklikeaduck.denhaven2.com/ Twitter: http://twitter.com/RickDeNatale WWR: http://www.workingwithrails.com/person/9021-rick-denatale LinkedIn: http://www.linkedin.com/in/rickdenatale
> I''m rambling now, and I''ve spent *way* too much of my workday on this > message, so I''ll just summarize: > > * Some easy things are complicated in RSpec; > * Incomplete understanding of RSpec and a framework being spec''ed can > create learning deadlock; > * Novices need different answers than competent practitioners; > * The community as a whole should be aware and respectful of the > growth curve difficulty. > > If anyone had the patience to read this far, any thoughts? Am I > identifying real issues, or am I just full of hot air? > > >Hi Stephen, I did in fact read your entire message, but I''m running out of time so I''m afraid I can''t reply with the thoughtful response that it deserves. Here is my brief gut reaction.. I totally agree about your observation WRT the Dreyfus model. I''ve thought about how how it relates to TDD/BDD in the past and have made similar conclusions. I think you are spot on about an expert not being able to really answer a novice''s question very well.. because they are suffering from the "curse of knowledge" and it is hard to remember what it was like without all that knowledge. In general though I don''t know if I really see all of this as a problem specific to RSpec. The problem is that many people coming to RSpec are not just RSpec novices but are in fact TDD novices (and Rails, and Ruby novices too boot at times as well!). TDD, like you said, is a skill apart from programming and is really separate from learning the mechanics of RSpec. A well practiced TDDer without much Ruby experience is in a much better position to use RSpec effectively than an experienced Ruby dev with no TDD experience. So if the real problem is that people don''t know TDD/BDD I think outside of the RSpec community there are a lot of resources to draw on. There are great TDD books in addition to lists dedicated to TDD, presentations on TDD, etc... Perhaps one step would be to link to these resources? I also agree with Rick in that it just takes experience to learn this stuff since the answers to questions are so context specific. Anyways, I gotta run, but I those were some of the thoughts I had. Thanks, Ben
Ben and Rick, Thanks very much to both of you for the encouraging responses. Your reply, Ben, came just in time as I was starting to wonder if I had made a complete and irrevocable ass of myself. On Fri, Jun 12, 2009 at 10:36 AM, Ben Mabey<ben at benmabey.com> wrote:> In general though I don''t know if I > really see all of this as a problem specific to RSpec. ?The problem is that > many people coming to RSpec are not just RSpec novices but are in fact TDD > novices (and Rails, and Ruby novices too boot at times as well!).FWIW, I agree totally. In fact I re-read my message last night, and I want to retract one of my statements for sure: the statement that RSpec makes "some easy things easy, but other easy things very complicated." On deeper thought, I don''t believe this is true. RSpec doesn''t make anything harder than it already is. There''s nothing _wrong_ with RSpec in this sense. I think a more incisive statement would be that with RSpec, some complicated things _feel like they should be easier._ If only because other things feel so easy. And newbies tend to confront those problem areas at a point in the growth curve where the complexity can be misinterpreted as them Just Not Getting It. Some are driven off as a result. You''re right about practice. You''re right that bringing together too many learning curves at once creates interference patterns that shouldn''t be blamed on RSpec. And Rick''s reference to Martin Fowler''s spiked learning curve -- "this is scary, this is nifty, WHOOPS I overdid it, this is okay in its place" -- is an excellent point. But I don''t think it''s totally out-of-bounds to frame it as an "RSpec problem," even if it isn''t RSpec''s fault. The thing is... Well... There are *opportunities* here. If RSpec -- along with its tools and resources and culture -- was a little better at some of this stuff, how much easier would it be to learn *everything else?* How many of us learned BDD at the same time we learned programming? ...Show of hands? Anyone? The only developers who could make that claim would have to be pretty fresh out of the pond. If that does apply to anyone reading this, please speak up -- I''d like to hear your thoughts on how you''re doing. How many of us used TDD or BDD out of the gate when we learned Rails or Ruby? This is plausible, but I get the sense it''s uncommon. Most of us don''t even think to work on good testing practices until we believe we''re getting the rest of the stuff down. I know I didn''t. Formal education in programming doesn''t help. Not by any evidence to my eyes. I''ve recently started seeing someone who''s majoring in CS at a well-known university, and he said the first class he took on software design was in C. Frickin'' *C*. _Then_ they did C++, and then Java, but there was nothing significant about testing. I told him about Cucumber and it made his heart go pitter-patter. The teaching resources for Rails and Ruby don''t help. Just about every book makes a *nod* toward testing -- there''s always a chapter, they always say "do this constantly" -- but the chapter''s inevitably somewhere near the back, after everything else has been introduced. Rails culture is passionate about testing, but most of us don''t start to tap into that culture until we''re at ''Competent'' or above. What if the world were different? What if teaching were different? What if the predominant lessons people used to get started with Rails *did not* start with "Type ''rails'', make a scaffold, watch it fly!" but instead started with, "What''s the most important single piece of information in your app? Okay, great. Write a spec like this. Now open up this file in /app/models.. [...] Your spec passed, you''re cooking! Now let''s spec a view so we can see that information... Now let''s spec a controller to bring them together..." (Even better would be to start with Cucumber, which would take up very little extra overhead in that type of tutorial.) You get the gist. What if learning Rails was *all about* learning BDD? Would it make the Rails stuff easier to understand and develop competence faster? I hypothesize that it would. Some things need to be easier in BDD before we can get there. To me that means things need to be easier in RSpec. I believe my tutorial above would be easy enough to do today on the model side, but it''s just too cumbersome to write a view or controller spec. You lose that "holy crap look how *fast* this is coming!" vibe which is so invigorating when you''re doing something new. We could make that tutorial happen when writing a controller spec takes about as long as writing the controller *code.* There are some mitigators for this -- the Shoulda macros may be great for this now that they work with RSpec -- but even so I think it''s probably a little bit of wading into the swamp. I would like to see that world. I''d like to help *make* it. That college guy I''m dating has asked me about learning Rails -- but I''m planning to talk his ass off about Cucumber and RSpec before I agree to help him with code. I''m hoping he''ll be my experiment to test my hypothesis, and I''ll get a better sense of just how easy or hard it is to learn agile Web development if BDD happens at the ground floor. Given time, I think the patterns facilitated by RSpec could *completely* revolutionize the way software happens everywhere, *if* the next generation of developers come to learn it first and not as an "add-on." And that''s why I''m picking on RSpec. I can''t think of any other technology in a better position to make this happen, with only some moderate shifts of emphasis in both the tools and the culture. Does anyone else think that''d be pretty cool? -- Have Fun, Steve Eley (sfeley at gmail.com) ESCAPE POD - The Science Fiction Podcast Magazine http://www.escapepod.org
Stephen Eley wrote:> Ben and Rick, > > Thanks very much to both of you for the encouraging responses. Your > reply, Ben, came just in time as I was starting to wonder if I had > made a complete and irrevocable ass of myself. > > > On Fri, Jun 12, 2009 at 10:36 AM, Ben Mabey<ben at benmabey.com> wrote: > >> In general though I don''t know if I >> really see all of this as a problem specific to RSpec. The problem is that >> many people coming to RSpec are not just RSpec novices but are in fact TDD >> novices (and Rails, and Ruby novices too boot at times as well!). >> > > FWIW, I agree totally. In fact I re-read my message last night, and I > want to retract one of my statements for sure: the statement that > RSpec makes "some easy things easy, but other easy things very > complicated." > > On deeper thought, I don''t believe this is true. RSpec doesn''t make > anything harder than it already is. There''s nothing _wrong_ with > RSpec in this sense. I think a more incisive statement would be that > with RSpec, some complicated things _feel like they should be easier._ > If only because other things feel so easy. And newbies tend to > confront those problem areas at a point in the growth curve where the > complexity can be misinterpreted as them Just Not Getting It. Some > are driven off as a result. > > You''re right about practice. You''re right that bringing together too > many learning curves at once creates interference patterns that > shouldn''t be blamed on RSpec. And Rick''s reference to Martin Fowler''s > spiked learning curve -- "this is scary, this is nifty, WHOOPS I > overdid it, this is okay in its place" -- is an excellent point. > > But I don''t think it''s totally out-of-bounds to frame it as an "RSpec > problem," even if it isn''t RSpec''s fault. The thing is... Well... > There are *opportunities* here. If RSpec -- along with its tools and > resources and culture -- was a little better at some of this stuff, > how much easier would it be to learn *everything else?* > > How many of us learned BDD at the same time we learned programming? > ...Show of hands? Anyone? The only developers who could make that > claim would have to be pretty fresh out of the pond. If that does > apply to anyone reading this, please speak up -- I''d like to hear your > thoughts on how you''re doing. > > How many of us used TDD or BDD out of the gate when we learned Rails > or Ruby? This is plausible, but I get the sense it''s uncommon. Most > of us don''t even think to work on good testing practices until we > believe we''re getting the rest of the stuff down. I know I didn''t. >When learning a new technology I generally don''t bother with testing let alone TDD (unless I think TDD could help the learning process). It is like spiking and most of the stuff you are creating is throw-away code that won''t need to be maintained. There are times, like this, where testing isn''t needed IMO. But that isn''t really your question is it.. You''re wondering what effect teaching good testing practices up front would bring... I''ve wondered this my self at times (in fact I''m helping prepare an intro rails course now and wresting with this very issue) and it seems like teaching good testing to people who don''t even know OO can be like jumping in to the deep end of the pool.> Formal education in programming doesn''t help. Not by any evidence to > my eyes. I''ve recently started seeing someone who''s majoring in CS at > a well-known university, and he said the first class he took on > software design was in C. Frickin'' *C*. _Then_ they did C++, and > then Java, but there was nothing significant about testing. I told > him about Cucumber and it made his heart go pitter-patter. >I hear this complaint quite often and FWIW it is an untrue stereotype IME. While I was at the university I was taught testing, design patterns, refactoring, Agile methodologies, pair programming, use cases, CI, etc... In fact the first time I heard about TDD was in my first Java class where the professor would TDD all of his work in the lectures (live coding!). We did team projects where we had a CI server setup and we were required to have tests and do pair programming at least part of the time. We also had class projects where different teams would create different components and we had to work out our communication protocols, touch points, and integrate it all. Anyways, I justed wanted to throw that out and say that formal education isn''t totally failing like some people claim it is. Only my "Software Engineering" courses taught and encouraged these practices, but I was still taught them.> The teaching resources for Rails and Ruby don''t help. Just about > every book makes a *nod* toward testing -- there''s always a chapter, > they always say "do this constantly" -- but the chapter''s inevitably > somewhere near the back, after everything else has been introduced. > Rails culture is passionate about testing, but most of us don''t start > to tap into that culture until we''re at ''Competent'' or above. > > What if the world were different? What if teaching were different? > What if the predominant lessons people used to get started with Rails > *did not* start with "Type ''rails'', make a scaffold, watch it fly!" > but instead started with, "What''s the most important single piece of > information in your app? Okay, great. Write a spec like this. Now > open up this file in /app/models.. [...] Your spec passed, you''re > cooking! Now let''s spec a view so we can see that information... Now > let''s spec a controller to bring them together..." > > (Even better would be to start with Cucumber, which would take up very > little extra overhead in that type of tutorial.) >So... From what I have seen people with little experience testing are able to pick up the notion of integration/acceptance testing a lot faster than object level testing. So, I do think having a tutorial drive all the features out with Cucumber or just webrat+some testing framework isn''t far fetched at all. Object level examples are a different story though IME...> You get the gist. What if learning Rails was *all about* learning > BDD? Would it make the Rails stuff easier to understand and develop > competence faster? I hypothesize that it would. > > Some things need to be easier in BDD before we can get there. To me > that means things need to be easier in RSpec. I believe my tutorial > above would be easy enough to do today on the model side, but it''s > just too cumbersome to write a view or controller spec. You lose that > "holy crap look how *fast* this is coming!" vibe which is so > invigorating when you''re doing something new. We could make that > tutorial happen when writing a controller spec takes about as long as > writing the controller *code.* There are some mitigators for this -- > the Shoulda macros may be great for this now that they work with RSpec > -- but even so I think it''s probably a little bit of wading into the > swamp. > > I would like to see that world. I''d like to help *make* it. That > college guy I''m dating has asked me about learning Rails -- but I''m > planning to talk his ass off about Cucumber and RSpec before I agree > to help him with code. I''m hoping he''ll be my experiment to test my > hypothesis, and I''ll get a better sense of just how easy or hard it is > to learn agile Web development if BDD happens at the ground floor. > > Given time, I think the patterns facilitated by RSpec could > *completely* revolutionize the way software happens everywhere, *if* > the next generation of developers come to learn it first and not as an > "add-on." And that''s why I''m picking on RSpec. I can''t think of any > other technology in a better position to make this happen, with only > some moderate shifts of emphasis in both the tools and the culture. > > Does anyone else think that''d be pretty cool? >Yeah, I think having such resources available would be very cool. The most common complaint I hear in IRC rooms is that there is no good sample rails app that makes good use of RSpec and Cucumber. I think there are some examples out there, but an app built as a tutorial would be much more helpful with the proper git tags and such. I mentioned before that I was helping prepare a rails course. This is with my local Ruby group and we have started on a tutorial app: http://github.com/urug/media_lender/tree/master It thought it would be cool to use this project and BDD all the way through and teach it like that. However, I have other commitments so other people in the group will probably do most of the work without BDD... It is something I would like to revisit someday and provide such an app. Have you started on a tutorial app like this yet? -Ben
Just to add a couple cents in...
I''ve been working as either a sysadmin or software developer for the
last 10 years (software development pretty much exclusively for the
last 5 or 6). I started picking up Rails about 2 years ago, and though
I was always aware of TDD - and even occasionally dabbled in it - I
never really embraced it. Not because I didn''t see its importance, but
because the barrier to entry always seemed too high, and schedules
never allowed for it (as a friend of mine is fond of saying, "You mean
we don''t have time to do it right, but we have time to do it over and
over again?").
Right now I''m working on a personal project for which I have no QA
team to watch my back, so instead I''m doggedly using top-down BDD with
Cucumber (and unit tests) to help keep me in line. I think the biggest
hurdle to learning it is this: Pretty much all the tutorials and docs
written on the topic center around some simple example app that''s easy
to test. But in the real world, software is messy, and tends to resist
being tested. I know the traditional counter argument is that this
means your design is flawed, but that isn''t always true. One of the
biggest challenges I faced was how to test authentication in my
application, in which users sign in using their Twitter credentials
via OAuth[1]. Another was in testing my own endpoints that create JSON
that''s rendered via JS in the browser[2]. Maybe Selenium could have
helped here, but creating tests in general was already slowing me to
the point that I''m a good deal behind schedule, and I couldn''t
take
another day or so to learn how to use Selenium with the risk that it
might not really suit my situation for one reason or another - because
software is messy.
One that I haven''t tackled yet but will have to soon is how to test
code that runs as a BackgrounDRb worker.
I came up with solutions for all of this (except for the BDRb stuff),
but I''m not entirely sure of their quality. They certainly feel
hackish, but I can''t put my finger on why. I suspect David Chelimsky
or Ben Mabey or Aslak or any of a hundred or more other people could
have come up with beautiful, elegant solutions to the challenges I
faced, but since none of them were standing over my shoulder I have no
way of knowing, and none of this is covered in the
write-a-blog-in-10-minutes Rails demo.
That has been my challenge in learning and using BDD.
[1] My solution: http://tr.im/oW8y
[2] I defined these two custom steps (JSON_CALLBACK is defined as the
:callback arg I pass in my ''render :json'' calls - but it will
trip the
JSON parser if its there):
Then /^I should get valid JSON$/ do
assert_nothing_raised do
assert JSON.parse(@response.body.gsub(/^#{JSON_CALLBACK}\((.*)\)/,
''\1''))
end
end
Then /^I should find "(.+)" in JSON/ do |xpath|
json_hash = JSON.parse(@response.body.gsub(/^#{JSON_CALLBACK}\((.*)\)/,
''\1''))
xmldoc = Nokogiri::XML(json_hash.to_xml)
xpath.gsub!(''_'', ''-'') # in XML,
underscores will be dashes.
xpath = "./hash" + xpath unless xpath =~ /^\/\//
assert !xmldoc.xpath(xpath).blank?
end
--
Bill Kocik
http://bkocik.net
Bill Kocik wrote:> Just to add a couple cents in... > > I''ve been working as either a sysadmin or software developer for the > last 10 years (software development pretty much exclusively for the > last 5 or 6). I started picking up Rails about 2 years ago, and though > I was always aware of TDD - and even occasionally dabbled in it - I > never really embraced it. Not because I didn''t see its importance, but > because the barrier to entry always seemed too high, and schedules > never allowed for it (as a friend of mine is fond of saying, "You mean > we don''t have time to do it right, but we have time to do it over and > over again?"). > > Right now I''m working on a personal project for which I have no QA > team to watch my back, so instead I''m doggedly using top-down BDD with > Cucumber (and unit tests) to help keep me in line. I think the biggest > hurdle to learning it is this: Pretty much all the tutorials and docs > written on the topic center around some simple example app that''s easy > to test. But in the real world, software is messy, and tends to resist > being tested. I know the traditional counter argument is that this > means your design is flawed, but that isn''t always true. One of the > biggest challenges I faced was how to test authentication in my > application, in which users sign in using their Twitter credentials > via OAuth[1]. Another was in testing my own endpoints that create JSON > that''s rendered via JS in the browser[2]. Maybe Selenium could have > helped here, but creating tests in general was already slowing me to > the point that I''m a good deal behind schedule, and I couldn''t take > another day or so to learn how to use Selenium with the risk that it > might not really suit my situation for one reason or another - because > software is messy. > > One that I haven''t tackled yet but will have to soon is how to test > code that runs as a BackgrounDRb worker. > > I came up with solutions for all of this (except for the BDRb stuff), > but I''m not entirely sure of their quality. They certainly feel > hackish, but I can''t put my finger on why. I suspect David Chelimsky > or Ben Mabey or Aslak or any of a hundred or more other people could > have come up with beautiful, elegant solutionsI''m flattered, but FWIW I am always struggling on how best to test things as well. In cases like you are describing it seems like testing it is often harder than the actual implementation. Making those tests valuable without being too brittle can be challenging.> to the challenges I > faced, but since none of them were standing over my shoulder I have no > way of knowing, and none of this is covered in the > write-a-blog-in-10-minutes Rails demo. > > That has been my challenge in learning and using BDD.Thanks for sharing Bill. I totally understand where you are coming from. IME, it is hard when writing a tutorial or giving a presentation to not teach to the least common denominator in the audience (the beginner). Time constraints usually set in too that don''t allow you to do more advanced topics that would require more explanation. Thats why for most presentations I try to focus more on the process and ideas rather than the example and tools (i.e stuff you can''t learn from a wiki so easily). I do see your point however. I often feel frustrated in other presentations at the simplicity of examples given and wish for more in-depth and challenging examples. Your point is noted: a good tutorial app for BDD should involve harder to test components (web services, asynchronous messaging/processing, caching, etc) since this is what pops up in real projects.> > [1] My solution: http://tr.im/oW8y >This is more or less how I would of done this with Cucumber. And you were complaining about no advanced tutorials. ;) Nice writeup.> [2] I defined these two custom steps (JSON_CALLBACK is defined as the > :callback arg I pass in my ''render :json'' calls - but it will trip the > JSON parser if its there): > > Then /^I should get valid JSON$/ do > assert_nothing_raised do > assert JSON.parse(@response.body.gsub(/^#{JSON_CALLBACK}\((.*)\)/, ''\1'')) > end > end > > Then /^I should find "(.+)" in JSON/ do |xpath| > json_hash = JSON.parse(@response.body.gsub(/^#{JSON_CALLBACK}\((.*)\)/, ''\1'')) > xmldoc = Nokogiri::XML(json_hash.to_xml) > xpath.gsub!(''_'', ''-'') # in XML, underscores will be dashes. > xpath = "./hash" + xpath unless xpath =~ /^\/\// > assert !xmldoc.xpath(xpath).blank? > end >I probably would of done this in RSpec instead of Cucumber. Then I would manually tested in a browser.. since you are going to want to do that anyways. I take this stance quite a bit as I feel adding Selenium (or another automated browser solution) can potentially add much more of a maintenance burden than webrat in :rails mode. Thanks, Ben
Thanks for the encouragement, Ben. :) On Thu, Jun 18, 2009 at 1:36 PM, Ben Mabey <ben at benmabey.com> wrote:>> Then /^I should find "(.+)" in JSON/ do |xpath| >> ?json_hash = JSON.parse(@response.body.gsub(/^#{JSON_CALLBACK}\((.*)\)/, ''\1'')) >> ?xmldoc = Nokogiri::XML(json_hash.to_xml) >> ?xpath.gsub!(''_'', ''-'') # in XML, underscores will be dashes. >> ?xpath = "./hash" + xpath unless xpath =~ /^\/\// >> ?assert !xmldoc.xpath(xpath).blank? >> end >> > > I probably would of done this in RSpec instead of Cucumber. ?Then I would manually tested in a browser.. since you are going to want to do that anyways. ?I take this stance quite a bit as I feel adding Selenium (or another automated browser solution) can potentially add much more of a maintenance burden than webrat in :rails mode.In this particular project, all the JS is written by my partner (I''m completely JS-clueless). Having a bunch of lines in my cucumber tests validating everything that should be in the JSON structure my controllers generate in order for the JS to work (and I have a lot of them) is the best way for me to be able to tell if I''ve broken something (and if I have, in what way I broke it). If the page doesn''t work, I will likely have difficulty figuring out why - unless Cucumber is there to tell me it didn''t find "/response/statusCode" in the JSON like it should have, then I know what to fix. :) Though I suppose there''s probably no reason RSpec couldn''t have done the same thing. I haven''t used it yet in this project. -- Bill Kocik http://bkocik.net
Thank you so much Stephen. I am ruby and rails beginner (created a simple site so far) that decided to dive into rspec and I feel the pains you described. I decided to start new rails app and do it with rspec (and to only test my models). Is there a github project with rspec code that test the models? it will be great to look at the examples before jumping to theory. also, I just bought the book and it looks amazing. Is it possible to only read the rspec parts and ignore the cucumber parts? Thanks again for the detailed post. it really helps me understand what I am getting into.
Thank you so much Stephen. I am ruby and rails beginner (created a simple site so far) that decided to dive into rspec and I feel the pains you described. I decided to start new rails app and do it with rspec (and to only test my models). Is there a github project with rspec code that test the models? it will be great to look at the examples before jumping to theory. also, I just bought the book and it looks amazing. Is it possible to only read the rspec parts and ignore the cucumber parts? Thanks again for the detailed post. it really helps me understand what I am getting into.
test On Jun 12, 9:41?am, Stephen Eley <sfe... at gmail.com> wrote:> Ben and Rick, > > Thanks very much to both of you for the encouraging responses. ?Your > reply, Ben, came just in time as I was starting to wonder if I had > made a complete and irrevocable ass of myself. > > On Fri, Jun 12, 2009 at 10:36 AM, Ben Mabey<b... at benmabey.com> wrote: > > In general though I don''t know if I > > really see all of this as a problem specific to RSpec. ?The problem is that > > many people coming to RSpec are not just RSpec novices but are in fact TDD > > novices (and Rails, and Ruby novices too boot at times as well!). > > FWIW, I agree totally. ?In fact I re-read my message last night, and I > want to retract one of my statements for sure: the statement that > RSpec makes "some easy things easy, but other easy things very > complicated." > > On deeper thought, I don''t believe this is true. ?RSpec doesn''t make > anything harder than it already is. ?There''s nothing _wrong_ with > RSpec in this sense. ?I think a more incisive statement would be that > with RSpec, some complicated things _feel like they should be easier._ > ?If only because other things feel so easy. ?And newbies tend to > confront those problem areas at a point in the growth curve where the > complexity can be misinterpreted as them Just Not Getting It. ?Some > are driven off as a result. > > You''re right about practice. ?You''re right that bringing together too > many learning curves at once creates interference patterns that > shouldn''t be blamed on RSpec. ?And Rick''s reference to Martin Fowler''s > spiked learning curve -- "this is scary, this is nifty, WHOOPS I > overdid it, this is okay in its place" -- is an excellent point. > > But I don''t think it''s totally out-of-bounds to frame it as an "RSpec > problem," even if it isn''t RSpec''s fault. ?The thing is... ?Well... > There are *opportunities* here. ?If RSpec -- along with its tools and > resources and culture -- was a little better at some of this stuff, > how much easier would it be to learn *everything else?* > > How many of us learned BDD at the same time we learned programming? > ...Show of hands? ?Anyone? ?The only developers who could make that > claim would have to be pretty fresh out of the pond. ?If that does > apply to anyone reading this, please speak up -- I''d like to hear your > thoughts on how you''re doing. > > How many of us used TDD or BDD out of the gate when we learned Rails > or Ruby? ?This is plausible, but I get the sense it''s uncommon. ?Most > of us don''t even think to work on good testing practices until we > believe we''re getting the rest of the stuff down. ?I know I didn''t. > > Formal education in programming doesn''t help. ?Not by any evidence to > my eyes. ?I''ve recently started seeing someone who''s majoring in CS at > a well-known university, and he said the first class he took on > software design was in C. ?Frickin'' *C*. ?_Then_ they did C++, and > then Java, but there was nothing significant about testing. ?I told > him about Cucumber and it made his heart go pitter-patter. > > The teaching resources for Rails and Ruby don''t help. ?Just about > every book makes a *nod* toward testing -- there''s always a chapter, > they always say "do this constantly" -- but the chapter''s inevitably > somewhere near the back, after everything else has been introduced. > Rails culture is passionate about testing, but most of us don''t start > to tap into that culture until we''re at ''Competent'' or above. > > What if the world were different? ?What if teaching were different? > What if the predominant lessons people used to get started with Rails > *did not* start with "Type ''rails'', make a scaffold, watch it fly!" > but instead started with, "What''s the most important single piece of > information in your app? ?Okay, great. ?Write a spec like this. ?Now > open up this file in /app/models.. ?[...] ?Your spec passed, you''re > cooking! ?Now let''s spec a view so we can see that information... ?Now > let''s spec a controller to bring them together..." > > (Even better would be to start with Cucumber, which would take up very > little extra overhead in that type of tutorial.) > > You get the gist. ?What if learning Rails was *all about* learning > BDD? ?Would it make the Rails stuff easier to understand and develop > competence faster? ?I hypothesize that it would. > > Some things need to be easier in BDD before we can get there. ?To me > that means things need to be easier in RSpec. ?I believe my tutorial > above would be easy enough to do today on the model side, but it''s > just too cumbersome to write a view or controller spec. ?You lose that > "holy crap look how *fast* this is coming!" vibe which is so > invigorating when you''re doing something new. ?We could make that > tutorial happen when writing a controller spec takes about as long as > writing the controller *code.* ?There are some mitigators for this -- > the Shoulda macros may be great for this now that they work with RSpec > -- but even so I think it''s probably a little bit of wading into the > swamp. > > I would like to see that world. ?I''d like to help *make* it. ?That > college guy I''m dating has asked me about learning Rails -- but I''m > planning to talk his ass off about Cucumber and RSpec before I agree > to help him with code. ?I''m hoping he''ll be my experiment to test my > hypothesis, and I''ll get a better sense of just how easy or hard it is > to learn agile Web development if BDD happens at the ground floor. > > Given time, I think the patterns facilitated by RSpec could > *completely* revolutionize the way software happens everywhere, *if* > the next generation of developers come to learn it first and not as an > "add-on." ?And that''s why I''m picking on RSpec. ?I can''t think of any > other technology in a better position to make this happen, with only > some moderate shifts of emphasis in both the tools and the culture. > > Does anyone else think that''d be pretty cool? > > -- > Have Fun, > ? ?Steve Eley (sfe... at gmail.com) > ? ?ESCAPE POD - The Science Fiction Podcast Magazine > ? ?http://www.escapepod.org > _______________________________________________ > rspec-users mailing list > rspec-us... at rubyforge.orghttp://rubyforge.org/mailman/listinfo/rspec-users
Thank you so much Stephen. I am ruby and rails beginner (created a simple site so far) that decided to dive into rspec and I feel the pains you described. I decided to start new rails app and do it with rspec (and to only test my models). is there a github project with rspec code that test the models? it will be great to look at the examples before jumping to theory. Also, I just bought ''the rspec book''. Can I only read the rspec parts and skip the cucumber parts or it doesn''t fit into the books structure and I must read it from start to finish? Thanks again for the detailed post. it really helps me understand what I am getting into.