Having used JUnit and Test::Unit, I''m quite used to having the ability to insert a failure message, which helps when tests fail. For instance, the example RSpec that is generated for a model class specifies that the model class is valid. Assuming this were supposed to be true, and it failed, I''ve now got to duplicate the code in the test in order to find out why it wasn''t valid. Whereas if I were writing the same code in Test::Unit, I might write: assert model.valid?, "Expected model to be valid, but found these errors: #{model.errors}" This means that when the model validation fails, I know /why/. I don''t see an easy way to include these sorts of messages in RSpec, which seems likely to cause me to waste time on test failures. Am I missing something? How are experienced RSpec users resolving this? Thanks, - Geoffrey -- Geoffrey Wiseman -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/rspec-users/attachments/20070904/558bcc38/attachment.html
On 9/4/07, Geoffrey Wiseman <geoffrey.wiseman at gmail.com> wrote:> Having used JUnit and Test::Unit, I''m quite used to having the ability to > insert a failure message, which helps when tests fail. > > For instance, the example RSpec that is generated for a model class > specifies that the model class is valid. Assuming this were supposed to be > true, and it failed, I''ve now got to duplicate the code in the test in order > to find out why it wasn''t valid. > > Whereas if I were writing the same code in Test::Unit, I might write: > assert model.valid?, "Expected model to be valid, but found these errors: > #{model.errors}" > > This means that when the model validation fails, I know /why/. I don''t see > an easy way to include these sorts of messages in RSpec, which seems likely > to cause me to waste time on test failures. Am I missing something? How > are experienced RSpec users resolving this?I come from the same background as you, so I hear where you''re coming from. We made a conscious decision, however, not to support custom messages almost two years ago and I''m not sure if its ever even come up before. If it has, it was a long time ago. If you follow the conventions of one expectation per example, and your example is well named, this is less of a problem. Here''s a common idiom: describe Person do def valid_attributes {:name => ''joe smith''} end before(:each) do @person = Person.new(valid_attributes) end it "should be valid with valid attributes" do @person.should be_valid end it "should be invalid with no name" do @person.name = nil @person.should_not be_valid end end Together, these different examples help to tell the whole story, and when one example fails you know why it''s failing - its just that the message is in the example''s name instead of a custom assertion message. Make sense?> > Thanks, > > - Geoffrey > -- > Geoffrey Wiseman > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
> > I come from the same background as you, so I hear where you''re coming > from. We made a conscious decision, however, not to support custom > messages almost two years ago and I''m not sure if its ever even come > up before. If it has, it was a long time ago.[nod] Perhaps as I get into the mindset, I''ll find this desire slips away. If you follow the conventions of one expectation per example, and your> example is well named, this is less of a problem. Here''s a common > idiom: > > describe Person do > def valid_attributes > {:name => ''joe smith''} > end > before(:each) do > @person = Person.new(valid_attributes) > end > it "should be valid with valid attributes" do > @person.should be_valid > end > it "should be invalid with no name" do > @person.name = nil > @person.should_not be_valid > end > endUsing this as an example, if a new validation rule is added, this test will fail without indicating /why/. Sure, I can get that answer in other ways, but I''d hate to discover things like: it "should be valid with valid attributes" do # puts @person.errors if !@person.valid @person.should be_valid end (Which I''ve seen when people have to repeatedly diagnose issues in a test; I''d prefer a failure message to the above) Together, these different examples help to tell the whole story, and> when one example fails you know why it''s failing - its just that the > message is in the example''s name instead of a custom assertion > message.Make sense? Yes and no; test isolation and good names is a decent practice even in XUnit, but clearly it''s that much stronger in RSpec, and I''m in favour of that. However, I find that often test failures involve unexpected changes ( e.g. the REST service didn''t return a status code of 201, as you expected, because a validation rule changed and the validation failed), which aren''t as easy to message. Still, i''m willing to give this approach a shot and see if this bothers me increasingly less. - Geoffrey -- Geoffrey Wiseman -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/rspec-users/attachments/20070904/9e09ed73/attachment.html
On 5/09/2007, at 8:51 AM, Geoffrey Wiseman wrote:> > Using this as an example, if a new validation rule is added, this > test will fail without indicating /why/. Sure, I can get that > answer in other ways, but I''d hate to discover things like: > > it "should be valid with valid attributes" do > # puts @person.errors if !@person.valid > @person.should be_valid > end >Sorry if I missed the point of this, but in the context of having an optional failure message.... I am curious as to how an optional message param to the assertion would help you in this case? Cheers Shane -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/rspec-users/attachments/20070905/78ece223/attachment-0001.html
On 9/4/07, Geoffrey Wiseman <geoffrey.wiseman at gmail.com> wrote:> > I come from the same background as you, so I hear where you''re coming > > from. We made a conscious decision, however, not to support custom > > messages almost two years ago and I''m not sure if its ever even come > > up before. If it has, it was a long time ago. > > [nod] Perhaps as I get into the mindset, I''ll find this desire slips away. > > > If you follow the conventions of one expectation per example, and your > > example is well named, this is less of a problem. Here''s a common > > idiom: > > > > describe Person do > > def valid_attributes > > {:name => ''joe smith''} > > end > > before(:each) do > > @person = Person.new(valid_attributes) > > end > > it "should be valid with valid attributes" do > > @person.should be_valid > > end > > it "should be invalid with no name" do > > @person.name = nil > > @person.should_not be_valid > > end > > end > > Using this as an example, if a new validation rule is added, this test will > fail without indicating /why/. Sure, I can get that answer in other ways, > but I''d hate to discover things like: > > it "should be valid with valid attributes" do > # puts @person.errors if !@person.valid > @person.should be_valid > end > > (Which I''ve seen when people have to repeatedly diagnose issues in a test; > I''d prefer a failure message to the above) > > > Together, these different examples help to tell the whole story, and > > when one example fails you know why it''s failing - its just that the > > message is in the example''s name instead of a custom assertion > > message. > > > Make sense? > > Yes and no; test isolation and good names is a decent practice even in > XUnit, but clearly it''s that much stronger in RSpec, and I''m in favour of > that. However, I find that often test failures involve unexpected changes ( > e.g. the REST service didn''t return a status code of 201, as you expected, > because a validation rule changed and the validation failed), which aren''t > as easy to message. > > Still, i''m willing to give this approach a shot and see if this bothers me > increasingly less.Personally, I''m open to the idea of custom messages - I just have no idea how that work syntactically. If you get to the point where you really feel the need for that feature (or before that point) please feel free to make suggestions about that. Cheers, David> > - Geoffrey > -- > Geoffrey Wiseman > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
Shane Mingins wrote:> > On 5/09/2007, at 8:51 AM, Geoffrey Wiseman wrote: > >> >> Using this as an example, if a new validation rule is added, this test >> will fail without indicating /why/. Sure, I can get that answer in >> other ways, but I''d hate to discover things like: >> >> it "should be valid with valid attributes" do >> # puts @person.errors if !@person.valid >> @person.should be_valid >> end >> > > Sorry if I missed the point of this, but in the context of having an > optional failure message.... > > I am curious as to how an optional message param to the assertion would > help you in this case?Seems like, in this case, he''d output @person.errors in his message so he could see *why* person was invalid; the "puts" is his current hack-around for the lack of custom messages. Not a bad idea, really. (the custom messages, not the hackaround!) Jay
I generally write custom expectation matchers when I want more specific information on failure scenarios. Granted this might not work in all scenarios (taking time to write a custom matcher I mean), but for most things it has made it very nice. -Chad On Sep 4, 2007, at 5:08 PM, Jay Levitt wrote:> Shane Mingins wrote: >> >> On 5/09/2007, at 8:51 AM, Geoffrey Wiseman wrote: >> >>> >>> Using this as an example, if a new validation rule is added, this >>> test >>> will fail without indicating /why/. Sure, I can get that answer in >>> other ways, but I''d hate to discover things like: >>> >>> it "should be valid with valid attributes" do >>> # puts @person.errors if !@person.valid >>> @person.should be_valid >>> end >>> >> >> Sorry if I missed the point of this, but in the context of having an >> optional failure message.... >> >> I am curious as to how an optional message param to the assertion >> would >> help you in this case? > > Seems like, in this case, he''d output @person.errors in his message so > he could see *why* person was invalid; the "puts" is his current > hack-around for the lack of custom messages. > > Not a bad idea, really. (the custom messages, not the hackaround!) > > Jay > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users
Chad Humphries wrote:> I generally write custom expectation matchers when I want more > specific information on failure scenarios. Granted this might not > work in all scenarios (taking time to write a custom matcher I mean), > but for most things it has made it very nice.Hmm, I wonder if rspec_on_rails should come with a custom be_valid matcher that shows .errors? I can''t think of a case where that would be bad... Jay
On 9/4/07, David Chelimsky <dchelimsky at gmail.com> wrote:> On 9/4/07, Geoffrey Wiseman <geoffrey.wiseman at gmail.com> wrote: > > > I come from the same background as you, so I hear where you''re coming > > > from. We made a conscious decision, however, not to support custom > > > messages almost two years ago and I''m not sure if its ever even come > > > up before. If it has, it was a long time ago. > > > > [nod] Perhaps as I get into the mindset, I''ll find this desire slips away. > > > > > If you follow the conventions of one expectation per example, and your > > > example is well named, this is less of a problem. Here''s a common > > > idiom: > > > > > > describe Person do > > > def valid_attributes > > > {:name => ''joe smith''} > > > end > > > before(:each) do > > > @person = Person.new(valid_attributes) > > > end > > > it "should be valid with valid attributes" do > > > @person.should be_valid > > > end > > > it "should be invalid with no name" do > > > @person.name = nil > > > @person.should_not be_valid > > > end > > > end > > > > Using this as an example, if a new validation rule is added, this test will > > fail without indicating /why/. Sure, I can get that answer in other ways, > > but I''d hate to discover things like: > > > > it "should be valid with valid attributes" do > > # puts @person.errors if !@person.valid > > @person.should be_valid > > end > > > > (Which I''ve seen when people have to repeatedly diagnose issues in a test; > > I''d prefer a failure message to the above) > > > > > Together, these different examples help to tell the whole story, and > > > when one example fails you know why it''s failing - its just that the > > > message is in the example''s name instead of a custom assertion > > > message. > > > > > Make sense? > > > > Yes and no; test isolation and good names is a decent practice even in > > XUnit, but clearly it''s that much stronger in RSpec, and I''m in favour of > > that. However, I find that often test failures involve unexpected changes ( > > e.g. the REST service didn''t return a status code of 201, as you expected, > > because a validation rule changed and the validation failed), which aren''t > > as easy to message. > > > > Still, i''m willing to give this approach a shot and see if this bothers me > > increasingly less. > > Personally, I''m open to the idea of custom messages - I just have no > idea how that work syntactically. If you get to the point where you > really feel the need for that feature (or before that point) please > feel free to make suggestions about that.What do you think about using a custom expectation matcher here? be_valid can be its own matcher instead of using the predicate matcher. That way we can include extra info without polluting the syntax, because as you said, this doesn''t come up. Of course that gets in the way of other objects that respond to valid?, so I guess you could do a little bit of type-checking (if it''s an AR object then display errors, otherwise delegate to predicate matcher) or create a separate matcher altogether. What about something like: it "should validate with valid attributes" do @person.should validate end ''Person should validate with valid attributes'' FAILED expected object to validate, failed with errors: Age can''t be blank Basically I think a custom expectation matcher works fine here, I just don''t know the best way to implement it. Pat
On 9/4/07, Pat Maddox <pergesu at gmail.com> wrote:> On 9/4/07, David Chelimsky <dchelimsky at gmail.com> wrote: > > On 9/4/07, Geoffrey Wiseman <geoffrey.wiseman at gmail.com> wrote: > > > > I come from the same background as you, so I hear where you''re coming > > > > from. We made a conscious decision, however, not to support custom > > > > messages almost two years ago and I''m not sure if its ever even come > > > > up before. If it has, it was a long time ago. > > > > > > [nod] Perhaps as I get into the mindset, I''ll find this desire slips away. > > > > > > > If you follow the conventions of one expectation per example, and your > > > > example is well named, this is less of a problem. Here''s a common > > > > idiom: > > > > > > > > describe Person do > > > > def valid_attributes > > > > {:name => ''joe smith''} > > > > end > > > > before(:each) do > > > > @person = Person.new(valid_attributes) > > > > end > > > > it "should be valid with valid attributes" do > > > > @person.should be_valid > > > > end > > > > it "should be invalid with no name" do > > > > @person.name = nil > > > > @person.should_not be_valid > > > > end > > > > end > > > > > > Using this as an example, if a new validation rule is added, this test will > > > fail without indicating /why/. Sure, I can get that answer in other ways, > > > but I''d hate to discover things like: > > > > > > it "should be valid with valid attributes" do > > > # puts @person.errors if !@person.valid > > > @person.should be_valid > > > end > > > > > > (Which I''ve seen when people have to repeatedly diagnose issues in a test; > > > I''d prefer a failure message to the above) > > > > > > > Together, these different examples help to tell the whole story, and > > > > when one example fails you know why it''s failing - its just that the > > > > message is in the example''s name instead of a custom assertion > > > > message. > > > > > > > Make sense? > > > > > > Yes and no; test isolation and good names is a decent practice even in > > > XUnit, but clearly it''s that much stronger in RSpec, and I''m in favour of > > > that. However, I find that often test failures involve unexpected changes ( > > > e.g. the REST service didn''t return a status code of 201, as you expected, > > > because a validation rule changed and the validation failed), which aren''t > > > as easy to message. > > > > > > Still, i''m willing to give this approach a shot and see if this bothers me > > > increasingly less. > > > > Personally, I''m open to the idea of custom messages - I just have no > > idea how that work syntactically. If you get to the point where you > > really feel the need for that feature (or before that point) please > > feel free to make suggestions about that. > > What do you think about using a custom expectation matcher here? > be_valid can be its own matcher instead of using the predicate > matcher. That way we can include extra info without polluting the > syntax, because as you said, this doesn''t come up. > > Of course that gets in the way of other objects that respond to > valid?, so I guess you could do a little bit of type-checking (if it''s > an AR object then display errors, otherwise delegate to predicate > matcher) or create a separate matcher altogether. > > What about something like: > > it "should validate with valid attributes" do > @person.should validatebe_valid_model????> end > > ''Person should validate with valid attributes'' FAILED > expected object to validate, failed with errors: > Age can''t be blank > > Basically I think a custom expectation matcher works fine here, I just > don''t know the best way to implement it. > > Pat > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
On 9/4/07, Jay Levitt <lists-rspec at shopwatch.org> wrote:> > Seems like, in this case, he''d output @person.errors in his message so > he could see *why* person was invalid; the "puts" is his current > hack-around for the lack of custom messages. > > Not a bad idea, really. (the custom messages, not the hackaround!) >Yes, exactly; I often write the simple case with no extra message first, then the first time the failing unit test doesn''t give me enough information to fix the problem, I add a custom message that makes it easier. Examples culled from Test::Unit code: - assert_select "a[href*=#{rev_1_url}]", true, "could not find #{rev_1_url} in #{@response.body}" - In this case, it failed to find the URL, but I wasn''t sure why that would be; adding the response body made it possible for me to diagnose that the expected URL I''d generated didn''t match because the ID was off. This kind of response is pretty spammy (entire response text), so I''d definitely only employ this if it was present only in the case of failure. - assert_response :ok, "Failed to restore contact_info: #{@ response.body}" - A similarly spammy @response.body output; wasn''t sure why the response wasn''t :ok, turned out to be a validation failure triggered by restoring a no-longer-valid revision. - assert customer.valid?, "Fixture customer is expected to be valid: #{customer.errors.full_messages.join('', '')}" - Pretty similar to the example I used above; unexpected validation failure, it''s nice to see the errors. Are any of these critical to RSpec''s main goals? Probably not. Will it prevent my adopting RSpec? Probably not. Is it still handy to have so that you don''t employ other means (e.g. commented out puts) for repeat diagnosis of failures? I think so. That said, if none of the above convinces you, then by all means, I''ll go ahead with the standard RSpec approach and see if, in a few months, I''m still missing this feature and, if so, I''ll have good RSpec examples in hand, with which to discuss. ;) - Geoffrey -- Geoffrey Wiseman -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/rspec-users/attachments/20070904/9bc522e4/attachment.html
Pat Maddox wrote:> On 9/4/07, David Chelimsky <dchelimsky at gmail.com> wrote: > >> On 9/4/07, Geoffrey Wiseman <geoffrey.wiseman at gmail.com> wrote: >> >>>> I come from the same background as you, so I hear where you''re coming >>>> from. We made a conscious decision, however, not to support custom >>>> messages almost two years ago and I''m not sure if its ever even come >>>> up before. If it has, it was a long time ago. >>>> >>> [nod] Perhaps as I get into the mindset, I''ll find this desire slips away. >>> >>> >>>> If you follow the conventions of one expectation per example, and your >>>> example is well named, this is less of a problem. Here''s a common >>>> idiom: >>>> >>>> describe Person do >>>> def valid_attributes >>>> {:name => ''joe smith''} >>>> end >>>> before(:each) do >>>> @person = Person.new(valid_attributes) >>>> end >>>> it "should be valid with valid attributes" do >>>> @person.should be_valid >>>> end >>>> it "should be invalid with no name" do >>>> @person.name = nil >>>> @person.should_not be_valid >>>> end >>>> end >>>> >>> Using this as an example, if a new validation rule is added, this test will >>> fail without indicating /why/. Sure, I can get that answer in other ways, >>> but I''d hate to discover things like: >>> >>> it "should be valid with valid attributes" do >>> # puts @person.errors if !@person.valid >>> @person.should be_valid >>> end >>> >>> (Which I''ve seen when people have to repeatedly diagnose issues in a test; >>> I''d prefer a failure message to the above) >>> >>> >>>> Together, these different examples help to tell the whole story, and >>>> when one example fails you know why it''s failing - its just that the >>>> message is in the example''s name instead of a custom assertion >>>> message. >>>> >>>> Make sense? >>>> >>> Yes and no; test isolation and good names is a decent practice even in >>> XUnit, but clearly it''s that much stronger in RSpec, and I''m in favour of >>> that. However, I find that often test failures involve unexpected changes ( >>> e.g. the REST service didn''t return a status code of 201, as you expected, >>> because a validation rule changed and the validation failed), which aren''t >>> as easy to message. >>> >>> Still, i''m willing to give this approach a shot and see if this bothers me >>> increasingly less. >>> >> Personally, I''m open to the idea of custom messages - I just have no >> idea how that work syntactically. If you get to the point where you >> really feel the need for that feature (or before that point) please >> feel free to make suggestions about that. >> > > What do you think about using a custom expectation matcher here? > be_valid can be its own matcher instead of using the predicate > matcher. That way we can include extra info without polluting the > syntax, because as you said, this doesn''t come up. > > Of course that gets in the way of other objects that respond to > valid?, so I guess you could do a little bit of type-checking (if it''s > an AR object then display errors, otherwise delegate to predicate > matcher) or create a separate matcher altogether. > > What about something like: > > it "should validate with valid attributes" do > @person.should validate > end > > ''Person should validate with valid attributes'' FAILED > expected object to validate, failed with errors: > Age can''t be blank > > Basically I think a custom expectation matcher works fine here, I just > don''t know the best way to implement it. > > Pat > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >Maybe this is what your thinking? http://opensoul.org/2007/4/18/rspec-model-should-be_valid -Ben
On 9/4/07, Ben Mabey <ben at benmabey.com> wrote:> Maybe this is what your thinking? > > http://opensoul.org/2007/4/18/rspec-model-should-be_valid >That should be the default matcher for be_valid... I use that and help me pinpoint some brittle specs (all related to new attributes added later in the development phase). As David himself said on that post: " Perfect use of custom matchers! Nice. Cheers, David " :-D -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi
El 4/9/2007, a las 22:51, Geoffrey Wiseman escribi?:> Using this as an example, if a new validation rule is added, this > test will > fail without indicating /why/. Sure, I can get that answer in > other ways, > but I''d hate to discover things like: > > it "should be valid with valid attributes" do > # puts @person.errors if !@person.valid > @person.should be_valid > end > > (Which I''ve seen when people have to repeatedly diagnose issues in > a test; > I''d prefer a failure message to the above)My personal approach when a spec fails and I can''t figure out *why* just by looking at it is to run the spec under the debugger and set a breakpoint on the line where the corresponding spec fails. In this way I can get as much information as I want about the problem and its context. The suggestion of using a custom matcher is a good one, but I personally don''t find it general enough. I actually *like* the way that RSpec doesn''t provide a mechanism for specifying error messages; it''s one of the things that makes the specs so easy to read. It''s difficult to imagine a syntax that would provide what you''re asking for without sacrificing readability. Cheers, Wincent
Scott Taylor
2007-Sep-08 20:19 UTC
[rspec-users] Failure Messages in RSpec + inline debugging of a spec
On Sep 5, 2007, at 4:25 AM, Wincent Colaiuta wrote:> El 4/9/2007, a las 22:51, Geoffrey Wiseman escribi?: > >> Using this as an example, if a new validation rule is added, this >> test will >> fail without indicating /why/. Sure, I can get that answer in >> other ways, >> but I''d hate to discover things like: >> >> it "should be valid with valid attributes" do >> # puts @person.errors if !@person.valid >> @person.should be_valid >> end >> >> (Which I''ve seen when people have to repeatedly diagnose issues in >> a test; >> I''d prefer a failure message to the above) > > My personal approach when a spec fails and I can''t figure out *why* > just by looking at it is to run the spec under the debugger and set a > breakpoint on the line where the corresponding spec fails. >I''ve been doing the same thing with ruby-debug. I have setup a snippet in textmate so that when I type debug(tab), I get the following line: require ''rubygems''; require ''ruby-debug''; debugger And I usually do this *inside* the failing spec. So far it has been a mixed experience - I have found that it doesn''t work very well with rails as you end up debugging more of rails then of your own code. It also doesn''t work with the DRB server. Any thoughts on how this could be better done?> I actually *like* the way that RSpec doesn''t provide a mechanism for > specifying error messages; it''s one of the things that makes the > specs so easy to read. It''s difficult to imagine a syntax that would > provide what you''re asking for without sacrificing readability. > > Cheers, > WincentI couldn''t agree more. I, like David, am not opposed to a custom matcher, but it''s hard to imagine how it would play nicely with the syntax. Scott
Jay Levitt
2007-Sep-08 21:41 UTC
[rspec-users] Failure Messages in RSpec + inline debugging of a spec
On 9/8/2007 4:19 PM, Scott Taylor wrote:> So far it has been > a mixed experience - I have found that it doesn''t work very well with > rails as you end up debugging more of rails then of your own code.Finally, a debugging environment that matches the production environment! Jay
Luis Lavena
2007-Sep-09 13:23 UTC
[rspec-users] Failure Messages in RSpec + inline debugging of a spec
On 9/8/07, Jay Levitt <lists-rspec at shopwatch.org> wrote:> On 9/8/2007 4:19 PM, Scott Taylor wrote: > > So far it has been > > a mixed experience - I have found that it doesn''t work very well with > > rails as you end up debugging more of rails then of your own code. > > Finally, a debugging environment that matches the production environment! >I agree. When debug is required, I switch to script/spec and specify the exact file and specification to run, in that way, reduce the problems loading ruby-debug and getting my feet wet with Rails code :) -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi