rspec users - Nov 2011 - customize description and failure messages when writing rspec matcher dsl

Dear all,

I recently tried to write a custom matcher using rspec dsl, which I''d
like
to use as the following:

==========hpgc_spec.rb==================
require "spec_helper"

describe Hpgc do
  specify do
    should be_structured_as(
    { :app => {
        :has => [
            :name,
            :author,
            :org,
            :version,
            :abstract,
            :uri
        ],

        :can_be_a => {
            :dataset => {
                :can_be_a => {
                    :vector => {},
                    :raster => {}
                }
            },

            :program => {
                :has => [
                    :language,
                    :parallel_programming_model,
                    :parallel_performance_overview,
                    :category
                ]
            }
        },

        :has_many => {
            :app_option => {
                :has => [
                    :switch,
                    :description
                ]
            },

            :app_io => {
                :has => [
                    :name,
                    :description,
                    :is_in
                ]
            }
        }
    }})
  end
end

==========================================
The purpose is to test the entire model structure(attributes, inheritance
and association). Then I wrote following code to implement it:

========be_structured_as.rb==================

# This is a matcher for test hpgc model structure

RSpec::Matchers.define :be_structured_as do |structure|
  def check(namespace, klass, desc)
    (desc[:has_many] ||= {}).keys.each { |item|
klass.to_s.camelize.constantize.new.should have_many
item.to_s.pluralize.to_sym }
    if !desc[:can_be_a] || desc[:can_be_a].empty?
      obj = Factory.create(klass)
      obj.should have_these_attributes (desc[:has] ||= [])
    else
      desc[:can_be_a].keys.map { |sub| sub.to_s }.should
=namespace.subclasses_of(klass.to_s)
      desc[:can_be_a].each { |key, value| check namespace, key, value }
    end
  end

  match do |namespace|
    self.class.send :include, namespace
    structure.each { |key, value| check namespace, key, value }
  end
end

==============================================
To simplify the matcher definition,  I used some other machers like
"obj.should have_these_attributes (desc[:has] ||= [])", this works
fine,
thanks to the excellent new matcher dsl syntax. Howerver, I got trouble
when running the test using "rspec spec/models/hpgc_spec.rb -fd". The
test
passed but print

[image: Screenshot.png]

instead of something like "Hpgc should be structured as ...". I think
it
may due to the other matcher I used because the message is from the matcher
"have_these_attributes". I try to add method "description"
but seems it
doesn''t work.

And the failure message is also a problem. I search the web and come across
some solutions, but all of them need manually build the message. I''ve
not
come out a clean way to do this(I must add some invasive clauses to track
the error) and think maybe the best behaviour would be that it "throw"
the
failure message of inner matchers or false-value expressions. But this time
it just  told me something like "Hpgc is expected to be structured as
...".

Any idea?

Best Regards,

YANG Anran
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20111124/ceb96a59/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 5244 bytes
Desc: not available
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20111124/ceb96a59/attachment-0001.png>

On Nov 24, 2011, at 12:48 AM, Anran Yang wrote:
> Dear all,
> 
> I recently tried to write a custom matcher using rspec dsl, which
I''d like to use as the following:
> 
> ==========hpgc_spec.rb==================> 
> require "spec_helper"
> 
> describe Hpgc do
>   specify do 
>     should be_structured_as(
>     { :app => {
>         :has => [
>             :name, 
>             :author, 
>             :org, 
>             :version, 
>             :abstract, 
>             :uri 
>         ],
> 
>         :can_be_a => { 
>             :dataset => { 
>                 :can_be_a => {
>                     :vector => {}, 
>                     :raster => {}
>                 }
>             },
> 
>             :program => { 
>                 :has => [
>                     :language, 
>                     :parallel_programming_model, 
>                     :parallel_performance_overview, 
>                     :category 
>                 ]
>             }
>         },
> 
>         :has_many => {
>             :app_option => {
>                 :has => [
>                     :switch, 
>                     :description 
>                 ]
>             },
> 
>             :app_io => { 
>                 :has => [
>                     :name, 
>                     :description, 
>                     :is_in 
>                 ]
>             }
>         }
>     }})
>   end
> end
> 
> ==========================================> 
> The purpose is to test the entire model structure(attributes, inheritance
and association).
Then term Behavior Driven Development came into being, in part, to discourage
this approach to testing. We believe testing should be about behavior, and that
testing structures leads to highly coupled tests that are hard to change.

I''d recommend thinking about the need for each of these attributes,
come up with examples of how they are used and write those instead. That said,
see below for more on how matchers work.
> Then I wrote following code to implement it:
> 
> ========be_structured_as.rb==================> 
> # This is a matcher for test hpgc model structure
> 
> RSpec::Matchers.define :be_structured_as do |structure|
>   def check(namespace, klass, desc)
>     (desc[:has_many] ||= {}).keys.each { |item|
klass.to_s.camelize.constantize.new.should have_many item.to_s.pluralize.to_sym
}
>     if !desc[:can_be_a] || desc[:can_be_a].empty?
>       obj = Factory.create(klass)
>       obj.should have_these_attributes (desc[:has] ||= [])
>     else
>       desc[:can_be_a].keys.map { |sub| sub.to_s }.should ==
namespace.subclasses_of(klass.to_s)
>       desc[:can_be_a].each { |key, value| check namespace, key, value }
>     end
>   end
> 
>   match do |namespace|
>     self.class.send :include, namespace
>     structure.each { |key, value| check namespace, key, value }
>   end
> end
> 
> ==============================================> 
> To simplify the matcher definition,  I used some other machers like
"obj.should have_these_attributes (desc[:has] ||= [])", this works
fine, thanks to the excellent new matcher dsl syntax. Howerver, I got trouble
when running the test using "rspec spec/models/hpgc_spec.rb -fd". The
test passed but print
> 
> <Screenshot.png>
> 
> instead of something like "Hpgc should be structured as ...". I
think it may due to the other matcher I used because the message is from the
matcher "have_these_attributes".
That''s correct. The default behavior is that the block passed to the
"match" method in the matcher DSL is expected to return a boolean. If
it raises an exception instead, that exception bubbles up and becomes the source
of the failure message.
> I try to add method "description" but seems it doesn''t
work.
"description" is used for the documentation formatter, not for failure
messages. If you run "rspec spec --format documentation"
you''ll see names for all your examples.
> And the failure message is also a problem. I search the web and come across
some solutions, but all of them need manually build the message. I''ve
not come out a clean way to do this(I must add some invasive clauses to track
the error) and think maybe the best behaviour would be that it "throw"
the failure message of inner matchers or false-value expressions. But this time
it just told me something like "Hpgc is expected to be structured as
...".
Instead of using the "match" method in the matcher, you can use
"match_unless_raises", which returns false if it captures an
exception, letting the matcher control the failure message (which is what I
think you''re looking for):

RSpec::Matchers.define :be_structured_as do |structure|
  def check(namespace, klass, desc)
    # ....
  end

  match_unless_raises do |namespace|
    self.class.send :include, namespace
    structure.each { |key, value| check namespace, key, value }
  end
end

HTH,
David
> Any idea?
> 
> Best Regards,
> 
> YANG Anran

On Nov 24, 2011, at 12:48 AM, Anran Yang wrote:
> Dear all,
>
> I recently tried to write a custom matcher using rspec dsl, which
I''d
like to use as the following:
>
> ==========hpgc_spec.rb==================>
> require "spec_helper"
>
> describe Hpgc do
>   specify do
>     should be_structured_as(
>     { :app => {
>         :has => [
>             :name,
>             :author,
>             :org,
>             :version,
>             :abstract,
>             :uri
>         ],
>
>         :can_be_a => {
>             :dataset => {
>                 :can_be_a => {
>                     :vector => {},
>                     :raster => {}
>                 }
>             },
>
>             :program => {
>                 :has => [
>                     :language,
>                     :parallel_programming_model,
>                     :parallel_performance_overview,
>                     :category
>                 ]
>             }
>         },
>
>         :has_many => {
>             :app_option => {
>                 :has => [
>                     :switch,
>                     :description
>                 ]
>             },
>
>             :app_io => {
>                 :has => [
>                     :name,
>                     :description,
>                     :is_in
>                 ]
>             }
>         }
>     }})
>   end
> end
>
> ==========================================>
> The purpose is to test the entire model structure(attributes, inheritance
and association).

Then term Behavior Driven Development came into being, in part, to
discourage this approach to testing. We believe testing should be about
behavior, and that testing structures leads to highly coupled tests that
are hard to change.

I''d recommend thinking about the need for each of these attributes,
come up
with examples of how they are used and write those instead. That said, see
below for more on how matchers work.

---------------------------------
It made me thinking a while. But at last I found that I''ve thought
about
it, though it may only happen unconsciously. So below is why I need this
kind of spec.
I must admit the Spec is against BDD''s spirit. However, I think BDD may
not
be enough for all kinds of projects in all kinds of circumstance. In my
project, the model itself is also a product, not only the "surface" of
the
software. Maybe the best way to accomplish this is to separate this kind of
test and the "real" BDD test. But rspec can support them perfectly so
I''m
lazy this time.
Maybe there are some other projects involved in this kind of situation? I
mean, some projects with a concrete domain model, so to some extent the
development is driven by both behavior and knowledge?

> Then I wrote following code to implement it:
>
> ========be_structured_as.rb==================>
> # This is a matcher for test hpgc model structure
>
> RSpec::Matchers.define :be_structured_as do |structure|
>   def check(namespace, klass, desc)
>     (desc[:has_many] ||= {}).keys.each { |item|
klass.to_s.camelize.constantize.new.should have_many
item.to_s.pluralize.to_sym }
>     if !desc[:can_be_a] || desc[:can_be_a].empty?
>       obj = Factory.create(klass)
>       obj.should have_these_attributes (desc[:has] ||= [])
>     else
>       desc[:can_be_a].keys.map { |sub| sub.to_s }.should
=namespace.subclasses_of(klass.to_s)
>       desc[:can_be_a].each { |key, value| check namespace, key, value }
>     end
>   end
>
>   match do |namespace|
>     self.class.send :include, namespace
>     structure.each { |key, value| check namespace, key, value }
>   end
> end
>
> ==============================================>
> To simplify the matcher definition,  I used some other machers like
"obj.should have_these_attributes (desc[:has] ||= [])", this works
fine,
thanks to the excellent new matcher dsl syntax. Howerver, I got trouble
when running the test using "rspec spec/models/hpgc_spec.rb -fd". The
test
passed but print
>
> <Screenshot.png>
>
> instead of something like "Hpgc should be structured as ...". I
think it
may due to the other matcher I used because the message is from the matcher
"have_these_attributes".

That''s correct. The default behavior is that the block passed to the
"match" method in the matcher DSL is expected to return a boolean. If
it
raises an exception instead, that exception bubbles up and becomes the
source of the failure message.
> I try to add method "description" but seems it doesn''t
work.
"description" is used for the documentation formatter, not for failure
messages. If you run "rspec spec --format documentation"
you''ll see names
for all your examples.

--------------------------
Maybe I failed express my problem clearly, but the question is for pass
message, not failure message. Knowing how rspec raise exception help little
to get my spec say "Hpgc should be structured as... " when it passes.
The
method "description" failed to present itself even when the spec
passed, as
long as there is an inner "should" clause.


> And the failure message is also a problem. I search the web and come
across some solutions, but all of them need manually build the message.
I''ve not come out a clean way to do this(I must add some invasive
clauses
to track the error) and think maybe the best behaviour would be that it
"throw" the failure message of inner matchers or false-value
expressions.
But this time it just told me something like "Hpgc is expected to be
structured as ...".

Instead of using the "match" method in the matcher, you can use
"match_unless_raises", which returns false if it captures an
exception,
letting the matcher control the failure message (which is what I think
you''re looking for):

RSpec::Matchers.define :be_structured_as do |structure|
 def check(namespace, klass, desc)
   # ....
 end

 match_unless_raises do |namespace|
   self.class.send :include, namespace
   structure.each { |key, value| check namespace, key, value }
 end
end

HTH,
David


----------------------------------------------------------------
 Is the method in rspec-expectation? I searched the local doc but
didn''t
find it. Also I don''t quite understand how the method can help track
the
failing message. It return false, but I still cannot figure out which line
fails, isn''t it?

Thank you for your inspiring reply and tolerance to my poor English :-)

Regards,
YANG
> Any idea?
>
> Best Regards,
>
> YANG Anran
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20111127/05996314/attachment-0001.html>

On Nov 27, 2011, at 7:22 AM, Anran Yang wrote:
> On Nov 24, 2011, at 12:48 AM, Anran Yang wrote:
> 
> > Dear all,
> >
> > I recently tried to write a custom matcher using rspec dsl, which
I''d like to use as the following:
> >
> > ==========hpgc_spec.rb==================> >
> > require "spec_helper"
> >
> > describe Hpgc do
> >   specify do
> >     should be_structured_as(
> >     { :app => {
> >         :has => [
> >             :name,
> >             :author,
> >             :org,
> >             :version,
> >             :abstract,
> >             :uri
> >         ],
> >
> >         :can_be_a => {
> >             :dataset => {
> >                 :can_be_a => {
> >                     :vector => {},
> >                     :raster => {}
> >                 }
> >             },
> >
> >             :program => {
> >                 :has => [
> >                     :language,
> >                     :parallel_programming_model,
> >                     :parallel_performance_overview,
> >                     :category
> >                 ]
> >             }
> >         },
> >
> >         :has_many => {
> >             :app_option => {
> >                 :has => [
> >                     :switch,
> >                     :description
> >                 ]
> >             },
> >
> >             :app_io => {
> >                 :has => [
> >                     :name,
> >                     :description,
> >                     :is_in
> >                 ]
> >             }
> >         }
> >     }})
> >   end
> > end
> >
> > ==========================================> >
> > The purpose is to test the entire model structure(attributes,
inheritance and association).
> 
> Then term Behavior Driven Development came into being, in part, to
discourage this approach to testing. We believe testing should be about
behavior, and that testing structures leads to highly coupled tests that are
hard to change.
> 
> I''d recommend thinking about the need for each of these
attributes, come up with examples of how they are used and write those instead.
That said, see below for more on how matchers work.
> 
> ---------------------------------
> It made me thinking a while. But at last I found that I''ve thought
about it, though it may only happen unconsciously. So below is why I need this
kind of spec.
> I must admit the Spec is against BDD''s spirit. However, I think
BDD may not be enough for all kinds of projects in all kinds of circumstance. In
my project, the model itself is also a product, not only the "surface"
of the software. Maybe the best way to accomplish this is to separate this kind
of test and the "real" BDD test. But rspec can support them perfectly
so I''m lazy this time.
> Maybe there are some other projects involved in this kind of situation? I
mean, some projects with a concrete domain model, so to some extent the
development is driven by both behavior and knowledge?
The specific problem is that tests about structure are more highly coupled to
structure than behavior, and therefore more brittle. This is true whether you
wrote the test first or wrote it after, and regardless of what methodology you
are thinking of when you write the tests.

I don''t understand what you mean when you say ''In my project,
the model itself is also a product, not only the "surface" of the
software.''. Can you elaborate?
> > Then I wrote following code to implement it:
> >
> > ========be_structured_as.rb==================> >
> > # This is a matcher for test hpgc model structure
> >
> > RSpec::Matchers.define :be_structured_as do |structure|
> >   def check(namespace, klass, desc)
> >     (desc[:has_many] ||= {}).keys.each { |item|
klass.to_s.camelize.constantize.new.should have_many item.to_s.pluralize.to_sym
}
> >     if !desc[:can_be_a] || desc[:can_be_a].empty?
> >       obj = Factory.create(klass)
> >       obj.should have_these_attributes (desc[:has] ||= [])
> >     else
> >       desc[:can_be_a].keys.map { |sub| sub.to_s }.should ==
namespace.subclasses_of(klass.to_s)
> >       desc[:can_be_a].each { |key, value| check namespace, key, value
}
> >     end
> >   end
> >
> >   match do |namespace|
> >     self.class.send :include, namespace
> >     structure.each { |key, value| check namespace, key, value }
> >   end
> > end
> >
> > ==============================================> >
> > To simplify the matcher definition,  I used some other machers like
"obj.should have_these_attributes (desc[:has] ||= [])", this works
fine, thanks to the excellent new matcher dsl syntax. Howerver, I got trouble
when running the test using "rspec spec/models/hpgc_spec.rb -fd". The
test passed but print
> >
> > <Screenshot.png>
> >
> > instead of something like "Hpgc should be structured as
...". I think it may due to the other matcher I used because the message is
from the matcher "have_these_attributes".
> 
> That''s correct. The default behavior is that the block passed to
the "match" method in the matcher DSL is expected to return a boolean.
If it raises an exception instead, that exception bubbles up and becomes the
source of the failure message.
> 
> > I try to add method "description" but seems it
doesn''t work.
> 
> "description" is used for the documentation formatter, not for
failure messages. If you run "rspec spec --format documentation"
you''ll see names for all your examples.
> 
> --------------------------
> Maybe I failed express my problem clearly, but the question is for pass
message, not failure message. Knowing how rspec raise exception help little to
get my spec say "Hpgc should be structured as... " when it passes. The
method "description" failed to present itself even when the spec
passed, as long as there is an inner "should" clause.
You can define your own description method in the same way:

description do |actual|
  #  ....
end
> > And the failure message is also a problem. I search the web and come
across some solutions, but all of them need manually build the message.
I''ve not come out a clean way to do this(I must add some invasive
clauses to track the error) and think maybe the best behaviour would be that it
"throw" the failure message of inner matchers or false-value
expressions. But this time it just told me something like "Hpgc is expected
to be structured as ...".
> 
> Instead of using the "match" method in the matcher, you can use
"match_unless_raises", which returns false if it captures an
exception, letting the matcher control the failure message (which is what I
think you''re looking for):
> 
> RSpec::Matchers.define :be_structured_as do |structure|
>  def check(namespace, klass, desc)
>    # ....
>  end
> 
>  match_unless_raises do |namespace|
>    self.class.send :include, namespace
>    structure.each { |key, value| check namespace, key, value }
>  end
> end
> 
> HTH,
> David
> 
> 
> ----------------------------------------------------------------
> Is the method in rspec-expectation? I searched the local doc but
didn''t find it.
It is in rspec-expectations but it is not documented. I added it to support
matchers in rspec-rails that delegate to minitest assertions (which raise their
own exception), and never took the time to formalize it as a public API. That
said, it should be considered public, will not change without appropriate
warning, and will be documented in the rdoc for the 2.8 release.
> Also I don''t quite understand how the method can help track the
failing message. It return false, but I still cannot figure out which line
fails, isn''t it?
It stores the exception from the failure in an instance variable named
@rescued_exception, which you can access through a method named
rescued_exception e.g.

failure_messsage_for_should do |actual|
  "Expected .... but something
happened\n#{rescued_exception.message}\n#{rescued_exception.backtrace}"
end

Another approach would be to keep track of each thing individually using the
standard matches method:

match do |actual|
  @reasons == []
  @reasons << "no foo" unless foo
  @reasons << "no bar" unless bar
  @reasons << "no baz" unless baz
  @reasons.empty?
end

That returns false if any conditions fail, but doesn''t raise an
exceptions. Then you can use @reasons in the failure message, e.g.

failure_message_for_should do |actual|
  "Failed because #{@reasons.join('', '')}"
end
> Thank you for your inspiring reply and tolerance to my poor English :-)
Your English is perfectly understandable.

Cheers,
David
> Regards,
> YANG
> 
> > Any idea?
> >
> > Best Regards,
> >
> > YANG Anran