thr3ads.net - rspec users - [rspec-users] Building higher-level DSLs [Feb 2013]

If this information is useful, please help other people find it:
Share via:

Ash Moran

2013-Feb-07 18:26 UTC

[rspec-users] Building higher-level DSLs

Hi

Someone on another mailing list I''m on recently posted asking for
people''s thoughts on test naming practices, and writing my reply made
me think about some of the techniques I use to improve naming and remove
duplication in my own spec files.

The most worked-through example I have is the contract test for my solutions to
the Tennis Kata[1]. (I''m not implying this is the best way to tackle
the Tennis Kata.) Like with everything other spec suite, I started out using
plain describe/context/it type language[2], which contains a lot of duplication:

  describe "scoring" do
    before(:each) { tennis.start_game }

    context "with no advantages" do
      context "A" do
        before(:each) { tennis.point_to_player_a }
        specify { expect(@score).to be == "15-0" }

        context "A" do
          before(:each) { tennis.point_to_player_a }
          specify { expect(@score).to be == "30-0" }
        end

        context "B" do
          before(:each) { tennis.point_to_player_b }
          specify { expect(@score).to be == "15-15" }

And it so goes on for quite a bit longer in the same style. The first step is to
factor out the duplication into context helpers, which leaves code like this:

  game_started do
    score_is_now "0-0"

    context "with no advantages" do
      point_to_player :a do
        score_is_now "15-0"

        point_to_player :a do
          score_is_now "30-0"
        end

The problem now is that the meaning of the specification is now hidden in the
implementation of the helpers, in this case it was in spec_helper[3]. This gives
very poorly composed methods with mixed levels of abstraction. So the final step
is to parameterise the spec DSL with blocks of code from the specs itself,
allowing you to write:

  specification_dsl :tennis do
    for_context :game_not_started do
      nothing
    end

    for_context :game_started do
      tennis.start_game
    end

    for_context :point_to_player do |player|
      # Heh, just noticed writing this email that I could be doing
      # tennis.send(:"point_to_player_#{player}") here, hey ho
      player == :a ? tennis.point_to_player_a : tennis.point_to_player_b
    end

    for_context :deuce do
      3.times do
        tennis.point_to_player_a
        tennis.point_to_player_b
      end
    end

    to_expect :score_is_now do |expected_score|
      expect(@scores.last).to be == expected_score
    end
    
This has finally put the spec and its definition back together[4], with the DSL
definition and its voodoo metaprogramming hidden away in spec_helper[5].

Unfortunately there''s a problem with this implementation, which is that
it fools RSpec into thinking expectation failures are all coming from
spec_helper.rb, which makes for rather useless error messages. I
haven''t investigated this.

Anyway, the point of explaining this example is to ask for people''s
opinions myself. A few obvious questions are:

* What sort of DSL-building have you tried/seen?
* Is this worth the effort over e.g. helper modules and custom matchers? (E.g.
is the terseness worth the indirection?)
* Is this possible in a simpler way with existing context tools in RSpec?
* If not, is it worth trying to make this DSL definition reusable?
* Are the situations where this is useful inherently best tackled another way?

I''m interested in any opinions though, especially if you have a valid
reason why this is a bad idea / approach. I''ve sat on it long enough
I''m clearly not going to have any more insights on my own any time
soon.

Cheers
Ash

[1] http://codingdojo.org/cgi-bin/wiki.pl?KataTennis
[2]
https://github.com/ashmoran/tennis_kata/blob/ff46b7e502337988940ef9ed881c934a04c53766/spec/tennis_spec.rb
[3]
https://github.com/ashmoran/tennis_kata/blob/a159bac3c3d6180c6f1b83770e6cd678fa750b33/spec/spec_helper.rb
[4] https://github.com/ashmoran/tennis_kata/blob/master/spec/tennis_contract.rb
[5] https://github.com/ashmoran/tennis_kata/blob/master/spec/spec_helper.rb

-- 
http://www.patchspace.co.uk/
http://www.linkedin.com/in/ashmoran

Sam Goldman

2013-Feb-07 23:32 UTC

head link

[rspec-users] Building higher-level DSLs

I''m not sure if this is what you are looking for, but I do have some
experience with writing DSLs on top of RSpec.

I made a project that lets you write tests for HTTP APIs in a bit more of a
straightforward manner[1]. This isn''t quite the level of DSL-ness that
it
seems you are going for?i.e., the specs using this library still very much
look like RSpec.

In the context of this project, I think that it was worth it. I very much
enjoy writing specs in the style afforded by this DSL.

I think the tradeoff is about test noise. Let me define noise as details
that are only relevant at a lower level of abstraction from the spec''s
level of abstraction. Ideally, your DSL will only hide noise. If your DSL
hides signal, then consumers of your dialect will need to understand the
internals.

That said, I think it''s a bad idea to start with a DSL. Write your
tests
directly, then once you have a good number of tests, ask yourself if a
non-leaky abstraction exists would let you convey your specs at a higher
level. You might find that this leads to a new domain concept for your
production code. Alternatively it might be better to augment your tests
with a little DSL.

Sorry if that''s a little too philosophical :P

Sam

1. https://github.com/smartlogic/http_spec


On Thu, Feb 7, 2013 at 8:26 AM, Ash Moran <ash.moran at
patchspace.co.uk>wrote:
> Hi
>
> Someone on another mailing list I''m on recently posted asking for
people''s
> thoughts on test naming practices, and writing my reply made me think about
> some of the techniques I use to improve naming and remove duplication in my
> own spec files.
>
> The most worked-through example I have is the contract test for my
> solutions to the Tennis Kata[1]. (I''m not implying this is the
best way to
> tackle the Tennis Kata.) Like with everything other spec suite, I started
> out using plain describe/context/it type language[2], which contains a lot
> of duplication:
>
>   describe "scoring" do
>     before(:each) { tennis.start_game }
>
>     context "with no advantages" do
>       context "A" do
>         before(:each) { tennis.point_to_player_a }
>         specify { expect(@score).to be == "15-0" }
>
>         context "A" do
>           before(:each) { tennis.point_to_player_a }
>           specify { expect(@score).to be == "30-0" }
>         end
>
>         context "B" do
>           before(:each) { tennis.point_to_player_b }
>           specify { expect(@score).to be == "15-15" }
>
> And it so goes on for quite a bit longer in the same style. The first step
> is to factor out the duplication into context helpers, which leaves code
> like this:
>
>   game_started do
>     score_is_now "0-0"
>
>     context "with no advantages" do
>       point_to_player :a do
>         score_is_now "15-0"
>
>         point_to_player :a do
>           score_is_now "30-0"
>         end
>
> The problem now is that the meaning of the specification is now hidden in
> the implementation of the helpers, in this case it was in spec_helper[3].
> This gives very poorly composed methods with mixed levels of abstraction.
> So the final step is to parameterise the spec DSL with blocks of code from
> the specs itself, allowing you to write:
>
>   specification_dsl :tennis do
>     for_context :game_not_started do
>       nothing
>     end
>
>     for_context :game_started do
>       tennis.start_game
>     end
>
>     for_context :point_to_player do |player|
>       # Heh, just noticed writing this email that I could be doing
>       # tennis.send(:"point_to_player_#{player}") here, hey ho
>       player == :a ? tennis.point_to_player_a : tennis.point_to_player_b
>     end
>
>     for_context :deuce do
>       3.times do
>         tennis.point_to_player_a
>         tennis.point_to_player_b
>       end
>     end
>
>     to_expect :score_is_now do |expected_score|
>       expect(@scores.last).to be == expected_score
>     end
>
> This has finally put the spec and its definition back together[4], with
> the DSL definition and its voodoo metaprogramming hidden away in
> spec_helper[5].
>
> Unfortunately there''s a problem with this implementation, which is
that it
> fools RSpec into thinking expectation failures are all coming from
> spec_helper.rb, which makes for rather useless error messages. I
haven''t
> investigated this.
>
> Anyway, the point of explaining this example is to ask for
people''s
> opinions myself. A few obvious questions are:
>
> * What sort of DSL-building have you tried/seen?
> * Is this worth the effort over e.g. helper modules and custom matchers?
> (E.g. is the terseness worth the indirection?)
> * Is this possible in a simpler way with existing context tools in RSpec?
> * If not, is it worth trying to make this DSL definition reusable?
> * Are the situations where this is useful inherently best tackled another
> way?
>
> I''m interested in any opinions though, especially if you have a
valid
> reason why this is a bad idea / approach. I''ve sat on it long
enough I''m
> clearly not going to have any more insights on my own any time soon.
>
> Cheers
> Ash
>
> [1] http://codingdojo.org/cgi-bin/wiki.pl?KataTennis
> [2]
>
https://github.com/ashmoran/tennis_kata/blob/ff46b7e502337988940ef9ed881c934a04c53766/spec/tennis_spec.rb
> [3]
>
https://github.com/ashmoran/tennis_kata/blob/a159bac3c3d6180c6f1b83770e6cd678fa750b33/spec/spec_helper.rb
> [4]
> https://github.com/ashmoran/tennis_kata/blob/master/spec/tennis_contract.rb
> [5]
> https://github.com/ashmoran/tennis_kata/blob/master/spec/spec_helper.rb
>
> --
> http://www.patchspace.co.uk/
> http://www.linkedin.com/in/ashmoran
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/rspec-users/attachments/20130207/27e26df7/attachment.html>

Ash Moran

2013-Feb-08 18:39 UTC

head link

[rspec-users] Building higher-level DSLs

On 7 Feb 2013, at 23:32, Sam Goldman <samwgoldman at gmail.com> wrote:
> I''m not sure if this is what you are looking for, but I do have
some experience with writing DSLs on top of RSpec.
> 
> I made a project that lets you write tests for HTTP APIs in a bit more of a
straightforward manner[1]. This isn''t quite the level of DSL-ness that
it seems you are going for?i.e., the specs using this library still very much
look like RSpec.
> 
> In the context of this project, I think that it was worth it. I very much
enjoy writing specs in the style afforded by this DSL.
This looks like a nice little DSL, and you''re doing exactly what I was
thinking of with the request contexts:

    describe "My Awesome App" do
      include HTTPSpec::DSL::Resource

      get "/foobar" do
        # ? expectations ?

I don''t know how much work would be involved, but a few extra methods
might make it cover HTTP completely, e.g. I could imagine

    it "should be successful" do
      do_request
      status.should eq(200)
    end

turning into

    response_has_status 200

or some such. This is the dilemma I run into, how long to spend adding things
like this.

> I think the tradeoff is about test noise. Let me define noise as details
that are only relevant at a lower level of abstraction from the spec''s
level of abstraction. Ideally, your DSL will only hide noise. If your DSL hides
signal, then consumers of your dialect will need to understand the internals.
Very well put. I like the signal metaphor, and I will be stealing it, hope you
don''t mind :)

I suspect this fear of signal-hiding is the thinking process people are using
when they don''t use RSpec because it has "too much magic".
(On a bigger scale, I suspect it''s the reason some people use Python
where Ruby is a viable option, as that seems to be a whole language founded on
the concept of "no magic". I''m basing this on very limited
knowledge though.)

Leaky abstractions are a huge waste of time for consumers of so it puts a burden
on the designer of a DSL to get it right. RSpec does this very well in my
experience. Actually buggy software in general is a huge cause of waste, a leaky
abstraction is just one type of bug.
> That said, I think it''s a bad idea to start with a DSL.
I''m torn between agreeing with this and arguing for the opposite, which
means there must be some rule I''m missing to decide between them.

When I use Cucumber, I write the DSL first and then figure out how to automate
it. This has advantages in that the tests are independent of the interface(s)
but does front-load some work writing an adapter layer. I''ve always got
in the back of my mind that this is how Christopher Alexander said we should
build buildings, create the language first and then use it to build a building,
even if only one, and he seemed to have a good understanding of design. Mind
you, he had several thousand years of architecture to extract patterns from
first.
> Write your tests directly, then once you have a good number of tests, ask
yourself if a non-leaky abstraction exists would let you convey your specs at a
higher level. You might find that this leads to a new domain concept for your
production code.
When I use RSpec I do this, and use duplication to locate missing domain
concepts in the tests, just like you say. Maybe this is because it''s
possible to get quicker feedback by committing to an interface immediately. But
that doesn''t explain why I''d do that in RSpec and yet I
wouldn''t dream of using ''When I click on the "foo"
div'' steps in Cucumber. OTOH I *do* use Aruba, which directly
contradicts that, but then I''ve found command line interfaces to be
more stable, and they''re also much simpler, and Aruba removes the
front-loading effort. HTTPSpec might fall into this category.
> Alternatively it might be better to augment your tests with a little DSL.
Agreed that you can always improve later when you know for certain what domain
concepts you need available first-class.
> Sorry if that''s a little too philosophical :P
Heh, not at all, it''s got me thinking (out loud, as usual) and is all
practical ideas I can test.

I think what I''ll do as an experiment is extract my RSpec meta-DSL, try
to fix the error reporting line number bug, and do a few katas DSL-first (i.e.
no plain `it` blocks and maybe no `context` blocks either - `describe` will
probably stay). Then I''ll be able to see what pain and/or benefits it
has.

Cheers
Ash

-- 
http://www.patchspace.co.uk/
http://www.linkedin.com/in/ashmoran

rspec users - Feb 2013 - Building higher-level DSLs

[rspec-users] Building higher-level DSLs

[rspec-users] Building higher-level DSLs

[rspec-users] Building higher-level DSLs