rspec users - Jan 2008 - unusual challenges speccing external software

This is something I''ve come up against twice. First, I have an
interactive editor which calls vi, emacs, or TextMate (etc.) via Unix,
and then loads the file handed to the editor back into IRB after it''s
edited and saved. I guess that makes it a special case of a more
general question, which is how do you spec external processes, with a
slightly more complicated version of the standard answer, mocks.

The second instance is more complicated. I''m writing a MIDI code
generator and using it to drive Propellerhead Reason, which is
third-party closed-source commercial software which will run on my OS
X box but which is not actually transparent to Unix in any meaningful
way that I''m aware of. (The code generator can very probably also
drive any arbitrary MIDI consumer.) I''m using code from Topher
Cyll''s
book "Practical Ruby Projects" to create Ruby wrappers to C methods in
Apple''s CoreMIDI library. The trap here is I''m calling those C
methods
to generate MIDI elsewhere in the OS in some unknown way. That makes
speccing weird.

I''ve been able to refactor this code in a kind-of spec-first way. I
write scripts which use the API successfully and actually generate
music I can hear. Then I change the internals of the API and run the
scripts again to ensure they still generate the same music. In that
sense I''m basically doing the right thing, even though it''s
not
particularly systematic or automated. I''m comparing the output with my
ears rather than in a way that code can grok, but it still has the
feel of a spec-first workflow.

For developing the API itself, especially the parts which generate
MIDI, I set up a layer of abstraction. Instead of directly driving the
code which sends MIDI to external software, the code which generates
the music drives an output adapter, and the output adapter sends MIDI
to external software - or, in the case of the TextOutputAdapter, just
text-dumps the music. Since it all comes back as text, it''s very easy
to write specs against.

But I''ve ended up with three artifacts that are *either* byproducts of
bad design *or* bad products of a compromised spec process. I created
three mock *classes* (as opposed to mock objects). For example, one is
the ArrayGenerator. In this system Generators return notes. The
ArrayGenerator "generates" an array of notes. It''s a silly
implementation, but it''s useful if you want to spec that some code can
take an arbitrary generator, which generates arbitrary notes, and then
process those notes correctly.

Sometimes I think my design has too many abstract parts. Sometimes I''m
happy with it. I want to improve my speccing approach so I can operate
with greater certainty. I think probably this approach is a good
direction, and that I should change the specs by making them more
specific. (For instance, there are places where I check that a
Generator subclass can do stuff, even though Generator itself isn''t
fully specced.) I''m not quite sure though.

-- 
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

At the risk of sounding a bit silly, what''s your question? I
couldn''t
find a question mark in the whole email...?

Daniel

On 17 Jan 2008, at 07:20 17 Jan 2008, Giles Bowkett wrote:
> This is something I''ve come up against twice. First, I have an
> interactive editor which calls vi, emacs, or TextMate (etc.) via Unix,
> and then loads the file handed to the editor back into IRB after
it''s
> edited and saved. I guess that makes it a special case of a more
> general question, which is how do you spec external processes, with a
> slightly more complicated version of the standard answer, mocks.
>
> The second instance is more complicated. I''m writing a MIDI code
> generator and using it to drive Propellerhead Reason, which is
> third-party closed-source commercial software which will run on my OS
> X box but which is not actually transparent to Unix in any meaningful
> way that I''m aware of. (The code generator can very probably also
> drive any arbitrary MIDI consumer.) I''m using code from Topher
Cyll''s
> book "Practical Ruby Projects" to create Ruby wrappers to C
methods in
> Apple''s CoreMIDI library. The trap here is I''m calling
those C methods
> to generate MIDI elsewhere in the OS in some unknown way. That makes
> speccing weird.
>
> I''ve been able to refactor this code in a kind-of spec-first way.
I
> write scripts which use the API successfully and actually generate
> music I can hear. Then I change the internals of the API and run the
> scripts again to ensure they still generate the same music. In that
> sense I''m basically doing the right thing, even though
it''s not
> particularly systematic or automated. I''m comparing the output
with my
> ears rather than in a way that code can grok, but it still has the
> feel of a spec-first workflow.
>
> For developing the API itself, especially the parts which generate
> MIDI, I set up a layer of abstraction. Instead of directly driving the
> code which sends MIDI to external software, the code which generates
> the music drives an output adapter, and the output adapter sends MIDI
> to external software - or, in the case of the TextOutputAdapter, just
> text-dumps the music. Since it all comes back as text, it''s very
easy
> to write specs against.
>
> But I''ve ended up with three artifacts that are *either*
byproducts of
> bad design *or* bad products of a compromised spec process. I created
> three mock *classes* (as opposed to mock objects). For example, one is
> the ArrayGenerator. In this system Generators return notes. The
> ArrayGenerator "generates" an array of notes. It''s a
silly
> implementation, but it''s useful if you want to spec that some code
can
> take an arbitrary generator, which generates arbitrary notes, and then
> process those notes correctly.
>
> Sometimes I think my design has too many abstract parts. Sometimes
I''m
> happy with it. I want to improve my speccing approach so I can operate
> with greater certainty. I think probably this approach is a good
> direction, and that I should change the specs by making them more
> specific. (For instance, there are places where I check that a
> Generator subclass can do stuff, even though Generator itself
isn''t
> fully specced.) I''m not quite sure though.
>
> -- 
> Giles Bowkett
>
> Podcast: http://hollywoodgrit.blogspot.com
> Blog: http://gilesbowkett.blogspot.com
> Portfolio: http://www.gilesgoatboy.org
> Tumblelog: http://giles.tumblr.com
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users

> At the risk of sounding a bit silly, what''s your question? I
couldn''t
> find a question mark in the whole email...?
OK, well, am I on the right track? Should I do it differently? If so,
in what particular way?

If you were speccing external software, would you do it this way? Do I
sound like a crazy person? If yes, more so than usual? Or only as much
as I usually do? Is this a sensible way to spec software which
interacts with but has no direct access to external software, or
should I just isolate my dependencies on external software and write
it off as unspeccable while speccing everything else? Is there a good
rule of thumb for a BDD workflow when you can''t programmatically
capture some of the output? Because that''s really what the problem is.
I built a whole set of spec objects to capture the output as text
because the system''s real output isn''t capturable, and if you
can''t
capture it, you can''t check it against the spec. My code has flaws but
with good specs it''s pretty easy to isolate the flaws. In this case I
can''t tell yet if the flaws are just flaws in my weird semi-BDD
approach or in the code itself. I''ve probably done enough question
marks but basically, is there a good set of guiding principles I can
use to apply RSpec well in unconventional contexts?

-- 
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

On Jan 17, 2008 10:24 AM, Giles Bowkett <gilesb at gmail.com>
wrote:
> > At the risk of sounding a bit silly, what''s your question? I
couldn''t
> > find a question mark in the whole email...?
>
> OK, well, am I on the right track? Should I do it differently? If so,
> in what particular way?
Giles - I think you''ll find more people with more experience in this
area on the testdrivendevelopment at yahoogroups.com list. That said, and
only having glanced at your mail, here''s a thought or two:

The TDD line on this, as I understand it, is that you should write
your main application code exactly as you want to write it, imagining
that you had the perfect API to talk to and stubbing that API in your
tests. Then, when hooking up to the real services, you write thin
adapters that simply convert messages to the right format. I *think*
that''s what you said you are doing.

HTH,
David
>
> If you were speccing external software, would you do it this way? Do I
> sound like a crazy person? If yes, more so than usual? Or only as much
> as I usually do? Is this a sensible way to spec software which
> interacts with but has no direct access to external software, or
> should I just isolate my dependencies on external software and write
> it off as unspeccable while speccing everything else? Is there a good
> rule of thumb for a BDD workflow when you can''t programmatically
> capture some of the output? Because that''s really what the problem
is.
> I built a whole set of spec objects to capture the output as text
> because the system''s real output isn''t capturable, and if
you can''t
> capture it, you can''t check it against the spec. My code has flaws
but
> with good specs it''s pretty easy to isolate the flaws. In this
case I
> can''t tell yet if the flaws are just flaws in my weird semi-BDD
> approach or in the code itself. I''ve probably done enough question
> marks but basically, is there a good set of guiding principles I can
> use to apply RSpec well in unconventional contexts?
>
>
> --
> Giles Bowkett
>
> Podcast: http://hollywoodgrit.blogspot.com
> Blog: http://gilesbowkett.blogspot.com
> Portfolio: http://www.gilesgoatboy.org
> Tumblelog: http://giles.tumblr.com
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

> Giles - I think you''ll find more people with more experience in
this
> area on the testdrivendevelopment at yahoogroups.com list. That said, and
> only having glanced at your mail, here''s a thought or two:
>
> The TDD line on this, as I understand it, is that you should write
> your main application code exactly as you want to write it, imagining
> that you had the perfect API to talk to and stubbing that API in your
> tests. Then, when hooking up to the real services, you write thin
> adapters that simply convert messages to the right format. I *think*
> that''s what you said you are doing.
Yes. Sorry about the Great American Novel - that is pretty much what
I''m doing.

-- 
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

On Jan 17, 2008 11:33 AM, Giles Bowkett <gilesb at gmail.com>
wrote:
> > Giles - I think you''ll find more people with more experience
in this
> > area on the testdrivendevelopment at yahoogroups.com list. That said,
and
> > only having glanced at your mail, here''s a thought or two:
> >
> > The TDD line on this, as I understand it, is that you should write
> > your main application code exactly as you want to write it, imagining
> > that you had the perfect API to talk to and stubbing that API in your
> > tests. Then, when hooking up to the real services, you write thin
> > adapters that simply convert messages to the right format. I *think*
> > that''s what you said you are doing.
>
> Yes. Sorry about the Great American Novel - that is pretty much what
I''m doing.
Then that''s sane :)

It lets you keep changes to your internal app separate from changes to
the service - even allows you to easily swap in new services - and
lets you "talk in your language" within the app rather than being
subject to an API that may or may not make sense to you most of the
time.
>
> --
>
> Giles Bowkett
>
> Podcast: http://hollywoodgrit.blogspot.com
> Blog: http://gilesbowkett.blogspot.com
> Portfolio: http://www.gilesgoatboy.org
> Tumblelog: http://giles.tumblr.com
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>