Puppet users - Dec 2013 - Puppet module testing & code coverage...

Afternoon all

I''ve read a fair few blogs, slideshares and other sources recently
around
Module testing in Puppet, and it has spurned me on to start writing a test 
framework around our internal modules... Yes, we aren''t currently
testing
our internal modules... Yes, we *should *be, and yes, I could give you a 
list of excuses why we aren''t, but the long and short is we (I) want to
start to :) 

Anyhow, I''ve been playing with puppetlabs_spec_helper, puppet-lint, 
hiera-puppet-helper, librarian-puppet and Jenkins over the past couple of 
days, and have got a working test framework finally... After lots of to-ing 
and fro-ing with Hiera helpers, Rspec helpers, etc... 

So I''ve currently got Jenkins running parser tests, lint tests, and am
guna
start running spec tests shortly, of which I currently have the grand sum 
of 1 (yes, one).
However one thing I''m struggling to find is how to generate Puppet test
code coverage stats... 

So Puppetlabs_spec_helper includes *''rake* *coverage*'', with a
description
of: 
"rake coverage         # Generate code coverage information".

However this doesn''t appear to currently do anything useful :( 

$ ruby -S rcov -Ispec:lib --exclude spec
./spec/classes/server/linux_spec.rb
> -bash-4.1$
>
I found a snippet on the Puppetlabs blog [1] which says:

''coverage'' runs rcov on your spec tests; it is only useful if
your module 
> includes Ruby extensions to Puppet.
>
So my reading of that is that rcov doesn''t check Puppet manifest
coverage,
only Ruby extensions? 
If that''s the case, is there any alternative that *will *generate
Puppet
manifest code coverage? 

Or am I completely barking up the wrong tree???

As always, thanks in advance for any responses/comments/etc... 

Cheers
Gavin 

[1] http://puppetlabs.com/blog/the-next-generation-of-puppet-module-testing

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1418d761-9675-4ef8-9135-ce3c8b2b70c5%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Gavin,

Congratulations on your momentous decision. And it is an important one.

You may want to go to www.amazon.com and pick up How Google Tests Software
by Whittaker, Arbon, and Carollo. 

It is also a serious effort.

Stuart

Afternoon all
>
> I''ve read a fair few blogs, slideshares and other sources recently
around
> Module testing in Puppet, and it has spurned me on to start writing a test 
> framework around our internal modules... Yes, we aren''t currently
testing
> our internal modules... Yes, we *should *be, and yes, I could give you a 
> list of excuses why we aren''t, but the long and short is we (I)
want to
> start to :) 
>
> Anyhow, I''ve been playing with puppetlabs_spec_helper,
puppet-lint,
> hiera-puppet-helper, librarian-puppet and Jenkins over the past couple of 
> days, and have got a working test framework finally... After lots of to-ing
> and fro-ing with Hiera helpers, Rspec helpers, etc... 
>
> So I''ve currently got Jenkins running parser tests, lint tests,
and am
> guna start running spec tests shortly, of which I currently have the grand 
> sum of 1 (yes, one).
> However one thing I''m struggling to find is how to generate Puppet
test
> code coverage stats... 
>
> So Puppetlabs_spec_helper includes *''rake* *coverage*'',
with a
> description of: 
> "rake coverage         # Generate code coverage information".
>
> However this doesn''t appear to currently do anything useful :( 
>
> $ ruby -S rcov -Ispec:lib --exclude spec 
>> ./spec/classes/server/linux_spec.rb
>> -bash-4.1$
>>
>
> I found a snippet on the Puppetlabs blog [1] which says:
>
> ''coverage'' runs rcov on your spec tests; it is only
useful if your module
>> includes Ruby extensions to Puppet.
>>
>
> So my reading of that is that rcov doesn''t check Puppet manifest
coverage,
> only Ruby extensions? 
> If that''s the case, is there any alternative that *will *generate
Puppet
> manifest code coverage? 
>
> Or am I completely barking up the wrong tree???
>
> As always, thanks in advance for any responses/comments/etc... 
>
> Cheers
> Gavin 
>
> [1] 
> http://puppetlabs.com/blog/the-next-generation-of-puppet-module-testing
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/82001dbc-cb72-455e-bda7-7dc193f10167%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi,

On 12/03/2013 06:23 PM, Stuart Cracraft wrote:
> You may want to go to www.amazon.com <http://www.amazon.com> and pick
up
> How Google Tests Software by Whittaker, Arbon, and Carollo.
I encourage the plugging of pertinent literature, but I feel
uncomfortable about linking to specific retailers. That just crosses the
line to blunt advertising.
> If that''s the case, is there any alternative that /will /generate
Puppet
> manifest code coverage?
I''m not sure what you expect from ''manifest code
coverage''.

If you write a spec to find out how a piece of manifest code (e.g. a
class) behaves, the blog you linked indicates that ''rake spec''
is the
task you''re looking for.

HTH,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/529F285D.4000308%40alumni.tu-berlin.de.
For more options, visit https://groups.google.com/groups/opt_out.