Puppet users - Sep 2013 - Parse puppet manifests in rspec tests

I want to do a syntax check of all our managed puppet manifests in rspec. 
It''s an easy way to generate build reports for use in a CI.

My first attempt looks like this:

    Puppet::Face[:parser,
''0.0.1''].validate(''puppet/manifests/site.pp'')

But this throws the following error: 

    Puppet::Error: Could not parse for environment production: Error 
converting value for param ''modulepath'': Could not find value
for $confdir
at        
    /Users/jkyle/Projects/puppet/occam/puppet/manifests/site.pp:2
    from 
/Users/jkyle/.rvm/gems/ruby-2.0.0-p247@puppet/gems/puppet-3.3.0/lib/puppet/parser/parser_support.rb:166:in
`rescue in parse''

Even though the same file passes if tested via

    puppet parser validate puppet/manifests/site.pp

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On 2013-16-09 24:05, James Kyle wrote:
> I want to do a syntax check of all our managed puppet manifests in
> rspec. It''s an easy way to generate build reports for use in a CI.
>
> My first attempt looks like this:
>
>      Puppet::Face[:parser,
''0.0.1''].validate(''puppet/manifests/site.pp'')
>
> But this throws the following error:
>
>      Puppet::Error: Could not parse for environment production: Error
> converting value for param ''modulepath'': Could not find
value for
> $confdir at
>      /Users/jkyle/Projects/puppet/occam/puppet/manifests/site.pp:2
>      from
>
/Users/jkyle/.rvm/gems/ruby-2.0.0-p247@puppet/gems/puppet-3.3.0/lib/puppet/parser/parser_support.rb:166:in
> `rescue in parse''
>
> Even though the same file passes if tested via
>
>      puppet parser validate puppet/manifests/site.pp
>
The puppet 3x parser is not really suitable as a "syntax checker"
only,
it does things while it is parsing that are unwanted in a pure "syntax 
checking" use-case (e.g. some evaluation IIRC, imports, if you try to 
use collection you need to have store-configs turned or or it will barf, 
it will stop on the first found error, etc.). It is simply not designed 
to support the syntax-checker use-case well.

The future parser performs parsing as a separate step, then validation, 
and finally translation to the 3x AST for evaluation. By using only the 
parse and validation steps you could construct a language validator that 
can be used in rspec tests. It should be possible to figure out how to 
do this by looking at the rspec tests in spec/unit/pops/validator (i.e. 
what you need to set up) - it is however not perfect as some of the 
checking is delegated to the transformation and runtime classes. (This 
will be much better with a new evaluator and further improved validator 
- i.e. hopefully in puppet 4). Still, you would capture many syntax and 
semantic issues this way.

There are alternatives though, the corresponding functionality in 
Geppetto is available as plugins to travis and jenkins (have to check if 
the geppetto-jenkins plugin is generally available yet). You can also 
run puppet-lint as part of this setup. The Geppetto CI plugins should 
give you a more high quality checking than what you can (easily) achieve 
by using ''puppet as a library''. These are used when publishing
to the
forge to check quality issues BTW.
One advantage with the Geppetto validator is that it also supports 
stylistic checks and checks for problematic expressions (like 
puppet-lint), something that the runtime parser does not do.

Then there are runtime issues naturally - none of the tools can capture 
those problems (rspec-puppet can find some).

Not being able to run the parser via Puppet Faces is the least of the 
problems IMO.

While I would love to get help to ensure that the Puppet runtime 
validator is perfect your best bet is to look at the Geppetto validator 
for CI if you want to do something right away.

Hope that helps.
Regards

- henrik


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.