Puppet users - Sep 2011 - How To Expand an .erb Template Without Doing a Full Puppet Run?

We use puppet to distribute named zone files, like many
of you do. We use git to maintain these files, which are
then pulled by the puppet master machine. These zone files are
actually puppet template .erb files.

The other day, I made a mistake which resulted in some zone
files with syntax errors in them. Puppet faithfully distributed
the erroneous zone files to the name servers, which killed them.

To prevent this from happening again, I want to be able to
somehow expand the template files and run named-checkzone
as part of a git pre-commit hook. I want this expansion to
occur without making a normal puppet run so that the bad
files won''t get distributed.

Can anybody point me to a technique for doing this?

Cordially,
Jon Forrest

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Sat, Sep 10, 2011 at 5:36 PM, Jon Forrest <nobozo@gmail.com> wrote:
> We use puppet to distribute named zone files, like many
> of you do. We use git to maintain these files, which are
> then pulled by the puppet master machine. These zone files are
> actually puppet template .erb files.
>
> The other day, I made a mistake which resulted in some zone
> files with syntax errors in them. Puppet faithfully distributed
> the erroneous zone files to the name servers, which killed them.
>
> To prevent this from happening again, I want to be able to
> somehow expand the template files and run named-checkzone
> as part of a git pre-commit hook. I want this expansion to
> occur without making a normal puppet run so that the bad
> files won''t get distributed.
>
> Can anybody point me to a technique for doing this?
>

git cat-file blob :0:$indexfile | erb -x -T - | ruby -c 2> $error_msg >
/dev/null ;

from:

http://projects.puppetlabs.com/projects/1/wiki/Puppet_Version_Control

should get you going.



>
> Cordially,
> Jon Forrest
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users+unsubscribe@**
> googlegroups.com <puppet-users%2Bunsubscribe@googlegroups.com>.
> For more options, visit this group at http://groups.google.com/**
>
group/puppet-users?hl=en<http://groups.google.com/group/puppet-users?hl=en>
> .
>
>

-- 
Nigel Kersten
Product Manager, Puppet Labs

*Join us for **PuppetConf *
<http://www.bit.ly/puppetconfsig>
Sept 22/23 Portland, Oregon, USA.
*
*

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Sep 10, 2011, at 9:26 PM, Nigel Kersten wrote:
> On Sat, Sep 10, 2011 at 5:36 PM, Jon Forrest <nobozo@gmail.com>
wrote:
> 
>> We use puppet to distribute named zone files, like many
>> of you do. We use git to maintain these files, which are
>> then pulled by the puppet master machine. These zone files are
>> actually puppet template .erb files.
>> 
>> The other day, I made a mistake which resulted in some zone
>> files with syntax errors in them. Puppet faithfully distributed
>> the erroneous zone files to the name servers, which killed them.
>> 
>> To prevent this from happening again, I want to be able to
>> somehow expand the template files and run named-checkzone
>> as part of a git pre-commit hook. I want this expansion to
>> occur without making a normal puppet run so that the bad
>> files won''t get distributed.
>> 
>> Can anybody point me to a technique for doing this?
>> 
> 
> 
> git cat-file blob :0:$indexfile | erb -x -T - | ruby -c 2> $error_msg
>
> /dev/null ;
> 
> from:
> 
> http://projects.puppetlabs.com/projects/1/wiki/Puppet_Version_Control
> 
> should get you going.
> 

Maybe I''m missing something here, but I think Jon was asking something
a bit different -- he doesn''t want to check the validity of the erb
template (i.e. ruby syntax check) but syntax check the named zone file generated
by the template.

So the tricky bit is how to get the variables out of the puppet manifests that
the erb template needs to generate the output file that
''would'' be generated by a new puppet run for this node --
without the actual puppet run (as he asks at the end of the email).

My first thought is that the only accurate way to do this is by doing a full
puppet run, as any part of the node''s manifest could effect the
variables used in the zone file template. You should be able to get away with a
--noop run so the changes won''t actually be applied (because noop does
generate files from templates, but you would need to have a way to capture the
newly generated zone file on the client host and run the named-checkzone there.

Cheers,

Jonathan
-------------------------------------------------------------------------------
Jonathan Stanton	 jonathan@spreadconcepts.com
Spread Group Messaging	www.spread.org
Spread Concepts LLC 	www.spreadconcepts.com
-------------------------------------------------------------------------------



> 
> 
> 
>> 
>> Cordially,
>> Jon Forrest
>> 
>> --
>> You received this message because you are subscribed to the Google
Groups
>> "Puppet Users" group.
>> To post to this group, send email to puppet-users@googlegroups.com.
>> To unsubscribe from this group, send email to
puppet-users+unsubscribe@**
>> googlegroups.com <puppet-users%2Bunsubscribe@googlegroups.com>.
>> For more options, visit this group at http://groups.google.com/**
>>
group/puppet-users?hl=en<http://groups.google.com/group/puppet-users?hl=en>
>> .
>> 
>> 
> 
> 
> -- 
> Nigel Kersten
> Product Manager, Puppet Labs
> 
> *Join us for **PuppetConf *
> <http://www.bit.ly/puppetconfsig>
> Sept 22/23 Portland, Oregon, USA.
> *
> *
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
>

On 9/10/2011 6:57 PM, Jonathan Stanton wrote:
>
> Maybe I''m missing something here, but I think Jon was asking
> something a bit different -- he doesn''t want to check the validity
of
> the erb template (i.e. ruby syntax check) but syntax check the named
> zone file generated by the template.
Precisely. Maybe later I''ll face the issue of ruby syntax
problems but right now I need to detect named syntax errors
before they cause problems.
> So the tricky bit is how to get the variables out of the puppet
> manifests that the erb template needs to generate the output file
> that ''would'' be generated by a new puppet run for this
node --
> without the actual puppet run (as he asks at the end of the email).
Precisely again.
> My first thought is that the only accurate way to do this is by doing
> a full puppet run, as any part of the node''s manifest could effect
> the variables used in the zone file template. You should be able to
> get away with a --noop run so the changes won''t actually be
applied
> (because noop does generate files from templates, but you would need
> to have a way to capture the newly generated zone file on the client
> host and run the named-checkzone there.
That''s what I figured. I was hoping that there would be an easier way
that could somehow do a facter run but only run the minimal amount
of puppet.

Jon


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Perhaps just add an exec step or two to your regular run? Put the file in a
temp location then move it to the live one if it checks out?
On Sep 10, 2011 8:01 PM, "Jon Forrest" <nobozo@gmail.com>
wrote:
> On 9/10/2011 6:57 PM, Jonathan Stanton wrote:
>>
>> Maybe I''m missing something here, but I think Jon was asking
>> something a bit different -- he doesn''t want to check the
validity of
>> the erb template (i.e. ruby syntax check) but syntax check the named
>> zone file generated by the template.
>
> Precisely. Maybe later I''ll face the issue of ruby syntax
> problems but right now I need to detect named syntax errors
> before they cause problems.
>
>> So the tricky bit is how to get the variables out of the puppet
>> manifests that the erb template needs to generate the output file
>> that ''would'' be generated by a new puppet run for
this node --
>> without the actual puppet run (as he asks at the end of the email).
>
> Precisely again.
>
>> My first thought is that the only accurate way to do this is by doing
>> a full puppet run, as any part of the node''s manifest could
effect
>> the variables used in the zone file template. You should be able to
>> get away with a --noop run so the changes won''t actually be
applied
>> (because noop does generate files from templates, but you would need
>> to have a way to capture the newly generated zone file on the client
>> host and run the named-checkzone there.
>
> That''s what I figured. I was hoping that there would be an easier
way
> that could somehow do a facter run but only run the minimal amount
> of puppet.
>
> Jon
>
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I just change the restart command on the named service.  Instead of
letting puppet do a:
    /sbin/service named stop and then start
I have it do
    /sbin/service named reload.

Which on redhat only runs
    /usr/sbin/rndc reload

If that fails, named keeps running with the previous config it loaded
with no interruption, puppet complains that the restart failed, I get
notified and fix the zone file.

On Sep 11, 12:00 am, Aaron Grewell <aaron.grew...@gmail.com>
wrote:
> Perhaps just add an exec step or two to youir regular run? Put the file in
a
> temp location then move it to the live one if it checks out?
> On Sep 10, 2011 8:01 PM, "Jon Forrest" <nob...@gmail.com>
wrote:
>
>
>
>
>
>
>
> > On 9/10/2011 6:57 PM, Jonathan Stanton wrote:
>
> >> Maybe I''m missing something here, but I think Jon was
asking
> >> something a bit different -- he doesn''t want to check the
validity of
> >> the erb template (i.e. ruby syntax check) but syntax check the
named
> >> zone file generated by the template.
>
> > Precisely. Maybe later I''ll face the issue of ruby syntax
> > problems but right now I need to detect named syntax errors
> > before they cause problems.
>
> >> So the tricky bit is how to get the variables out of the puppet
> >> manifests that the erb template needs to generate the output file
> >> that ''would'' be generated by a new puppet run
for this node --
> >> without the actual puppet run (as he asks at the end of the
email).
>
> > Precisely again.
>
> >> My first thought is that the only accurate way to do this is by
doing
> >> a full puppet run, as any part of the node''s manifest
could effect
> >> the variables used in the zone file template. You should be able
to
> >> get away with a --noop run so the changes won''t actually
be applied
> >> (because noop does generate files from templates, but you would
need
> >> to have a way to capture the newly generated zone file on the
client
> >> host and run the named-checkzone there.
>
> > That''s what I figured. I was hoping that there would be an
easier way
> > that could somehow do a facter run but only run the minimal amount
> > of puppet.
>
> > Jon
>
> > --
> > You received this message because you are subscribed to the Google
Groups
>
> "Puppet Users" group.> To post to this group, send email to
puppet-users@googlegroups.com.
> > To unsubscribe from this group, send email to
>
> puppet-users+unsubscribe@googlegroups.com.> For more options, visit this
group at
>
> http://groups.google.com/group/puppet-users?hl=en.
>
>
>
>
>
>
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

@Jon - If you find an answer to your question, please post it to the  
list.  I''d like to see the answer and there are probably lots of other
folks who would benefit from the info being in the mailing list  
archives.

Good luck.

On Sep 11, 2011, at 12:00 AM, Aaron Grewell wrote:
> Perhaps just add an exec step or two to your regular run? Put the  
> file in a temp location then move it to the live one if it checks out?
>
> On Sep 10, 2011 8:01 PM, "Jon Forrest" <nobozo@gmail.com>
wrote:
> > On 9/10/2011 6:57 PM, Jonathan Stanton wrote:
> >>
> >> Maybe I''m missing something here, but I think Jon was
asking
> >> something a bit different -- he doesn''t want to check the
> validity of
> >> the erb template (i.e. ruby syntax check) but syntax check the  
> named
> >> zone file generated by the template.
> >
> > Precisely. Maybe later I''ll face the issue of ruby syntax
> > problems but right now I need to detect named syntax errors
> > before they cause problems.
> >
> >> So the tricky bit is how to get the variables out of the puppet
> >> manifests that the erb template needs to generate the output file
> >> that ''would'' be generated by a new puppet run
for this node --
> >> without the actual puppet run (as he asks at the end of the
email).
> >
> > Precisely again.
> >
> >> My first thought is that the only accurate way to do this is by  
> doing
> >> a full puppet run, as any part of the node''s manifest
could effect
> >> the variables used in the zone file template. You should be able
to
> >> get away with a --noop run so the changes won''t actually
be applied
> >> (because noop does generate files from templates, but you would  
> need
> >> to have a way to capture the newly generated zone file on the  
> client
> >> host and run the named-checkzone there.
> >
> > That''s what I figured. I was hoping that there would be an
easier
> way
> > that could somehow do a facter run but only run the minimal amount
> > of puppet.
> >
> > Jon
> >
> >
> > --
> > You received this message because you are subscribed to the Google  
> Groups "Puppet Users" group.
> > To post to this group, send email to puppet-users@googlegroups.com.
> > To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
> .
> > For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
> .
> >
>
> -- 
> You received this message because you are subscribed to the Google  
> Groups "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
> .
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
> .
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I''m thinking the answer is to update your init script, to make sure
that
restart calls rndc reload, and specify "hasrestart" in your resource
definition.

I suggest this because bind should ignore bad zone files, and keep serving
the old zones, when reloaded via rndc.

Maybe the answer is to also incorporate named-checkzone and some sort of
email notification into the restart section of your init script.

As always test, and you milage may vary.

Cheers,
Brian

On Sun, Sep 11, 2011 at 10:35 AM, Dan White <ygor@comcast.net> wrote:
> @Jon - If you find an answer to your question, please post it to the list.
>  I''d like to see the answer and there are probably lots of other
folks who
> would benefit from the info being in the mailing list archives.
>
> Good luck.
>
> On Sep 11, 2011, at 12:00 AM, Aaron Grewell wrote:
>
> Perhaps just add an exec step or two to your regular run? Put the file in a
> temp location then move it to the live one if it checks out?
> On Sep 10, 2011 8:01 PM, "Jon Forrest" <nobozo@gmail.com>
wrote:
> > On 9/10/2011 6:57 PM, Jonathan Stanton wrote:
> >>
> >> Maybe I''m missing something here, but I think Jon was
asking
> >> something a bit different -- he doesn''t want to check the
validity of
> >> the erb template (i.e. ruby syntax check) but syntax check the
named
> >> zone file generated by the template.
> >
> > Precisely. Maybe later I''ll face the issue of ruby syntax
> > problems but right now I need to detect named syntax errors
> > before they cause problems.
> >
> >> So the tricky bit is how to get the variables out of the puppet
> >> manifests that the erb template needs to generate the output file
> >> that ''would'' be generated by a new puppet run
for this node --
> >> without the actual puppet run (as he asks at the end of the
email).
> >
> > Precisely again.
> >
> >> My first thought is that the only accurate way to do this is by
doing
> >> a full puppet run, as any part of the node''s manifest
could effect
> >> the variables used in the zone file template. You should be able
to
> >> get away with a --noop run so the changes won''t actually
be applied
> >> (because noop does generate files from templates, but you would
need
> >> to have a way to capture the newly generated zone file on the
client
> >> host and run the named-checkzone there.
> >
> > That''s what I figured. I was hoping that there would be an
easier way
> > that could somehow do a facter run but only run the minimal amount
> > of puppet.
> >
> > Jon
> >
> >
> > --
> > You received this message because you are subscribed to the Google
Groups
> "Puppet Users" group.
> > To post to this group, send email to puppet-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> > For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>


-- 
<http://aws.amazon.com/solutions/solution-providers/brandorr/>

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Sat, Sep 10, 2011 at 7:59 PM, Jon Forrest <nobozo@gmail.com> wrote:
> On 9/10/2011 6:57 PM, Jonathan Stanton wrote:
>
>>
>> Maybe I''m missing something here, but I think Jon was asking
>> something a bit different -- he doesn''t want to check the
validity of
>> the erb template (i.e. ruby syntax check) but syntax check the named
>> zone file generated by the template.
>>
>
> Precisely. Maybe later I''ll face the issue of ruby syntax
> problems but right now I need to detect named syntax errors
> before they cause problems.

Ah, yes, I did misread your question.

You could adopt the above method to produce the actual templatized output
and then use named to verify it, but as others have said, distros are much
better at letting you verify configs before restarting nowadays, and
I''d
probably push this there.

In the past I tend to solve this with a canary node that implements the new
configuration, and then out of band testing verifies that DNS is still
working happily for critical infrastructure entries before before the
configuration gets pushed to "live" name servers.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hello,

On 09/11/2011 04:59 AM, Jon Forrest wrote:
> On 9/10/2011 6:57 PM, Jonathan Stanton wrote:
>>
>> Maybe I''m missing something here, but I think Jon was asking
>> something a bit different -- he doesn''t want to check the
validity of
>> the erb template (i.e. ruby syntax check) but syntax check the named
>> zone file generated by the template.
>
> Precisely. Maybe later I''ll face the issue of ruby syntax
> problems but right now I need to detect named syntax errors
> before they cause problems.
>
>> So the tricky bit is how to get the variables out of the puppet
>> manifests that the erb template needs to generate the output file
>> that ''would'' be generated by a new puppet run for
this node --
>> without the actual puppet run (as he asks at the end of the email).
>
> Precisely again.
>
>> My first thought is that the only accurate way to do this is by doing
>> a full puppet run, as any part of the node''s manifest could
effect
>> the variables used in the zone file template. You should be able to
>> get away with a --noop run so the changes won''t actually be
applied
>> (because noop does generate files from templates, but you would need
>> to have a way to capture the newly generated zone file on the client
>> host and run the named-checkzone there.
>
> That''s what I figured. I was hoping that there would be an easier
way
> that could somehow do a facter run but only run the minimal amount
> of puppet.
named-checkzone(8) should be suitable tool for your needs.

Best,
-- 
Dominik Zyla

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 9/14/2011 1:58 PM, Dominik Zyla wrote:
> named-checkzone(8) should be suitable tool for your needs.
I already knew that. To repeat, what I''m trying to find out
is a way to get an expanded template without doing a full
run of puppet. I could then run named-checkzone on that
and then do a regular run.

In fact, this is only a special case of the more
general problem of getting access to any file generated
by a puppet run without actually putting the files
in the ultimate destination. In other words, it would
be like a --test run except files will be generated and
kept.

Jon Forrest



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 15/09/11 04:24, Jon Forrest wrote:
> In fact, this is only a special case of the more
> general problem of getting access to any file generated
> by a puppet run without actually putting the files
> in the ultimate destination. In other words, it would
> be like a --test run except files will be generated and
> kept.
I have two ideas. The first is to introduce a preflight check run stage 
which has a clone of whatever you use to generate the zonefile, but puts 
it somewhere else and then does an exec with named-checkzone etc. If run 
stage "main" depends on "preflight" that should do the right
thing
assuming that failures of earlier run stages prevent execution of later 
ones.

I''m not sure if that could lead to a race condition should someone 
update the data between the preflight check and the main manifest run 
though. Also you may find that your preflight stage requires things 
which are in your main stage (installation of packages etc) which will 
just get messy.

The other option which I think I prefer is to have puppet deploy your 
zone files to a staging area, exec named-checkzone and whatever else you 
want to do and then have an exec or similar which moves the staged 
zonefiles into place on your nameserver. Make that exec depend on your 
sanity checks and it will only run if the zonefiles check out OK.

You could even go so far as to stage your zonefiles into a git repo or 
similar on your nameserver and end up with something like:

vcsrepo { "/staging/location":
	# Set up git repo
}

file { "/staging/location/zone.a":
	.... # whatever gets zone.a there
}

exec { "check /staging/location/zone.a":
	command => "named-checkzone /staging/location/zone.a",
	require =>
[File["/staging/location/zone.a"],Vcsrepo["/staging/location"]],
}

exec { "commit staged zonefiles",
	command => "git commit ...",
	require => Exec["check /staging/location/zone.a"],
	# ...
}

exec { "deploy zonefiles to nameserver",
	command => "tar cp . -C /staging/location | tar xp -C 
/var/lib/named/zones",
	require => Exec["commit staged zonefiles"],
	notify  => Service["named"],
}

-- 
Russell Howe
rhowe@moonfruit.com

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hello Jon,

Here is a process that I follow to avoid the type of misconfiguration
problem you encountered.  This process is used to *avoid* the mis-
configured system problem.  It is not a direct test that Puppet will
generate a valid configuration file from a template.

1) A "local" environment is used to deploy the entire system (which is
called a ''role'' for the system).  While I may be developing a
single
class, I still confirm that the entire role can be deployed to my
local environment.  The local server is typically rebuilt 30-50 times
during the course of development of a new manifest.  This allows me to
catch errors early, as many of the mistakes I make result in spurious
errors on Puppet.

The manifests are then committed to SVN and automatically deployed to
a DEV environment, which is identical to the production environment.
This allows live testing against my other servers.  The manifests then
are merged to the ''trunk'' branch and get deployed to a TESTING
environment where selenium regression tests exercise the applications
to make sure they still work as expected.  The manifests get deployed
to QA for manual testing and review and finally to PRODUCTION.

2) Include configuration checks before making a configuration file
live.

One example is with the sudoers file.  I use a file-fragment approach
to generate a sudoers testing file and only make that file live if the
check passes.  Your zone file can use a similar check.

In situations where checks are not possible, I include a
''testing''
flag to prevent a configuration file from being made live.  I manually
check the testing file to make sure it is valid.  The APF firewall
script is an example where this is important.  When I first deployed
APF, puppet exploded with spurious errors because I forgot to open a
port.  I now confirm (using the below process) that the APF
configuration is valid before allowing APF to be started on the remote
servers.

3) Each of my servers has a Dokuwiki page that is generated by puppet
during the deployment.

I use a file fragments pattern for this as well, where each class and
define puts a dokuwik-formatted fragment into a specific directory.
The configuration templates are included into the dokuwiki file
fragments using this approach.  These fragments are assembled into a
single wiki page that is SCP''d to my documentation server.  I then
review the server''s configuration directly on this wiki page to
confirm that the destination server is configured in the manner I
expect.  In the case of APF, I then change the ''testing'' flag
to false
and re-run puppet to make the configuration live.


This is a process, not an answer to your original question about
Puppet generating the configuration file from a template without
actually applying the changes.  That would be a nice feature which I
don''t believe currently exists.  Your mis-configured zone file would
have been spotted in steps 1 and 2; maybe step 3.

Regards,

Jon

On Sep 10, 8:36 pm, Jon Forrest <nob...@gmail.com>
wrote:
> We use puppet to distribute named zone files, like many
> of you do. We use git to maintain these files, which are
> then pulled by the puppet master machine. These zone files are
> actually puppet template .erb files.
>
> The other day, I made a mistake which resulted in some zone
> files with syntax errors in them. Puppet faithfully distributed
> the erroneous zone files to the name servers, which killed them.
>
> To prevent this from happening again, I want to be able to
> somehow expand the template files and run named-checkzone
> as part of a git pre-commit hook. I want this expansion to
> occur without making a normal puppet run so that the bad
> files won''t get distributed.
>
> Can anybody point me to a technique for doing this?
>
> Cordially,
> Jon Forrest
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Sep 14, 2011, at 8:24 PM, Jon Forrest wrote:
> On 9/14/2011 1:58 PM, Dominik Zyla wrote:
> 
>> named-checkzone(8) should be suitable tool for your needs.
> 
> I already knew that. To repeat, what I''m trying to find out
> is a way to get an expanded template without doing a full
> run of puppet. I could then run named-checkzone on that
> and then do a regular run.
> 
> In fact, this is only a special case of the more
> general problem of getting access to any file generated
> by a puppet run without actually putting the files
> in the ultimate destination. In other words, it would
> be like a --test run except files will be generated and
> kept.
----
I use /etc/puppet/deployment_files for situations such as this. 
Puppet pseudo code...

file {/etc/puppet/deployment_files/zone_name
  source => puppet:///modules/bind/zone_name.erb
  notify => Exec["zone_name.un-deploy"]

exec {"zone_name.un-deploy"
  /bin/rm "/etc/puppet/deployment_files/zone_name.deployed"

exec {"zone_name.deploy"
  touch "/etc/puppet/deployment_files/zone_name.deployed"
  notify => Exec["test - deploy if test successful"]

exec {"test - deploy if test successful"
     command => cat /etc/puppet/deployment_files/zone_name
     unless  => bind-checkzone /etc/puppet/deployment_files/zone_name > 0
AND ls ! /etc/puppet/deployment_files/zone_name.deployed
     require => File["/etc/puppet/deployment_files/zone_name"]
     notify  => [ Class["bind::service"],
Exec["zone_name.deploy"] ]

That''s not all - you might want some notice statements but
that''s the general idea anyway.

Craig

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I have a similar requirement as Jon Forrest who opened this topic. I notice 
some time has passed since then, so I thought I''d just tap you guys up
to
see if anyone had any further solutions.

I''d like to use Jenkins to Unit Test come configs we use whenever
someone
commits them. The idea is that we''d take a particular config,
"compile" the
templates and copy them to test box and fire up the relevant apps. We can 
then shoot our Unit Tests at applications on the test box and report on how 
many tests passed or failed.

In order to do this, I need to find a way to "compile" the .erb
templates
into their resulting config files. I''ll then push them onto my test
server
and start up the apps. This would check the syntax of the configs, but 
mostly means I have an box running that is pretty much the same as what
I''m
going to deploy in the future. In Jon''s case, this would mean
he''d have a
name server that not only has a valid syntax, but also could respond to 
queries (so could be unit tested). In my case, I can just fire some HTTP 
requests at the box and make sure it''s what I thought it was going to
be.

Does anyone know if it''s possible to do something like
"compile" the
templates? In an ideal world, I''d just run something like 
"puppet_generate_file /etc/httpd/conf.d/vhosts.conf" and get the
resulting
file on STDOUT (possibly on the machine that would normally receive the 
file in question). Even better would be if I could do this on an entirely 
different machine and still get the same result (maybe something like 
"puppet_generate_file 
otherserver.example.com:/etc/httpd/conf.d/vhosts.conf").

Any ideas?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Sunday, 11 September 2011 02:36:39 UTC+2, Jon Forrest
wrote:
>
> [...] I want to be able to
>
> somehow expand the template files and run [some check]
> as part of a git pre-commit hook. I want this expansion to
> occur without making a normal puppet run so that the bad
> files won''t get distributed.
>
> Can anybody point me to a technique for doing this?
>
> I have seen lots of interesting techniques in the reply-posts but not that 
IMO satisfactorily address your original requirement.  I faced a similar 
problem trying to do a syntax check on perl script thrown into .erb format 
to fill in some hard-coded directories and stuff.  The solution I quickly 
hacked together might be helpful...

The puppet server config for our datacentres are stored in a git 
repository, and I have local clone on my workstation for development 
($HOME/gitrepos/puppet-isa/). Starting from a copy (in a temp dir) of the 
manifest that generates the script in question,  I remove the surrounding 
class myclass { ... }, and the include/require of other manifests. In fact 
keep little more than the file resource that generates the script (with 
owner => ''root'', etc removed!), and assign hard-coded
string values to the
variables used in the erb. One of those variables is the dir where the file 
will be created and I point that to my same tmpdir! Then I just run puppet 
apply to generate my script.  Something like this:

$ mkdir $HOME/tmp/testpuppet
$ cd $HOME/tmp/testpuppet
$ cp $HOME/gitrepos/puppet-isa/modules/mymodule/manifests/handlers.pp 
./genhandlers.pp
$ vi genhandlers.pp # edit the manifest as described above
$ cat genhandlers.pp
# genhandlers.pp - generate the handler script

$handlerdir = ''/home/me/tmp/testspuppet''

# next two vars are for use in erb, normaly the values
# are taken from included config class...
# $mylibdir = $mymodule::config::mylibdir
$mylibdir   = ''/usr/share/mylibs''
$myconffile = ''/etc/myconf.conf''

file { "${handlerdir}/myscript.pl" :
  ensure  => file,
  content => template(''mymodule/myscript.pl.erb''),
  mode    => ''0755'',
}
# EOF - genhandlers.pp
$ puppet apply --modulepath=$HOME/gitrepos/puppet-isa/modules./genhandlers.pp
$ ls -1tr
genhandlers.pp
myscript.pl

--- end example
I have no experience with commit hooks but I guess it shouldn''t be hard
to
wrap the above into a script.  One could use a directory from mktemp and 
generate a minimal manifest from a here-document or by pulling it out of 
the original manifest with awk/sed...

Hope that helps,
Brandon

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.