Jon Forrest
2011-Sep-11 00:36 UTC
[Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
We use puppet to distribute named zone files, like many of you do. We use git to maintain these files, which are then pulled by the puppet master machine. These zone files are actually puppet template .erb files. The other day, I made a mistake which resulted in some zone files with syntax errors in them. Puppet faithfully distributed the erroneous zone files to the name servers, which killed them. To prevent this from happening again, I want to be able to somehow expand the template files and run named-checkzone as part of a git pre-commit hook. I want this expansion to occur without making a normal puppet run so that the bad files won''t get distributed. Can anybody point me to a technique for doing this? Cordially, Jon Forrest -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nigel Kersten
2011-Sep-11 01:26 UTC
Re: [Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
On Sat, Sep 10, 2011 at 5:36 PM, Jon Forrest <nobozo@gmail.com> wrote:> We use puppet to distribute named zone files, like many > of you do. We use git to maintain these files, which are > then pulled by the puppet master machine. These zone files are > actually puppet template .erb files. > > The other day, I made a mistake which resulted in some zone > files with syntax errors in them. Puppet faithfully distributed > the erroneous zone files to the name servers, which killed them. > > To prevent this from happening again, I want to be able to > somehow expand the template files and run named-checkzone > as part of a git pre-commit hook. I want this expansion to > occur without making a normal puppet run so that the bad > files won''t get distributed. > > Can anybody point me to a technique for doing this? >git cat-file blob :0:$indexfile | erb -x -T - | ruby -c 2> $error_msg > /dev/null ; from: http://projects.puppetlabs.com/projects/1/wiki/Puppet_Version_Control should get you going.> > Cordially, > Jon Forrest > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@** > googlegroups.com <puppet-users%2Bunsubscribe@googlegroups.com>. > For more options, visit this group at http://groups.google.com/** > group/puppet-users?hl=en<http://groups.google.com/group/puppet-users?hl=en> > . > >-- Nigel Kersten Product Manager, Puppet Labs *Join us for **PuppetConf * <http://www.bit.ly/puppetconfsig> Sept 22/23 Portland, Oregon, USA. * * -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Jonathan Stanton
2011-Sep-11 01:57 UTC
Re: [Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
On Sep 10, 2011, at 9:26 PM, Nigel Kersten wrote:> On Sat, Sep 10, 2011 at 5:36 PM, Jon Forrest <nobozo@gmail.com> wrote: > >> We use puppet to distribute named zone files, like many >> of you do. We use git to maintain these files, which are >> then pulled by the puppet master machine. These zone files are >> actually puppet template .erb files. >> >> The other day, I made a mistake which resulted in some zone >> files with syntax errors in them. Puppet faithfully distributed >> the erroneous zone files to the name servers, which killed them. >> >> To prevent this from happening again, I want to be able to >> somehow expand the template files and run named-checkzone >> as part of a git pre-commit hook. I want this expansion to >> occur without making a normal puppet run so that the bad >> files won''t get distributed. >> >> Can anybody point me to a technique for doing this? >> > > > git cat-file blob :0:$indexfile | erb -x -T - | ruby -c 2> $error_msg > > /dev/null ; > > from: > > http://projects.puppetlabs.com/projects/1/wiki/Puppet_Version_Control > > should get you going. >Maybe I''m missing something here, but I think Jon was asking something a bit different -- he doesn''t want to check the validity of the erb template (i.e. ruby syntax check) but syntax check the named zone file generated by the template. So the tricky bit is how to get the variables out of the puppet manifests that the erb template needs to generate the output file that ''would'' be generated by a new puppet run for this node -- without the actual puppet run (as he asks at the end of the email). My first thought is that the only accurate way to do this is by doing a full puppet run, as any part of the node''s manifest could effect the variables used in the zone file template. You should be able to get away with a --noop run so the changes won''t actually be applied (because noop does generate files from templates, but you would need to have a way to capture the newly generated zone file on the client host and run the named-checkzone there. Cheers, Jonathan ------------------------------------------------------------------------------- Jonathan Stanton jonathan@spreadconcepts.com Spread Group Messaging www.spread.org Spread Concepts LLC www.spreadconcepts.com -------------------------------------------------------------------------------> > > >> >> Cordially, >> Jon Forrest >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To post to this group, send email to puppet-users@googlegroups.com. >> To unsubscribe from this group, send email to puppet-users+unsubscribe@** >> googlegroups.com <puppet-users%2Bunsubscribe@googlegroups.com>. >> For more options, visit this group at http://groups.google.com/** >> group/puppet-users?hl=en<http://groups.google.com/group/puppet-users?hl=en> >> . >> >> > > > -- > Nigel Kersten > Product Manager, Puppet Labs > > *Join us for **PuppetConf * > <http://www.bit.ly/puppetconfsig> > Sept 22/23 Portland, Oregon, USA. > * > * > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. >
Jon Forrest
2011-Sep-11 02:59 UTC
Re: [Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
On 9/10/2011 6:57 PM, Jonathan Stanton wrote:> > Maybe I''m missing something here, but I think Jon was asking > something a bit different -- he doesn''t want to check the validity of > the erb template (i.e. ruby syntax check) but syntax check the named > zone file generated by the template.Precisely. Maybe later I''ll face the issue of ruby syntax problems but right now I need to detect named syntax errors before they cause problems.> So the tricky bit is how to get the variables out of the puppet > manifests that the erb template needs to generate the output file > that ''would'' be generated by a new puppet run for this node -- > without the actual puppet run (as he asks at the end of the email).Precisely again.> My first thought is that the only accurate way to do this is by doing > a full puppet run, as any part of the node''s manifest could effect > the variables used in the zone file template. You should be able to > get away with a --noop run so the changes won''t actually be applied > (because noop does generate files from templates, but you would need > to have a way to capture the newly generated zone file on the client > host and run the named-checkzone there.That''s what I figured. I was hoping that there would be an easier way that could somehow do a facter run but only run the minimal amount of puppet. Jon -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Aaron Grewell
2011-Sep-11 04:00 UTC
Re: [Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
Perhaps just add an exec step or two to your regular run? Put the file in a temp location then move it to the live one if it checks out? On Sep 10, 2011 8:01 PM, "Jon Forrest" <nobozo@gmail.com> wrote:> On 9/10/2011 6:57 PM, Jonathan Stanton wrote: >> >> Maybe I''m missing something here, but I think Jon was asking >> something a bit different -- he doesn''t want to check the validity of >> the erb template (i.e. ruby syntax check) but syntax check the named >> zone file generated by the template. > > Precisely. Maybe later I''ll face the issue of ruby syntax > problems but right now I need to detect named syntax errors > before they cause problems. > >> So the tricky bit is how to get the variables out of the puppet >> manifests that the erb template needs to generate the output file >> that ''would'' be generated by a new puppet run for this node -- >> without the actual puppet run (as he asks at the end of the email). > > Precisely again. > >> My first thought is that the only accurate way to do this is by doing >> a full puppet run, as any part of the node''s manifest could effect >> the variables used in the zone file template. You should be able to >> get away with a --noop run so the changes won''t actually be applied >> (because noop does generate files from templates, but you would need >> to have a way to capture the newly generated zone file on the client >> host and run the named-checkzone there. > > That''s what I figured. I was hoping that there would be an easier way > that could somehow do a facter run but only run the minimal amount > of puppet. > > Jon > > > -- > You received this message because you are subscribed to the Google Groups"Puppet Users" group.> To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email topuppet-users+unsubscribe@googlegroups.com.> For more options, visit this group athttp://groups.google.com/group/puppet-users?hl=en.>-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Roger
2011-Sep-11 13:56 UTC
[Puppet Users] Re: How To Expand an .erb Template Without Doing a Full Puppet Run?
I just change the restart command on the named service. Instead of letting puppet do a: /sbin/service named stop and then start I have it do /sbin/service named reload. Which on redhat only runs /usr/sbin/rndc reload If that fails, named keeps running with the previous config it loaded with no interruption, puppet complains that the restart failed, I get notified and fix the zone file. On Sep 11, 12:00 am, Aaron Grewell <aaron.grew...@gmail.com> wrote:> Perhaps just add an exec step or two to youir regular run? Put the file in a > temp location then move it to the live one if it checks out? > On Sep 10, 2011 8:01 PM, "Jon Forrest" <nob...@gmail.com> wrote: > > > > > > > > > On 9/10/2011 6:57 PM, Jonathan Stanton wrote: > > >> Maybe I''m missing something here, but I think Jon was asking > >> something a bit different -- he doesn''t want to check the validity of > >> the erb template (i.e. ruby syntax check) but syntax check the named > >> zone file generated by the template. > > > Precisely. Maybe later I''ll face the issue of ruby syntax > > problems but right now I need to detect named syntax errors > > before they cause problems. > > >> So the tricky bit is how to get the variables out of the puppet > >> manifests that the erb template needs to generate the output file > >> that ''would'' be generated by a new puppet run for this node -- > >> without the actual puppet run (as he asks at the end of the email). > > > Precisely again. > > >> My first thought is that the only accurate way to do this is by doing > >> a full puppet run, as any part of the node''s manifest could effect > >> the variables used in the zone file template. You should be able to > >> get away with a --noop run so the changes won''t actually be applied > >> (because noop does generate files from templates, but you would need > >> to have a way to capture the newly generated zone file on the client > >> host and run the named-checkzone there. > > > That''s what I figured. I was hoping that there would be an easier way > > that could somehow do a facter run but only run the minimal amount > > of puppet. > > > Jon > > > -- > > You received this message because you are subscribed to the Google Groups > > "Puppet Users" group.> To post to this group, send email to puppet-users@googlegroups.com. > > To unsubscribe from this group, send email to > > puppet-users+unsubscribe@googlegroups.com.> For more options, visit this group at > > http://groups.google.com/group/puppet-users?hl=en. > > > > > > > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Dan White
2011-Sep-11 14:35 UTC
Re: [Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
@Jon - If you find an answer to your question, please post it to the list. I''d like to see the answer and there are probably lots of other folks who would benefit from the info being in the mailing list archives. Good luck. On Sep 11, 2011, at 12:00 AM, Aaron Grewell wrote:> Perhaps just add an exec step or two to your regular run? Put the > file in a temp location then move it to the live one if it checks out? > > On Sep 10, 2011 8:01 PM, "Jon Forrest" <nobozo@gmail.com> wrote: > > On 9/10/2011 6:57 PM, Jonathan Stanton wrote: > >> > >> Maybe I''m missing something here, but I think Jon was asking > >> something a bit different -- he doesn''t want to check the > validity of > >> the erb template (i.e. ruby syntax check) but syntax check the > named > >> zone file generated by the template. > > > > Precisely. Maybe later I''ll face the issue of ruby syntax > > problems but right now I need to detect named syntax errors > > before they cause problems. > > > >> So the tricky bit is how to get the variables out of the puppet > >> manifests that the erb template needs to generate the output file > >> that ''would'' be generated by a new puppet run for this node -- > >> without the actual puppet run (as he asks at the end of the email). > > > > Precisely again. > > > >> My first thought is that the only accurate way to do this is by > doing > >> a full puppet run, as any part of the node''s manifest could effect > >> the variables used in the zone file template. You should be able to > >> get away with a --noop run so the changes won''t actually be applied > >> (because noop does generate files from templates, but you would > need > >> to have a way to capture the newly generated zone file on the > client > >> host and run the named-checkzone there. > > > > That''s what I figured. I was hoping that there would be an easier > way > > that could somehow do a facter run but only run the minimal amount > > of puppet. > > > > Jon > > > > > > -- > > You received this message because you are subscribed to the Google > Groups "Puppet Users" group. > > To post to this group, send email to puppet-users@googlegroups.com. > > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com > . > > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en > . > > > > -- > You received this message because you are subscribed to the Google > Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com > . > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en > .-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Brian Gupta
2011-Sep-11 15:50 UTC
Re: [Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
I''m thinking the answer is to update your init script, to make sure that restart calls rndc reload, and specify "hasrestart" in your resource definition. I suggest this because bind should ignore bad zone files, and keep serving the old zones, when reloaded via rndc. Maybe the answer is to also incorporate named-checkzone and some sort of email notification into the restart section of your init script. As always test, and you milage may vary. Cheers, Brian On Sun, Sep 11, 2011 at 10:35 AM, Dan White <ygor@comcast.net> wrote:> @Jon - If you find an answer to your question, please post it to the list. > I''d like to see the answer and there are probably lots of other folks who > would benefit from the info being in the mailing list archives. > > Good luck. > > On Sep 11, 2011, at 12:00 AM, Aaron Grewell wrote: > > Perhaps just add an exec step or two to your regular run? Put the file in a > temp location then move it to the live one if it checks out? > On Sep 10, 2011 8:01 PM, "Jon Forrest" <nobozo@gmail.com> wrote: > > On 9/10/2011 6:57 PM, Jonathan Stanton wrote: > >> > >> Maybe I''m missing something here, but I think Jon was asking > >> something a bit different -- he doesn''t want to check the validity of > >> the erb template (i.e. ruby syntax check) but syntax check the named > >> zone file generated by the template. > > > > Precisely. Maybe later I''ll face the issue of ruby syntax > > problems but right now I need to detect named syntax errors > > before they cause problems. > > > >> So the tricky bit is how to get the variables out of the puppet > >> manifests that the erb template needs to generate the output file > >> that ''would'' be generated by a new puppet run for this node -- > >> without the actual puppet run (as he asks at the end of the email). > > > > Precisely again. > > > >> My first thought is that the only accurate way to do this is by doing > >> a full puppet run, as any part of the node''s manifest could effect > >> the variables used in the zone file template. You should be able to > >> get away with a --noop run so the changes won''t actually be applied > >> (because noop does generate files from templates, but you would need > >> to have a way to capture the newly generated zone file on the client > >> host and run the named-checkzone there. > > > > That''s what I figured. I was hoping that there would be an easier way > > that could somehow do a facter run but only run the minimal amount > > of puppet. > > > > Jon > > > > > > -- > > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > > To post to this group, send email to puppet-users@googlegroups.com. > > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. >-- <http://aws.amazon.com/solutions/solution-providers/brandorr/> -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nigel Kersten
2011-Sep-12 20:20 UTC
Re: [Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
On Sat, Sep 10, 2011 at 7:59 PM, Jon Forrest <nobozo@gmail.com> wrote:> On 9/10/2011 6:57 PM, Jonathan Stanton wrote: > >> >> Maybe I''m missing something here, but I think Jon was asking >> something a bit different -- he doesn''t want to check the validity of >> the erb template (i.e. ruby syntax check) but syntax check the named >> zone file generated by the template. >> > > Precisely. Maybe later I''ll face the issue of ruby syntax > problems but right now I need to detect named syntax errors > before they cause problems.Ah, yes, I did misread your question. You could adopt the above method to produce the actual templatized output and then use named to verify it, but as others have said, distros are much better at letting you verify configs before restarting nowadays, and I''d probably push this there. In the past I tend to solve this with a canary node that implements the new configuration, and then out of band testing verifies that DNS is still working happily for critical infrastructure entries before before the configuration gets pushed to "live" name servers. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Dominik Zyla
2011-Sep-14 20:58 UTC
Re: [Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
Hello, On 09/11/2011 04:59 AM, Jon Forrest wrote:> On 9/10/2011 6:57 PM, Jonathan Stanton wrote: >> >> Maybe I''m missing something here, but I think Jon was asking >> something a bit different -- he doesn''t want to check the validity of >> the erb template (i.e. ruby syntax check) but syntax check the named >> zone file generated by the template. > > Precisely. Maybe later I''ll face the issue of ruby syntax > problems but right now I need to detect named syntax errors > before they cause problems. > >> So the tricky bit is how to get the variables out of the puppet >> manifests that the erb template needs to generate the output file >> that ''would'' be generated by a new puppet run for this node -- >> without the actual puppet run (as he asks at the end of the email). > > Precisely again. > >> My first thought is that the only accurate way to do this is by doing >> a full puppet run, as any part of the node''s manifest could effect >> the variables used in the zone file template. You should be able to >> get away with a --noop run so the changes won''t actually be applied >> (because noop does generate files from templates, but you would need >> to have a way to capture the newly generated zone file on the client >> host and run the named-checkzone there. > > That''s what I figured. I was hoping that there would be an easier way > that could somehow do a facter run but only run the minimal amount > of puppet.named-checkzone(8) should be suitable tool for your needs. Best, -- Dominik Zyla -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Jon Forrest
2011-Sep-15 03:24 UTC
Re: [Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
On 9/14/2011 1:58 PM, Dominik Zyla wrote:> named-checkzone(8) should be suitable tool for your needs.I already knew that. To repeat, what I''m trying to find out is a way to get an expanded template without doing a full run of puppet. I could then run named-checkzone on that and then do a regular run. In fact, this is only a special case of the more general problem of getting access to any file generated by a puppet run without actually putting the files in the ultimate destination. In other words, it would be like a --test run except files will be generated and kept. Jon Forrest -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Russell Howe
2011-Sep-15 09:50 UTC
Re: [Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
On 15/09/11 04:24, Jon Forrest wrote:> In fact, this is only a special case of the more > general problem of getting access to any file generated > by a puppet run without actually putting the files > in the ultimate destination. In other words, it would > be like a --test run except files will be generated and > kept.I have two ideas. The first is to introduce a preflight check run stage which has a clone of whatever you use to generate the zonefile, but puts it somewhere else and then does an exec with named-checkzone etc. If run stage "main" depends on "preflight" that should do the right thing assuming that failures of earlier run stages prevent execution of later ones. I''m not sure if that could lead to a race condition should someone update the data between the preflight check and the main manifest run though. Also you may find that your preflight stage requires things which are in your main stage (installation of packages etc) which will just get messy. The other option which I think I prefer is to have puppet deploy your zone files to a staging area, exec named-checkzone and whatever else you want to do and then have an exec or similar which moves the staged zonefiles into place on your nameserver. Make that exec depend on your sanity checks and it will only run if the zonefiles check out OK. You could even go so far as to stage your zonefiles into a git repo or similar on your nameserver and end up with something like: vcsrepo { "/staging/location": # Set up git repo } file { "/staging/location/zone.a": .... # whatever gets zone.a there } exec { "check /staging/location/zone.a": command => "named-checkzone /staging/location/zone.a", require => [File["/staging/location/zone.a"],Vcsrepo["/staging/location"]], } exec { "commit staged zonefiles", command => "git commit ...", require => Exec["check /staging/location/zone.a"], # ... } exec { "deploy zonefiles to nameserver", command => "tar cp . -C /staging/location | tar xp -C /var/lib/named/zones", require => Exec["commit staged zonefiles"], notify => Service["named"], } -- Russell Howe rhowe@moonfruit.com -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Jon Jaroker
2011-Sep-15 13:53 UTC
[Puppet Users] Re: How To Expand an .erb Template Without Doing a Full Puppet Run?
Hello Jon, Here is a process that I follow to avoid the type of misconfiguration problem you encountered. This process is used to *avoid* the mis- configured system problem. It is not a direct test that Puppet will generate a valid configuration file from a template. 1) A "local" environment is used to deploy the entire system (which is called a ''role'' for the system). While I may be developing a single class, I still confirm that the entire role can be deployed to my local environment. The local server is typically rebuilt 30-50 times during the course of development of a new manifest. This allows me to catch errors early, as many of the mistakes I make result in spurious errors on Puppet. The manifests are then committed to SVN and automatically deployed to a DEV environment, which is identical to the production environment. This allows live testing against my other servers. The manifests then are merged to the ''trunk'' branch and get deployed to a TESTING environment where selenium regression tests exercise the applications to make sure they still work as expected. The manifests get deployed to QA for manual testing and review and finally to PRODUCTION. 2) Include configuration checks before making a configuration file live. One example is with the sudoers file. I use a file-fragment approach to generate a sudoers testing file and only make that file live if the check passes. Your zone file can use a similar check. In situations where checks are not possible, I include a ''testing'' flag to prevent a configuration file from being made live. I manually check the testing file to make sure it is valid. The APF firewall script is an example where this is important. When I first deployed APF, puppet exploded with spurious errors because I forgot to open a port. I now confirm (using the below process) that the APF configuration is valid before allowing APF to be started on the remote servers. 3) Each of my servers has a Dokuwiki page that is generated by puppet during the deployment. I use a file fragments pattern for this as well, where each class and define puts a dokuwik-formatted fragment into a specific directory. The configuration templates are included into the dokuwiki file fragments using this approach. These fragments are assembled into a single wiki page that is SCP''d to my documentation server. I then review the server''s configuration directly on this wiki page to confirm that the destination server is configured in the manner I expect. In the case of APF, I then change the ''testing'' flag to false and re-run puppet to make the configuration live. This is a process, not an answer to your original question about Puppet generating the configuration file from a template without actually applying the changes. That would be a nice feature which I don''t believe currently exists. Your mis-configured zone file would have been spotted in steps 1 and 2; maybe step 3. Regards, Jon On Sep 10, 8:36 pm, Jon Forrest <nob...@gmail.com> wrote:> We use puppet to distribute named zone files, like many > of you do. We use git to maintain these files, which are > then pulled by the puppet master machine. These zone files are > actually puppet template .erb files. > > The other day, I made a mistake which resulted in some zone > files with syntax errors in them. Puppet faithfully distributed > the erroneous zone files to the name servers, which killed them. > > To prevent this from happening again, I want to be able to > somehow expand the template files and run named-checkzone > as part of a git pre-commit hook. I want this expansion to > occur without making a normal puppet run so that the bad > files won''t get distributed. > > Can anybody point me to a technique for doing this? > > Cordially, > Jon Forrest-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Craig White
2011-Sep-15 15:32 UTC
Re: [Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
On Sep 14, 2011, at 8:24 PM, Jon Forrest wrote:> On 9/14/2011 1:58 PM, Dominik Zyla wrote: > >> named-checkzone(8) should be suitable tool for your needs. > > I already knew that. To repeat, what I''m trying to find out > is a way to get an expanded template without doing a full > run of puppet. I could then run named-checkzone on that > and then do a regular run. > > In fact, this is only a special case of the more > general problem of getting access to any file generated > by a puppet run without actually putting the files > in the ultimate destination. In other words, it would > be like a --test run except files will be generated and > kept.---- I use /etc/puppet/deployment_files for situations such as this. Puppet pseudo code... file {/etc/puppet/deployment_files/zone_name source => puppet:///modules/bind/zone_name.erb notify => Exec["zone_name.un-deploy"] exec {"zone_name.un-deploy" /bin/rm "/etc/puppet/deployment_files/zone_name.deployed" exec {"zone_name.deploy" touch "/etc/puppet/deployment_files/zone_name.deployed" notify => Exec["test - deploy if test successful"] exec {"test - deploy if test successful" command => cat /etc/puppet/deployment_files/zone_name unless => bind-checkzone /etc/puppet/deployment_files/zone_name > 0 AND ls ! /etc/puppet/deployment_files/zone_name.deployed require => File["/etc/puppet/deployment_files/zone_name"] notify => [ Class["bind::service"], Exec["zone_name.deploy"] ] That''s not all - you might want some notice statements but that''s the general idea anyway. Craig -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Ralph Bolton
2013-Feb-11 17:00 UTC
Re: [Puppet Users] How To Expand an .erb Template Without Doing a Full Puppet Run?
I have a similar requirement as Jon Forrest who opened this topic. I notice some time has passed since then, so I thought I''d just tap you guys up to see if anyone had any further solutions. I''d like to use Jenkins to Unit Test come configs we use whenever someone commits them. The idea is that we''d take a particular config, "compile" the templates and copy them to test box and fire up the relevant apps. We can then shoot our Unit Tests at applications on the test box and report on how many tests passed or failed. In order to do this, I need to find a way to "compile" the .erb templates into their resulting config files. I''ll then push them onto my test server and start up the apps. This would check the syntax of the configs, but mostly means I have an box running that is pretty much the same as what I''m going to deploy in the future. In Jon''s case, this would mean he''d have a name server that not only has a valid syntax, but also could respond to queries (so could be unit tested). In my case, I can just fire some HTTP requests at the box and make sure it''s what I thought it was going to be. Does anyone know if it''s possible to do something like "compile" the templates? In an ideal world, I''d just run something like "puppet_generate_file /etc/httpd/conf.d/vhosts.conf" and get the resulting file on STDOUT (possibly on the machine that would normally receive the file in question). Even better would be if I could do this on an entirely different machine and still get the same result (maybe something like "puppet_generate_file otherserver.example.com:/etc/httpd/conf.d/vhosts.conf"). Any ideas? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Brandon
2013-Feb-23 22:22 UTC
[Puppet Users] Re: How To Expand an .erb Template Without Doing a Full Puppet Run?
On Sunday, 11 September 2011 02:36:39 UTC+2, Jon Forrest wrote:> > [...] I want to be able to > > somehow expand the template files and run [some check] > as part of a git pre-commit hook. I want this expansion to > occur without making a normal puppet run so that the bad > files won''t get distributed. > > Can anybody point me to a technique for doing this? > > I have seen lots of interesting techniques in the reply-posts but not thatIMO satisfactorily address your original requirement. I faced a similar problem trying to do a syntax check on perl script thrown into .erb format to fill in some hard-coded directories and stuff. The solution I quickly hacked together might be helpful... The puppet server config for our datacentres are stored in a git repository, and I have local clone on my workstation for development ($HOME/gitrepos/puppet-isa/). Starting from a copy (in a temp dir) of the manifest that generates the script in question, I remove the surrounding class myclass { ... }, and the include/require of other manifests. In fact keep little more than the file resource that generates the script (with owner => ''root'', etc removed!), and assign hard-coded string values to the variables used in the erb. One of those variables is the dir where the file will be created and I point that to my same tmpdir! Then I just run puppet apply to generate my script. Something like this: $ mkdir $HOME/tmp/testpuppet $ cd $HOME/tmp/testpuppet $ cp $HOME/gitrepos/puppet-isa/modules/mymodule/manifests/handlers.pp ./genhandlers.pp $ vi genhandlers.pp # edit the manifest as described above $ cat genhandlers.pp # genhandlers.pp - generate the handler script $handlerdir = ''/home/me/tmp/testspuppet'' # next two vars are for use in erb, normaly the values # are taken from included config class... # $mylibdir = $mymodule::config::mylibdir $mylibdir = ''/usr/share/mylibs'' $myconffile = ''/etc/myconf.conf'' file { "${handlerdir}/myscript.pl" : ensure => file, content => template(''mymodule/myscript.pl.erb''), mode => ''0755'', } # EOF - genhandlers.pp $ puppet apply --modulepath=$HOME/gitrepos/puppet-isa/modules./genhandlers.pp $ ls -1tr genhandlers.pp myscript.pl --- end example I have no experience with commit hooks but I guess it shouldn''t be hard to wrap the above into a script. One could use a directory from mktemp and generate a minimal manifest from a here-document or by pulling it out of the original manifest with awk/sed... Hope that helps, Brandon -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.