Hi All, For most of the config files we manage via our Puppet setup, we either serve flat files or use templates and dashboard parameters and Facts to dynamically create the config file. However, in some cases we are not comfortable with either of these methods due to the possibility of small but numerous variances across different versions of operating systems. Also there is always the worry that a package update from the OS vendor will make changes you did not forsee. Managing the file with a File resource or template in Puppet would stomp out these vendor changes. Currently in such cases, we use diff-style patches and use the ''patch'' command in our RPMs to apply custom changes to Red Hat''s baseline file. This is flexible and fairly safe. However we''ve been transitioning from using custom RPMs in a Kickstarted environment, to bootstrapping our systems using Kickstart and then using Puppet wherever possible. I''d like to manage even these tricky files with Puppet, but I''m not sure the best way to handle it. I don''t want to use a File resource or templates, as we do for most everything else. I considered augeas and have played around with it a bit, but in so many cases it is clumsy and a little dangerous, i.e. items in config files are often referenced by file line number or some other possibly-volatile key. This uncertainty with augeas sorta defeats the whole purpose of avoiding File and templates. Line numbers can change from one version of the file to the next, and if I hardcode changes to specific line numbers, that seems like a recipe for incorrect config files in the future. Anyone have a good idea for how to apply Diff-style patches using Puppet? I was thinking a File and an Exec could get it done, the former being the patch file, the latter being a command to patch the file. But in this case, how to keep from download the patch file on every Puppet run, whether or not we use it? Hope I''ve been clear on my questioning, and thanks in advance! -Kent -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
You can use a file resource to copy a diff to a temp/diff area that
notifies an exec that does the diff as you have mentioned. This means
the diff would only get downloaded once, and the patch only applied if
the diff changes.
So roughly:
file {"/var/lib/diffpatches/foo.patch":
source => "puppet:///modules/foo/foo.patch",
notify => Exec["apply-foo-patch"],
}
exec {"apply-foo-patch":
command => "patch -d /path/to/patch -p1 < /var/lib/diffpatches/
foo.patch",
refreshonly => true,
}
You could wrap this sucker in a define and make it a re-usable
function if you wanted to as well so you can re-use it. You may also
want to add an ''onlyif'' section to the exec with the patch
command in
a dry-run perhaps?
Other ideas would include concatenating multiple files:
http://forge.puppetlabs.com/ripienaar/concat
https://github.com/puppet-modules/puppet-concat
And there are various regexp based patterns for deleting lines and
search and replace:
http://projects.puppetlabs.com/projects/1/wiki/Simple_Text_Patterns
Normally I don''t diverge much from concat or managing the file in a
template myself to be honest. The vendor may add new configuration
items, but I generally want to be aware of this anyway if I''m
upgrading a package :-).
ken.
On Feb 22, 7:55 pm, Kent <kentmshu...@gmail.com>
wrote:> Hi All,
>
> For most of the config files we manage via our Puppet setup, we either
> serve flat files or use templates and dashboard parameters and Facts
> to dynamically create the config file. However, in some cases we are
> not comfortable with either of these methods due to the possibility of
> small but numerous variances across different versions of operating
> systems. Also there is always the worry that a package update from the
> OS vendor will make changes you did not forsee. Managing the file
> with a File resource or template in Puppet would stomp out these
> vendor changes.
>
> Currently in such cases, we use diff-style patches and use the
''patch''
> command in our RPMs to apply custom changes to Red Hat''s baseline
> file. This is flexible and fairly safe. However we''ve been
> transitioning from using custom RPMs in a Kickstarted environment, to
> bootstrapping our systems using Kickstart and then using Puppet
> wherever possible. I''d like to manage even these tricky files with
> Puppet, but I''m not sure the best way to handle it.
>
> I don''t want to use a File resource or templates, as we do for
most
> everything else. I considered augeas and have played around with it a
> bit, but in so many cases it is clumsy and a little dangerous, i.e.
> items in config files are often referenced by file line number or some
> other possibly-volatile key. This uncertainty with augeas sorta
> defeats the whole purpose of avoiding File and templates. Line numbers
> can change from one version of the file to the next, and if I hardcode
> changes to specific line numbers, that seems like a recipe for
> incorrect config files in the future.
>
> Anyone have a good idea for how to apply Diff-style patches using
> Puppet? I was thinking a File and an Exec could get it done, the
> former being the patch file, the latter being a command to patch the
> file. But in this case, how to keep from download the patch file on
> every Puppet run, whether or not we use it?
>
> Hope I''ve been clear on my questioning, and thanks in advance!
>
> -Kent
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Take a look at augeas: <http://docs.puppetlabs.com/references/latest/type.html#augeas> It can match based on various information and append/change/delete lines from the configuration file. On Tue, Feb 22, 2011 at 11:05 PM, Ken Barber <ken@bob.sh> wrote:> You can use a file resource to copy a diff to a temp/diff area that > notifies an exec that does the diff as you have mentioned. This means > the diff would only get downloaded once, and the patch only applied if > the diff changes. > > So roughly: > > file {"/var/lib/diffpatches/foo.patch": > source => "puppet:///modules/foo/foo.patch", > notify => Exec["apply-foo-patch"], > } > exec {"apply-foo-patch": > command => "patch -d /path/to/patch -p1 < /var/lib/diffpatches/ > foo.patch", > refreshonly => true, > } > > You could wrap this sucker in a define and make it a re-usable > function if you wanted to as well so you can re-use it. You may also > want to add an ''onlyif'' section to the exec with the patch command in > a dry-run perhaps? > > Other ideas would include concatenating multiple files: > > http://forge.puppetlabs.com/ripienaar/concat > https://github.com/puppet-modules/puppet-concat > > And there are various regexp based patterns for deleting lines and > search and replace: > > http://projects.puppetlabs.com/projects/1/wiki/Simple_Text_Patterns > > Normally I don''t diverge much from concat or managing the file in a > template myself to be honest. The vendor may add new configuration > items, but I generally want to be aware of this anyway if I''m > upgrading a package :-). > > ken. > > On Feb 22, 7:55 pm, Kent <kentmshu...@gmail.com> wrote: >> Hi All, >> >> For most of the config files we manage via our Puppet setup, we either >> serve flat files or use templates and dashboard parameters and Facts >> to dynamically create the config file. However, in some cases we are >> not comfortable with either of these methods due to the possibility of >> small but numerous variances across different versions of operating >> systems. Also there is always the worry that a package update from the >> OS vendor will make changes you did not forsee. Managing the file >> with a File resource or template in Puppet would stomp out these >> vendor changes. >> >> Currently in such cases, we use diff-style patches and use the ''patch'' >> command in our RPMs to apply custom changes to Red Hat''s baseline >> file. This is flexible and fairly safe. However we''ve been >> transitioning from using custom RPMs in a Kickstarted environment, to >> bootstrapping our systems using Kickstart and then using Puppet >> wherever possible. I''d like to manage even these tricky files with >> Puppet, but I''m not sure the best way to handle it. >> >> I don''t want to use a File resource or templates, as we do for most >> everything else. I considered augeas and have played around with it a >> bit, but in so many cases it is clumsy and a little dangerous, i.e. >> items in config files are often referenced by file line number or some >> other possibly-volatile key. This uncertainty with augeas sorta >> defeats the whole purpose of avoiding File and templates. Line numbers >> can change from one version of the file to the next, and if I hardcode >> changes to specific line numbers, that seems like a recipe for >> incorrect config files in the future. >> >> Anyone have a good idea for how to apply Diff-style patches using >> Puppet? I was thinking a File and an Exec could get it done, the >> former being the patch file, the latter being a command to patch the >> file. But in this case, how to keep from download the patch file on >> every Puppet run, whether or not we use it? >> >> Hope I''ve been clear on my questioning, and thanks in advance! >> >> -Kent > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Feb 22, 7:55 pm, Kent <kentmshu...@gmail.com> wrote:> I don''t want to use a File resource or templates, as we do for most > everything else. I considered augeas and have played around with it a > bit, but in so many cases it is clumsy and a little dangerous, i.e. > items in config files are often referenced by file line number or some > other possibly-volatile key. This uncertainty with augeas sorta > defeats the whole purpose of avoiding File and templates. Line numbers > can change from one version of the file to the next, and if I hardcode > changes to specific line numbers, that seems like a recipe for > incorrect config files in the future.Agreed, augeas is a bit convoluted at times, definitely a tool for a specific job. As you''ve seen, it all depends on the lense for the file in question. File lenses with fully unique keys are great, lenses for files that allow multiple, duplicate keys, or are line order dependent (think /etc/hosts and /etc/pam.d/) then things become more annoying, but I haven''t run across many cases that I''d consider terribly volatile to consider not using it, except maybe /etc/logrotate.conf, can you give me any examples? While we''re talking Augeas, I''ve attached some examples people might find helpful. One of my augeas edits to add one of the lines for Kerberos to /etc/ pam.d/system-auth-ac, an order dependent file. This one inserts the line after "auth required pam_env.so", if that line''s missing then the augeas resource will fail, in which case the format of this PAM file has seriously changed and I''d want to manually intervene anyway: #Edit PAM system-auth-ac and add auth line for kerberos. #Eg: #auth sufficient pam_krb5.so try_first_pass augeas { "add kerberos auth to ${system_auth_file}": context => "/files${system_auth_file}", changes => [ "rm *[type=''auth''][module=''pam_krb5.so'']", "ins 1000000 after *[type=''auth''][control=''required''] [module=''pam_env.so'']", "set 1000000/type auth", "set 1000000/control sufficient", "set 1000000/module pam_krb5.so", "set 1000000/argument try_first_pass", ], onlyif => "match *[type=''auth''][control=''sufficient''] [module=''pam_krb5.so''][argument=''try_first_pass''] size == 0", } #====Define: insert_comment # #Insert a comment at the top of a file using Augeas. # #This is a wrapper around an Augeas function that inserts a comment at the top #of any file with a given comment if that comment doesn''t exist. You can override #the Augeas ''comment'' node name but it defaults to ''\#comment''. The ''description'' #parameter is only used in the name of the Augeas resource, so it''s for information #purposes only. # #====Parameters # #comment:: The contents of the comment to be added. #file:: The file to comment, must be supported by Augeas. #description:: Description of the Augeas resource which translates to it''s namevar. #aug_comment:: The comment node name for this file. Defaults to ''#comment'', you need to change this if your augeas lense describes it''s comment nodes differently, like '';comment'' or ''//comment''. #load_path:: Load any additional Augeas lense paths. define insert_comment($comment, $file, $description, $aug_comment="#comment", $load_path=undef) { augeas { "comment ${file} for ${description}": context => "/files${file}", changes => [ "ins ${aug_comment} before *[1]", "set ${aug_comment}[1] ''${comment}''", ], onlyif => "match ${aug_comment}[.=''${comment}''] size == 0", load_path => $load_path, } } #====Define: exports_entry # #This define is a shortcut for other classes to use Augeas to make inline edits on #/etc/exports. To add multiple clients with the same path then just create multiple #exports_entry resources. This define also notifies the Exec[reload_exportfs_file] #resource to have the NFS server reload the exports file. # #This define only uses Augeas ''set'' commands, so it can only ADD to / etc/exports, #it will not clean up old entries or change NFS server options from clients. # #====Parameters #path:: The file system path to export. #client:: The hostname or IP address of the NFS client. #ro:: Boolean to turn on the ''ro'' NFS mount option. Defaults to false. #async:: Boolean to turn on the ''async'' NFS mount option. Defaults to false. #no_root_squash:: Boolean to turn on the ''no_root_squash'' NFS mount option. Defaults to false. define exports_entry($path, $client, $ro = false, $async = false, $no_root_squash = false) { #The basic set of Augeas changes to set up an NFS export. $base_changes = [ "set dir[.=''$path''] $path", "set dir[.=''$path'']/client[.=''${client}''] ${client}" ] #additional augeas changes are defined as separate variables. Unfortunately #we can''t append to the array above (Puppet "design feature"). if ($ro) { $ro_change = "set dir[.=''$path'']/client[.=''${client}'']/option[1] ro" } if ($no_root_squash) { $no_root_squash_change = "set dir[.=''$path'']/client[.=''${client}'']/ option[2] no_root_squash" } if ($async) { $async_change = "set dir[.=''$path'']/client[.=''${client}'']/ option[3] async" } #Use Augeas to add an NFS export for the given client to the given path. The #''changes'' array also lists all optional variables that get defined when the #optional parameters to this define are made True. augeas { "/etc/exports add $client to $path": context => "/files/etc/exports", changes => [ $base_changes, $ro_change, $no_root_squash_change, $async_change, ], notify => Exec["reload_exportfs_file"], } } -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Feb 22, 1:55 pm, Kent <kentmshu...@gmail.com> wrote:> Currently in such cases, we use diff-style patches and use the ''patch'' > command in our RPMs to apply custom changes to Red Hat''s baseline > file. This is flexible and fairly safe. However we''ve been > transitioning from using custom RPMs in a Kickstarted environment, to > bootstrapping our systems using Kickstart and then using Puppet > wherever possible. I''d like to manage even these tricky files with > Puppet, but I''m not sure the best way to handle it.It is not clear to me whether it''s RPMs you want to move away from, or just kickstart. If the latter, then the path of least resistance may be to apply your existing (and new) custom RPMs under Puppet''s direction. Certainly that should be quick and easy to do at least as a transitional step. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Thanks for the helpful replies, everyone. I think I''m going to keep it easy as I can for now and go with the File/Exec wrapped in a define. Good idea to have the exec refreshonly and subscribe to the patch file...> It is not clear to me whether it''s RPMs you want to move away from, or > just kickstart. If the latter, then the path of least resistance may > be to apply your existing (and new) custom RPMs under Puppet''s > direction. Certainly that should be quick and easy to do at least as > a transitional step.We are going to keep Kickstarting, but want to move away from custom RPMs as much as possible. That way when RHEL releases updated RPMs, we don''t have to roll our changes back in. -Kent -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.