Puppet users - Nov 2011 - Using puppet+augeas to modify multiple files

Hi all,

I have a task to convince puppet and augeas to modify multiple postfix
files, most specifically /etc/postfix/main.cf and /etc/postifx/
acceptdomains.cf (and other LDAP related files), and I''m struggling to
find out how to do that. I have found examples on how to use augeas to
modify just one file at a time, but no information seems to be
available when multiple files are involved.

I started with the config published here, and have no idea what to
change to convince puppet+augeas to edit both /etc/postfix/main.cf
and /etc/postfix/acceptdomains.cf at the same time:

https://github.com/camptocamp/puppet-postfix/blob/master/manifests/config.pp

Does anyone have a sample configuration on how you would modify
multiple files at once, or have a clue on where to start?

Regards,
Graham
--

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 27 Nov 2011, at 20:04, Graham Leggett wrote:
> I have a task to convince puppet and augeas to modify multiple postfix
> files, most specifically /etc/postfix/main.cf and /etc/postifx/
> acceptdomains.cf (and other LDAP related files), and I''m
struggling to
> find out how to do that. I have found examples on how to use augeas to
> modify just one file at a time, but no information seems to be
> available when multiple files are involved.
> 
> I started with the config published here, and have no idea what to
> change to convince puppet+augeas to edit both /etc/postfix/main.cf
> and /etc/postfix/acceptdomains.cf at the same time:
> 
>
https://github.com/camptocamp/puppet-postfix/blob/master/manifests/config.pp
> 
> Does anyone have a sample configuration on how you would modify
> multiple files at once, or have a clue on where to start?
Answering my own question, I''ve discovered that augeas doesn''t
support multiple files at all without a custom lens being created to support it.

Or to declare it more accurately, augeas only knows about specific files defined
in specific locations, it has no concept of a general file in a general location
that you might want to apply a lens to.

I eventually worked around the issue by making a custom lens copied from the
augeas postfix_main.aug lens that listed the extra ldap templates as matching a
wildcard, at this point augeas started working correctly.

Something else I discovered is that augeas has no error handling of any kind -
if it doesn''t recognise a file as matching one of the predetermined
locations built into the lens, it will silently ignore the file, and not tell
you why.

Regards,
Graham
--

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I''m not aware of any file editing method in Puppet that will modify
multiple files as part of the same resource.  The usual approach would
be to create multiple resources, one for each file, and then cause
each resource to notify the postfix service to restart.  Puppet''s
smart enough to only restart it once.  As far as setting up the files
the way you want them, if you need to edit the files in place Augeas
is your only built-in option.  You can also do hacky things with exec
but it''s not pleasant.  Most folks don''t edit in place, they
replace
the file with known good contents.  For that you can use templates or
something like puppet-concat (a third-party module that allows you to
build files from fragments).

On Mon, Nov 28, 2011 at 6:17 AM, Graham Leggett <minfrin@sharp.fm>
wrote:
> On 27 Nov 2011, at 20:04, Graham Leggett wrote:
>
>> I have a task to convince puppet and augeas to modify multiple postfix
>> files, most specifically /etc/postfix/main.cf and /etc/postifx/
>> acceptdomains.cf (and other LDAP related files), and I''m
struggling to
>> find out how to do that. I have found examples on how to use augeas to
>> modify just one file at a time, but no information seems to be
>> available when multiple files are involved.
>>
>> I started with the config published here, and have no idea what to
>> change to convince puppet+augeas to edit both /etc/postfix/main.cf
>> and /etc/postfix/acceptdomains.cf at the same time:
>>
>>
https://github.com/camptocamp/puppet-postfix/blob/master/manifests/config.pp
>>
>> Does anyone have a sample configuration on how you would modify
>> multiple files at once, or have a clue on where to start?
>
> Answering my own question, I''ve discovered that augeas
doesn''t support multiple files at all without a custom lens being
created to support it.
>
> Or to declare it more accurately, augeas only knows about specific files
defined in specific locations, it has no concept of a general file in a general
location that you might want to apply a lens to.
>
> I eventually worked around the issue by making a custom lens copied from
the augeas postfix_main.aug lens that listed the extra ldap templates as
matching a wildcard, at this point augeas started working correctly.
>
> Something else I discovered is that augeas has no error handling of any
kind - if it doesn''t recognise a file as matching one of the
predetermined locations built into the lens, it will silently ignore the file,
and not tell you why.
>
> Regards,
> Graham
> --
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mon, Nov 28, 2011 at 9:59 AM, Aaron Grewell
<aaron.grewell@gmail.com>wrote:
> I''m not aware of any file editing method in Puppet that will
modify
> multiple files as part of the same resource.  The usual approach would
> be to create multiple resources, one for each file, and then cause
> each resource to notify the postfix service to restart.

Note that you can wrap these multiple resources in a single defined
resource so that it "looks" like a single resource from the point of
view
of the person editing the manifest code.




>  Puppet''s
> smart enough to only restart it once.  As far as setting up the files
> the way you want them, if you need to edit the files in place Augeas
> is your only built-in option.  You can also do hacky things with exec
> but it''s not pleasant.  Most folks don''t edit in place,
they replace
> the file with known good contents.  For that you can use templates or
> something like puppet-concat (a third-party module that allows you to
> build files from fragments).
>
> On Mon, Nov 28, 2011 at 6:17 AM, Graham Leggett <minfrin@sharp.fm>
wrote:
> > On 27 Nov 2011, at 20:04, Graham Leggett wrote:
> >
> >> I have a task to convince puppet and augeas to modify multiple
postfix
> >> files, most specifically /etc/postfix/main.cf and /etc/postifx/
> >> acceptdomains.cf (and other LDAP related files), and I''m
struggling to
> >> find out how to do that. I have found examples on how to use
augeas to
> >> modify just one file at a time, but no information seems to be
> >> available when multiple files are involved.
> >>
> >> I started with the config published here, and have no idea what to
> >> change to convince puppet+augeas to edit both /etc/postfix/main.cf
> >> and /etc/postfix/acceptdomains.cf at the same time:
> >>
> >>
>
https://github.com/camptocamp/puppet-postfix/blob/master/manifests/config.pp
> >>
> >> Does anyone have a sample configuration on how you would modify
> >> multiple files at once, or have a clue on where to start?
> >
> > Answering my own question, I''ve discovered that augeas
doesn''t support
> multiple files at all without a custom lens being created to support it.
> >
> > Or to declare it more accurately, augeas only knows about specific
files
> defined in specific locations, it has no concept of a general file in a
> general location that you might want to apply a lens to.
> >
> > I eventually worked around the issue by making a custom lens copied
from
> the augeas postfix_main.aug lens that listed the extra ldap templates as
> matching a wildcard, at this point augeas started working correctly.
> >
> > Something else I discovered is that augeas has no error handling of
any
> kind - if it doesn''t recognise a file as matching one of the
predetermined
> locations built into the lens, it will silently ignore the file, and not
> tell you why.
>
Do you get the same results with the command line ''augtool''
Aaron?

If not, please bug report that we need to fix this in the Puppet Augeas
integration.

-- 
Nigel Kersten
Product Manager, Puppet Labs

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 29 November 2011 01:17, Graham Leggett <minfrin@sharp.fm> wrote:
> On 27 Nov 2011, at 20:04, Graham Leggett wrote:
>
> > I have a task to convince puppet and augeas to modify multiple postfix
> > files, most specifically /etc/postfix/main.cf and /etc/postifx/
> > acceptdomains.cf (and other LDAP related files), and I''m
struggling to
> > find out how to do that. I have found examples on how to use augeas to
>
Graham - the initial allure of Augeas is attractive, but we ended up using
puppet''s File resource with files and templates. It enabled us to have
the
files & templates under version control (big win) and gave us the
flexibility if we need to change other parts of the config file - you
don''t
end up with an Augeas resource per line of the file! And its much more
intuitive at a glance as to what is going on

Regards

John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi Graham,

On Nov 28, 9:17 am, Graham Leggett <minf...@sharp.fm>
wrote:
> On 27 Nov 2011, at 20:04, Graham Leggett wrote:
>
> > I have a task to convince puppet and augeas to modify multiple postfix
> > files, most specifically /etc/postfix/main.cf and /etc/postifx/
> > acceptdomains.cf (and other LDAP related files), and I''m
struggling to
> > find out how to do that. I have found examples on how to use augeas to
> > modify just one file at a time, but no information seems to be
> > available when multiple files are involved.
>
> > I started with the config published here, and have no idea what to
> > change to convince puppet+augeas to edit both /etc/postfix/main.cf
> > and /etc/postfix/acceptdomains.cf at the same time:
>
>
>https://github.com/camptocamp/puppet-postfix/blob/master/manifests/co...
>
> > Does anyone have a sample configuration on how you would modify
> > multiple files at once, or have a clue on where to start?
>
> Answering my own question, I''ve discovered that augeas
doesn''t support multiple files at all without a custom lens being
created to support it.
>
It sure does. Once Augeas maps all the files it knows about to the
tree, you can manipulate them altogether. In fact, you don''t even have
to care about files, since you''re manipulating a single tree which
contains them all.

For this reason, you can actually modify multiple files with a single
command.

> Or to declare it more accurately, augeas only knows about specific files
defined in specific locations, it has no concept of a general file in a general
location that you might want to apply a lens to.
I''m not sure to understand what you mean there. Can you be specific
about your need?

> I eventually worked around the issue by making a custom lens copied from
the augeas postfix_main.aug lens that listed the extra ldap templates as
matching a wildcard, at this point augeas started working correctly.

Are there standard loading paths missing from postfix_main.aug? In
this case, I''d be happy to add them so everybody can benefit from it.

> Something else I discovered is that augeas has no error handling of any
kind - if it doesn''t recognise a file as matching one of the
predetermined locations built into the lens, it will silently ignore the file,
and not tell you why.
If Augeas fails to load a file it knows about, it reports the errors
in /augeas/files/${file_path}/error, so you can list all the errors
generated by Augeas with a `print /augeas//error`.


Cheers,

Raphaël

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I''m not the OP Nigel, I think you meant Graham.  It sounds like Augeas
has some capabilities that I just wasn''t aware of.  In any case Augeas
is not something I use if I can avoid it.  Templates and concat are my
preferred methods for managing config files.  Straightforward > clever
for the general case.

On Mon, Nov 28, 2011 at 12:47 PM, Nigel Kersten <nigel@puppetlabs.com>
wrote:
>
>
> On Mon, Nov 28, 2011 at 9:59 AM, Aaron Grewell
<aaron.grewell@gmail.com>
> wrote:
>>
>> I''m not aware of any file editing method in Puppet that will
modify
>> multiple files as part of the same resource.  The usual approach would
>> be to create multiple resources, one for each file, and then cause
>> each resource to notify the postfix service to restart.
>
>
> Note that you can wrap these multiple resources in a single defined
resource
> so that it "looks" like a single resource from the point of view
of the
> person editing the manifest code.
>
>
>
>
>>
>>  Puppet''s
>> smart enough to only restart it once.  As far as setting up the files
>> the way you want them, if you need to edit the files in place Augeas
>> is your only built-in option.  You can also do hacky things with exec
>> but it''s not pleasant.  Most folks don''t edit in
place, they replace
>> the file with known good contents.  For that you can use templates or
>> something like puppet-concat (a third-party module that allows you to
>> build files from fragments).
>>
>> On Mon, Nov 28, 2011 at 6:17 AM, Graham Leggett
<minfrin@sharp.fm> wrote:
>> > On 27 Nov 2011, at 20:04, Graham Leggett wrote:
>> >
>> >> I have a task to convince puppet and augeas to modify multiple
postfix
>> >> files, most specifically /etc/postfix/main.cf and
/etc/postifx/
>> >> acceptdomains.cf (and other LDAP related files), and
I''m struggling to
>> >> find out how to do that. I have found examples on how to use
augeas to
>> >> modify just one file at a time, but no information seems to be
>> >> available when multiple files are involved.
>> >>
>> >> I started with the config published here, and have no idea
what to
>> >> change to convince puppet+augeas to edit both
/etc/postfix/main.cf
>> >> and /etc/postfix/acceptdomains.cf at the same time:
>> >>
>> >>
>> >>
https://github.com/camptocamp/puppet-postfix/blob/master/manifests/config.pp
>> >>
>> >> Does anyone have a sample configuration on how you would
modify
>> >> multiple files at once, or have a clue on where to start?
>> >
>> > Answering my own question, I''ve discovered that augeas
doesn''t support
>> > multiple files at all without a custom lens being created to
support it.
>> >
>> > Or to declare it more accurately, augeas only knows about specific
files
>> > defined in specific locations, it has no concept of a general file
in a
>> > general location that you might want to apply a lens to.
>> >
>> > I eventually worked around the issue by making a custom lens
copied from
>> > the augeas postfix_main.aug lens that listed the extra ldap
templates as
>> > matching a wildcard, at this point augeas started working
correctly.
>> >
>> > Something else I discovered is that augeas has no error handling
of any
>> > kind - if it doesn''t recognise a file as matching one of
the predetermined
>> > locations built into the lens, it will silently ignore the file,
and not
>> > tell you why.
>
>
> Do you get the same results with the command line
''augtool'' Aaron?
>
> If not, please bug report that we need to fix this in the Puppet Augeas
> integration.
>
> --
> Nigel Kersten
> Product Manager, Puppet Labs
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.