Puppet users - May 2009 - dealing with numbered items in Augeas

I haven''t seen this really discussed in the wiki or on the list. I  
suspect anyone with enough knowledge to help me can already tell where  
I''m going from the subject, but here''s a lengthy description
of the
problem anyway…

For files like sshd_config or Postfix''s main.cf, using Augeas is very  
straightforward. You have a bunch of unique keys, you set the values  
for the ones you care about, Puppet can tell what does and does not  
need to be changed and you get on with your life.

However, what seems like a majority of files are broken up into an  
"array" of values. Take the `/etc/exports` [example from the wiki][1].
Existing items are assigned numbers 1-4 by Augeas when the file is  
loaded. If you add an item with an index of 10000, the tree will  
contain items 1, 2, 3, 4 and 10000 until you save the changes. But on  
the next Puppet run, Augeas will load the file and number the items  
1-5. There won''t be an item 10000, so what''s to stop it from
being
added over and over again on every run? The same can be said for  
inserting things. Say I want to insert something after /2 in `/etc/ 
inittab`. There will be a /2 in the tree on every single run, so  
what''s to stop it from inserting again and again?

I guess what I''m getting at is that the Augeas type seems extremely  
whatever-the-opposite-of-idempotent-is by default. Or at least it''s  
very easy for a person to accidentally make a mess.

I realize the numbering of items in certain files is unavoidable, but  
is there a good way to handle these situations in Puppet? Here are  
some real-world examples from my manifests that seem to work.

     # adds rh:06:wait:/etc/rc.shutdown
     augeas { "shutdown":
       require => File["shutdown"],
       context => "/files/etc/inittab",
       changes => [
         "ins 3 after 2",
         "set 3[1]/id rh",
         "set 3[1]/runlevels 06",
         "set 3[1]/action wait",
         "set 3[1]/process /etc/rc.shutdown",
       ],
       onlyif => "get 3/id != rh",
     }

This inserts a line after the `/etc/rc.d/rc.sysinit` line. Works fine  
now, but what if the sysinit line isn''t item 2 in every future version
of the OS? Maybe I could add "only if 2''s id is
''si''" to make sure,
but what if it''s *not* ''si''? It won''t get
added to the wrong place,
but it won''t get added anywhere else either. Clearly not what
I''m
shooting for.

     # boot systems to runlevel 3
     augeas { "runlevel":
       context => "/files",
       changes => [
         "set /etc/inittab/1/runlevels 3",
       ],
       onlyif => "get /etc/inittab/1/action == initdefault",
     }

Again, this works, but what if initdefault was *not* item 1? I''ve  
prevented the wrong thing from happening, but I haven''t ensured the  
right thing. (By the way, why are items numbered in inittab when they  
all have an "id"?)

On a related note, there doesn''t seem to be a good way to add comments
to a particular spot to explain the changes you''ve made. This is true  
even for the "straightforward" files mentioned above. Any advice?

Please view this as a question and not a complaint. If I didn''t think  
Augeas and its Puppet type were awesome, I wouldn''t have tried to use  
them heavily enough to encounter the above. Thanks.

[1]: http://reductivelabs.com/trac/puppet/wiki/PuppetAugeas

-- 
Rob McBroom
<http://www.skurfer.com/>


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

I think that the heart of the problem is the convention of using
"10000" as
an Augeas key to mean "add this to the end of the list".  As Rob
points out,
you don''t know how many items are already on the list.  If there are
less
than 10,000 entries already, you wind up with funky numbering (1, 2, 3, 4,
5, 10000) for the duration of the Augeas run.  If there are more than 10,000
entries in the list already, you wind up inadvertently overwriting one of
those entries.
If your goal is to append something to an Augeas list, then I think
it''s
much better to use the [last()+1] notation.  The
http://reductivelabs.com/trac/puppet/wiki/PuppetAugeas wiki page shows the
following example for adding a new entry to the /etc/exports file:

augtool> set /files/etc/exports/dir[10000] /foo
augtool> set /files/etc/exports/dir[last()]/client weeble
augtool> set /files/etc/exports/dir[last()]/client/option[1] ro
augtool> set /files/etc/exports/dir[last()]/client/option[2] all_squash
augtool> save
Saved 1 file(s)


I think that the following (which appears to work) would be much more
reliable (note that I''m simply replacing the "10000" in the
first line with
"last()+1":

augtool> set /files/etc/exports/dir[last()+1] /foo
augtool> set /files/etc/exports/dir[last()]/client weeble
augtool> set /files/etc/exports/dir[last()]/client/option[1] ro
augtool> set /files/etc/exports/dir[last()]/client/option[2] all_squash
augtool> save
Saved 1 file(s)


This essentially says "create a new entry numbered one greater than the
current last entry, and then set a bunch of options on that entry (which is
now the last one)".

As for the second problem which Rob raises, that of making sure you''re
editing the section of an Augeas tree that you think you are, I''d again
suggest that the problem is in using the Augeas index numbers directly.  Rob
gives the following example, to add a new 3rd line to /etc/inittab, which
assumes that line 2 is the "/etc/rc.d/rc.sysinit" line:

    # adds rh:06:wait:/etc/rc.shutdown
    augeas { "shutdown":
      require => File["shutdown"],
      context => "/files/etc/inittab",
      changes => [
        "ins 3 after 2",
        "set 3[1]/id rh",
        "set 3[1]/runlevels 06",
        "set 3[1]/action wait",
        "set 3[1]/process /etc/rc.shutdown",
      ],
      onlyif => "get 3/id != rh",
    }

What I''d suggest here is, instead of referring to the relevant entries
by
Augeas index number (which happens to correspond to line number, in this
case), refer to them using the Augeas "path expressions" mechanism (
http://augeas.net/page/Path_expressions).  In other words, don''t say
"add
this after line 2", but rather "add this after the line whose
''process''
field is ''/etc/rc.d/rc.sysinit''".

Try this instead; I think it will do what you want:

    # adds rh:06:wait:/etc/rc.shutdown
    augeas { "shutdown":
      require => File["shutdown"],
      context => "/files/etc/inittab",
      changes => [
        "ins 0 after *[process=\"/etc/rc.d/rc.sysinit\"]",
        "set 0/id rh",
        "set 0/runlevels 06",
        "set 0/action wait",
        "set 0/process /etc/rc.shutdown",
      ],
    }

Note that I''ve changed the label for the new entry from "3"
to "0".  It
doesn''t much matter what the label is, as long as its a non-negative
integer, because that''s what the Augeas inittab lens wants the labels
for
entries in /files/etc/inittab to be (it won''t properly generate the new
file
otherwise, for instance if the label you create is "X"; the lens
generates
the entries in the order they appear in the tree, however, _not_ in numeric
order).

As a general practice, I''d say that you want to avoid explicitly using
the
"line number" indexes that Augeas generates when it loads a file.  To
append
a record to the end of a list, instead of "10000" (which makes
assumptions
about how many records there might be), use "last()+1" to create a new
record and then simply "last()" to reference that new record.  To add
something after a particular line in a file, use the Augeas path expression
search methods to find the relevant line, rather than assuming that the line
number will never change.

Treat the numeric indexes produced by Augeas as if they were pointers in C.
 You never want to hardcode the value of a pointer into your code, because
the value might be different the next time the program is run.  You might do
some simple relative arithmetic against a pointer ("last()+1" is an
example
of that), but that''s about all.

Good luck!

-Brent
-- 
Brent Chapman <brent@netomata.com>
Founder and CEO // Netomata, Inc. // www.netomata.com
Making networks more cost-effective, reliable, and flexible by automating
network configuration

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On 2009-May-16, at 12:42 PM, Brent Chapman wrote:
> If your goal is to append something to an Augeas list, then I think  
> it''s much better to use the [last()+1] notation.
There are a couple of problems with that. Using last() is a better  
method as far as general Augeas technique goes, but it doesn''t address
the problem with using Augeas in Puppet. Also, the last() function  
only seems to be available for labels with multiple values. It doesn''t
seem to work for things like /etc/inittab or /etc/hosts where the  
numbers assigned represent lines in the file. For instance:

     augtool> ls /files/etc/inittab/last()
     augtool> ls /files/etc/inittab/20
     id = x
     runlevels = 5
     action = respawn
     process = /etc/X11/prefdm -nodaemon

Here are a couple of examples to further illustrate what I''m trying to
avoid. First, something as simple as adding a host to /etc/hosts  
(using 10 since last() isn''t an option here):

     augeas { "repeathost":
       context => "/files/etc/hosts",
       changes => [
         "set 10/ipaddr 10.10.10.10",
         "set 10/canonical foo",
         "set 10/alias foo.localdomain",
       ]
     }

This adds the same host on every puppet run until there are 10 hosts  
total, at which point it compares the changes to the existing item 10,  
sees that there is nothing to be done, and moves on (as it should).

The same happens if you use last() instead of an arbitrary high number:

     augeas { "repeatalias":
       context => "/files/etc/hosts",
       changes => [
         "set 4/ipaddr 10.10.10.10",
         "set 4/canonical foo",
         "set 4/alias foo.localdomain",
         "set 4/alias[last()+1] bar.localdomain",
       ]
     }

This will add the "bar" alias over and over on every puppet run until
the disk fills up or something else breaks.

I understand *why* this is happening. It just doesn''t seem very Puppet-
like, so I wanted to make sure I''m not overlooking something. It seems
that to use the Augeas type, you have to explicitly tell it "unless  
this has already been done" with nearly every augeas resource you  
define. With user, file, yumrepo, package, and every other type of  
resource I''ve worked with, you just tell it what you want to be true  
and Puppet will determine if any action is required to make it happen.

If this is the way it is, then fine. Maybe it''s not technically  
feasible to make the augeas type figure this stuff out on its own. I  
just want to confirm that before I put a lot of time into working  
around this limitation.
> As for the second problem which Rob raises, that of making sure  
> you''re editing the section of an Augeas tree that you think you
are,
> I''d again suggest that the problem is in using the Augeas index  
> numbers directly.  Rob gives the following example, to add a new 3rd  
> line to /etc/inittab, which assumes that line 2 is the "/etc/rc.d/ 
> rc.sysinit" line:
>
>     # adds rh:06:wait:/etc/rc.shutdown
>     augeas { "shutdown":
>       require => File["shutdown"],
>       context => "/files/etc/inittab",
>       changes => [
>         "ins 3 after 2",
>         "set 3[1]/id rh",
>         "set 3[1]/runlevels 06",
>         "set 3[1]/action wait",
>         "set 3[1]/process /etc/rc.shutdown",
>       ],
>       onlyif => "get 3/id != rh",
>     }
>
> What I''d suggest here is, instead of referring to the relevant  
> entries by Augeas index number (which happens to correspond to line  
> number, in this case), refer to them using the Augeas "path  
> expressions" mechanism (http://augeas.net/page/Path_expressions).
I''ll look into that.
> As a general practice, I''d say that you want to avoid explicitly  
> using the "line number" indexes that Augeas generates when it
loads
> a file.
Oh, I surely do. I just wasn''t sure how. Looks like path expressions  
will help in some situations.

Thanks again.

-- 
Rob McBroom
<http://www.skurfer.com/>

The magnitude of a problem does not affect its ownership.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

While this is kind of kludg-y, might you try building the multipart
file with Puppet itself:

http://reductivelabs.com/trac/puppet/wiki/Recipes/BuildingMultipartFiles

It''s useful for building files where you have chunked sections and, of
course, you can use templates etc... where necessary.

I have no idea what it does to performance though.

Trevor

On Fri, May 15, 2009 at 14:41, Rob McBroom <mailinglist0@skurfer.com>
wrote:
>
> I haven''t seen this really discussed in the wiki or on the list. I
> suspect anyone with enough knowledge to help me can already tell where
> I''m going from the subject, but here''s a lengthy
description of the
> problem anyway…
>
> For files like sshd_config or Postfix''s main.cf, using Augeas is
very
> straightforward. You have a bunch of unique keys, you set the values
> for the ones you care about, Puppet can tell what does and does not
> need to be changed and you get on with your life.
>
> However, what seems like a majority of files are broken up into an
> "array" of values. Take the `/etc/exports` [example from the
wiki][1].
> Existing items are assigned numbers 1-4 by Augeas when the file is
> loaded. If you add an item with an index of 10000, the tree will
> contain items 1, 2, 3, 4 and 10000 until you save the changes. But on
> the next Puppet run, Augeas will load the file and number the items
> 1-5. There won''t be an item 10000, so what''s to stop it
from being
> added over and over again on every run? The same can be said for
> inserting things. Say I want to insert something after /2 in `/etc/
> inittab`. There will be a /2 in the tree on every single run, so
> what''s to stop it from inserting again and again?
>
> I guess what I''m getting at is that the Augeas type seems
extremely
> whatever-the-opposite-of-idempotent-is by default. Or at least
it''s
> very easy for a person to accidentally make a mess.
>
> I realize the numbering of items in certain files is unavoidable, but
> is there a good way to handle these situations in Puppet? Here are
> some real-world examples from my manifests that seem to work.
>
>     # adds rh:06:wait:/etc/rc.shutdown
>     augeas { "shutdown":
>       require => File["shutdown"],
>       context => "/files/etc/inittab",
>       changes => [
>         "ins 3 after 2",
>         "set 3[1]/id rh",
>         "set 3[1]/runlevels 06",
>         "set 3[1]/action wait",
>         "set 3[1]/process /etc/rc.shutdown",
>       ],
>       onlyif => "get 3/id != rh",
>     }
>
> This inserts a line after the `/etc/rc.d/rc.sysinit` line. Works fine
> now, but what if the sysinit line isn''t item 2 in every future
version
> of the OS? Maybe I could add "only if 2''s id is
''si''" to make sure,
> but what if it''s *not* ''si''? It won''t
get added to the wrong place,
> but it won''t get added anywhere else either. Clearly not what
I''m
> shooting for.
>
>     # boot systems to runlevel 3
>     augeas { "runlevel":
>       context => "/files",
>       changes => [
>         "set /etc/inittab/1/runlevels 3",
>       ],
>       onlyif => "get /etc/inittab/1/action == initdefault",
>     }
>
> Again, this works, but what if initdefault was *not* item 1? I''ve
> prevented the wrong thing from happening, but I haven''t ensured
the
> right thing. (By the way, why are items numbered in inittab when they
> all have an "id"?)
>
> On a related note, there doesn''t seem to be a good way to add
comments
> to a particular spot to explain the changes you''ve made. This is
true
> even for the "straightforward" files mentioned above. Any advice?
>
> Please view this as a question and not a complaint. If I didn''t
think
> Augeas and its Puppet type were awesome, I wouldn''t have tried to
use
> them heavily enough to encounter the above. Thanks.
>
> [1]: http://reductivelabs.com/trac/puppet/wiki/PuppetAugeas
>
> --
> Rob McBroom
> <http://www.skurfer.com/>
>
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Mon, May 18, 2009 at 9:01 AM, Rob McBroom
<mailinglist0@skurfer.com>wrote:
>
> On 2009-May-16, at 12:42 PM, Brent Chapman wrote:
>
> > If your goal is to append something to an Augeas list, then I think
> > it''s much better to use the [last()+1] notation.
>
> There are a couple of problems with that. Using last() is a better
> method as far as general Augeas technique goes, but it doesn''t
address
> the problem with using Augeas in Puppet. Also, the last() function
> only seems to be available for labels with multiple values. It
doesn''t
> seem to work for things like /etc/inittab or /etc/hosts where the
> numbers assigned represent lines in the file. For instance:
>
>     augtool> ls /files/etc/inittab/last()

The correct syntax in this case would be:

augtool> ls /files/etc/inittab/*[last()]
id = x
runlevels = 5
action = respawn
process = /etc/X11/prefdm -nodaemon


-Brent
-- 
Brent Chapman <brent@netomata.com>
Founder and CEO // Netomata, Inc. // www.netomata.com
Making networks more reliable and flexible by automating network
configuration

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Mon, May 18, 2009 at 10:53 AM, Trevor Vaughan
<peiriannydd@gmail.com>wrote:
>
> While this is kind of kludg-y, might you try building the multipart
> file with Puppet itself:
>
> http://reductivelabs.com/trac/puppet/wiki/Recipes/BuildingMultipartFiles
>
> It''s useful for building files where you have chunked sections
and, of
> course, you can use templates etc... where necessary.
>
> I have no idea what it does to performance though.
>
> Trevor
>
This is a cool trick, but it''s solving a fundamentally different
problem
than using Augeas would.  With this approach, you''re saying
"here''s exactly
what should be in the file; I don''t care what''s there already,
replace it
with this".  In contrast, the Augeas approach is "I''m not
sure what''s in the
file already, but here is the list of changes that need to be made to the
file; make these changes, but leave the rest of the file (including
comments) alone."

I tend to favor the former approach, of specifying exactly what should be in
a given config file; it''s the approach I take with our Netomata Config
Generator (NCG) tool (http://www.netomata.com/products/ncg).  It''s not
the
right approach for all circumstances, though, and Augeas might be useful in
those cases.

-Brent
-- 
Brent Chapman <brent@netomata.com>
Founder and CEO // Netomata, Inc. // www.netomata.com
Making networks more reliable and flexible by automating network
configuration

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Brent Chapman wrote:
> On Mon, May 18, 2009 at 10:53 AM, Trevor Vaughan
<peiriannydd@gmail.com>wrote:
> 
>> While this is kind of kludg-y, might you try building the multipart
>> file with Puppet itself:
>>
>>
http://reductivelabs.com/trac/puppet/wiki/Recipes/BuildingMultipartFiles
>>
>> It''s useful for building files where you have chunked sections
and, of
>> course, you can use templates etc... where necessary.
>>
>> I have no idea what it does to performance though.
>>
>> Trevor
>>
> 
> This is a cool trick, but it''s solving a fundamentally different
problem
> than using Augeas would.  With this approach, you''re saying
"here''s exactly
> what should be in the file; I don''t care what''s there
already, replace it
> with this".  In contrast, the Augeas approach is "I''m
not sure what''s in the
> file already, but here is the list of changes that need to be made to the
> file; make these changes, but leave the rest of the file (including
> comments) alone."
> 
> I tend to favor the former approach, of specifying exactly what should be
in
> a given config file; it''s the approach I take with our Netomata
Config
> Generator (NCG) tool (http://www.netomata.com/products/ncg).  It''s
not the
> right approach for all circumstances, though, and Augeas might be useful in
> those cases.
> 
> -Brent
The augeas approach is useful when many modules which do not know about 
each other need to handle the same file. As you said, this is useful in 
some cases where the exact final topology is not known until the modules 
are combined.

-- bk


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Brent,

Thanks for the feedback.

It felt like a bit of a hack when I wrote it but it very much follows
the Puppet mentality of idempotency.

I''m just starting to fiddle with Augeas since 90% of my problems fall
into the precise declaration arena.

Trevor

On Mon, May 18, 2009 at 17:16, Brent Chapman <brent@netomata.com>
wrote:
> On Mon, May 18, 2009 at 10:53 AM, Trevor Vaughan
<peiriannydd@gmail.com>
> wrote:
>>
>> While this is kind of kludg-y, might you try building the multipart
>> file with Puppet itself:
>>
>>
http://reductivelabs.com/trac/puppet/wiki/Recipes/BuildingMultipartFiles
>>
>> It''s useful for building files where you have chunked sections
and, of
>> course, you can use templates etc... where necessary.
>>
>> I have no idea what it does to performance though.
>>
>> Trevor
>
> This is a cool trick, but it''s solving a fundamentally different
problem
> than using Augeas would.  With this approach, you''re saying
"here''s exactly
> what should be in the file; I don''t care what''s there
already, replace it
> with this".  In contrast, the Augeas approach is "I''m
not sure what''s in the
> file already, but here is the list of changes that need to be made to the
> file; make these changes, but leave the rest of the file (including
> comments) alone."
> I tend to favor the former approach, of specifying exactly what should be
in
> a given config file; it''s the approach I take with our Netomata
Config
> Generator (NCG) tool (http://www.netomata.com/products/ncg).  It''s
not the
> right approach for all circumstances, though, and Augeas might be useful in
> those cases.
>
> -Brent
> --
> Brent Chapman <brent@netomata.com>
> Founder and CEO // Netomata, Inc. // www.netomata.com
> Making networks more reliable and flexible by automating network
> configuration
>
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Fri, 2009-05-15 at 14:41 -0400, Rob McBroom wrote:
> However, what seems like a majority of files are broken up into an  
> "array" of values. Take the `/etc/exports` [example from the
wiki][1].
> Existing items are assigned numbers 1-4 by Augeas when the file is  
> loaded. 
> If you add an item with an index of 10000, 
I started the ''10000'' idiom because I couldn''t think
of anything better,
and because in TeX 10000 is infinity, but in practice it''s of course a
very finite value.

Thinking about that I realize that there''s a very easy way to create
labels that are (a) numbers and (b) unique that you can use instead of
10000: prefix them with one or more ''0''s. The label itself has
no real
meaning, it just has to be a valid number (more precisely: it needs to
match the regexp /[0-9]+/) and Augeas will never use leading zeroes when
it reads in a file.
> the tree will  
> contain items 1, 2, 3, 4 and 10000 until you save the changes. But on  
> the next Puppet run, Augeas will load the file and number the items  
> 1-5. There won''t be an item 10000, so what''s to stop it
from being
> added over and over again on every run? The same can be said for  
> inserting things. Say I want to insert something after /2 in `/etc/ 
> inittab`. There will be a /2 in the tree on every single run, so  
> what''s to stop it from inserting again and again?
Think of the subtree under /2 etc. as a record - in almost all cases,
these records have some notion of primary key; for /etc/hosts, that''s
the ipaddr. If the ''127.0.0.1'' line is the first in
/etc/hosts, you can
access the rest of the line as /files/etc/hosts/1/... _or_ by using
XPath notation to go by the primary key: /files/etc/hosts/*[ipaddr
''127.0.0.1''] The latter will always give you the localhost
line,
regardless of where it is in the file.
> I realize the numbering of items in certain files is unavoidable, but  
> is there a good way to handle these situations in Puppet? Here are  
> some real-world examples from my manifests that seem to work.
> 
>      # adds rh:06:wait:/etc/rc.shutdown
>      augeas { "shutdown":
>        require => File["shutdown"],
>        context => "/files/etc/inittab",
>        changes => [
>          "ins 3 after 2",
You don''t need the explicit ''ins'' to append - set
will do that when you
access a nonexistant node automatically.
>          "set 3[1]/id rh",
>          "set 3[1]/runlevels 06",
>          "set 3[1]/action wait",
>          "set 3[1]/process /etc/rc.shutdown",
>        ],
>        onlyif => "get 3/id != rh",
>      }
Putting all the above together, you can write this as

             # adds rh:06:wait:/etc/rc.shutdown
             augeas { "shutdown":
               require => File["shutdown"],
               context => "/files/etc/inittab",
               changes => [
                 "set 01/id rh",
                 "set 01/runlevels 06",
                 "set 01/action wait",
                 "set 01/process /etc/rc.shutdown",
               ],
               onlyif => "match *[id = ''rh''] size ==
0",
             }
        
> This inserts a line after the `/etc/rc.d/rc.sysinit` line. Works fine  
> now, but what if the sysinit line isn''t item 2 in every future
version
> of the OS? Maybe I could add "only if 2''s id is
''si''" to make sure,
> but what if it''s *not* ''si''?
If you always want your entry to show up after the ''si'' line,
you ned to
do the explicit ''ins'' as the first thing in your changes,
something like
''ins 01 after */[id = ''si'']''
>  It won''t get added to the wrong place,  
> but it won''t get added anywhere else either. Clearly not what
I''m
> shooting for.
> 
>      # boot systems to runlevel 3
>      augeas { "runlevel":
>        context => "/files",
>        changes => [
>          "set /etc/inittab/1/runlevels 3",
>        ],
>        onlyif => "get /etc/inittab/1/action == initdefault",
>      }
> 
> Again, this works, but what if initdefault was *not* item 1? 
Use "set /etc/inittab/*[action = ''initdefault'']/runlevels
3" - for that,
you don''t even need an onlyif. If runlevels is already 3, Augeas will
notice that nothing has changed when the tree is saved, and won''t touch
the file at all.

The XPath notation for Augeas is described in more detail at [1]
> On a related note, there doesn''t seem to be a good way to add
comments
> to a particular spot to explain the changes you''ve made. This is
true
> even for the "straightforward" files mentioned above. Any advice?
Most lenses now map comments as nodes labelled ''#comment'', and
you can
do something like the following to add a comment just before a given
node:

        ins #comment before /files/etc/hosts/*[ipaddr =
''127.0.0.1'']
        set /files/etc/hosts/#comment[. = ''''] "My new
comment"
        
though the ''set'' is a bit of a kludge (it looks for comment
nodes with
an empty comment) - the Augeas path expressions clearly need a better
way to refer to newly created nodes, or to refer to nodes by a more
complicated criterion, i.e. so that you can say ''the #comment node just
before the one with an ipaddr child with value 127.0.0.1''.
> Please view this as a question and not a complaint.
This is great feedback.

David

[1] http://augeas.net/page/Path_expressions



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Fri, 2009-05-15 at 14:41 -0400, Rob McBroom wrote:
> However, what seems like a majority of files are broken up into an  
> "array" of values. Take the `/etc/exports` [example from the
wiki][1].
> Existing items are assigned numbers 1-4 by Augeas when the file is  
> loaded.
I should have looked at the way /etc/exports is mapped before my last
reply - because there, the entries aren''t really numbered. In the tree
you just have a list of ''dir'' nodes underneath
/files/etc/exports - the
notation dir[1], dir[2], ... is only used to disambiguate those nodes,
but the ''[1]'' etc. are not really part of the node name, and
you can
indeed force a new one to be created by saying

  set /files/etc/exports/dir[last() + 1] /myexport
  set /files/etc/exports/dir[last()]/client 192.168.0.0/24
  ....

Note that the first ''set'' creates a new
''dir'' node (since there can
never be a node with index last() + 1), but once that node is created
it''s referenced as dir[last()]

David



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On 2009-May-19, at 4:09 PM, David Lutterkort wrote:
> Think of the subtree under /2 etc. as a record - in almost all cases,
> these records have some notion of primary key;
Which is why I wonder why /etc/inittab doesn''t just use the
''id'' field
instead of assigning a number, or why /etc/hosts doesn''t just use the  
IP address. But like I said, treating things as a numbered sequence is  
going to be unavoidable in some places, so my time is probably better  
spent figuring out how to handle it, rather than nitpicking this or  
that lens.
> Putting all the above together, you can write this as
>
>             # adds rh:06:wait:/etc/rc.shutdown
>             augeas { "shutdown":
>               require => File["shutdown"],
>               context => "/files/etc/inittab",
>               changes => [
>                 "set 01/id rh",
>                 "set 01/runlevels 06",
>                 "set 01/action wait",
>                 "set 01/process /etc/rc.shutdown",
>               ],
>               onlyif => "match *[id = ''rh''] size
== 0",
>             }
If I don''t care about order (which I probably shouldn''t), this
works
well. Thanks. One minor correction: I had to remove whitespace inside  
the square brackets to get it to work, so it ended up being

     onlyif => "match *[id=''rh''] size == 0",
> If you always want your entry to show up after the ''si''
line, you
> ned to
> do the explicit ''ins'' as the first thing in your changes,
something
> like
> ''ins 01 after */[id = ''si'']''
For the sake of discussion, let''s say I was going to try an insert in  
a specific place. I haven''t been able to get the path expression to  
work in that context. I have tried each of the following:

     "ins 0 after *[id=''si'']",
     "ins 0 after *[id=\"si\"]",
     "ins 0 after *[id=\''si\'']",
     ''ins 0 after *[id="si"]'',
     ''ins 0 after *[id=\''si\'']'',
     ''ins 0 after *[id=\"si\"]'',
     "ins 0 after \"*[id=''si'']\"",

And in every case, I get an error implying that it sees ''si''
and the
closing ] as two additional parameters, rather than as part of the  
expression.

     err: //was/Augeas[shutdown]: Failed to retrieve current state of  
resource: Error sending command ''ins 0 after *[id='' with
params ["si",
"]"]/Command ''ins 0 after *[id='' is not supported

The following version will go through without error, but it doesn''t  
actually match anything so the new item gets inserted at the end of  
the file:

     "ins 0 after *[id=si]",

For what it''s worth, this works fine in augtool:

     ins 0 after /files/etc/inittab/*[id=''si'']

Should I file a bug or am I doing something wrong here? I''m running  
RHEL 5.3 with Puppet 0.24.8 and Augeas 0.5.0 from EPEL.
>>     # boot systems to runlevel 3
>>     augeas { "runlevel":
>>       context => "/files",
>>       changes => [
>>         "set /etc/inittab/1/runlevels 3",
>>       ],
>>       onlyif => "get /etc/inittab/1/action ==
initdefault",
>>     }
>>
>> Again, this works, but what if initdefault was *not* item 1?
>
> Use "set /etc/inittab/*[action =
''initdefault'']/runlevels 3" - for
> that,
> you don''t even need an onlyif. If runlevels is already 3, Augeas
will
> notice that nothing has changed when the tree is saved, and won''t
> touch
> the file at all.
Great! That seems much cleaner. Unfortunately, I get the same type of  
errors as the above when trying to use it.

     err: //Augeas[runlevel]: Failed to retrieve current state of  
resource: Error sending command ''set /etc/inittab/*[action=''
with
params ["initdefault", "]/runlevels 3"]/Command
''set /etc/inittab/
*[action='' is not supported
>> On a related note, there doesn''t seem to be a good way to add
>> comments
>> to a particular spot to explain the changes you''ve made. This
is true
>> even for the "straightforward" files mentioned above. Any
advice?
>
> Most lenses now map comments as nodes labelled
''#comment'', and you can
> do something like the following to add a comment just before a given
> node:
>
>        ins #comment before /files/etc/hosts/*[ipaddr =
''127.0.0.1'']
>        set /files/etc/hosts/#comment[. = ''''] "My new
comment"
>
> though the ''set'' is a bit of a kludge (it looks for
comment nodes with
> an empty comment) - the Augeas path expressions clearly need a better
> way to refer to newly created nodes, or to refer to nodes by a more
> complicated criterion, i.e. so that you can say ''the #comment node
> just
> before the one with an ipaddr child with value 127.0.0.1''.
Sure, there''s room for improvement, but I think this trick will  
accomplish what I want as long as I can count on changes being  
processed in order. For instance, canonical_maps doesn''t exist by  
default, so in order to put a comment before it, I need to set it first.

     augeas { "main.cf":
       require => Package["postfix"],
       context => "/files/etc/postfix/main.cf",
       changes => [
         "set canonical_maps ldap:/etc/postfix/addresses.ldap",
         "ins #comment before canonical_maps",
         "set #comment[.=''''] ''refer to LDAP for
e-mail address''",
       ],
       notify => Service["postfix"],
     }

Of course, the "#comment[.='''']" expression throws
errors too, so I
can''t use this until that''s sorted out. :)

Lots of good info in this thread. Thanks everyone.

-- 
Rob McBroom
<http://www.skurfer.com/>


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi Rob,

Rob McBroom wrote:
>       err: //was/Augeas[shutdown]: Failed to retrieve current state of
> resource: Error sending command ''ins 0 after *[id='' with
params ["si",
> "]"]/Command ''ins 0 after *[id='' is not
supported
I think you''re hitting this bug that I reported:

http://projects.reductivelabs.com/issues/2141

However, I don''t know git/ruby well enough to actually test the
attached
fix, so it''s still broken in my install.

Hope that helps you, though. :)

Thanks,
Avi

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Wed, 2009-05-20 at 09:45 -0400, Rob McBroom wrote:
> On 2009-May-19, at 4:09 PM, David Lutterkort wrote:
> 
> > Think of the subtree under /2 etc. as a record - in almost all cases,
> > these records have some notion of primary key;
> 
> Which is why I wonder why /etc/inittab doesn''t just use the
''id'' field
> instead of assigning a number, or why /etc/hosts doesn''t just use
the
> IP address. But like I said, treating things as a numbered sequence is  
> going to be unavoidable in some places, so my time is probably better  
> spent figuring out how to handle it, rather than nitpicking this or  
> that lens.
Heh ;) The reason to use numbered entries vs. the same label repeatedly
is a little subtle and has to do with how spacing is preserved when you
add or delete entries (with numbered entries, the spacing on hte line
''belongs'' to the entry with that number, whereas with the same
labels
the spacing of the lines is used in the order it appeared in the file,
even if a new entry is inserted in the middle)
> > If you always want your entry to show up after the
''si'' line, you
> > ned to
> > do the explicit ''ins'' as the first thing in your
changes, something
> > like
> > ''ins 01 after */[id = ''si'']''
> 
> For the sake of discussion, let''s say I was going to try an insert
in
> a specific place. I haven''t been able to get the path expression
to
> work in that context. I have tried each of the following:
> 
>      "ins 0 after *[id=''si'']",
>      "ins 0 after *[id=\"si\"]",
>      "ins 0 after *[id=\''si\'']",
>      ''ins 0 after *[id="si"]'',
>      ''ins 0 after *[id=\''si\'']'',
>      ''ins 0 after *[id=\"si\"]'',
>      "ins 0 after \"*[id=''si'']\"",
> 
> And in every case, I get an error implying that it sees
''si'' and the
> closing ] as two additional parameters, rather than as part of the  
> expression.
> 
>      err: //was/Augeas[shutdown]: Failed to retrieve current state of  
> resource: Error sending command ''ins 0 after *[id='' with
params ["si",
> "]"]/Command ''ins 0 after *[id='' is not
supported
Yeah, as Avi mentioned, you''re being bitten by bug #2141 - if your git
is up to it, you could cherrypick the fix into your tree, otherwise
you''ll have to wait for the next puppet release :(
> Should I file a bug or am I doing something wrong here? I''m
running
> RHEL 5.3 with Puppet 0.24.8 and Augeas 0.5.0 from EPEL.
No need - Avi already did that ;)
> Sure, there''s room for improvement, but I think this trick will  
> accomplish what I want as long as I can count on changes being  
> processed in order. For instance, canonical_maps doesn''t exist by
> default, so in order to put a comment before it, I need to set it first.
> 
>      augeas { "main.cf":
>        require => Package["postfix"],
>        context => "/files/etc/postfix/main.cf",
>        changes => [
>          "set canonical_maps ldap:/etc/postfix/addresses.ldap",
>          "ins #comment before canonical_maps",
>          "set #comment[.=''''] ''refer to LDAP
for e-mail address''",
>        ],
>        notify => Service["postfix"],
>      }
> 
> Of course, the "#comment[.='''']" expression
throws errors too, so I
> can''t use this until that''s sorted out. :)
Yeah, #2141 at work again ;) One warning: since you are inserting a new
node, you''d need to guard this with an onlyif checking if
canonical_maps
is there already - otherwise every puppet run will add a new comment
line.

David



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---