Puppet users - Feb 2009 - Integrating puppet with etckeeper and similar tools.

G''day.

We are currently looking to integrate etckeeper[1] into our puppet
managed hosts, especially the legacy hosts that are still partially
under manual control.

etckeeper is, essentially, a wrapper around a VCS for /etc, tracking
changes in file content.

When I say "integrate" I mean, specifically, that we would like puppet
to commit any changes using etckeeper after running, to capture all the
changes into the revision history.[2]


So, essentially, what I would like is to run an ''exec'' command
at the
very end of puppet processing, after *all* other actions are complete.

It there any sane way to express this in the puppet language?

(We do already run puppet from a shell script, so I will integrate this
 there if I can''t do it within the manifest, but I would rather avoid
 that path if I can...)

Regards,
        Daniel

Footnotes: 
[1]  http://kitenet.net/~joey/code/etckeeper/

[2]  This also allows us to push those changes to a central host, do
     change reporting and other similar VCS based activity.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Feb 5, 2009, at 8:14 PM, Daniel Pittman wrote:
>
> G''day.
>
> We are currently looking to integrate etckeeper[1] into our puppet
> managed hosts, especially the legacy hosts that are still partially
> under manual control.
>
> etckeeper is, essentially, a wrapper around a VCS for /etc, tracking
> changes in file content.
>
> When I say "integrate" I mean, specifically, that we would like
puppet
> to commit any changes using etckeeper after running, to capture all  
> the
> changes into the revision history.[2]
>
>
> So, essentially, what I would like is to run an ''exec''
command at the
> very end of puppet processing, after *all* other actions are complete.
>
> It there any sane way to express this in the puppet language?
>
> (We do already run puppet from a shell script, so I will integrate  
> this
> there if I can''t do it within the manifest, but I would rather
avoid
> that path if I can...)

I''ve never been able to come up with a clean way of forcing a given  
resource to the end of the graph.  We could just implement
''first'' and
''last'' metaparams, but what happens when you have multiple of
them?

Really, if someone has some experience with directed graphs and is  
interested in spending a bit of time on Puppet, we could probably come  
up with a straightforward way of doing this, but it requires a new way  
of specifying relationships plus some skullduggery while building the  
graph.

I actually just had an idea for how to fix this:

Currently, the whole resource graph starts with Class[main], and could  
even be said to be contained by Class[main].  Given that the resource  
graph is a directed acyclic graph, this means that if you specify a  
relationship to Class[main], it''s either a cycle or a noop, depending  
on the direction of the graph.

However, we could add something special that said, if you specify a  
relationship to Class[main], it removes your other containment  
relationship.  This would allow you to say, X resource should happen  
before/after the rest of the graph.  It would have some limitations,  
though.

Take a simple configuration like this:

service { "foo": subscribe => MyResource[stuff] }

myresource { stuff: do => stuff }

define myresource($do) {
   exec { "$do $stuff": require => Class[main] }
}

The exec is ''contained by'' the ''myresource''
instance, so there''s an
edge between them.  My proposal would sever that edge, thus forcing  
the exec to run after the rest of the configuration.   But the service  
is subscribed to Myresource, with the implication being that it''s  
actually interested in events from the Exec.  In this situation,  
you''ve broken the edge between myresource and the exec, so the service
could never see events cascading that way.

However, it''s the closest I''ve gotten to a solution.

Any comments?

-- 
SELF-EVIDENT, adj. Evident to one''s self and to nobody else.
     -- Ambrose Bierce
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Luke Kanies <luke@madstop.com> writes:
> On Feb 5, 2009, at 8:14 PM, Daniel Pittman wrote:
>>
>> We are currently looking to integrate etckeeper[1] into our puppet
>> managed hosts, especially the legacy hosts that are still partially
>> under manual control.
>>
>> etckeeper is, essentially, a wrapper around a VCS for /etc, tracking
>> changes in file content.
>>
>> When I say "integrate" I mean, specifically, that we would
like
>> puppet to commit any changes using etckeeper after running, to
>> capture all the changes into the revision history.[2]
>>
>>
>> So, essentially, what I would like is to run an
''exec'' command at the
>> very end of puppet processing, after *all* other actions are
>> complete.
>>
>> It there any sane way to express this in the puppet language?
>>
>> (We do already run puppet from a shell script, so I will integrate
>> this there if I can''t do it within the manifest, but I would
rather
>> avoid that path if I can...)
>
>
> I''ve never been able to come up with a clean way of forcing a
given
> resource to the end of the graph.  We could just implement
''first'' and
> ''last'' metaparams, but what happens when you have
multiple of them?
Hmmmm.  My expectation of multiple items with a "last" tag is that
they
would behave like any other round of resources: unordered, unless they
had internal dependencies.

  exec { "example 1": last => true }
  exec { "example 2": last => true, require =>
Exec["example 1"] }
  exec { "example 2": last => true }

That would, I would anticipate, run example 2 after example 1, but
without defined order to example 3.


Perhaps something like "weight" would be more appropriate, to allow a
resource to be pushed around in the graph?

  exec { "example 1": weight => 100 }  # 100 is standard
  exec { "example 2": weight => 200 }  # gonna happen later...
  exec { "example 3": weight => 150 }  # half way between...
  exec { "example 4": weight => 300, before =>
Exec["example 1"] }

In that, I would imagine that the resolver would run example 4 prior to
example 1, but might emit a warning about the constraint being
roughed-up, perhaps?
> Really, if someone has some experience with directed graphs and is
> interested in spending a bit of time on Puppet, we could probably come
> up with a straightforward way of doing this, but it requires a new way
> of specifying relationships plus some skullduggery while building the
> graph.
I think the biggest problem, though, is that I don''t have a formal
notion of what I want to achieve, just an informal desire to integrate
with a bit of software in a specific fashion.

I might, perhaps, think a bit more about this and see if I can make some
more concrete proposal about the whole thing...

Regards,
        Daniel

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

+1 for the weight argument, overridden by inherent ordering of course
(notify/subscribe).

I''ve been in a few situations where it would be *really nice* to have
something happen at the end, or beginning, of everything.

This does, of course, lend itself to ordering abuse, but every bit of
flexibility has a down side.  Not adding weights would limit the
flexibility of the solution but would also make it just annoying
enough to only use it when you actually need to.

My suggestion for running the script after everything is to, for now,
just blob it onto the end of your script that''s running Puppet.  The
only other thing that I can think of is to have some global exec that
you have everything notify at the top of your site.pp but that''s a
complete kludge.

Basically:

--- site.pp ---

exec { "do_my_stuff": command => "/bin/random_stuff",
refreshonly => true }

File [ notify => Exec["do_my_stuff"] ]
Service [ notify => Exec["do_my_stuff"] ]
etc...

... the rest of your manifest ...

I find this entirely unpleasant, but it would probably do what you
want it to in general.  You would just have to be careful to use
''notify +>'' everywhere else in your manifests.

Ew...

Trevor

On Tue, Feb 10, 2009 at 22:54, Daniel Pittman <daniel@rimspace.net>
wrote:
>
> Luke Kanies <luke@madstop.com> writes:
>> On Feb 5, 2009, at 8:14 PM, Daniel Pittman wrote:
>>>
>>> We are currently looking to integrate etckeeper[1] into our puppet
>>> managed hosts, especially the legacy hosts that are still partially
>>> under manual control.
>>>
>>> etckeeper is, essentially, a wrapper around a VCS for /etc,
tracking
>>> changes in file content.
>>>
>>> When I say "integrate" I mean, specifically, that we
would like
>>> puppet to commit any changes using etckeeper after running, to
>>> capture all the changes into the revision history.[2]
>>>
>>>
>>> So, essentially, what I would like is to run an
''exec'' command at the
>>> very end of puppet processing, after *all* other actions are
>>> complete.
>>>
>>> It there any sane way to express this in the puppet language?
>>>
>>> (We do already run puppet from a shell script, so I will integrate
>>> this there if I can''t do it within the manifest, but I
would rather
>>> avoid that path if I can...)
>>
>>
>> I''ve never been able to come up with a clean way of forcing a
given
>> resource to the end of the graph.  We could just implement
''first'' and
>> ''last'' metaparams, but what happens when you have
multiple of them?
>
> Hmmmm.  My expectation of multiple items with a "last" tag is
that they
> would behave like any other round of resources: unordered, unless they
> had internal dependencies.
>
>  exec { "example 1": last => true }
>  exec { "example 2": last => true, require =>
Exec["example 1"] }
>  exec { "example 2": last => true }
>
> That would, I would anticipate, run example 2 after example 1, but
> without defined order to example 3.
>
>
> Perhaps something like "weight" would be more appropriate, to
allow a
> resource to be pushed around in the graph?
>
>  exec { "example 1": weight => 100 }  # 100 is standard
>  exec { "example 2": weight => 200 }  # gonna happen later...
>  exec { "example 3": weight => 150 }  # half way between...
>  exec { "example 4": weight => 300, before =>
Exec["example 1"] }
>
> In that, I would imagine that the resolver would run example 4 prior to
> example 1, but might emit a warning about the constraint being
> roughed-up, perhaps?
>
>> Really, if someone has some experience with directed graphs and is
>> interested in spending a bit of time on Puppet, we could probably come
>> up with a straightforward way of doing this, but it requires a new way
>> of specifying relationships plus some skullduggery while building the
>> graph.
>
> I think the biggest problem, though, is that I don''t have a formal
> notion of what I want to achieve, just an informal desire to integrate
> with a bit of software in a specific fashion.
>
> I might, perhaps, think a bit more about this and see if I can make some
> more concrete proposal about the whole thing...
>
> Regards,
>        Daniel
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---