Puppet users - Jul 2009 - Managing upgrading of puppet recipes

In order to facilitate puppet manifest upgrades, I need to manage puppet
recipes on a per-server basis.

My puppet classifier spits out a variable called $puppet_iteration.

I tried putting this variable in my site.pp:

import
"/var/lib/puppet/files/static/applications/puppet/$puppet_iteration/system"

but it looks like the variable does not get interpreted in the import
statement:

err: Could not retrieve catalog: Could not parse for environment mgmt: No
file(s) found for import of
''/var/lib/puppet/files/static/applications/puppet/$puppet_iteration/system''
at /etc/puppet/manifests/site.pp:33

Can I get some suggestions for how to load specific manifests based on a
path partially determined by my puppet classifier?

Thanks in advance,
Pete

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

why don''t you use puppet environments?

Ohad

On Tue, Jul 7, 2009 at 7:46 AM, Pete Emerson <pemerson@gmail.com> wrote:
> In order to facilitate puppet manifest upgrades, I need to manage puppet
> recipes on a per-server basis.
>
> My puppet classifier spits out a variable called $puppet_iteration.
>
> I tried putting this variable in my site.pp:
>
> import
>
"/var/lib/puppet/files/static/applications/puppet/$puppet_iteration/system"
>
> but it looks like the variable does not get interpreted in the import
> statement:
>
> err: Could not retrieve catalog: Could not parse for environment mgmt: No
> file(s) found for import of
>
''/var/lib/puppet/files/static/applications/puppet/$puppet_iteration/system''
> at /etc/puppet/manifests/site.pp:33
>
> Can I get some suggestions for how to load specific manifests based on a
> path partially determined by my puppet classifier?
>
> Thanks in advance,
> Pete
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

I take it you''re referring to this:

http://reductivelabs.com/trac/puppet/wiki/UsingMultipleEnvironments

So on the puppetmaster server I''d need to generate a puppet.conf to
create a
[section] for each version I need to run. Since iterating over arrays
doesn''t appear to be allowed I guess I''d need to look into
writing my own
function, or simply externalize this to a script that I''d write.

On the client I''d manage the puppet.conf to set the environment
parameter to
the proper version (this I can easily do with a puppet recipe).

I''ll take a look at this today and see how easy it is to write my own
function to generate a file like this. I''ve never done this or written
ruby.
An adventure, for sure!

Any other suggestions on how to skin this cat?

Thanks, Ohad, for pointing me to this suggestion.

Pete

On Mon, Jul 6, 2009 at 8:32 PM, Ohad Levy <ohadlevy@gmail.com> wrote:
> why don''t you use puppet environments?
>
> Ohad
>
>
> On Tue, Jul 7, 2009 at 7:46 AM, Pete Emerson <pemerson@gmail.com>
wrote:
>
>> In order to facilitate puppet manifest upgrades, I need to manage
puppet
>> recipes on a per-server basis.
>>
>> My puppet classifier spits out a variable called $puppet_iteration.
>>
>> I tried putting this variable in my site.pp:
>>
>> import
>>
"/var/lib/puppet/files/static/applications/puppet/$puppet_iteration/system"
>>
>> but it looks like the variable does not get interpreted in the import
>> statement:
>>
>> err: Could not retrieve catalog: Could not parse for environment mgmt:
No
>> file(s) found for import of
>>
''/var/lib/puppet/files/static/applications/puppet/$puppet_iteration/system''
>> at /etc/puppet/manifests/site.pp:33
>>
>> Can I get some suggestions for how to load specific manifests based on
a
>> path partially determined by my puppet classifier?
>>
>> Thanks in advance,
>> Pete
>>
>>
>>
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Pete Emerson wrote:
> On the client I''d manage the puppet.conf to set the environment
> parameter to the proper version (this I can easily do with a puppet
recipe).
> 
> I''ll take a look at this today and see how easy it is to write my
own
> function to generate a file like this. I''ve never done this or
written
> ruby. An adventure, for sure!
Feel free to post your function or any code queries to the puppet-dev
list.  The people there may be able to assist or provide feedback and ideas.

Regards

James Turnbull

-- 
Author of:
* Pro Linux Systems Administration
(http://tinyurl.com/linuxadmin)
* Pulling Strings with Puppet
(http://tinyurl.com/pupbook)
* Pro Nagios 2.0
(http://tinyurl.com/pronagios)
* Hardening Linux
(http://tinyurl.com/hardeninglinux)

Maybe this can save you some time...

http://github.com/ohadlevy/puppet-multipuppetmaster/blob/582f98840e2c7bbea9d9e820a3060b09b0916306/templates/puppet.conf

Cheers,
Ohad

On 7/7/09, Pete Emerson <pemerson@gmail.com>
wrote:
> I take it you''re referring to this:
>
> http://reductivelabs.com/trac/puppet/wiki/UsingMultipleEnvironments
>
> So on the puppetmaster server I''d need to generate a puppet.conf
to create a
> [section] for each version I need to run. Since iterating over arrays
> doesn''t appear to be allowed I guess I''d need to look
into writing my own
> function, or simply externalize this to a script that I''d write.
>
> On the client I''d manage the puppet.conf to set the environment
parameter to
> the proper version (this I can easily do with a puppet recipe).
>
> I''ll take a look at this today and see how easy it is to write my
own
> function to generate a file like this. I''ve never done this or
written ruby.
> An adventure, for sure!
>
> Any other suggestions on how to skin this cat?
>
> Thanks, Ohad, for pointing me to this suggestion.
>
> Pete
>
> On Mon, Jul 6, 2009 at 8:32 PM, Ohad Levy <ohadlevy@gmail.com> wrote:
>
>> why don''t you use puppet environments?
>>
>> Ohad
>>
>>
>> On Tue, Jul 7, 2009 at 7:46 AM, Pete Emerson <pemerson@gmail.com>
wrote:
>>
>>> In order to facilitate puppet manifest upgrades, I need to manage
puppet
>>> recipes on a per-server basis.
>>>
>>> My puppet classifier spits out a variable called $puppet_iteration.
>>>
>>> I tried putting this variable in my site.pp:
>>>
>>> import
>>>
"/var/lib/puppet/files/static/applications/puppet/$puppet_iteration/system"
>>>
>>> but it looks like the variable does not get interpreted in the
import
>>> statement:
>>>
>>> err: Could not retrieve catalog: Could not parse for environment
mgmt: No
>>> file(s) found for import of
>>>
''/var/lib/puppet/files/static/applications/puppet/$puppet_iteration/system''
>>> at /etc/puppet/manifests/site.pp:33
>>>
>>> Can I get some suggestions for how to load specific manifests based
on a
>>> path partially determined by my puppet classifier?
>>>
>>> Thanks in advance,
>>> Pete
>>>
>>>
>>>
>>
>> >
>>
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Brilliant! I think that will save me tremendously. I''ll play with it
and let
you know. Thanks very much.

Pete

On Tue, Jul 7, 2009 at 8:22 AM, Ohad Levy <ohadlevy@gmail.com> wrote:
>
> Maybe this can save you some time...
>
>
>
http://github.com/ohadlevy/puppet-multipuppetmaster/blob/582f98840e2c7bbea9d9e820a3060b09b0916306/templates/puppet.conf
>
> Cheers,
> Ohad
>
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

So far I''ve gone the route of having two static environments,
"production"
(which seems to be the default) and "development". I had my own
$environment
definition which was overriding puppet''s definition of $environment
that
caused some problems for a little while, but I''ve got that all sorted
out,
and now by changing my database (which in turn changes the results in the
puppet node classifier) I can easily flip back and forth between the
production and development environments. Very slick!

If I wind up playing with your dynamic creation of puppet.conf, I''ll
let you
know how it goes.

Thanks again for your pointer.

Pete

On Tue, Jul 7, 2009 at 8:45 AM, Pete Emerson <pemerson@gmail.com> wrote:
> Brilliant! I think that will save me tremendously. I''ll play with
it and
> let you know. Thanks very much.
>
> Pete
>
>
> On Tue, Jul 7, 2009 at 8:22 AM, Ohad Levy <ohadlevy@gmail.com> wrote:
>
>>
>> Maybe this can save you some time...
>>
>>
>>
http://github.com/ohadlevy/puppet-multipuppetmaster/blob/582f98840e2c7bbea9d9e820a3060b09b0916306/templates/puppet.conf
>>
>> Cheers,
>> Ohad
>>
>>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Pete Emerson wrote:
> I take it you''re referring to this:
> 
> http://reductivelabs.com/trac/puppet/wiki/UsingMultipleEnvironments
> 
> So on the puppetmaster server I''d need to generate a puppet.conf
to
> create a [section] for each version I need to run. Since iterating over 
> arrays doesn''t appear to be allowed I guess I''d need to
look into
> writing my own function, or simply externalize this to a script that
I''d
> write.
Here''s a definition I wrote to manage puppet.conf environments:

     define puppet_environment()
     {
	$envdir = "$puppet_conf_prefix/$name"
	augeas {
	    "puppet-environ--$name":
		context => "/files/etc/puppet/puppet.conf/$name",
		changes => [
		    "rm *",
		    "set manifestdir $envdir/manifests",
		    "set manifest    $envdir/manifests/site.pp",
		    "set modulepath  $envdir/modules",
		],
		require => Package["puppetmaster"],
		notify => Service["puppetmaster"];
	}
	file {
	    $envdir:
		ensure => directory;
	}
     }

Then use it like this:

     $puppet_conf_prefix = "/etc/puppet/environments"
     $puppet_environments = [ "production", "testing",
"devel-1", "devel-2" ]

     puppet_environment {
	$puppet_environments: ;
     }

The advantage of this over a template is that Puppet won''t overwrite
the
entire puppet.conf, just the environments in $puppet_environments.

I tend to use lots of "throw-away" environments for my development,
that
may live for anywhere between an hour or several weeks, depending on how
long time the development of a certain feature takes.  I use them somewhat
similar to how one uses topic branches in Git.  When I develop a new
feature, I create an environment for it, do my editing there, and when
I feel ready to test it, I try to run it once (using puppetd --onetime)
on a node that normally runs the production environment.  Often with
--noop as well until I think it actually does what I want.  When I feel
more confident about my changes, I change that node to that environment
and run it that way for a few days, and when I think it''s stable, I
check
in my changes to the VCS, and check them out in the production environment,
switching back my test nodes to production as well.  And then remove that
development environment.

Depending on how big or riskful a change I''m doing is, I may test my
changes on several different nodes, and/or I may shorten the time my
test node runs my development environment.  (Most of the times they
are small enough that I can skip semi-permanent running entirely.  I
also have a special development environment that I never erase, but
re-use again and again for very small changes, the kind that takes
just a few minutes; in fact, each sysadmin here has their own such
environment.)

I can have several of these temporary environments open in parallell,
if I''m developing and testing several different things.

With the above workflow, it would be quite disruptive to have Puppet
suddenly overwrite puppet.conf on the master with a version that
doesn''t
have my environment in it.  And having to add my environment to my
Puppet manifests before being able to use it feels like too much
overhead.  (In fact, I would like to not have to edit puppet.conf at
all to add or remove environments.  Being able to configure a
"wildcard"
environment in puppet.conf would be nice.)


	/Thomas Bellman

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Pete Emerson wrote:
> So on the puppetmaster server I''d need to generate a puppet.conf
to
> create a [section] for each version I need to run. 
> **Since iterating over arrays doesn''t appear to be allowed **
[emphasis mine]
> I guess I''d need to look into 
> writing my own function, or simply externalize this to a script that
I''d
> write.
This is soooo wrong. Sorry, but puppet allows you very easily to process 
arrays with defines:

define puppet::config_section() {
   include puppet::config_section_base
   concatenated_file_part {
     $name:
       dir => "/var/lib/puppet/modules/puppet/config.d",
       content => template("puppet/config_section.erb");
   }
}

puppet::config_section{
   [ "production", "development", "testing",
"blah" ]:
}





Regards, DavidS

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Here''s the solution I wound up using. I created a puppet.conf.erb
template file and had puppet create the puppet.conf file dynamically.

The critical pieces (I only wanted directories starting with v, like
v01, v02, et cetera):

<%
dirs = Dir.entries("/etc/puppet/manifests").sort
-%>

[main]
...
[puppetmasterd]
...
[puppetd]
...

<% for dir in dirs -%>
        <% if File.directory?("/etc/puppet/manifests/" + dir) and
dir
!= ''.'' and dir != ''..'' and dir =~ /^v/
-%>

[<%= dir %>]
    manifest = /etc/puppet/manifests/<%= dir %>/site.pp

        <% end -%>
<% end -%>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---