llowder@oreillyauto.com
2012-Jul-11 16:06 UTC
[Puppet Users] Environment and layout questions
If I have a module in /etc/puppet/modules/module_a And I also have /etc/puppet/environments/<env>modules/module_a and the one in the environments is different (say, it''s a dev or test env that I am using to test new module code or something like that) Will puppet use the test one if the environment setting matches, then fall back on the one from /etc/puppet/modules for other environments? Or would I be better off to use /etc/puppet/modules for distribution of custom facts/plugins, and have a copy of the modules in each environment folder? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/qzh5I1RM8JgJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 11/07/2012 09:06, llowder@oreillyauto.com wrote:> If I have a module in /etc/puppet/modules/module_a > > And I also have /etc/puppet/environments/<env>modules/module_a > > and the one in the environments is different (say, it''s a dev or test > env that I am using to test new module code or something like that) > > Will puppet use the test one if the environment setting matches, then > fall back on the one from /etc/puppet/modules for other environments?"Fall back" under what circumstances? if the module isn''t found in the environment folder? If you want Puppet to search your environment specific folder first, and then use a generic module folder if the module isn''t available then you can do that by adding the following module path in your puppet.conf: modulepath = /etc/puppet/environments/${environment}-modules/:/etc/puppet/modules Although it''s a not a layout I''ve seen anyone use, people generally maintain a full set of modules in a particular environment module directory. Is that what you meant? Craig -- Craig Dunn Professional Services Puppet Labs Inc. http://www.puppetlabs.com -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
llowder@oreillyauto.com
2012-Jul-11 18:37 UTC
Re: [Puppet Users] Environment and layout questions
Comments inline. On Wednesday, July 11, 2012 1:29:46 PM UTC-5, Craig Dunn wrote: <snip>> "Fall back" under what circumstances? if the module isn''t found in the > environment folder? >> If you want Puppet to search your environment specific folder first, and > then use a generic module folder if the module isn''t available then you > can do that by adding the following module path in your puppet.conf: > > modulepath = > /etc/puppet/environments/${environment}-modules/:/etc/puppet/modules > > Although it''s a not a layout I''ve seen anyone use, people generally > maintain a full set of modules in a particular environment module > directory. > >That does answer my question. I''m relatively new to puppet and am looking at ways to better use it. I''ve got a lot of questions still, but between this list and the IRC channel slowly whittling them down.> Is that what you meant? > Craig > > -- > Craig Dunn > Professional Services > Puppet Labs Inc. > http://www.puppetlabs.com > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/ZFvENRbceDIJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi Tim
I had a similar question in
https://groups.google.com/forum/?fromgroups#!topic/puppet-users/fhHYT3LkBoE
but spent a while figuring out and testing what is possible. Hopefully
this will help you or someone work their way around a corner.
Puppet Environments
* Common modules, environment/node specific configs
* Environment specific modules and common or environment/node specific
configs
It is possible to use both and failback/iterate over configs per
$environment and per node.
And it is possible to have a shared/common module in (modules) that is
specific to all environments and have the specific $environment or node
configs being served via environments. It is a little complicated but does
work and allows for quite a bit of flexibility serving environments.
I find that I do not want to maintain all modules per environment in
puppet, as that means maintaining $(( modules * number_of_environments )),
but do as shown later :)
Some modules lend then themselves to a common module but environment
specific configuration. To achieve this some puppet magic is required, a
few spells :) I shall try and describe the concept and steps here.
So for example it is possible to achieve something like the following
(caveat manifests/${environment} dirs are there if you are also using an
include ${environment}/*.pp for node definitions):
|+-environments/
| +-dev/
| | +-configs/
| | | |+-mysql_proxy/
| | | | +-${puppet_env}.mysql-proxy.erb
| | | | +-dev-server-1.mysql-proxy.erb
| | | |+-mysql_server/
| | | +-$hostname.cnf
| | | +-mysql.conf
| | +-modules/
| | | |+-httpd/
| | | |+-manifests/
| | | | +-init.pp
| | | |+-templates/
| | | +-httpd.conf.erb
| | +-repo/
| | |+-*.rpm
| +-prod/
| | +-configs/
| | | |+-mysql_proxy/
| | | | +-${puppet_env}.mysql-proxy.erb
| | | | +-prod-server-2.mysql-proxy.erb
| | | |+-mysql_server/
| | | +-$hostname.cnf
| | | +-mysql.conf
| | +-modules/
| | | |+-httpd/
| | | |+-manifests/
| | | | +-init.pp
| | | |+-templates/
| | | +-httpd.conf.erb
| | +-repo/
| | |+-*.rpm
|+-manifests/
| |+-dev/
| | +-dev-server-1.pp
| | +-dev-server-2.pp
| |+-prod/
| | +-prod-server-1.pp
| | +-prod-server-2.pp
| |+-extdata/
| | +-dev.csv
| | +-dev.csv
| |+-dev.pp
| |+-prod.pp
| |+-site.pp
|+-modules/
|+-mysql_proxy/
| |+-manifests/
| | +-init.pp
| |+-files/
| +-mysql-proxy.erb
|+-mysql_server/
|+-manifests/
+-init.pp
To achieve this you need to ensure environments are set in the puppet.conf
<SNIP>
[agent]
environment = <%= environment %>
</SNIP>
Just for clarification I map $environment to $puppet_env (for some backward
compatability issues as puppet environments has not always been avaiable).
<SNIP>
[agent]
environment = <%= puppet_env %>
</SNIP>
,
In the node manifest:
<SNIP>
$puppet_env = ''$::environment''
</SNIP>
In the puppetmaster puppet.conf:
<SNIP>
[master]
# Where the puppet manifests live
templatedir = /opt/puppet/manifests
modulepath = $confdir/environments/$environment/modules:$confdir/modules
manifest = $confdir/manifests/unknown_environment.pp
[dev]
manifest = $confdir/manifests/dev.pp
[prod]
manifest = $confdir/manifests/prod.pp
</SNIP>
Environment specific manifests. As you can see above, here if the node is
a dev node, the manifest/dev.pp will be served and it does:
<SNIP>
import ''site.pp''
import ''dev/*.pp''
</SNIP>
site.pp being common dev and prod variables, etc.
Then in the puppetmaster filerserver.conf (example from a erb template),
change the /opt/puppet path as appropriate.
For further clarification I use extdata and the extlookup function in erb
templates and have dev.csv and prod.csv extdata files and there is a top
scope variable of $puppet_repo = ''/opt/puppet''
Here is a snippet that serves
environments/${environment}/{configs,modules,repo} directories to the nodes.
fileserver.conf(.erb):
<SNIP>
[configs]
path /opt/puppet/environments/<%= environment %>/configs
####
# Environment nodes
<% node_ips.each do |val| -%>
allow <%= val %>
<% end -%>
<% if cloud_provider == "aws" %># aws allow private IP
allow <%= ec2_local_ipv4 %><% end %>
[repo]
path <%= puppet_repo %>/environments/<%= puppet_env %>/repo
####
# Environment nodes
<% node_ips.each do |val| -%>
allow <%= val %>
<% end -%>
<% if cloud_provider == "aws" %># aws allow private IP
allow <%= ec2_local_ipv4 %><% end %>
</SNIP>
A note regarding modules and environments, if you have environments
configured and you have a module is the
environments/${environment}/modules/module_a you cannot have
modules/module_a.
Putting it all together....
mysql_proxy example init.pp:
<SNIP>
# mysql-proxy config
file { ''/etc/sysconfig/mysql-proxy'':
owner => ''root'',
group => ''root'',
mode => ''0644'',
# Here we use an inline_template that calls the file() resource. The
template
# resource does not allow to use the "first file found" like the
source
resource
# does. However the file () resource exits after it finds the first file
# declared and therefore passes the first template found :)
# Novel as per Wolf Nobel''s method:
# http://www.mail-archive.com/puppet-users@googlegroups.com/msg27226.html
content => inline_template(
file(
"${puppet_repo}/environments/${puppet_env}/configs/mysql_proxy/files/${hostname}.mysql-proxy.erb",
"${puppet_repo}/environments/${puppet_env}/configs/mysql_proxy/files/${puppet_env}.${cloud}.mysql-proxy.erb",
"${puppet_repo}/environments/${puppet_env}/configs/mysql_proxy/files/${puppet_env}.mysql-proxy.erb")),
notify => Service[''mysql-proxy''],
}
</SNIP>
The inline_template method allows for you to specify different LOCAL
targets (not via fileserver), so this config would be equally valid and
work (failover):
<SNIP>
content => inline_template(
file(
"${puppet_repo}/environments/${puppet_env}/configs/mysql_proxy/files/${hostname}.mysql-proxy.erb",
"${puppet_repo}/environments/${puppet_env}/configs/mysql_proxy/files/${puppet_env}.${cloud}.mysql-proxy.erb",
"${puppet_repo}/modules/mysql_proxy/files/mysql-proxy.erb")),
</SNIP>
As I said, a few puppet magic spells are required. However this allows for
very flexible environment management.
I use a git repo for puppet and in the puppet repo I have a dev and prod
branch and automatically update the puppetmasters with git pull origin
$enviornment, this ensure that the environments are TOTALLY segregated at
an operational level and a mistaken dev change will not mistakenly be made
to a prod resource. Although this also means that there are 2 copies of
modules, etc, but prod updates are "special" tasks and this allows for
easy
visual directory diffs and diffs, package/repo management per environment,
etc, etc.
Locally I have:
git/dev/puppet
git/prod/puppet
My working dir is always dev and I always push with "git push origin
$environment", habit.
Anyway hope some of this is useful.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/sd-DFUDxYEAJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.