Puppet users - Mar 2010 - plugin in modules - Environments - documentation?

Hey there,

i''m stuck with the plugin in modules section.
i''ve got 2 environments set called
development and production.

my puppet.conf on the puppetmasterd looks like this.

[development]
    manifest = /etc/puppet/development/manifests/site.pp
    manifestdir = /etc/puppet/development/manifests
    templatedir = /etc/puppet/development/manifests/templates
    modulepath = /etc/puppet/development/modules

[production]
    manifest = /etc/puppet/production/manifests/site.pp
    manifestdir = /etc/puppet/production/manifests
    templatedir = /etc/puppet/production/manifests/templates
    modulepath = /etc/puppet/production/modules

[puppetmasterd]
    dbuser = puppet
    dbname = puppet
    dbpassword = password
    dbadapter = mysql
    dbserver = localhost
    dbsocket = /var/lib/mysql/mysql.sock
    masterport = 8140
    group = root
    storeconfigs = true
    node_name = facter
    user = root
    autosign = true
    libdir = /opt/puppet-dashboard

Al my clients are connecting with this puppet.conf:

[puppetd]
    pluginsync = true
    factsync = true
    environment = development
    listen = false
    report = true
    yamldir = /var/puppet/yaml
    node_name = facter
    rrddir = /var/puppet/rrd
    rrdinterval = 1800
    railslog = /var/puppet/log/rails.log
    templatedir = /var/puppet/templates

now i wanted to have the sysctl plugin inserted in my development
environment without harming anything in production.
http://github.com/duritong/puppet-sysctl

Now this page is refering to me, that this will not work...
http://projects.puppetlabs.com/projects/puppet/wiki/Using_Multiple_Environments
"Note that currently Plugins In Modules don’t mix well with
environments."

meaning that i should create this?
/etc/puppet/development/modules/facts
/etc/puppet/development/modules/plugins/files/puppet/provider/sysctl/
parsed.rb
/etc/puppet/development/modules/plugins/files/puppet/type/sysctl.rb

but of course here it still stating:
http://docs.reductivelabs.com/guides/plugins_in_modules.html

in 0.25.0 and later releases, Puppet changes uses ‘lib’ for the name
of the plugins directory.
This change was introducued in 0.25.0 and modules with an outdated
plugins directory name will generate a deprecation warning. The
plugins directory will be formally deprecated in the Rowlf release of
Puppet. This deprecation changes your module paths to:

<MODULEPATH>/<module>/lib/puppet/type
For providers, place them in:

<MODULEPATH>/<module>/lib/puppet/provider
Similarly, Facter facts belong in the facter subdirectory of the
library directory:

<MODULEPATH>/<module>/lib/facter
So, if we are using our custom module and our modulepath is /etc/
puppet/modules then types and facts would be stored in the following
directories:

/etc/puppet/modules/custom/lib/puppet/type
/etc/puppet/modules/custom/lib/puppet/provider
/etc/puppet/modules/custom/lib/facter


so with other words:
how do i needs to setup the filesystem to actually sync my custom
modules over to the client etc...
hope anybody can help.. i''ve lost the complete overview...

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

JeFFreaK wrote:
> i''m stuck with the plugin in modules section.
> i''ve got 2 environments set called
> development and production.
[...]
> now i wanted to have the sysctl plugin inserted in my development
> environment without harming anything in production.
> http://github.com/duritong/puppet-sysctl
Unfortunately, not possible.  There is some hope that this will
work at least a little bit better in Puppet 0.26 (or whatever it
will be called), but currently you are out of luck.

For custom types and custom functions, puppetmasterd will only
look at the *global* value of modulepath, i.e the one you have
set under [puppetmasterd] or under [main].  The modulepath you
have set under each environment will not be used for those.  You
didn''t have a global modulepath setting at all in puppet.conf;
you need to add one to get plugins in modules to work *at*all*.

(Note, though, that the per-environment modulepath setting *will*
be used for custom facts.  If you are using 0.25 on both clients
and server, it will even work correctly, but if you use 0.24, it
will give you strange results.)
> Now this page is refering to me, that this will not work...
>
http://projects.puppetlabs.com/projects/puppet/wiki/Using_Multiple_Environments
> "Note that currently Plugins In Modules don’t mix well with
> environments."
Indeed, they don''t.  You cannot have custom types or custom
functions per environment.  All environments will share the same
set of plugins (except for facts).
> so with other words:
> how do i needs to setup the filesystem to actually sync my custom
> modules over to the client etc...
> hope anybody can help.. i''ve lost the complete overview...
The complete picture is somewhat lacking in the documentation, yes.

The way I have set up things, is that at the global level (under the
[puppetmasterd] heading), I set modulepath to point to the modules
directory of my production environment.  The only way to change my
custom types or custom functions, is to update them in the production
environment, and I must ensure that my other environments work with
what is available in production.

To develop new versions of my plugins, I start with using plain
puppet, instead of the puppetd/puppetmasterd combo.  Depending on
how confident I am with new plugins, I may then either promote
them directly to production, or I may set up a separate puppetmaster
server for testing more, before promoting them to production.


	/Bellman

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tue, Mar 30, 2010 at 7:49 AM, Thomas Bellman <bellman@nsc.liu.se>
wrote:
> JeFFreaK wrote:
>
>> i''m stuck with the plugin in modules section.
>> i''ve got 2 environments set called
>> development and production.
>
> [...]
>>
>> now i wanted to have the sysctl plugin inserted in my development
>> environment without harming anything in production.
>> http://github.com/duritong/puppet-sysctl
>
> Unfortunately, not possible.  There is some hope that this will
> work at least a little bit better in Puppet 0.26 (or whatever it
> will be called), but currently you are out of luck.
>
> For custom types and custom functions, puppetmasterd will only
> look at the *global* value of modulepath, i.e the one you have
> set under [puppetmasterd] or under [main].  The modulepath you
> have set under each environment will not be used for those.  You
> didn''t have a global modulepath setting at all in puppet.conf;
> you need to add one to get plugins in modules to work *at*all*.
>
> (Note, though, that the per-environment modulepath setting *will*
> be used for custom facts.  If you are using 0.25 on both clients
> and server, it will even work correctly, but if you use 0.24, it
> will give you strange results.)
>
>> Now this page is refering to me, that this will not work...
>>
>>
http://projects.puppetlabs.com/projects/puppet/wiki/Using_Multiple_Environments
>> "Note that currently Plugins In Modules don’t mix well with
>> environments."
>
> Indeed, they don''t.  You cannot have custom types or custom
> functions per environment.  All environments will share the same
> set of plugins (except for facts).
That''s not actually quite true as I understand it.

You can indeed serve different sets of plugins to different
environments for pluginsync, however the server needs to "know" about
the existence of all these plugins in it''s own libdir.

>
>> so with other words:
>> how do i needs to setup the filesystem to actually sync my custom
>> modules over to the client etc...
>> hope anybody can help.. i''ve lost the complete overview...
>
> The complete picture is somewhat lacking in the documentation, yes.
>
> The way I have set up things, is that at the global level (under the
> [puppetmasterd] heading), I set modulepath to point to the modules
> directory of my production environment.  The only way to change my
> custom types or custom functions, is to update them in the production
> environment, and I must ensure that my other environments work with
> what is available in production.
>
> To develop new versions of my plugins, I start with using plain
> puppet, instead of the puppetd/puppetmasterd combo.  Depending on
> how confident I am with new plugins, I may then either promote
> them directly to production, or I may set up a separate puppetmaster
> server for testing more, before promoting them to production.
>
>
>        /Bellman
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>


-- 
nigel

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 3/30/10 8:49 AM, Thomas Bellman wrote:
> JeFFreaK wrote:
>
>> i''m stuck with the plugin in modules section.
>> i''ve got 2 environments set called
>> development and production.
> [...]
>> now i wanted to have the sysctl plugin inserted in my development
>> environment without harming anything in production.
>> http://github.com/duritong/puppet-sysctl
>
> Unfortunately, not possible.  There is some hope that this will
> work at least a little bit better in Puppet 0.26 (or whatever it
> will be called), but currently you are out of luck.
This isn''t quite the same - we have a custom package provider (as test 
to show we could do it) - what we''re doing is specifying different 
"modulepath" for each environment in the server''s puppet.conf
file.

To make it work we had to put the provider in our puppetmaster''s 
$libdir/puppet/provider/package directory, as well as the right place in 
the module..

That seems to be working for us.

We haven''t tried creating a separate type
though..
>
> For custom types and custom functions, puppetmasterd will only
> look at the *global* value of modulepath, i.e the one you have
> set under [puppetmasterd] or under [main].  The modulepath you
> have set under each environment will not be used for those.  You
> didn''t have a global modulepath setting at all in puppet.conf;
> you need to add one to get plugins in modules to work *at*all*.
So it doesn''t look at "modulepath" under an environment
definition in
the puppet.conf on the pupetmaster?

[snip...]

Regards,
-Roy


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Nigel Kersten wrote:
> On Tue, Mar 30, 2010 at 7:49 AM, Thomas Bellman <bellman@nsc.liu.se>
wrote:
>> Indeed, they don''t.  You cannot have custom types or custom
>> functions per environment.  All environments will share the same
>> set of plugins (except for facts).
> That''s not actually quite true as I understand it.
> You can indeed serve different sets of plugins to different
> environments for pluginsync, however the server needs to "know"
about
> the existence of all these plugins in it''s own libdir.
You might be able to have different custom *providers* (for either
custom types or for standard types), for each environment.  But
anything that executes on the puppet master, like parameter validation,
or the existence of the custom type, is always the same for all
environments.  But I haven''t actually tried that myself (the custom
types I have written doesn''t make any sense for to have multiple
providers).  I also seem to remember seeing a thread in puppet-dev
a couple of days ago where they said Puppet had some bug that pulled
in the providers on the master even where it shouldn''t have needed to.

And custom functions execute entirely on the master, so those cannot
be per environment at all.  Puppetd will pull them down to itself, but
they won''t be of any use to it, so I don''t understand why it
does so.

I think it is at least a good first approximation to say that custom
types and custom functions cannot be per environment.  At least then
you will not tear out your hair trying to understand what is happening
when you get strange results. :-)


	/Bellman

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tue, Mar 30, 2010 at 8:21 AM, Roy Nielsen <rsn@lanl.gov>
wrote:
> On 3/30/10 8:49 AM, Thomas Bellman wrote:
>>
>> JeFFreaK wrote:
>>
>>> i''m stuck with the plugin in modules section.
>>> i''ve got 2 environments set called
>>> development and production.
>>
>> [...]
>>>
>>> now i wanted to have the sysctl plugin inserted in my development
>>> environment without harming anything in production.
>>> http://github.com/duritong/puppet-sysctl
>>
>> Unfortunately, not possible.  There is some hope that this will
>> work at least a little bit better in Puppet 0.26 (or whatever it
>> will be called), but currently you are out of luck.
>
> This isn''t quite the same - we have a custom package provider (as
test to
> show we could do it) - what we''re doing is specifying different
"modulepath"
> for each environment in the server''s puppet.conf file.
>
> To make it work we had to put the provider in our puppetmaster''s
> $libdir/puppet/provider/package directory, as well as the right place in
the
> module..
>
> That seems to be working for us.
>
> We haven''t tried creating a separate type though..
>>
>> For custom types and custom functions, puppetmasterd will only
>> look at the *global* value of modulepath, i.e the one you have
>> set under [puppetmasterd] or under [main].  The modulepath you
>> have set under each environment will not be used for those.  You
>> didn''t have a global modulepath setting at all in puppet.conf;
>> you need to add one to get plugins in modules to work *at*all*.
>
> So it doesn''t look at "modulepath" under an environment
definition in the
> puppet.conf on the pupetmaster?
It does for *delivering* to clients, but for validating the manifests
in order to turn them into a catalog, it looks at the local libdir for
the puppetmaster.

As Thomas says in the next message that just arrived, functions are
different as they execute on the server, and types are different due
to parameter validation being employed.

The problem I reported on puppet-dev with providers being pulled in is
only if you''re setting global resource defaults in your manifests.


>
> [snip...]
>
> Regards,
> -Roy
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>


-- 
nigel

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 3/30/10 9:33 AM, Nigel Kersten wrote:
> On Tue, Mar 30, 2010 at 8:21 AM, Roy Nielsen<rsn@lanl.gov>  wrote:
>    
>> On 3/30/10 8:49 AM, Thomas Bellman wrote:
>>      
>>> JeFFreaK wrote:
>>>
>>>        
>>>> i''m stuck with the plugin in modules section.
>>>> i''ve got 2 environments set called
>>>> development and production.
>>>>          
>>> [...]
>>>        
>>>> now i wanted to have the sysctl plugin inserted in my
development
>>>> environment without harming anything in production.
>>>> http://github.com/duritong/puppet-sysctl
>>>>          
>>> Unfortunately, not possible.  There is some hope that this will
>>> work at least a little bit better in Puppet 0.26 (or whatever it
>>> will be called), but currently you are out of luck.
>>>        
>> This isn''t quite the same - we have a custom package provider
(as test to
>> show we could do it) - what we''re doing is specifying
different "modulepath"
>> for each environment in the server''s puppet.conf file.
>>
>> To make it work we had to put the provider in our
puppetmaster''s
>> $libdir/puppet/provider/package directory, as well as the right place
in the
>> module..
>>
>> That seems to be working for us.
>>
>> We haven''t tried creating a separate type though..
>>      
>>> For custom types and custom functions, puppetmasterd will only
>>> look at the *global* value of modulepath, i.e the one you have
>>> set under [puppetmasterd] or under [main].  The modulepath you
>>> have set under each environment will not be used for those.  You
>>> didn''t have a global modulepath setting at all in
puppet.conf;
>>> you need to add one to get plugins in modules to work *at*all*.
>>>        
>> So it doesn''t look at "modulepath" under an
environment definition in the
>> puppet.conf on the pupetmaster?
>>      
> It does for *delivering* to clients, but for validating the manifests
> in order to turn them into a catalog, it looks at the local libdir for
> the puppetmaster.
>
> As Thomas says in the next message that just arrived, functions are
> different as they execute on the server, and types are different due
> to parameter validation being employed.
>
> The problem I reported on puppet-dev with providers being pulled in is
> only if you''re setting global resource defaults in your manifests.
>    
ahh.. ok.  Thanks,
-Roy


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tue, Mar 30, 2010 at 9:09 AM, Roy Nielsen <rsn@lanl.gov>
wrote:
> On 3/30/10 9:33 AM, Nigel Kersten wrote:
>>
>> On Tue, Mar 30, 2010 at 8:21 AM, Roy Nielsen<rsn@lanl.gov>
 wrote:
>>
>>>
>>> On 3/30/10 8:49 AM, Thomas Bellman wrote:
>>>
>>>>
>>>> JeFFreaK wrote:
>>>>
>>>>
>>>>>
>>>>> i''m stuck with the plugin in modules section.
>>>>> i''ve got 2 environments set called
>>>>> development and production.
>>>>>
>>>>
>>>> [...]
>>>>
>>>>>
>>>>> now i wanted to have the sysctl plugin inserted in my
development
>>>>> environment without harming anything in production.
>>>>> http://github.com/duritong/puppet-sysctl
>>>>>
>>>>
>>>> Unfortunately, not possible.  There is some hope that this will
>>>> work at least a little bit better in Puppet 0.26 (or whatever
it
>>>> will be called), but currently you are out of luck.
>>>>
>>>
>>> This isn''t quite the same - we have a custom package
provider (as test to
>>> show we could do it) - what we''re doing is specifying
different
>>> "modulepath"
>>> for each environment in the server''s puppet.conf file.
>>>
>>> To make it work we had to put the provider in our
puppetmaster''s
>>> $libdir/puppet/provider/package directory, as well as the right
place in
>>> the
>>> module..
>>>
>>> That seems to be working for us.
>>>
>>> We haven''t tried creating a separate type though..
>>>
>>>>
>>>> For custom types and custom functions, puppetmasterd will only
>>>> look at the *global* value of modulepath, i.e the one you have
>>>> set under [puppetmasterd] or under [main].  The modulepath you
>>>> have set under each environment will not be used for those.
 You
>>>> didn''t have a global modulepath setting at all in
puppet.conf;
>>>> you need to add one to get plugins in modules to work *at*all*.
>>>>
>>>
>>> So it doesn''t look at "modulepath" under an
environment definition in the
>>> puppet.conf on the pupetmaster?
>>>
>>
>> It does for *delivering* to clients, but for validating the manifests
>> in order to turn them into a catalog, it looks at the local libdir for
>> the puppetmaster.
>>
>> As Thomas says in the next message that just arrived, functions are
>> different as they execute on the server, and types are different due
>> to parameter validation being employed.
>>
>> The problem I reported on puppet-dev with providers being pulled in is
>> only if you''re setting global resource defaults in your
manifests.
>>
>
> ahh.. ok.  Thanks,
This is all quite horrific to grok conceptually, and I''m not
pretending otherwise by the way.

The completely reasonable expectation is that functions, types and
providers are completely self-contained within an environment, but
that''s not the case.



-- 
nigel

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.