Puppet users - Jun 2012 - Is this possible with Hiera - Puppet Module Development and using Hiera

Hi Puppet Users,

I have just started playing with Puppet and Hiera.  I am curious if it is
possible to control how hiera does variable lookup within a module I am creating
to test it.

I have read through the documentation I can find on Hiera and Puppet however I
cannot find any documentation suggesting a way to it.

For the record these are the versions that I am using (taken from the Puppetlabs
Debian/Devel):
hiera (1.0.0-0.1rc3)
hiera-puppet (1.0.0-0.1rc1-1-g3e68ff0)
puppet-common (3.0.0-0.1rc3puppetlabs1)
puppet (3.0.0-0.1rc3puppetlabs1)

Essentially I would like to define a hiera.yaml file (just the hierarchy and
puppet datasource location) within my module and use it to clearly separate
Operating System differences to the default state.  For example:

------==== {modulepath}/hiera.yaml =======------
:hierarchy:
           - %{operatingsystem}
            - defaults

:puppet:
   :datasource: %{modulename}/config/
------===========------

So in my module directory I could have:
{modulepath}/config/debian.pp
{modulepath}/config/centos.pp
{modulepath}/config/defaults.pp

(It would be even better if in the file name somehow we could grip up similar OS
types, ie debian_ubuntu.pp)

I know that I can use puppet logic statements (case statements, if, ...) to set
variables per OS but I feel that it would be cleaner and clearer to separate the
OS specific stuff and it would be easier to update a module for a specific OS
and override something set in the defaults.pp file.

So is this possible or is this something that could be considered as a feature
request.

Thanks,

Peter.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Monday, 18 June 2012 13:02:33 UTC+2, Peter wrote:
>
> Hi Puppet Users, 
>
> I have just started playing with Puppet and Hiera.  I am curious if it is 
> possible to control how hiera does variable lookup within a module I am 
> creating to test it.
>
>  
>
> I have read through the documentation I can find on Hiera and Puppet 
> however I cannot find any documentation suggesting a way to it.
>
>  
>
> For the record these are the versions that I am using (taken from the 
> Puppetlabs Debian/Devel):
>
> hiera (1.0.0-0.1rc3)
>
> hiera-puppet (1.0.0-0.1rc1-1-g3e68ff0)
>
> puppet-common (3.0.0-0.1rc3puppetlabs1)
>
> puppet (3.0.0-0.1rc3puppetlabs1)
>
>  
>
> Essentially I would like to define a hiera.yaml file (just the hierarchy 
> and puppet datasource location) within my module and use it to clearly 
> separate Operating System differences to the default state.  For example:
>
>  
>
The hiera.yaml file is the global configuration for all paths, however it 
could contain a 

path of 
/etc/puppet/modules/%{module_name}/config/operatingsystem
/etc/puppet/modules/%{module_name}/config/defaults


 
> ------==== {modulepath}/hiera.yaml =======------
>
> :hierarchy: 
>
>            - %{operatingsystem}
>
>             - defaults
>
>  
>
> :puppet:
>
>    :datasource: %{modulename}/config/
>
> ------===========------
>
>  
>
> So in my module directory I could have:
>
> {modulepath}/config/debian.pp
>
> {modulepath}/config/centos.pp
>
> {modulepath}/config/defaults.pp
>
>  
>
> (It would be even better if in the file name somehow we could grip up 
> similar OS types, ie debian_ubuntu.pp)
>
>  
>
Look at the "osfamily" fact, this debian on ubuntu and redhat.
 
> I know that I can use puppet logic statements (case statements, if, …) to 
> set variables per OS but I feel that it would be cleaner and clearer to 
> separate the OS specific stuff and it would be easier to update a module 
> for a specific OS and override something set in the defaults.pp file.
>
>  
>
> So is this possible or is this something that could be considered as a 
> feature request.
>
>
So it''s possible but you can''t distribute the hiera.yaml with
your module
since this is a site setting.




 
>  
>
> Thanks,
>
>  
>
> Peter.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/dlrUWu1b8OsJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tuesday, 19 June 2012 03:26:14 UTC+10, Steve Traylen
wrote:
>
>
> On Monday, 18 June 2012 13:02:33 UTC+2, Peter wrote:
>>
>> Hi Puppet Users, 
>>
>> Essentially I would like to define a hiera.yaml file (just the
hierarchy
>> and puppet datasource location) within my module and use it to clearly 
>> separate Operating System differences to the default state.  For
example:
>>
> The hiera.yaml file is the global configuration for all paths, however it 
> could contain a path of 
> /etc/puppet/modules/%{module_name}/config/operatingsystem
> /etc/puppet/modules/%{module_name}/config/defaults
>
Hi Steve,

Thanks for the reply.

Although this will work it is not something I believe we should be aiming 
for.  I am hoping to eventually make some of the modules I am working on 
available on the Puppet Forge.

Sorry you may have miss-understood me, I was not talking about over-riding 
the hiera.yaml puppet file, I was talking about a version that describes 
how hiera should interact with the puppet module it is currently 
interacting with.

Using the solution you propose for module users it would we require 
additional work updating and maintaining hiera.yaml settings that could be 
non-trivial (what if a module uses a sub-directory data instead of config, 
or OSFamily instead of OS, or other filters).  Additionally what if the 
module developer expects a particular ordering of the files but the module 
users has something specific for there environment.

If we could implement a way for module developers to signal to hiera 
information on how to interact with values specific to modules it provides 
the best of both worlds:
*  Module Users can specify the order of hiera values and control the 
values they set; and
*  Module Develoers can specify the location of hiera values and what is 
the order is all without the module user having to work it out themselves.

Obviously in the puppet data extension to hiera it has been hardcoded to 
look for a data.pp file so it could be configured to re-use some of the 
other hiera code to interrogate a module hiera.yaml file.

Thanks,

Peter

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/Lq8aE-cfsMgJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tuesday, June 19, 2012 8:27:55 AM UTC-5, Peter wrote:
>
> On Tuesday, 19 June 2012 03:26:14 UTC+10, Steve Traylen wrote:
>>
>>
>> On Monday, 18 June 2012 13:02:33 UTC+2, Peter wrote:
>>>
>>> Hi Puppet Users, 
>>>
>>> Essentially I would like to define a hiera.yaml file (just the
hierarchy
>>> and puppet datasource location) within my module and use it to
clearly
>>> separate Operating System differences to the default state.  For
example:
>>>
>> The hiera.yaml file is the global configuration for all paths, however
it
>> could contain a path of 
>> /etc/puppet/modules/%{module_name}/config/operatingsystem
>> /etc/puppet/modules/%{module_name}/config/defaults
>>
>
> Hi Steve,
>
> Thanks for the reply.
>
> Although this will work it is not something I believe we should be aiming 
> for.  I am hoping to eventually make some of the modules I am working on 
> available on the Puppet Forge.
>
> Sorry you may have miss-understood me, I was not talking about over-riding 
> the hiera.yaml puppet file, I was talking about a version that describes 
> how hiera should interact with the puppet module it is currently 
> interacting with.
>
> Using the solution you propose for module users it would we require 
> additional work updating and maintaining hiera.yaml settings that could be 
> non-trivial (what if a module uses a sub-directory data instead of config, 
> or OSFamily instead of OS, or other filters).  Additionally what if the 
> module developer expects a particular ordering of the files but the module 
> users has something specific for there environment.
>
> If we could implement a way for module developers to signal to hiera 
> information on how to interact with values specific to modules it provides 
> the best of both worlds:
> *  Module Users can specify the order of hiera values and control the 
> values they set; and
> *  Module Develoers can specify the location of hiera values and what is 
> the order is all without the module user having to work it out themselves.
>
> Obviously in the puppet data extension to hiera it has been hardcoded to 
> look for a data.pp file so it could be configured to re-use some of the 
> other hiera code to interrogate a module hiera.yaml file.
>
I confess that I don''t follow what you are suggesting.  Hiera already 
supports module developers providing data, and module users overlaying 
their own on top, in the order of their choice.  What more would your 
suggestion provide in that area?  And why do module developers need to 
configure specific locations for (the root of) their data on the 
filesystem?  That should be under the control of module users (== the 
administrators of the master).  What am I missing?


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/MBqhQDrOOKgJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi Joh,

On Wednesday, 20 June 2012 03:18:46 UTC+10, jcbollinger
wrote:
>
>
> I confess that I don''t follow what you are suggesting.  Hiera
already
> supports module developers providing data, and module users overlaying 
> their own on top, in the order of their choice.  What more would your 
> suggestion provide in that area?  And why do module developers need to 
> configure specific locations for (the root of) their data on the 
> filesystem?  That should be under the control of module users (== the 
> administrators of the master).  What am I missing?
>
>
Thanks for taking the time to reply.

Yes hiera provides support for module developers ... however I would argue 
that it is limited for example, the hiera-puppet plugin is *hard-coded* to 
only look for values in the module manifest directory for a file called 
data.pp.

This does allow for a very low threshold for module developers to start 
implementing hiera in their modules (rename their existing params.pp files).

I believe by extending the hiera-puppet plugin for module developers to 
follow a similar convention as hiera for module users would make certain 
things easier for module developers.

By providing the ability to signal the hiera-puppet plugin (using a 
module-hiera.yaml file (for lack of a better name)) a module developer 
could reduce complexity and use the DRY principle to setup sane defaults 
(either in a defaults.pp or in a defaults.yaml file) and than use layer on 
top specific settings for Operating systems or even Hardware Types (as 
examples).

Sorry I don''t have time right now to provide something in depth but a
brief
example would be setting up ISCSI, the module I currently have has a 
complex params.pp file for detecting particular setups.

As a module developer I am interested in setting up a hierarchy such that 
from most specific to least specific:   
* Hardware_platform
* Operating System
* Defaults

By defining this in the module-hiera.yaml file it makes it very easy for me 
to signal to the module user what settings override which.

I am not suggesting taking any control from module users, and to be 
explicit user settings (if set) would override the module variable.

Hopefully I have done a better job of explaining what I am thinking.

Thanks,

Peter.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/hmxDrwmX6UMJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Fri, Jun 22, 2012 at 7:02 PM, Peter <peter@ifoley.id.au>
wrote:
>
>
> Thanks for taking the time to reply.
>
> Yes hiera provides support for module developers ... however I would argue
> that it is limited for example, the hiera-puppet plugin is *hard-coded* to
> only look for values in the module manifest directory for a file called
> data.pp.
>
> This does allow for a very low threshold for module developers to start
> implementing hiera in their modules (rename their existing params.pp
files).
>
> I believe by extending the hiera-puppet plugin for module developers to
> follow a similar convention as hiera for module users would make certain
> things easier for module developers.
>
> By providing the ability to signal the hiera-puppet plugin (using a
> module-hiera.yaml file (for lack of a better name)) a module developer
> could reduce complexity and use the DRY principle to setup sane defaults
> (either in a defaults.pp or in a defaults.yaml file) and than use layer on
> top specific settings for Operating systems or even Hardware Types (as
> examples).
>
> Sorry I don''t have time right now to provide something in depth
but a
> brief example would be setting up ISCSI, the module I currently have has a
> complex params.pp file for detecting particular setups.
>
> As a module developer I am interested in setting up a hierarchy such that
> from most specific to least specific:
> * Hardware_platform
> * Operating System
> * Defaults
>
> By defining this in the module-hiera.yaml file it makes it very easy for
> me to signal to the module user what settings override which.
>
> I am not suggesting taking any control from module users, and to be
> explicit user settings (if set) would override the module variable.
>
> Hopefully I have done a better job of explaining what I am thinking.
>
> Thanks,
>
> Peter.
>
If I understand your idea correctly then I too would like to see this.  We
would be able to use a hierarchy within the module (when sharing on the
forge) to provide various defaults without having to do a bunch of
selectors on $operatingsystem etc and make it easier for someone to start
providing overrides from their own hiera data.  Right now to provide
defaults that differ for an operating system you would have to do a bunch
of logic in params.pp that the person would want to strip out and replace
with a call to hiera anyway.  By allowing us to create params.pp as a
series of hiera calls in the first place it''ll reduce the amount of
modifications (hopefully to zero) when taking a module off the forge.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Friday, June 22, 2012 6:02:33 PM UTC-5, Peter wrote:
>
> I believe by extending the hiera-puppet plugin for module developers to 
> follow a similar convention as hiera for module users would make certain 
> things easier for module developers.
>
> By providing the ability to signal the hiera-puppet plugin (using a 
> module-hiera.yaml file (for lack of a better name)) a module developer 
> could reduce complexity and use the DRY principle to setup sane defaults 
> (either in a defaults.pp or in a defaults.yaml file) and than use layer on 
> top specific settings for Operating systems or even Hardware Types (as 
> examples).
>
What I''m getting is that the central idea you propose is to give
Hiera''s
Puppet plugin the ability to collect data not only from different 
namespaces, as it does now, but also from different classes in each 
namespace.  I also get the impression that you are reserving use of the 
YAML backend to module users.

I''m not prepared to discount your proposal, but I do think you are 
artificially limiting yourself if as a module developer you refuse to use 
the YAML backend.  If you want more flexibility *today* then that''s
where
you should look.
 

John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/9rtwA8ZgbQAJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Friday, June 22, 2012 7:02:33 PM UTC-4, Peter wrote:
>
> Hi Joh,
>
> On Wednesday, 20 June 2012 03:18:46 UTC+10, jcbollinger wrote:
>>
>>
>> I confess that I don''t follow what you are suggesting.  Hiera
already
>> supports module developers providing data, and module users overlaying 
>> their own on top, in the order of their choice.  What more would your 
>> suggestion provide in that area?  And why do module developers need to 
>> configure specific locations for (the root of) their data on the 
>> filesystem?  That should be under the control of module users (== the 
>> administrators of the master).  What am I missing?
>>
>>
> Thanks for taking the time to reply.
>
> Yes hiera provides support for module developers ... however I would argue 
> that it is limited for example, the hiera-puppet plugin is *hard-coded* to 
> only look for values in the module manifest directory for a file called 
> data.pp.
>
> This does allow for a very low threshold for module developers to start 
> implementing hiera in their modules (rename their existing params.pp
files).
>
> I believe by extending the hiera-puppet plugin for module developers to 
> follow a similar convention as hiera for module users would make certain 
> things easier for module developers.
>
> By providing the ability to signal the hiera-puppet plugin (using a 
> module-hiera.yaml file (for lack of a better name)) a module developer 
> could reduce complexity and use the DRY principle to setup sane defaults 
> (either in a defaults.pp or in a defaults.yaml file) and than use layer on 
> top specific settings for Operating systems or even Hardware Types (as 
> examples).
>
> Sorry I don''t have time right now to provide something in depth
but a
> brief example would be setting up ISCSI, the module I currently have has a 
> complex params.pp file for detecting particular setups.
>
> As a module developer I am interested in setting up a hierarchy such that 
> from most specific to least specific:   
> * Hardware_platform
> * Operating System
> * Defaults
>
> By defining this in the module-hiera.yaml file it makes it very easy for 
> me to signal to the module user what settings override which.
>
> I am not suggesting taking any control from module users, and to be 
> explicit user settings (if set) would override the module variable.
>
> Hopefully I have done a better job of explaining what I am thinking.
>
> Thanks,
>
> Peter.
>
I like where this idea is going. You would like a backend, hiera-puppet for 
now, to be able to lookup "default" data from the current module
directory
during parse time. Let''s open a ticket (feature request) for this and 
workout the details.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/Ib1O42g43JYJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.