thr3ads.net - Puppet users - [Puppet Users] Puppet 3.0 and Hiera [May 2012]

If this information is useful, please help other people find it:
Share via:

Pieter van de Bruggen

2012-May-03 17:05 UTC

[Puppet Users] Puppet 3.0 and Hiera

Greetings,

As many of you may be aware, Hiera will be tightly integrated into Puppet
in the upcoming release of Puppet 3.0. As a final sanity check of this
work, I’d like to open our current plan for integration up for feedback.
This is particularly for feedback from existing Hiera users, but I hope to
solicit good feedback from those not using Hiera as well.

The problem, as it currently exists, is that Puppet (core) has no good
first-class mechanism for separating configuration data from manifests.
Everything from $faked_namespaces__in__variable_names to specialty “params”
classes have been tried, with varying degrees of success.

Hiera came along as another solution to this problem, and provided a useful
abstraction for hierarchical data lookup, but life as a plugin meant that
certain integrations were still difficult. Our aim in Puppet 3.0 is to
tighten up these integrations (making Hiera more useful), provide
first-class data separation solution (for those not already using Hiera),
and to provide a simple and safe migration for those currently using Hiera.

Here’s what’s new:

   - Hiera data keys can be namespaced
      - (e.g. ''dns::nameserver'': ''8.8.8.8'')
   - Namespaced data will automatically populate class parameters
      - (e.g. ''dns::nameserver'' will be automatically looked
up for the
      $nameserver parameter when you include dns)

Here’s what’s changed:

   - The hiera-puppet module will no longer be required
      - It should, however, still continue to work
   - The heira() function (from the hiera-puppet module) will be superceded
   by the lookup() function (in Puppet core)
   - hiera_include() will not be ported; include now properly accept
   arrays, making this redundant

Here’s what were still wondering about:

   - How should we integrate hiera_array() and hiera_hash()?
   - How should we integrate hiera’s “default” and “override” parameters?
   - How should we handle overlaps between data supplied by Hiera and data
   supplied in a parameterized class include?

If you’re interested in test-driving the new functionality for yourself,
checkout out the master branch of the Puppet repository on
Github<https://github.com/puppetlabs/puppet>,
install Hiera (with gem install hiera) and make the following configuration
changes:

   - Set data_binding_terminus to hiera
   - If you have a Hiera configuration file, set hiera_config to point to
   it.
   - If you don’t, create a file in ${confdir}/hiera.yaml with these
   contents:

     :backend:

       - yaml

     :yaml:

       :datadir: $confdir/data

     :hierarchy:

       - %{certname}

       - %{environment}

       - global

This sets up a default hierarchy that looks for values in ${confdir}/data,
first in the ${certname}.yml file, then in the ${environment}.yml file,
then in global.yml.

That should be it!  Please, let us know if you''re having trouble
getting
started, or if you have questions, concerns, thoughts, or opinions about
any of this.

Thanks!

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Craig Dunn

2012-May-04 08:40 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

All sounds very good...
> Here’s what’s new:
>
>   * Hiera data keys can be namespaced
>       o (e.g. ''dns::nameserver'':
''8.8.8.8'')
Namespace scoping is defiantly a massive improvement to Hiera - Im 
curious how this affects back ends though... Will lookup("nameserver")
called from Puppet in a class called DNS be always mapped to 
dns::nameserver and that value passed as the lookup key to Hiera and on 
to whatever back end you''re using?  If so is that going to make
existing
Hiera setups incompatible?
>   * Namespaced data will automatically populate class parameters
>       o (e.g. ''dns::nameserver'' will be automatically
looked up for the
>         $nameserver parameter when you include dns)
I currently try and achieve this with using blah/%{calling_module} in my 
hierarchy and therefore class dns ( $nameserver=hiera("nameserver" ) 
will look for nameserver: in blah/dns.yaml which also adds some nice 
grouping/separation for setups with many Hiera enabled modules - will 
the use of %{calling_module} still be supportable?

Also - is there any indication that tying a lookup to a particular back 
end may be supported at some point? 
(http://projects.puppetlabs.com/issues/12951)

Sounds like a lot of great new functionality though, looking forward to 
testing it out!

Craig


-- 
Craig Dunn | http://www.craigdunn.org
Yahoo/Skype: craigrdunn | Twitter: @crayfishX

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

R.I.Pienaar

2012-May-04 09:05 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

----- Original Message -----> From: "Pieter van de Bruggen" <pieter@puppetlabs.com>
<snip>
> 
>     * How should we integrate hiera_array() and hiera_hash() ?
>     * How should we integrate hiera ’s “default” and “override”
>     parameters?
>     * How should we handle overlaps between data supplied by Hiera
>     and data supplied in a parameterized class include?
Given:

class foo($something="default") { }

I think the plan was that there would be a priority order as below:

 - someone wrote in a manifest: class{"foo": something =>
something}
 - an ENC supplied the values for something on the class foo
 - someone did "include foo" or class{"foo": } this would
consult hiera
 - if hiera does not have an answer it would default to "default"

This gives people with more complex needs than those matched by hiera
the ability to use ENCs and get exactly the data model they desire as
well as people who do not want any magical data lookup for some class
the ability to hard code values.

And at the same time it lets module authors provide out of the box
defaults that work without placing a load on module users where they
might have to read every class and set hiera values for every argument
before they can use a class if all they wanted was the default out of
the box behaviour.

Does this sound sane?
> If you’re interested in test-driving the new functionality for
> yourself, checkout out the master branch of the Puppet repository on
> Github , install Hiera (with gem install hiera ) and make the
> following configuration changes:
> 
>     * Set data_binding_terminus to hiera
>     * If you have a Hiera configuration file, set hiera_config to
>     point to it.
>     * If you don’t, create a file in ${confdir}/hiera.yaml with these
>     contents:
> 
> 
> :backend:
> 
> - yaml
> 
> :yaml:
> 
> :datadir: $confdir/data
> 
> :hierarchy:
> 
> - %{certname}
> 
> - %{environment}
> 
> - global
> 
> This sets up a default hierarchy that looks for values in
> ${confdir}/data , first in the ${certname}.yml file, then in the
> ${environment}.yml file, then in global.yml .
These should probably %{::certname} etc right?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Ashley Penney

2012-May-04 12:47 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On Fri, May 4, 2012 at 5:05 AM, R.I.Pienaar <rip@devco.net> wrote:
>
> I think the plan was that there would be a priority order as below:
>
>  - someone wrote in a manifest: class{"foo": something =>
something}
>  - an ENC supplied the values for something on the class foo
>  - someone did "include foo" or class{"foo": } this
would consult hiera
>  - if hiera does not have an answer it would default to "default"
>
> This gives people with more complex needs than those matched by hiera
> the ability to use ENCs and get exactly the data model they desire as
> well as people who do not want any magical data lookup for some class
> the ability to hard code values.
>
> And at the same time it lets module authors provide out of the box
> defaults that work without placing a load on module users where they
> might have to read every class and set hiera values for every argument
> before they can use a class if all they wanted was the default out of
> the box behaviour.
>
> Does this sound sane?

I can''t speak for anyone else but this sounds fantastic to me. 
I''ve already
ran into this problem where I want a bunch of defaults to be used unless
they need to specifically overwrite them in Hiera and I don''t always
want
to add those into Hiera (because of the lack of namespacing it can get
messy).  All of the changes mentioned so far will be of very big benefit
and really improve things. :)

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Josh

2012-May-04 13:00 UTC

head link

[Puppet Users] Re: Puppet 3.0 and Hiera

I''m not sure if its in there but one thing that would be *amazing*
would be
support for empty arrays/hashes. E.g.

$hash1 = hiera_hash(''hash1'', {})
$hash1 = hiera_hash(''hash2'', {})

This means that now I can do a:

merge_hash($hash1, $hash2)

This will now work regardless of whether I have declared hash1 and hash2 
for a given node or not, where as at the moment if you don''t declare
both
of them in hiera the merge fails because the empty hash is seen as a string.

Native support for the merging would be handy as well.

Josh

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/bmYUG6203roJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Jeremy Carroll

2012-May-04 16:19 UTC

head link

[Puppet Users] Re: Puppet 3.0 and Hiera

We use Hiera backed Puppet here at Klout. Mainly for the reasons 
of separating configuration data from puppet modules. We have had great 
success, and I welcome this change to puppet core. Reminds me of some of 
Chef''s "Deep Merge" functionality. Here are a few of my
thoughts.

   - Name spacing variable names is a welcome improvement. It will help 
   with variable name collisions we have experienced (Ex: module1::hosts, 
   module2::hosts). We currently use the %{calling_class},
%{calling_module}method of scoping hiera variables.
   - I currently like the flexibility of defining my own order for merging 
   of variables in hiera.yaml. But I would like Puppet to formalize the merge 
   order so everybody can use the same style when creating modules for reuse. 
   Also have some methods to specify overrides instead of merging from top to 
   bottom for hashes / arrays. Helps for configurations for systems that
don''t
   want to inherit everything. Chef has a lot of work for Deep Merging 
   ordering that I believe Hiera would benefit from.
   - hiera_hash would be improved by sorting the hash natively. We 
   currently repeat this pattern every time we use a hiera hash. We create a 
   sorted array of each hash key, then iterate over the array to lookup values 
   in the hash. If we fail to do this, puppet run re-orders the file every run 
   as hash orders are not guaranteed. This causes a lot of un-necessary work 
   and change notifications. Example below.

*hieradata*
> --- 
hiera_var:>   user:
>     value:  ''my_value''
*module/params.pp*
> $hiera_var   = hiera_hash(''hiera_var'')
> $sorted_var  = sort(keys($hiera_var))
*module/template/sometemplate.pp*
> <% scope.lookupvar(''module::params::sorted_var'').each
do |item| -%><%
> scope.lookupvar(''hadoop::params::hiera_var'')[item].each
do |key,value| -%>
> <%= user %> <%= key %> <%= value %>
> <% end -%><% end -%>

On Thursday, May 3, 2012 10:05:14 AM UTC-7, Pieter van de Bruggen
wrote:>
> Greetings,
>
> As many of you may be aware, Hiera will be tightly integrated into Puppet 
> in the upcoming release of Puppet 3.0. As a final sanity check of this 
> work, I’d like to open our current plan for integration up for feedback. 
> This is particularly for feedback from existing Hiera users, but I hope to 
> solicit good feedback from those not using Hiera as well.
>
> The problem, as it currently exists, is that Puppet (core) has no good 
> first-class mechanism for separating configuration data from manifests. 
> Everything from $faked_namespaces__in__variable_names to specialty 
> “params” classes have been tried, with varying degrees of success.
>
> Hiera came along as another solution to this problem, and provided a 
> useful abstraction for hierarchical data lookup, but life as a plugin meant
> that certain integrations were still difficult. Our aim in Puppet 3.0 is to
> tighten up these integrations (making Hiera more useful), provide 
> first-class data separation solution (for those not already using Hiera), 
> and to provide a simple and safe migration for those currently using Hiera.
>
> Here’s what’s new:
>
>    - Hiera data keys can be namespaced 
>       - (e.g. ''dns::nameserver'':
''8.8.8.8'')
>    - Namespaced data will automatically populate class parameters 
>       - (e.g. ''dns::nameserver'' will be automatically
looked up for the
>       $nameserver parameter when you include dns) 
>    
> Here’s what’s changed:
>
>    - The hiera-puppet module will no longer be required 
>       - It should, however, still continue to work 
>    - The heira() function (from the hiera-puppet module) will be 
>    superceded by the lookup() function (in Puppet core) 
>    - hiera_include() will not be ported; include now properly accept 
>    arrays, making this redundant 
>
> Here’s what were still wondering about:
>
>    - How should we integrate hiera_array() and hiera_hash()? 
>    - How should we integrate hiera’s “default” and “override” parameters? 
>    - How should we handle overlaps between data supplied by Hiera and 
>    data supplied in a parameterized class include? 
>
> If you’re interested in test-driving the new functionality for yourself, 
> checkout out the master branch of the Puppet repository on
Github<https://github.com/puppetlabs/puppet>,
> install Hiera (with gem install hiera) and make the following 
> configuration changes:
>
>    - Set data_binding_terminus to hiera 
>    - If you have a Hiera configuration file, set hiera_config to point to 
>    it. 
>    - If you don’t, create a file in ${confdir}/hiera.yaml with these 
>    contents: 
>
>      :backend:
>
>        - yaml
>
>      :yaml:
>
>        :datadir: $confdir/data
>
>      :hierarchy:
>
>        - %{certname}
>
>        - %{environment}
>
>        - global
>
> This sets up a default hierarchy that looks for values in ${confdir}/data, 
> first in the ${certname}.yml file, then in the ${environment}.yml file, 
> then in global.yml.
>
> That should be it!  Please, let us know if you''re having trouble
getting
> started, or if you have questions, concerns, thoughts, or opinions about 
> any of this.
>
> Thanks!
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/GxVliXSX5u0J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

R.I.Pienaar

2012-May-04 16:27 UTC

head link

Re: [Puppet Users] Re: Puppet 3.0 and Hiera

----- Original Message -----> From: "Jeremy Carroll" <jeremy@klout.com>
> To: puppet-users@googlegroups.com
> Sent: Friday, May 4, 2012 5:19:37 PM
> Subject: [Puppet Users] Re: Puppet 3.0 and Hiera
> 
> We use Hiera backed Puppet here at Klout. Mainly for the reasons of
> separating configuration data from puppet modules. We have had great
> success, and I welcome this change to puppet core. Reminds me of
> some of Chef''s "Deep Merge" functionality. Here are a
few of my
> thoughts.
> 
> 
>     * Name spacing variable names is a welcome improvement. It will
>     help with variable name collisions we have experienced (Ex:
>     module1::hosts, module2::hosts). We currently use the
>     %{calling_class}, %{calling_module} method of scoping hiera
>     variables.
>     * I currently like the flexibility of defining my own order for
>     merging of variables in hiera.yaml. But I would like Puppet to
>     formalize the merge order so everybody can use the same style
>     when creating modules for reuse. Also have some methods to
>     specify overrides instead of merging from top to bottom for
>     hashes / arrays. Helps for configurations for systems that
don''t
>     want to inherit everything. Chef has a lot of work for Deep
>     Merging ordering that I believe Hiera would benefit from.
Do you mean you want the hierarchy to be standardised? 

can you expand this with examples please?

>     * hiera_hash would be improved by sorting the hash natively. We
>     currently repeat this pattern every time we use a hiera hash. We
>     create a sorted array of each hash key, then iterate over the
>     array to lookup values in the hash. If we fail to do this,
>     puppet run re-orders the file every run as hash orders are not
>     guaranteed. This causes a lot of un-necessary work and change
>     notifications. Example below.
Unfortunately ruby hashes has no order so we cant sort the hashes for you :(

Things are improving in 1.9, but I do not think there is much we can do there
> 
> 
> hieradata
> 
> 
> ---
> 
> hiera_var:
> user:
> value: ''my_value''
> module/params.pp
> 
> $hiera_var = hiera_hash(''hiera_var'')
> $sorted_var = sort(keys($hiera_var))
> module/template/sometemplate.pp
> 
> 
> <% scope.lookupvar(''module::params::sorted_var'').each
do |item| -%><%
> scope.lookupvar(''hadoop::params::hiera_var'')[item].each
do
> |key,value| -%>
> <%= user %> <%= key %> <%= value %>
> <% end -%><% end -%>
> On Thursday, May 3, 2012 10:05:14 AM UTC-7, Pieter van de Bruggen
> wrote:
> 
> 
> 
> 
> 
> Greetings,
> 
> As many of you may be aware, Hiera will be tightly integrated into
> Puppet in the upcoming release of Puppet 3.0. As a final sanity
> check of this work, I’d like to open our current plan for
> integration up for feedback. This is particularly for feedback from
> existing Hiera users, but I hope to solicit good feedback from those
> not using Hiera as well.
> 
> The problem, as it currently exists, is that Puppet (core) has no
> good first-class mechanism for separating configuration data from
> manifests. Everything from $faked_namespaces__in__variable_names to
> specialty “params” classes have been tried, with varying degrees of
> success.
> 
> Hiera came along as another solution to this problem, and provided a
> useful abstraction for hierarchical data lookup, but life as a
> plugin meant that certain integrations were still difficult. Our aim
> in Puppet 3.0 is to tighten up these integrations (making Hiera more
> useful), provide first-class data separation solution (for those not
> already using Hiera), and to provide a simple and safe migration for
> those currently using Hiera.
> 
> Here’s what’s new:
> 
>     * Hiera data keys can be namespaced
> 
> 
>         * (e.g. ''dns::nameserver'':
''8.8.8.8'' )
>     * Namespaced data will automatically populate class parameters
> 
>         * (e.g. ''dns::nameserver'' will be automatically
looked up for
>         the $nameserver parameter when you include dns )
> 
> 
> Here’s what’s changed:
> 
>     * The hiera-puppet module will no longer be required
> 
> 
>         * It should, however, still continue to work
>     * The heira() function (from the hiera-puppet module) will be
>     superceded by the lookup() function (in Puppet core)
>     * hiera_include() will not be ported; include now properly accept
>     arrays, making this redundant
> 
> 
> Here’s what were still wondering about:
> 
>     * How should we integrate hiera_array() and hiera_hash() ?
>     * How should we integrate hiera ’s “default” and “override”
>     parameters?
>     * How should we handle overlaps between data supplied by Hiera
>     and data supplied in a parameterized class include?
> 
> 
> If you’re interested in test-driving the new functionality for
> yourself, checkout out the master branch of the Puppet repository on
> Github , install Hiera (with gem install hiera ) and make the
> following configuration changes:
> 
>     * Set data_binding_terminus to hiera
>     * If you have a Hiera configuration file, set hiera_config to
>     point to it.
>     * If you don’t, create a file in ${confdir}/hiera.yaml with these
>     contents:
> 
> 
> :backend:
> 
> - yaml
> 
> :yaml:
> 
> :datadir: $confdir/data
> 
> :hierarchy:
> 
> - %{certname}
> 
> - %{environment}
> 
> - global
> 
> This sets up a default hierarchy that looks for values in
> ${confdir}/data , first in the ${certname}.yml file, then in the
> ${environment}.yml file, then in global.yml .
> 
> That should be it! Please, let us know if you''re having trouble
> getting started, or if you have questions, concerns, thoughts, or
> opinions about any of this.
> 
> Thanks!
> 
> 
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/GxVliXSX5u0J .
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Jeremy Carroll

2012-May-04 16:29 UTC

head link

[Puppet Users] Re: Puppet 3.0 and Hiera

We use Hiera backed Puppet here at Klout. Mainly for the reasons 
of separating configuration data from puppet modules. We have had great 
success, and I welcome this change to puppet core. Reminds me of some of 
Chef''s "Deep Merge" functionality. Here are a few of my
thoughts.

   - Name spacing variable names is a welcome improvement. It will help 
   with variable name collisions we have experienced (Ex: module1::hosts, 
   module2::hosts). We currently use the %{calling_class}, %{calling_module}
method
   of scoping hiera variables.
   - I currently like the flexibility of defining my own order for merging 
   of variables in hiera.yaml. But I would like Puppet to formalize the merge 
   order so everybody can use the same style when creating modules for reuse. 
   Also have some methods to specify overrides instead of merging from top to 
   bottom for hashes / arrays. Helps for configurations for systems that
don''t
   want to inherit everything.  Something like this.
      - overrides
      - hieradata
      - module default params
   - hiera_hash would be improved by sorting the hash natively. We 
   currently repeat this pattern every time we use a hiera hash. We create a 
   sorted array of each hash key, then iterate over the array to lookup values 
   in the hash. If we fail to do this, puppet run re-orders the file every run 
   as hash orders are not guaranteed. This causes a lot of un-necessary work 
   and change notifications. Example below.

*hieradata*
> --- 
hiera_var:>   user:
>     value:  ''my_value''
*module/params.pp*
> $hiera_var   = hiera_hash(''hiera_var'')
> $sorted_var  = sort(keys($hiera_var))
*module/template/sometemplate.pp*
> <% scope.lookupvar(''module::params::sorted_var'').each
do |item| -%><%
> scope.lookupvar(''module::params::hiera_var'')[item].each
do |key,value| -%>
> <%= user %> <%= key %> <%= value %>
> <% end -%><% end -%>

On Thursday, May 3, 2012 10:05:14 AM UTC-7, Pieter van de Bruggen
wrote:>
> Greetings,
>
> As many of you may be aware, Hiera will be tightly integrated into Puppet 
> in the upcoming release of Puppet 3.0. As a final sanity check of this 
> work, I’d like to open our current plan for integration up for feedback. 
> This is particularly for feedback from existing Hiera users, but I hope to 
> solicit good feedback from those not using Hiera as well.
>
> The problem, as it currently exists, is that Puppet (core) has no good 
> first-class mechanism for separating configuration data from manifests. 
> Everything from $faked_namespaces__in__variable_names to specialty 
> “params” classes have been tried, with varying degrees of success.
>
> Hiera came along as another solution to this problem, and provided a 
> useful abstraction for hierarchical data lookup, but life as a plugin meant
> that certain integrations were still difficult. Our aim in Puppet 3.0 is to
> tighten up these integrations (making Hiera more useful), provide 
> first-class data separation solution (for those not already using Hiera), 
> and to provide a simple and safe migration for those currently using Hiera.
>
> Here’s what’s new:
>
>    - Hiera data keys can be namespaced 
>       - (e.g. ''dns::nameserver'':
''8.8.8.8'')
>    - Namespaced data will automatically populate class parameters 
>       - (e.g. ''dns::nameserver'' will be automatically
looked up for the
>       $nameserver parameter when you include dns) 
>    
> Here’s what’s changed:
>
>    - The hiera-puppet module will no longer be required 
>       - It should, however, still continue to work 
>    - The heira() function (from the hiera-puppet module) will be 
>    superceded by the lookup() function (in Puppet core) 
>    - hiera_include() will not be ported; include now properly accept 
>    arrays, making this redundant 
>
> Here’s what were still wondering about:
>
>    - How should we integrate hiera_array() and hiera_hash()? 
>    - How should we integrate hiera’s “default” and “override” parameters? 
>    - How should we handle overlaps between data supplied by Hiera and 
>    data supplied in a parameterized class include? 
>
> If you’re interested in test-driving the new functionality for yourself, 
> checkout out the master branch of the Puppet repository on
Github<https://github.com/puppetlabs/puppet>,
> install Hiera (with gem install hiera) and make the following 
> configuration changes:
>
>    - Set data_binding_terminus to hiera 
>    - If you have a Hiera configuration file, set hiera_config to point to 
>    it. 
>    - If you don’t, create a file in ${confdir}/hiera.yaml with these 
>    contents: 
>
>      :backend:
>
>        - yaml
>
>      :yaml:
>
>        :datadir: $confdir/data
>
>      :hierarchy:
>
>        - %{certname}
>
>        - %{environment}
>
>        - global
>
> This sets up a default hierarchy that looks for values in ${confdir}/data, 
> first in the ${certname}.yml file, then in the ${environment}.yml file, 
> then in global.yml.
>
> That should be it!  Please, let us know if you''re having trouble
getting
> started, or if you have questions, concerns, thoughts, or opinions about 
> any of this.
>
> Thanks!
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/JUwQTdQzdlMJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Ohad Levy

2012-May-04 19:05 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On Fri, May 4, 2012 at 12:05 PM, R.I.Pienaar <rip@devco.net> wrote:
>
>
> ----- Original Message -----
> > From: "Pieter van de Bruggen" <pieter@puppetlabs.com>
>
> <snip>
>
> >
> >     * How should we integrate hiera_array() and hiera_hash() ?
> >     * How should we integrate hiera ’s “default” and “override”
> >     parameters?
> >     * How should we handle overlaps between data supplied by Hiera
> >     and data supplied in a parameterized class include?
>
> Given:
>
> class foo($something="default") { }
>
> I think the plan was that there would be a priority order as below:
>
>  - someone wrote in a manifest: class{"foo": something =>
something}
>  - an ENC supplied the values for something on the class foo
>  - someone did "include foo" or class{"foo": } this
would consult hiera
>  - if hiera does not have an answer it would default to "default"
>
 Would be possible to define which function is called in this case? by
default that should be heira, but in case someone wants something else, I
think we should allow that?

Ohad

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Daniel Pittman

2012-May-04 20:01 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On Fri, May 4, 2012 at 12:05 PM, Ohad Levy <ohadlevy@gmail.com>
wrote:> On Fri, May 4, 2012 at 12:05 PM, R.I.Pienaar <rip@devco.net> wrote:
>> ----- Original Message -----
>> > From: "Pieter van de Bruggen"
<pieter@puppetlabs.com>
>>
>> <snip>
>>
>> >
>> >     * How should we integrate hiera_array() and hiera_hash() ?
>> >     * How should we integrate hiera ’s “default” and “override”
>> >     parameters?
>> >     * How should we handle overlaps between data supplied by Hiera
>> >     and data supplied in a parameterized class include?
>>
>> Given:
>>
>> class foo($something="default") { }
>>
>> I think the plan was that there would be a priority order as below:
>>
>>  - someone wrote in a manifest: class{"foo": something =>
something}
>>  - an ENC supplied the values for something on the class foo
>>  - someone did "include foo" or class{"foo": } this
would consult hiera
>>  - if hiera does not have an answer it would default to
"default"
>
>  Would be possible to define which function is called in this case? by
> default that should be heira, but in case someone wants something else, I
> think we should allow that?
The implementation of the automatic lookup is through an indirection
and terminus - so you can (theoretically) run with `none` as the
back-end, and you can supply your own Ruby code that implements the
lookup in some other way.

-- 
Daniel Pittman
⎋ Puppet Labs Developer – http://puppetlabs.com
♲ Made with 100 percent post-consumer electrons

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Trevor Vaughan

2012-May-05 16:26 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

With the integration of Hiera, is the extlookup functionality being deprecated?

If so, I would like to ask that the team add a Hiera backend for the existing
extlookup data sources.

I know that it won''t support hashes but it would help to break the view
that Puppet routinely breaks existing functionality on upgrades.

Thanks,

Trevor

On 05/03/2012 01:05 PM, Pieter van de Bruggen wrote:> Greetings,
> 
> As many of you may be aware, Hiera will be tightly integrated into Puppet
> in the upcoming release of Puppet 3.0. As a final sanity check of this
> work, I’d like to open our current plan for integration up for feedback.
> This is particularly for feedback from existing Hiera users, but I hope to
> solicit good feedback from those not using Hiera as well.
> 
> The problem, as it currently exists, is that Puppet (core) has no good
> first-class mechanism for separating configuration data from manifests.
> Everything from $faked_namespaces__in__variable_names to specialty “params”
> classes have been tried, with varying degrees of success.
> 
> Hiera came along as another solution to this problem, and provided a useful
> abstraction for hierarchical data lookup, but life as a plugin meant that
> certain integrations were still difficult. Our aim in Puppet 3.0 is to
> tighten up these integrations (making Hiera more useful), provide
> first-class data separation solution (for those not already using Hiera),
> and to provide a simple and safe migration for those currently using Hiera.
> 
> Here’s what’s new:
> 
>    - Hiera data keys can be namespaced
>       - (e.g. ''dns::nameserver'':
''8.8.8.8'')
>    - Namespaced data will automatically populate class parameters
>       - (e.g. ''dns::nameserver'' will be automatically
looked up for the
>       $nameserver parameter when you include dns)
> 
> Here’s what’s changed:
> 
>    - The hiera-puppet module will no longer be required
>       - It should, however, still continue to work
>    - The heira() function (from the hiera-puppet module) will be superceded
>    by the lookup() function (in Puppet core)
>    - hiera_include() will not be ported; include now properly accept
>    arrays, making this redundant
> 
> Here’s what were still wondering about:
> 
>    - How should we integrate hiera_array() and hiera_hash()?
>    - How should we integrate hiera’s “default” and “override” parameters?
>    - How should we handle overlaps between data supplied by Hiera and data
>    supplied in a parameterized class include?
> 
> If you’re interested in test-driving the new functionality for yourself,
> checkout out the master branch of the Puppet repository on
> Github<https://github.com/puppetlabs/puppet>,
> install Hiera (with gem install hiera) and make the following configuration
> changes:
> 
>    - Set data_binding_terminus to hiera
>    - If you have a Hiera configuration file, set hiera_config to point to
>    it.
>    - If you don’t, create a file in ${confdir}/hiera.yaml with these
>    contents:
> 
>      :backend:
> 
>        - yaml
> 
>      :yaml:
> 
>        :datadir: $confdir/data
> 
>      :hierarchy:
> 
>        - %{certname}
> 
>        - %{environment}
> 
>        - global
> 
> This sets up a default hierarchy that looks for values in ${confdir}/data,
> first in the ${certname}.yml file, then in the ${environment}.yml file,
> then in global.yml.
> 
> That should be it!  Please, let us know if you''re having trouble
getting
> started, or if you have questions, concerns, thoughts, or opinions about
> any of this.
> 
> Thanks!
> 
- -- 
Trevor Vaughan
 Vice President, Onyx Point, Inc.
 email: tvaughan@onyxpoint.com
 phone: 410-541-ONYX (6699)
 pgp: 0x6C701E94

- -- This account not approved for unencrypted sensitive information --
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQEcBAEBAgAGBQJPpVScAAoJECNCGV1OLcypcesIAJT9gARlmQNxR7ftUo4soR1r
R2aCh/k2B+Wylj/xfF3o9j3gZh/1WrOLogMGThNgYT9Vcv50zCcbeFjbCTVr2bBo
18gtlcoJhTBeUebBk6CsFsG+sIApwja3UDbpSm49FRquCK/KI+fBV8gHhAQt9fgm
1P0ZXB1KU61nhqrLEpHNRmJVKBvLCwDuqEMJltMuGoysczWEv0uVROtYari1T3g+
tNqov2EkSj5NV+Au5DeNNKviToEDn+wR7JvtXEqRdwSc142U9PnZwp8MvM+/i2+W
RGKqg8HZuMMLEbbSnimdRnS6pCwKdgVJDPwBb+CoiZJJ0ftWCF2F9lNDjgUNjcI=/h12
-----END PGP SIGNATURE-----

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

R.I.Pienaar

2012-May-06 10:26 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On 5 May 2012, at 17:26, Trevor Vaughan <tvaughan@onyxpoint.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> With the integration of Hiera, is the extlookup functionality being
deprecated?
> 
> If so, I would like to ask that the team add a Hiera backend for the
existing extlookup data sources.
> 
> I know that it won''t support hashes but it would help to break the
view that Puppet routinely breaks existing functionality on upgrades.

The hiera-puppet project provides a script to convert extlookup CSV to json or
yaml, using that to migrate would be my preferred way with the extlookup
function logging deprecation warnings

If this reflects the actual plan I don''t know, I did expect the people
working on the merging project to engage on this thread more since they know
best and there seems to be a few points for discussion here already that needs
clarification

> 
> Thanks,
> 
> Trevor
> 
> On 05/03/2012 01:05 PM, Pieter van de Bruggen wrote:
>> Greetings,
>> 
>> As many of you may be aware, Hiera will be tightly integrated into
Puppet
>> in the upcoming release of Puppet 3.0. As a final sanity check of this
>> work, I’d like to open our current plan for integration up for
feedback.
>> This is particularly for feedback from existing Hiera users, but I hope
to
>> solicit good feedback from those not using Hiera as well.
>> 
>> The problem, as it currently exists, is that Puppet (core) has no good
>> first-class mechanism for separating configuration data from manifests.
>> Everything from $faked_namespaces__in__variable_names to specialty
“params”
>> classes have been tried, with varying degrees of success.
>> 
>> Hiera came along as another solution to this problem, and provided a
useful
>> abstraction for hierarchical data lookup, but life as a plugin meant
that
>> certain integrations were still difficult. Our aim in Puppet 3.0 is to
>> tighten up these integrations (making Hiera more useful), provide
>> first-class data separation solution (for those not already using
Hiera),
>> and to provide a simple and safe migration for those currently using
Hiera.
>> 
>> Here’s what’s new:
>> 
>>   - Hiera data keys can be namespaced
>>      - (e.g. ''dns::nameserver'':
''8.8.8.8'')
>>   - Namespaced data will automatically populate class parameters
>>      - (e.g. ''dns::nameserver'' will be automatically
looked up for the
>>      $nameserver parameter when you include dns)
>> 
>> Here’s what’s changed:
>> 
>>   - The hiera-puppet module will no longer be required
>>      - It should, however, still continue to work
>>   - The heira() function (from the hiera-puppet module) will be
superceded
>>   by the lookup() function (in Puppet core)
>>   - hiera_include() will not be ported; include now properly accept
>>   arrays, making this redundant
>> 
>> Here’s what were still wondering about:
>> 
>>   - How should we integrate hiera_array() and hiera_hash()?
>>   - How should we integrate hiera’s “default” and “override”
parameters?
>>   - How should we handle overlaps between data supplied by Hiera and
data
>>   supplied in a parameterized class include?
>> 
>> If you’re interested in test-driving the new functionality for
yourself,
>> checkout out the master branch of the Puppet repository on
>> Github<https://github.com/puppetlabs/puppet>,
>> install Hiera (with gem install hiera) and make the following
configuration
>> changes:
>> 
>>   - Set data_binding_terminus to hiera
>>   - If you have a Hiera configuration file, set hiera_config to point
to
>>   it.
>>   - If you don’t, create a file in ${confdir}/hiera.yaml with these
>>   contents:
>> 
>>     :backend:
>> 
>>       - yaml
>> 
>>     :yaml:
>> 
>>       :datadir: $confdir/data
>> 
>>     :hierarchy:
>> 
>>       - %{certname}
>> 
>>       - %{environment}
>> 
>>       - global
>> 
>> This sets up a default hierarchy that looks for values in
${confdir}/data,
>> first in the ${certname}.yml file, then in the ${environment}.yml file,
>> then in global.yml.
>> 
>> That should be it!  Please, let us know if you''re having
trouble getting
>> started, or if you have questions, concerns, thoughts, or opinions
about
>> any of this.
>> 
>> Thanks!
>> 
> 
> - -- 
> Trevor Vaughan
> Vice President, Onyx Point, Inc.
> email: tvaughan@onyxpoint.com
> phone: 410-541-ONYX (6699)
> pgp: 0x6C701E94
> 
> - -- This account not approved for unencrypted sensitive information --
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.12 (GNU/Linux)
> 
> iQEcBAEBAgAGBQJPpVScAAoJECNCGV1OLcypcesIAJT9gARlmQNxR7ftUo4soR1r
> R2aCh/k2B+Wylj/xfF3o9j3gZh/1WrOLogMGThNgYT9Vcv50zCcbeFjbCTVr2bBo
> 18gtlcoJhTBeUebBk6CsFsG+sIApwja3UDbpSm49FRquCK/KI+fBV8gHhAQt9fgm
> 1P0ZXB1KU61nhqrLEpHNRmJVKBvLCwDuqEMJltMuGoysczWEv0uVROtYari1T3g+
> tNqov2EkSj5NV+Au5DeNNKviToEDn+wR7JvtXEqRdwSc142U9PnZwp8MvM+/i2+W
> RGKqg8HZuMMLEbbSnimdRnS6pCwKdgVJDPwBb+CoiZJJ0ftWCF2F9lNDjgUNjcI> =/h12
> -----END PGP SIGNATURE-----
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
> 
> <tvaughan.vcf>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Nigel Kersten

2012-May-06 21:54 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On Sat, May 5, 2012 at 9:26 AM, Trevor Vaughan
<tvaughan@onyxpoint.com>wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> With the integration of Hiera, is the extlookup functionality being
> deprecated?
>
> If so, I would like to ask that the team add a Hiera backend for the
> existing extlookup data sources.
>
> I know that it won''t support hashes but it would help to break the
view
> that Puppet routinely breaks existing functionality on upgrades.
>
We''ll deprecate it, but not *break* it.

We''d go through a whole release cycle before actually removing the
extlookup function, and as RI pointed out, we can migrate the backend
stores from extlookup CSV to straight up JSON/YAML.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Derek J. Balling

2012-May-07 14:22 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On May 3, 2012, at 1:05 PM, Pieter van de Bruggen wrote:> As many of you may be aware, Hiera will be tightly integrated into Puppet
in the upcoming release of Puppet 3.0.
> What will this mean for sites that have specifically avoided using Hiera? I
would presume that this should just mean "business as usual", with no
significant impact?




-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Nigel Kersten

2012-May-07 15:54 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On Mon, May 7, 2012 at 7:22 AM, Derek J. Balling <dredd@megacity.org>
wrote:
>
> On May 3, 2012, at 1:05 PM, Pieter van de Bruggen wrote:
>
> As many of you may be aware, Hiera will be tightly integrated into Puppet
> in the upcoming release of Puppet 3.0.
>
> What will this mean for sites that have specifically avoided using Hiera?
> I would presume that this should just mean "business as usual",
with no
> significant impact?
>
Yes. If you''re not planning to externalize data at all, the only
difference
you''ll see is that you''ll now have Hiera installed on your
system when you
install Puppet 3.0.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Derek J. Balling

2012-May-07 16:26 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On May 7, 2012, at 11:54 AM, Nigel Kersten wrote:> Yes. If you''re not planning to externalize data at all, the only
difference you''ll see is that you''ll now have Hiera installed
on your system when you install Puppet 3.0.
Cool. Was really panicking there for a moment.

I kinda like having "human readable" manifests as opposed to
"machine readable" YAML  :-)

Cheers,
D


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Nigel Kersten

2012-May-07 16:45 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On Mon, May 7, 2012 at 9:26 AM, Derek J. Balling <dredd@megacity.org>
wrote:
>
> On May 7, 2012, at 11:54 AM, Nigel Kersten wrote:
> > Yes. If you''re not planning to externalize data at all, the
only
> difference you''ll see is that you''ll now have Hiera
installed on your
> system when you install Puppet 3.0.
>
> Cool. Was really panicking there for a moment.
>
> I kinda like having "human readable" manifests as opposed to
"machine
> readable" YAML  :-)

Externalizing data actually helps things be much more human readable imho,
as you end up having semantically useful labels in the puppet manifests
rather than naked data.

But yes, if you''re not interested in externalizing data, so you
haven''t put
anything in the Hiera store, and continue to embed data in your manifests,
everything will still work as before.




-- 
Nigel Kersten | http://puppetlabs.com | @nigelkersten

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Christopher Wood

2012-May-07 16:47 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On Mon, May 07, 2012 at 12:26:32PM -0400, Derek J. Balling
wrote:> 
> On May 7, 2012, at 11:54 AM, Nigel Kersten wrote:
> > Yes. If you''re not planning to externalize data at all, the
only difference you''ll see is that you''ll now have Hiera
installed on your system when you install Puppet 3.0.
> 
> Cool. Was really panicking there for a moment.
> 
> I kinda like having "human readable" manifests as opposed to
"machine readable" YAML  :-)
Wrapper script (similar concept for anywhere with a yaml reader):

#!/usr/bin/perl
use warnings;
use strict;
use YAML::Syck;
use File::Slurp;
use Data::Dumper;
my $file = $ARGV[0];
my $text = read_file($file);
my $yaml = Load($text);
print Dumper($yaml);


Out of nosiness, how do you handle the edgier cases in your manifests? For
example, how everything is a standard syslog client, except for this host which
is loghost, and these couple of clients which all do different things with
syslog?


> Cheers,
> D
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
> 
> 
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Derek J. Balling

2012-May-08 10:00 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On May 7, 2012, at 12:47 PM, Christopher Wood wrote:> Wrapper script (similar concept for anywhere with a yaml reader):
Still have to *write* the YAML files.
> Out of nosiness, how do you handle the edgier cases in your manifests? For
example, how everything is a standard syslog client, except for this host which
is loghost, and these couple of clients which all do different things with
syslog?
We would have the syslog module have all the "common"
files/packages/services/etc. declared, and then simply do something like:

if ($fqdn =~ /^syslog/)
{
..... log host specific stuff .....
}
else
{
..... NON-log host specific stuff .....
}

doesn''t seem like rocket science, and makes it clear, all in one place,
what''s "common", and what''s "special".

Puppet provides some wonderful decision-making capabilities in the manifests
themselves, I''ve yet to see any need to have some ugly machine-readable
formatted file to solve a problem.

But I *have* seen the parts of our organization that DO use Hiera complaining
loudly that they can''t easily figure out which of a slurry of YAML
files is changing some setting/variable to a value they''re not
expecting.

Cheers,
D

--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

John Warburton

2012-May-08 11:14 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On 8 May 2012 20:00, Derek J. Balling <dredd@megacity.org> wrote:
>
> On May 7, 2012, at 12:47 PM, Christopher Wood wrote:
> > Wrapper script (similar concept for anywhere with a yaml reader):
>
> Still have to *write* the YAML files.
>
> Oh yeah. This is my big motivator sticking with extlookup and notconverting to Hiera - anyone in the team can do CSV. But YAML is starting
to push well into the dev camp and far away from ops (IMHO)

John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Craig Dunn

2012-May-08 11:22 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

>     Still have to *write* the YAML files.
>
> Oh yeah. This is my big motivator sticking with extlookup and not
> converting to Hiera - anyone in the team can do CSV. But YAML is
> starting to push well into the dev camp and far away from ops (IMHO)
Unless you''re doing rich data types such as hashes, which if
you''re
using CSV then it sounds like you''re not, then a simple YAML 
representation becomes "key: val" - YAML only starts pushing into dev 
camp when and if you use some of the more advanced features like anchors 
IMO.

Plus, when talking about Hiera we shouldn''t be focusing too much on the
credits or pitfalls of YAML - there are other DB and file based back 
ends available, and if one of those doesn''t fit the bill it''s
very
possible to extend Hiera with a new backend that supports a different 
format.

Craig


-- 
Craig Dunn | http://www.craigdunn.org
Yahoo/Skype: craigrdunn | Twitter: @crayfishX

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

R.I.Pienaar

2012-May-08 11:43 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

----- Original Message -----> From: "John Warburton" <jwarburton@gmail.com>
> To: puppet-users@googlegroups.com
> Sent: Tuesday, May 8, 2012 12:14:48 PM
> Subject: Re: [Puppet Users] Puppet 3.0 and Hiera
> 
> 
> On 8 May 2012 20:00, Derek J. Balling < dredd@megacity.org > wrote:
> 
> 
> 
> 
> On May 7, 2012, at 12:47 PM, Christopher Wood wrote:
> > Wrapper script (similar concept for anywhere with a yaml reader):
> 
> Still have to *write* the YAML files.
> 
> 
> 
> Oh yeah. This is my big motivator sticking with extlookup and not
> converting to Hiera - anyone in the team can do CSV. But YAML is
> starting to push well into the dev camp and far away from ops (IMHO)
If you want the same abilities that CSV has then this would be your YAML:

sysadmin: you@your.com
nameservers: [1.2.3.4, 2.3.4.5]

just that simple, I think this is much clearer than CSV.  If people really are
attached to CSV i guess it would be pretty trivial to write a CSV backend for
hiera but in the past one of the big complaints I had about extlookup was CSV
files.

When extlookup was written puppet did not have hashes and so CSV was a perfect
fit, now though it does and CSV just cant store all the kinds of data we needed
so rather than provide a crippled CSV implementation I decided to skip it.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

John Warburton

2012-May-08 12:06 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

Thanks R.I. & Craig Dunn

On 8 May 2012 21:43, R.I.Pienaar <rip@devco.net> wrote:
>
> If you want the same abilities that CSV has then this would be your YAML:
>
> sysadmin: you@your.com
> nameservers: [1.2.3.4, 2.3.4.5]
>
> just that simple, I think this is much clearer than CSV.  If people really
> are

That''s great - an easy migration then

Cheers

John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Christopher Wood

2012-May-08 13:39 UTC

head link

Re: [Puppet Users] Puppet 3.0 and Hiera

On Tue, May 08, 2012 at 06:00:07AM -0400, Derek J. Balling
wrote:> 
> On May 7, 2012, at 12:47 PM, Christopher Wood wrote:
> > Wrapper script (similar concept for anywhere with a yaml reader):
> 
> Still have to *write* the YAML files.
I don''t think I''ve ever actually opened a text editor and
written YAML. (Tweaked, sure, but not written.) This concept is useful across
any number of scripting languages:

#!/usr/bin/perl
use warnings;
use strict;
use YAML::Syck;
my %opts = (
    ''key1'' => ''value1'',
    ''key2'' => [ ''list2a'',
''list2b'' ],
    ''key3'' => { ''key3a'' =>
''value3a'', ''key3b'' => [
''list3b1'', ''list3b2'' ], },
);
$YAML::Syck::SortKeys = 1;
my $yaml = Dump( \%opts );
print $yaml;

It''s probably easier to think in your preferred language''s
data structure and then translate that to YAML. As discussed in other branches,
most of our data structure requirements really aren''t that complex.
> > Out of nosiness, how do you handle the edgier cases in your manifests?
For example, how everything is a standard syslog client, except for this host
which is loghost, and these couple of clients which all do different things with
syslog?
> 
> We would have the syslog module have all the "common"
files/packages/services/etc. declared, and then simply do something like:
> 
> if ($fqdn =~ /^syslog/)
> {
> ..... log host specific stuff .....
> }
> else
> {
> ..... NON-log host specific stuff .....
> }
> 
> doesn''t seem like rocket science, and makes it clear, all in one
place, what''s "common", and what''s
"special".
> 
> Puppet provides some wonderful decision-making capabilities in the
manifests themselves, I''ve yet to see any need to have some ugly
machine-readable formatted file to solve a problem.
> 
> But I *have* seen the parts of our organization that DO use Hiera
complaining loudly that they can''t easily figure out which of a slurry
of YAML files is changing some setting/variable to a value they''re not
expecting.
> 
> Cheers,
> D
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
> 
> 
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Pieter van de Bruggen

2012-May-08 22:01 UTC

head link

[Puppet Users] Re: Puppet 3.0 and Hiera

As a courtesy update for our user community:

My earlier claims about the "lookup" function were exaggerated. 
Instead,
the "hiera", "hiera_array", and "hiera_hash"
functions will all be
available, provided by Hiera.

Also, as mentioned earlier, class parameters will be sourced from multiple
locations, with the following priorities:
 - Hardcoded in the manifest (e.g. class { "foo": bar =>
"baz" })
 - Supplied by an ENC
 - Automatic namespaced data lookup in Hiera
 - Class-specified default value

Feel free to ask if you have any further questions.

Thanks,
  - Pieter

On Thu, May 3, 2012 at 10:05 AM, Pieter van de Bruggen <
pieter@puppetlabs.com> wrote:
> Greetings,
>
> As many of you may be aware, Hiera will be tightly integrated into Puppet
> in the upcoming release of Puppet 3.0. As a final sanity check of this
> work, I’d like to open our current plan for integration up for feedback.
> This is particularly for feedback from existing Hiera users, but I hope to
> solicit good feedback from those not using Hiera as well.
>
> The problem, as it currently exists, is that Puppet (core) has no good
> first-class mechanism for separating configuration data from manifests.
> Everything from $faked_namespaces__in__variable_names to specialty
> “params” classes have been tried, with varying degrees of success.
>
> Hiera came along as another solution to this problem, and provided a
> useful abstraction for hierarchical data lookup, but life as a plugin meant
> that certain integrations were still difficult. Our aim in Puppet 3.0 is to
> tighten up these integrations (making Hiera more useful), provide
> first-class data separation solution (for those not already using Hiera),
> and to provide a simple and safe migration for those currently using Hiera.
>
> Here’s what’s new:
>
>    - Hiera data keys can be namespaced
>       - (e.g. ''dns::nameserver'':
''8.8.8.8'')
>    - Namespaced data will automatically populate class parameters
>       - (e.g. ''dns::nameserver'' will be automatically
looked up for the
>       $nameserver parameter when you include dns)
>
> Here’s what’s changed:
>
>    - The hiera-puppet module will no longer be required
>       - It should, however, still continue to work
>    - The heira() function (from the hiera-puppet module) will be
>    superceded by the lookup() function (in Puppet core)
>    - hiera_include() will not be ported; include now properly accept
>    arrays, making this redundant
>
> Here’s what were still wondering about:
>
>    - How should we integrate hiera_array() and hiera_hash()?
>    - How should we integrate hiera’s “default” and “override” parameters?
>    - How should we handle overlaps between data supplied by Hiera and
>    data supplied in a parameterized class include?
>
> If you’re interested in test-driving the new functionality for yourself,
> checkout out the master branch of the Puppet repository on
Github<https://github.com/puppetlabs/puppet>,
> install Hiera (with gem install hiera) and make the following
> configuration changes:
>
>    - Set data_binding_terminus to hiera
>    - If you have a Hiera configuration file, set hiera_config to point to
>    it.
>    - If you don’t, create a file in ${confdir}/hiera.yaml with these
>    contents:
>
>      :backend:
>
>        - yaml
>
>      :yaml:
>
>        :datadir: $confdir/data
>
>      :hierarchy:
>
>        - %{certname}
>
>        - %{environment}
>
>        - global
>
> This sets up a default hierarchy that looks for values in ${confdir}/data,
> first in the ${certname}.yml file, then in the ${environment}.yml file,
> then in global.yml.
>
> That should be it!  Please, let us know if you''re having trouble
getting
> started, or if you have questions, concerns, thoughts, or opinions about
> any of this.
>
> Thanks!
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Puppet users - May 2012 - Puppet 3.0 and Hiera

[Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

[Puppet Users] Re: Puppet 3.0 and Hiera

[Puppet Users] Re: Puppet 3.0 and Hiera

Re: [Puppet Users] Re: Puppet 3.0 and Hiera

[Puppet Users] Re: Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

Re: [Puppet Users] Puppet 3.0 and Hiera

[Puppet Users] Re: Puppet 3.0 and Hiera