Puppet users - Mar 2013 - hiera_hash lookups for included classes?

When using "include" to include a class Hiera helpfully performs an 
autolookup on the parameters of the included class. Specially-named 
variable names are automatically pulled from the Hiera datastore(s) and 
passed into the included module. This is pretty awesome.

But it looks like included resources _only_ perform a hiera() lookup. If we 
have a hash defined in several levels of our Hiera data, and we want the 
hash to be collapsed into a single hash following hierarchy order, does 
Hiera offer a way to perform a hiera_hash() lookup for the autolookup?

The documentation discourages mixing defined-type module inclusion and 
"include"-type module inclusion. So if there''s no
hiera_hash() option for
autolookups, we''re required to use defined-type inclusion, yes?

Thanks.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

You can just call hiera_hash() from within your manifest, no? It''s not 
quite as pretty as auto lookup, but it works, and we can hope we get some 
prettier syntax in the future.

On Wednesday, March 13, 2013 6:28:00 PM UTC-7, Scott Merrill
wrote:
>
> When using "include" to include a class Hiera helpfully performs
an
> autolookup on the parameters of the included class. Specially-named 
> variable names are automatically pulled from the Hiera datastore(s) and 
> passed into the included module. This is pretty awesome.
>
> But it looks like included resources _only_ perform a hiera() lookup. If 
> we have a hash defined in several levels of our Hiera data, and we want the
> hash to be collapsed into a single hash following hierarchy order, does 
> Hiera offer a way to perform a hiera_hash() lookup for the autolookup?
>
> The documentation discourages mixing defined-type module inclusion and 
> "include"-type module inclusion. So if there''s no
hiera_hash() option for
> autolookups, we''re required to use defined-type inclusion, yes?
>
> Thanks.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Second Ellison''s idea.
That''s what I do for hashes as well.



On 14 March 2013 11:51, Ellison Marks <gtyaoi@gmail.com> wrote:
> You can just call hiera_hash() from within your manifest, no? It''s
not
> quite as pretty as auto lookup, but it works, and we can hope we get some
> prettier syntax in the future.
>
>
> On Wednesday, March 13, 2013 6:28:00 PM UTC-7, Scott Merrill wrote:
>>
>> When using "include" to include a class Hiera helpfully
performs an
>> autolookup on the parameters of the included class. Specially-named
>> variable names are automatically pulled from the Hiera datastore(s) and
>> passed into the included module. This is pretty awesome.
>>
>> But it looks like included resources _only_ perform a hiera() lookup.
If
>> we have a hash defined in several levels of our Hiera data, and we want
the
>> hash to be collapsed into a single hash following hierarchy order, does
>> Hiera offer a way to perform a hiera_hash() lookup for the autolookup?
>>
>> The documentation discourages mixing defined-type module inclusion and
>> "include"-type module inclusion. So if there''s no
hiera_hash() option for
>> autolookups, we''re required to use defined-type inclusion,
yes?
>>
>> Thanks.
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscribe@googlegroups.com.
> To post to this group, send email to puppet-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Wednesday, March 13, 2013 8:51:04 PM UTC-5, Ellison Marks
wrote:
>
> You can just call hiera_hash() from within your manifest, no?


No.  The OP wants hash merging, and hiera_hash() doesn''t do that (as
far as
I understand).  It will instead return a hash whose keys are hierarchy 
levels, and whose values are those associated with the requested key at the 
corresponding level.  Hash merging is supposedly included in Hiera 1.2, 
which is now at release candidate stage, but I don''t know what the API
for
it looks like.

With older hiera, you can use hiera_array() to get the component hashes in 
hierarchy order, and then merge them manually.  The puppetlabs-stdlib 
module provides a merge function that does almost what the OP wants, but 
although it will merge an arbitrary number of hashes, I don''t think it
will
accept them in array-of-hashes form.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Thursday, March 14, 2013 10:26:59 AM UTC-4, jcbollinger
wrote:
>
>
>
> On Wednesday, March 13, 2013 8:51:04 PM UTC-5, Ellison Marks wrote:
>>
>> You can just call hiera_hash() from within your manifest, no?
>
>
>
> No.  The OP wants hash merging, and hiera_hash() doesn''t do that
(as far
> as I understand).  It will instead return a hash whose keys are hierarchy 
> levels, and whose values are those associated with the requested key at the
> corresponding level.  Hash merging is supposedly included in Hiera 1.2, 
> which is now at release candidate stage, but I don''t know what the
API for
> it looks like.
>
Although we''re just dipping out toes into the Hiera waters, I think 
hiera_hash is exactly what we want.

Given this hiera.yaml:
---
:backends:
  - yaml
:yaml:
  :datadir: /etc/puppet/hieradata
:hierarchy:
  - %{::clientcert}
  - %{::environment}
  - common

and these yaml files:

lab.yaml
---
email_aliases:
    root: someone-else@company.com

common.yaml
---
email_aliases:
    root: someone@company.com
    foo: foo@company.com
    bar: bar@company.com
 

And a manifest like this:
class mail::aliases ( $alias_list )
{
    file { ''/etc/aliases'':
        mode => 0644,
        owner => root,
        group => root,
        content => template(''mail/aliases.erb'')
    }

We could then do something like this:
node foo {
    $environment = ''lab''
    class{ ''mail::aliases'':
        alias_list => hiera_hash( ''email_aliases'' )
}

Node foo should have "root: someone-else@company.com" for the root
alias,
but also still get foo and bar, because the hiera_hash() function will 
construct a hash from the Hiera data while preserving hierarchy order.  
Environment ("lab") occurs before common, so the "root"
element of the
email_aliases hash should be the one from the lab.yaml file.

We can -- indeed, must -- use declared-type declarations for our modules in 
order to benefit from hiera_hash() in this way.

The reason I was asking about include-type declarations was because
we''d
eventually like to consume some Puppet Forge modules.  To do so means we 
need an intermediary class that performs the hiera_hash() lookups and 
passes the values on to the destination module.  It would be neat if we 
could just name or Hiera values according to the module''s parameter
names
and enjoy the magic of autolookup.

I just wanted to confirm that my understanding of the situation is 
correct.  If I''m wrong, I''d love to be corrected.

Thanks!
Scott


 

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

I''m pretty sure it does do merging, in the yaml backend at least.

case resolution_type
...
when :hash
            raise Exception, "Hiera type mismatch: expected Hash and got 
#{new_answer.class}" unless new_answer.kind_of? Hash
            answer ||= {}
            answer = new_answer.merge answer

On Thursday, March 14, 2013 7:26:59 AM UTC-7, jcbollinger
wrote:
>
>
>
> On Wednesday, March 13, 2013 8:51:04 PM UTC-5, Ellison Marks wrote:
>>
>> You can just call hiera_hash() from within your manifest, no?
>
>
>
> No.  The OP wants hash merging, and hiera_hash() doesn''t do that
(as far
> as I understand).  It will instead return a hash whose keys are hierarchy 
> levels, and whose values are those associated with the requested key at the
> corresponding level.  Hash merging is supposedly included in Hiera 1.2, 
> which is now at release candidate stage, but I don''t know what the
API for
> it looks like.
>
> With older hiera, you can use hiera_array() to get the component hashes in 
> hierarchy order, and then merge them manually.  The puppetlabs-stdlib 
> module provides a merge function that does almost what the OP wants, but 
> although it will merge an arbitrary number of hashes, I don''t
think it will
> accept them in array-of-hashes form.
>
>
> John
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Thursday, March 14, 2013 3:39:41 PM UTC-5, Ellison Marks
wrote:
>
> I''m pretty sure it does do merging, in the yaml backend at least.
>
> case resolution_type
> ...
> when :hash
>             raise Exception, "Hiera type mismatch: expected Hash and
got
> #{new_answer.class}" unless new_answer.kind_of? Hash
>             answer ||= {}
>             answer = new_answer.merge answer
>
>
Never mind.  I somehow got a totally wrong idea of what hiera_hash() does.  
I probably formed it by analogy with hiera_array(), but in fact they
aren''t
very analogous in that hiera_array() gives you all the data in your 
hierarchy, whereas hiera_hash() gives you a digested form.  It looks like 
it does merge (but only shallowly until 1.2).

I got another surprise when studying the code, too: hiera_array() is much 
less general than I thought.  It collects only String and Array values, not 
numeric, boolean, or hash values.  It raises an exception if any of the 
collected values for the give key is of any of the latter types, and it is 
not clear to me why.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

The difference between array and hash resolution makes sense, from a 
certain perspective. As an array is fine with holding duplicate values, it 
just gets all the data and flattens it down, but since a hash needs unique 
keys, to return a fairly flat hash, it can only take one value for each 
key, and prefers the more specific.

The string and array restriction is quite odd though, looking at it now...

On Friday, March 15, 2013 8:28:56 AM UTC-7, jcbollinger
wrote:
>
>
>
> On Thursday, March 14, 2013 3:39:41 PM UTC-5, Ellison Marks wrote:
>>
>> I''m pretty sure it does do merging, in the yaml backend at
least.
>>
>> case resolution_type
>> ...
>> when :hash
>>             raise Exception, "Hiera type mismatch: expected Hash
and got
>> #{new_answer.class}" unless new_answer.kind_of? Hash
>>             answer ||= {}
>>             answer = new_answer.merge answer
>>
>>
>
> Never mind.  I somehow got a totally wrong idea of what hiera_hash() 
> does.  I probably formed it by analogy with hiera_array(), but in fact they
> aren''t very analogous in that hiera_array() gives you all the data
in your
> hierarchy, whereas hiera_hash() gives you a digested form.  It looks like 
> it does merge (but only shallowly until 1.2).
>
> I got another surprise when studying the code, too: hiera_array() is much 
> less general than I thought.  It collects only String and Array values, not
> numeric, boolean, or hash values.  It raises an exception if any of the 
> collected values for the give key is of any of the latter types, and it is 
> not clear to me why.
>
>
> John
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.