Hi,
I''m trying to build a module for haproxy which fetches all the
configuration data from Hiera to populate the haproxy config file. I''ve
run
into a number of issues though when I try to use hashes. Ideally, I want to
use something like this:
haproxy_listeners :
"cluster1" :
ip : ''192.168.0.2''
port : ''80''
servers :
"server1" :
ip : ''192.168.0.3''
port : ''8080''
So a hash of clusters with each cluster containing a nested hash of
servers. Is this possible with Hiera/ERB? It''s easy enough to iterate
over
the first hash, but I can''t work out how to extract the contents of the
nested hash. Or I might just be approaching this in entirely the wrong
way... Any help would be much appreciated.
Thanks,
Andy
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/vtr8gCCwuBsJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:> > Hi, > > I''m trying to build a module for haproxy which fetches all the > configuration data from Hiera to populate the haproxy config file. I''ve run > into a number of issues though when I try to use hashes. Ideally, I want to > use something like this: > > haproxy_listeners : > "cluster1" : > ip : ''192.168.0.2'' > port : ''80'' > servers : > "server1" : > ip : ''192.168.0.3'' > port : ''8080'' > > So a hash of clusters with each cluster containing a nested hash of > servers. Is this possible with Hiera/ERB? It''s easy enough to iterate over > the first hash, but I can''t work out how to extract the contents of the > nested hash. Or I might just be approaching this in entirely the wrong > way... Any help would be much appreciated. > >I haven''t used the function myself, but this looks like it would be a good case for a define + create_resources(), which I think is part of stdlib. You might need to restructure the hashes slightly, but I think that will be the best approach.> Thanks, > > Andy >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/DPkuzR8Q8soJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Wolf Noble
2013-Jan-04 16:37 UTC
Re: [Puppet Users] Hiera hashes and arrays in ERB templates
not sure if this is the "best" way to do this, but, taking from a
different module I have:
hiera yaml:
snmp_filesystems:
root: {
mountpoint: ''/'',
threshold: ''10%''
}
snmp_collector:
test: {
ip: ''1.2.3.4'',
group: ''readonly'',
community: ''somecommunity''
}
class:
class snmp::params(
filesystems = hiera(snmp_filesystems, undef),
collector = hiera(snmp_collector, undef)
{
#…
}
template:
<%
collector = scope.lookupvar(''snmp::params::collector'')
filesystems = scope.lookupvar(''snmp::params::filesystems'')
-%>
<% collector.each_pair do |key, hash| -%>
group <%=hash[''group'']-%> v2c <%=key%>
<%end-%>
# Filesystem monitoring enabled
<% filesystems.each_pair do |key, hash| -%>
# <%=key%>
disk <%=hash[''mountpoint'']-%>
<%=hash[''threshold'']%>
<%end-%>
does that help at all?
Wolf Noble
UNIX Team Lead
Datapipe Managed IT Services
1.201.792.4847 (international) x2910
1.888.749.5821 (toll free) x2910
On Jan 4, 2013, at 9:37 AM, llowder <llowderiv@gmail.com> wrote:
>
>
> On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:
> Hi,
>
> I''m trying to build a module for haproxy which fetches all the
configuration data from Hiera to populate the haproxy config file. I''ve
run into a number of issues though when I try to use hashes. Ideally, I want to
use something like this:
>
> haproxy_listeners :
> "cluster1" :
> ip : ''192.168.0.2''
> port : ''80''
> servers :
> "server1" :
> ip : ''192.168.0.3''
> port : ''8080''
>
> So a hash of clusters with each cluster containing a nested hash of
servers. Is this possible with Hiera/ERB? It''s easy enough to iterate
over the first hash, but I can''t work out how to extract the contents
of the nested hash. Or I might just be approaching this in entirely the wrong
way... Any help would be much appreciated.
>
>
> I haven''t used the function myself, but this looks like it would
be a good case for a define + create_resources(), which I think is part of
stdlib. You might need to restructure the hashes slightly, but I think that will
be the best approach.
>
>
> Thanks,
>
> Andy
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/DPkuzR8Q8soJ.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
________________________________
This message may contain confidential or privileged information. If you are not
the intended recipient, please advise us immediately and delete this message.
See http://www.datapipe.com/legal/email_disclaimer/ for further information on
confidentiality and the risks of non-secure electronic communication. If you
cannot access these links, please notify us by reply message and we will send
the contents to you.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Gavin Williams
2013-Jan-04 16:47 UTC
[Puppet Users] Re: Hiera hashes and arrays in ERB templates
I''m with Andy on this one... I''m doing something very similar
with my
NetApp volume provider (https://github.com/fatmcgav/fatmcgav-netapp/).
I''ve created a define with the following contents:
define util::netapp::volume (
$ensure = present,
$size,
$aggr = ''aggr1'',
$snapresv = 0,
$autoincrement = true,
$snapschedule = {"minutes" => 0, "hours" => 0,
"days" => 0, "weeks"
=> 0}
) {
netapp_notify {"volume_define_${name}":
message => "Processing Volume ${name}",
}
->
netapp_volume { "v_${name}":
ensure => $ensure,
initsize => $size,
aggregate => $aggr,
spaceres => "none",
snapreserve => $snapresv,
autoincrement => $autoincrement,
options => {''convert_ucode'' =>
''on'', ''no_atime_update'' =>
''on'', ''try_first'' =>
''volume_grow''},
snapschedule => $snapschedule
}
->
netapp_qtree { "q_${name}":
ensure => $ensure,
volume => "v_${name}"
}
->
netapp_export { "/vol/v_${name}/q_${name}":
ensure => $ensure,
persistent => true
}
}
I''ve added a default hash to ''snapschedule'' in the
options list, but that
can be over-ridden from the Hiera data.
Then use the following to pull the data from hiera and call the define:
create_resources( util::netapp::volume, hiera(''volumes'') )
''Volumes'' in hiera yaml looks like:
volumes:
vol1:
ensure: present
size: ''500m''
vol2:
ensure: present
size: ''20g''
snapschedule:
minutes: 0
hours: 36
days: 0
weeks: 0
You can also use the ''hiera'' command to test your yaml
structure:
$ hiera -c hiera.yaml volumes clientcert=act-star-nactl01
{"vol1"=>{"ensure"=>"present",
"size"=>"500m"},
"vol2"=>{"ensure"=>"present",
"size"=>"20g",
"snapschedule"=>{"days"=>0,
"weeks"=>0, "hours"=>36, "minutes"=>0}}}
As you can see from the above output, *snapschedule* for *vol2* is a nested
hash. This assumes that your resource provider can support hashes on the
relevant param/property ;)
HTH
Gav
On Friday, 4 January 2013 15:37:25 UTC, llowder wrote:>
>
>
> On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:
>>
>> Hi,
>>
>> I''m trying to build a module for haproxy which fetches all the
>> configuration data from Hiera to populate the haproxy config file.
I''ve run
>> into a number of issues though when I try to use hashes. Ideally, I
want to
>> use something like this:
>>
>> haproxy_listeners :
>> "cluster1" :
>> ip : ''192.168.0.2''
>> port : ''80''
>> servers :
>> "server1" :
>> ip : ''192.168.0.3''
>> port : ''8080''
>>
>> So a hash of clusters with each cluster containing a nested hash of
>> servers. Is this possible with Hiera/ERB? It''s easy enough to
iterate over
>> the first hash, but I can''t work out how to extract the
contents of the
>> nested hash. Or I might just be approaching this in entirely the wrong
>> way... Any help would be much appreciated.
>>
>>
> I haven''t used the function myself, but this looks like it would
be a good
> case for a define + create_resources(), which I think is part of stdlib.
> You might need to restructure the hashes slightly, but I think that will be
> the best approach.
>
>
>
>> Thanks,
>>
>> Andy
>>
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/UjWLd-yp7iIJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Andy Taylor
2013-Jan-04 17:05 UTC
[Puppet Users] Re: Hiera hashes and arrays in ERB templates
Thanks for your suggestions guys. I did consider using create_resource, but
don''t see how I can when I''m trying to apply this Hiera data
to a single
file. To expand on my initial post, what I need to do is create multiple
config blocks within one file resource. So this Hiera data:
haproxy_listeners :
"cluster1" :
ip : ''192.168.0.2''
port : ''80''
servers :
"server1" :
ip : ''192.168.0.3''
port : ''8080''
"server2" :
ip : ''192.168.0.4''
port : ''8080''
"cluster2" :
ip : ''192.168.0.5''
port : ''80''
servers :
"server3"
ip : ''192.168.0.6''
port : ''8080''
"server4"
ip : ''192.168.0.7''
port : ''8080''
will result in this being generated in the haproxy config file:
listen cluster1 192.168.0.2:80
server server1 192.168.0.3:8080
server server2 192.168.0.4:8080
listen cluster2 192.168.0.5:80
server server3 192.168.0.6:8080
server server4 192.168.0.7:8080
So I don''t see how create_resources can handle this, as that''s
for creating
multiple Puppet resources, as opposed to multiple blocks within a single
file. The only alternative I can think of at the moment is using
create_resources with a define which utilizes Augeas, but I don''t know
how
well that will work.
Thanks,
Andy
On Friday, 4 January 2013 16:47:13 UTC, Gavin Williams
wrote:>
> I''m with Andy on this one... I''m doing something very
similar with my
> NetApp volume provider (https://github.com/fatmcgav/fatmcgav-netapp/).
>
> I''ve created a define with the following contents:
> define util::netapp::volume (
> $ensure = present,
> $size,
> $aggr = ''aggr1'',
> $snapresv = 0,
> $autoincrement = true,
> $snapschedule = {"minutes" => 0, "hours"
=> 0, "days" => 0,
> "weeks" => 0}
> ) {
>
> netapp_notify {"volume_define_${name}":
> message => "Processing Volume ${name}",
> }
> ->
> netapp_volume { "v_${name}":
> ensure => $ensure,
> initsize => $size,
> aggregate => $aggr,
> spaceres => "none",
> snapreserve => $snapresv,
> autoincrement => $autoincrement,
> options => {''convert_ucode'' =>
''on'', ''no_atime_update'' =>
> ''on'', ''try_first'' =>
''volume_grow''},
> snapschedule => $snapschedule
> }
> ->
> netapp_qtree { "q_${name}":
> ensure => $ensure,
> volume => "v_${name}"
> }
> ->
> netapp_export { "/vol/v_${name}/q_${name}":
> ensure => $ensure,
> persistent => true
> }
>
> }
>
> I''ve added a default hash to ''snapschedule'' in
the options list, but that
> can be over-ridden from the Hiera data.
>
> Then use the following to pull the data from hiera and call the define:
> create_resources( util::netapp::volume, hiera(''volumes'')
)
>
> ''Volumes'' in hiera yaml looks like:
> volumes:
> vol1:
> ensure: present
> size: ''500m''
> vol2:
> ensure: present
> size: ''20g''
> snapschedule:
> minutes: 0
> hours: 36
> days: 0
> weeks: 0
>
> You can also use the ''hiera'' command to test your yaml
structure:
> $ hiera -c hiera.yaml volumes clientcert=act-star-nactl01
> {"vol1"=>{"ensure"=>"present",
"size"=>"500m"},
> "vol2"=>{"ensure"=>"present",
"size"=>"20g",
"snapschedule"=>{"days"=>0,
> "weeks"=>0, "hours"=>36,
"minutes"=>0}}}
>
> As you can see from the above output, *snapschedule* for *vol2* is a
> nested hash. This assumes that your resource provider can support hashes on
> the relevant param/property ;)
>
> HTH
>
> Gav
>
> On Friday, 4 January 2013 15:37:25 UTC, llowder wrote:
>>
>>
>>
>> On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:
>>>
>>> Hi,
>>>
>>> I''m trying to build a module for haproxy which fetches all
the
>>> configuration data from Hiera to populate the haproxy config file.
I''ve run
>>> into a number of issues though when I try to use hashes. Ideally, I
want to
>>> use something like this:
>>>
>>> haproxy_listeners :
>>> "cluster1" :
>>> ip : ''192.168.0.2''
>>> port : ''80''
>>> servers :
>>> "server1" :
>>> ip : ''192.168.0.3''
>>> port : ''8080''
>>>
>>> So a hash of clusters with each cluster containing a nested hash of
>>> servers. Is this possible with Hiera/ERB? It''s easy enough
to iterate over
>>> the first hash, but I can''t work out how to extract the
contents of the
>>> nested hash. Or I might just be approaching this in entirely the
wrong
>>> way... Any help would be much appreciated.
>>>
>>>
>> I haven''t used the function myself, but this looks like it
would be a
>> good case for a define + create_resources(), which I think is part of
>> stdlib. You might need to restructure the hashes slightly, but I think
that
>> will be the best approach.
>>
>>
>>
>>> Thanks,
>>>
>>> Andy
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/W3UBJBXuT24J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Steven Nemetz
2013-Jan-04 17:56 UTC
RE: [Puppet Users] Re: Hiera hashes and arrays in ERB templates
I''m using stdlib to help with this $listeners =
hiera(''haproxy_listeners'', undef)$listener_keys =
keys($listeners) then pass $listener_keys to a define to create all the
instances
The entre hash is in memory ($listeners) and the define will have the key it is
working on ($name)So you can access anything in the data structure to build your
resources.
Steven
Date: Fri, 4 Jan 2013 09:05:15 -0800
From: andytayloruk@gmail.com
To: puppet-users@googlegroups.com
Subject: [Puppet Users] Re: Hiera hashes and arrays in ERB templates
Thanks for your suggestions guys. I did consider using create_resource, but
don''t see how I can when I''m trying to apply this Hiera data
to a single file. To expand on my initial post, what I need to do is create
multiple config blocks within one file resource. So this Hiera data:
haproxy_listeners : "cluster1" : ip : ''192.168.0.2''
port : ''80'' servers : "server1" : ip :
''192.168.0.3'' port : ''8080''
"server2" : ip : ''192.168.0.4'' port :
''8080'' "cluster2" : ip :
''192.168.0.5'' port : ''80'' servers :
"server3" ip : ''192.168.0.6'' port :
''8080'' "server4" ip :
''192.168.0.7'' port : ''8080''
will result in this being generated in the haproxy config file:
listen cluster1 192.168.0.2:80 server server1 192.168.0.3:8080
server server2 192.168.0.4:8080
listen cluster2 192.168.0.5:80 server server3 192.168.0.6:8080
server server4 192.168.0.7:8080
So I don''t see how create_resources can handle this, as that''s
for creating multiple Puppet resources, as opposed to multiple blocks within a
single file. The only alternative I can think of at the moment is using
create_resources with a define which utilizes Augeas, but I don''t know
how well that will work.
Thanks,
Andy
On Friday, 4 January 2013 16:47:13 UTC, Gavin Williams wrote:I''m with
Andy on this one... I''m doing something very similar with my NetApp
volume provider (https://github.com/fatmcgav/fatmcgav-netapp/).
I''ve created a define with the following contents:
define util::netapp::volume (
$ensure = present,
$size,
$aggr = ''aggr1'',
$snapresv = 0,
$autoincrement = true,
$snapschedule = {"minutes" => 0, "hours" => 0,
"days" => 0, "weeks" => 0}
) {
netapp_notify {"volume_define_${name}":
message => "Processing Volume ${name}",
}
->
netapp_volume { "v_${name}":
ensure => $ensure,
initsize => $size,
aggregate => $aggr,
spaceres => "none",
snapreserve => $snapresv,
autoincrement => $autoincrement,
options => {''convert_ucode'' =>
''on'', ''no_atime_update'' =>
''on'', ''try_first'' =>
''volume_grow''},
snapschedule => $snapschedule
}
->
netapp_qtree { "q_${name}":
ensure => $ensure,
volume => "v_${name}"
}
->
netapp_export { "/vol/v_${name}/q_${name}":
ensure => $ensure,
persistent => true
}
}
I''ve added a default hash to ''snapschedule'' in the
options list, but that can be over-ridden from the Hiera data.
Then use the following to pull the data from hiera and call the define:
create_resources( util::netapp::volume, hiera(''volumes'') )
''Volumes'' in hiera yaml looks like:
volumes:
vol1:
ensure: present
size: ''500m''
vol2:
ensure: present
size: ''20g''
snapschedule:
minutes: 0
hours: 36
days: 0
weeks: 0
You can also use the ''hiera'' command to test your yaml
structure:
$ hiera -c hiera.yaml volumes clientcert=act-star-nactl01
{"vol1"=>{"ensure"=>"present",
"size"=>"500m"},
"vol2"=>{"ensure"=>"present",
"size"=>"20g",
"snapschedule"=>{"days"=>0, "weeks"=>0,
"hours"=>36, "minutes"=>0}}}
As you can see from the above output, snapschedule for vol2 is a nested hash.
This assumes that your resource provider can support hashes on the relevant
param/property ;)
HTH
Gav
On Friday, 4 January 2013 15:37:25 UTC, llowder wrote:
On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:Hi,
I''m trying to build a module for haproxy which fetches all the
configuration data from Hiera to populate the haproxy config file. I''ve
run into a number of issues though when I try to use hashes. Ideally, I want to
use something like this:
haproxy_listeners : "cluster1" : ip : ''192.168.0.2''
port : ''80'' servers : "server1" : ip :
''192.168.0.3'' port : ''8080''
So a hash of clusters with each cluster containing a nested hash of servers. Is
this possible with Hiera/ERB? It''s easy enough to iterate over the
first hash, but I can''t work out how to extract the contents of the
nested hash. Or I might just be approaching this in entirely the wrong way...
Any help would be much appreciated.
I haven''t used the function myself, but this looks like it would be a
good case for a define + create_resources(), which I think is part of stdlib.
You might need to restructure the hashes slightly, but I think that will be the
best approach.
Thanks,
Andy
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/W3UBJBXuT24J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
jcbollinger
2013-Jan-04 21:55 UTC
[Puppet Users] Re: Hiera hashes and arrays in ERB templates
On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:> > Hi, > > I''m trying to build a module for haproxy which fetches all the > configuration data from Hiera to populate the haproxy config file. I''ve run > into a number of issues though when I try to use hashes. Ideally, I want to > use something like this: > > haproxy_listeners : > "cluster1" : > ip : ''192.168.0.2'' > port : ''80'' > servers : > "server1" : > ip : ''192.168.0.3'' > port : ''8080'' > > So a hash of clusters with each cluster containing a nested hash of > servers. Is this possible with Hiera/ERB? It''s easy enough to iterate over > the first hash, but I can''t work out how to extract the contents of the > nested hash. Or I might just be approaching this in entirely the wrong > way... Any help would be much appreciated. > >The data can be expressed in YAML, loaded into a Puppet variable via Hiera, and processed in an ERB template to construct your file. In other words, yes, it is possible. May I ask, however, what the advantage is to encoding the data in YAML and then transcoding it into the haproxy configuration language? I mean, if you also use bits and pieces of that object elsewhere in your manifests then perhaps it makes sense, but if all you want to do with it is generate the config file then I don''t see the point. Why not just serve up the config as a static or mostly-static file? Leaving aside the question of merits of your design, - Note that your example data are a quadruply-nested hash, not just a doubly-nested one as you say. You have a hash of clusters, each of which is a hash itself and contains a third hash, each of whose values is * another* hash. - The same Ruby mechanisms that allow your template to iterate and otherwise inspect the outer hash also work for the inner ones. - Perhaps, though, you are tripping over the fact that the values in your outer hash are of uniform type and meaning, whereas the values in the next-level hash are of varying type and meaning. You probably don''t actually want to iterate the second-level hash; instead, you want to retrieve specific values from it by key. So, you might do something of this general form (not intended to resemble an actual haproxy configuration file): haproxy.conf.erb: ===========<% @clusters.each_pair | cluster, properties | { -%> <cluster name="<%= cluster %>" ip="<%= properties[''ip''] %>" port="<%= properties[''port''] %>" > <% properties[''servers''].each_pair | server, server_props | { -%> <server name="<%= server %>" ip="<%= server_props[''ip''] %>" port="<%= server_props[''port''] %>" /> <% } -%> </cluster> <% } -%> Once corrected for the inevitable typos, that should produce output similar to <cluster name="cluster1" ip="192.168.0.2" port="80" > <server name="server1" ip="192.168.0.3" port="8080" /> </cluster> John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/uZSpX_tBuM8J. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.