Hi, I''m trying to build a module for haproxy which fetches all the configuration data from Hiera to populate the haproxy config file. I''ve run into a number of issues though when I try to use hashes. Ideally, I want to use something like this: haproxy_listeners : "cluster1" : ip : ''192.168.0.2'' port : ''80'' servers : "server1" : ip : ''192.168.0.3'' port : ''8080'' So a hash of clusters with each cluster containing a nested hash of servers. Is this possible with Hiera/ERB? It''s easy enough to iterate over the first hash, but I can''t work out how to extract the contents of the nested hash. Or I might just be approaching this in entirely the wrong way... Any help would be much appreciated. Thanks, Andy -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/vtr8gCCwuBsJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:> > Hi, > > I''m trying to build a module for haproxy which fetches all the > configuration data from Hiera to populate the haproxy config file. I''ve run > into a number of issues though when I try to use hashes. Ideally, I want to > use something like this: > > haproxy_listeners : > "cluster1" : > ip : ''192.168.0.2'' > port : ''80'' > servers : > "server1" : > ip : ''192.168.0.3'' > port : ''8080'' > > So a hash of clusters with each cluster containing a nested hash of > servers. Is this possible with Hiera/ERB? It''s easy enough to iterate over > the first hash, but I can''t work out how to extract the contents of the > nested hash. Or I might just be approaching this in entirely the wrong > way... Any help would be much appreciated. > >I haven''t used the function myself, but this looks like it would be a good case for a define + create_resources(), which I think is part of stdlib. You might need to restructure the hashes slightly, but I think that will be the best approach.> Thanks, > > Andy >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/DPkuzR8Q8soJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Wolf Noble
2013-Jan-04 16:37 UTC
Re: [Puppet Users] Hiera hashes and arrays in ERB templates
not sure if this is the "best" way to do this, but, taking from a different module I have: hiera yaml: snmp_filesystems: root: { mountpoint: ''/'', threshold: ''10%'' } snmp_collector: test: { ip: ''1.2.3.4'', group: ''readonly'', community: ''somecommunity'' } class: class snmp::params( filesystems = hiera(snmp_filesystems, undef), collector = hiera(snmp_collector, undef) { #… } template: <% collector = scope.lookupvar(''snmp::params::collector'') filesystems = scope.lookupvar(''snmp::params::filesystems'') -%> <% collector.each_pair do |key, hash| -%> group <%=hash[''group'']-%> v2c <%=key%> <%end-%> # Filesystem monitoring enabled <% filesystems.each_pair do |key, hash| -%> # <%=key%> disk <%=hash[''mountpoint'']-%> <%=hash[''threshold'']%> <%end-%> does that help at all? Wolf Noble UNIX Team Lead Datapipe Managed IT Services 1.201.792.4847 (international) x2910 1.888.749.5821 (toll free) x2910 On Jan 4, 2013, at 9:37 AM, llowder <llowderiv@gmail.com> wrote:> > > On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote: > Hi, > > I''m trying to build a module for haproxy which fetches all the configuration data from Hiera to populate the haproxy config file. I''ve run into a number of issues though when I try to use hashes. Ideally, I want to use something like this: > > haproxy_listeners : > "cluster1" : > ip : ''192.168.0.2'' > port : ''80'' > servers : > "server1" : > ip : ''192.168.0.3'' > port : ''8080'' > > So a hash of clusters with each cluster containing a nested hash of servers. Is this possible with Hiera/ERB? It''s easy enough to iterate over the first hash, but I can''t work out how to extract the contents of the nested hash. Or I might just be approaching this in entirely the wrong way... Any help would be much appreciated. > > > I haven''t used the function myself, but this looks like it would be a good case for a define + create_resources(), which I think is part of stdlib. You might need to restructure the hashes slightly, but I think that will be the best approach. > > > Thanks, > > Andy > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/DPkuzR8Q8soJ. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.________________________________ This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/legal/email_disclaimer/ for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Gavin Williams
2013-Jan-04 16:47 UTC
[Puppet Users] Re: Hiera hashes and arrays in ERB templates
I''m with Andy on this one... I''m doing something very similar with my NetApp volume provider (https://github.com/fatmcgav/fatmcgav-netapp/). I''ve created a define with the following contents: define util::netapp::volume ( $ensure = present, $size, $aggr = ''aggr1'', $snapresv = 0, $autoincrement = true, $snapschedule = {"minutes" => 0, "hours" => 0, "days" => 0, "weeks" => 0} ) { netapp_notify {"volume_define_${name}": message => "Processing Volume ${name}", } -> netapp_volume { "v_${name}": ensure => $ensure, initsize => $size, aggregate => $aggr, spaceres => "none", snapreserve => $snapresv, autoincrement => $autoincrement, options => {''convert_ucode'' => ''on'', ''no_atime_update'' => ''on'', ''try_first'' => ''volume_grow''}, snapschedule => $snapschedule } -> netapp_qtree { "q_${name}": ensure => $ensure, volume => "v_${name}" } -> netapp_export { "/vol/v_${name}/q_${name}": ensure => $ensure, persistent => true } } I''ve added a default hash to ''snapschedule'' in the options list, but that can be over-ridden from the Hiera data. Then use the following to pull the data from hiera and call the define: create_resources( util::netapp::volume, hiera(''volumes'') ) ''Volumes'' in hiera yaml looks like: volumes: vol1: ensure: present size: ''500m'' vol2: ensure: present size: ''20g'' snapschedule: minutes: 0 hours: 36 days: 0 weeks: 0 You can also use the ''hiera'' command to test your yaml structure: $ hiera -c hiera.yaml volumes clientcert=act-star-nactl01 {"vol1"=>{"ensure"=>"present", "size"=>"500m"}, "vol2"=>{"ensure"=>"present", "size"=>"20g", "snapschedule"=>{"days"=>0, "weeks"=>0, "hours"=>36, "minutes"=>0}}} As you can see from the above output, *snapschedule* for *vol2* is a nested hash. This assumes that your resource provider can support hashes on the relevant param/property ;) HTH Gav On Friday, 4 January 2013 15:37:25 UTC, llowder wrote:> > > > On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote: >> >> Hi, >> >> I''m trying to build a module for haproxy which fetches all the >> configuration data from Hiera to populate the haproxy config file. I''ve run >> into a number of issues though when I try to use hashes. Ideally, I want to >> use something like this: >> >> haproxy_listeners : >> "cluster1" : >> ip : ''192.168.0.2'' >> port : ''80'' >> servers : >> "server1" : >> ip : ''192.168.0.3'' >> port : ''8080'' >> >> So a hash of clusters with each cluster containing a nested hash of >> servers. Is this possible with Hiera/ERB? It''s easy enough to iterate over >> the first hash, but I can''t work out how to extract the contents of the >> nested hash. Or I might just be approaching this in entirely the wrong >> way... Any help would be much appreciated. >> >> > I haven''t used the function myself, but this looks like it would be a good > case for a define + create_resources(), which I think is part of stdlib. > You might need to restructure the hashes slightly, but I think that will be > the best approach. > > > >> Thanks, >> >> Andy >> >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/UjWLd-yp7iIJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Andy Taylor
2013-Jan-04 17:05 UTC
[Puppet Users] Re: Hiera hashes and arrays in ERB templates
Thanks for your suggestions guys. I did consider using create_resource, but don''t see how I can when I''m trying to apply this Hiera data to a single file. To expand on my initial post, what I need to do is create multiple config blocks within one file resource. So this Hiera data: haproxy_listeners : "cluster1" : ip : ''192.168.0.2'' port : ''80'' servers : "server1" : ip : ''192.168.0.3'' port : ''8080'' "server2" : ip : ''192.168.0.4'' port : ''8080'' "cluster2" : ip : ''192.168.0.5'' port : ''80'' servers : "server3" ip : ''192.168.0.6'' port : ''8080'' "server4" ip : ''192.168.0.7'' port : ''8080'' will result in this being generated in the haproxy config file: listen cluster1 192.168.0.2:80 server server1 192.168.0.3:8080 server server2 192.168.0.4:8080 listen cluster2 192.168.0.5:80 server server3 192.168.0.6:8080 server server4 192.168.0.7:8080 So I don''t see how create_resources can handle this, as that''s for creating multiple Puppet resources, as opposed to multiple blocks within a single file. The only alternative I can think of at the moment is using create_resources with a define which utilizes Augeas, but I don''t know how well that will work. Thanks, Andy On Friday, 4 January 2013 16:47:13 UTC, Gavin Williams wrote:> > I''m with Andy on this one... I''m doing something very similar with my > NetApp volume provider (https://github.com/fatmcgav/fatmcgav-netapp/). > > I''ve created a define with the following contents: > define util::netapp::volume ( > $ensure = present, > $size, > $aggr = ''aggr1'', > $snapresv = 0, > $autoincrement = true, > $snapschedule = {"minutes" => 0, "hours" => 0, "days" => 0, > "weeks" => 0} > ) { > > netapp_notify {"volume_define_${name}": > message => "Processing Volume ${name}", > } > -> > netapp_volume { "v_${name}": > ensure => $ensure, > initsize => $size, > aggregate => $aggr, > spaceres => "none", > snapreserve => $snapresv, > autoincrement => $autoincrement, > options => {''convert_ucode'' => ''on'', ''no_atime_update'' => > ''on'', ''try_first'' => ''volume_grow''}, > snapschedule => $snapschedule > } > -> > netapp_qtree { "q_${name}": > ensure => $ensure, > volume => "v_${name}" > } > -> > netapp_export { "/vol/v_${name}/q_${name}": > ensure => $ensure, > persistent => true > } > > } > > I''ve added a default hash to ''snapschedule'' in the options list, but that > can be over-ridden from the Hiera data. > > Then use the following to pull the data from hiera and call the define: > create_resources( util::netapp::volume, hiera(''volumes'') ) > > ''Volumes'' in hiera yaml looks like: > volumes: > vol1: > ensure: present > size: ''500m'' > vol2: > ensure: present > size: ''20g'' > snapschedule: > minutes: 0 > hours: 36 > days: 0 > weeks: 0 > > You can also use the ''hiera'' command to test your yaml structure: > $ hiera -c hiera.yaml volumes clientcert=act-star-nactl01 > {"vol1"=>{"ensure"=>"present", "size"=>"500m"}, > "vol2"=>{"ensure"=>"present", "size"=>"20g", "snapschedule"=>{"days"=>0, > "weeks"=>0, "hours"=>36, "minutes"=>0}}} > > As you can see from the above output, *snapschedule* for *vol2* is a > nested hash. This assumes that your resource provider can support hashes on > the relevant param/property ;) > > HTH > > Gav > > On Friday, 4 January 2013 15:37:25 UTC, llowder wrote: >> >> >> >> On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote: >>> >>> Hi, >>> >>> I''m trying to build a module for haproxy which fetches all the >>> configuration data from Hiera to populate the haproxy config file. I''ve run >>> into a number of issues though when I try to use hashes. Ideally, I want to >>> use something like this: >>> >>> haproxy_listeners : >>> "cluster1" : >>> ip : ''192.168.0.2'' >>> port : ''80'' >>> servers : >>> "server1" : >>> ip : ''192.168.0.3'' >>> port : ''8080'' >>> >>> So a hash of clusters with each cluster containing a nested hash of >>> servers. Is this possible with Hiera/ERB? It''s easy enough to iterate over >>> the first hash, but I can''t work out how to extract the contents of the >>> nested hash. Or I might just be approaching this in entirely the wrong >>> way... Any help would be much appreciated. >>> >>> >> I haven''t used the function myself, but this looks like it would be a >> good case for a define + create_resources(), which I think is part of >> stdlib. You might need to restructure the hashes slightly, but I think that >> will be the best approach. >> >> >> >>> Thanks, >>> >>> Andy >>> >>-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/W3UBJBXuT24J. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Steven Nemetz
2013-Jan-04 17:56 UTC
RE: [Puppet Users] Re: Hiera hashes and arrays in ERB templates
I''m using stdlib to help with this $listeners = hiera(''haproxy_listeners'', undef)$listener_keys = keys($listeners) then pass $listener_keys to a define to create all the instances The entre hash is in memory ($listeners) and the define will have the key it is working on ($name)So you can access anything in the data structure to build your resources. Steven Date: Fri, 4 Jan 2013 09:05:15 -0800 From: andytayloruk@gmail.com To: puppet-users@googlegroups.com Subject: [Puppet Users] Re: Hiera hashes and arrays in ERB templates Thanks for your suggestions guys. I did consider using create_resource, but don''t see how I can when I''m trying to apply this Hiera data to a single file. To expand on my initial post, what I need to do is create multiple config blocks within one file resource. So this Hiera data: haproxy_listeners : "cluster1" : ip : ''192.168.0.2'' port : ''80'' servers : "server1" : ip : ''192.168.0.3'' port : ''8080'' "server2" : ip : ''192.168.0.4'' port : ''8080'' "cluster2" : ip : ''192.168.0.5'' port : ''80'' servers : "server3" ip : ''192.168.0.6'' port : ''8080'' "server4" ip : ''192.168.0.7'' port : ''8080'' will result in this being generated in the haproxy config file: listen cluster1 192.168.0.2:80 server server1 192.168.0.3:8080 server server2 192.168.0.4:8080 listen cluster2 192.168.0.5:80 server server3 192.168.0.6:8080 server server4 192.168.0.7:8080 So I don''t see how create_resources can handle this, as that''s for creating multiple Puppet resources, as opposed to multiple blocks within a single file. The only alternative I can think of at the moment is using create_resources with a define which utilizes Augeas, but I don''t know how well that will work. Thanks, Andy On Friday, 4 January 2013 16:47:13 UTC, Gavin Williams wrote:I''m with Andy on this one... I''m doing something very similar with my NetApp volume provider (https://github.com/fatmcgav/fatmcgav-netapp/). I''ve created a define with the following contents: define util::netapp::volume ( $ensure = present, $size, $aggr = ''aggr1'', $snapresv = 0, $autoincrement = true, $snapschedule = {"minutes" => 0, "hours" => 0, "days" => 0, "weeks" => 0} ) { netapp_notify {"volume_define_${name}": message => "Processing Volume ${name}", } -> netapp_volume { "v_${name}": ensure => $ensure, initsize => $size, aggregate => $aggr, spaceres => "none", snapreserve => $snapresv, autoincrement => $autoincrement, options => {''convert_ucode'' => ''on'', ''no_atime_update'' => ''on'', ''try_first'' => ''volume_grow''}, snapschedule => $snapschedule } -> netapp_qtree { "q_${name}": ensure => $ensure, volume => "v_${name}" } -> netapp_export { "/vol/v_${name}/q_${name}": ensure => $ensure, persistent => true } } I''ve added a default hash to ''snapschedule'' in the options list, but that can be over-ridden from the Hiera data. Then use the following to pull the data from hiera and call the define: create_resources( util::netapp::volume, hiera(''volumes'') ) ''Volumes'' in hiera yaml looks like: volumes: vol1: ensure: present size: ''500m'' vol2: ensure: present size: ''20g'' snapschedule: minutes: 0 hours: 36 days: 0 weeks: 0 You can also use the ''hiera'' command to test your yaml structure: $ hiera -c hiera.yaml volumes clientcert=act-star-nactl01 {"vol1"=>{"ensure"=>"present", "size"=>"500m"}, "vol2"=>{"ensure"=>"present", "size"=>"20g", "snapschedule"=>{"days"=>0, "weeks"=>0, "hours"=>36, "minutes"=>0}}} As you can see from the above output, snapschedule for vol2 is a nested hash. This assumes that your resource provider can support hashes on the relevant param/property ;) HTH Gav On Friday, 4 January 2013 15:37:25 UTC, llowder wrote: On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:Hi, I''m trying to build a module for haproxy which fetches all the configuration data from Hiera to populate the haproxy config file. I''ve run into a number of issues though when I try to use hashes. Ideally, I want to use something like this: haproxy_listeners : "cluster1" : ip : ''192.168.0.2'' port : ''80'' servers : "server1" : ip : ''192.168.0.3'' port : ''8080'' So a hash of clusters with each cluster containing a nested hash of servers. Is this possible with Hiera/ERB? It''s easy enough to iterate over the first hash, but I can''t work out how to extract the contents of the nested hash. Or I might just be approaching this in entirely the wrong way... Any help would be much appreciated. I haven''t used the function myself, but this looks like it would be a good case for a define + create_resources(), which I think is part of stdlib. You might need to restructure the hashes slightly, but I think that will be the best approach. Thanks, Andy -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/W3UBJBXuT24J. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
jcbollinger
2013-Jan-04 21:55 UTC
[Puppet Users] Re: Hiera hashes and arrays in ERB templates
On Friday, January 4, 2013 9:11:28 AM UTC-6, Andy Taylor wrote:> > Hi, > > I''m trying to build a module for haproxy which fetches all the > configuration data from Hiera to populate the haproxy config file. I''ve run > into a number of issues though when I try to use hashes. Ideally, I want to > use something like this: > > haproxy_listeners : > "cluster1" : > ip : ''192.168.0.2'' > port : ''80'' > servers : > "server1" : > ip : ''192.168.0.3'' > port : ''8080'' > > So a hash of clusters with each cluster containing a nested hash of > servers. Is this possible with Hiera/ERB? It''s easy enough to iterate over > the first hash, but I can''t work out how to extract the contents of the > nested hash. Or I might just be approaching this in entirely the wrong > way... Any help would be much appreciated. > >The data can be expressed in YAML, loaded into a Puppet variable via Hiera, and processed in an ERB template to construct your file. In other words, yes, it is possible. May I ask, however, what the advantage is to encoding the data in YAML and then transcoding it into the haproxy configuration language? I mean, if you also use bits and pieces of that object elsewhere in your manifests then perhaps it makes sense, but if all you want to do with it is generate the config file then I don''t see the point. Why not just serve up the config as a static or mostly-static file? Leaving aside the question of merits of your design, - Note that your example data are a quadruply-nested hash, not just a doubly-nested one as you say. You have a hash of clusters, each of which is a hash itself and contains a third hash, each of whose values is * another* hash. - The same Ruby mechanisms that allow your template to iterate and otherwise inspect the outer hash also work for the inner ones. - Perhaps, though, you are tripping over the fact that the values in your outer hash are of uniform type and meaning, whereas the values in the next-level hash are of varying type and meaning. You probably don''t actually want to iterate the second-level hash; instead, you want to retrieve specific values from it by key. So, you might do something of this general form (not intended to resemble an actual haproxy configuration file): haproxy.conf.erb: ===========<% @clusters.each_pair | cluster, properties | { -%> <cluster name="<%= cluster %>" ip="<%= properties[''ip''] %>" port="<%= properties[''port''] %>" > <% properties[''servers''].each_pair | server, server_props | { -%> <server name="<%= server %>" ip="<%= server_props[''ip''] %>" port="<%= server_props[''port''] %>" /> <% } -%> </cluster> <% } -%> Once corrected for the inevitable typos, that should produce output similar to <cluster name="cluster1" ip="192.168.0.2" port="80" > <server name="server1" ip="192.168.0.3" port="8080" /> </cluster> John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/uZSpX_tBuM8J. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.