Puppet users - May 2008 - Complex data structures in templates

Hi.

I wonder if it is possible to use more complex data structures in
templates than simple variables and arrays? I have been trying nested
arrays, as well as nested arrays and hashes, but nothing seems to work
(more specifically, nested arrays seem to be flattened into a single
array). Having only simple variables and arrays is a little limiting.

Something like the Perl-based Template Toolkit or the PHP-based Smarty
template engine would be really nice.

Regards
Ingemar

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Mon, May 12, 2008 at 1:40 PM, Init <ingemar.nilsson1@gmail.com>
wrote:
>
>  Hi.
>
>  I wonder if it is possible to use more complex data structures in
>  templates than simple variables and arrays? I have been trying nested
>  arrays, as well as nested arrays and hashes, but nothing seems to work
>  (more specifically, nested arrays seem to be flattened into a single
>  array). Having only simple variables and arrays is a little limiting.
>
>  Something like the Perl-based Template Toolkit or the PHP-based Smarty
>  template engine would be really nice.
>
Um, any thing oyu can do in Ruby you can do in

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Mon, May 12, 2008 at 1:42 PM, Evan Hisey <ehisey@gmail.com>
wrote:
> On Mon, May 12, 2008 at 1:40 PM, Init <ingemar.nilsson1@gmail.com>
wrote:
>  >
>  >  Hi.
>  >
>  >  I wonder if it is possible to use more complex data structures in
>  >  templates than simple variables and arrays? I have been trying
nested
>  >  arrays, as well as nested arrays and hashes, but nothing seems to
work
>  >  (more specifically, nested arrays seem to be flattened into a single
>  >  array). Having only simple variables and arrays is a little
limiting.
>  >
>  >  Something like the Perl-based Template Toolkit or the PHP-based
Smarty
>  >  template engine would be really nice.
>  >
>  Um, any thing oyu can do in Ruby you can do in
Sorry about the butter fingered send. As I was typing,anything you do
in Ruby you can also do in a template. My question is why do you need
complex arrays in the tmeplates? It ounds lke you may be usin gthe
templates to do somethign that your really should not be. What are you
trying to do/manage?

evan

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On May 12, 8:45 pm, "Evan Hisey" <ehi...@gmail.com> wrote:
> Sorry about the butter fingered send. As I was typing,anything you do
> in Ruby you can also do in a template. My question is why do you need
> complex arrays in the tmeplates? It ounds lke you may be usin gthe
> templates to do somethign that your really should not be. What are you
> trying to do/manage?
One of my co-workers wrote a definition that creates a DRBD device,
with templates for the configuration files. Since I now want to create
another DRBD device on the same host, I figured that an array of
hashes would be a nice way to allow me to create multiple device
sections. But I haven''t been able to get it to work.

The problem is probbly not in the templates themselves (as they take
ordinary Ruby code), but in the data structure specifications in the
puppet manifest files. They only seem to accept a small subset of Ruby
code, e.g. throwing a hissy-fit when I try to specify a hash literal.
And then there is the flattening problem, i.e. [ a, b, [c, d], [e, [f,
g], h], i, j] seems to get flattened to an [a, b, c, d, e, f, g, h, i,
j] array.

I''m new to Ruby by the way, but I''ve read more than 100 pages
in "The
Ruby Programming Language" (O''Reilly), so I''m not
completely lost. :)

Regards
Ingemar

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On May 12, 2008, at 3:33 PM, Init wrote:
>
> On May 12, 8:45 pm, "Evan Hisey" <ehi...@gmail.com> wrote:
>
>> Sorry about the butter fingered send. As I was typing,anything you do
>> in Ruby you can also do in a template. My question is why do you need
>> complex arrays in the tmeplates? It ounds lke you may be usin gthe
>> templates to do somethign that your really should not be. What are  
>> you
>> trying to do/manage?
>
> One of my co-workers wrote a definition that creates a DRBD device,
> with templates for the configuration files. Since I now want to create
> another DRBD device on the same host, I figured that an array of
> hashes would be a nice way to allow me to create multiple device
> sections. But I haven''t been able to get it to work.
>
> The problem is probbly not in the templates themselves (as they take
> ordinary Ruby code), but in the data structure specifications in the
> puppet manifest files. They only seem to accept a small subset of Ruby
> code, e.g. throwing a hissy-fit when I try to specify a hash literal.
> And then there is the flattening problem, i.e. [ a, b, [c, d], [e, [f,
> g], h], i, j] seems to get flattened to an [a, b, c, d, e, f, g, h, i,
> j] array.
>
> I''m new to Ruby by the way, but I''ve read more than 100
pages in "The
> Ruby Programming Language" (O''Reilly), so I''m not
completely lost. :)
The difference is the puppet language is a _completely_ separate  
language.  It''s not ruby at all.  The primary resource for information
about the language is here:
http://reductivelabs.com/trac/puppet/wiki/LanguageTutorial

Templates (accessed by using the content => template("file.erb")  
syntax) are ERb, and so can do pretty much anything Ruby can do.

-Blake

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On May 12, 12:40 pm, Init <ingemar.nilss...@gmail.com>
wrote:
>
> I wonder if it is possible to use more complex data structures in
> templates than simple variables and arrays?
As mentioned, the templates are ERB and can use any amount of
arbitrary Ruby code that you want to write. However, to clarify,
Puppet only understands variables and flat arrays, so you can''t pass a
hash from Puppet to a template. You would need to pass variables, and
then build a hash from those within Ruby code in your template. That
sounds like a disaster waiting to happen, though.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Wiadomość napisana w dniu 2008-05-13, o godz 00:33, przez Init:
>>
>
> One of my co-workers wrote a definition that creates a DRBD device,
> with templates for the configuration files. Since I now want to create
> another DRBD device on the same host, I figured that an array of
> hashes would be a nice way to allow me to create multiple device
> sections. But I haven''t been able to get it to work.
>
> The problem is probbly not in the templates themselves (as they take
> ordinary Ruby code), but in the data structure specifications in the
> puppet manifest files. They only seem to accept a small subset of Ruby
> code, e.g. throwing a hissy-fit when I try to specify a hash literal.
> And then there is the flattening problem, i.e. [ a, b, [c, d], [e, [f,
> g], h], i, j] seems to get flattened to an [a, b, c, d, e, f, g, h, i,
> j] array.

Hi!

I''ve got simillar problem - I store info about what to backup in  
database, and then I''d like to use this definition in puppet template  
to configure bacula. But... What I need is hash (each key is hostname  
of machine to backup), which contains hash (basic parameters, like:  
how often to backup), and then, under key "files", I need array of  
paths to really backup. Having this informations I can construct  
template for bacula directory.

So, I use something like this:
in bacula/templates/bacula-dir.conf.erb:

<%
require "yaml"
backup_client = YAML.load(backup_client_yaml.to_s)
-%>

then you can do anywhere in template stuff like this:

<% backup_client.keys.each do |client_name| -%>
Client {
	Name = <%= client_name %>
	Address = <%= backup_client[client_name]["address"] %>
...
<% end -%>

Etc. etc.

Now you just need to put proper yaml string in $bacula_client_yaml  
variable in you puppet manifests. I do that using code in database  
app, and then external nodes, but I don''t see any problem to put that  
in your manifests, like this:

$bacula_client_yaml = "---
client2:
   files:
   - /home
   address: 192.168.0.1
client1:
   files:
   - c:/path1
   - c:/path2
   address: 127.0.0.1"

This method (that is using yaml i manifests) could be very nice - you  
can have data structure as complex as you like, you don''t need to  
touch ANY puppet internals, and it is still very easy to edit, use in  
templates, and understand. I also has many puppet functions which just  
extract data from yaml passed to puppet and, ie.: returns arrays of  
clients. Then you can do ie.:

$example_var = get_data_from_this_yaml($example_var_yaml)
define example-macro {
	file { "/etc/somedaemonconfig/$name":
		content => "hello world on $name",
	}
}
example-macro { $example_var: }

...and you created many files.

cheers
---
Grzegorz Marszałek
graf0@post.pl



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

I thought about posting something just like this, but refrained when I
thought of the pain it's caused me in the past.

Because Puppet templates are just ERB, you can do crazy things like
pass in strings (or YAML fragments, or whatever) and call eval on them
to turn them into valid Ruby objects.

Initially this seems like a fantastic idea - you can pass around all
sorts of complex data structures in your configuration and eval them
in the templates. All of a sudden your complex data structure problems
are solved, and everything is right in the world.

Then when you end up actually maintaining this in the future, you end
up realising what a horrid house of pain you have created for
yourself.

I've used this to describe some reasonably complex LDAP replication
structures (and dynamically generate shell scripts in the process),
and it just wasn't maintainable. You'll set and forget, and then when
you open up the templates in 12 months to change something you won't
be able to work out what the hell you were doing.

I strongly advise anyone thinking of this *not* to do it - there is
almost always a way to simplify the problem and break it down into
something more maintainable.

Lindsay

2008/5/30 Grzegorz Marszałek <graf0@post.pl>:
>
>
> Wiadomość napisana w dniu 2008-05-13, o godz 00:33, przez Init:
>
>>>
>>
>> One of my co-workers wrote a definition that creates a DRBD device,
>> with templates for the configuration files. Since I now want to create
>> another DRBD device on the same host, I figured that an array of
>> hashes would be a nice way to allow me to create multiple device
>> sections. But I haven't been able to get it to work.
>>
>> The problem is probbly not in the templates themselves (as they take
>> ordinary Ruby code), but in the data structure specifications in the
>> puppet manifest files. They only seem to accept a small subset of Ruby
>> code, e.g. throwing a hissy-fit when I try to specify a hash literal.
>> And then there is the flattening problem, i.e. [ a, b, [c, d], [e, [f,
>> g], h], i, j] seems to get flattened to an [a, b, c, d, e, f, g, h, i,
>> j] array.
>
>
> Hi!
>
> I've got simillar problem - I store info about what to backup in
> database, and then I'd like to use this definition in puppet template
> to configure bacula. But... What I need is hash (each key is hostname
> of machine to backup), which contains hash (basic parameters, like:
> how often to backup), and then, under key "files", I need array
of
> paths to really backup. Having this informations I can construct
> template for bacula directory.
>
> So, I use something like this:
> in bacula/templates/bacula-dir.conf.erb:
>
> <%
> require "yaml"
> backup_client = YAML.load(backup_client_yaml.to_s)
> -%>
>
> then you can do anywhere in template stuff like this:
>
> <% backup_client.keys.each do |client_name| -%>
> Client {
>        Name = <%= client_name %>
>        Address = <%= backup_client[client_name]["address"]
%>
> ...
> <% end -%>
>
> Etc. etc.
>
> Now you just need to put proper yaml string in $bacula_client_yaml
> variable in you puppet manifests. I do that using code in database
> app, and then external nodes, but I don't see any problem to put that
> in your manifests, like this:
>
> $bacula_client_yaml = "---
> client2:
>   files:
>   - /home
>   address: 192.168.0.1
> client1:
>   files:
>   - c:/path1
>   - c:/path2
>   address: 127.0.0.1"
>
> This method (that is using yaml i manifests) could be very nice - you
> can have data structure as complex as you like, you don't need to
> touch ANY puppet internals, and it is still very easy to edit, use in
> templates, and understand. I also has many puppet functions which just
> extract data from yaml passed to puppet and, ie.: returns arrays of
> clients. Then you can do ie.:
>
> $example_var = get_data_from_this_yaml($example_var_yaml)
> define example-macro {
>        file { "/etc/somedaemonconfig/$name":
>                content => "hello world on $name",
>        }
> }
> example-macro { $example_var: }
>
> ...and you created many files.
>
> cheers
> ---
> Grzegorz Marszałek
> graf0@post.pl
>
>
>
> >
>


-- 
http://slug.org.au/ (the Sydney Linux Users Group)
http://holmwood.id.au/~lindsay/ (me)

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

>
> Then when you end up actually maintaining this in the future, you end
> up realising what a horrid house of pain you have created for
> yourself.
>
Yes, that could be problem. I''ve got ruby-on-rails application, where  
my users just click on button and voila - you have backup of our  
customers data, or working dns server... But yes - it really needs a  
very good documentation, and very careful checking what your changes  
are REALLY doing. In some way you are designing new protocol -  
database to puppet communication :) And as such it should be treated  
as a development task - rather than just system administration.

---
Grzegorz Marszałek
graf0@post.pl



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---