Puppet users - Feb 2013 - Some thoughts on design patterns and hiera includes

I have been thinking about a good way to include things in Hiera--that is, 
to insert a directive to add another stream.  The problem is one of 
attachment:  inserting things becomes complex when you think much about it.

Consider the following:

    ---
    key1:  value1
    key2:  value2
    key3:  ''%include values/me.yaml''

And a me.yaml:

    ---
    :value: value3

Does that make sense?  The result would be:

    ---
    key1:  value1
    key2:  value2
    key3:  value3

  How about this me.yaml:

    ---
    :value:
      - element3.1
      - element3.2
      - element3.3

Giving the result:

    ---
    key1:  value1
    key2:  value2
    key3:
      - element3.1
      - element3.2
      - element3.3

And predictably with a hash:

    ---
    :value:
      subkey1:  element3.1
      subkey2:  element3.2
      subkey3:  element3.3

And the result:

    ---
    key1:  value1
    key2:  value2
    key3:
      subkey1:  element3.1
      subkey2:  element3.2
      subkey3:  element3.3

We could even predict shell globs, such that the following works:

    ---
    key1:    value1
    key2:    value2
    vhosts:  ''%include vhosts/*.yaml''

Now here''s the trick:  What if you want to expand from a base?

    ---
    key1:  value1
    key2:  value2
    key3:
      - element1
      - element2
      - ''%include values/me.yaml''

And you want a result like

    ---
    key1:  value1
    key2:  value2
    key3:
      - element1
      - element2
      - element3
      - element4

You might say, "But wait, with an array this always makes sense because an 
array can''t blind-insert a value" (can it?), so something like
this would
make no sense:

    ---
    key1:  value1
    key2:  value2
    key3:
      - element1
      - element2
      -
        - element3
        - element4

But what happens when you want to blindly append to a hash?

    ---
    key1:  value1
    key2:  value2
    key3:
      subkey1:  element1
      subkey2:  element2
      ''%include values/key3.yaml''

This does not appear well-formed... and similarly unpalatable is the idea 
of using a different semantic for this case:

    ---
    key1:  value1
    key2:  value2
    key3:
      subkey1:      element1
      subkey2:      element2
      '':%include'':  values/key3.yaml

This is fairly significant because there are instances--for example,
I''m
sitting on one--where you have an Apache httpd.conf that''s, oh, what is
it... 4976 lines long, about 4500 of which are vhost definitions.

The Puppetlabs Apache module breaks vhosts out into individual files, just 
like debian, even on CentOS/RHEL.  This is a better way... trust me
I''ve
done it both ways, extensively.  I''ve worked in environments where
having a
huge httpd.conf caused major pain and interruptions to our SLAs that we 
handwaved away by having fine-print text that reads "99.99% uptime refers 
to network uptime.  We cannot guarantee against software or hardware 
failures."  Turns out multi-editing httpd.conf from vi causes problems 
sometimes.

But what you get for your trouble is 300-500 virtual hosts defined in one 
big friggin'' YAML file, just so you can create a node that calls:

    create_resources(''apache::vhost'',
hiera(''vhosts''))
    create_resources(''nginx::vhost'',
hiera(''vhosts''))

And, you see, therein lies the problem.  Sometimes you really, really want 
a hierarchy like:

    data/
         common.yaml
         nodes/
               web-01.yaml
               web/
                   vhosts-enabled/
                                  *.yaml

Especially if you have automated systems dropping files in and out of these 
directories, i.e. to create/delete/enable/disable sites.  Kind of removes 
data hotspots.

Perhaps a merge directive would be easier, like hiera_hash():

    # web.yaml
    ---
    :merge:    web/vhosts/*
    vhosts:
    # web/vhosts/vhost1.yaml
    ---
    vhosts:
      www.example.com:
        docroot:  ''/var/www/sites/www.example.com/html''
        priority: 1

Thoughts?  I kind of want to file a competent Feature Request on this that 
proscribes a good way forward.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.