Puppet users - Dec 2012 - Poll for thoughts on hierifying modules and OS default differences

Hello lovelies,

Our team is deliberating a few different options for the hierification of our
modules; and wanted to poll the collective genius before making our decision.

As we all know, there are certain attributes of a package which are, by default,
consistent on an OS... say, the name of a package.....

class foobar {
include foobar::params
  $package = $foobar::params::package
  Package {$package:
    ensure => installed
  }
}

class foobar::params {
case $osfamily:
  redhat: {
    $package = ''foobar''
  }
  debian: {
    $package = ''foobard''
  }
  default: {
    fail
  }
}


well, we want to be able to overide those defaults with hiera:

class foobar {
include foobar::params
  $package = $foobar::params::package
  Package {$package:
    ensure => installed
  }
}

class foobar::params {
case $osfamily:
  redhat: {
    $package =
hiera(''foobar_package'',''foobar'')
  }
  debian: {
    $package =
hiera(''foobar_package'',''foobard'')
  }
  default: {
    fail
  }
}


which is great, except now the package name is wrong by default on debian
boxes...

now sure, I can argue that my hierarchy should have places to logically separate
out and override those ''inconvenient'' defaults, ie for an
environment that uses debian...

one of our engineers suggested the following paradigm:


class foobar {
include foobar::params
  $package = $foobar::params::package
  Package {$package:
    ensure => installed
  }
}

class foobar::params {
case $osfamily:
  redhat: {
    $package =
hiera(''foobar_os_rhel_package'',''foobar'')
  }
  debian: {
    $package =
hiera(''foobar_os_deb_package'',''foobard'')
  }
  default: {
    fail
  }
}

which makes our topmost hierarchy (which is not segregated by OS, because the
relevant hierarchies doen''t split very cleanly at the OS level.) a
little more bloated:

foobar_os_deb_package: ''foobard''
foobar_os_rhel_package: ''foobar''


but makes for less overriding and additions for "default" setups...


to be honest, I''m not sure which idea (or what other as yet unthought
of idea) has the most merit, and I''d love to hear your thoughts on the
matter before we get too much further


Thanks so much in advance


Wolf Noble

________________________________

This message may contain confidential or privileged information. If you are not
the intended recipient, please advise us immediately and delete this message.
See http://www.datapipe.com/legal/email_disclaimer/ for further information on
confidentiality and the risks of non-secure electronic communication. If you
cannot access these links, please notify us by reply message and we will send
the contents to you.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

----- Original Message -----
> From: "Wolf Noble" <wnoble@datapipe.com>
> To: "<puppet-users@googlegroups.com>"
<puppet-users@googlegroups.com>
> Sent: Friday, December 7, 2012 8:08:40 PM
> Subject: [Puppet Users] Poll for thoughts on hierifying modules and OS
default differences
> 
> Hello lovelies,
> 
> Our team is deliberating a few different options for the
> hierification of our modules; and wanted to poll the collective
> genius before making our decision.
> 
> As we all know, there are certain attributes of a package which are,
> by default, consistent on an OS... say, the name of a package.....
> 
> class foobar {
> include foobar::params
>   $package = $foobar::params::package
>   Package {$package:
>     ensure => installed
>   }
> }
> 
> class foobar::params {
> case $osfamily:
>   redhat: {
>     $package = ''foobar''
>   }
>   debian: {
>     $package = ''foobard''
>   }
>   default: {
>     fail
>   }
> }
> 
> 
> well, we want to be able to overide those defaults with hiera:
> 
> class foobar {
> include foobar::params
>   $package = $foobar::params::package
>   Package {$package:
>     ensure => installed
>   }
> }
> 
> class foobar::params {
> case $osfamily:
>   redhat: {
>     $package =
hiera(''foobar_package'',''foobar'')
>   }
>   debian: {
>     $package =
hiera(''foobar_package'',''foobard'')
>   }
>   default: {
>     fail
>   }
> }
> 
> 
> which is great, except now the package name is wrong by default on
> debian boxes...
> 
> now sure, I can argue that my hierarchy should have places to
> logically separate out and override those ''inconvenient''
defaults,
> ie for an environment that uses debian...
> 
> one of our engineers suggested the following paradigm:
> 
> 
> class foobar {
> include foobar::params
>   $package = $foobar::params::package
>   Package {$package:
>     ensure => installed
>   }
> }
> 
> class foobar::params {
> case $osfamily:
>   redhat: {
>     $package =
hiera(''foobar_os_rhel_package'',''foobar'')
>   }
>   debian: {
>     $package =
hiera(''foobar_os_deb_package'',''foobard'')
>   }
>   default: {
>     fail
>   }
> }
> 
> which makes our topmost hierarchy (which is not segregated by OS,
> because the relevant hierarchies doen''t split very cleanly at the
OS
> level.) a little more bloated:
> 
> foobar_os_deb_package: ''foobard''
> foobar_os_rhel_package: ''foobar''
> 
> 
> but makes for less overriding and additions for "default"
setups...
> 
> 
> to be honest, I''m not sure which idea (or what other as yet
unthought
> of idea) has the most merit, and I''d love to hear your thoughts on
> the matter before we get too much further
hiera() takes 3 arguments, the last is a temporary hierarchy pre-pend.

hiera("foobar_pkg", "foobar", $::osfamily)

if your hierarchy was ["your.box.com", "common"] then for
this single
hiera lookup it would become: ["RedHat", "your.box.com",
"common"].

It''s not perfect but worth considering.  Additionally we''re
seeking
feedback on a possible approach towards dealing with this problem in
the issue http://projects.puppetlabs.com/issues/16856 it would be good
to know if this might sort you out

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I tend to take the stance that things in modules that AREN''T specific
to
your company (i.e. things that could help others - like names of packages
on other OS platforms, config file paths, etc) should be kept in the Module
and not locked up in Hiera data (that way, both people using and NOT using
Hiera can benefit from them).  I HAVE in the past put that data in Hiera,
but have also find that it can bloat the hierarchy.  I''m curious as to
what
others think, as this is my opinion and not necessarily ''best
practice''.



On Fri, Dec 7, 2012 at 10:13 AM, R.I.Pienaar <rip@devco.net> wrote:
>
> feedback on a possible approach towards dealing with this problem in



-- 

Gary Larizza
Professional Services Engineer
Puppet Labs

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

----- Original Message -----
> From: "Gary Larizza" <gary@puppetlabs.com>
> To: puppet-users@googlegroups.com
> Sent: Friday, December 7, 2012 8:35:48 PM
> Subject: Re: [Puppet Users] Poll for thoughts on hierifying modules and OS
default differences
> 
> I tend to take the stance that things in modules that AREN''T
specific
> to your company (i.e. things that could help others - like names of
> packages on other OS platforms, config file paths, etc) should be
> kept in the Module and not locked up in Hiera data (that way, both
> people using and NOT using Hiera can benefit from them). I HAVE in
> the past put that data in Hiera, but have also find that it can
> bloat the hierarchy. I''m curious as to what others think, as this
is
> my opinion and not necessarily ''best practice''.

Did you read http://projects.puppetlabs.com/issues/16856 ? it seems this
would address your concerns with this kind of information in hiera

The commit message @ https://github.com/puppetlabs/puppet/pull/1217 might
do a better job of explaining

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi Gary,


I know what you mean, and agree for modules destined to be released to the wild.
unfortunately, we have a diverse enough environment that even the defaults
occasionally need to be overridden; hence the hierification.
(I''ve got all that logic wrapped into a case statement based on if the
global parameter hiera_enabled is true or false, with sane-ish defaults when
hiera isn''t enabled)


On Dec 7, 2012, at 12:35 PM, Gary Larizza <gary@puppetlabs.com>
 wrote:
> I tend to take the stance that things in modules that AREN''T
specific to your company (i.e. things that could help others - like names of
packages on other OS platforms, config file paths, etc) should be kept in the
Module and not locked up in Hiera data (that way, both people using and NOT
using Hiera can benefit from them).  I HAVE in the past put that data in Hiera,
but have also find that it can bloat the hierarchy.  I''m curious as to
what others think, as this is my opinion and not necessarily ''best
practice''.
>

________________________________

This message may contain confidential or privileged information. If you are not
the intended recipient, please advise us immediately and delete this message.
See http://www.datapipe.com/legal/email_disclaimer/ for further information on
confidentiality and the risks of non-secure electronic communication. If you
cannot access these links, please notify us by reply message and we will send
the contents to you.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Friday, December 7, 2012 1:45:23 PM UTC-6, Wolf Noble
wrote:
>
> Hi Gary, 
>
>
> I know what you mean, and agree for modules destined to be released to the 
> wild. 
> unfortunately, we have a diverse enough environment that even the defaults 
> occasionally need to be overridden; hence the hierification. 
> (I''ve got all that logic wrapped into a case statement based on if
the
> global parameter hiera_enabled is true or false, with sane-ish defaults 
> when hiera isn''t enabled) 
>

You''re not talking about the same thing, then.  The non-site specific 
defaults Gary is talking about would be (in this case) the package names 
used by the OS distribution.  I''m inclined to agree with him that it
makes
the most sense to associate those tightly with the module, whether by 
encoding them somehow in your manifests or by putting them in an Hiera data 
file that is bound to the module and distributed with it (as R.I. has been 
demonstrating).

For site-specific details stored in hiera, you really have only two basic 
alternatives:

   1. distinguish data by the files in which they are recorded, or
   2. distinguish data by the keys with which they are associated
   
In fact, you always use some combination of both, so your question boils 
down to choosing the best mix.  Unfortunately, we can''t really answer
that
for you.  It''s too dependent on the priorities, practices, and
preferences
of your organization and its people.

Myself, I prefer to minimize conditionals in my manifests where I can by 
relying on data instead.  Thus, for example, I might consider a variation 
on your engineer''s suggestion such as this:

class foobar::params { 
  $default_package = {
    ''RedHat'' => ''foobar'',
    ''Debian'' => ''foobard''
  }

  $package = hiera("foobar_os_${::osfamily}_package",
    $default_package[$::osfamily])
}

The approach R.I. is testing amounts to doing more or less the same thing, 
but using an hiera data file belonging to the module instead of 
$foobar::params::default_package, which has the great advantage that your 
manifests don''t need to change when you add support for a new OS
family.
The approach R.I. first suggested differs from that mainly in the location 
of the additional data file and its position in the hierarchy (first 
instead of last).

Alternatively, you could go meta with it, along the lines of this model:

class foobar::params { 
  $package_key = hiera(''foo_package_key'')
  $package = hiera($package_key)
}

That abandons the idea of non-site-specific data being tightly bound to the 
module, but the extra layer of indirection through Hiera gives you a great 
deal of additional flexibility.  If that appeals to you then I''m sure
you
can see several directions to run with it.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/yb_CNvnn7G0J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I can only strongly suggest you look at the new Hiera-in-modules ticket and
pull request by R.I.Pienaar as it really solves a lot of problems many
people encounter in this situation.
It needs more testing and debugging so all help on this is very welcome.

Regards,

Stefan Goethals.

On Fri, Dec 7, 2012 at 7:08 PM, Wolf Noble <wnoble@datapipe.com> wrote:
> Hello lovelies,
>
> Our team is deliberating a few different options for the hierification of
> our modules; and wanted to poll the collective genius before making our
> decision.
>
> As we all know, there are certain attributes of a package which are, by
> default, consistent on an OS... say, the name of a package.....
>
> class foobar {
> include foobar::params
>   $package = $foobar::params::package
>   Package {$package:
>     ensure => installed
>   }
> }
>
> class foobar::params {
> case $osfamily:
>   redhat: {
>     $package = ''foobar''
>   }
>   debian: {
>     $package = ''foobard''
>   }
>   default: {
>     fail
>   }
> }
>
>
> well, we want to be able to overide those defaults with hiera:
>
> class foobar {
> include foobar::params
>   $package = $foobar::params::package
>   Package {$package:
>     ensure => installed
>   }
> }
>
> class foobar::params {
> case $osfamily:
>   redhat: {
>     $package =
hiera(''foobar_package'',''foobar'')
>   }
>   debian: {
>     $package =
hiera(''foobar_package'',''foobard'')
>   }
>   default: {
>     fail
>   }
> }
>
>
> which is great, except now the package name is wrong by default on debian
> boxes...
>
> now sure, I can argue that my hierarchy should have places to logically
> separate out and override those ''inconvenient'' defaults,
ie for an
> environment that uses debian...
>
> one of our engineers suggested the following paradigm:
>
>
> class foobar {
> include foobar::params
>   $package = $foobar::params::package
>   Package {$package:
>     ensure => installed
>   }
> }
>
> class foobar::params {
> case $osfamily:
>   redhat: {
>     $package =
hiera(''foobar_os_rhel_package'',''foobar'')
>   }
>   debian: {
>     $package =
hiera(''foobar_os_deb_package'',''foobard'')
>   }
>   default: {
>     fail
>   }
> }
>
> which makes our topmost hierarchy (which is not segregated by OS, because
> the relevant hierarchies doen''t split very cleanly at the OS
level.) a
> little more bloated:
>
> foobar_os_deb_package: ''foobard''
> foobar_os_rhel_package: ''foobar''
>
>
> but makes for less overriding and additions for "default"
setups...
>
>
> to be honest, I''m not sure which idea (or what other as yet
unthought of
> idea) has the most merit, and I''d love to hear your thoughts on
the matter
> before we get too much further
>
>
> Thanks so much in advance
>
>
> Wolf Noble
>
> ________________________________
>
> This message may contain confidential or privileged information. If you
> are not the intended recipient, please advise us immediately and delete
> this message. See http://www.datapipe.com/legal/email_disclaimer/ for
> further information on confidentiality and the risks of non-secure
> electronic communication. If you cannot access these links, please notify
> us by reply message and we will send the contents to you.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.