Puppet users - Oct 2012 - Using regex to match hostnames in hiera

Hi, I''m having a problem with extlookup not respecting the
''certname'' parameter[1].  When executing a puppet run with
either the --certname or --fqdn parameters, it ends up using the specified SSL
certificate and gets the correct node definition applied from the puppetmaster. 
However, it still retrieves extlookup data using the node''s actual
FQDN, not the one manually specified using the parameter.

Anyway, I thought this might be a bug in the extlookup code so I decided to try
out hiera as an alternative.  It seems like a nice tool, but in my case there is
a very big downside - the inability to match the fqdn based on a regular
expression.  This was brought up in a previous thread[2].

As an example, here is my existing extlookup configuration from site.pp:

$hostgroup = regsubst($hostname, ''-*\d+$'',
'''')
$extlookup_datadir =
"/etc/puppet/environments/${environment}/manifests/extdata"
$extlookup_precedence = [ ''hostnames/%{fqdn}'',
''hostgroups/%{hostgroup}'', ''common'' ]

This layout is quite beautiful, as it has three possible matching scenarios.

1. A CSV file for a specific hostname exists.  Example:
hostnames/web1.mydomain.com.csv applies to a single host - web1.mydomain.com.
2. A CSV file for a regular expression based on hostname exists.  Example:
hostgroups/web.csv applies to all hosts whose hostname begins with
''web'' - web1.mydomain.com or web999.mydomain.com.
3. No matches are found, so default values from common.csv are used.

I''d like to replicate this behavior using hiera.  Is it possible?

P.S.  The first reply to the aforementioned mailing list thread suggested
creating a custom fact and using that to specify the node''s hostgroup. 
Please don''t recommend that!  To begin with, such measures
aren''t necessary with extlookup.  Hiera is the shiny/new/better
successor to extlookup, I shouldn''t have to create a custom fact to
reproduce the old functionality.  Secondly, I have another good reason but it is
complicated and would take too long to explain.  Just take my word for it :-)

Thank you.

[1] https://projects.puppetlabs.com/issues/17198
[2] https://groups.google.com/forum/?fromgroups=#!topic/puppet-users/aGFSQ2SYgL8

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

----- Original Message -----
> From: "Edwin Starkey" <edwinstarkey@yahoo.com>
> To: puppet-users@googlegroups.com
> Sent: Friday, October 26, 2012 4:45:27 AM
> Subject: [Puppet Users] Using regex to match hostnames in hiera
> 
> Hi, I''m having a problem with extlookup not respecting the
''certname''
> parameter[1].  When executing a puppet run with either the
> --certname or --fqdn parameters, it ends up using the specified SSL
> certificate and gets the correct node definition applied from the
> puppetmaster.  However, it still retrieves extlookup data using the
> node''s actual FQDN, not the one manually specified using the
> parameter.
> 
> Anyway, I thought this might be a bug in the extlookup code so I
> decided to try out hiera as an alternative.  It seems like a nice
> tool, but in my case there is a very big downside - the inability to
> match the fqdn based on a regular expression.  This was brought up
> in a previous thread[2].
> 
> As an example, here is my existing extlookup configuration from
> site.pp:
> 
> $hostgroup = regsubst($hostname, ''-*\d+$'',
'''')
> $extlookup_datadir >
"/etc/puppet/environments/${environment}/manifests/extdata"
> $extlookup_precedence = [ ''hostnames/%{fqdn}'',
> ''hostgroups/%{hostgroup}'', ''common'' ]
> 
> This layout is quite beautiful, as it has three possible matching
> scenarios.
> 
> 1. A CSV file for a specific hostname exists.  Example:
> hostnames/web1.mydomain.com.csv applies to a single host -
> web1.mydomain.com.
> 2. A CSV file for a regular expression based on hostname exists.
>   Example: hostgroups/web.csv applies to all hosts whose hostname
> begins with ''web'' - web1.mydomain.com or
web999.mydomain.com.
> 3. No matches are found, so default values from common.csv are used.
> 
> I''d like to replicate this behavior using hiera.  Is it possible?
Create a simple custom fact that has the same logic as your $hostgroup
variable, use that in the hierarchy
> P.S.  The first reply to the aforementioned mailing list thread
> suggested creating a custom fact and using that to specify the
> node''s hostgroup.  Please don''t recommend that!  To begin
with, such
> measures aren''t necessary with extlookup.  Hiera is the
> shiny/new/better successor to extlookup, I shouldn''t have to
create
> a custom fact to reproduce the old functionality.  Secondly, I have
> another good reason but it is complicated and would take too long to
> explain.  Just take my word for it :-)
hiera can access any variable in its hierarchy, so set it however you
want, however facts are there for a reason to solve this exact problem.
you should use them.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, 2012-10-25 at 20:45 -0700, Edwin Starkey wrote:
> Hi, I''m having a problem with extlookup not respecting the
''certname''
> parameter[1].  When executing a puppet run with either the --certname
> or --fqdn parameters, it ends up using the specified SSL certificate
> and gets the correct node definition applied from the puppetmaster.
> However, it still retrieves extlookup data using the node''s actual
> FQDN, not the one manually specified using the parameter.
> As an example, here is my existing extlookup configuration from
> site.pp:
> 
> $hostgroup = regsubst($hostname, ''-*\d+$'',
'''')
> $extlookup_datadir =
"/etc/puppet/environments/${environment}/manifests/extdata"
> $extlookup_precedence = [ ''hostnames/%{fqdn}'',
''hostgroups/%{hostgroup}'', ''common'' ]
The certname value is exposed in the puppet manifests in the $clientcert
variable. Something like the following might work:

$hostgroup = regsubst($clientcert, ''-*\d+$'',
'''')
$extlookup_datadir =
"/etc/puppet/environments/${environment}/manifests/extdata"
$extlookup_precedence = [ ''hostnames/%{clientcert}'',
''hostgroups/%{hostgroup}'', ''common'' ]
> This layout is quite beautiful, as it has three possible matching
> scenarios.
> 
> 1. A CSV file for a specific hostname exists.  Example:
> hostnames/web1.mydomain.com.csv applies to a single host -
> web1.mydomain.com.
> 2. A CSV file for a regular expression based on hostname exists.
> Example: hostgroups/web.csv applies to all hosts whose hostname begins
> with ''web'' - web1.mydomain.com or web999.mydomain.com.
> 3. No matches are found, so default values from common.csv are used.
> 
> I''d like to replicate this behavior using hiera.  Is it possible?
Absolutely. Hiera lookups can be done using arbitrary variables defined
in your puppet manifests. Something like the following should be
sufficient:

in site.pp toplevel:
$hostgroup = regsubst($clientcert, ''-*\d+$'',
'''')

in hiera.yaml:
:yaml:
  :datadir: /etc/puppet/environments/${environment}/manifests/hiera
:hierarchy:
  - hostnames/%{clientcert}
  - hostgroups/%{hostgroup}
  - common

Then you can use puppet3''s automatic class parameter lookup, or call
the
hiera() functions anywhere in your manifest.

-- 
Calvin Walton <calvin.walton@kepstin.ca>