Puppet users - Jun 2009 - Server Hierarchies Part Two (Puppet's Revenge)

Let me apologize in advance for being dense.

When I manage systems- I organize them into hierarchies.

All my systems share certain global bits in common- NTP servers or the
root domain name for example.

All the systems within a given site or environment share other bits in
common- LDAP servers, a sub domain, default routes, etc. Specific
sites may even override some of the global settings- for example a
local NTP server for a mission critical site.

Then I have servers within those sites that once again might override
the more general site or global settings. The NTP master for a site is
not going to use itself as a server- it should use a higher precision
clock from somewhere else. There may be a special ldap base for the
research department, etc.

I can find no good way in puppet to create these sorts of hierarchies.

If I defined base nodes and try to inherit values- puppet is
declarative so later values do not get imported.

If I define classes for global, then site, then system and try to do
something like:

node ntpserver {
  $ntp_master = "accurate.example.com"
  include site::nyresearch
  include site::global
}

None of the values set in nyresearch will be available to classes in
global because of scoping rules.

How are folks making this work? Have I completely missed a totally
awesome feature or language keyword  somewhere along the line? Are
there fun tricks with inheritance that I am missing?

Explanations or links to good docs would be greatly appreciate.

If I''m trying to go about this all wrong and someone can explain the
error of my ways, that too would be appreciated.

-Don
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Don wrote:
> When I manage systems- I organize them into hierarchies.
> 
> All my systems share certain global bits in common- NTP servers or the
> root domain name for example.
> 
> All the systems within a given site or environment share other bits in
> common- LDAP servers, a sub domain, default routes, etc. Specific
> sites may even override some of the global settings- for example a
> local NTP server for a mission critical site.
> 
> Then I have servers within those sites that once again might override
> the more general site or global settings. The NTP master for a site is
> not going to use itself as a server- it should use a higher precision
> clock from somewhere else. There may be a special ldap base for the
> research department, etc.
> 
> I can find no good way in puppet to create these sorts of hierarchies.
> 
> If I defined base nodes and try to inherit values- puppet is
> declarative so later values do not get imported.
> 
> If I define classes for global, then site, then system and try to do
> something like:
> 
> node ntpserver {
>   $ntp_master = "accurate.example.com"
>   include site::nyresearch
>   include site::global
> }
> 
> None of the values set in nyresearch will be available to classes in
> global because of scoping rules.
One possibility would be to include site::global within 
site::nyresearch. Then the class would receive all variables.

Another possibility is to define a naming convention like this:

   $global_ntp
   $nyresearch_ntp
   $local_ntp

and use a custom function to make a site specific lookup in the ntp class:

   $ntp_master = site_value("ntp")




Regards, DavidS






--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hello,
> I can find no good way in puppet to create these sorts of
> hierarchies.
I use a parser function that reads data from a CSV, you can make it read from a
database or whatever.

http://nephilim.ml.org/~rip/puppet/extlookup.rb

Using this you can model all sorts of complex data structures and provide many
levels of overrides.

something as simple as: 

$ntp_master = extlookup("ntp_master")

could allow for setting a specific value per host, per domains, per country, per
vlan or any variable/fact combination available in your manifests and allow for
setting sane defaults - or just produce parser errors if no data is available.



> 
> If I defined base nodes and try to inherit values- puppet is
> declarative so later values do not get imported.
> 
> If I define classes for global, then site, then system and try to do
> something like:
> 
> node ntpserver {
>   $ntp_master = "accurate.example.com"
>   include site::nyresearch
>   include site::global
> }
> 
> None of the values set in nyresearch will be available to classes in
> global because of scoping rules.
> 
> How are folks making this work? Have I completely missed a totally
> awesome feature or language keyword  somewhere along the line? Are
> there fun tricks with inheritance that I am missing?
> 
> Explanations or links to good docs would be greatly appreciate.
> 
> If I''m trying to go about this all wrong and someone can explain
the
> error of my ways, that too would be appreciated.
> 
> -Don
> 
-- 
R.I.Pienaar

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

> One possibility would be to include site::global within
> site::nyresearch. Then the class would receive all variables.
Wow, how did I miss that? That may very well solve a bunch of my
problems- it''s not as transparent as I would prefer (you can''t
see at
a glance what resources a node references) but it has got to be better
than what I''m dealing with now. Thanks for the feedback.
>    $global_ntp
>    $nyresearch_ntp
>    $local_ntp
>
> and use a custom function to make a site specific lookup in the ntp class:
>
>    $ntp_master = site_value("ntp")
I considered doing that but it just feels like a lot of work for
something I think should just be part of the language.

Thanks for all the good tips everyone.

-Don
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

> Wow, how did I miss that? That may very well solve a bunch of my
> problems- it''s not as transparent as I would prefer (you
can''t see at
> a glance what resources a node references) but it has got to be better
> than what I''m dealing with now. Thanks for the feedback.
Nope spoke too soon. Scoping screws this up.

For example:
class zones::global {
  $ntp_servers = [''ntp01.example.com'',
                          ''ntp02.example.com'']
  $ntp_acls = ['''']
  include ntp
}

class zones::nyc {
  $ntp_servers = [''ntp01.othersite.com'',
                          ''ntp02.othersite.com'']

  include zones::global
}

node host1.example.com {
  include zones::nylrc
}

The variables in global are available to NYC but the variables I have
overridden in NYC are not available to global. In the end,
host1.example.com ends up with ntp01.example.com as it''s NTP server
instead of ntp01.othersite.com because of scoping.

Am I crazy for thinking this sort of hierarchical structure would be
amazingly useful and is basically impossible with puppet?

Is there some reason I am missing that would make the above a terrible
idea?

Without the above sort of hierarchy I''m going to end up with a jumble
of logic statements and node descriptions that could otherwise be made
incredibly simple.

What am I missing?

-Don
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

In James Turnbull''s book there''s a good discussion on the
variable
scoping issue (excellent book btw, a must have).  He offered a
workaround by defining the variable you want to override outside the
class scopes.  So you could try

$ntp_servers = [''ntp01.example.com'',
                          ''ntp02.example.com'']

class zones::global {
  $ntp_acls = ['''']
  include ntp
}

class zones::nyc {
  $ntp_servers = [''ntp01.othersite.com'',
                          ''ntp02.othersite.com'']
  include zones::global
}

node host1.example.com {
  include zones::nylrc
}


On Jun 23, 7:32 pm, Don <d...@blacksun.org> wrote:
> > Wow, how did I miss that? That may very well solve a bunch of my
> > problems- it''s not as transparent as I would prefer (you
can''t see at
> > a glance what resources a node references) but it has got to be better
> > than what I''m dealing with now. Thanks for the feedback.
>
> Nope spoke too soon. Scoping screws this up.
>
> For example:
> class zones::global {
>   $ntp_servers = [''ntp01.example.com'',
>                           ''ntp02.example.com'']
>   $ntp_acls = ['''']
>   include ntp
>
> }
>
> class zones::nyc {
>   $ntp_servers = [''ntp01.othersite.com'',
>                           ''ntp02.othersite.com'']
>
>   include zones::global
>
> }
>
> node host1.example.com {
>   include zones::nylrc
>
> }
>
> The variables in global are available to NYC but the variables I have
> overridden in NYC are not available to global. In the end,
> host1.example.com ends up with ntp01.example.com as it''s NTP
server
> instead of ntp01.othersite.com because of scoping.
>
> Am I crazy for thinking this sort of hierarchical structure would be
> amazingly useful and is basically impossible with puppet?
>
> Is there some reason I am missing that would make the above a terrible
> idea?
>
> Without the above sort of hierarchy I''m going to end up with a
jumble
> of logic statements and node descriptions that could otherwise be made
> incredibly simple.
>
> What am I missing?
>
> -Don
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

iuhh wrote:
> In James Turnbull''s book there''s a good discussion on the
variable
> scoping issue (excellent book btw, a must have).  He offered a
> workaround by defining the variable you want to override outside the
> class scopes.  So you could try
> 
Thanks! :)

There''s also some discussion at:

http://reductivelabs.com/trac/puppet/wiki/CommonMisconceptions

Regards

James Turnbull

- --
Author of:
* Pro Linux Systems Administration
(http://tinyurl.com/linuxadmin)
* Pulling Strings with Puppet
(http://tinyurl.com/pupbook)
* Pro Nagios 2.0
(http://tinyurl.com/pronagios)
* Hardening Linux
(http://tinyurl.com/hardeninglinux)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKQ0eD9hTGvAxC30ARAi2dAJ9Jim7CIkumaRjVk2eAUIJGFrdaQQCfXLoF
EIavzehdYq7l26DfvqepdGE=RkBh
-----END PGP SIGNATURE-----

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

> In James Turnbull''s book there''s a good discussion on the
variable
> scoping issue (excellent book btw, a must have).  He offered a
I have it, and while a good book, I don''t think my problems are with
understanding the issues- I just want puppet to do things it can''t.
> $ntp_servers = [''ntp01.example.com'',
>                           ''ntp02.example.com'']
>
> class zones::global {
>   $ntp_acls = ['''']
>   include ntp
> }
See- now you''re defining a role (ntp) and settings ($ntp_acls) for
that role all in the same place. I wanted to use zones to define
various settings and then just keep overriding them. If I include
classes then the scoping rules prevent the variables in my zone class
from being seen by my role class so that doesn''t work either.

What I''ve started doing is to use inheritance to build a variable
tree:

node global {
  ntp_servers = ['''']
}

node zonenyc inherits global {
  $ntp_servers = [''10..1.1.10'']
}

node client inherits zonenyc {
  include roles::general
}

node ntpmaster inherits zonenyc {
  $ntp_servers = [''pool.ntp.org'']

  include roles::general
  include roles::ntpmaster
}

It''s not nearly as clean as I would like, but at least I can override
variables and accomplish most of what I want to do.

-Don
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---