Puppet users - Aug 2011 - Does Node Inheritance work for people?

We have a bunch of problems people regularly run into with node inheritance,
and it''s something we''d like to find a better solution for.

Is anyone using node inheritance and happy with how it works? If so, can you
describe your setup briefly?

-- 
Nigel Kersten
Product Manager, Puppet Labs

*Join us for **PuppetConf *
<http://www.bit.ly/puppetconfsig>
Sept 22/23 Portland, Oregon, USA.
*
*

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 08/05/2011 09:33 AM, Nigel Kersten wrote:
> We have a bunch of problems people regularly run into with node 
> inheritance, and it''s something we''d like to find a
better solution for.
>
Let me guess:

         - nodes are not classes
         - node scope is not global scope

Is there something else that trips people up?

> Is anyone using node inheritance and happy with how it works? If so, 
> can you describe your setup briefly?
I just include classes in my nodes. I save the fancy stuff for classes.
I don''t expect anything from nodes, so I am never disappointed.

Here''s a suggestion: Get rid of nodes as something separate from
classes.
Consider ''node'' to be a keyword meaning ''here is a
class, automatically
include it in the manifest for these hosts''.  These automatically
assigned
classes can inherit from other classes.  Make ''node'' be the
name of the
scope for that automatically assigned class.

If there is no node statement, including no default node, then magically
create an empty class so that the node::  scope can be tested.


-- 
vagn

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

----- Original Message -----
> On 08/05/2011 09:33 AM, Nigel Kersten wrote:
> > We have a bunch of problems people regularly run into with node
> > inheritance, and it''s something we''d like to find a
better solution
> > for.
> >
> Let me guess:
> 
>          - nodes are not classes
>          - node scope is not global scope
> 
> Is there something else that trips people up?
node default {
  $foo = 1

  notify{$foo: }
}

node foo inherits default {
  $foo = 2
}

this will print 1, inheritance is badly designed since this is a common pattern

But also now I think $foo should be considered a top level variable in 2.7
when its not, making node variables more or less useless.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 08/05/2011 10:21 AM, R.I.Pienaar wrote:
> But also now I think $foo should be considered a top level variable in 2.7
> when its not, making node variables more or less useless.
>    
I like it that node variables are local and private.

node default {

         $foo = "$backupdir/${hostname}"

         backup { $foo: when => ''daily'', }
         archive { $foo: when => ''weekly'', }
}

If $foo were global it might have unintended side effects.

I think one could make a case for getting rid of global variables entirely.

-- 
vagn

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Fri, Aug 05, 2011 at 06:33:12AM -0700, Nigel Kersten
wrote:
> We have a bunch of problems people regularly run into with node
inheritance,
> and it''s something we''d like to find a better solution
for.
> 
> Is anyone using node inheritance and happy with how it works? If so, can
you
> describe your setup briefly?
I use node inheritance purely for setting variables that are relevant to
environments or roles.  So I''ll have a production node or a staging
node
and the host node may inherit from that, or from a "production
webservers" node
which inherits from production.  I give all the role and environement
based nodes names ending in ".template" to avoid potential name
clashes.

I use class inheritance for all the configuration and include classes in
the actual host node.  I use role-based classes to tie things together
and minimise the number of classes.  I make sure we never, ever include
a class in anything except the host node (not in any of the template
nodes from which it inherits), so that we always have the option to
override default values etc. without having to specially design our
classes for every eventuality.

So I''ll have a webservers "role" class.  This gives me things
like

node web01 inherits webserver.production.template {
	enabled_sites = [ ''test'', ''sample'',
''whatever'' ]
	include webserver
}

This is the only way I have found to make puppet work for me
practically.  I don''t like it much, but it works.  The inheritance
model
is, without question, my absolute least favourite thing about Puppet.
Most hated thing, to be accurate.  I hate having to specify things at
the node level when the only things declared at the node level should be
the things which are *unique* to the node.  I don''t much like having to
have parallel node and class inheritance.

A more traditional inheritance model would be much more useful for both
nodes and classes.  Oh, and as said by somebody else already, I have
always considered nodes to be a redundant duplication.  Nodes are just a
special case of classes and only serve to complicate Puppet''s
structure.
I appreciate the practical reasons why they were used in the first
place, but I think Puppet would benefit from scrapping nodes and
implementing a better inheritance model for classes.  I appreciate
that''s quite something of a redesign and would make optimisation more
difficult (at lest, starting from where we are now).

-- 
Bruce

A problem shared brings the consolation that someone else is now
feeling as miserable as you.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Aug 5, 8:33 am, Nigel Kersten <ni...@puppetlabs.com>
wrote:
> We have a bunch of problems people regularly run into with node
inheritance,
> and it''s something we''d like to find a better solution
for.
>
> Is anyone using node inheritance and happy with how it works? If so, can
you
> describe your setup briefly?

I am using node inheritance and I am happy with how it works, within
the limited scope of the things I do with it.

My setup is fairly simple:
- I use a nodes.pp manifest and no ENC.
- I have only two levels in my inheritance hierarchy, which correspond
approximately to machine type defaults (workstation vs. server) on one
level and individual machines on the other level.  All per-machine
node definitions inherit from one of the machine type definitions.
- My node definitions do nothing but include classes.  There are no
variable definitions or resource declarations.


As I tell people here from time to time, node (single-) inheritance
doesn''t fit the problem space very well because few sites admit a
satisfactory node taxonomy of any complexity.  Moreover, a lot of the
things one might conceive of doing via node inheritance can be done as
well or better with classes (classes in general, not necessarily class
inheritance).  If I were using an ENC then I would have no use for
node inheritance: I would achieve everything I currently get from node
inheritance through my ENC implementation instead.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

vagn scott:
> I think one could make a case for getting rid of global variables
> entirely.
Functionally, this is common advice for avoiding the pitfalls of
puppet''s peculiar dynamic scope.  Newcomers are often advised to use
$::foo::bar::baz to avoid picking up data from unexpected places in the
chain of invocations.

-- 
"Some of us figured out in the 1950s
that blacklists were a bad idea.
Some of us have that lesson still ahead of us."
           -- John Gilmore, on RBLs.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Aug 5, 9:38 am, vagn scott <vagnsc...@gmail.com> wrote:
> I think one could make a case for getting rid of global variables entirely.
Sort of.  There are alternatives that could fill the same role, such
as class variables of a class that every node includes, so yes, you
could dump global variables themselves, but I can''t see failing to
support that role somehow.  With that said, then, the biggest
advantage of global variables may be that they are easy for manifest
writers to use compared to the available alternatives.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 08/05/2011 01:18 PM, jcbollinger wrote:
>
> On Aug 5, 9:38 am, vagn scott<vagnsc...@gmail.com>  wrote:
>
>    
>> I think one could make a case for getting rid of global variables
entirely.
>>      
> Sort of.  There are alternatives that could fill the same role, such
> as class variables of a class that every node includes, so yes, you
> could dump global variables themselves, but I can''t see failing to
> support that role somehow.  With that said, then, the biggest
> advantage of global variables may be that they are easy for manifest
> writers to use compared to the available alternatives.
>
>
> John
>
>    
After writing that I recalled that I like
doing this:


         #! /usr/bin/puppet apply
         $prog_name = "shebang.pp"
         notice("running: $program_name")


so I take back all the nasty things I said
about global variables.

But I don''t use them in my modules, and
I would not recommend using them for anything
non-trivial.

-- 
vagn

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I''ve ALWAYS used nodes inheritance and have NEVER had problems with
them:
- no scoping issues (well just at the beginning when I didn''t realize
what I''m going to write below)
- no problems with resource defaults (I actually just set few general
defaults without overriding them).

I''ve always wondered what''s this fuzz about nodes''
inheritance "bad
reputation", even in James'' book.
You just have to take care of one thing:
- Use nodes inheritance to set and override variables NOT to include
classes.
NEVER include a class in the inheritance tree (well you can if you
don''t redefine variables used in that class but still is not
necessary).
In a nodes inheritance layout BEFORE you set the variables and THEN
you include classes (in my case "role classes").

In my typical layout I''ve one or more inheritance levels where I set
and, according to needs, override variables and in the final node (the
one with the actual node hostname) I just include a role class.
In the role class sometimes I set some more variables (typically
related to the classes included in the role), THEN include a "general"
class (which includes all the common classes) and FINALLY include the
classes and defines related to that role.

So the point IMHO is much better to explain the minimal
recommendations that the nodes'' inheritance approach require rather
than just saying that it''s wrong or not recommended. Said this
I''m not
saying that this is the BEST possible (still wonder what could be the
"best", in Puppet''s world), but that it works well and
without
problems, that scales well.

An example of such a layout is here:
https://github.com/example42/puppet-infrastructures

My2c
Al

On Aug 5, 3:33 pm, Nigel Kersten <ni...@puppetlabs.com>
wrote:
> We have a bunch of problems people regularly run into with node
inheritance,
> and it''s something we''d like to find a better solution
for.
>
> Is anyone using node inheritance and happy with how it works? If so, can
you
> describe your setup briefly?
>
> --
> Nigel Kersten
> Product Manager, Puppet Labs
>
> *Join us for **PuppetConf *
> <http://www.bit.ly/puppetconfsig>
> Sept 22/23 Portland, Oregon, USA.
> *
> *
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

[Following up a little late]
> Is anyone using node inheritance and happy with how it works? If so, can
you describe your setup briefly?
I make very limited use of node inheritance. I use

node basenode {
  include $operatingsystem
 [other stuff I want absolutely everywhere]
}

node default inherits basenode{}

node X inherits basenode {
  [more stuff]
}


I have little to add that the others haven''t said, except to emphasize
that the whole puppet inheritance model is broken.

I''m increasingly of the opinion that, were I designing puppet 3, I
would completely ditch the custom syntax in favor of a more straight-up ruby DSL
and take advantage of ruby inheritance abilities.

Darrell





-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

ENC + include/require = no need for inheritance.

Classify roles for your servers, assign classes and parameters to the roles.


On Wed, Aug 10, 2011 at 4:33 PM, Darrell Fuhriman
<darrell@garnix.org>wrote:
> [Following up a little late]
>
> Is anyone using node inheritance and happy with how it works? If so, can
> you describe your setup briefly?
>
>
> I make very limited use of node inheritance. I use
>
> node basenode {
>   include $operatingsystem
>  [other stuff I want absolutely everywhere]
> }
>
> node default inherits basenode{}
>
> node X inherits basenode {
>   [more stuff]
> }
>
>
> I have little to add that the others haven''t said, except to
emphasize that
> the whole puppet inheritance model is broken.
>
> I''m increasingly of the opinion that, were I designing puppet 3, I
would
> completely ditch the custom syntax in favor of a more straight-up ruby DSL
> and take advantage of ruby inheritance abilities.
>
> Darrell
>
>
>
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mon, Aug 08, 2011 at 06:25:20AM -0700, Alessandro Franceschi
wrote:
> NEVER include a class in the inheritance tree (well you can if you
> don''t redefine variables used in that class but still is not
> necessary).
> In a nodes inheritance layout BEFORE you set the variables and THEN
> you include classes (in my case "role classes").
This is precisely why Puppet''s inheritance model is painful.  To be
safe, you have to include all classes at the specific node.  You can
hide some of this in role-based classes, but only so much.  It causes
duplication, noise in the config and is an unnecessary potential source
of error.  Yes, if you use an ENC, you can avoid some of this by, for
example, having a script that queries a database in which the
duplication has been removed by normalisation, or by using a language
with a sane inheritance model to describe your nodes.  That, though, is
only hiding the problem and still means much more noise than is
necessary in the communication between the ECN and Puppet (which is
still a potential source of error, as the same datum transmitted many
times is more likely to be corrupted/distorted in one of those
instances).

-- 
Bruce

I unfortunately do not know how to turn cheese into gold.

We probably have the quintessential setup for using node inheritance.

We now have about 1000 nodes in a HPC cluster with CentOS 5.x. About 950
nodes are identical, and the naming scheme is simplistic (vmpxxxx). The
remaining nodes are gateways that users log into to launch jobs,
or infrastructure nodes (DNS, NTP, etc.).

We have a site.pp that imports "modules", "nodes" and
"templates."

Templates defines

node default {
  include envvars
  include selinux
}

node prodnode {
  [include all production node modules]
}

node prodgateway {
  [include all production gateway modules]
}

And a sprinkling of other special nodes.

Nodes file defines the nodes like so:

node ''vmps04.vampire'' inherits prodgateway {}
node ''vmp001.vampire'' inherits prodnode {}

The modules file imports all the modules.

The modules themselves are fairly fine grained to avoid the class this::that
syntax. We started that way, but we have eliminated it.

This works well because of the relatively simple, and massively uniform
environment with which we work.

~Charles~

On Fri, Aug 5, 2011 at 8:33 AM, Nigel Kersten <nigel@puppetlabs.com>
wrote:
> We have a bunch of problems people regularly run into with node
> inheritance, and it''s something we''d like to find a
better solution for.
>
> Is anyone using node inheritance and happy with how it works? If so, can
> you describe your setup briefly?
>
> --
> Nigel Kersten
> Product Manager, Puppet Labs
>
> *Join us for **PuppetConf *
> <http://www.bit.ly/puppetconfsig>
> Sept 22/23 Portland, Oregon, USA.
> *
> *
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

To be honest I don''t find painful at all to include at node level ONE
and only one (role) class.
I wonder where you find all this duplication. If your node names are
standardized (ie: web01 web02 web03.. ) you can even use wildcards for
all the nodes that share the same role.

I personally find more quick and handy to add a node via vi that
clicking around a web interface. And most of the times when I''ve to
add a node I just insert the node name and the role class that it
includes.
I''ve worked with this layout in different environments with various
different colleagues, more or less Puppet experts (some never used it
before) and nobody has ever made mistakes while managing nodes, and
once explained the override logic of variables in the inheritance tree
they were autonomous in managing customizations.
So somehow, for me, this is a hint, if not a proof, that this layout
is not so confusing or problematic.
Once again, I don''t intend to convince anyone that this is the best or
the only possible one, but I''m ready to demonstrate any time that it
works and scales well and, once explained is easy to work with.

My2c
Al

On Aug 11, 11:02 am, Bruce Richardson <itsbr...@workshy.org>
wrote:
> On Mon, Aug 08, 2011 at 06:25:20AM -0700, Alessandro Franceschi wrote:
> > NEVER include a class in the inheritance tree (well you can if you
> > don''t redefine variables used in that class but still is not
> > necessary).
> > In a nodes inheritance layout BEFORE you set the variables and THEN
> > you include classes (in my case "role classes").
>
> This is precisely why Puppet''s inheritance model is painful.  To
be
> safe, you have to include all classes at the specific node.  You can
> hide some of this in role-based classes, but only so much.  It causes
> duplication, noise in the config and is an unnecessary potential source
> of error.  Yes, if you use an ENC, you can avoid some of this by, for
> example, having a script that queries a database in which the
> duplication has been removed by normalisation, or by using a language
> with a sane inheritance model to describe your nodes.  That, though, is
> only hiding the problem and still means much more noise than is
> necessary in the communication between the ECN and Puppet (which is
> still a potential source of error, as the same datum transmitted many
> times is more likely to be corrupted/distorted in one of those
> instances).
>
> --
> Bruce
>
> I unfortunately do not know how to turn cheese into gold.
>
>  signature.asc
> < 1KViewDownload
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.