Héctor Rivas Gándara
2010-Sep-29 12:59 UTC
[Puppet Users] Overwrite default settings in nodes using external nodes.
In my puppet environment I tryed to implement default configuration that
can be extended in child node definition.
For instance:
* All the linux SSH servers must allow connect two groups: group1 and
group2
* Each node (or classnode) should can have more groups allowed to
connect.
* Some "special" nodes can overwrite this value.
First I implemented it using parametrized defines, and using the
redefinition of the instance (I think that I can do this with parametrized
classes)
In this case:
define linux($connect_allowed_groups) {
$ssh_allowed_groups = $connect_allowed_groups
include ssh_server
}
# The general linuxserver case
node linuxserver {
linux{"base":
connect_allowed_groups => [ "group1",
"group2" ]
}
}
# A node with an extra group allowed to connect
node ''node1.mydomain.com'' inherits linuxserver {
Linux["base"]{
connect_allowed_groups +> [ "group3" ]
}
}
# A node where there is an the connect groups are overwriten
node ''node2.mydomain.com'' inherits linuxserver {
Linux["base"]{
connect_allowed_groups => [ "group1",
"group3" ]
}
}
Now I am trying to use external nodes, but in external nodes you only can
include non-parametrized classes and set parameters.
Also, from my tests I checked that the parameters (please correct me if I am
wrong):
- Are stored in global scope: All classes has access to it.
- But if a class defines that parameter, it is used the class value.
To simulate the behaviour exposed before I think that the unique way that I
think I can use is:
class linux{
case $overwrite_connect_allowed_groups {
'''': { ssh_allowed_groups
$default_connect_allowed_groups }
default: { ssh_allowed_groups $overwrite_connect_allowed_groups }
}
case $extra_connect_allowed_groups {
'''': { }
default: { ssh_allowed_groups += $extra_connect_allowed_groups }
}
include ssh::base
}
And have a external node classifier with:
- name: node1.mydomain.com
parameters:
extra_connect_allowed_groups: ["group3"]
classes:
- linux
- name: node2.mydomain.com
parameters:
overwrite_connect_allowed_groups: ["group1","group3"]
classes:
- linux
But it looks extremelly weird and unreadable, specially as it grows in
number of variables.
Other way could be use extlookup, as proposed here:
in site.pp:
$extlookup_datadir = "/etc/puppet/data/common/extdata/"
$extlookup_precedence = ["hosts/%{fqdn}",
"domain_%{domain}", "common"]
the linux class:
class linux{
ssh_allowed_groups =
extlookup(''connect_allowed_groups'')
include ssh::base
}
And just have a files for extlookup:
/etc/puppet/data/common/extdata/common.csv
connect_allowed_groups,group1,group2
/etc/puppet/data/common/extdata/hosts/node1.mydomain.com.csv
connect_allowed_groups,group1,group2,group3
/etc/puppet/data/common/extdata/hosts/node2.mydomain.com.csv
connect_allowed_groups,group1,group3
But I do not like too much the extlookup solution because:
* I can not define a common subset of groups that all host will inheret
(except the ones that overwrite this value). May be using again the
"default_connect_allowed_groups" variable.
* I think that is bad to have configuration in two places: external nodes
and cvs''s files.
Do you have an idea to implement this better?
--
Atentamente
Héctor Rivas
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Bruce Richardson
2010-Sep-29 13:46 UTC
Re: [Puppet Users] Overwrite default settings in nodes using external nodes.
On Wed, Sep 29, 2010 at 02:59:28PM +0200, Héctor Rivas Gándara wrote:> > Also, from my tests I checked that the parameters (please correct me if I am > wrong): > - Are stored in global scope: All classes has access to it.No. The parameters have "node" scope. Any class included within the node will be able to see the node''s values.> - But if a class defines that parameter, it is used the class value.Yes. If you want to be able to set a default within a class, but have that possible to be overridden, you can do something like this: class example { if $example_variable { $variable = $example_variable } else { $variable = ''Default Value'' } } Personally, I try to avoid setting default values within classes.> But I do not like too much the extlookup solution because: > * I can not define a common subset of groups that all host will inheret > (except the ones that overwrite this value). May be using again the > "default_connect_allowed_groups" variable.There''s no nice way to do it. Aas I understand it, Puppet''s external nodes code doesn''t support arrays, let alone appending to arrays. Every time I look at using external node classifiers, I walk away from it again. If I were managing sites of such a scale that it were unavoidable, I''d be an unhappy person (time to dust off the Ruby skills).> * I think that is bad to have configuration in two places: external nodes > and cvs''s files.I agree. -- Bruce A problem shared brings the consolation that someone else is now feeling as miserable as you. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Dan Bode
2010-Sep-29 17:10 UTC
Re: [Puppet Users] Overwrite default settings in nodes using external nodes.
On Wed, Sep 29, 2010 at 6:46 AM, Bruce Richardson <itsbruce@workshy.org>wrote:> On Wed, Sep 29, 2010 at 02:59:28PM +0200, Héctor Rivas Gándara wrote: > > > > Also, from my tests I checked that the parameters (please correct me if I > am > > wrong): > > - Are stored in global scope: All classes has access to it. > > No. The parameters have "node" scope. Any class included within the > node will be able to see the node''s values. > > > - But if a class defines that parameter, it is used the class value. > > Yes. If you want to be able to set a default within a class, but have > that possible to be overridden, you can do something like this: > > class example { > > if $example_variable { > $variable = $example_variable > } else { > $variable = ''Default Value'' > } > } > > Personally, I try to avoid setting default values within classes. > > > But I do not like too much the extlookup solution because: > > * I can not define a common subset of groups that all host will inheret > > (except the ones that overwrite this value). May be using again the > > "default_connect_allowed_groups" variable. > > There''s no nice way to do it. Aas I understand it, Puppet''s external > nodes code doesn''t support arrays,the external node classifier interface allows parameters to be set as arrays, the dashboard currently does not.> let alone appending to arrays.the interface is just a yaml serialization. Logic for appending arrays could be supported, but it is up to the author of the external node classifier to support it.> Every > time I look at using external node classifiers, I walk away from it > again. If I were managing sites of such a scale that it were > unavoidable, I''d be an unhappy person (time to dust off the Ruby > skills). > > > * I think that is bad to have configuration in two places: external > nodes > > and cvs''s files. > > I agree. > > -- > Bruce > > A problem shared brings the consolation that someone else is now > feeling as miserable as you. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Héctor Rivas Gándara
2010-Sep-29 17:20 UTC
Re: [Puppet Users] Overwrite default settings in nodes using external nodes.
On Wed, Sep 29, 2010 at 3:46 PM, Bruce Richardson <itsbruce@workshy.org> wrote:> On Wed, Sep 29, 2010 at 02:59:28PM +0200, Héctor Rivas Gándara wrote: >> Also, from my tests I checked that the parameters (please correct me if I am >> wrong): >> - Are stored in global scope: All classes has access to it. > > No. The parameters have "node" scope. Any class included within the > node will be able to see the node''s values.Ok, thanks. But I still don''t have really clear the different scopes. Do you know any clear documentation about it. A simple diagram like this woud be great: Scope precedence: Subclass => Included Class => Parent Class => Global => Node??>> But I do not like too much the extlookup solution because: >> * I can not define a common subset of groups that all host will inheret >> (except the ones that overwrite this value). May be using again the >> "default_connect_allowed_groups" variable. > > There''s no nice way to do it. Aas I understand it, Puppet''s external > nodes code doesn''t support arrays, let alone appending to arrays. Every > time I look at using external node classifiers, I walk away from it > again. If I were managing sites of such a scale that it were > unavoidable, I''d be an unhappy person (time to dust off the Ruby > skills).First: I have to say you that external nodes DOES support arrays, just writing [ ''a'', ''b'', ''c'' ] Well, I think that a fast solution could be create a "wrapper" script (an script that parses and modifies the output of the other) for external nodes that would implement simple inheritance and other parameter substitution. We could add a section "super: <name>" and allow the use of reference to other parameters (of actual node or parents) into the parameters. Something like this: - name: linuxserver parameters: connect_allowed_groups: ["group1", "group2"] classes: - linux - name: node1.mydomain.com super: linux parameters: connect_allowed_groups: %{connect_allowed_groups}+["group3"] - name: node2.mydomain.com super: linux parameters: connect_allowed_groups: ["group1", "group3"] This way we could easily implement inheritance in all external nodes implementations. -- Atentamente Héctor Rivas -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nigel Kersten
2010-Sep-29 17:27 UTC
Re: [Puppet Users] Overwrite default settings in nodes using external nodes.
On Wed, Sep 29, 2010 at 10:20 AM, Héctor Rivas Gándara <keymon@gmail.com> wrote:> On Wed, Sep 29, 2010 at 3:46 PM, Bruce Richardson <itsbruce@workshy.org> wrote: >> On Wed, Sep 29, 2010 at 02:59:28PM +0200, Héctor Rivas Gándara wrote: >>> Also, from my tests I checked that the parameters (please correct me if I am >>> wrong): >>> - Are stored in global scope: All classes has access to it. >> >> No. The parameters have "node" scope. Any class included within the >> node will be able to see the node''s values. > > Ok, thanks. > > But I still don''t have really clear the different scopes. Do you know > any clear documentation about it. A simple diagram like this woud be > great: > > Scope precedence: Subclass => Included Class => Parent Class => > Global => Node?? > >>> But I do not like too much the extlookup solution because: >>> * I can not define a common subset of groups that all host will inheret >>> (except the ones that overwrite this value). May be using again the >>> "default_connect_allowed_groups" variable. >> >> There''s no nice way to do it. Aas I understand it, Puppet''s external >> nodes code doesn''t support arrays, let alone appending to arrays. Every >> time I look at using external node classifiers, I walk away from it >> again. If I were managing sites of such a scale that it were >> unavoidable, I''d be an unhappy person (time to dust off the Ruby >> skills). > > First: I have to say you that external nodes DOES support arrays, just > writing [ ''a'', ''b'', ''c'' ] > > Well, I think that a fast solution could be create a "wrapper" script > (an script that parses and modifies the output of the other) for > external nodes that would implement simple inheritance and other > parameter substitution. > We could add a section "super: <name>" and allow the use of reference > to other parameters (of actual node or parents) into the parameters.We have simple inheritance in our external node classifier like this: # nodes/foo.mydomain.yaml includes: - pillar - puppetmaster parameters: puppetmaster_role: catalog_server # nodes/includes/puppetmaster --- classes: - puppetmaster parameters: puppetmaster_track: stable puppetmaster_role: catalog_server puppet_development: false puppet_autotest: false We only allow one level, and includes cannot include other includes. Parameters specified in a node yaml override the same parameter in an include yaml. Oh, and we have a default node that all of this is appended to. This is a really simple classifier, less than 100 lines of real code.> > Something like this: > > - name: linuxserver > parameters: > connect_allowed_groups: ["group1", "group2"] > classes: > - linux > > - name: node1.mydomain.com > super: linux > parameters: > connect_allowed_groups: %{connect_allowed_groups}+["group3"] > > - name: node2.mydomain.com > super: linux > parameters: > connect_allowed_groups: ["group1", "group3"] > > This way we could easily implement inheritance in all external nodes > implementations. > -- > Atentamente > Héctor Rivas > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. > >-- nigel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Héctor Rivas Gándara
2010-Sep-30 10:20 UTC
Re: [Puppet Users] Overwrite default settings in nodes using external nodes.
> We have simple inheritance in our external node classifier like this:Actually, yaml itself allows the definition of Relational Trees (http://en.wikipedia.org/wiki/YAML#Relational_trees) that allow you to do something like: - &linux name: linux parameters: one_parameter: "value1" other_parameter: "value2" classes: - linuxserver - <<: *linux name: server.domain.com environment: test parameters: another_parameter2: " y2 " txt_parameter3: " y3 " array_parameter1: [ " y_a_1.1 ", " y_a_1.2 " ] array_parameter2: [ " y_a_2.1 ", " y_a_2.2 " ] array_parameter3: [ " y_a_3.1-", " y_a_3.2 " ] array_parameter4: [ " y_a_4.1 ", " y_a_4.2 " ] classes: - linuxserver - webserver But you can not extend "classes" since it is a attribute, not a dictionary.> We only allow one level, and includes cannot include other includes. > Parameters specified in a node yaml override the same parameter in an > include yaml. > > Oh, and we have a default node that all of this is appended to. > This is a really simple classifier, less than 100 lines of real code.I will try to do something like that. -- Atentamente Héctor Rivas> > > We only allow one level, and includes cannot include other includes. > Parameters specified in a node yaml override the same parameter in an > include yaml. > > Oh, and we have a default node that all of this is appended to. > > This is a really simple classifier, less than 100 lines of real code. > > > > > Something like this: > > > > - name: linuxserver > > parameters: > > connect_allowed_groups: ["group1", "group2"] > > classes: > > - linux > > > > - name: node1.mydomain.com > > super: linux > > parameters: > > connect_allowed_groups: %{connect_allowed_groups}+["group3"] > > > > - name: node2.mydomain.com > > super: linux > > parameters: > > connect_allowed_groups: ["group1", "group3"] > > > > This way we could easily implement inheritance in all external nodes > > implementations. > > -- > > Atentamente > > Héctor Rivas > > > > -- > > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > > To post to this group, send email to puppet-users@googlegroups.com. > > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. > > > > > > > > -- > nigel > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Héctor Rivas Gándara
2010-Sep-30 13:12 UTC
Re: [Puppet Users] Overwrite default settings in nodes using external nodes.
>> We have simple inheritance in our external node classifier like this: >> We only allow one level, and includes cannot include other includes. >> Parameters specified in a node yaml override the same parameter in an >> include yaml. >> Oh, and we have a default node that all of this is appended to. >> This is a really simple classifier, less than 100 lines of real code. > > I will try to do something like that.I''ve just made a small python script that implements a simple external node manager with inheritance support. http://gist.github.com/604530 It allows define "includes" to other nodes/definitions in yaml by adding classes named "external:<name>" or adding a new attribute "include". It resolves references to other parameters using the sintax "%{parameter_name}". This way you can use something like: - name: server parameters: guest_group: "guests" classes: - generic_server - name: linux parameters: connect_allowed_groups: [ "group1", "group2" ] classes: - external:server - name: node1.mydomain.com parameters: connect_allowed_groups: [ "%{connect_allowed_groups}", "group3"] classes: - dbserver - external:linux - name: node2.mydomain.com parameters: connect_allowed_groups: ["group1","group3", "%{guest_group}"] include: - linux classes: - webserver And this is exactly what I was needing. $ ./simple-node-classifier.py node2.mydomain.com --- classes: [webserver, generic_server] include: [linux] name: node2.mydomain.com parameters: connect_allowed_groups: [group1, group3, guests] guest_group: guests $ ./simple-node-classifier.py node1.mydomain.com --- classes: [generic_server, dbserver] include: [linux] name: node1.mydomain.com parameters: connect_allowed_groups: [group1, group2, group3] guest_group: guests Right now it reads from a plain YAML file, but I will add code to allow delegate the node queries to other external command. This will allow add inheritance support to other external nodes implementations easily. Actually, I will use plain YAML files for a time. I think that is easy to manage and convenient to have all this configuration in a yaml file. And I can add it to a CVS repository. Later import it to other system (like dashboard) would be easy. In fact, I recomend all new users to start working with plain YAML files and a script like this one as external node classifier. -- Atentamente Héctor Rivas -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nigel Kersten
2010-Sep-30 13:23 UTC
Re: [Puppet Users] Overwrite default settings in nodes using external nodes.
2010/9/30 Héctor Rivas Gándara <keymon@gmail.com>:>>> We have simple inheritance in our external node classifier like this: >>> We only allow one level, and includes cannot include other includes. >>> Parameters specified in a node yaml override the same parameter in an >>> include yaml. >>> Oh, and we have a default node that all of this is appended to. >>> This is a really simple classifier, less than 100 lines of real code. >> >> I will try to do something like that. > > I''ve just made a small python script that implements a simple external > node manager with inheritance support. > > http://gist.github.com/604530 > > It allows define "includes" to other nodes/definitions in yaml by > adding classes named "external:<name>" or adding a new attribute > "include". It resolves references to other parameters using the sintax > "%{parameter_name}". > > This way you can use something like: > > - name: server > parameters: > guest_group: "guests" > classes: > - generic_server > > - name: linux > parameters: > connect_allowed_groups: [ "group1", "group2" ] > classes: > - external:server > > - name: node1.mydomain.com > parameters: > connect_allowed_groups: [ "%{connect_allowed_groups}", "group3"] > classes: > - dbserver > - external:linux > > - name: node2.mydomain.com > parameters: > connect_allowed_groups: ["group1","group3", "%{guest_group}"] > include: > - linux > classes: > - webserver > > And this is exactly what I was needing. > > $ ./simple-node-classifier.py node2.mydomain.com > --- > classes: [webserver, generic_server] > include: [linux] > name: node2.mydomain.com > parameters: > connect_allowed_groups: [group1, group3, guests] > guest_group: guests > > $ ./simple-node-classifier.py node1.mydomain.com > --- > classes: [generic_server, dbserver] > include: [linux] > name: node1.mydomain.com > parameters: > connect_allowed_groups: [group1, group2, group3] > guest_group: guests > > > > Right now it reads from a plain YAML file, but I will add code to > allow delegate the node queries to other external command. This will > allow add inheritance support to other external nodes implementations > easily. > > Actually, I will use plain YAML files for a time. I think that is easy > to manage and convenient to have all this configuration in a yaml > file. And I can add it to a CVS repository. Later import it to other > system (like dashboard) would be easy. > > In fact, I recomend all new users to start working with plain YAML > files and a script like this one as external node classifier.Absolutely. It''s really easy to write a node classifier. You can pick any language you want, you can do it however you want, you just need to supply appropriate output. Nice work on the array appending solution you came up with. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.