About two years ago, when Stanford headed down the path of doing a large Puppet deployment, really pushing the limits of what Puppet could do at the time, we decided to wrap our findings and methodologies into a "best practices" document with Luke''s blessings. Since that time, we have made changes to the way we run things. I have finally found the time to update the best practices document to reflect that. Comments and questions are welcome: <http://reductivelabs.com/trac/puppet/wiki/PuppetBestPractice> -- Digant C Kasundra <digant@stanford.edu> Technical Lead, ITS Unix Systems and Applications, Stanford University --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Hi,
my 2 cents on your post on the best practices.
1 - What you have called template classes for me are roles: similar
logic, a class used by nodes with the same function that imports
common modules, but with a unique $role variable that may be used
mostly when sourcing files for a modules that differ just for the role
of the server.
Something like what follows has been quite useful for me, with the use
of a single httpd class for different kind of servers with the
flexibility to make per host or per role configurations.
file {
"/etc/httpd/conf/httpd.conf":
source => [
"puppet://$servername/
apache/httpd.conf-$hostname",
"puppet://$servername/
apache/httpd.conf-$role",
"puppet://$servername/
apache/httpd.conf"
]
}
Examples or roles definitions here:
http://live.lab42.it/puppetinfrastructure/browser/modules/project_general/manifests/roles.pp
2 - What you place in /manifests/ where you can actually define the
architecture of your infrastructure for me is useful to place in a
dedicated infrastructural module (the project_general in the example
before). Doing this it''s possible to present as modules different
infrastructure layouts (from quite easy to complex) with project_**
naming convention (totally arbitrary) which can be adapted and
populated with own nodes.
3- I wonder why:
class somehost_postfix inherits postfix {
# blah blah blah
}
node somehost {
include somehost_postfix
}
should not be something like:
class postfix::somehost inherits postfix {
# blah blah blah
}
node somehost {
include postfix::somehost
}
Even if generally prefer to keep class names generic, subclass names
related to twists on the main class (ie: postfix::mysql) and handle
nodes differences in the class themselves (as in the example in point
1).
4- If''ve not fully understood the meaning of the /service/ and the
real necessity of the /clients/ directories. Or better, if I''d
understood well your point, I''d place them in dedicated
project_something modules.
5- I Like the "ssh::disabled" logic and I think I''ll follow
it
Thank you for your document, I think that when you approach a tool so
flexible as puppet actually there''s not the "right" way but
the best
way for own needs.
Nevertheless seeing as other have faced and solved same, similar or
related problems is always very interesting and may give unexpected
perspectives.
I''ve used and introduced puppet in different architectures, from very
simple to quite complex and what I''ve seen (which is quite obvious
after all) is that the architecture logic shapes the puppet logic.
I''m trying to build a set of modules that can be easily adapted to
different architectures and I''ve definitvely have seen that trying to
provide a general purpose, adaptable logic, handling differect
architectures, distros, repositories, is much more difficult than
expected.
But I would like to design a puppet logic that adapts easily and
smoothly to different architecture logics.
Just at the beginning... :-)
Best regards,
al
On 21 Apr, 23:17, Digant C Kasundra <dig...@stanford.edu>
wrote:> About two years ago, when Stanford headed down the path of doing a large
> Puppet deployment, really pushing the limits of what Puppet could do at the
> time, we decided to wrap our findings and methodologies into a "best
> practices" document with Luke''s blessings. Since that time,
we have made
> changes to the way we run things. I have finally found the time to update
> the best practices document to reflect that. Comments and questions are
> welcome:
>
> <http://reductivelabs.com/trac/puppet/wiki/PuppetBestPractice>
>
> --
> Digant C Kasundra <dig...@stanford.edu>
> Technical Lead, ITS Unix Systems and Applications, Stanford University
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---
--On Wednesday, April 23, 2008 09:55:12 AM -0700 "Al @ Lab42" <lab42.it@gmail.com> wrote:> > Hi, > my 2 cents on your post on the best practices. > > 1 - What you have called template classes for me are roles: similar > logic, a class used by nodes with the same function that imports > common modules, but with a unique $role variable that may be used > mostly when sourcing files for a modules that differ just for the role > of the server. > Something like what follows has been quite useful for me, with the use > of a single httpd class for different kind of servers with the > flexibility to make per host or per role configurations. > file { > "/etc/httpd/conf/httpd.conf": > source => [ > "puppet://$servername/ > apache/httpd.conf-$hostname", > "puppet://$servername/ > apache/httpd.conf-$role", > "puppet://$servername/ > apache/httpd.conf" > ] > } > Examples or roles definitions here: > http://live.lab42.it/puppetinfrastructure/browser/modules/project_general > /manifests/roles.ppThat is an interesting approach. We actually don''t use templates at all. I leave it in just because I know others do use templates (or roles as you call them). Using the variable to indicate what role the server will use and then using logic in a class to make decisions on configuration is to me equivalent to having explicitly named classes that drop in the appropriate files. In my experience, I''ve found the explicit naming of classes to be less ambiguous when one admin is deciphering another''s catalog. When I look at a server''s catalog and I see something like ldap::master, I know that this server is configuring itself as an ldap master. But if if I had simply set a variable in the node, perhaps to indicate that role and only saw ldap in the catalog, it would be ambiguous. In your example, I would look for something like ldapmaster_role, which again is equivalent to ldap::master so really, one approach seems similar to another. But with the vagaries of variables and scoping, I prefer not to use variables to indicate these sorts of things unless facter provides that value. (For instance, what happens if someone includes scan_role and ids_role?)> > 2 - What you place in /manifests/ where you can actually define the > architecture of your infrastructure for me is useful to place in a > dedicated infrastructural module (the project_general in the example > before). Doing this it''s possible to present as modules different > infrastructure layouts (from quite easy to complex) with project_** > naming convention (totally arbitrary) which can be adapted and > populated with own nodes.Also interesting. So instead of a site.pp file, I suppose you start the puppetmaster by pointing at the init.pp file of the particular module? In general, the site.pp and nodes.pp will be going away with the completion of better node-class-mapping tools (used to be call kinial but I think Luke changed that name).> > 3- I wonder why: > class somehost_postfix inherits postfix { > # blah blah blah > } > > node somehost { > include somehost_postfix > } > should not be something like: > class postfix::somehost inherits postfix { > # blah blah blah > } > > node somehost { > include postfix::somehost > } > Even if generally prefer to keep class names generic, subclass names > related to twists on the main class (ie: postfix::mysql) and handle > nodes differences in the class themselves (as in the example in point > 1).postfix::somehost only make sense if you plan to keep all your host specific subclass in the main postfix module, which to me is counterintuitive. We generally have modules specific to each client (external department) and core infrastructure service and keep subclasses specific to those grouped with those modules. So really, what we would do is actually: class s_ldap::postfix inherits postfix { # blah blah blah } class s_ldap { include s_ldap::postfix } node ldap-server inherits basenode { include s_ldap } The problem right now is that a bug in Puppet prevents "class s_ldap::postfix inherits postfix" from working.> > 4- If''ve not fully understood the meaning of the /service/ and the > real necessity of the /clients/ directories. Or better, if I''d > understood well your point, I''d place them in dedicated > project_something modules.Service is for modules related to core infrastructure services that we run as the central information technology services department. Clients is for modules related to different servers we run for external departments. We use three module areas instead of one and wouldn''t use project_* because you lose the benefit of magic namespace mapping. It can also create isolated pockets of puppet deployments which is counter to the reason we adopted puppet: to ensure all our admins adhere to the same set of practices regardless of the servers they are supporting. So, we actively discourage sys admins from duplicating a manner in which a certain computing server (ssh for instance) is managed. I guess I would need to better understand how you are using project_*. If I understand it correctly, it sounds similar to what we intend to use environments for.> > 5- I Like the "ssh::disabled" logic and I think I''ll follow it > > Thank you for your document, I think that when you approach a tool so > flexible as puppet actually there''s not the "right" way but the best > way for own needs.Absolutely. This document started as just the Stanford Best Practice, which translates into "if you work at Stanford, you better be doing things this way or I''ll beat you with a stick," and was later generalized as a guide for others. But with different sets of needs and different business rules, everyone is going to have to come up with their own set of practices. Really, maybe that is something we should extract from my document and make more clear. I think what both you and I are absolutely agreeing on is that modules should be used wherever and whenever possible.> Nevertheless seeing as other have faced and solved same, similar or > related problems is always very interesting and may give unexpected > perspectives.I''m always curious to see how others have approached their setups.> I''ve used and introduced puppet in different architectures, from very > simple to quite complex and what I''ve seen (which is quite obvious > after all) is that the architecture logic shapes the puppet logic. > I''m trying to build a set of modules that can be easily adapted to > different architectures and I''ve definitvely have seen that trying to > provide a general purpose, adaptable logic, handling differect > architectures, distros, repositories, is much more difficult than > expected.It is indeed hard. I tried to use the best practices guide to get everyone off on the same page so that it would be easier to exchange modules if everyone took a similar approach to the architecture. But alas, that never happened. For instance, I have doubts if your modules will be easily usable/adaptable with our architecture. But only time will tell.> But I would like to design a puppet logic that adapts easily and > smoothly to different architecture logics. > > Just at the beginning... :-)That''s where you have to start, right? :) I''ll try to find time to put our Puppet infrastructure in depth on the web so others can compare notes. That will likely take a few months as we''re all pretty busy at the moment. -- Digant C Kasundra <digant@stanford.edu> Technical Lead, ITS Unix Systems and Applications, Stanford University --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Apr 23, 12:25 pm, Digant C Kasundra <dig...@stanford.edu> wrote:> > I''m always curious to see how others have approached their setups.Having no prior exposure to a configuration management tool, I approached our initial setup as described in your Best Practice document. It was coherent and sensible enough to follow, and it works pretty well for our purposes. The main ''difference'' is that we went heavy on modules, and didn''t have much of anything in the master/ manifests directory, preferring instead to create a module for everything. -joshua --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
--On Monday, April 28, 2008 01:30:25 PM -0700 jtimberman <grumpysmurf@gmail.com> wrote:> > On Apr 23, 12:25 pm, Digant C Kasundra <dig...@stanford.edu> wrote: >> >> I''m always curious to see how others have approached their setups. > > Having no prior exposure to a configuration management tool, I > approached our initial setup as described in your Best Practice > document. It was coherent and sensible enough to follow, and it works > pretty well for our purposes. The main ''difference'' is that we went > heavy on modules, and didn''t have much of anything in the master/ > manifests directory, preferring instead to create a module for > everything. > > -joshuaThat''s excellent news! We too are going heavy modules and no longer use the master area for much of anything except the site.pp and the nodes.pp files. The first best practice was drafted long before modules even existed, which is why I''m glad I finally found some time to revise and update it. -- Digant C Kasundra <digant@stanford.edu> Technical Lead, ITS Unix Systems and Applications, Stanford University --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Apr 29, 10:45 am, Digant C Kasundra <dig...@stanford.edu> wrote:> That''s excellent news! We too are going heavy modules and no longer use > the master area for much of anything except the site.pp and the nodes.pp > files. The first best practice was drafted long before modules even > existed, which is why I''m glad I finally found some time to revise and > update it.One thing to note, I know you''ve mentioned you guys aren''t using the node templates anymore. We''ve taken that idea and modified it somewhat, combining the idea of a node "type" class from David Schmitt or James Turnbull (can''t remember who, but I think it is reflected in the BP doc). Example: puppet/master/manifests/nodetype.pp: class nodetype { base node type includes classes on all systems } class nodetype::web inherits nodetype { include statements and setup for web servers } puppet/master/manifests/nodes.pp node default { include nodetype } node web1 { include nodetype::web, node specific settings } This wound up being very clean and easy to implement since we''re not using an external node classification tool. We''ve also successfully switched nodes from one nodetype class to another very quickly without impacting anything. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
--On Tuesday, April 29, 2008 10:30:50 AM -0700 jtimberman <grumpysmurf@gmail.com> wrote:> > On Apr 29, 10:45 am, Digant C Kasundra <dig...@stanford.edu> wrote: >> That''s excellent news! We too are going heavy modules and no longer use >> the master area for much of anything except the site.pp and the nodes.pp >> files. The first best practice was drafted long before modules even >> existed, which is why I''m glad I finally found some time to revise and >> update it. > > One thing to note, I know you''ve mentioned you guys aren''t using the > node templates anymore. We''ve taken that idea and modified it > somewhat, combining the idea of a node "type" class from David Schmitt > or James Turnbull (can''t remember who, but I think it is reflected in > the BP doc). Example: > > puppet/master/manifests/nodetype.pp: > class nodetype { base node type includes classes on all systems } > class nodetype::web inherits nodetype { include statements and setup > for web servers } > > puppet/master/manifests/nodes.pp > node default { include nodetype } > node web1 { include nodetype::web, node specific settings } > > This wound up being very clean and easy to implement since we''re not > using an external node classification tool. We''ve also successfully > switched nodes from one nodetype class to another very quickly without > impacting anything.That''s actually a very elegant approach. I like it! -- Digant C Kasundra <digant@stanford.edu> Technical Lead, ITS Unix Systems and Applications, Stanford University --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
> <lab42...@gmail.com> wrote: > > 1 - What you have called template classes for me are roles: similar > > logic, a class used by nodes with the same function that imports > > common modules, but with a unique $role variable that may be used > > mostly when sourcing files for a modules that differ just for the role > > of the server. > > [...] > > Examples or roles definitions here: > >http://live.lab42.it/puppetinfrastructure/browser/modules/project_gen... > > /manifests/roles.pp > > That is an interesting approach. We actually don''t use templates at all. I > leave it in just because I know others do use templates (or roles as you > call them). Using the variable to indicate what role the server will use > and then using logic in a class to make decisions on configuration is to me > equivalent to having explicitly named classes that drop in the appropriate > files. In my experience, I''ve found the explicit naming of classes to be > less ambiguous when one admin is deciphering another''s catalog. When I > look at a server''s catalog and I see something like ldap::master, I know > that this server is configuring itself as an ldap master. But if if I had > simply set a variable in the node, perhaps to indicate that role and only > saw ldap in the catalog, it would be ambiguous. In your example, I would > look for something like ldapmaster_role, which again is equivalent to > ldap::master so really, one approach seems similar to another.> But with the vagaries of variables and scoping, I prefer not to use > variables to indicate these sorts of things unless facter provides that > value. (For instance, what happens if someone includes scan_role and > ids_role?)<grumpysm...@gmail.com> wrote:> One thing to note, I know you''ve mentioned you guys aren''t using the > node templates anymore. We''ve taken that idea and modified it > somewhat, combining the idea of a node "type" class from David Schmitt > or James Turnbull (can''t remember who, but I think it is reflected in > the BP doc). Example:> puppet/master/manifests/nodetype.pp: > class nodetype { base node type includes classes on all systems } > class nodetype::web inherits nodetype { include statements and setup > for web servers }> puppet/master/manifests/nodes.pp > node default { include nodetype } > node web1 { include nodetype::web, node specific settings }I do like too this solution, and maybe I would implement it too. I''d tend to keep on using the $role variable (or similar) which I''ve found quite useful to keep a simple module structure adaptable to different "roles". In my scenario, a node must have one and only one role (to reply to Digant''s question) and possibly no node specific settings. The role is oriented to the node''s function, for example a monitoring server with Nagios would have a role colled "nagios" or "monitor" which includes also an apache module, such as another node that would have a webserver role. In the apache class differencies in httpd.conf accoring to roles (or if needed for single nodes) would be handled in this way (I generally prefer to source static files than use heavy templating): class apache { file { "httpd.conf": mode => 644, owner => root, group => root, require => Package[apache], backup => local, ensure => present, path => $operatingsystem ?{ default => "/etc/httpd/conf/ httpd.conf", }, source => [ "puppet://$servername/apache/ httpd.conf-$hostname", "puppet://$servername/apache/ httpd.conf-$role", "puppet://$servername/apache/ httpd.conf" ], } }> > > 2 - What you place in /manifests/ where you can actually define the > > architecture of your infrastructure for me is useful to place in a > > dedicated infrastructural module (the project_general in the example > > before). Doing this it''s possible to present as modules different > > infrastructure layouts (from quite easy to complex) with project_** > > naming convention (totally arbitrary) which can be adapted and > > populated with own nodes. > > Also interesting. So instead of a site.pp file, I suppose you start the > puppetmaster by pointing at the init.pp file of the particular module?Exactly. But you may decide to point to the init.pp of different project_modules, if you like/need to manage different projects with the same puppetmaster. I also would tend to keep a project specific files in different paths and using different subclasses: for example: class apache::lab42 inherits apache { file ["httpd.conf] { source => [ "puppet://$servername/apache/lab42/ httpd.conf-$hostname", "puppet://$servername/apache/lab42/ httpd.conf" ], } }> I think what both you and I are absolutely > agreeing on is that modules should be used wherever and whenever possible.Sure. My major problem is always how to make modules generic enough that can be used to: - differnet environments/projects - different operating systems - different yum repositories (is someone has tried to make a nagios- plugins module handling either Epel or RPMforge yumrepos for RedHat knows what I mean) - different software version (this is an issue that I still haven''t faced but surely needs a deeper look: what is the best way to handle, for example, different versions of apache (1.3, 2...) with not totally compatible configuration syntax? Differnet module names? Switch cases inside the apache class?)> It is indeed hard. I tried to use the best practices guide to get everyone > off on the same page so that it would be easier to exchange modules if > everyone took a similar approach to the architecture. But alas, that never > happened. For instance, I have doubts if your modules will be easily > usable/adaptable with our architecture. But only time will tell.True. for this reason before tuning, completing and enlarging my set of modules I would like to shape them in the most possibile "compatible" way, in order to ease adaptability and interchange. These discussions are quite helpful for me to try to achieve this goal. Best regards Al --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
--On Saturday, May 03, 2008 06:21:41 AM -0700 "Al @ Lab42" <lab42.it@gmail.com> wrote:> - different software version (this is an issue that I still haven''t > faced but surely needs a deeper look: what is the best way to handle, > for example, different versions of apache (1.3, 2...) with not totally > compatible configuration syntax? Differnet module names? Switch cases > inside the apache class?)This is also very interesting because we found that generic modules for a service like apache was not suitable for us. In other words, we have apache subclasses for each version of apache that builds off of a generic apache class that handles things that are similar between the two (if any). This is because we have very specific requirements about which apache we want to run on which server and this is the most explicit way to declare that. Again, it comes down to being able to look at a servers catalog and know exactly what classes are being associated with a server and being able to determine from the classname alone exactly what is being done, which is why we use classes instead of variables. I can tell from class names alone which version of apache I''m configuring on any given server. If this isn''t a requirement than a generic "handle any version" class would be more appropriate. So it depends on the intention. -- Digant C Kasundra <digant@stanford.edu> Technical Lead, ITS Unix Systems and Applications, Stanford University --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---