A new project has been started, called Puppet Common Modules. The goals of this project are as follows: 1) To develop a set of standards for writing puppet modules. This is currently at http://reductivelabs.com/trac/puppet/wiki/ModuleStandards . 2) To develop a set of standards for documenting puppet modules. This is currently at http://reductivelabs.com/trac/puppet/wiki/ModuleDocumentationStandards . 3) To develop a standard set of modules that users can just download and use. These modules will be tested, documented and independent. A RedMine project has been started for this at http://reductivelabs.com/redmine/projects/show/pcm . If you would like to get involved with Puppet but are not a Ruby programmer, here is your chance! We will need help writing documentation, standards and the modules themselves. I would like to start the discussion on the standards for writing and documenting modules both via email and via the wiki. In particular, generating documentation will require some sort of external tool. We are open to any ideas on what that tool should be and how the documentation should be structured. Also, the module writing standards on the wiki are a first draft and suggestions will be welcomed. Once the standards are set, everyone will be able to write and submit modules to this collection and begin using them. Martha Greenberg marthag@mit.edu --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to puppet-dev+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
Having recently seen TiddlyWiki in action, I would like to put in a strong vote for this tool. For those unfamiliar with TiddlyWiki, it''s essentially a complete Wiki in one page of html using Javascript. The value here is that it requires no back end, can be sent around like a normal document, has full Wiki capabilities, and can be checked in as part of the module code base, including being revisioned alongside of the code itself. Randy Bias, chief tactician, neoTactics, Inc. (877) NEO-TKTX, randyb@neotactics.com On Jul 29, 2008, at 6:22 PM, Martha Greenberg wrote:> In particular, generating > documentation will require some sort of external tool. We are open > to any ideas > on what that tool should be and how the documentation should be > structured--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
this is great idea! I''d love having any kind of reusable puppet assets. which modules will be there? will they be small and general or bigger and more specific? maybe we need a list of modules to start with. if I write a module or class for each system package, at the end of the day I have a set of classes with the typical “service ensure running”, the required “package ensure installed” and maybe some package configuration files with a “notify the service”. sure this is useful, but often not enough. for me al least. but if I start with a class named “my_first_mail_server” something entirely different happens. the result is a manifest that expresses all the components and complex interconnections of a very specific mail-server: the one I want to install and manage. David Escala On 30 Jul, 03:22, Martha Greenberg <mart...@mit.edu> wrote:> A new project has been started, called Puppet Common Modules. The goals of this > project are as follows: > > 1) To develop a set of standards for writing puppet modules. This is currently > athttp://reductivelabs.com/trac/puppet/wiki/ModuleStandards. > > 2) To develop a set of standards for documenting puppet modules. This is > currently athttp://reductivelabs.com/trac/puppet/wiki/ModuleDocumentationStandards. > > 3) To develop a standard set of modules that users can just download and use. > These modules will be tested, documented and independent. A RedMine project has > been started for this athttp://reductivelabs.com/redmine/projects/show/pcm. > > If you would like to get involved with Puppet but are not a Ruby programmer, > here is your chance! We will need help writing documentation, standards and the > modules themselves. > > I would like to start the discussion on the standards for writing and > documenting modules both via email and via the wiki. In particular, generating > documentation will require some sort of external tool. We are open to any ideas > on what that tool should be and how the documentation should be structured. > Also, the module writing standards on the wiki are a first draft and suggestions > will be welcomed. > > Once the standards are set, everyone will be able to write and submit modules to > this collection and begin using them. > > Martha Greenberg > mart...@mit.edu--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Bryan Kearney
2008-Jul-30 12:18 UTC
[Puppet Users] Re: Puppet Common Modules (Also, Augeas Module)
Martha Greenberg wrote:> A new project has been started, called Puppet Common Modules. The goals of this > project are as follows:<snip>> > I would like to start the discussion on the standards for writing and > documenting modules both via email and via the wiki. In particular, generating > documentation will require some sort of external tool. We are open to any ideas > on what that tool should be and how the documentation should be structured. > Also, the module writing standards on the wiki are a first draft and suggestions > will be welcomed.I had talked to David Lutterkort about sharing an augeas module [1], so this seems like a great way to do that and to discuss the common modules effort as well. First, on the module. It is built to expose the augeas tool. It accepts a set of changes as either a single string, or an array of strings. Any comments on this approach would be appreciated. The goal was to allow a single puppet task to execute multiple changes as a single unit of work. This is especially important if one change requires adding several lines to a file. Now, on to the common modules. I went through the docs, and tried to apply them to the augeas module below. The augeas module is really only a single custom type. However, I like packaging it as a module as this seems like a good level for re-distribution. The fact that it is a single custom type may influence the comments below. Here are some comments I came away with I) On standards, I have always found that it is difficult to do any sort of interesting dependencies with items in the form of Foo::Bar. Having require => [Package[X], Foo::Bar[Y]] always causes me fits. Is there a documented way to do this, or can we look to make the standard not use ::? II) For types, should the custom doc be in the type class (as I have it) or in the mainfest file? III) I have found that working on many manifests causes me to use the path to understand what file I am working on. Can we make it the standard to have the init.pp files import manifests with sensible names? IV) How should the license information be incorporated with the documentation? Should it be a separate box, or part of the main doco? V) Is there any way to pull the documentation from the plugin and use it to generate the actual documentation? Alot of it is the same information -- bk [1] http://git.et.redhat.com/?p=ace.git;a=tree;f=modules/augeas --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
> > Among the various issues of a standardization, I begin with a simple > question for a specific case I''ve faced. > This is an example, but quite common, I suppose, considering also > David Escala''s post: > How to organize in standard modules the configuration of a mailscanner > server? > - Mailscanner can be used with different MTA (postfix, sendmail...). > - It can have some useful addons (mailwatch for mailscanner is a > lovely web interface). > - A working mailscanner module for a complete single box mail server > solution (example: Postfix+MailScanner+Clamav+Spamassassin+SqlGrey > +Mailwatch+Dovecot) requires the configuration of different files of > different services. > How to organize them? > > More generally, how a to organize a module for a "major software > package or service" that requires interaction and configuration of > other "major software packages or services"? >In this vain, a standard approache to variable passing form the either the site.pp or the node decsriptions to the models should probably be developed. Evan --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
I would prefer the comments being inline with the code, not in a separate file. I don''t think there is any chance a separate file would stay up to date. Randy Bias wrote:> Having recently seen TiddlyWiki in action, I would like to put in a > strong vote for this tool. For those unfamiliar with TiddlyWiki, it''s > essentially a complete Wiki in one page of html using Javascript. The > value here is that it requires no back end, can be sent around like a > normal document, has full Wiki capabilities, and can be checked in as > part of the module code base, including being revisioned alongside of > the code itself. > > > > > Randy Bias, chief tactician, neoTactics, Inc. > (877) NEO-TKTX, randyb@neotactics.com > > > On Jul 29, 2008, at 6:22 PM, Martha Greenberg wrote: > >> In particular, generating >> documentation will require some sort of external tool. We are open >> to any ideas >> on what that tool should be and how the documentation should be >> structured > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Tue, Jul 29, 2008 at 8:38 PM, Randy Bias <randyb@neotactics.com> wrote:> > Having recently seen TiddlyWiki in action, I would like to put in a > strong vote for this tool. For those unfamiliar with TiddlyWiki, it''s > essentially a complete Wiki in one page of html using Javascript.I think the gist of the ''external documentation'' tool is something like Rdoc, where special comment markup can create nice documentation. There was discussion about this topic on IRC yesterday. TiddlyWiki is a great tool (I use it daily), but I don''t think it fits this need. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Wed, 2008-07-30 at 04:22 +0300, Martha Greenberg wrote:> A new project has been started, called Puppet Common Modules.This is great news. In a similar vein, the Fedora Infrastructure guys are trying to come up with guides for best practices for sysadmins for various situations[1], and I think they intend to back those with puppet modules. Not sure what (if any) collaboration there makes the most sense, but probably something to keep an eye on.> 1) To develop a set of standards for writing puppet modules. This is currently > at http://reductivelabs.com/trac/puppet/wiki/ModuleStandards .Some comments: * ''(1) Modules should be independent'' ... that seems to be more based on a limitation of the current tools than something that is inherently necessary. I think modules fall into two broad categories: (a) utility modules with general purpose defines etc. (b) modules that describe some aspect of a system''s configuration (in the sense of Alva Couch''s closures) These should be the defining characteristics of a module, not its dependencies * ''(2) The standard modules should be downloaded ...'' ... is again more a tools issue; a fixed modulepath is really only helpful if we have tools to help deploy modules, similar to a package manager * ''(5) Objects are defined in modules in alphabetical order'' Why ? It would make more sense to organize them in an order that somehow follows the dependencies between resources. David [1] https://fedorahosted.org/csi --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Hrm... I think I misunderstood. I''m less concerned about the inline documentation, which is handled well by an RDoc solution. I just assumed that would be so. I''m more concerned with a non-sysadmin accessible ''user guide'' for people trying to use Puppet modules at a higher level. But then we''re generally trying to make Puppet more generally accessible. I get that this particular crowd has less use for a user guide. To put it a different way, RDoc style == API docs. TiddlyWiki == user guide docs. A different audience in both cases, but perhaps it''s just something we''ll solve separately for CloudScale. Thanks, --Randy Randy Bias, chief tactician, neoTactics, Inc. (877) NEO-TKTX, randyb@neotactics.com On Jul 30, 2008, at 8:24 AM, Martha Greenberg wrote:> > I would prefer the comments being inline with the code, not in a > separate file. > I don''t think there is any chance a separate file would stay up > to date.--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
David Lutterkort schrieb:> On Wed, 2008-07-30 at 04:22 +0300, Martha Greenberg wrote: >> A new project has been started, called Puppet Common Modules. > > This is great news. [...]Yeah, really! A big "Thanks!" shout out to Martha for getting the ball rolling! You can count me definitely in and it would be great for me to donate my modules and help on further working on them (although I do have to admit that I have less time to spend recently).>> 1) To develop a set of standards for writing puppet modules. This is currently >> at http://reductivelabs.com/trac/puppet/wiki/ModuleStandards . > > Some comments: > > * ''(1) Modules should be independent'' ... that seems to be more > based on a limitation of the current tools than something that > is inherently necessary. I think modules fall into two broad > categories: (a) utility modules with general purpose defines > etc. (b) modules that describe some aspect of a system''s > configuration (in the sense of Alva Couch''s closures) These > should be the defining characteristics of a module, not its > dependencies > * ''(2) The standard modules should be downloaded ...'' ... is again > more a tools issue; a fixed modulepath is really only helpful if > we have tools to help deploy modules, similar to a package > manager > * ''(5) Objects are defined in modules in alphabetical order'' Why ? > It would make more sense to organize them in an order that > somehow follows the dependencies between resources.Exactly what I wanted to say. Additionally: 1) If two modules need the same (ruby) function, there is no way that should work[1] except to move the function to a shared module (or add a dependency from one module to the other). [1] that also means that pluginsync should probably fail if there are two definitions for the same function. 5) I can''t write how bad an idea I think this is. Later today I''ll take a shot at polishing my common module with the new Doc Standards and restructuring it a bit for a more "modern" manifest style. We can see how it progresses from there. I''ll be on IRC the next hours ... Regards, DavidS --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Mike Pountney wrote: > I think this is a very positive step, and would love to get involved.> Can I make a suggestion that we use a ''common'' namespace and module- > name prefix for each of these modules however, to distinguish them > from all the non-standard modules out there? I think most of us have > (eg) an ''ssh'' module, with the ''ssh::server'' namespace already > defined. I think that using simply ''ssh::server'' for the common module > will make migration and integration difficult. > > How about we name the modules: > > puppet_common_{name} (eg puppet_common_ssh) > > [I prefix with puppet_ as this seems to be the standard on github, > though i guess this is not necessary] > > and the underlying classes/defines in the modules: > > common::{name}::{client,server} (eg common::ssh::server) > > This way we can easily just add the modules into our manifest trees > and start using and inheriting them as we see fit, for example: > > class ssh::service inherits common::ssh:server { > file {"/etc/ssh/sshd_config": > content("ssh/my_sshd_config_template") > } > }I''m not convinced that the common:: namespace prefix would be a good idea. Quite to the contrary, I would recommend that users put their local changes into the namespace _below_ the PCMs: | class ssh::server::extra_secure inherits ssh::server { | # ... | } On the point of migration, I think the namespace clashing is the lesser problem. The fear of change in established systems is a much greater hindrance. Having a central "blessed" repository from reductivelabs will help here in the long run much more than hiding stuff in another namespace. See also Bryan Kerney''s comment I about not liking the "require => Foo::Bar[]" stuff. While I can''t say I totally support his point, I''m definitely in favour of less namespace nesting. Regards, DavidS --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
[Moving this to -users where I think this belongs] Jeroen van Meeuwen (GMail) schrieb:> David Lutterkort wrote: >> On Wed, 2008-07-30 at 04:22 +0300, Martha Greenberg wrote: >>> A new project has been started, called Puppet Common Modules. >> This is great news. In a similar vein, the Fedora Infrastructure guys >> are trying to come up with guides for best practices for sysadmins for >> various situations[1], and I think they intend to back those with puppet >> modules. Not sure what (if any) collaboration there makes the most >> sense, but probably something to keep an eye on. >> > > As one of the authors on the ConfigurationManagement document (which is > a work in progress still), and initiator of puppetmanaged.org, I will > initially be using the modules available there -which doesn''t mean no > collaboration can take place of course. > > puppetmanaged.org has a very specific target though, and that is > providing modules that do not need to be changed downstream in order to > start using them, yet still providing the alternative locations of > files, modules and manifests for (multiple-) domain specific environments. > > I hope this makes sense but if it doesn''t let me know and I''ll try to > elaborate some more.I do think that any "Common" repository will have to have modules which do not need to be locally patched to make them work in the local environment. Surely, using git and stuff makes local patches easier, but this should be mostly work that can be pushed "upstream".> I''ll certainly be lurking on this list and PRM and DavidS'' manifests and > other interesting places.See you there ;) Regards, DavidS --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to puppet-dev+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
On 1 Aug 2008, at 11:25, David Schmitt wrote:>> ... >> This way we can easily just add the modules into our manifest trees >> and start using and inheriting them as we see fit, for example: >> >> class ssh::service inherits common::ssh:server { >> file {"/etc/ssh/sshd_config": >> content("ssh/my_sshd_config_template") >> } >> } > > I''m not convinced that the common:: namespace prefix would be a good > idea. Quite to the contrary, I would recommend that users put their > local changes into the namespace _below_ the PCMs: > > | class ssh::server::extra_secure inherits ssh::server { > | # ... > | } > >But this will then force us to alter the PCMs for our own environments, making it difficult to maintain upstream changes. Using a common:: prefix (or common_ for the module directory) means that we do not need to change the PCMs for our own environments, where naturally template files and such are going to be more variable - we can override the ''golden'' PCM modules as we see fit in our own modules. I guess my point is that I see the PCMs as a useful base for our own modules - base *::server and *::client classes, useful defines, plugins, parser functions, that will help us easily create our own modules for our more tailored environments. I don''t think it is possible, certainly with the more complicated subsystems, to provide a PCM module that will handle everyone''s needs directly. We will have to alter them someway, and I think by extension into our own (non common) modules is the cleanest way of preserving the sanctity of the PCMs themselves.> On the point of migration, I think the namespace clashing is the > lesser > problem. The fear of change in established systems is a much greater > hindrance.I''m not sure what you mean here. Replacing my existing ssh module with an entirely new one that I have to understand prior to deployment, is not a task I relish, and ssh is quite a simple example. I''d rather install the PCMs alongside my existing modules and gradually shift over to using them: not through fear, but to minimise risk.> Having a central "blessed" repository from reductivelabs will > help here in the long run much more than hiding stuff in another > namespace. >This is what i would love to see - a stable, well tested, base module system. With this, we can make sensible assumptions about other peoples manifest trees, and use the PCM module functions, classes, et al as we see fit. Regards, Mike --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Mike Pountney wrote:> > On 1 Aug 2008, at 11:25, David Schmitt wrote: > >>> ... >>> This way we can easily just add the modules into our manifest trees >>> and start using and inheriting them as we see fit, for example: >>> >>> class ssh::service inherits common::ssh:server { >>> file {"/etc/ssh/sshd_config": >>> content("ssh/my_sshd_config_template") >>> } >>> } >> I''m not convinced that the common:: namespace prefix would be a good >> idea. Quite to the contrary, I would recommend that users put their >> local changes into the namespace _below_ the PCMs: >> >> | class ssh::server::extra_secure inherits ssh::server { >> | # ... >> | } >> >> > > But this will then force us to alter the PCMs for our own > environments, making it difficult to maintain upstream changes. >Actually what I think it forces you to do -nevermind the advocacy thing going on here-, is to think about how you can change the changes you make into changes that can go upstream, submit them as a proposal regardless of whether you implement them in your own environment straight-up, and then follow-up with upstream to see what might further be needed to improve adoption of your changes in the upstream module, if anything, guarding that your motivation for changing your modules in the first place isn''t being neglected as your changes move into a acceptable changeset to apply to the upstream module, and then rebasing when upstream merges in your changes. This (and here''s the advocacy I was talking about) ensures you that; - your changes are long-term sustainable as upstream takes into account changes made upstream are tested for all use-cases covered in current-release-1-minor, or migrate applicable use-cases into current-release+1-major before letting the general public put it in production stage (*uch*, insert grain of salt here) - you do not have to test, re-test, rebase, merge back in, the changes you apply to version X in order to be able to update to version Y, which saves you headaches, frustration, and in the end, a lot of time (and thus money, which is were you get your customer or employer to appreciate investing a comparatively little amount of time in getting your changes accepted by upstream) - you can go away from your customer or employer without having to instruct them to hire another expensive puppet master (no pun intended), because it''s actually upstream maintaining the modules they want to use and apply to their environment which at some point is a core-business impacting piece of infrastructure and such pieces glue together with using puppet and using standardized modules. All you need to tell your customer or employer is to hire someone who can spell "puppet" right. Kind regards, Jeroen van Meeuwen -kanarip --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Mike, I''m pretty recent to Puppet. My expectation was that there''d be a a common set of modules, and that they''d have short and obvious names. I''d have been shocked to discover I had to include common::ssh — or, worse, require => Common::something["name"]. Or is it common::Something["name"]? I''d prefer to rename my modules to custom_* or companyname_*, keep the PCM modules unprefixed, and make life easier for newcomers to Puppet. They''ll outnumber us in the medium to long term. (In a later post, you refer to the difficulty of replacing your ssh module with the common ssh module. You shouldn''t. Rename your ssh module, instead. We''re treating configurations as code, after all, so why not refactor it?) Yours, Garth. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Jeroen, this is a key point: as administrators, we want to avoid private implementations of common modules unless absolutely necessary. The whole idea with Puppet, after all, is to not have to re-invent the automation wheel every time you change jobs. If I needed features lacking in the common module, I''d fork from it rather than rewrite, and submit patches to hopefully close the gap. Yours, Garth. 2008/8/2 Jeroen van Meeuwen (GMail) <kanarip@gmail.com>: …> - you can go away from your customer or employer without having to > instruct them to hire another expensive puppet master (no pun intended), > because it''s actually upstream maintaining the modules they want to use > and apply to their environment which at some point is a core-business > impacting piece of infrastructure and such pieces glue together with > using puppet and using standardized modules. All you need to tell your > customer or employer is to hire someone who can spell "puppet" right.--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Garth Roxburgh-Kidd wrote:> Jeroen, this is a key point: as administrators, we want to avoid > private implementations of common modules unless absolutely necessary. > The whole idea with Puppet, after all, is to not have to re-invent the > automation wheel every time you change jobs. > > If I needed features lacking in the common module, I''d fork from it > rather than rewrite, and submit patches to hopefully close the gap. >Well, that is your choice and I respect that, as I think will everyone else on this planet, but I hope you can appreciate where the statements I''ve made are, to some extend, valuable for you and anyone else to consider, maybe on a case-by-case basis, regardless of whether you choose to go by them or ignore them. Kind regards, Jeroen van Meeuwen -kanarip --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Garth Roxburgh-Kidd wrote:> Jeroen, this is a key point: as administrators, we want to avoid > private implementations of common modules unless absolutely necessary. > The whole idea with Puppet, after all, is to not have to re-invent the > automation wheel every time you change jobs. > > If I needed features lacking in the common module, I''d fork from it > rather than rewrite, and submit patches to hopefully close the gap. >Hmm... reading your message a second time I think I may have misunderstood you the first time I read it. My bad if that''s the case, I''m sorry, and thank you. Kind regards, Jeroen "not a native english speaker" van Meeuwen -kanarip --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Jeroen, I apologise for not being clearer. Indeed, I agree: if there''s a decent common module, it''s odd to write your own. If the common module lacks something you need, you should minimize your changes and submit them as a patch. The Python community has a term: "Pythonic". The language is absurdly clean. If your code isn''t clean, there''s probably something wrong. This is doubly true for libraries. If using your library makes my code look odd, there''s probably something wrong with the design of your library. Similarly: Puppet minimizes — almost eliminates — the effort of controlling the configuration of large numbers of hosts. If you find yourself spending as much time hand-coding your Puppet modules as you use to spend administering machines, there''s probably something wrong. Sadly, "puppetic" sounds awful. Yours, Garth. 2008/8/2 Jeroen van Meeuwen (GMail) <kanarip@gmail.com>:> > Garth Roxburgh-Kidd wrote: >> Jeroen, this is a key point: as administrators, we want to avoid >> private implementations of common modules unless absolutely necessary. >> The whole idea with Puppet, after all, is to not have to re-invent the >> automation wheel every time you change jobs. >> >> If I needed features lacking in the common module, I''d fork from it >> rather than rewrite, and submit patches to hopefully close the gap. >> > > Hmm... reading your message a second time I think I may have > misunderstood you the first time I read it. My bad if that''s the case, > I''m sorry, and thank you. > > Kind regards, > > Jeroen "not a native english speaker" van Meeuwen > -kanarip > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Saturday 02 August 2008, Mike Pountney wrote:> On 1 Aug 2008, at 11:25, David Schmitt wrote: > >> ... > >> This way we can easily just add the modules into our manifest trees > >> and start using and inheriting them as we see fit, for example: > >> > >> class ssh::service inherits common::ssh:server { > >> file {"/etc/ssh/sshd_config": > >> content("ssh/my_sshd_config_template") > >> } > >> } > > > > I''m not convinced that the common:: namespace prefix would be a good > > idea. Quite to the contrary, I would recommend that users put their > > > > local changes into the namespace _below_ the PCMs: > > | class ssh::server::extra_secure inherits ssh::server { > > | # ... > > | } > > But this will then force us to alter the PCMs for our own > environments, making it difficult to maintain upstream changes. > > Using a common:: prefix (or common_ for the module directory) means > that we do not need to change the PCMs for our own environments, where > naturally template files and such are going to be more variable - we > can override the ''golden'' PCM modules as we see fit in our own modules.This is a strawman. Nobody needs to touch anything within the ssh module to create a ssh::server::extra_secure class. Of course, autoloading won''t help you with this design, but that''s no big deal for local classes and surely less a problem than getting autoload to work around the common:: prefix.> > On the point of migration, I think the namespace clashing is the > > lesser > > problem. The fear of change in established systems is a much greater > > hindrance. > > I''m not sure what you mean here. Replacing my existing ssh module with > an entirely new one that I have to understand prior to deployment, is > not a task I relish, and ssh is quite a simple example. I''d rather > install the PCMs alongside my existing modules and gradually shift > over to using them: not through fear, but to minimise risk.One can do a quite risk-less global rename of the local ssh module to ssh_local to avoid having to replace it immediately. Either way won''t spare you the neccessary migration and testing. But using a common:: prefix breaks autoloading and will be a millstone around everyones neck forever. Regards, DavidS - -- The primary freedom of open source is not the freedom from cost, but the free- dom to shape software to do what you want. This freedom is /never/ exercised without cost, but is available /at all/ only by accepting the very different costs associated with open source, costs not in money, but in time and effort. - -- http://www.schierer.org/~luke/log/20070710-1129/on-forks-and-forking -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iD8DBQFIlGHi/Pp1N6Uzh0URAjseAKCDw9LfSxQhhk7gD7NSYOVwVF8mzACfeIIh Es4SXBn3hXTYpiHvWbhUKpI=wNdW -----END PGP SIGNATURE----- --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 David Schmitt wrote:> One can do a quite risk-less global rename of the local ssh module to > ssh_local to avoid having to replace it immediately. Either way won''t spare > you the neccessary migration and testing. But using a common:: prefix breaks > autoloading and will be a millstone around everyones neck forever.+1. Regards James - -- Author of: * Pulling Strings with Puppet (http://www.amazon.com/gp/product/1590599780/) * Pro Nagios 2.0 (http://www.amazon.com/gp/product/1590596099/) * Hardening Linux (http://www.amazon.com/gp/product/1590594444/) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIlJDD9hTGvAxC30ARAqxlAJ9AMy+A5QkeoqgVvyLhAod/EFmXwgCfUKJA 2yzAjkgT6LPeql1UEBwaaB4=prdp -----END PGP SIGNATURE----- --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
(*) Firstly, after reading the responses to my post, I recognise that my ''common::'' idea (''scuse the pun) is not sensible. Stop reading now if you like :) Hopefully this discussion has raised some interesting points though (which have taught me some new puppet techniques, woo!), hence the comments to the comments below: On 2 Aug 2008, at 14:32, David Schmitt wrote:>> >> Using a common:: prefix (or common_ for the module directory) means >> that we do not need to change the PCMs for our own environments, >> where >> naturally template files and such are going to be more variable - we >> can override the ''golden'' PCM modules as we see fit in our own >> modules. > > This is a strawman. Nobody needs to touch anything within the ssh > module to > create a ssh::server::extra_secure class. Of course, autoloading > won''t help > you with this design, but that''s no big deal for local classes and > surely > less a problem than getting autoload to work around the common:: > prefix. >Good point, I''d never considered the fact that the namespaces and module ''packaging'' are separate things, (with the exception of autoloading, as you mention below).> > One can do a quite risk-less global rename of the local ssh module to > ssh_local to avoid having to replace it immediately. Either way > won''t spare > you the neccessary migration and testing.In my particular example, it''s also going to involve me renaming many of my classes and defines. It''s a reasonable job, but not a huge task I admit. This is a minor quibble compared to the benefit new users get from PCM modules though. See (*) ;)> But using a common:: prefix breaks > autoloading and will be a millstone around everyones neck forever. >I see what you mean. This would force the use of ''puppet_common_ssh::server'' as the module name (based on my suggestion of using puppet_common_ssh as the module name), which is just horrid. See (*) My main reasoning behind this discussion was that the PCM modules are likely to be fairly immutable, yet will need to be altered by some to fit into their (eg) company policy or existing environments. Templates in particular are likely going to need to be changed by people, in ways that are not necessarily going to be useful to upstream. Puppet does (as you show above) provide many ways to allow us to enhance/alter the functionality of them locally, without requiring changing the PCM modules themselves. This makes me happy, as it solves my problem. I guess my point then is, in the PCM Best Practices, how would we recommend that this is done? Options so far seem to be (forgive my rough notes): *) Extend the PCM namespace, as per David''s ssh::server::secure_config, in a separate module (my_ssh) - maintains the namespace hierarchy - breaks autoloading *) Create a new top-level namespace, eg my_ssh::server, in my_ssh module - avoids namespace clashes with the PCM module - breaks the namespace hierarchy (eg auto-tagging, will need to do puppetd -v -o --tags ''my_ssh,ssh'' rather than just --tags ''ssh'') *) Fork/Branch the PCM module - changes are easy to make, no new modules to create. - requires branch management/merging to keep PCM module up-to-date. - best solution for changes that are expected to move upstream. Regards, Mike --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Saturday 02 August 2008, Mike Pountney wrote:> (*) Firstly, after reading the responses to my post, I recognise that > my ''common::'' idea (''scuse the pun) is not sensible. Stop reading now > if you like :)Not at all, people who learn are much more interesting to read, than people who don''t!> Options so far seem to be (forgive my rough notes): > > *) Extend the PCM namespace, as per David''s > ssh::server::secure_config, in a separate module (my_ssh) > - maintains the namespace hierarchy > - breaks autoloading > > *) Create a new top-level namespace, eg my_ssh::server, in my_ssh module > - avoids namespace clashes with the PCM module > - breaks the namespace hierarchy (eg auto-tagging, will need to do > puppetd -v -o --tags ''my_ssh,ssh'' rather than just --tags ''ssh'') > > *) Fork/Branch the PCM module > - changes are easy to make, no new modules to create. > - requires branch management/merging to keep PCM module up-to-date. > - best solution for changes that are expected to move upstream.Don''t forget that local classes do not _have_ to be in modules. A simple "import ''local_changes/*.pp''" can help much here. Therefore I think the first option would be the best. Regards, DavidS - -- The primary freedom of open source is not the freedom from cost, but the free- dom to shape software to do what you want. This freedom is /never/ exercised without cost, but is available /at all/ only by accepting the very different costs associated with open source, costs not in money, but in time and effort. - -- http://www.schierer.org/~luke/log/20070710-1129/on-forks-and-forking -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iD8DBQFIlYJn/Pp1N6Uzh0URAiDzAJ9+XjkSnnqKQvF1zTbAoEw+0QhkdgCeJ+vW dR128zPe/WPhqSkj+qsRfwk=8cPp -----END PGP SIGNATURE----- --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Martha Greenberg
2008-Aug-03 14:15 UTC
[Puppet Users] Re: [Puppet-dev] Re: [Puppet Users] Puppet Common Modules
I''m resending this to puppet-users, since a lot of same questions are coming up here: The truth is, that PCM is really three projects: 1) Coming up with a set of standards for the "right" way to write a common module, so it can be used by as many people as possible. In particular, setting clear ways to handle as many OSs as possible, and getting the modules to be stand-alone. I would like Puppet beginners to be able to download starter modules and have them just work. The modules that are out right now are great. However, they are split into lots of different repositories, they often only work on specific OSs, and they contain lots of site-specific code. This means they are not as useful as they could be. Most of the time, a user cannot use them as-is. Here''s what I''d like the new Puppet user experience to be: 1) download puppet 2) check out/download PCM 3) modulepath=/etc/puppet/modules:/etc/puppet/pcm-modules 4) user set up nodes with pcm modules. 5) oh wait, you need a different sshd config file? set this up: class mysite::ssh-server inherits ssh::server { File["sshd_config"] source => "puppet:///mysite/sshd_config" } or class myssh:server inherits ssh::server {} 2) Devising a way to document modules. Ideally, coming up with a tool that could create stub docs from puppet code. This should probably be split out into a separate project. 3) Getting current and new module developers under one roof and taking the existing modules and standardizing and documenting them. I don''t think we should be starting from scratch. But right now, we have no way of coordinating this work, or letting other people contribute. Also, I would like to reduce the amount of duplication. Most likely, everyone on this list has written an SSH module. 90% of us could use the same module, if it was done right. Just like Puppet abstracts out a lot of repetitive sysadmin work, I''d like PCM to take that a step further... Coordinating and organizing these projects is certainly much hard than the technical aspects. I''m willing to work on this, but we need lots of help. The announcement was intended more as a call for volunteers and a way to start the discussion than "This is how things are going to be". Martha PS - On a another note, I see this project as a good way to give people who are not Ruby programmers a way to contribute to Puppet. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Jeroen van Meeuwen (GMail)
2008-Aug-03 16:23 UTC
[Puppet Users] Re: [Puppet-dev] Re: [Puppet Users] Puppet Common Modules
Martha Greenberg wrote:> I''m resending this to puppet-users, since a lot of same questions are coming up > here: > > > The truth is, that PCM is really three projects: > > 1) Coming up with a set of standards for the "right" way to write a common > module, so it can be used by as many people as possible. In particular, setting > clear ways to handle as many OSs as possible, and getting the modules to be > stand-alone. I would like Puppet beginners to be able to download starter > modules and have them just work. >A concern here on my part is that if the modules are required to be entirely standalone, a lot of duplication (in different namespaces) is going to happen. Just one example is managing puppet (and the puppetmaster) with a puppet module, that enables one to have puppet pull updated module sources from an upstream SCM (branching included, so that MultipleEnvironments are properly facilitated). Right now this requires one to also have the module for the preferred SCM in use. Although that might not be a priority for PCM right now, I would like to take into consideration to not require the modules to be stand-alone. Then again, maybe, because of the stand-alone requirement, such a puppet module should not require the SCM module and not pull from upstream automagically, until you set $pull_from_scm = 1 (and only then require the SCM module). This we should take into very careful consideration as I think having common puppet modules is one thing, and then having a million settings you need to go through to get it to do anything is another. One of the other things I think common puppet modules are also great for, is to enable a sort of de-facto standard in common system administration, although again that might not at all be a priority at this point.> 2) Devising a way to document modules. Ideally, coming up with a tool that > could create stub docs from puppet code. This should probably be split out into > a separate project. >I''m afraid the documentation needed for some of the modules out there (and I don''t mean the ssh module everyone has), needs to be so extensive as to explain use-cases in a clear and understandable way that it''ll be obfuscating the code just a little too much. I think little code vs. lots of comments/documentation preferably does not come from the same file. Just some thoughts ;-) Kind regards, Jeroen van Meeuwen -kanarip --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Hi all, I''m not very active on the mailinglist, but... On 3 aug, 16:15, Martha Greenberg <mart...@mit.edu> wrote:> PS - On a another note, I see this project as a good way to give people who are > not Ruby programmers a way to contribute to Puppet.... just wanted to say that I really support that and will be sure to spend some time on this. Especially since we plan to release our internal modules soon. (David asked me to voice my support, so consider this my vote of confidence! ;-)) -- Kind regards, Tim --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
--On Wednesday, July 30, 2008 04:22:20 AM +0300 Martha Greenberg <marthag@mit.edu> wrote:> > 1) To develop a set of standards for writing puppet modules. This is > currently at http://reductivelabs.com/trac/puppet/wiki/ModuleStandards .I''m not sure I like the ::common naming scheme. To me, ssh::server implies a class that handles the most common case and the less common would inherit from that and go in a namespace below that (ssh::server::weird). To me, this more closely resembles an "object-oriented" model. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to puppet-dev+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
--On Friday, August 01, 2008 12:25:28 PM +0200 David Schmitt <david@schmitt.edv-bus.at> wrote:>| class ssh::server::extra_secure inherits ssh::server { >| # ... >| } > > > On the point of migration, I think the namespace clashing is the lesser > problem. The fear of change in established systems is a much greater > hindrance. Having a central "blessed" repository from reductivelabs will > help here in the long run much more than hiding stuff in another > namespace.I can see where this could be a problem if one PCM module were to depend on another. There would either have to be a standard practice where PCM gets the top level or the local institution reserves the top level. Mixing and matching would be a problem because eventually namespaces would definitely collide and if a PCM module ssh::blah relied on PCM module tivoli::blah but an institution already have a tivoli::blah then something has to give. Maybe everything should go under pcm:: namespace? This is what you sort of do with things like URNs or to a lesser degree OIDs in LDAP. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
--On Sunday, August 03, 2008 02:52:19 AM +1000 James Turnbull <james@lovedthanlost.net> wrote:>> One can do a quite risk-less global rename of the local ssh module to >> ssh_local to avoid having to replace it immediately. Either way won''t >> spare you the neccessary migration and testing. But using a common:: >> prefix breaks autoloading and will be a millstone around everyones neck >> forever. > > +1.Ah, right, I forgot about the autoloading. Locally, we actually have 3 module paths. One for just regular modules, and one for clients and one for services (as in IT services, not system level services) and to make autoloading work, we''ve had to prefix the actual module names with c_ or s_, which is less then ideal. Having one big common module or pcm module would not be good. It would be nice to say that a particular namespace actually correlates to a module path. I smell a feature request. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
I''m just wondering... humor me. I mean this in the best possible way. I''ll take ssh::server as an example... What is it you can customize to ssh::server, other then the source the sshd_config is coming from, whether you harvest and manage the host keys and then... what else? Then, what do you really need to customize and what part of that customization is it you really need to manage from the manifest? Putting the right files in the right place is one thing, putting the right content in the right files in the right place is another. Here''s an example of a setup that runs in several organizations. Like always, one needs a little different customization then the other. The sshd_config file is pulled from (in order of appearance): puppet:///ssh/sshd_config.$hostname puppet:///ssh/sshd_config Any customization for a small number of hosts goes directly into a sshd_config.$hostname file, whereas customizations per group-of-hosts goes into sshd_config.group, having symlinks from sshd_config.$hostname to sshd_config.group. Another example: sshd_config.bofh-only only allows members of group bofh (a little evil administrators group) to log on via ssh. Servers that need this customization have a sshd_config.$hostname symlink to the sshd_config.bofh-only file. Eg: sshd_config.ldap01 -> sshd_config.bofh-only sshd_config.ldap02 -> sshd_config.bofh-only sshd_config.xen01 -> sshd_config.bofh-only sshd_config.xen02 -> sshd_config.bofh-only (etc, etc) Given this mechanism, whether you use $hostname or $fqdn, anyone can do any type of customization to the sshd_config file as long as they put the file (or link) in the right place with the right name. We all know that once we start templating sshd_config there is no end to it, and we could better find a way to parse the sshd_config man-page into a puppet manifest, forcing the "user" (who is actually an administrator) to walk through numerous settings before being able to start using the module. Now, am I missing something here? How far can we take this relatively simple mechanism? I already know how far in case of the ssh module, and some other modules, but I''m asking you... Would a simple mechanism like that serve your use-case, given a change here and there? For reference, I''ve also included a link to the actual module running this show[1]. It takes into account seperate source trees for domain name spaces (private/$domain), organizations (files/) and module defaults (ssh/). I hope you get the point, and I''d love to hear your answers, Kind regards, Jeroen van Meeuwen -kanarip [1] http://git.puppetmanaged.org/?p=modules/ssh/.git;a=blob;f=manifests/init.pp --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
To use our modules as an example, we change several things about our ssh server and I prefer to have the different explicit in the class name rather than automagic file loading. The reason is simple: when I use a tool or just list classes.txt and browse the classes applied to a system, I want to be able to tell right there what the system is doing or should be doing. For instance, if I can''t connect to a server from an off campus IP, I want to be able to look at the classes and see if perhaps I''m not using the ssh::global class, which doesn''t restrict ssh connections to just campus IPs. So, yes, sshd_config could be handled just by setting up the manifests to look for .$hostname and then fall back to a more general config but then I have to dig through my manifests to see what behaviors I have setup a particular manifests to do instead of knowing from just the name. Plus, not all things are just the config changes. For instance, the example I use of ssh version ssh::global is actually related to the iptables rules that get dropped in and not anything to do with sshd configuration. And I''ll reiterate just now nice it is to examine the classes being applied to a server to catch things like a server using ssh::rootenabled instead of interpreting the manifests or examining all the files in the files dir to see if a server is using that. That ability to rely on the class names to peer into that sort of thing is going to be critical for our "one portal" setup that will allow us to examine all facts about a system from one page. --On Tuesday, August 05, 2008 12:51:21 AM +0200 "Jeroen van Meeuwen (GMail)" <kanarip@gmail.com> wrote:> > I''m just wondering... humor me. I mean this in the best possible way. > > I''ll take ssh::server as an example... > > What is it you can customize to ssh::server, other then the source the > sshd_config is coming from, whether you harvest and manage the host keys > and then... what else? > > Then, what do you really need to customize and what part of that > customization is it you really need to manage from the manifest? Putting > the right files in the right place is one thing, putting the right > content in the right files in the right place is another. > > Here''s an example of a setup that runs in several organizations. Like > always, one needs a little different customization then the other. > > The sshd_config file is pulled from (in order of appearance): > > puppet:///ssh/sshd_config.$hostname > puppet:///ssh/sshd_config > > Any customization for a small number of hosts goes directly into a > sshd_config.$hostname file, whereas customizations per group-of-hosts > goes into sshd_config.group, having symlinks from sshd_config.$hostname > to sshd_config.group. Another example: > > sshd_config.bofh-only only allows members of group bofh (a little evil > administrators group) to log on via ssh. > > Servers that need this customization have a sshd_config.$hostname > symlink to the sshd_config.bofh-only file. Eg: > > sshd_config.ldap01 -> sshd_config.bofh-only > sshd_config.ldap02 -> sshd_config.bofh-only > sshd_config.xen01 -> sshd_config.bofh-only > sshd_config.xen02 -> sshd_config.bofh-only > (etc, etc) > > Given this mechanism, whether you use $hostname or $fqdn, anyone can do > any type of customization to the sshd_config file as long as they put > the file (or link) in the right place with the right name. > > We all know that once we start templating sshd_config there is no end to > it, and we could better find a way to parse the sshd_config man-page > into a puppet manifest, forcing the "user" (who is actually an > administrator) to walk through numerous settings before being able to > start using the module. > > Now, am I missing something here? How far can we take this relatively > simple mechanism? I already know how far in case of the ssh module, and > some other modules, but I''m asking you... Would a simple mechanism like > that serve your use-case, given a change here and there? > > For reference, I''ve also included a link to the actual module running > this show[1]. It takes into account seperate source trees for domain > name spaces (private/$domain), organizations (files/) and module > defaults (ssh/). > > I hope you get the point, and I''d love to hear your answers, > > Kind regards, > > Jeroen van Meeuwen > -kanarip > > [1] > http://git.puppetmanaged.org/?p=modules/ssh/.git;a=blob;f=manifests/init. > pp > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Digant C Kasundra wrote:> To use our modules as an example, we change several things about our ssh > server and I prefer to have the different explicit in the class name rather > than automagic file loading. The reason is simple: when I use a tool or > just list classes.txt and browse the classes applied to a system, I want to > be able to tell right there what the system is doing or should be doing. > For instance, if I can''t connect to a server from an off campus IP, I want > to be able to look at the classes and see if perhaps I''m not using the > ssh::global class, which doesn''t restrict ssh connections to just campus > IPs. > > So, yes, sshd_config could be handled just by setting up the manifests to > look for .$hostname and then fall back to a more general config but then I > have to dig through my manifests to see what behaviors I have setup a > particular manifests to do instead of knowing from just the name. > > Plus, not all things are just the config changes. For instance, the > example I use of ssh version ssh::global is actually related to the > iptables rules that get dropped in and not anything to do with sshd > configuration. > > And I''ll reiterate just now nice it is to examine the classes being applied > to a server to catch things like a server using ssh::rootenabled instead of > interpreting the manifests or examining all the files in the files dir to > see if a server is using that. That ability to rely on the class names to > peer into that sort of thing is going to be critical for our "one portal" > setup that will allow us to examine all facts about a system from one page. >Thank you for the details on how you think what I set out in my previous email wouldn''t work for you! If I understand you correctly, and please do correct me if I''m wrong, you''re saying that having separate (sub-)classes for separate ssh setups is a matter of convenience to you, over having multiple sources, am I right? This led me to conclude it is not so much a technical issue, hence that''s why I''m asking. Kind regards, Jeroen van Meeuwen -kanarip --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Jeroen van Meeuwen (GMail) wrote:> I''m just wondering... humor me. I mean this in the best possible way. > > I''ll take ssh::server as an example... > > What is it you can customize to ssh::server, other then the source the > sshd_config is coming from, whether you harvest and manage the host keys > and then... what else?On the nodes in the clusters i manage using Puppet, I do the following: * Copy /etc/ssh/ssh_host_{rsa,dsa}_key{,.pub} from a central location (four files; but each node has its own set of four files). * Make sure that "HostbasedAuthentication" is set to "yes" in /etc/ssh/sshd_config. * Make sure that "IgnoreRhosts" is set to "no" in /etc/ssh/sshd_config. * Install /etc/ssh/shosts.equiv from a template file. * Install /root/.shosts from the same template file as above. Note especially that I do *not* manage the entire /etc/ssh/sshd_config, just a couple of individual settings. I believe my OS vendor is better than me on knowing what is sensible and safe values for most of those settings. I don''t want to spend time tracking the changes they make between updates of the openssh-server package, between OS releases, or between OS distributions. And other sites are likely to want to change other settings than I do. Similarly, for the ssh client I do: * Install /etc/ssh/ssh_known_hosts from a template (that reads the various host keys from a central location). * Ensure that the settings HostbasedAuthentication yes EnableSSHKeysign yes exists in the "Host *" section in /etc/ssh/ssh_config.> Putting > the right files in the right place is one thing, putting the right > content in the right files in the right place is another.On another site I run multiple instances of the ssh server, with different config files, on the same machine. So it''s not always to easy to even know *where* to put the config files... /Thomas Bellman --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Digant C Kasundra schrieb:> --On Wednesday, July 30, 2008 04:22:20 AM +0300 Martha Greenberg > <marthag@mit.edu> wrote: > >> 1) To develop a set of standards for writing puppet modules. This is >> currently at http://reductivelabs.com/trac/puppet/wiki/ModuleStandards . > > I''m not sure I like the ::common naming scheme. To me, ssh::server implies > a class that handles the most common case and the less common would inherit > from that and go in a namespace below that (ssh::server::weird). To me, > this more closely resembles an "object-oriented" model. >In my reading there is only one reason a module contains a ::common class, and that is "sharing resources between multiple user-visible classes and defines." In #6 of the Standards(-to-be) I used it for example to split out the os-independent part of the manifest. As can be seen there, the real user-visible class is still "ssh::server", which multiplexes according to $operatingsystem into the right "implementation". I guess I should make that more clear, or did I misunderstand you completely? Regards, DavidS --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to puppet-dev+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
--On Tuesday, August 05, 2008 09:41:01 AM +0200 "Jeroen van Meeuwen (GMail)" <kanarip@gmail.com> wrote:> Thank you for the details on how you think what I set out in my previous > email wouldn''t work for you! If I understand you correctly, and please > do correct me if I''m wrong, you''re saying that having separate > (sub-)classes for separate ssh setups is a matter of convenience to you, > over having multiple sources, am I right? This led me to conclude it is > not so much a technical issue, hence that''s why I''m asking.It is both a technical and convenience issue. The technical side is that we don''t just configure the ssh server in that class but also the associated firewall rules, which are different for different classes depending on where we want to allow connections from. Having multiple sources doesn''t quite address that. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Digant C Kasundra
2008-Aug-05 20:24 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
--On Tuesday, August 05, 2008 12:13:14 PM +0200 David Schmitt <david@schmitt.edv-bus.at> wrote:>>> 1) To develop a set of standards for writing puppet modules. This is >>> currently at http://reductivelabs.com/trac/puppet/wiki/ModuleStandards >>> . >> >> I''m not sure I like the ::common naming scheme. To me, ssh::server >> implies a class that handles the most common case and the less common >> would inherit from that and go in a namespace below that >> (ssh::server::weird). To me, this more closely resembles an >> "object-oriented" model. >> > > In my reading there is only one reason a module contains a ::common > class, and that is "sharing resources between multiple user-visible > classes and defines." > > In #6 of the Standards(-to-be) I used it for example to split out the > os-independent part of the manifest. As can be seen there, the real > user-visible class is still "ssh::server", which multiplexes according > to $operatingsystem into the right "implementation". > > I guess I should make that more clear, or did I misunderstand you > completely?So, ssh::server includes ssh::server::common and ssh::server::debian? That definitely seems reverse of what I would consider an inheritance model approach. What doesn''t the os-independant stuff go into ssh::server and ssh::server::debian inherit from that? Maybe we''re both trying to say the same thing but are talking around each other. Am I still misunderstanding? --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
James Turnbull
2008-Aug-05 20:34 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Digant C Kasundra wrote:>> I guess I should make that more clear, or did I misunderstand you >> completely? > > So, ssh::server includes ssh::server::common and ssh::server::debian? That > definitely seems reverse of what I would consider an inheritance model > approach. What doesn''t the os-independant stuff go into ssh::server and > ssh::server::debian inherit from that? Maybe we''re both trying to say the > same thing but are talking around each other. Am I still misunderstanding?- From my reading you''re both saying the same thing. Regards James Turnbull - -- Author of: * Pulling Strings with Puppet (http://www.amazon.com/gp/product/1590599780/) * Pro Nagios 2.0 (http://www.amazon.com/gp/product/1590596099/) * Hardening Linux (http://www.amazon.com/gp/product/1590594444/) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFImLlB9hTGvAxC30ARAt/uAKCevMDPD4cCuhct2OLr0xJwTBg+oACdGdDa sFVr7qNOsGw3qokfff1344E=78H/ -----END PGP SIGNATURE----- --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
David Schmitt
2008-Aug-06 14:38 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
[I think this should be kept to -users, since it doesn''t really pertains to the ruby core] Digant C Kasundra schrieb:> --On Tuesday, August 05, 2008 12:13:14 PM +0200 David Schmitt > <david@schmitt.edv-bus.at> wrote: > >>>> 1) To develop a set of standards for writing puppet modules. This is >>>> currently at http://reductivelabs.com/trac/puppet/wiki/ModuleStandards >>>> . >>> I''m not sure I like the ::common naming scheme. To me, ssh::server >>> implies a class that handles the most common case and the less common >>> would inherit from that and go in a namespace below that >>> (ssh::server::weird). To me, this more closely resembles an >>> "object-oriented" model. >>> >> In my reading there is only one reason a module contains a ::common >> class, and that is "sharing resources between multiple user-visible >> classes and defines." >> >> In #6 of the Standards(-to-be) I used it for example to split out the >> os-independent part of the manifest. As can be seen there, the real >> user-visible class is still "ssh::server", which multiplexes according >> to $operatingsystem into the right "implementation". >> >> I guess I should make that more clear, or did I misunderstand you >> completely? > > So, ssh::server includes ssh::server::common and ssh::server::debian? That > definitely seems reverse of what I would consider an inheritance model > approach. What doesn''t the os-independant stuff go into ssh::server and > ssh::server::debian inherit from that? Maybe we''re both trying to say the > same thing but are talking around each other. Am I still misunderstanding?Yes, I think I failed to explain the difference between actual "user-visible" parts and the "under-the-hood" parts which make up the mechanics. To copy my (slightly enhanced) example from wiki:ModuleStandards : | class ssh::server::common { | service { ''ssh::server'': ... } | } | class ssh::server::debian inherits ssh::server::common { | Service[''ssh::server''] { name => ''ssh'' } | } | class ssh::server { | include "ssh::server::$operatingsystem" | } This structure works by splitting the different responsibilities into separate parts: 1) "Type": A common basic structure for all: The ssh::server::common class defines all major parts of a ssh server. In this simplified example, only the Service[''ssh::server''] is described. A higher quality implementation could provide a placeholder for the package and default users for privilege separation as well as the basic relationships between this resources. 2) "Provider": OS-specific implementation of details. In this case, there is only one implementation: the class ssh::server::debian. There the actual name of the init script is provided. A higher quality implementation would provide hasstatus/hasrestart and so on, fill out other details and add release-specific overrides. 3) "Resource": Easy way to use for the common user. Just "include ssh::server" will bring in all the goodness for enabling a ssh server on this node. This structure has the following properties: *) fail-save: a missing implementation for a OS leads to a compile error; having two implementations for a OS (e.g. new module version and user provided one) will lead to a obvious error too. *) common interface: every implementation will provide a "Service[''ssh::server'']" because it is specified in the ::common class. This enables the user to blindly notify=>Service[''ssh::server''] (or whatever else is needed/available) without having to think about the specific OS'' details underneath. Also the ::common class leaves out all the gritty and changing details, like release differences and such thing. This makes the base class much more legible and helps to distinguish the relevant structure from the implementation details. *) support for a yet unsupported OS is easily added: add a new file with ssh::server::new-os. As soon as this class gets loaded, puppet will use it where applicable. no need to touch the module at all for this and the file can be sent as-is for inclusion in the module. *) general extensibility: class ssh::server::weird inherits ssh::server { # include OS-dependent weirdness here include "ssh::server::${operatingsystem}::weird" # Add OS-independent weirdness ssh::conf { "SshBanner": value => "very weird server 2.3"; } } Again, this has the same fail-save properties as the original structure, since it won''t compile on a unsupported OS, and the same pattern can be applied to add another layer again. This pattern can also be used to add OS-specific extensions. Regards, DavidS --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
I''ve been following this thread, and there is one thing that is bothering me. Basically, you''re looking at a heck of a lot of string creep with some of these suggestions. I''ve been trying to be good and properly nest things but some users of my results get a bit tired of mis-typing things like: ssh::server::debian::slightmod::evenslightermod And tend to prefer: ssh::evenslightermod which uses ''inherits'' to mask everything. It''s not as elegant, but it''s way more typeable. I''m not sure what the actual solution is, but it seems that this way lies madness if not.....*gasp*....GUIs! T --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to puppet-dev+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
Evan Hisey
2008-Aug-06 20:03 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
On Wed, Aug 6, 2008 at 12:35 PM, Trevor Vaughan <peiriannydd@gmail.com> wrote:> > I''ve been following this thread, and there is one thing that is bothering me. > > Basically, you''re looking at a heck of a lot of string creep with some > of these suggestions. > > I''ve been trying to be good and properly nest things but some users of > my results get a bit tired of mis-typing things like: > ssh::server::debian::slightmod::evenslightermod > > And tend to prefer: > ssh::evenslightermod which uses ''inherits'' to mask everything. > > It''s not as elegant, but it''s way more typeable. > > I''m not sure what the actual solution is, but it seems that this way > lies madness if not.....*gasp*....GUIs! >In this same vain, it really looks like we are trying to do to much work to amek teh common modules to general. No matter how flexible we try and make them they will _never_ be right for most use cases. I thkn the best route here is to make the whole PCM manifest work cleanly in itself and set up good sound default system setting that can be easily customized _in_ the module. this way some one can take a complete generic configuration of say a web server, ssh, and a hardening module, to create a local puppet manifest an d have all the parts play nice together. From there they can then customize or grow teh solution out to meet the specific site needs. After all , we really should not be assuming this system is for use by dummies. When it comes to server and infrastructure maintenance issues, anyone in charge of that should really take the time to learn how the tools work. Evan --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Wed, Aug 06, 2008 at 01:35:16PM -0400, Trevor Vaughan wrote:> I''ve been trying to be good and properly nest things but some users of > my results get a bit tired of mis-typing things like: > ssh::server::debian::slightmod::evenslightermod > > And tend to prefer: > ssh::evenslightermod which uses ''inherits'' to mask everything.I agree. In fact, here we use koumbit::ssh::server. ie. we *prefix* our namespace in front of the regular namespace for our customizations. But I think that ssh::server::debian does make some sense and is important as you''ll also need ssh::client::debian and therefore can''t shortcut to ssh::debian (although there might be some commonality between "servers" and "clients" that might be factored out in ssh::debian...) A. -- Information is not knowledge Knowledge is not wisdom Wisdom is not truth - Frank Zappa
Nigel Kersten
2008-Aug-06 23:34 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
On Wed, Aug 6, 2008 at 1:03 PM, Evan Hisey <ehisey@gmail.com> wrote:> In this same vain, it really looks like we are trying to do to much > work to amek teh common modules to general. No matter how flexible we > try and make them they will _never_ be right for most use cases. I > thkn the best route here is to make the whole PCM manifest work > cleanly in itself and set up good sound default system setting that > can be easily customized _in_ the module. this way some one can take a > complete generic configuration of say a web server, ssh, and a > hardening module, to create a local puppet manifest an d have all the > parts play nice together. From there they can then customize or grow > teh solution out to meet the specific site needs. > > After all , we really should not be assuming this system is for use > by dummies. When it comes to server and infrastructure maintenance > issues, anyone in charge of that should really take the time to learn > how the tools work.Evan has nicely summarized the disquiet I''ve felt throughout this thread. -- Nigel Kersten Systems Administrator Tech Lead - MacOps --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
James Turnbull
2008-Aug-06 23:59 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Nigel Kersten wrote: | On Wed, Aug 6, 2008 at 1:03 PM, Evan Hisey <ehisey@gmail.com> wrote: | |> In this same vain, it really looks like we are trying to do to much |> work to amek teh common modules to general. No matter how flexible we |> try and make them they will _never_ be right for most use cases. I |> thkn the best route here is to make the whole PCM manifest work |> cleanly in itself and set up good sound default system setting that |> can be easily customized _in_ the module. this way some one can take a |> complete generic configuration of say a web server, ssh, and a |> hardening module, to create a local puppet manifest an d have all the |> parts play nice together. From there they can then customize or grow |> teh solution out to meet the specific site needs. |> |> After all , we really should not be assuming this system is for use |> by dummies. When it comes to server and infrastructure maintenance |> issues, anyone in charge of that should really take the time to learn |> how the tools work. | | Evan has nicely summarized the disquiet I''ve felt throughout this thread. | I agree totally with Evan/Nigel that all sys admins should understand how the module works - no one should blindly apply configuration. I don''t think anyone sees PCM as doing that though. PCM is like a framework - a skeleton that demonstrates capability and best practice. But I think the nature of the modules we produce could significantly impact how easy it is for a new Puppet user to understand those modules. ~ In this spirit we should produce generic modules - and expect customisation to happen. My principal concern is to what definition of "generic" should modules adhere? To a single OS/platform? To a "conceptual" model? To multiple platforms - Linux, BSD, OSX? I''d like to see a generic module - for example apache - that specifies all the required base configuration and demonstrates/supports multiple platforms using sub-classes. Probably not all platforms but broadly speaking I think Debian/RH/*BSD~OSX are a good baseline. Maybe Solaris too for some/all modules. This approach isn''t going to meet everyone''s needs - not possible - but it will: a) Demonstrate the basics b) Demonstrate the cross-platform nature of Puppet - a critical concept c) Hopefully be easily extensible/customisable. There is a question (a mostly technical/style one) of HOW that customisation should be achieved and articulated - namespaces, inheritance, overrides, etc, etc. That''s just a technical issue to resolve though and I think some good discussion is going on here. I personally don''t overly care how we do it and if the agreed result is something like apache::server::debian and people simply modify the existing module rather than override I''d be happy. Regards James Turnbull - -- Author of: * Pulling Strings with Puppet (http://www.amazon.com/gp/product/1590599780/) * Pro Nagios 2.0 (http://www.amazon.com/gp/product/1590596099/) * Hardening Linux (http://www.amazon.com/gp/product/1590594444/) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFImjrF9hTGvAxC30ARAi8wAKC4hjJ8/7OYpA24NsQBdcUWA9MIqQCeJkJA T7Axprf6iE4gFtEigGmWblk=Qu9O -----END PGP SIGNATURE----- --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
David Schmitt
2008-Aug-07 08:17 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
James Turnbull schrieb:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Nigel Kersten wrote: > | On Wed, Aug 6, 2008 at 1:03 PM, Evan Hisey <ehisey@gmail.com> wrote: > | > |> In this same vain, it really looks like we are trying to do to much > |> work to amek teh common modules to general. No matter how flexible we > |> try and make them they will _never_ be right for most use cases. IEvery apache admin will need apache::conf_snippet and a significant portion will need apache::virtual_host.> |> thkn the best route here is to make the whole PCM manifest work > |> cleanly in itself and set up good sound default system setting that > |> can be easily customized _in_ the module. this way some one can take a > |> complete generic configuration of say a web server, ssh, and a > |> hardening module, to create a local puppet manifest an d have all the > |> parts play nice together. From there they can then customize or grow > |> teh solution out to meet the specific site needs. > |> > |> After all , we really should not be assuming this system is for use > |> by dummies. When it comes to server and infrastructure maintenance > |> issues, anyone in charge of that should really take the time to learn > |> how the tools work. > | > | Evan has nicely summarized the disquiet I''ve felt throughout this thread. > | > > I agree totally with Evan/Nigel that all sys admins should understand > how the module works - no one should blindly apply configuration. I > don''t think anyone sees PCM as doing that though. PCM is like a > framework - a skeleton that demonstrates capability and best practice. > > But I think the nature of the modules we produce could significantly > impact how easy it is for a new Puppet user to understand those modules. > ~ In this spirit we should produce generic modules - and expect > customisation to happen. My principal concern is to what definition of > "generic" should modules adhere? To a single OS/platform? To a > "conceptual" model? To multiple platforms - Linux, BSD, OSX?Conceptual model, like "ssh means OpenSSH, and Service[ssh::server] is running" (which, coincidentally is what ssh::server::common would describe, following my example from yesterday evening.> I''d like to see a generic module - for example apache - that specifies > all the required base configuration and demonstrates/supports multiple > platforms using sub-classes. Probably not all platforms but broadly > speaking I think Debian/RH/*BSD~OSX are a good baseline. Maybe Solaris > too for some/all modules.I''m all for a big group-hugging collaboration with people responsible for apps (single module, concept, prototype on a single os) or platforms (single os implementation in multiple modules) to support as much platforms as puppet can do.> This approach isn''t going to meet everyone''s needs - not possible - but > it will: > > a) Demonstrate the basics > b) Demonstrate the cross-platform nature of Puppet - a critical concept > c) Hopefully be easily extensible/customisable. > > There is a question (a mostly technical/style one) of HOW that > customisation should be achieved and articulated - namespaces, > inheritance, overrides, etc, etc. That''s just a technical issue to > resolve though and I think some good discussion is going on here.I''m totally with you there. Regards, DavidS --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Digant C Kasundra
2008-Aug-07 22:44 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
--On Wednesday, August 06, 2008 01:35:16 PM -0400 Trevor Vaughan <peiriannydd@gmail.com> wrote:> I''ve been following this thread, and there is one thing that is bothering > me. > > Basically, you''re looking at a heck of a lot of string creep with some > of these suggestions. > > I''ve been trying to be good and properly nest things but some users of > my results get a bit tired of mis-typing things like: > ssh::server::debian::slightmod::evenslightermod > > And tend to prefer: > ssh::evenslightermod which uses ''inherits'' to mask everything. > > It''s not as elegant, but it''s way more typeable. > > I''m not sure what the actual solution is, but it seems that this way > lies madness if not.....*gasp*....GUIs! > > TWe''ve avoided this many subclasses. There is something to be said for not breaking every little thing into different classes. For instance, do you have lots of scenarios where you have ssh server but not the client or visa-versa? We don''t, so we just have an ssh module that handles both server and client, thus eliminating one level of subclassing. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Digant C Kasundra
2008-Aug-07 22:49 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
--On Thursday, August 07, 2008 09:59:01 AM +1000 James Turnbull <james@lovedthanlost.net> wrote:> PCM is like a > framework - a skeleton that demonstrates capability and best practice.I''m not sure how much people will agree on a best practice for some of the things PCM will model. Even with configuring and running puppet, something we all on this list have in common, there is much disagreement on what is "best." But it will certainly be a framework for people that want to take it for one way to do things that will work and if nothing else, as you stated, it will serve as a good starting ground for someone trying to tackle writing manifests for a particular purpose. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Digant C Kasundra
2008-Aug-07 22:58 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
--On Wednesday, August 06, 2008 04:38:37 PM +0200 David Schmitt <david@schmitt.edv-bus.at> wrote:> Yes, I think I failed to explain the difference between actual > "user-visible" parts and the "under-the-hood" parts which make up the > mechanics. > > To copy my (slightly enhanced) example from wiki:ModuleStandards : > >| class ssh::server::common { >| service { ''ssh::server'': ... } >| } >| class ssh::server::debian inherits ssh::server::common { >| Service[''ssh::server''] { name => ''ssh'' } >| } >| class ssh::server { >| include "ssh::server::$operatingsystem" >| }Very novel. I probably need to think a bit more about how this would work in a practical sense. I can imagine it getting a bit annoying if you have a lot of subclasses and have to therefore have a lot of ::$operatingsystem:: namespaces. I would say for something simple, it would still be okay to just put the os specific difference in ssh::server. For instance, in our ssh (our module handles both client and server b/c we never install one without the other so no real need to break apart), we use a case statement off of operatingsystem to determine what the package name is. Other than that, everything is the same. So, not sure if I would want to break that out into ssh::debian and ssh::redhat. But in the case where there are lot of differences, for legibility alone, it is a good idea to follow the structure you''ve outlined. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
David Schmitt
2008-Aug-08 08:14 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
Digant C Kasundra schrieb:> --On Wednesday, August 06, 2008 04:38:37 PM +0200 David Schmitt > <david@schmitt.edv-bus.at> wrote: > >> Yes, I think I failed to explain the difference between actual >> "user-visible" parts and the "under-the-hood" parts which make up the >> mechanics. >> >> To copy my (slightly enhanced) example from wiki:ModuleStandards : >> >> | class ssh::server::common { >> | service { ''ssh::server'': ... } >> | } >> | class ssh::server::debian inherits ssh::server::common { >> | Service[''ssh::server''] { name => ''ssh'' } >> | } >> | class ssh::server { >> | include "ssh::server::$operatingsystem" >> | } > > Very novel. I probably need to think a bit more about how this would work > in a practical sense. > > I can imagine it getting a bit annoying if you have a lot of subclasses and > have to therefore have a lot of ::$operatingsystem:: namespaces. I would > say for something simple, it would still be okay to just put the os > specific difference in ssh::server.Definitely. On the up side, that can be much more compact, on the downside it is harder to extend locally (patch instead of added file).> For instance, in our ssh (our module handles both client and server b/c we > never install one without the other so no real need to break apart), we use > a case statement off of operatingsystem to determine what the package name > is. Other than that, everything is the same. So, not sure if I would want > to break that out into ssh::debian and ssh::redhat. But in the case where > there are lot of differences, for legibility alone, it is a good idea to > follow the structure you''ve outlined.If you look at my actual ssh module at [1], you see that my ssh::server actually inherits from ssh::client and I do handle a bit more than package+service --- specifically fixing the user and group that are created for priviledge separation and stuff. I don''t know whether other distros might not differ more on this and on which files to repair ownership afterwards. I also don''t know yet how to adapt the structure as it is currently to my outlined method. I presume I will fold the classes into one and provide a override to disable all server functionality. Regards, DavidS [1] http://git.black.co.at/?p=module-ssh;a=blob;f=manifests/init.pp;hb=HEAD --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Trevor Vaughan
2008-Aug-08 10:30 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
Personally, I think that the operating system should be hidden in as many cases as possible. We have the $operatingsystem and related facter variables and all of the types that I''ve used try to do something sane given an OS (or other fact about the system). So, it would make sense to me that modules follow the same general concept. It seems that a general rule for a given module would be: class foo { case $operatingsystem (or whatever, it would be really nice to be able to OR or AND these) { os1: { do os1 stuff }, os2: { do os2 stuff }, default: { if there is something sane for everyone, do it, otherwise throw an error as running a module on an unsupported OS } } } Unfortunately, I''ve got a few cases that look like the following: class foo { case $operatingsystem { os1: { case $operatingsystemversion { 2: { do os1.2 stuff }, 3: { do os1.3 stuff} }, os2: { case $operatingsystemversion { 8: { do os 2.8 stuff}, 12: { do os 2.12 stuff} }, etc... } } And, as you can see, that''s really ugly but absolutely functional. T On Thu, Aug 7, 2008 at 6:58 PM, Digant C Kasundra <digant@stanford.edu> wrote:> > --On Wednesday, August 06, 2008 04:38:37 PM +0200 David Schmitt > <david@schmitt.edv-bus.at> wrote: > >> Yes, I think I failed to explain the difference between actual >> "user-visible" parts and the "under-the-hood" parts which make up the >> mechanics. >> >> To copy my (slightly enhanced) example from wiki:ModuleStandards : >> >>| class ssh::server::common { >>| service { ''ssh::server'': ... } >>| } >>| class ssh::server::debian inherits ssh::server::common { >>| Service[''ssh::server''] { name => ''ssh'' } >>| } >>| class ssh::server { >>| include "ssh::server::$operatingsystem" >>| } > > Very novel. I probably need to think a bit more about how this would work > in a practical sense. > > I can imagine it getting a bit annoying if you have a lot of subclasses and > have to therefore have a lot of ::$operatingsystem:: namespaces. I would > say for something simple, it would still be okay to just put the os > specific difference in ssh::server. > > For instance, in our ssh (our module handles both client and server b/c we > never install one without the other so no real need to break apart), we use > a case statement off of operatingsystem to determine what the package name > is. Other than that, everything is the same. So, not sure if I would want > to break that out into ssh::debian and ssh::redhat. But in the case where > there are lot of differences, for legibility alone, it is a good idea to > follow the structure you''ve outlined. > > > > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Garth Roxburgh-Kidd
2008-Aug-11 04:14 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
2008/8/8 Trevor Vaughan <peiriannydd@gmail.com>:> Personally, I think that the operating system should be hidden in as > many cases as possible.+1 --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Francois Deppierraz
2008-Aug-11 09:51 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
Trevor Vaughan wrote:> And tend to prefer: > ssh::evenslightermod which uses ''inherits'' to mask everything.You can always create a specific module for your company which contains customizations to publically availables modules. class mycompany::ssh inherits ssh::server::foo::bar François --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Trevor Vaughan
2008-Aug-11 11:17 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
François, That''s definitely true, and not a bad idea at all. It looks like I''m going to have to finally put together that puppet manifest parser to graphviz parser after all. All of the inheritance is getting too confusing without a map. Does anyone know if there are any hooks into the current manifest parser, or would it be an inline mod? I thought about parsing the YAML, but it doesn''t seem to keep the inheritance metadata. . . Trevor On Mon, Aug 11, 2008 at 5:51 AM, Francois Deppierraz <francois@ctrlaltdel.ch> wrote:> > Trevor Vaughan wrote: > >> And tend to prefer: >> ssh::evenslightermod which uses ''inherits'' to mask everything. > > You can always create a specific module for your company which contains > customizations to publically availables modules. > > class mycompany::ssh inherits ssh::server::foo::bar > > François > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Digant C Kasundra
2008-Aug-11 20:06 UTC
[Puppet Users] Re: [Puppet-dev] Re: Puppet Common Modules
--On Friday, August 08, 2008 06:30:01 AM -0400 Trevor Vaughan <peiriannydd@gmail.com> wrote:> Unfortunately, I''ve got a few cases that look like the following: > > class foo { > case $operatingsystem { > os1: { > case $operatingsystemversion { > 2: { do os1.2 stuff }, > 3: { do os1.3 stuff} > }, > os2: { > case $operatingsystemversion { > 8: { do os 2.8 stuff}, > 12: { do os 2.12 stuff} > }, > etc... > } > }Me too. What was proposed was to break the stuff into foo::debian so you could just do: class foo { include foo::$operatingsystem } class foo::os1 { case $operatingsystemversion { 2: { do os1.2 stuff }, 3: { do os1.3 stuff} } } All you get is legability so you don''t have one big honking class with a tone of case statements. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Puppet can generate graphs of the relationships. I forgot the option, but check genconfig and you''ll find it. On 8/11/08, Trevor Vaughan <peiriannydd@gmail.com> wrote:> > François, > > That''s definitely true, and not a bad idea at all. > > It looks like I''m going to have to finally put together that puppet > manifest parser to graphviz parser after all. All of the inheritance > is getting too confusing without a map. > > Does anyone know if there are any hooks into the current manifest > parser, or would it be an inline mod? > > I thought about parsing the YAML, but it doesn''t seem to keep the > inheritance metadata. . . > > Trevor > > On Mon, Aug 11, 2008 at 5:51 AM, Francois Deppierraz > <francois@ctrlaltdel.ch> wrote: >> >> Trevor Vaughan wrote: >> >>> And tend to prefer: >>> ssh::evenslightermod which uses ''inherits'' to mask everything. >> >> You can always create a specific module for your company which contains >> customizations to publically availables modules. >> >> class mycompany::ssh inherits ssh::server::foo::bar >> >> François >> >> > >> > > > >-- Sent from Gmail for mobile | mobile.google.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Trevor Vaughan
2008-Aug-12 11:13 UTC
[Puppet Users] Re: [Puppet-dev] Puppet Common Modules
Paul, I thought that was post-compile, not pre-compile. I''ll try it and see what I get. Thanks, Trevor On Mon, Aug 11, 2008 at 7:34 PM, Paul Lathrop <paul@tertiusfamily.net> wrote:> > Puppet can generate graphs of the relationships. I forgot the option, > but check genconfig and you''ll find it. > > On 8/11/08, Trevor Vaughan <peiriannydd@gmail.com> wrote: >> >> François, >> >> That''s definitely true, and not a bad idea at all. >> >> It looks like I''m going to have to finally put together that puppet >> manifest parser to graphviz parser after all. All of the inheritance >> is getting too confusing without a map. >> >> Does anyone know if there are any hooks into the current manifest >> parser, or would it be an inline mod? >> >> I thought about parsing the YAML, but it doesn''t seem to keep the >> inheritance metadata. . . >> >> Trevor >> >> On Mon, Aug 11, 2008 at 5:51 AM, Francois Deppierraz >> <francois@ctrlaltdel.ch> wrote: >>> >>> Trevor Vaughan wrote: >>> >>>> And tend to prefer: >>>> ssh::evenslightermod which uses ''inherits'' to mask everything. >>> >>> You can always create a specific module for your company which contains >>> customizations to publically availables modules. >>> >>> class mycompany::ssh inherits ssh::server::foo::bar >>> >>> François >>> >>> > >>> >> >> > >> > > -- > Sent from Gmail for mobile | mobile.google.com > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---