Puppet users - Oct 2012 - How do design next-gen modules, any guidelines ? a question for gurus...

Hi,

My dilemma is how should I write my module to be "next-gen" ?
Not from a code point of view but from a design/layout point of view.

We wrote our modules for 2.6.x like every beginner would have done: 
write everything in init.pp.

But now, the fashion is to use parameterized classes and Hiera which we 
will use with puppet 2.7.x.

I mainly followed/watched what Example42 was doing. They were using 
$classname::{install|remove|disable|blah} to actually call an action in 
their old modules. It was almost a sub class ($subclass.pp) per action. 
It was easy to read and possibly to maintain.

But now Alessandro from Example42 seems to put everything back in 
init.pp with his nextgen modules...

I like the idea of segregating types or action per files to avoid 
mistakes during edits but at the same time I am wondering why this 
change for example42, is it a limitation for parameterized classes ?

Like everybody, I''d like to write my module using the best and most 
flexible way so it will be easy to modify and to maintain. I don''t like
to redo my modules from scratch each time :)

So, any inputs/insights are welcome. What direction should I/we take ?

I really believe that puppetlabs or a set of gurus should set these 
guidelines so everybody would develop the same way. At the moment, it''s
like "there is more than one way to do it" (perl) but a lot of people 
are after a "this is the best way to do it", to not be confused with 
"there is only way to do it" :)

Note: I like the puppet style guide but it only talks about syntax not
       how to design module the best way.

-Thomas

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mon, Oct 22, 2012 at 12:33 AM, Thomas Guthmann
<tguthmann@iseek.com.au>wrote:
>
> But now, the fashion is to use parameterized classes and Hiera which we
> will use with puppet 2.7.x.
>
On this point, I''d suggest you read this docs page with some points on
writing classes with other versions in mind. Personally, if you''re
starting
from scratch, I''d suggest you write with Puppet 3.0 in mind.

http://docs.puppetlabs.com/puppet/3/reference/lang_classes.html#aside-writing-for-multiple-puppet-versions

>
>
> Note: I like the puppet style guide but it only talks about syntax not
>       how to design module the best way.
>
>
Indeed! I''m quite interested in writing such a guide. It''s
only a matter of
getting the time at this point to start a draft and get everyone involved
in hacking on it.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Monday, October 22, 2012 2:33:18 AM UTC-5, Thomas Guthmann
wrote:
>
> Hi, 
>
> My dilemma is how should I write my module to be "next-gen" ? 
> Not from a code point of view but from a design/layout point of view.
>

I don''t think much has changed in that area in a long time.  Styles
have
come and gone, but the only new thing I can think of since 0.24 that had 
much impact on the fundamental questions is hiera.

 
>
> We wrote our modules for 2.6.x like every beginner would have done: 
> write everything in init.pp. 
>
> But now, the fashion is to use parameterized classes and Hiera which we 
> will use with puppet 2.7.x. 
>

It would be wise to make your decisions based on an evaluation of technical 
merit, not based on what happens to be fashionable in some segment of the 
Internet at the moment.  And naturally, you should be wary of taking advice 
from random people on the ''net.  (Except me -- I''m TOTALLY
reliable.  :-))

 
>
> I mainly followed/watched what Example42 was doing. They were using 
> $classname::{install|remove|disable|blah} to actually call an action in 
> their old modules. It was almost a sub class ($subclass.pp) per action. 
> It was easy to read and possibly to maintain. 
>

I am all for modular code (including inside Puppet ''modules''),
but what I
think you''re asking about here is not so much module implementation,
but
rather modules'' interfaces to the rest of Puppet.  Thus I take it as a 
question about whether to convey information about the desired 
configuration details via class names or data (parameters or external 
data).  The two approaches provide different advantages.

Controlling everything via data (meaning either / both class parameters and 
external data) can make your main manifests very simple, and / or it may 
have advantages for the design of complementary ENCs.  On the other hand, 
putting information into class names can enable class dependencies to 
depend on that information.  For example:
"Class[''tomcat::running''] ->
Class[''mywebapp::running'']".  As you say, using class
names in that way
also can make manifests clearer to human readers.

 
>
> But now Alessandro from Example42 seems to put everything back in 
> init.pp with his nextgen modules... 
>
> I like the idea of segregating types or action per files to avoid 
> mistakes during edits but at the same time I am wondering why this 
> change for example42, is it a limitation for parameterized classes ? 
>

I cannot speak for Example42, but if you''ve been following this group
you
will know by now that my technical opinion of parametrized classes is that 
you should not write them, and as much as possible you should avoid using 
them (unless via the ''include'' or ''require''
function).  I have not studied
the Example42 modules, but it is conceivable that some of the technical 
limitations of parametrized classes influenced the decision to move to 
unitary interfaces.

The main potential issue there would be that any parametrized-style 
declaration for a given class must be the *first* declaration Puppet parses 
for that class.  It follows that there can only be one such declaration of 
each class (whereas there can be any number of ''include'' or
''require''
declarations of a class).  There are many implications, among them that if 
the module supports users declaring, say, mymodule::installed, then other 
classes in the module (mymodule::running, for instance) must not use 
parametrized-style declarations of that class.  That''s a bit of a
catch-22,
however, because if the user *doesn''t* declare the first class, then
other
classes in the module may need to declare it with parameters.

Alternatively, if the example42 modules are truly subclassing (which is not 
inherently implied by namespace nesting ala "classname::blah") then
the
perception may just be that conditionalizing resource declarations based on 
class parameters is preferable to inheriting/overriding resources.  There 
are valid arguments for that position, but I think the jury is very much 
still out.
 
>
> Like everybody, I''d like to write my module using the best and
most
> flexible way so it will be easy to modify and to maintain. I don''t
like
> to redo my modules from scratch each time :) 
>
> So, any inputs/insights are welcome. What direction should I/we take ?
>

It''s just too broad a question.  I don''t think there are very
many rules
that would reasonably apply to every module in every context.  Here are two 
that you should consider adopting, however:

   - Declarations of interface classes (those intended to be declared 
   outside the module) should be performed via the ''include''
or ''require''
   function, as appropriate, not via the newer parametrized-style class 
   declaration syntax.  Such classes should rely on hiera for any needed 
   customization data.  In Puppet 2.x, that probably means avoiding 
   parametrizing such classes in the first place.
   - Class, definition, and parameter names should be nouns or noun phrases 
   to emphasize that they describe state, not the actions required to achieve 
   that state.  Good: "apache", "apache::installed",
"apache::disabled".  Bad:
   "tomcat::remove", "perl::update". Exceptions may be made
for Exec resources.

 
>
> I really believe that puppetlabs or a set of gurus should set these 
> guidelines so everybody would develop the same way. At the moment,
it''s
> like "there is more than one way to do it" (perl) but a lot of
people
> are after a "this is the best way to do it", to not be confused
with
> "there is only way to do it" :) 
>
>
That''s just not going to happen.  That is, Puppetlabs or some group of 
gurus might produce such a document, but it would be unlikely to garner 
universal support, much less universal compliance.  The best I think you 
can do is come up with standards for your own organization.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/yCim_UTPeCwJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 10/22/2012 06:17 PM, jcbollinger wrote:
> 
> 
> On Monday, October 22, 2012 2:33:18 AM UTC-5, Thomas Guthmann wrote:
> 
>     Hi,
> 
>     My dilemma is how should I write my module to be "next-gen" ?
>     Not from a code point of view but from a design/layout point of view.
> 
> 
> 
> I don''t think much has changed in that area in a long time. 
Styles have
> come and gone, but the only new thing I can think of since 0.24 that had
> much impact on the fundamental questions is hiera.
> 
>  
> 
> 
>     We wrote our modules for 2.6.x like every beginner would have done:
>     write everything in init.pp.
> 
>     But now, the fashion is to use parameterized classes and Hiera which we
>     will use with puppet 2.7.x.
> 
> 
> 
> It would be wise to make your decisions based on an evaluation of
> technical merit, not based on what happens to be fashionable in some
> segment of the Internet at the moment.  And naturally, you should be
> wary of taking advice from random people on the ''net.  (Except me
-- I''m
> TOTALLY reliable.  :-))
> 
>  
> 
> 
>     I mainly followed/watched what Example42 was doing. They were using
>     $classname::{install|remove|disable|blah} to actually call an action in
>     their old modules. It was almost a sub class ($subclass.pp) per action.
>     It was easy to read and possibly to maintain.
> 
> 
> 
> I am all for modular code (including inside Puppet
''modules''), but what
> I think you''re asking about here is not so much module
implementation,
> but rather modules'' interfaces to the rest of Puppet.  Thus I take
it as
> a question about whether to convey information about the desired
> configuration details via class names or data (parameters or external
> data).  The two approaches provide different advantages.
> 
> Controlling everything via data (meaning either / both class parameters
> and external data) can make your main manifests very simple, and / or it
> may have advantages for the design of complementary ENCs.  On the other
> hand, putting information into class names can enable class dependencies
> to depend on that information.  For example:
"Class[''tomcat::running'']
> -> Class[''mywebapp::running'']".  As you say,
using class names in that
> way also can make manifests clearer to human readers.
> 
>  
> 
> 
>     But now Alessandro from Example42 seems to put everything back in
>     init.pp with his nextgen modules...
> 
>     I like the idea of segregating types or action per files to avoid
>     mistakes during edits but at the same time I am wondering why this
>     change for example42, is it a limitation for parameterized classes ?
> 
> 
> 
> I cannot speak for Example42, but if you''ve been following this
group
> you will know by now that my technical opinion of parametrized classes
> is that you should not write them, and as much as possible you should
> avoid using them (unless via the ''include'' or
''require'' function).  I
> have not studied the Example42 modules, but it is conceivable that some
> of the technical limitations of parametrized classes influenced the
> decision to move to unitary interfaces.
IMHO it''s OK to write them as long as you don''t declare them
as
resources. Benefits:

- backwards compatible
- no direct hiera calls in classes
- "class::params" has all the defaults

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi Thomas and John,
just bumped in this thread  so sorry for replying so late...

On Monday, October 22, 2012 6:17:50 PM UTC+2, jcbollinger
wrote:
>
>
>
> On Monday, October 22, 2012 2:33:18 AM UTC-5, Thomas Guthmann wrote:
>>
>> Hi, 
>>
>> My dilemma is how should I write my module to be "next-gen" ?
>> Not from a code point of view but from a design/layout point of view.
>>
>
>
> I don''t think much has changed in that area in a long time. 
Styles have
> come and gone, but the only new thing I can think of since 0.24 that had 
> much impact on the fundamental questions is hiera.
>
>  
>
>>
>> We wrote our modules for 2.6.x like every beginner would have done: 
>> write everything in init.pp. 
>>
>> But now, the fashion is to use parameterized classes and Hiera which we
>> will use with puppet 2.7.x. 
>>
>
>
> It would be wise to make your decisions based on an evaluation of 
> technical merit, not based on what happens to be fashionable in some 
> segment of the Internet at the moment.  And naturally, you should be wary 
> of taking advice from random people on the ''net.  (Except me --
I''m TOTALLY
> reliable.  :-))
>
>  
>
>>
>> I mainly followed/watched what Example42 was doing. They were using 
>> $classname::{install|remove|disable|blah} to actually call an action in
>> their old modules. It was almost a sub class ($subclass.pp) per action.
>> It was easy to read and possibly to maintain. 
>>
>
>
Actually it was easier to read but harder to maintain, especially when 
adding resources to the class.
Also I preferred a less verbose and more compact alternative.

 
>
> I am all for modular code (including inside Puppet
''modules''), but what I
> think you''re asking about here is not so much module
implementation, but
> rather modules'' interfaces to the rest of Puppet.  Thus I take it
as a
> question about whether to convey information about the desired 
> configuration details via class names or data (parameters or external 
> data).  The two approaches provide different advantages.
>
> Controlling everything via data (meaning either / both class parameters 
> and external data) can make your main manifests very simple, and / or it 
> may have advantages for the design of complementary ENCs.  On the other 
> hand, putting information into class names can enable class dependencies to
> depend on that information.  For example:
"Class[''tomcat::running''] ->
> Class[''mywebapp::running'']".  As you say, using
class names in that way
> also can make manifests clearer to human readers.
>
>  
>
>>
>> But now Alessandro from Example42 seems to put everything back in 
>> init.pp with his nextgen modules... 
>>
>> I like the idea of segregating types or action per files to avoid 
>> mistakes during edits but at the same time I am wondering why this 
>> change for example42, is it a limitation for parameterized classes ? 
>
>
> I cannot speak for Example42, but if you''ve been following this
group you
> will know by now that my technical opinion of parametrized classes is that 
> you should not write them, and as much as possible you should avoid using 
> them (unless via the ''include'' or
''require'' function).  I have not studied
> the Example42 modules, but it is conceivable that some of the technical 
> limitations of parametrized classes influenced the decision to move to 
> unitary interfaces.
>
Exactly.

Also, a unitary interface actually doesn''t require to use the module
only
as parametrized class, but requires a parametrized class.
At first I was somehow skeptical about their usage (the non duplicability 
inside the same catalog seemed a PITA)  I used happily the "set variables /
include class" pattern, also because it was the only possible thing to do.
So, I decided to make the new modules usable in different ways:
- As pure parametrized classes
- With the "set variable (at top scope/ENC or via Hiera) & include
class"
pattern 
- Eventually mixing these patterns.

Seems somehow weird,
but it''s supposed to not enforce a specific "data separation"
approach
(remember that Hiera is default only from Puppet 3.0).

Then I was convinced that  with parametrized classes you provide an API to 
your module, but this doesn''t force to use is as a parametrized class.

So instead of 
include foo::absent

You can have these basic alternatives:
class { ''foo'':
  absent => true,
}

or:
$::foo_absent = true
include foo

or 
hiera(''foo_absent'')
include foo

> The main potential issue there would be that any parametrized-style 
> declaration for a given class must be the *first* declaration Puppet 
> parses for that class.  It follows that there can only be one such 
> declaration of each class (whereas there can be any number of
''include'' or
> ''require'' declarations of a class).  There are many
implications, among
> them that if the module supports users declaring, say, mymodule::installed,
> then other classes in the module (mymodule::running, for instance) must not
> use parametrized-style declarations of that class.  That''s a bit
of a
> catch-22, however, because if the user *doesn''t* declare the first
class,
> then other classes in the module may need to declare it with parameters.
>
> Alternatively, if the example42 modules are truly subclassing (which is 
> not inherently implied by namespace nesting ala
"classname::blah") then the
> perception may just be that conditionalizing resource declarations based on
> class parameters is preferable to inheriting/overriding resources.  There 
> are valid arguments for that position, but I think the jury is very much 
> still out.
>
True.
 
>  
>
>>
>> Like everybody, I''d like to write my module using the best and
most
>> flexible way so it will be easy to modify and to maintain. I
don''t like
>> to redo my modules from scratch each time :) 
>>
>> So, any inputs/insights are welcome. What direction should I/we take ?
>>
>
My 2 cents:
- Try to define a module template that works for each of your main cases: 
typical package-service-configfile pattern, simple package+optional config 
module, module for a [java|php|rails..] web application ... when doing this 
avoid to specify module''s specific naming... do it "sed
friendly" use
internal variables to manage the module behaviour and place os differences 
in a params.pp (or equivalent) class. This allows you to write a script 
that quickly generates a new module from your template (like this : 
https://github.com/example42/Example42-tools/blob/master/module_clone.sh ) :
- 5 seconds to create a new module
- 10 minutes to customize the params.pp for different OS support
- x minutes to add all the module''s specific stuff
=  a very coherent set of modules that fits your needs.

- When defining this template for your modules consider if you want to 
reuse them, and let people use them, or you prefer/need to optimize/tailor 
them for your current needs.
The second option is somehow easier and quicker, but obviously  less 
reusable.
If you want to reuse or distribute your modules you must allow as much 
freedom as possible for module''s users to be able to adapt the modules
to
their sites WITHOUT the need to alter the module''s code. If you want
this,
well, let''s discuss about it... Example42 modules are all about
reusability
(with some, optional, opinionated add-ons for puppi, monitoring, 
firewalling) any advancement on this field is welcomed. 

- You do not need to make all your new modules via templates. For complex 
applications look for the existing modules around, evaluate how much effort 
you eventually need to adapt them to your patterns: decide if to fork or 
pull request, possibly the second. To the basic layout you may want to add 
module specific parameters, that make it easier to configure it.


>
> It''s just too broad a question.  I don''t think there are
very many rules
> that would reasonably apply to every module in every context.  Here are two
> that you should consider adopting, however:
>
>    - Declarations of interface classes (those intended to be declared 
>    outside the module) should be performed via the
''include'' or ''require''
>    function, as appropriate, not via the newer parametrized-style class 
>    declaration syntax.  Such classes should rely on hiera for any needed 
>    customization data.  In Puppet 2.x, that probably means avoiding 
>    parametrizing such classes in the first place.
>    - Class, definition, and parameter names should be nouns or noun 
>    phrases to emphasize that they describe state, not the actions required
to
>    achieve that state.  Good: "apache",
"apache::installed",
>    "apache::disabled".  Bad: "tomcat::remove",
"perl::update". Exceptions may
>    be made for Exec resources.
>
>  
>
>>
>> I really believe that puppetlabs or a set of gurus should set these 
>> guidelines so everybody would develop the same way. At the moment,
it''s
>> like "there is more than one way to do it" (perl) but a lot
of people
>> are after a "this is the best way to do it", to not be
confused with
>> "there is only way to do it" :) 
>>
>>
> That''s just not going to happen.  That is, Puppetlabs or some
group of
> gurus might produce such a document, but it would be unlikely to garner 
> universal support, much less universal compliance.  
>
The best I think you can do is come up with standards for your own
> organization.
>
>
Actually something can be tried, again.
At least at the API level some guidelines can be suggested:
- Provide parameters to manage the module behaviour (present, absent, 
disable...) 
- Allow alternative options on how to provide configurations (source, 
template, options) 
- Allow the saferuns and debugs (audit, debug)
- Set a naming convention for these parameters.
and lets the gurus propose guidelines.

Oh my cents,
Al

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/GM8R6c44wukJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.