Puppet users - Apr 2008 - Lab42 Puppet Infrastructure: request for comments

Dear Puppet community,
I would like to anticipate you the "Lab42 Puppet Infrastructure" a
growing set of recipes, freely usable, with these goals:

- Provide a coherent puppet environment
Paths, filenames, classnames and other elements have common structure.
Logical, intuitive, simple.
Modules structure follows Reductive Labs'' guidelines.
Modules names are application oriented ( apache, lighttpd ..)

- Modular, easy to use and integrate logic
Modules can be simply imported and use default variables, that can be
overriden with custom values, defined in a clean, single inheritance,
nodes grouping tree.

- System Administrator oriented
Puppet code tends to be simple, as a preference configurations are
provided as files (easier to integrate from an existing non puppet
automated infrastructure). Templates are used but not overrused.
You do not need to be a programmer in order to understand how to use
and adapt the modules.

- Live from scratch. Adapt according to custom needs
Modules work "out of the box" with default configuration but
customization and personalization is easy (and reccomended)

- Cross Operating System
Modules are ready for easy customization according to operative
system, version and architecture.
Default values are Redhat based (work generally with Fedora and
Centos).
Future planned support also for Suse and Debian breeds.

- Fitting for heterogenous projects and network infrastructures
The logic of the infrastructure is defined in a single custom
"project" module.
Here you define nodes, inheritance from more general grouping, roles.
The logic of nodes grouping and inheritance can be adapted to own
reality.

- Best practices references
Application installation and default configurations are oriented to
Lab42 best practices.
Subclasses are defined in order to use optimized configurations and
extend standard functionalities.

Work is in progress (let''s say we are at alpha stage, but it works)
you can find sources at
http://live.lab42.it/puppetinfrastructure/

You can consider as "standard" module for an application this one:
http://live.lab42.it/puppetinfrastructure/browser/modules/lighttpd
and view how different custom infrastructures can be defined in single
modules here:
http://live.lab42.it/puppetinfrastructure/browser/modules/project_general
or here:
http://live.lab42.it/puppetinfrastructure/browser/modules/project_lab42
(this is a live working infrastructure for the Lab42 servers (who said
security though obscurity?)

I''d love to receive from Puppet experts and general users a feedback
on the module organization logic and approach, in order to sum up
ideas and start to leverage modules layout (now they still don''t share
the same logic) and make more coherent the whole project.
I know and respect the great work of David Schmitt but I''ve chosen a
quite different approach (cross distro and with few templates) and
would like to discuss if it can be a usable alternative.
Any comment or suggestion is welcomed.

Best regards & Puppet for All !

Alessandro Franceschi
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Wow, I just wanted to be the first to thank you for your generous gift to us
all.   Tomorrow at work I''ll upload my current state of modules and
drop you
a link to the URL so you can have a look through.  Most of my work so far
mirrors the kind of modules you''ve written, but maybe someone will get
something useful out of them!

Thanks,

On Wed, Apr 16, 2008 at 6:20 PM, Al @ Lab42 <lab42.it@gmail.com> wrote:
>
> Dear Puppet community,
> I would like to anticipate you the "Lab42 Puppet Infrastructure"
a
> growing set of recipes, freely usable, with these goals:
>
> - Provide a coherent puppet environment
> Paths, filenames, classnames and other elements have common structure.
> Logical, intuitive, simple.
> Modules structure follows Reductive Labs'' guidelines.
> Modules names are application oriented ( apache, lighttpd ..)
>
> - Modular, easy to use and integrate logic
> Modules can be simply imported and use default variables, that can be
> overriden with custom values, defined in a clean, single inheritance,
> nodes grouping tree.
>
> - System Administrator oriented
> Puppet code tends to be simple, as a preference configurations are
> provided as files (easier to integrate from an existing non puppet
> automated infrastructure). Templates are used but not overrused.
> You do not need to be a programmer in order to understand how to use
> and adapt the modules.
>
> - Live from scratch. Adapt according to custom needs
> Modules work "out of the box" with default configuration but
> customization and personalization is easy (and reccomended)
>
> - Cross Operating System
> Modules are ready for easy customization according to operative
> system, version and architecture.
> Default values are Redhat based (work generally with Fedora and
> Centos).
> Future planned support also for Suse and Debian breeds.
>
> - Fitting for heterogenous projects and network infrastructures
> The logic of the infrastructure is defined in a single custom
> "project" module.
> Here you define nodes, inheritance from more general grouping, roles.
> The logic of nodes grouping and inheritance can be adapted to own
> reality.
>
> - Best practices references
> Application installation and default configurations are oriented to
> Lab42 best practices.
> Subclasses are defined in order to use optimized configurations and
> extend standard functionalities.
>
> Work is in progress (let''s say we are at alpha stage, but it
works)
> you can find sources at
> http://live.lab42.it/puppetinfrastructure/
>
> You can consider as "standard" module for an application this
one:
> http://live.lab42.it/puppetinfrastructure/browser/modules/lighttpd
> and view how different custom infrastructures can be defined in single
> modules here:
> http://live.lab42.it/puppetinfrastructure/browser/modules/project_general
> or here:
> http://live.lab42.it/puppetinfrastructure/browser/modules/project_lab42
> (this is a live working infrastructure for the Lab42 servers (who said
> security though obscurity?)
>
> I''d love to receive from Puppet experts and general users a
feedback
> on the module organization logic and approach, in order to sum up
> ideas and start to leverage modules layout (now they still don''t
share
> the same logic) and make more coherent the whole project.
> I know and respect the great work of David Schmitt but I''ve chosen
a
> quite different approach (cross distro and with few templates) and
> would like to discuss if it can be a usable alternative.
> Any comment or suggestion is welcomed.
>
> Best regards & Puppet for All !
>
> Alessandro Franceschi
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi
> I would like to anticipate you the "Lab42 Puppet Infrastructure"
a
> growing set of recipes, freely usable, with these goals:
as several people are coming up more and more with publishing their
modules, we should begin to think about a "central" community driven
platform for modules.
DavidS published already some modules, me as well ours, which might be
forks of DavidS one''s as well some own new modules and their are a lot
of others as well.

So after some prework and proposal of DavidS on
http://reductivelabs.com/trac/puppet/wiki/CommonModules I would propose
to continue with his proposal and try to figure out how we could do some
kind of community place for modules which won''t get cluttered into
x-different module versions which won''t be compatible to each another.

So after some discussion yesterday with lak and someone else (sorry lost
my backlog :-/ ) I would note the following points:

- a central wiki page which would contain all modules inside this
community process
- a central git-repository (proposal was to use github.com) with one
maintainer for each module.
- possibilitities to add other development git-repos for the specific
module on the wiki on a sublevel. which are forks of this specific modules
- avoid cluttering of modules, the goal is to get one module which
should fit for nearly all people. (might be an utopia ;) ) and at the
end there shouldn''t be in my opinion for example a dozen different
modules for apache.
- documentation: every module should fit some documentation standards we
define and each maintainer tries to achieve them for their module.
- free license: huh let''s start another flamewar :P but what
i''d like to
say is that the modules should be published in the sense of free
software (so bsd or gpl) and it should be possible to use them each
another. i don''t care which license the specific module have, the
important thing is that they are free and can be used along each another.
- definition of dependencies: if modules use each another they should
document their dependencies. there was once a discussion about how that
could be done, but i don''t know how far this is yet.

so this are my notes on that, any other thought ideas?

greets pete


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

cool idea.

i can provide hosting for that. I have a vps and also can get a more cool
thing through the company.

Cheers.

On Thu, Apr 17, 2008 at 9:40 AM, Peter Meier <peter.meier@immerda.ch>
wrote:
>
> Hi
>
> > I would like to anticipate you the "Lab42 Puppet
Infrastructure" a
> > growing set of recipes, freely usable, with these goals:
>
> as several people are coming up more and more with publishing their
> modules, we should begin to think about a "central" community
driven
> platform for modules.
> DavidS published already some modules, me as well ours, which might be
> forks of DavidS one''s as well some own new modules and their are a
lot
> of others as well.
>
> So after some prework and proposal of DavidS on
> http://reductivelabs.com/trac/puppet/wiki/CommonModules I would propose
> to continue with his proposal and try to figure out how we could do some
> kind of community place for modules which won''t get cluttered into
> x-different module versions which won''t be compatible to each
another.
>
> So after some discussion yesterday with lak and someone else (sorry lost
> my backlog :-/ ) I would note the following points:
>
> - a central wiki page which would contain all modules inside this
> community process
> - a central git-repository (proposal was to use github.com) with one
> maintainer for each module.
> - possibilitities to add other development git-repos for the specific
> module on the wiki on a sublevel. which are forks of this specific modules
> - avoid cluttering of modules, the goal is to get one module which
> should fit for nearly all people. (might be an utopia ;) ) and at the
> end there shouldn''t be in my opinion for example a dozen different
> modules for apache.
> - documentation: every module should fit some documentation standards we
> define and each maintainer tries to achieve them for their module.
> - free license: huh let''s start another flamewar :P but what
i''d like to
> say is that the modules should be published in the sense of free
> software (so bsd or gpl) and it should be possible to use them each
> another. i don''t care which license the specific module have, the
> important thing is that they are free and can be used along each another.
> - definition of dependencies: if modules use each another they should
> document their dependencies. there was once a discussion about how that
> could be done, but i don''t know how far this is yet.
>
> so this are my notes on that, any other thought ideas?
>
> greets pete
>
>
> >
>

-- 
-------> Dive to Live, Live to Dive <-------

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Peter Meier wrote:
> So after some prework and proposal of DavidS on
> http://reductivelabs.com/trac/puppet/wiki/CommonModules I would propose
> to continue with his proposal and try to figure out how we could do some
> kind of community place for modules which won''t get cluttered into
> x-different module versions which won''t be compatible to each
another.
This is going to be interesting- some thoughts:

- - how do you handle the issue of distribution/platform inter-operability
in a consistent manner?

- - Also how do you handle cross-module testing, inter-relationships and
compatibility?  You need to consider some form of testing methodology or
even test scripts to ensure updates to some modules don''t break other
modules.
> - a central wiki page which would contain all modules inside this
> community process
Happy to help with this.
> - a central git-repository (proposal was to use github.com) with one
> maintainer for each module.
Hmm - I think you''d need to consider more than one maintainer - easier
to ensure a module isn''t orphaned.  Perhaps a group who look after
collections of modules.
> - possibilitities to add other development git-repos for the specific
> module on the wiki on a sublevel. which are forks of this specific modules
What would the purpose of the forks be?
> - avoid cluttering of modules, the goal is to get one module which
> should fit for nearly all people. (might be an utopia ;) ) and at the
> end there shouldn''t be in my opinion for example a dozen different
> modules for apache.
Maybe a core module and sub-modules by OS?
> - documentation: every module should fit some documentation standards we
> define and each maintainer tries to achieve them for their module.
Agreed - happy to help develop the standards.
> - free license: huh let''s start another flamewar :P but what
i''d like to
> say is that the modules should be published in the sense of free
> software (so bsd or gpl) and it should be possible to use them each
> another. i don''t care which license the specific module have, the
> important thing is that they are free and can be used along each another.
It''d be nice if the license was consistent with Puppet - GPLv2 - but I
guess not critical.
> - definition of dependencies: if modules use each another they should
> document their dependencies. there was once a discussion about how that
> could be done, but i don''t know how far this is yet.
This is a function of a module README.  I think every module needs one
of these - best based on a template that ensures all required
information is available for the module.
> so this are my notes on that, any other thought ideas?
Sounds like a great concept.  Be good to hear from all the current
module maintainers.

Cheers

James

- --
James Turnbull (james@lovedthanlost.net)
- --
Author of:
- - Pulling Strings with Puppet
(http://www.amazon.com/gp/product/1590599780/)
- - Pro Nagios 2.0
(http://www.amazon.com/gp/product/1590596099/)
- - Hardening Linux
(http://www.amazon.com/gp/product/1590594444/)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIB05Q9hTGvAxC30ARAmhDAKCP0HX7IKJSmDLqw0zQVJxN2vg5CACgwrXY
W3VWO3jSwrhgjiFohtFh0oM=hO6I
-----END PGP SIGNATURE-----

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Peter,
I agree.
A central community driven platform for modules is a natural evolution
of the Puppet user''s community.
I''m going to read
http://reductivelabs.com/trac/puppet/wiki/CommonModules
in order to have the idea of what could be the way to follow.
> - a central wiki page
> - a central git-repository
> - possibilitities to add other development git-repos for the specific
> module o
> - avoid cluttering of modules
> - documentation: every module should fit some documentation standards
> - free license
> - definition of dependencies
> so this are my notes on that, any other thought ideas?
I generally agree on everything, let me add some few notes which may
be too specific or just implicit in what you''ve written:
- modules should be operating system indipendent and support different
versions of an os
- by default a module should not overwrite any configuration file
(classes define the stage, subclasses configures)
- there should be somehow freedom (on user''s side) on how they should
provide configurations (some times templates are more useful, other
times plain sourced files to edit according to custom needs are more
comfortable)
- "infrastructural modules" could be defined/suggested in order to
setup ready to adapt IT scenarios: here I''d define nodes, nodes
inheritances, roles and inclusion of the other "application" modules.
- there should be a common backup wrapper module, with recovery
facilities, which can use different backup systems (defined by the
user) but should be invoked in a standard way

A good starting point in order to define a common module layout would
be to provide a proposal for a typical application (who said apache?)
and see how the different solution could be merged and what approach
fit better various scenarios.

My time, when available, and efforts are at disposal
al
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Apr 17, 6:40 am, Peter Meier <peter.me...@immerda.ch>
wrote:
>
> - a central git-repository (proposal was to use github.com) with one
> maintainer for each module.
> - possibilitities to add other development git-repos for the specific
> module on the wiki on a sublevel. which are forks of this specific modules
Is there a general bias towards git in the puppet community? I haven''t
used it. We use subversion for our puppet manifests because that is
what the web team uses for content and development version control. I
have a number of modules I would like to share with the puppet
community, but because the Common Modules page refers to git, I don''t
particularly want to set up a separate repo just for this purpose.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Apr 17, 2008, at 11:43 AM, jtimberman wrote:
> Is there a general bias towards git in the puppet community? I
haven''t
> used it. We use subversion for our puppet manifests because that is
> what the web team uses for content and development version control. I
> have a number of modules I would like to share with the puppet
> community, but because the Common Modules page refers to git, I
don''t
> particularly want to set up a separate repo just for this purpose.

The probable reason for the apparent git bias is because I somewhat  
arbitrarily decided to switch Puppet to git.

It might make sense to support other SCM tools, but I can''t recommend  
highly enough that only distributed version control tools be used.  In  
particular, git''s support for in-repository branches is really just  
fantastic, I can''t say enough good about it.

For modules especially, where lots of people will have various small  
modifications to a module, it makes a lot of sense.

-- 
He played the king as if afraid someone else would play the ace.
--John Mason Brown
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi
> Is there a general bias towards git in the puppet community? I
haven''t
> used it. We use subversion for our puppet manifests because that is
> what the web team uses for content and development version control. I
> have a number of modules I would like to share with the puppet
> community, but because the Common Modules page refers to git, I
don''t
> particularly want to set up a separate repo just for this purpose.
i could provide some ruby script i use to dump, on a half-daily basis,
our modules from one subversion repo into different git-repos.

it''s just that sharing your code on a horizontal, community and
distributed manner is with git very easy. as well to merge different
sources of modules (branches) together.

greets pete

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Apr 17, 2008, at 6:27 AM, Al @ Lab42 wrote:
>
> Peter,
> I agree.
> A central community driven platform for modules is a natural evolution
> of the Puppet user''s community.
> I''m going to read
http://reductivelabs.com/trac/puppet/wiki/CommonModules
> in order to have the idea of what could be the way to follow.
>
>> - a central wiki page
>> - a central git-repository
>> - possibilitities to add other development git-repos for the specific
>> module o
>> - avoid cluttering of modules
>> - documentation: every module should fit some documentation standards
>> - free license
>> - definition of dependencies
>> so this are my notes on that, any other thought ideas?
>
> I generally agree on everything, let me add some few notes which may
> be too specific or just implicit in what you''ve written:
> - modules should be operating system indipendent and support different
> versions of an os
> - by default a module should not overwrite any configuration file
> (classes define the stage, subclasses configures)
> - there should be somehow freedom (on user''s side) on how they
should
> provide configurations (some times templates are more useful, other
> times plain sourced files to edit according to custom needs are more
> comfortable)
> - "infrastructural modules" could be defined/suggested in order
to
> setup ready to adapt IT scenarios: here I''d define nodes, nodes
> inheritances, roles and inclusion of the other "application"
modules.
> - there should be a common backup wrapper module, with recovery
> facilities, which can use different backup systems (defined by the
> user) but should be invoked in a standard way
>
> A good starting point in order to define a common module layout would
> be to provide a proposal for a typical application (who said apache?)
> and see how the different solution could be merged and what approach
> fit better various scenarios.
>
> My time, when available, and efforts are at disposal
See my suggestion for a metadata file for modules that would help  
define much of this:
http://groups.google.com/group/puppet-users/browse_thread/thread/66b90ee0e3a229c2?hl=en

-Blake

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

I''ve also offered here to support a "dedicated" unmanaged VPS
to the
puppet project, mentioned to Luke before already had something in
place.

-L
--
Larry Ludwig
Empowering Media
1-866-792-0489 x600
Managed and Unmanaged Xen VPSes
http://www.hostcube.com/

On Apr 17, 8:45 am, "Nicolas Arias" <nicoar...@gmail.com>
wrote:
> cool idea.
>
> i can provide hosting for that. I have a vps and also can get a more cool
> thing through the company.
>
> Cheers.
>
>
>
> On Thu, Apr 17, 2008 at 9:40 AM, Peter Meier <peter.me...@immerda.ch>
wrote:
>
> > Hi
>
> > > I would like to anticipate you the "Lab42 Puppet
Infrastructure" a
> > > growing set of recipes, freely usable, with these goals:
>
> > as several people are coming up more and more with publishing their
> > modules, we should begin to think about a "central"
community driven
> > platform for modules.
> > DavidS published already some modules, me as well ours, which might be
> > forks of DavidS one''s as well some own new modules and their
are a lot
> > of others as well.
>
> > So after some prework and proposal of DavidS on
> >http://reductivelabs.com/trac/puppet/wiki/CommonModulesI would propose
> > to continue with his proposal and try to figure out how we could do
some
> > kind of community place for modules which won''t get cluttered
into
> > x-different module versions which won''t be compatible to each
another.
>
> > So after some discussion yesterday with lak and someone else (sorry
lost
> > my backlog :-/ ) I would note the following points:
>
> > - a central wiki page which would contain all modules inside this
> > community process
> > - a central git-repository (proposal was to use github.com) with one
> > maintainer for each module.
> > - possibilitities to add other development git-repos for the specific
> > module on the wiki on a sublevel. which are forks of this specific
modules
> > - avoid cluttering of modules, the goal is to get one module which
> > should fit for nearly all people. (might be an utopia ;) ) and at the
> > end there shouldn''t be in my opinion for example a dozen
different
> > modules for apache.
> > - documentation: every module should fit some documentation standards
we
> > define and each maintainer tries to achieve them for their module.
> > - free license: huh let''s start another flamewar :P but what
i''d like to
> > say is that the modules should be published in the sense of free
> > software (so bsd or gpl) and it should be possible to use them each
> > another. i don''t care which license the specific module have,
the
> > important thing is that they are free and can be used along each
another.
> > - definition of dependencies: if modules use each another they should
> > document their dependencies. there was once a discussion about how
that
> > could be done, but i don''t know how far this is yet.
>
> > so this are my notes on that, any other thought ideas?
>
> > greets pete
>
> --
> -------> Dive to Live, Live to Dive <-------
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

In order to get fast into practical land I would list the main
activities suggested in this thread:
1- Setup of a central community driven platform site (wiki, vps and
all the rest)
2- Definition of the collaboration method and code management /
maintenance procedures (http://reductivelabs.com/trac/puppet/wiki/
CommonModules can be a starting point)
3- Definition of how to standardize modules layout and templates
4- Definition of possible module metadata layouts ( such as what is
suggested in
http://groups.google.com/group/puppet-users/browse_thread/thread/66b90ee0e3a229c2?hl=en
)
5- Definition (possibile?) of a general module organization that can
deal with conflicts, forks, parallel modules, interdipendencies and so
on

Since I need to define as soon as possible a general module layout,
and would like to do it in a way compatible with "community standards"
or, at least, best practices, I''m particularly interested in pushing
point 3.
For this reason I''ve opened a ticket in
http://live.lab42.it/puppetinfrastructure/ticket/1
in order to discuss how to design a module standard in the Lab42
Puppet Infrastructure but I''m at disposal to discuss this matter
anywhere: on this list, on a dedicated site, or wherever.
What I think is importart is to start to actively share ideas on this
specific issue, proposing how everyone has made a typical module and
how we can merge/evaluate the various approaches.

The way is not so easy, because maybe we don''t even agree of WHAT a
modules should be, and that''s an issue to consider before definiting
HOW it should be.
I anticipate here what I''ve written in the cited link, according to me
priorities to the module layout are:
- Module must be cross-platform. Support for different distros/os
should not be done on subclasses (ex: lighttpd should work with Debian/
RedHat etc, withouth the need of a lighttpd::debian)
- One module - one application. Standard layout. Standard READMEs and
requirements/conflicts definitions
- Standard, non invasive, behaviour (ex: class lighttpd ) should
install packages, run services but not overwrite any existing file by
default
- Specific configurations (ex: class lighttpd::php ,
lighttpd::hardened) can provide custom configuration files (via
templates or static files)
- Templates should not be over-used: an user should not be compelled
to understand HOW the module work and all its variables in order to
use it
- Use of defines is good to autoconfigure cross-application modules
(ex: checks of Nagios, Munin ecc.)
- Use of sourced static files or templates could be a choice of the
user, according to custom needs, to define in custom subclassess (ex:
lighttpd::lab42 )

Many of these points should be verified and compared or bare puppet
code, so I invite whoever is interested to post, link, provide in some
forms how he would make a standard module for a typical application.
We can choose any one, I''ve started with
http://live.lab42.it/puppetinfrastructure/browser/modules/lighttpd
even if I don''t know Lighttpd much, but we can use any application as
reference and start to provide modules proposals for it.

Al
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi
> - Also how do you handle cross-module testing, inter-relationships and
> compatibility?  You need to consider some form of testing methodology or
> even test scripts to ensure updates to some modules don''t break
other
> modules.
I agree that this point is quite hard. And looking at the current state
of writing modules that there is (yet) no way to write module
documentation not module depenecies nor tests, it might be something
that can evolve with the growing module repository? do we really have
this from the beginning on?
>> - a central wiki page which would contain all modules inside this
>> community process
> 
> Happy to help with this.
How about to start on wiki:CommonModules to document the process an idea
a bit more. I really like the work already done by DavidS, it helped me
a lot! (so thanks!!) and to start wiki:CommonModulesList and try to get
a starting point?

Maybe it would be good to start with one (rather simple and easy)
module, to see that there a plenty of people with different ideas :)
>> - a central git-repository (proposal was to use github.com) with one
>> maintainer for each module.
> 
> Hmm - I think you''d need to consider more than one maintainer -
easier
> to ensure a module isn''t orphaned.  Perhaps a group who look after
> collections of modules.
> 
>> - possibilitities to add other development git-repos for the specific
>> module on the wiki on a sublevel. which are forks of this specific
modules
> 
> What would the purpose of the forks be?
this was actually the idea of this kind of group of maintainers, but I
agree that it might be better to have more than one which have push
access to the official one. This one should be treated as the stable and
working one. All the others might be forks or modules in development and
therefor to bee seen as rather unstable.
>> - avoid cluttering of modules, the goal is to get one module which
>> should fit for nearly all people. (might be an utopia ;) ) and at the
>> end there shouldn''t be in my opinion for example a dozen
different
>> modules for apache.
> 
> Maybe a core module and sub-modules by OS?
yeah maybe. however the core-module should do one every OS the same.
>> - documentation: every module should fit some documentation standards
we
>> define and each maintainer tries to achieve them for their module.
> 
> Agreed - happy to help develop the standards.
might be a starting point for the wiki page.
>> - free license: huh let''s start another flamewar :P but what
i''d like to
>> say is that the modules should be published in the sense of free
>> software (so bsd or gpl) and it should be possible to use them each
>> another. i don''t care which license the specific module have,
the
>> important thing is that they are free and can be used along each
another.
> 
> It''d be nice if the license was consistent with Puppet - GPLv2 -
but I
> guess not critical.
I would be fine with that and I think it makes sense that it stay
consistent.

greets Pete

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thursday 17 April 2008, James Turnbull wrote:
> Peter Meier wrote:
> > So after some prework and proposal of DavidS on
> > http://reductivelabs.com/trac/puppet/wiki/CommonModules I would
propose
> > to continue with his proposal and try to figure out how we could do
some
> > kind of community place for modules which won''t get cluttered
into
> > x-different module versions which won''t be compatible to each
another.
>
> This is going to be interesting- some thoughts:
>
> - how do you handle the issue of distribution/platform inter-operability
> in a consistent manner?
After suggestions from the immerda.ch guys, I''ve created a distro/os
aware
branch of my apache module:

http://git.black.co.at/?p=module-apache;a=shortlog;h=distros

The main class dispatches to a specific subclass by $operatingsystem:

class apache {
	include "apache::${operatingsystem}"
}

and the additional code can be autoloaded:

apache/manifests/debian.pp contains the apache::debian class:

http://git.black.co.at/?p=module-apache;a=blob;f=manifests/debian.pp;h=1ad54ea9a34d73b62ccd6a35199eb9d6b61e4515;hb=distros

class apache::debian inherits apache::base { 
	Package["apache"] { name => apache2 }
	# ...
}

This class contains all the Debian (etch) specifica. Since I''ve no
other
systems to test and not enough time anyways, this is rather unfinished, but 
the structure and intent should be clear.

> - Also how do you handle cross-module testing, inter-relationships and
> compatibility?  You need to consider some form of testing methodology or
> even test scripts to ensure updates to some modules don''t break
other
> modules.
This was already discussed at length without any definitife outcome. People 
seem to do manual testing, report watching and nagios testing.
> > - a central wiki page which would contain all modules inside this
> > community process
> Happy to help with this.
As already mentioned, I started CommonModules and CompleteConfiguration on the 
wiki with the intention of kicking off such a community effort, but never 
came around to actually do a formal release.
> > - a central git-repository (proposal was to use github.com) with one
> > maintainer for each module.
> Hmm - I think you''d need to consider more than one maintainer -
easier
> to ensure a module isn''t orphaned.  Perhaps a group who look after
> collections of modules.
I think most people will have a single Platform but multiple Applications, 
because they come from a "Redhat shop" or - like myself - from a
"Debian
shop", but have to run apache, nagios, munin, etc. 

This suggests to me that modules (which are mostly per-application) are not 
the best fit for maintenance units anyways.
> > - avoid cluttering of modules, the goal is to get one module which
> > should fit for nearly all people. (might be an utopia ;) ) and at the
> > end there shouldn''t be in my opinion for example a dozen
different
> > modules for apache.
> Maybe a core module and sub-modules by OS?
As detailled above, OS-support within a module can mostly be constrained to a 
single autoloaded file in the style of native "providers".
> > - free license: huh let''s start another flamewar :P but what
i''d like to
> > say is that the modules should be published in the sense of free
> > software (so bsd or gpl) and it should be possible to use them each
> > another. i don''t care which license the specific module have,
the
> > important thing is that they are free and can be used along each
another.
>
> It''d be nice if the license was consistent with Puppet - GPLv2 -
but I
> guess not critical.
I released all my modules under a simple BSD license because I don''t
care. So
GPLv2 would be OK for me too.



Regards, DavidS

- -- 
The primary freedom of open source is not the freedom from cost, but the free-
dom to shape software to do what you want. This freedom is /never/ exercised
without cost, but is available /at all/ only by accepting the very different
costs associated with open source, costs not in money, but in time and effort.
- -- http://www.schierer.org/~luke/log/20070710-1129/on-forks-and-forking
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIDyop/Pp1N6Uzh0URAj5xAJwPT8xZtlEJNkDWz5BOTsS0TAo4PwCfdJof
e51d40DOqRIkQcRuT0UX8mg=p0v+
-----END PGP SIGNATURE-----

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

> class apache {
>         include "apache::${operatingsystem}"
>
> }
>
> and the additional code can be autoloaded:
>
> apache/manifests/debian.pp contains the apache::debian class:
>
> http://git.black.co.at/?p=module-apache;a=blob;f=manifests/debian.pp;...
>
> class apache::debian inherits apache::base {
>         Package["apache"] { name => apache2 }
>         # ...
>
> }
>
> This class contains all the Debian (etch) specifica. Since I''ve no
other
> systems to test and not enough time anyways, this is rather unfinished, but
> the structure and intent should be clear.
Yes it is, and actually I like it too.
I''m seriously reconsidering my approach on how to handle different os
on a module and follow this way.

I tended to handle everything on a single class, where there''s
something like:
        service { apache:
 	                name => $operatingsystem ? {
 	                        debian  => "apache2",
 	                        default => "httpd",
 	                        },
 	                ensure => running,
 	                enable => true,
 	                pattern => $operating ? {
 	                        debian  => "/usr/sbin/apache2",
 	                        default => "/usr/sbin/httpd",
 	                        },
	                require => Package["apache"],
 	                subscribe => File["httpd.conf"],
 	        }

handling os specific objects with something like
case $operatingsystem  {
 	                        debian:  { # Block of os specific code }
 	                        redhat:  { # Block of os specific code }
 	                        }
but that would be less readable and more error prone.

This is the kind of practical discussion I''m looking for.

I love your work David but I think that sometimes you overruse
variables and templates, which leads to some kind of over complexity
in the modules usage (the user has to understand the module logic and
know its variables in order to use it).
Sometimes an user has just to place his configuration file, with
custom values that don''t change much site-wide, on his nodes (read
"source" rather than "content").
I wonder if you''ve considered an elegant way to leave to the user
(possibly using site-specific subclasses) the decision on how to
provide configuration files, with templates or sourcing files
(eventually with different suffixes on the puppetmaster according or
nodenames or other variables).

Looking forward to get into deeper discussions and contributions from
anyone about modules organization.

Al

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wednesday 23 April 2008, Al @ Lab42 wrote:
> I love your work David but I think that sometimes you overruse
> variables and templates, which leads to some kind of over complexity
> in the modules usage (the user has to understand the module logic and
> know its variables in order to use it).
> Sometimes an user has just to place his configuration file, with
> custom values that don''t change much site-wide, on his nodes (read
> "source" rather than "content").
> I wonder if you''ve considered an elegant way to leave to the user
> (possibly using site-specific subclasses) the decision on how to
> provide configuration files, with templates or sourcing files
> (eventually with different suffixes on the puppetmaster according or
> nodenames or other variables).

While I _do_ tend to overengineer, I believe that this is rather a problem of 
documentation than anything else. Additional possibilites for overriding 
sources / contents can be added without cost to the basic functioning of the 
module like this: 

define something(override = '''') {
	file { blah: }
	case $override {
		'''': {
			File[blah] { content => template ( ''blah_default.erb'' )
}
		}
		default: {
			File[blah] { source => $override }
		}
	}
}

Other possibilities include using default places in source arrays or multiple 
templates:

file { blah: source => 
[ "puppet:///local/modulename/$fqdn",
"puppet:///modulename/default.conf" ] }


See above. Though I''d rather expect this to be implemented on a
case-by-case
basis if someone comes up with an actual specific use-case.



Regards, DavidS
- -- 
The primary freedom of open source is not the freedom from cost, but the free-
dom to shape software to do what you want. This freedom is /never/ exercised
without cost, but is available /at all/ only by accepting the very different
costs associated with open source, costs not in money, but in time and effort.
- -- http://www.schierer.org/~luke/log/20070710-1129/on-forks-and-forking
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFID6Fi/Pp1N6Uzh0URAjBLAJ9CGMfedIjpEwUg9bug5MeA+pd1HACffHyV
gedKkdBsuzjoN7jO6Km7cCw=Cgtt
-----END PGP SIGNATURE-----

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

David,
the discussion about module (and plugins) community standardization is
not important to let it fade out.

I''ve re-post here some thoughts, written in the wrong place, about how
we could move towards  standardization.

---CUT---
Puppet code and modules sharing and standardization inside the
community is really a necessity, for everybody, to save time already
spent by others.

I''m convinced that we should start with few simple steps in order to
define a Common Public Community Puppet Module Repositoy.

I can imagine basic elements which should not have incompatibility
problems:
- custom facts with possibly standard convention for variables names
- custom, well tested, functions for typical operations on files
- standard, cross operating system, and extremely basic modules for
applications

With standard basic modules I mean really basic (and easily extendable
with standard naming convention) modules that should just install the
program, manage the services and ensures the presence of the main
configuration file, being natively platfrom indipendent for these
types (see example later).

All the rest is left to own tastes, skills and needs:
- submodules to handle different distros peculiarities
- submodules to configure the application in whatever way (according
to anyone taste, since I''ve realized that every solution can be a good
solution if it fits the local needs).
So that if I decide to configure Apache with single files, templates,
building blocks of texts with whatever custom function, method and
preferred variables names, I can do it.
Starting from the basic apache module and possibily using the public
community alternative modules of different sources (ex: apache::nsc,
apache::davids, apache::lab42 or whatever public repository)

Upon these basic-common-shared and rather static modules (they should
not change too much over time) everyone, I think, can easily adapt and
extend his set of configurations as preferred.

An example of such a basic class could be something like (the use of
"custom" $update variable, which I find useful to easily defin if a
node should be automatically updated or not, can be discussed, as
everything in this proposal):

class apache {

        # This breaks if the included class does not exist
        # A way to make this work even if there''s no
apache::operatingsystem ?
        include "apache::${operatingsystem}"

        package { apache:
                name => $operatingsystem ? {
                        debian  => "apache2",
                        default => "httpd",
                        },
                ensure => $update ?{
                        true    => "latest",
                        false   => "present",
                        default => "present",
                        },
        }

        service { apache:
                name => $operatingsystem ? {
                        debian  => "apache2",
                        # community efforts would be to gather here
all alternatives
                        default => "httpd",
                        },
                ensure => running,
                enable => true,
                pattern => $operatingsystem ? {
                        debian  => "/usr/sbin/apache2",
                        default => "/usr/sbin/httpd",
                        },
                hasrestart => true,
                hasstatus => true,
                require => Package["apache"],
                subscribe => File["httpd.conf"],
        }

        file {
                "httpd.conf":
                        mode => 644, owner => root, group => root,
                        require => Package[apache],
                        backup => local,
                        ensure => present,
                        path => $operatingsystem ?{
                                default => "/etc/httpd/conf/
httpd.conf",
                        },
        }

}

Incidentally a standard structure for a basic module can be easily
generated automatically by  scripts or tools that can inherit
packages/
service names and file pathnames from existing sources. Moreover it
can be easily managed with a extendable web frontend that would gift
the point and click masses with glimpses of the power of puppet.

---CUT---

Now, everything written above should be discussed (possibly by
examples), but the small step of defining a common background upon
which variations and customizations can move, is according to me,
vital for everyone working with puppet.

Let''s place all of us in the position to know that the way we write
our custom modules is based on simple, strong, generally accepted,
shared basic classes and can benefit, when and if wanted, of a pool of
shared functions, types and generally available code.

I think that convergence on these basic properties of a module
(package, service, configuration file), considering various scenarious
for various applications, can be done quite rapidly and then
integrated in the wiki.

Let''s start to share sample ideas on how an apache basic module could
be.
End of tunnel should be be far :-)

al

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thursday 15 May 2008, Al @ Lab42 wrote:
> David,
> the discussion about module (and plugins) community standardization is
> not important to let it fade out.
Thanks for picking this up again. I''m currently very much strapped for 
time :-(
> Let''s start to share sample ideas on how an apache basic module
could
> be.
> End of tunnel should be be far :-)
After feedback from Peter, I modified my apache module to start supporting 
multiple distros. Basically that means that there is 

a) a common "contract" how others can "communicate" with the
apache. This is
handled by providing the apache::site and apache::module defines as well as 
managed directories and default names, like Service[''apache'']
and
Exec[''reload-apache''].

e.g.:
http://git.black.co.at/?p=module-apache;a=blob;f=manifests/site.pp



b) an ''apache'' class that multiplexes to the correct subclass,
basically ''include "apache::${operatingsystem}"'' and
an apache::base class
defining the primary parts of the module.

http://git.black.co.at/?p=module-apache;a=blob;f=manifests/init.pp



c) operating system specific overrides to handle the diversity.

http://git.black.co.at/?p=module-apache;a=blob;f=manifests/debian.pp




This structure is IMHO very flexible and maintainable, although this specific 
instance is not yet split enough (there are still debian bits in ::base).


Regards, DavidS


- -- 
The primary freedom of open source is not the freedom from cost, but the free-
dom to shape software to do what you want. This freedom is /never/ exercised
without cost, but is available /at all/ only by accepting the very different
costs associated with open source, costs not in money, but in time and effort.
- -- http://www.schierer.org/~luke/log/20070710-1129/on-forks-and-forking
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIMnVY/Pp1N6Uzh0URAlo4AJ96ECCHWS04e0NyLt27KbeebYW+3gCbByIG
zT1emSBOWxLPK45g5HV/GOU=wn+C
-----END PGP SIGNATURE-----

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---