Puppet users - Oct 2013 - A note on upcoming module metadata improvements

We''ve been working hard to reduce the complexity involved in publishing
modules to the Forge and make it simpler to find great modules. I''m
writing
today to give you some background on the problems we''re working to
solve
and the approach we''d like to take to help solve them.

To publish a Forge module today, you must provide some metadata in a form
on forge.puppetlabs.com and create a file named Modulefile in your module
root which contains additional (and in some cases duplicate) metadata. Then
you must run `puppet module build` or follow a workflow in Geppetto which
both transform the Modulefile into the metadata.json artifact that Puppet
and the Forge consume.

On top of that, the types of metadata you can enter into the Modulefile is
static and requires an update to Puppet to improve. You''ve been waiting
for
a way to find modules that work with your version of Puppet and on the
operating systems you run in your datacenter. The Forge is ready to display
this information, allow you to filter search results on it and more. The
last step is to allow for additional metadata to be supplied by module
authors in a single, simple, source of metadata.


So, what are we planning to simplify module publishing and enable better
module discovery on Forge?

- The web form on Forge currently required to publish a new module is going
away.
- The Modulefile metadata file will be deprecated in the next major release
of Puppet
- metadata.json (part of the `puppet module build` output) will become the
single-source of metadata
- http://docs.puppetlabs.com/puppet/latest/reference/modules_publishing.htmlwill
be updated to reflect these changes and to document brand new metadata
keys including:
 - puppet_version : The major.minor versions of Puppet your module supports
like 2.7 or 3.3
 - operating_system : OSes your module supports stated like the
$operatingsystem Facter values Ubuntu or RedHat
- This new metadata will be used on forge.puppetlabs.com on module pages,
search results and will be available over its (soon to be released) REST
API.

There are a couple of notable caveats to the above which will be included
in the documentation along with much greater detail on each metadata key.
For example, we''d like to allow for more flexible expressions in
puppet_version (not just major.minor) and we''d like to allow for
version
comparison in operating_system. We plan to introduce these keys as
described above until Forge is ready to accept more complicated
expressions. Once able, we''ll update the documentation to document the
new
capabilities and accept both forms of metadata.



Will this break all my modules and when can I start using this stuff?

We know that these changes aren''t trivial and even though Puppet and
the
Forge use metadata.json already, some tools interact with Modulefile
directly. We''re doing several things to make this transition seamless
and
pleasant.

First, rest easy knowing that the modules you build with the module tool
available in 2.7.14 and later will continue to work. Though you won''t
be
able to express the new metadata or publish to the Forge without visiting
the website, older versions of Puppet and Geppetto will continue to create
and interact with modules just the same.

Users of Puppet 3.3.0 and later will enjoy extra flexibility during this
transition. It includes a change to respect the new metadata fields
expressed in metadata.json when building your release tarball based on the
metadata entered in Modulefile. Though you won''t be living the single
source of metadata dream, you can still express new metadata with few
manual steps.

The next major version of Puppet will include a greatly revamped Puppet
Module Tool that (amongst other improvements) will make it easy to express
and validate the new metadata directly in metadata.json. A future release
of Geppetto will be the easiest choice when it provides a simple form
editor to express, edit and validate the new metadata.

The updated documentation which should be available within the next month.
It will also describe these changes in greater detail and provide all the
information you need to get started.

Best of all, once module authors begin publishing new releases which
contain the new metadata keys, you''ll be able to filter your Forge
search
results on the Puppet versions and operating systems that you care about.
Three sources of module metadata will be reduced to one authoritative
source that doesn''t require an upgrade of your entire Puppet
infrastructure
to improve. Publishing modules to the Forge will no longer require web-site
interaction enabling vastly improved workflows like GitHub-based publishing
that we hope to work on in the coming months.

On behalf of all of us at Puppet Labs, I want to thank you for being such
an awesome community and I''m looking forward to the next year of module
development. You''re always welcome to email me directly or find me as
ryanycoleman in #puppet on Freenode. There are many more things coming from
us in the next few months and I look forward to sharing those with you soon.


--Ryan

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On 10/16/2013 12:07 PM, Ryan Coleman wrote:
>   - operating_system : OSes your module supports stated like the
> $operatingsystem Facter values Ubuntu or RedHat
> - This new metadata will be used on forge.puppetlabs.com
> <http://forge.puppetlabs.com> on module pages, search results and
will
> be available over its (soon to be released) REST API.
>
Ok, I''ll bite :)

How does this plan to deal with all of the RedHat flavors?  If the 
module author doesn''t list out RedHat, CentOS, OEL, Scientific, etc, 
will it work?

Might osfamily be better here? (With the caveat that Fedora should 
probably be it''s own osfamily and not redhat).

Jason

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Ryan,

Thanks for the update. The improvements that are coming sound awesome! (I 
was in fact about to begin a thread to discuss this issue - glad to see 
you''re way ahead of me).

Some insight into how I currently experience the Puppet forge (you may be 
aware of it all, yet anyway):
- From an end-users perspective, the web form has been my biggest struggle 
for me. I have one script to publish new versions of my modules (run tests, 
create tag, push to github). Anything related to getting a new version on 
the forge should be doable in Bash. A popular tool in the PHP community 
named Composer has done this pretty neatly. You may want to take a look at 
this tool, before reinventing the wheel (it utilizes the Github api and 
simply scans for new tags iirc).
- In its current state, the module tool complains if modules don''t
match,
even though an alternative may provide the same interface. The concat 
module comes to mind, there are several versions of the concat module 
around that all share the same interface. It would be nice if there''s
some
''provides'' or ''replaces'' syntax that allows
to define if a module can also
provide functionality for that of another module.
- It''s hard to find the ''best'' module on the Forge.
Rather than only seeing
the number of downloads, I''d also like to see some qualitative score,
like
a 1-5 star rating. 
- There exists a possibility to specify an alternative repo. I would 
however like to have the option to also configure multiple repos. In 
commercial organizations it''s not rare to have a private set of
modules,
and a public one as well.
- Judging from the number of modules that use Github as project page, it 
seems Git is a rather popular tool. I personally prefer to use submodules 
instead of embedding all the extra code in my own repository. Would be nice 
if the module tool could simply manage my submodules instead.
- There''s currently no support for optional dependencies. Many of the 
Example42 modules ( https://github.com/example42?tab=repositories ) can 
integrate with example42/firewall and example42/monitoring, but you''re
in
no way obliged to use this integration. It would be nice if these could be 
listed as optional dependencies.

Because of the reasons above I don''t use the module tool myself.
I''m sorry
if I have misunderstood any functionality or suggested that some isn''t 
available while it actually is - I''m just not aware of it (nor can I
find
it).

Lastly, I can''t stress enough to take a look at Composer (tool: 
http://getcomposer.org/ & forge: https://packagist.org/ ). Although
it''s
based around PHP, I''m fairly certain they face(d) the exact same
challenges
as there are with the Puppet Module tool. It would be a shame if you''d 
reinvent the wheel here.

Regards,

Dolf
-- Freeaqingme

On Wednesday, October 16, 2013 6:07:57 PM UTC+2, Ryan Coleman
wrote:
>
> We''ve been working hard to reduce the complexity involved in
publishing
> modules to the Forge and make it simpler to find great modules.
I''m writing
> today to give you some background on the problems we''re working to
solve
> and the approach we''d like to take to help solve them.
>
> To publish a Forge module today, you must provide some metadata in a form 
> on forge.puppetlabs.com and create a file named Modulefile in your module 
> root which contains additional (and in some cases duplicate) metadata. Then
> you must run `puppet module build` or follow a workflow in Geppetto which 
> both transform the Modulefile into the metadata.json artifact that Puppet 
> and the Forge consume.
>
> On top of that, the types of metadata you can enter into the Modulefile is 
> static and requires an update to Puppet to improve. You''ve been
waiting for
> a way to find modules that work with your version of Puppet and on the 
> operating systems you run in your datacenter. The Forge is ready to display
> this information, allow you to filter search results on it and more. The 
> last step is to allow for additional metadata to be supplied by module 
> authors in a single, simple, source of metadata.
>
>
> So, what are we planning to simplify module publishing and enable better 
> module discovery on Forge?
>
> - The web form on Forge currently required to publish a new module is 
> going away.
> - The Modulefile metadata file will be deprecated in the next major 
> release of Puppet
> - metadata.json (part of the `puppet module build` output) will become the 
> single-source of metadata
> - 
>
http://docs.puppetlabs.com/puppet/latest/reference/modules_publishing.htmlwill
be updated to reflect these changes and to document brand new metadata
> keys including:
>  - puppet_version : The major.minor versions of Puppet your module 
> supports like 2.7 or 3.3
>  - operating_system : OSes your module supports stated like the 
> $operatingsystem Facter values Ubuntu or RedHat
> - This new metadata will be used on forge.puppetlabs.com on module pages, 
> search results and will be available over its (soon to be released) REST 
> API.
>
> There are a couple of notable caveats to the above which will be included 
> in the documentation along with much greater detail on each metadata key. 
> For example, we''d like to allow for more flexible expressions in 
> puppet_version (not just major.minor) and we''d like to allow for
version
> comparison in operating_system. We plan to introduce these keys as 
> described above until Forge is ready to accept more complicated 
> expressions. Once able, we''ll update the documentation to document
the new
> capabilities and accept both forms of metadata.
>
>
>
> Will this break all my modules and when can I start using this stuff?
>
> We know that these changes aren''t trivial and even though Puppet
and the
> Forge use metadata.json already, some tools interact with Modulefile 
> directly. We''re doing several things to make this transition
seamless and
> pleasant.
>
> First, rest easy knowing that the modules you build with the module tool 
> available in 2.7.14 and later will continue to work. Though you
won''t be
> able to express the new metadata or publish to the Forge without visiting 
> the website, older versions of Puppet and Geppetto will continue to create 
> and interact with modules just the same.
>
> Users of Puppet 3.3.0 and later will enjoy extra flexibility during this 
> transition. It includes a change to respect the new metadata fields 
> expressed in metadata.json when building your release tarball based on the 
> metadata entered in Modulefile. Though you won''t be living the
single
> source of metadata dream, you can still express new metadata with few 
> manual steps.
>
> The next major version of Puppet will include a greatly revamped Puppet 
> Module Tool that (amongst other improvements) will make it easy to express 
> and validate the new metadata directly in metadata.json. A future release 
> of Geppetto will be the easiest choice when it provides a simple form 
> editor to express, edit and validate the new metadata.
>
> The updated documentation which should be available within the next month. 
> It will also describe these changes in greater detail and provide all the 
> information you need to get started.
>
> Best of all, once module authors begin publishing new releases which 
> contain the new metadata keys, you''ll be able to filter your Forge
search
> results on the Puppet versions and operating systems that you care about. 
> Three sources of module metadata will be reduced to one authoritative 
> source that doesn''t require an upgrade of your entire Puppet
infrastructure
> to improve. Publishing modules to the Forge will no longer require web-site
> interaction enabling vastly improved workflows like GitHub-based publishing
> that we hope to work on in the coming months.
>
> On behalf of all of us at Puppet Labs, I want to thank you for being such 
> an awesome community and I''m looking forward to the next year of
module
> development. You''re always welcome to email me directly or find me
as
> ryanycoleman in #puppet on Freenode. There are many more things coming from
> us in the next few months and I look forward to sharing those with you
soon.
>
>
> --Ryan 
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Reposting my last post as I got an error that it could not be delivered 
because of my SPF records. Now with a different email address
----------

Hi Ryan,

Thanks for the update. The improvements that are coming sound awesome! (I 
was in fact about to begin a thread to discuss this issue - glad to see 
you''re way ahead of me).

Some insight into how I currently experience the Puppet forge (you may be 
aware of it all, yet anyway):
- From an end-users perspective, the web form has been my biggest struggle 
for me. I have one script to publish new versions of my modules (run tests, 
create tag, push to github). Anything related to getting a new version on 
the forge should be doable in Bash. A popular tool in the PHP community 
named Composer has done this pretty neatly. You may want to take a look at 
this tool, before reinventing the wheel (it utilizes the Github api and 
simply scans for new tags iirc).
- In its current state, the module tool complains if modules don''t
match,
even though an alternative may provide the same interface. The concat 
module comes to mind, there are several versions of the concat module 
around that all share the same interface. It would be nice if there''s
some
''provides'' or ''replaces'' syntax that allows
to define if a module can also
provide functionality for that of another module.
- It''s hard to find the ''best'' module on the Forge.
Rather than only seeing
the number of downloads, I''d also like to see some qualitative score,
like
a 1-5 star rating. 
- There exists a possibility to specify an alternative repo. I would 
however like to have the option to also configure multiple repos. In 
commercial organizations it''s not rare to have a private set of
modules,
and a public one as well.
- Judging from the number of modules that use Github as project page, it 
seems Git is a rather popular tool. I personally prefer to use submodules 
instead of embedding all the extra code in my own repository. Would be nice 
if the module tool could simply manage my submodules instead.
- There''s currently no support for optional dependencies. Many of the 
Example42 modules ( https://github.com/example42?tab=repositories ) can 
integrate with example42/firewall and example42/monitoring, but you''re
in
no way obliged to use this integration. It would be nice if these could be 
listed as optional dependencies.

Because of the reasons above I don''t use the module tool myself.
I''m sorry
if I have misunderstood any functionality or suggested that some isn''t 
available while it actually is - I''m just not aware of it (nor can I
find
it).

Lastly, I can''t stress enough to take a look at Composer (tool: 
http://getcomposer.org/ & forge: https://packagist.org/ ). Although
it''s
based around PHP, I''m fairly certain they face(d) the exact same
challenges
as there are with the Puppet Module tool. It would be a shame if you''d 
reinvent the wheel here.

Regards,

Dolf
-- Freeaqingme

On Wednesday, October 16, 2013 6:07:57 PM UTC+2, Ryan Coleman
wrote:
>
> We''ve been working hard to reduce the complexity involved in
publishing
> modules to the Forge and make it simpler to find great modules.
I''m writing
> today to give you some background on the problems we''re working to
solve
> and the approach we''d like to take to help solve them.
>
> To publish a Forge module today, you must provide some metadata in a form 
> on forge.puppetlabs.com and create a file named Modulefile in your module 
> root which contains additional (and in some cases duplicate) metadata. Then
> you must run `puppet module build` or follow a workflow in Geppetto which 
> both transform the Modulefile into the metadata.json artifact that Puppet 
> and the Forge consume.
>
> On top of that, the types of metadata you can enter into the Modulefile is 
> static and requires an update to Puppet to improve. You''ve been
waiting for
> a way to find modules that work with your version of Puppet and on the 
> operating systems you run in your datacenter. The Forge is ready to display
> this information, allow you to filter search results on it and more. The 
> last step is to allow for additional metadata to be supplied by module 
> authors in a single, simple, source of metadata.
>
>
> So, what are we planning to simplify module publishing and enable better 
> module discovery on Forge?
>
> - The web form on Forge currently required to publish a new module is 
> going away.
> - The Modulefile metadata file will be deprecated in the next major 
> release of Puppet
> - metadata.json (part of the `puppet module build` output) will become the 
> single-source of metadata
> - 
>
http://docs.puppetlabs.com/puppet/latest/reference/modules_publishing.htmlwill
be updated to reflect these changes and to document brand new metadata
> keys including:
>  - puppet_version : The major.minor versions of Puppet your module 
> supports like 2.7 or 3.3
>  - operating_system : OSes your module supports stated like the 
> $operatingsystem Facter values Ubuntu or RedHat
> - This new metadata will be used on forge.puppetlabs.com on module pages, 
> search results and will be available over its (soon to be released) REST 
> API.
>
> There are a couple of notable caveats to the above which will be included 
> in the documentation along with much greater detail on each metadata key. 
> For example, we''d like to allow for more flexible expressions in 
> puppet_version (not just major.minor) and we''d like to allow for
version
> comparison in operating_system. We plan to introduce these keys as 
> described above until Forge is ready to accept more complicated 
> expressions. Once able, we''ll update the documentation to document
the new
> capabilities and accept both forms of metadata.
>
>
>
> Will this break all my modules and when can I start using this stuff?
>
> We know that these changes aren''t trivial and even though Puppet
and the
> Forge use metadata.json already, some tools interact with Modulefile 
> directly. We''re doing several things to make this transition
seamless and
> pleasant.
>
> First, rest easy knowing that the modules you build with the module tool 
> available in 2.7.14 and later will continue to work. Though you
won''t be
> able to express the new metadata or publish to the Forge without visiting 
> the website, older versions of Puppet and Geppetto will continue to create 
> and interact with modules just the same.
>
> Users of Puppet 3.3.0 and later will enjoy extra flexibility during this 
> transition. It includes a change to respect the new metadata fields 
> expressed in metadata.json when building your release tarball based on the 
> metadata entered in Modulefile. Though you won''t be living the
single
> source of metadata dream, you can still express new metadata with few 
> manual steps.
>
> The next major version of Puppet will include a greatly revamped Puppet 
> Module Tool that (amongst other improvements) will make it easy to express 
> and validate the new metadata directly in metadata.json. A future release 
> of Geppetto will be the easiest choice when it provides a simple form 
> editor to express, edit and validate the new metadata.
>
> The updated documentation which should be available within the next month. 
> It will also describe these changes in greater detail and provide all the 
> information you need to get started.
>
> Best of all, once module authors begin publishing new releases which 
> contain the new metadata keys, you''ll be able to filter your Forge
search
> results on the Puppet versions and operating systems that you care about. 
> Three sources of module metadata will be reduced to one authoritative 
> source that doesn''t require an upgrade of your entire Puppet
infrastructure
> to improve. Publishing modules to the Forge will no longer require web-site
> interaction enabling vastly improved workflows like GitHub-based publishing
> that we hope to work on in the coming months.
>
> On behalf of all of us at Puppet Labs, I want to thank you for being such 
> an awesome community and I''m looking forward to the next year of
module
> development. You''re always welcome to email me directly or find me
as
> ryanycoleman in #puppet on Freenode. There are many more things coming from
> us in the next few months and I look forward to sharing those with you
soon.
>
>
> --Ryan 
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Il giorno mercoledì 16 ottobre 2013 18:07:57 UTC+2, Ryan Coleman ha
scritto:
>
> We''ve been working hard to reduce the complexity involved in
publishing
> modules to the Forge and make it simpler to find great modules.
I''m writing
> today to give you some background on the problems we''re working to
solve
> and the approach we''d like to take to help solve them.
>
> To publish a Forge module today, you must provide some metadata in a form 
> on forge.puppetlabs.com and create a file named Modulefile in your module 
> root which contains additional (and in some cases duplicate) metadata. Then
> you must run `puppet module build` or follow a workflow in Geppetto which 
> both transform the Modulefile into the metadata.json artifact that Puppet 
> and the Forge consume.
>
> On top of that, the types of metadata you can enter into the Modulefile is 
> static and requires an update to Puppet to improve. You''ve been
waiting for
> a way to find modules that work with your version of Puppet and on the 
> operating systems you run in your datacenter. The Forge is ready to display
> this information, allow you to filter search results on it and more. The 
> last step is to allow for additional metadata to be supplied by module 
> authors in a single, simple, source of metadata.
>
>
> So, what are we planning to simplify module publishing and enable better 
> module discovery on Forge?
>
> - The web form on Forge currently required to publish a new module is 
> going away.
> - The Modulefile metadata file will be deprecated in the next major 
> release of Puppet
> - metadata.json (part of the `puppet module build` output) will become the 
> single-source of metadata
> - 
>
http://docs.puppetlabs.com/puppet/latest/reference/modules_publishing.htmlwill
be updated to reflect these changes and to document brand new metadata
> keys including:
>  - puppet_version : The major.minor versions of Puppet your module 
> supports like 2.7 or 3.3
>  - operating_system : OSes your module supports stated like the 
> $operatingsystem Facter values Ubuntu or RedHat
> - This new metadata will be used on forge.puppetlabs.com on module pages, 
> search results and will be available over its (soon to be released) REST 
> API.
>
All this sound great,
as Dolf has written in his reply,  and others like Gareth have suggested at 
the PuppetConf, a hugely useful key would be something like 
''Optional-dependency'' with relevant option(s) on the module
tool on how to
manage it.
Maybe it could also be nice to add a key where are expressed the external 
resources the module depends upon (either mandatory or optional): most of 
the times you depend on an apache module, for example, just because you use 
its apache::vhost and that single dependency more or less easily fixable, 
may prevent different modules from being used together.
Both these keys would greatly help interoperability among modules, IMHO.

Can open a feature request, if there''s not already one.
Al

 
>
> There are a couple of notable caveats to the above which will be included 
> in the documentation along with much greater detail on each metadata key. 
> For example, we''d like to allow for more flexible expressions in 
> puppet_version (not just major.minor) and we''d like to allow for
version
> comparison in operating_system. We plan to introduce these keys as 
> described above until Forge is ready to accept more complicated 
> expressions. Once able, we''ll update the documentation to document
the new
> capabilities and accept both forms of metadata.
>
>
>
> Will this break all my modules and when can I start using this stuff?
>
> We know that these changes aren''t trivial and even though Puppet
and the
> Forge use metadata.json already, some tools interact with Modulefile 
> directly. We''re doing several things to make this transition
seamless and
> pleasant.
>
> First, rest easy knowing that the modules you build with the module tool 
> available in 2.7.14 and later will continue to work. Though you
won''t be
> able to express the new metadata or publish to the Forge without visiting 
> the website, older versions of Puppet and Geppetto will continue to create 
> and interact with modules just the same.
>
> Users of Puppet 3.3.0 and later will enjoy extra flexibility during this 
> transition. It includes a change to respect the new metadata fields 
> expressed in metadata.json when building your release tarball based on the 
> metadata entered in Modulefile. Though you won''t be living the
single
> source of metadata dream, you can still express new metadata with few 
> manual steps.
>
> The next major version of Puppet will include a greatly revamped Puppet 
> Module Tool that (amongst other improvements) will make it easy to express 
> and validate the new metadata directly in metadata.json. A future release 
> of Geppetto will be the easiest choice when it provides a simple form 
> editor to express, edit and validate the new metadata.
>
> The updated documentation which should be available within the next month. 
> It will also describe these changes in greater detail and provide all the 
> information you need to get started.
>
> Best of all, once module authors begin publishing new releases which 
> contain the new metadata keys, you''ll be able to filter your Forge
search
> results on the Puppet versions and operating systems that you care about. 
> Three sources of module metadata will be reduced to one authoritative 
> source that doesn''t require an upgrade of your entire Puppet
infrastructure
> to improve. Publishing modules to the Forge will no longer require web-site
> interaction enabling vastly improved workflows like GitHub-based publishing
> that we hope to work on in the coming months.
>
> On behalf of all of us at Puppet Labs, I want to thank you for being such 
> an awesome community and I''m looking forward to the next year of
module
> development. You''re always welcome to email me directly or find me
as
> ryanycoleman in #puppet on Freenode. There are many more things coming from
> us in the next few months and I look forward to sharing those with you
soon.
>
>
> --Ryan 
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Forgive me for top-posting cherry-picked responses. I wanted to reply to
each of you without hitting the list with 3 emails. Thanks again for the
great feedback and support. I can''t wait to get it all shipped.

On Wed, Oct 16, 2013 at 9:36 AM, Jason Slagle <raistlin@tacorp.net> wrote:
>
>
> On 10/16/2013 12:07 PM, Ryan Coleman wrote:
>
>   - operating_system : OSes your module supports stated like the
>> $operatingsystem Facter values Ubuntu or RedHat
>>
>
> How does this plan to deal with all of the RedHat flavors?  If the module
> author doesn''t list out RedHat, CentOS, OEL, Scientific, etc, will
it work?
>
> Might osfamily be better here? (With the caveat that Fedora should
> probably be it''s own osfamily and not redhat).

Hi Jason, I''ve truly gone back and forth on this one. Being as granular
as
$operatingsystem can be incredibly verbose as you''ve indicated. On the
other hand, if the metadata is as broad as $osfamily and you simply say
RedHat, you could hit situations where users of Amazon Linux expect a
functional module and run into breakage.

So the idea here is to provide the more flexible option to module authors
and set the expectation that you need to be verbose about each operating
system you support. The documentation will provide a list of common values
and point to examples. I''d like to avoid using two keys to express the
same
thing and wanted to err on treating this metadata as a tested promise
instead of a general catch-all that is probably tested on just a few
platforms in the osfamily. Do you think it''s worth the additional
initial
effort? Thanks for your feedback.


On Wed, Oct 16, 2013 at 9:44 AM, Dolf Schimmel - Freeaqingme <
dolf@dolfschimmel.nl> wrote:
> Hi Ryan,
>
> Thanks for the update. The improvements that are coming sound awesome! (I
> was in fact about to begin a thread to discuss this issue - glad to see
> you''re way ahead of me).
>
> Some insight into how I currently experience the Puppet forge (you may be
> aware of it all, yet anyway):
>
>
<snip>

Thanks for all the feedback Dolf. I''m with you on each of those points.
We''re a small team and though we''d like to address each of
them tomorrow,
we''ll get to each as we can.

> - There''s currently no support for optional dependencies. Many of
the
> Example42 modules ( https://github.com/example42?tab=repositories ) can
> integrate with example42/firewall and example42/monitoring, but
you''re in
> no way obliged to use this integration. It would be nice if these could be
> listed as optional dependencies.
>
I''ll reply to both you and Al here. Thank you both.

Evolving the metadata format in the way we''re planning opens the door
to
exactly these kinds of features. Though we''re kicking this off with
expressions of Puppet and Operating System compatibility, I think the next
logical step is to allow for optional dependencies and obsoleting a module.

Both will require new module metadata. Unfortunately they''ll also
require
an update to the Puppet Module Tool to take advantage of the new metadata.
We''re about to spin up plans for a new major version of the module tool
which will allow for things like this and we''re also planning to make
updating the module tool more flexible so that we''re no longer in a
situation where you must upgrade Puppet on your masters in order to get new
module tool features. I''ll send out a similar note soon to get a larger
discussion going on what you''d like to see there.

>
> Because of the reasons above I don''t use the module tool myself.
I''m sorry
> if I have misunderstood any functionality or suggested that some
isn''t
> available while it actually is - I''m just not aware of it (nor can
I find
> it).
>
> Lastly, I can''t stress enough to take a look at Composer (tool:
> http://getcomposer.org/ & forge: https://packagist.org/ ). Although
it''s
> based around PHP, I''m fairly certain they face(d) the exact same
challenges
> as there are with the Puppet Module tool. It would be a shame if
you''d
> reinvent the wheel here.
>
>
I hadn''t seen these tools before. Thank you, I''ll check them
out.



On Wed, Oct 16, 2013 at 9:07 AM, Ryan Coleman <ryan@puppetlabs.com> wrote:
> We''ve been working hard to reduce the complexity involved in
publishing
> modules to the Forge and make it simpler to find great modules.
I''m writing
> today to give you some background on the problems we''re working to
solve
> and the approach we''d like to take to help solve them.
>
> To publish a Forge module today, you must provide some metadata in a form
> on forge.puppetlabs.com and create a file named Modulefile in your module
> root which contains additional (and in some cases duplicate) metadata. Then
> you must run `puppet module build` or follow a workflow in Geppetto which
> both transform the Modulefile into the metadata.json artifact that Puppet
> and the Forge consume.
>
> On top of that, the types of metadata you can enter into the Modulefile is
> static and requires an update to Puppet to improve. You''ve been
waiting for
> a way to find modules that work with your version of Puppet and on the
> operating systems you run in your datacenter. The Forge is ready to display
> this information, allow you to filter search results on it and more. The
> last step is to allow for additional metadata to be supplied by module
> authors in a single, simple, source of metadata.
>
>
> So, what are we planning to simplify module publishing and enable better
> module discovery on Forge?
>
> - The web form on Forge currently required to publish a new module is
> going away.
> - The Modulefile metadata file will be deprecated in the next major
> release of Puppet
> - metadata.json (part of the `puppet module build` output) will become the
> single-source of metadata
> -
>
http://docs.puppetlabs.com/puppet/latest/reference/modules_publishing.htmlwill
be updated to reflect these changes and to document brand new metadata
> keys including:
>  - puppet_version : The major.minor versions of Puppet your module
> supports like 2.7 or 3.3
>  - operating_system : OSes your module supports stated like the
> $operatingsystem Facter values Ubuntu or RedHat
> - This new metadata will be used on forge.puppetlabs.com on module pages,
> search results and will be available over its (soon to be released) REST
> API.
>
> There are a couple of notable caveats to the above which will be included
> in the documentation along with much greater detail on each metadata key.
> For example, we''d like to allow for more flexible expressions in
> puppet_version (not just major.minor) and we''d like to allow for
version
> comparison in operating_system. We plan to introduce these keys as
> described above until Forge is ready to accept more complicated
> expressions. Once able, we''ll update the documentation to document
the new
> capabilities and accept both forms of metadata.
>
>
>
> Will this break all my modules and when can I start using this stuff?
>
> We know that these changes aren''t trivial and even though Puppet
and the
> Forge use metadata.json already, some tools interact with Modulefile
> directly. We''re doing several things to make this transition
seamless and
> pleasant.
>
> First, rest easy knowing that the modules you build with the module tool
> available in 2.7.14 and later will continue to work. Though you
won''t be
> able to express the new metadata or publish to the Forge without visiting
> the website, older versions of Puppet and Geppetto will continue to create
> and interact with modules just the same.
>
> Users of Puppet 3.3.0 and later will enjoy extra flexibility during this
> transition. It includes a change to respect the new metadata fields
> expressed in metadata.json when building your release tarball based on the
> metadata entered in Modulefile. Though you won''t be living the
single
> source of metadata dream, you can still express new metadata with few
> manual steps.
>
> The next major version of Puppet will include a greatly revamped Puppet
> Module Tool that (amongst other improvements) will make it easy to express
> and validate the new metadata directly in metadata.json. A future release
> of Geppetto will be the easiest choice when it provides a simple form
> editor to express, edit and validate the new metadata.
>
> The updated documentation which should be available within the next month.
> It will also describe these changes in greater detail and provide all the
> information you need to get started.
>
> Best of all, once module authors begin publishing new releases which
> contain the new metadata keys, you''ll be able to filter your Forge
search
> results on the Puppet versions and operating systems that you care about.
> Three sources of module metadata will be reduced to one authoritative
> source that doesn''t require an upgrade of your entire Puppet
infrastructure
> to improve. Publishing modules to the Forge will no longer require web-site
> interaction enabling vastly improved workflows like GitHub-based publishing
> that we hope to work on in the coming months.
>
> On behalf of all of us at Puppet Labs, I want to thank you for being such
> an awesome community and I''m looking forward to the next year of
module
> development. You''re always welcome to email me directly or find me
as
> ryanycoleman in #puppet on Freenode. There are many more things coming from
> us in the next few months and I look forward to sharing those with you
soon.
>
>
> --Ryan
>


-- 
Ryan Coleman | Modules & Forge | ryanycoleman on twitter & #puppet IRC

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.