Puppet users - Jun 2011 - RFC: Adding implicit stages to Puppet

https://projects.puppetlabs.com/issues/7697

One problem people producing modules that make use of stages are hitting is
that it''s difficult to create something reusable that integrates
seamlessly
into existing setups.

This feature request is to add several more implicit stages to Puppet so we
have:

bootstrap
pre
main
post

existing by default, making it easier for authors to specify stages in their
modules.

Thoughts?

-- 
Nigel Kersten
Product, Puppet Labs
@nigelkersten

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, 09 Jun 2011 18:42:54 -0700, Nigel Kersten wrote:
> 
> https://projects.puppetlabs.com/issues/7697
> 
> One problem people producing modules that make use of stages are hitting is
> that it''s difficult to create something reusable that integrates
seamlessly
> into existing setups.
> 
> This feature request is to add several more implicit stages to Puppet so we
> have:
> 
> bootstrap
> pre
> main
> post
> 
> existing by default, making it easier for authors to specify stages in
their
> modules.
> 
> Thoughts?
> 
The answer to question "Which comes first, ''bootstrap'' or
''pre''?" seems
awfully ambiguous from just the names.

What''s the reason for separating it out?

-- 
Jacob Helwig

A while back I wrote down all the puppet patterns
I could think of, and this was one of them.
I named it Cradle To Grave, but probably that''s not
appropriate.  However, I was only focusing on
puppet runs at the time, so that name popped into my head.
It is an instance of the more general pattern Convention
Over Configuration.

Of course, sites will define their own stage sequences.
But there is still benefit to having a canonical sequence
to refer to when publishing modules.

I think the trick to making it work generally
is to allow renaming of the stages.
That is, the stages are referred to through
variables, so the stage names can be
changed by, for example, sub-classing
in order to make them fit into a site design.

I''ve got one module that uses stages.
I think I will change it to make the stage names
an abstraction and see how that works.

-vagn

CRADLE TO GRAVE

problem:

         There are many steps that must be done in sequence.
         Expressing that through dependencies is tedious,
         and violates the principle of encapsulation (or something).

solution:

         Convention over configuration:
         have some canonical stages.

         also have module specific stages
         that relate to the canonical stages.

canonical stages (but, don''t take this particular list seriously):

         local-pre
         local
         local-post
         network-pre
         network
         network-post
         .. as above, there is a -pre and -post version of
         provisioning
         package
         package-update
         lib
         comms
         mounts
         scripts
         services
         db
         web
         bikeshed
         main
         app
         ha
         env
         restart

module stages:

         foo-local-pre
         foo-network
         foo-mounts-post
         foo-whatever

pros:

         don''t have to constantly mess with dependencies,
         just slot the module in where it belongs in the
         grand scheme.

cons:

         deciding where to put the bikeshed



On 06/09/2011 09:50 PM, Jacob Helwig wrote:
> On Thu, 09 Jun 2011 18:42:54 -0700, Nigel Kersten wrote:
>    
>> https://projects.puppetlabs.com/issues/7697
>>
>> One problem people producing modules that make use of stages are
hitting is
>> that it''s difficult to create something reusable that
integrates seamlessly
>> into existing setups.
>>
>> This feature request is to add several more implicit stages to Puppet
so we
>> have:
>>
>> bootstrap
>> pre
>> main
>> post
>>
>> existing by default, making it easier for authors to specify stages in
their
>> modules.
>>
>> Thoughts?
>>
>>      
> The answer to question "Which comes first,
''bootstrap'' or ''pre''?" seems
> awfully ambiguous from just the names.
>
> What''s the reason for separating it out?
>
>    
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I prefer having a small number of predefined stages in puppet.
This makes it easier to share modules which use stages.
My suggestion:
- a small number of stages is easier to remeber
- a samll numer of possibilities makes people think in advance in which
stage they need to put their module

Additionally I would still like to have the possibility to add
individual stages.


My proposal for names:

(the proposed names from Nigel#s mail are fine for me)
- bootstrap
- pre ( another proposal: init?)
- main
- post

(and a name for the very last stage I am using on some setups)
- cleanup




On 06/10/2011 06:47 AM, vagn scott wrote:
> 
> A while back I wrote down all the puppet patterns
> I could think of, and this was one of them.
> I named it Cradle To Grave, but probably that''s not
> appropriate.  However, I was only focusing on
> puppet runs at the time, so that name popped into my head.
> It is an instance of the more general pattern Convention
> Over Configuration.
> 
> Of course, sites will define their own stage sequences.
> But there is still benefit to having a canonical sequence
> to refer to when publishing modules.
> 
> I think the trick to making it work generally
> is to allow renaming of the stages.
> That is, the stages are referred to through
> variables, so the stage names can be
> changed by, for example, sub-classing
> in order to make them fit into a site design.
> 
> I''ve got one module that uses stages.
> I think I will change it to make the stage names
> an abstraction and see how that works.
> 
> -vagn
> 
> CRADLE TO GRAVE
> 
> problem:
> 
>         There are many steps that must be done in sequence.
>         Expressing that through dependencies is tedious,
>         and violates the principle of encapsulation (or something).
> 
> solution:
> 
>         Convention over configuration:
>         have some canonical stages.
> 
>         also have module specific stages
>         that relate to the canonical stages.
> 
> canonical stages (but, don''t take this particular list seriously):
> 
>         local-pre
>         local
>         local-post
>         network-pre
>         network
>         network-post
>         .. as above, there is a -pre and -post version of
>         provisioning
>         package
>         package-update
>         lib
>         comms
>         mounts
>         scripts
>         services
>         db
>         web
>         bikeshed
>         main
>         app
>         ha
>         env
>         restart
> 
> module stages:
> 
>         foo-local-pre
>         foo-network
>         foo-mounts-post
>         foo-whatever
> 
> pros:
> 
>         don''t have to constantly mess with dependencies,
>         just slot the module in where it belongs in the
>         grand scheme.
> 
> cons:
> 
>         deciding where to put the bikeshed
> 
> 
> 
> On 06/09/2011 09:50 PM, Jacob Helwig wrote:
>> On Thu, 09 Jun 2011 18:42:54 -0700, Nigel Kersten wrote:
>>   
>>> https://projects.puppetlabs.com/issues/7697
>>>
>>> One problem people producing modules that make use of stages are
>>> hitting is
>>> that it''s difficult to create something reusable that
integrates
>>> seamlessly
>>> into existing setups.
>>>
>>> This feature request is to add several more implicit stages to
Puppet
>>> so we
>>> have:
>>>
>>> bootstrap
>>> pre
>>> main
>>> post
>>>
>>> existing by default, making it easier for authors to specify stages
>>> in their
>>> modules.
>>>
>>> Thoughts?
>>>
>>>      
>> The answer to question "Which comes first,
''bootstrap'' or ''pre''?" seems
>> awfully ambiguous from just the names.
>>
>> What''s the reason for separating it out?
>>
>>    
> 
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 10 June 2011 02:50, Jacob Helwig <jacob@puppetlabs.com> wrote:
> On Thu, 09 Jun 2011 18:42:54 -0700, Nigel Kersten wrote:
> >
> > https://projects.puppetlabs.com/issues/7697
> >
> > One problem people producing modules that make use of stages are
hitting
> is
> > that it''s difficult to create something reusable that
integrates
> seamlessly
> > into existing setups.
> >
> > This feature request is to add several more implicit stages to Puppet
so
> we
> > have:
> >
> > bootstrap
> > pre
> > main
> > post
> >
> > existing by default, making it easier for authors to specify stages in
> their
> > modules.
> >
> > Thoughts?
> >
>
Seems like a very good idea to me. What about these stages relating to being
able regenerate facts after a stage possibly? A thread the other day about
nagios configs meant delivering a new fact file and then going back on the
next run to retrieve the facts. I think this required two runs, can''t
swear
to it right now though, but a formal re-evaluation of facts after a
bootstrap could be good. You could be real confusing and have a proper
"POST" section too, right at the start, BIOS style. Not to be confused
with
the "post" stage of course! ;-)

The answer to question "Which comes first, ''bootstrap'' or
''pre''?" seems
> awfully ambiguous from just the names.
>
I would see that a bootstrap stage would be for modules fundamental to
puppet itself, e.g. if puppet.conf is updated. And pre would be normal run
stuff that just needs to happen early on in the run itself.

What''s the reason for separating it out?


For the reasons he already described...

Thanks

Chris

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Chris Phillips <chris@untrepid.com> wrote:



On 10 June 2011 02:50, Jacob Helwig <jacob@puppetlabs.com> wrote:

On Thu, 09 Jun 2011 18:42:54 -0700, Nigel Kersten wrote:
>
> https://projects.puppetlabs.com/issues/7697
>
> One problem people producing modules that make use of stages are hitting is
> that it''s difficult to create something reusable that integrates
seamlessly
> into existing setups.
>
> This feature request is to add several more implicit stages to Puppet so we
> have:
>
> bootstrap
> pre
> main
> post
>
> existing by default, making it easier for authors to specify stages in
their
> modules.
>
> Thoughts?
>

Seems like a very good idea to me. What about these stages relating to being
able regenerate facts after a stage possibly? A thread the other day about
nagios configs meant delivering a new fact file and then going back on the next
run to retrieve the facts. I think this required two runs, can''t swear
to it right now though, but a formal re-evaluation of facts after a bootstrap
could be good. You could be real confusing and have a proper "POST"
section too, right at the start, BIOS style. Not to be confused with the
"post" stage of course! ;-)


The answer to question "Which comes first, ''bootstrap'' or
''pre''?" seems
awfully ambiguous from just the names.


I would see that a bootstrap stage would be for modules fundamental to puppet
itself, e.g. if puppet.conf is updated. And pre would be normal run stuff that
just needs to happen early on in the run itself.


What''s the reason for separating it out?


For the reasons he already described...  


Thanks


Chris


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.


The question was specifically about splitting out "pre" and
"bootstrap". All the rest made sense, especially with all the talk
that''s been going on about prerun and postrun commands.
-- 
Sent from my phone. Please excuse my brevity.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 10 Jun 2011 07:52, "Jacob Helwig" <jacob@puppetlabs.com>
wrote:
>
> Chris Phillips <chris@untrepid.com> wrote:
>>
>>
>>
>> On 10 June 2011 02:50, Jacob Helwig <jacob@puppetlabs.com> wrote:
>>>
>>> On Thu, 09 Jun 2011 18:42:54 -0700, Nigel Kersten wrote:
>>> >
>>> > https://projects.puppetlabs.com/issues/7697
>>> >
>>> > One problem people producing modules that make use of stages
are
hitting is
>>> > that it''s difficult to create something reusable that
integrates
seamlessly
>>> > into existing setups.
>>> >
>>> > This feature request is to add several more implicit stages to
Puppet
so we
>>> > have:
>>> >
>>> > bootstrap
>>> > pre
>>> > main
>>> > post
>>> >
>>> > existing by default, making it easier for authors to specify
stages in
their
>>> > modules.
>>> >
>>> > Thoughts?
>>> >
>>
>>
>> Seems like a very good idea to me. What about these stages relating to
being able regenerate facts after a stage possibly? A thread the other day
about nagios configs meant delivering a new fact file and then going back on
the next run to retrieve the facts. I think this required two runs,
can''t
swear to it right now though, but a formal re-evaluation of facts after a
bootstrap could be good. You could be real confusing and have a proper
"POST" section too, right at the start, BIOS style. Not to be confused
with
the "post" stage of course! ;-)
>>
>>> The answer to question "Which comes first,
''bootstrap'' or ''pre''?" seems
>>> awfully ambiguous from just the names.
>>
>>
>> I would see that a bootstrap stage would be for modules fundamental to
puppet itself, e.g. if puppet.conf is updated. And pre would be normal run
stuff that just needs to happen early on in the run
itself.
>>
>>> What''s the reason for separating it out?
>>
>>
>> For the reasons he already described...
>>
>> Thanks
>>
>> Chris
>>
>> --
>> You received this message because you are subscribed to the Google
Groups
"Puppet Users" group.
>> To post to this group, send email to puppet-users@googlegroups.com.
>> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
>> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
>
>
> The question was specifically about splitting out "pre" and
"bootstrap".
All the rest made sense, especially with all the talk that''s been going
on
about prerun and postrun commands.

Sure, well I have an example where my manifests use the lsbmajdistrelease
fact a lot. If the lsb-release rpm isn''t installed the fact
doesn''t exist.
So a boot stage can be used to deal with ensuring the rpm gets installed,
facts could then be really evaluated and the fact I need then be ready in
the pre stage.
> --
> Sent from my phone. Please excuse my brevity.
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, 2011-06-09 at 18:50 -0700, Jacob Helwig wrote:
> On Thu, 09 Jun 2011 18:42:54 -0700, Nigel Kersten wrote:
> > 
> > https://projects.puppetlabs.com/issues/7697
> > 
> > One problem people producing modules that make use of stages are
hitting is
> > that it''s difficult to create something reusable that
integrates seamlessly
> > into existing setups.
> > 
> > This feature request is to add several more implicit stages to Puppet
so we
> > have:
> > 
> > bootstrap
> > pre
> > main
> > post
> > 
> > existing by default, making it easier for authors to specify stages in
their
> > modules.
> > 
> > Thoughts?
> > 
> 
> The answer to question "Which comes first,
''bootstrap'' or ''pre''?" seems
> awfully ambiguous from just the names.
> 
> What''s the reason for separating it out?
One of the reason would be for the bootstrap phase to happen in its own
run instead of being part of the standard run. That would allow to
pre-install stuff that plugins could use (like for instance mysqladmin
for the mysql types). Then the 3 other stages would happen in the same
run.

It would also be great to have this stage being optional in subsequent
runs, allowing you to use the bootstrap stage during provisioning (ie
just after a pre-seed or kickstart), but never again. This would help
bootstrap from bare-metal.

-- 
Brice Figureau
Follow the latest Puppet Community evolutions on www.planetpuppet.org!

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 10 June 2011 09:06, Brice Figureau <brice-puppet@daysofwonder.com>
wrote:
> On Thu, 2011-06-09 at 18:50 -0700, Jacob Helwig wrote:
> > On Thu, 09 Jun 2011 18:42:54 -0700, Nigel Kersten wrote:
> > >
> > > https://projects.puppetlabs.com/issues/7697
> > >
> > > One problem people producing modules that make use of stages are
> hitting is
> > > that it''s difficult to create something reusable that
integrates
> seamlessly
> > > into existing setups.
> > >
> > > This feature request is to add several more implicit stages to
Puppet
> so we
> > > have:
> > >
> > > bootstrap
> > > pre
> > > main
> > > post
> > >
> > > existing by default, making it easier for authors to specify
stages in
> their
> > > modules.
> > >
> > > Thoughts?
> > >
> >
> > The answer to question "Which comes first,
''bootstrap'' or ''pre''?" seems
> > awfully ambiguous from just the names.
> >
> > What''s the reason for separating it out?
>
> One of the reason would be for the bootstrap phase to happen in its own
> run instead of being part of the standard run. That would allow to
> pre-install stuff that plugins could use (like for instance mysqladmin
> for the mysql types). Then the 3 other stages would happen in the same
> run.
>
> It would also be great to have this stage being optional in subsequent
> runs, allowing you to use the bootstrap stage during provisioning (ie
> just after a pre-seed or kickstart), but never again. This would help
> bootstrap from bare-metal.
>
Whilst it initially sounds useful, is that distinction not somewhat
arbitrary? The standard logic conventions can deal with that already and
you''d then also need to define what constitutes the provisioning run. I
suppose a command line parameter on the call in the kickstart is easy
enough, but then surely that model would need to be generic across all
stages and not only targeted at one stage?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Puppet already has stage[ Main ] which is the only
stage it needs to define.  All other stages
can be defined relative to main and each other, and should
be a matter of convention.  So I think it would be more
fruitful to talk about the purpose of stages, along with
their proposed names.

For example, I have a class that sets up the apt proxy.
It defines two stages: preupdate followed by update.
Their purpose is to set up the network environment
needed for downloading packages, so the implication is
that any class that installs packages must be
expressed later in the run.

I would diagram it like this:

preupdate -> update -> ... -> main
                      |
                      +---- no apt proxy before this point


-- 
vagn

On 06/10/2011 04:20 AM, Chris Phillips wrote:
>
>
> On 10 June 2011 09:06, Brice Figureau <brice-puppet@daysofwonder.com
> <mailto:brice-puppet@daysofwonder.com>> wrote:
>
>     On Thu, 2011-06-09 at 18:50 -0700, Jacob Helwig wrote:
>      > On Thu, 09 Jun 2011 18:42:54 -0700, Nigel Kersten wrote:
>      > >
>      > > https://projects.puppetlabs.com/issues/7697
>      > >
>      > > One problem people producing modules that make use of stages
>     are hitting is
>      > > that it''s difficult to create something reusable
that
>     integrates seamlessly
>      > > into existing setups.
>      > >
>      > > This feature request is to add several more implicit stages
to
>     Puppet so we
>      > > have:
>      > >
>      > > bootstrap
>      > > pre
>      > > main
>      > > post
>      > >
>      > > existing by default, making it easier for authors to specify
>     stages in their
>      > > modules.
>      > >
>      > > Thoughts?
>      > >
>      >
>      > The answer to question "Which comes first,
''bootstrap'' or ''pre''?"
>     seems
>      > awfully ambiguous from just the names.
>      >
>      > What''s the reason for separating it out?
>
>     One of the reason would be for the bootstrap phase to happen in its own
>     run instead of being part of the standard run. That would allow to
>     pre-install stuff that plugins could use (like for instance mysqladmin
>     for the mysql types). Then the 3 other stages would happen in the same
>     run.
>
>     It would also be great to have this stage being optional in subsequent
>     runs, allowing you to use the bootstrap stage during provisioning (ie
>     just after a pre-seed or kickstart), but never again. This would help
>     bootstrap from bare-metal.
>
>
> Whilst it initially sounds useful, is that distinction not somewhat
> arbitrary? The standard logic conventions can deal with that already and
> you''d then also need to define what constitutes the provisioning
run. I
> suppose a command line parameter on the call in the kickstart is easy
> enough, but then surely that model would need to be generic across all
> stages and not only targeted at one stage?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jun 10, 3:06 am, Brice Figureau <brice-pup...@daysofwonder.com>
wrote:
> On Thu, 2011-06-09 at 18:50 -0700, Jacob Helwig wrote:
> > On Thu, 09 Jun 2011 18:42:54 -0700, Nigel Kersten wrote:
>
> > >https://projects.puppetlabs.com/issues/7697
>
> > > One problem people producing modules that make use of stages are
hitting is
> > > that it''s difficult to create something reusable that
integrates seamlessly
> > > into existing setups.
>
> > > This feature request is to add several more implicit stages to
Puppet so we
> > > have:
>
> > > bootstrap
> > > pre
> > > main
> > > post
>
> > > existing by default, making it easier for authors to specify
stages in their
> > > modules.
>
> > > Thoughts?
>
> > The answer to question "Which comes first,
''bootstrap'' or ''pre''?" seems
> > awfully ambiguous from just the names.
>
> > What''s the reason for separating it out?
>
> One of the reason would be for the bootstrap phase to happen in its own
> run instead of being part of the standard run. That would allow to
> pre-install stuff that plugins could use (like for instance mysqladmin
> for the mysql types). Then the 3 other stages would happen in the same
> run.
>
> It would also be great to have this stage being optional in subsequent
> runs, allowing you to use the bootstrap stage during provisioning (ie
> just after a pre-seed or kickstart), but never again. This would help
> bootstrap from bare-metal.

The usual recommendation for tackling this problem used to be
environments.  You would use a different environment for bootstrap,
and you might even have a resource in the bootstrap environment that
changes the node''s environment for future runs.  Does this approach no
longer work?

I think endowing a particular stage with special case rules is a
really poor idea.  I guarantee that doing so will cause grief for both
users and developers.

Maybe what is needed is an easier way to perform two or more
successive puppet runs in different environments.  I think that''s
essentially what those asking for fact re-evaluation between want.
Instead of making run stages a lot more complicated, why not take
something that already does the job, and make it better?


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, Jun 9, 2011 at 6:50 PM, Jacob Helwig <jacob@puppetlabs.com> wrote:
> On Thu, 09 Jun 2011 18:42:54 -0700, Nigel Kersten wrote:
> >
> > https://projects.puppetlabs.com/issues/7697
> >
> > One problem people producing modules that make use of stages are
hitting
> is
> > that it''s difficult to create something reusable that
integrates
> seamlessly
> > into existing setups.
> >
> > This feature request is to add several more implicit stages to Puppet
so
> we
> > have:
> >
> > bootstrap
> > pre
> > main
> > post
> >
> > existing by default, making it easier for authors to specify stages in
> their
> > modules.
> >
> > Thoughts?
> >
>
> The answer to question "Which comes first,
''bootstrap'' or ''pre''?" seems
> awfully ambiguous from just the names.
>
Agreed. I''d like a clearer distinction.

> What''s the reason for separating it out?


We have a default stage when the stage is undefined. [main]

You need a stage before this to do things like lay down normal package
repositories [pre]

You need a stage before this to set up required libraries/binaries for
providers to lay down repositories. [bootstrap]


You could have pre,main,post only if the default stage were
''post'', but that
feels counter-intuitive.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Fri, Jun 10, 2011 at 4:35 AM, Vagn Scott <vagnscott@gmail.com> wrote:
> Puppet already has stage[ Main ] which is the only
> stage it needs to define.  All other stages
> can be defined relative to main and each other, and should
> be a matter of convention.

This is true, but only if you don''t care about sharing your puppet code
with
anyone else, or consuming anyone else''s code.

Say I build a module to set up a service that requires installing libraries,
laying down package repos, installing packages, and setting up a service.

You can''t use stages for this in your module, because there are no
conventions around stages other than [main] that you can rely existing in
deployments other than your own.

>  So I think it would be more
> fruitful to talk about the purpose of stages, along with
> their proposed names.
>
As the OP, I''m going to say that I''m really interested in
hearing about
this, but please start another thread :)

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 06/10/2011 08:55 AM, jcbollinger wrote:
> Maybe what is needed is an easier way to perform two or more
> successive puppet runs in different environments.
>    
It would be nice to have a puppet restart command.
Tell puppet that it should reinitialize and try again.

Then you could, as you say, have a module that
sets up the next environment.  With an immediate
restart you could then run it without waiting.

-- 
vagn

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, Jun 9, 2011 at 6:42 PM, Nigel Kersten <nigel@puppetlabs.com>
wrote:
> https://projects.puppetlabs.com/issues/7697
>
> One problem people producing modules that make use of stages are hitting is
> that it''s difficult to create something reusable that integrates
seamlessly
> into existing setups.
>
> This feature request is to add several more implicit stages to Puppet so we
> have:
>
> bootstrap
> pre
> main
> post
>
> existing by default, making it easier for authors to specify stages in
> their modules.
>
I agree it would be convenient to have implicit stages (and I opened the
ticket :) ). Even with these implicit stages, it would still be hard to use
them in a reusable way in modules b/c stages can only be applied to param
classes.


> Thoughts?
>
> --
> Nigel Kersten
> Product, Puppet Labs
> @nigelkersten
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Fri, Jun 10, 2011 at 6:35 AM, Dan Bode <dan@puppetlabs.com> wrote:
>
>
> On Thu, Jun 9, 2011 at 6:42 PM, Nigel Kersten
<nigel@puppetlabs.com>wrote:
>
>> https://projects.puppetlabs.com/issues/7697
>>
>> One problem people producing modules that make use of stages are
hitting
>> is that it''s difficult to create something reusable that
integrates
>> seamlessly into existing setups.
>>
>> This feature request is to add several more implicit stages to Puppet
so
>> we have:
>>
>> bootstrap
>> pre
>> main
>> post
>>
>> existing by default, making it easier for authors to specify stages in
>> their modules.
>>
>
> I agree it would be convenient to have implicit stages (and I opened the
> ticket :) ). Even with these implicit stages, it would still be hard to use
> them in a reusable way in modules b/c stages can only be applied to param
> classes.
>
That could be solved relatively easily with a stage() function just like the
tag() function we have now.

>
>
>
>> Thoughts?
>>
>> --
>> Nigel Kersten
>> Product, Puppet Labs
>> @nigelkersten
>>
>>  --
>> You received this message because you are subscribed to the Google
Groups
>> "Puppet Users" group.
>> To post to this group, send email to puppet-users@googlegroups.com.
>> To unsubscribe from this group, send email to
>> puppet-users+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/puppet-users?hl=en.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>


-- 
Nigel Kersten
Product, Puppet Labs
@nigelkersten

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Fri, Jun 10, 2011 at 6:12 AM, Nigel Kersten <nigel@puppetlabs.com>
wrote:
>
>
> On Thu, Jun 9, 2011 at 6:50 PM, Jacob Helwig <jacob@puppetlabs.com>
wrote:
>>
>> On Thu, 09 Jun 2011 18:42:54 -0700, Nigel Kersten wrote:
>> >
>> > https://projects.puppetlabs.com/issues/7697
>> >
>> > One problem people producing modules that make use of stages are
hitting
>> > is
>> > that it''s difficult to create something reusable that
integrates
>> > seamlessly
>> > into existing setups.
>> >
>> > This feature request is to add several more implicit stages to
Puppet so
>> > we
>> > have:
>> >
>> > bootstrap
>> > pre
>> > main
>> > post
>> >
>> > existing by default, making it easier for authors to specify
stages in
>> > their
>> > modules.
>> >
>> > Thoughts?
>> >
>>
>> The answer to question "Which comes first,
''bootstrap'' or ''pre''?" seems
>> awfully ambiguous from just the names.
>
> Agreed. I''d like a clearer distinction.
>
>>
>> What''s the reason for separating it out?
>
> We have a default stage when the stage is undefined. [main]
> You need a stage before this to do things like lay down normal package
> repositories [pre]
> You need a stage before this to set up required libraries/binaries for
> providers to lay down repositories. [bootstrap]
>
> You could have pre,main,post only if the default stage were
''post'', but that
> feels counter-intuitive.
>
There are two intended use case for stages and I was hoping we can
split them up into:

1. stages for bootstrap that will halt processing of dependent stages.
2. stages for relationship organization that will not halt processing
of subsequent stages.

stage { ''bootstrap'':
  ensure  => enforcement,
  before   => Stage[''pre''],
}
stage { ''pre'':
  ensure => enforcement,
  before => Stage[''main'']
}
stage { ''main'':
  ensure => relationship,
  before => Stage[''post'']
}
stage { ''post'':
}

Stages that are enforcement must have all resource completed, stages
that are purely relationship do not block subsequent stages, but
resource dependency still apply within the stage. Would something like
this make sense?

Thanks,

Nan

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Let''s see: Then I could do

Stage { ensure => enforcement, }

at the top and get the behavior I want:
prerequisites are satisfied before moving on.

But, what''s the use case for relationship?
Why would I want that?

consider three stages in various
combinations of enforcement (e) and relationship (r)

0. e e e
1. e e r
2. e r e
3. e r r
4. r e e
5. r e r
6. r r e
7. r r r

Does it matter how we define the third stage?
I think it is a don''t-care:

[0,1]. e e -
[2,3]. e r -
[4,5]. r e -
[6,7]. r r -

And then the 2nd stage is a don''t-care:

[0,1,2,3]. e - -
[4,5,6,7]. r - -

And then none of them matter:

[*]. - - -

It seems to me that stages are only sensible if they
are either all enforcement or all relationship,
but not mixed.  So, if I had to choose I
would choose enforcement, because I have other ways to
express relationship (before/after/require/notify).

Also, suppose I set

Stage { ensure => enforcement, }

for the site and some module decides to do:

stage { ''unique_snowflake'':
	before => Stage[''whatever''],
	ensure => relationship,
}

What does that do to my design?

Since enforcement subsumes relationship, anyway,
why don''t stages just do enforcement?

-- 
vagn

On 06/10/2011 11:15 PM, Nan Liu wrote:
>
> There are two intended use case for stages and I was hoping we can
> split them up into:
>
> 1. stages for bootstrap that will halt processing of dependent stages.
> 2. stages for relationship organization that will not halt processing
> of subsequent stages.
>
> stage { ''bootstrap'':
>    ensure  =>  enforcement,
>    before   =>  Stage[''pre''],
> }
> stage { ''pre'':
>    ensure =>  enforcement,
>    before =>  Stage[''main'']
> }
> stage { ''main'':
>    ensure =>  relationship,
>    before =>  Stage[''post'']
> }
> stage { ''post'':
> }
>
> Stages that are enforcement must have all resource completed, stages
> that are purely relationship do not block subsequent stages, but
> resource dependency still apply within the stage. Would something like
> this make sense?
>
> Thanks,
>
> Nan
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Sat, Jun 11, 2011 at 6:39 AM, Vagn Scott <vagnscott@gmail.com>
wrote:
> Let''s see: Then I could do
>
> Stage { ensure => enforcement, }
>
> at the top and get the behavior I want:
> prerequisites are satisfied before moving on.
>
> But, what''s the use case for relationship?
> Why would I want that?
>
> consider three stages in various
> combinations of enforcement (e) and relationship (r)
>
> 0. e e e
> 1. e e r
> 2. e r e
> 3. e r r
> 4. r e e
> 5. r e r
> 6. r r e
> 7. r r r
>
> Does it matter how we define the third stage?
> I think it is a don''t-care:
>
> [0,1]. e e -
> [2,3]. e r -
> [4,5]. r e -
> [6,7]. r r -
>
> And then the 2nd stage is a don''t-care:
>
> [0,1,2,3]. e - -
> [4,5,6,7]. r - -
>
> And then none of them matter:
>
> [*]. - - -
>
> It seems to me that stages are only sensible if they
> are either all enforcement or all relationship,
> but not mixed.  So, if I had to choose I
> would choose enforcement, because I have other ways to
> express relationship (before/after/require/notify).
I''m not sure if I captured the intention well, essentially still want
a way to coarsely organize classes, however without adding any
dependency requirement. So deploy user accounts/customization after
application deployment, however still proceed with user deployment
even if there was any issues with the application deployment.
> Also, suppose I set
>
> Stage { ensure => enforcement, }
>
> for the site and some module decides to do:
>
> stage { ''unique_snowflake'':
>        before => Stage[''whatever''],
>        ensure => relationship,
> }
>
> What does that do to my design?
>
> Since enforcement subsumes relationship, anyway,
> why don''t stages just do enforcement?
>
> --
> vagn
>
> On 06/10/2011 11:15 PM, Nan Liu wrote:
>>
>> There are two intended use case for stages and I was hoping we can
>> split them up into:
>>
>> 1. stages for bootstrap that will halt processing of dependent stages.
>> 2. stages for relationship organization that will not halt processing
>> of subsequent stages.
>>
>> stage { ''bootstrap'':
>>   ensure  =>  enforcement,
>>   before   =>  Stage[''pre''],
>> }
>> stage { ''pre'':
>>   ensure =>  enforcement,
>>   before =>  Stage[''main'']
>> }
>> stage { ''main'':
>>   ensure =>  relationship,
>>   before =>  Stage[''post'']
>> }
>> stage { ''post'':
>> }
>>
>> Stages that are enforcement must have all resource completed, stages
>> that are purely relationship do not block subsequent stages, but
>> resource dependency still apply within the stage. Would something like
>> this make sense?
>>
>> Thanks,
>>
>> Nan
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 06/11/2011 01:06 PM, Nan Liu wrote:
> I''m not sure if I captured the intention well, essentially still
want
> a way to coarsely organize classes, however without adding any
> dependency requirement. So deploy user accounts/customization after
> application deployment, however still proceed with user deployment
> even if there was any issues with the application deployment.
If you are not expressing dependency, then where is the
need for organizing the classes?  Couldn''t you
just as well run the classes in alphabetical order?

We already have Class[ foo ] -> Class[ baz ]

Which implies a topological sort.  But it fails to
capture that one set of classes must succeed
before another set can be attempted.  I think stages
should express that kind of boundary, and not just
be another expression of sequencing.

Are you perhaps talking about a rendezvous, where
multiple sequences of stages all have to
complete before the rendezvous will unblock?

a -> b -> c -> g
d -> f -> c

With c as a rendezvous d and f can complete even
if a stalls. Nothing in c or g will run until all
of [ a b d f ] complete.

In my thinking this was implicit in enforcement.
But maybe rendezvous is a more descriptive term.

-- 
vagn

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.