Puppet users - Feb 2010 - Building a better puppetrun and related ideas

Teyo, Bruce, and I were bouncing around some ideas resently for an
simple but enhanced puppetrun.

Basically the idea is merging the ideas behind Func and Puppetrun.
Obviously other tools like mcollective have various other advantaged
features so this will be fairly primative by comparison, though it
won''t require a message bus.  If you want something more advanced
obviously try out those tools, this is covering a much smaller use case.

This is something I am going to take a crack at this in the coming
weeks.    This would be something pretty simple and lightweight, and
could
probably fix a lot of the use cases around making puppetrun (or
staggering large sets of hosts) a lot easier.

Features I''m thinking of:

Requires no additonal ports, setup, or config files -- use existing
puppet listening capability and puppetca, just a /usr/bin app
Be able to query dashboard DB to run against tagged nodes or hosts
that have certain data there (or in storeconfigs???)
Be able to run against wildcarded nodes based on what certs are
present on the puppetmaster (we know the hostnames)
Be able to be used easily from an API perspective from any ruby application
Be able to invoke ralsh remotely for querying things (and for debug,
and one off tasks)
Be able to run shell commands for things that are one offs (emergency
security power down now)

Example syntax:

punc --hosts *.example.org --puppetize  # get new catalog and run
punc --hosts *.example.org --ralsh "service name=foo ensure=running"
# perform an action through ralsh
punc --hosts *.example.org --shell "/bin/emergency_script"   # run a
shell script... for the one-off cases
punc --hosts foo.*.example.org --ralsh "service name=foo"
--format=json  # query something with ralsh and generate a report
punc --hosts foo.*.example.org --facter fact --format=json # similarly
generate a facter report
punc --tags webservers [...ditto...]
punc --critiera "fact==foo" [..ditto...]
punc --critiera "fact==foo" [some operation to run only if fact
matches] [...ditto...]

So for example we could choose to reboot all the servers that match a
given fact, etc.

It should also allow easier staged deployments and environment usage
from apps that want to use the API.

Additional ideas for stuff you would like to see?

--Michael

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Michael DeHaan wrote:
> Example syntax:
> 
> punc
Python or Ruby?

-scott

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wed, Feb 10, 2010 at 12:49 PM, Scott Smith <scott@ohlol.net>
wrote:
> Michael DeHaan wrote:
>>
>> Example syntax:
>>
>> punc
>
> Python or Ruby?
>
> -scott

Totally Ruby :)

I think the RAL really makes up for and eliminates the need of a lot
of the stuff we tried to do as Func modules.   Good stuff.   I think
most people use Func to run bash regardless... however the main thing
we want to do here is just tell Puppet to wake up and run.
(Combining this workflow with no-op mode for instance)

Similarly, facter and everything is nicely extensible and can be
easily rigged up to do things like the "Exploding Laptop Finder" demo
script Func had ... I''d also like to do something similar to
https://fedorahosted.org/func/wiki/FuncInventory ... basically it''s a
really quick way to get an RSS feed of an arbitrary anything.

--Michael

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Sounds promising. What about a combination of shell execution and a
normal puppetrun? Something like run a shell command, apply catalog,
run another command.

On Wed, Feb 10, 2010 at 6:45 PM, Michael DeHaan
<michael@reductivelabs.com> wrote:
> Teyo, Bruce, and I were bouncing around some ideas resently for an
> simple but enhanced puppetrun.
>
> Basically the idea is merging the ideas behind Func and Puppetrun.
> Obviously other tools like mcollective have various other advantaged
> features so this will be fairly primative by comparison, though it
> won''t require a message bus.  If you want something more advanced
> obviously try out those tools, this is covering a much smaller use case.
>
> This is something I am going to take a crack at this in the coming
> weeks.    This would be something pretty simple and lightweight, and
> could
> probably fix a lot of the use cases around making puppetrun (or
> staggering large sets of hosts) a lot easier.
>
> Features I''m thinking of:
>
> Requires no additonal ports, setup, or config files -- use existing
> puppet listening capability and puppetca, just a /usr/bin app
> Be able to query dashboard DB to run against tagged nodes or hosts
> that have certain data there (or in storeconfigs???)
> Be able to run against wildcarded nodes based on what certs are
> present on the puppetmaster (we know the hostnames)
> Be able to be used easily from an API perspective from any ruby application
> Be able to invoke ralsh remotely for querying things (and for debug,
> and one off tasks)
> Be able to run shell commands for things that are one offs (emergency
> security power down now)
>
> Example syntax:
>
> punc --hosts *.example.org --puppetize  # get new catalog and run
> punc --hosts *.example.org --ralsh "service name=foo
ensure=running"
> # perform an action through ralsh
> punc --hosts *.example.org --shell "/bin/emergency_script"   #
run a
> shell script... for the one-off cases
> punc --hosts foo.*.example.org --ralsh "service name=foo"
> --format=json  # query something with ralsh and generate a report
> punc --hosts foo.*.example.org --facter fact --format=json # similarly
> generate a facter report
> punc --tags webservers [...ditto...]
> punc --critiera "fact==foo" [..ditto...]
> punc --critiera "fact==foo" [some operation to run only if fact
> matches] [...ditto...]
>
> So for example we could choose to reboot all the servers that match a
> given fact, etc.
>
> It should also allow easier staged deployments and environment usage
> from apps that want to use the API.
>
> Additional ideas for stuff you would like to see?
>
> --Michael
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
>
>


-- 

Cheers,

Daniel

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Daniel wrote:
> Sounds promising. What about a combination of shell execution and a
> normal puppetrun? Something like run a shell command, apply catalog,
> run another command.
>
>   
Possibly --- What''s the shell command in that example so I can grok the
use case?

OT -- one thing I didn''t explain about Func is it had the notion of
"do
this N at a time", in our particular case,
there may be an opportunity to teach N hosts to update at a time, and 
then do N hosts later... so in very large
setups, it would be possible to do easy rolling puppetmaster updates.  

Func actually handled this by forking and running synchronously, so this 
would need to be a bit different.

--Michael

> On Wed, Feb 10, 2010 at 6:45 PM, Michael DeHaan
> <michael@reductivelabs.com> wrote:
>   
>> Teyo, Bruce, and I were bouncing around some ideas resently for an
>> simple but enhanced puppetrun.
>>
>> Basically the idea is merging the ideas behind Func and Puppetrun.
>> Obviously other tools like mcollective have various other advantaged
>> features so this will be fairly primative by comparison, though it
>> won''t require a message bus.  If you want something more
advanced
>> obviously try out those tools, this is covering a much smaller use
case.
>>
>> This is something I am going to take a crack at this in the coming
>> weeks.    This would be something pretty simple and lightweight, and
>> could
>> probably fix a lot of the use cases around making puppetrun (or
>> staggering large sets of hosts) a lot easier.
>>
>> Features I''m thinking of:
>>
>> Requires no additonal ports, setup, or config files -- use existing
>> puppet listening capability and puppetca, just a /usr/bin app
>> Be able to query dashboard DB to run against tagged nodes or hosts
>> that have certain data there (or in storeconfigs???)
>> Be able to run against wildcarded nodes based on what certs are
>> present on the puppetmaster (we know the hostnames)
>> Be able to be used easily from an API perspective from any ruby
application
>> Be able to invoke ralsh remotely for querying things (and for debug,
>> and one off tasks)
>> Be able to run shell commands for things that are one offs (emergency
>> security power down now)
>>
>> Example syntax:
>>
>> punc --hosts *.example.org --puppetize  # get new catalog and run
>> punc --hosts *.example.org --ralsh "service name=foo
ensure=running"
>> # perform an action through ralsh
>> punc --hosts *.example.org --shell "/bin/emergency_script"  
# run a
>> shell script... for the one-off cases
>> punc --hosts foo.*.example.org --ralsh "service name=foo"
>> --format=json  # query something with ralsh and generate a report
>> punc --hosts foo.*.example.org --facter fact --format=json # similarly
>> generate a facter report
>> punc --tags webservers [...ditto...]
>> punc --critiera "fact==foo" [..ditto...]
>> punc --critiera "fact==foo" [some operation to run only if
fact
>> matches] [...ditto...]
>>
>> So for example we could choose to reboot all the servers that match a
>> given fact, etc.
>>
>> It should also allow easier staged deployments and environment usage
>> from apps that want to use the API.
>>
>> Additional ideas for stuff you would like to see?
>>
>> --Michael
>>
>> --
>> You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
>> To post to this group, send email to puppet-users@googlegroups.com.
>> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
>> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
>>
>>
>>     
>
>
>
>   
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Something like "/etc/init.d/tomcat stop", "deploy",
"/etc/init.d/tomcat start". It also would be great to manage services
in general. Like stop/start/restart service X.

On Wed, Feb 10, 2010 at 8:49 PM, Michael DeHaan
<michael@reductivelabs.com> wrote:
> Daniel wrote:
>>
>> Sounds promising. What about a combination of shell execution and a
>> normal puppetrun? Something like run a shell command, apply catalog,
>> run another command.
>>
>>
>
> Possibly --- What''s the shell command in that example so I can
grok the use
> case?
>
> OT -- one thing I didn''t explain about Func is it had the notion
of "do this
> N at a time", in our particular case,
> there may be an opportunity to teach N hosts to update at a time, and then
> do N hosts later... so in very large
> setups, it would be possible to do easy rolling puppetmaster updates.
> Func actually handled this by forking and running synchronously, so this
> would need to be a bit different.
>
> --Michael
>
>
>> On Wed, Feb 10, 2010 at 6:45 PM, Michael DeHaan
>> <michael@reductivelabs.com> wrote:
>>
>>>
>>> Teyo, Bruce, and I were bouncing around some ideas resently for an
>>> simple but enhanced puppetrun.
>>>
>>> Basically the idea is merging the ideas behind Func and Puppetrun.
>>> Obviously other tools like mcollective have various other
advantaged
>>> features so this will be fairly primative by comparison, though it
>>> won''t require a message bus.  If you want something more
advanced
>>> obviously try out those tools, this is covering a much smaller use
case.
>>>
>>> This is something I am going to take a crack at this in the coming
>>> weeks.    This would be something pretty simple and lightweight,
and
>>> could
>>> probably fix a lot of the use cases around making puppetrun (or
>>> staggering large sets of hosts) a lot easier.
>>>
>>> Features I''m thinking of:
>>>
>>> Requires no additonal ports, setup, or config files -- use existing
>>> puppet listening capability and puppetca, just a /usr/bin app
>>> Be able to query dashboard DB to run against tagged nodes or hosts
>>> that have certain data there (or in storeconfigs???)
>>> Be able to run against wildcarded nodes based on what certs are
>>> present on the puppetmaster (we know the hostnames)
>>> Be able to be used easily from an API perspective from any ruby
>>> application
>>> Be able to invoke ralsh remotely for querying things (and for
debug,
>>> and one off tasks)
>>> Be able to run shell commands for things that are one offs
(emergency
>>> security power down now)
>>>
>>> Example syntax:
>>>
>>> punc --hosts *.example.org --puppetize  # get new catalog and run
>>> punc --hosts *.example.org --ralsh "service name=foo
ensure=running"
>>> # perform an action through ralsh
>>> punc --hosts *.example.org --shell
"/bin/emergency_script"   # run a
>>> shell script... for the one-off cases
>>> punc --hosts foo.*.example.org --ralsh "service name=foo"
>>> --format=json  # query something with ralsh and generate a report
>>> punc --hosts foo.*.example.org --facter fact --format=json #
similarly
>>> generate a facter report
>>> punc --tags webservers [...ditto...]
>>> punc --critiera "fact==foo" [..ditto...]
>>> punc --critiera "fact==foo" [some operation to run only
if fact
>>> matches] [...ditto...]
>>>
>>> So for example we could choose to reboot all the servers that match
a
>>> given fact, etc.
>>>
>>> It should also allow easier staged deployments and environment
usage
>>> from apps that want to use the API.
>>>
>>> Additional ideas for stuff you would like to see?
>>>
>>> --Michael
>>>
>>> --
>>> You received this message because you are subscribed to the Google
Groups
>>> "Puppet Users" group.
>>> To post to this group, send email to puppet-users@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> puppet-users+unsubscribe@googlegroups.com.
>>> For more options, visit this group at
>>> http://groups.google.com/group/puppet-users?hl=en.
>>>
>>>
>>>
>>
>>
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>


-- 

Cheers,

Daniel

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Michael DeHaan wrote:
> Additional ideas for stuff you would like to see?
>
> --Michael
>
>   
Please take out the ''feature'' that you need LDAP hosts to run
puppetrun
on a wide scale. The utility becomes useless for a large portion of 
people. I searched the thread quickly and didn''t see this mentioned...

-- 
Joe McDonagh
AIM: YoosingYoonickz
IRC: joe-mac on freenode
L''ennui est contre-révolutionnaire

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Daniel wrote:
> Something like "/etc/init.d/tomcat stop", "deploy",
> "/etc/init.d/tomcat start". It also would be great to manage
services
> in general. Like stop/start/restart service X.
>   
Seems like this would be better modelled as:

punc --server-tags webservers --puppetrun --tags "tomcat"

And model the restarting of the service in Puppet.

punc could support running just tagged aspects of the catalog.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Joe McDonagh wrote:
> Michael DeHaan wrote:
>> Additional ideas for stuff you would like to see?
>>
>> --Michael
>>
>>   
> Please take out the ''feature'' that you need LDAP hosts to
run
> puppetrun on a wide scale. The utility becomes useless for a large 
> portion of people. I searched the thread quickly and didn''t see
this
> mentioned...
>
I hadn''t even thought of supporting LDAP, actually :)

First data sources will be simple wildcards of the certs we have, second 
data source will be the dashboard DB (and anything else that follows 
this schema).     LDAP would be a good patch for later, I''m guessing, 
but probably not something I''d do initially.

--Michael


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Michael DeHaan wrote:
> OT -- one thing I didn''t explain about Func is it had the notion
of "do
> this N at a time", in our particular case,
> there may be an opportunity to teach N hosts to update at a time, and 
> then do N hosts later... so in very large
> setups, it would be possible to do easy rolling puppetmaster updates. 
> Func actually handled this by forking and running synchronously, so this 
> would need to be a bit different.
Cool, sounds like dsh.

-scott

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Michael DeHaan wrote:
> Joe McDonagh wrote:
>> Michael DeHaan wrote:
>>> Additional ideas for stuff you would like to see?
>>>
>>> --Michael
>>>
>>>   
>> Please take out the ''feature'' that you need LDAP
hosts to run
>> puppetrun on a wide scale. The utility becomes useless for a large 
>> portion of people. I searched the thread quickly and didn''t
see this
>> mentioned...
>>
> I hadn''t even thought of supporting LDAP, actually :)
>
> First data sources will be simple wildcards of the certs we have, 
> second data source will be the dashboard DB (and anything else that 
> follows this schema).     LDAP would be a good patch for later,
I''m
> guessing, but probably not something I''d do initially.
>
> --Michael
>
>
It already does support LDAP. In fact if you try to use puppetrun -a it 
*requires* LDAP. That''s my beef with it. Why can''t it look at
my
nodes.pp and get all the nodes from there?

-- 
Joe McDonagh
AIM: YoosingYoonickz
IRC: joe-mac on freenode
L''ennui est contre-révolutionnaire

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Joe McDonagh wrote:
> Michael DeHaan wrote:
>> Joe McDonagh wrote:
>>> Michael DeHaan wrote:
>>>> Additional ideas for stuff you would like to see?
>>>>
>>>> --Michael
>>>>
>>>>   
>>> Please take out the ''feature'' that you need LDAP
hosts to run
>>> puppetrun on a wide scale. The utility becomes useless for a large 
>>> portion of people. I searched the thread quickly and
didn''t see this
>>> mentioned...
>>>
>> I hadn''t even thought of supporting LDAP, actually :)
>>
>> First data sources will be simple wildcards of the certs we have, 
>> second data source will be the dashboard DB (and anything else that 
>> follows this schema).     LDAP would be a good patch for later,
I''m
>> guessing, but probably not something I''d do initially.
>>
>> --Michael
>>
>>
> It already does support LDAP. In fact if you try to use puppetrun -a 
> it *requires* LDAP. That''s my beef with it. Why can''t it
look at my
> nodes.pp and get all the nodes from there?
>
Yes, I''m aware.   We may need to support that going forward for 
mantainability, I need to investigate but we definitely won''t require
LDAP.

As for nodes.pp --
nodes.pp can contain regexen, IIRC.    I suppose technically that might 
be possible, but short term, we probably won''t do this.

There''s nothing saying we can''t look at the list of certs,
which is what
Func did, but I also want it to be able to use extended metadata and 
info from the dashboard interface (or anything else that uses that 
schema).  It may also be interesting to maintain an external json 
catalog of nodes and tags, TBD... I don''t want to overdesign that bit 
too much up front -- "YAGNI", and what all.

(We also need a CLI for dashboard... so it''s easy to add nodes and tag 
them here... we don''t want people using
the extended DB to have to click around a WebUI if they don''t want to, 
and it would be helpful with batch population).

--Michael



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Michael DeHaan wrote:
> Joe McDonagh wrote:
>> Michael DeHaan wrote:
>>> Joe McDonagh wrote:
>>>> Michael DeHaan wrote:
>>>>> Additional ideas for stuff you would like to see?
>>>>>
>>>>> --Michael
>>>>>
>>>>>   
>>>> Please take out the ''feature'' that you need
LDAP hosts to run
>>>> puppetrun on a wide scale. The utility becomes useless for a
large
>>>> portion of people. I searched the thread quickly and
didn''t see
>>>> this mentioned...
>>>>
>>> I hadn''t even thought of supporting LDAP, actually :)
>>>
>>> First data sources will be simple wildcards of the certs we have, 
>>> second data source will be the dashboard DB (and anything else that
>>> follows this schema).     LDAP would be a good patch for later,
I''m
>>> guessing, but probably not something I''d do initially.
>>>
>>> --Michael
>>>
>>>
>> It already does support LDAP. In fact if you try to use puppetrun -a 
>> it *requires* LDAP. That''s my beef with it. Why can''t
it look at my
>> nodes.pp and get all the nodes from there?
>>
> Yes, I''m aware.   We may need to support that going forward for 
> mantainability, I need to investigate but we definitely won''t
require
> LDAP.
>
> As for nodes.pp --
> nodes.pp can contain regexen, IIRC.    I suppose technically that 
> might be possible, but short term, we probably won''t do this.
>
> There''s nothing saying we can''t look at the list of
certs, which is
> what Func did, but I also want it to be able to use extended metadata 
> and info from the dashboard interface (or anything else that uses that 
> schema).  It may also be interesting to maintain an external json 
> catalog of nodes and tags, TBD... I don''t want to overdesign that
bit
> too much up front -- "YAGNI", and what all.
>
> (We also need a CLI for dashboard... so it''s easy to add nodes and
tag
> them here... we don''t want people using
> the extended DB to have to click around a WebUI if they don''t want
to,
> and it would be helpful with batch population).
>
> --Michael
>
>
>
Good point, your idea for list of certs is also valid, I am also down 
with metadata, I just think the tool right now is kinda useless. If this 
is improved I can ditch capistrano for scalable administration and leave 
it to deploys.

-- 
Joe McDonagh
AIM: YoosingYoonickz
IRC: joe-mac on freenode
L''ennui est contre-révolutionnaire

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Michael DeHaan wrote:
> (We also need a CLI for dashboard... so it''s easy to add nodes and
tag
> them here... we don''t want people using
> the extended DB to have to click around a WebUI if they don''t want
to,
> and it would be helpful with batch population).
http://github.com/ohlol/puppet-dashboard/blob/master/lib/tasks/node.rake

The code sucks! I refactored it but need to push.

-scott

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wed, Feb 10, 2010 at 5:30 PM, Scott Smith <scott@ohlol.net>
wrote:
> Michael DeHaan wrote:
>>
>> (We also need a CLI for dashboard... so it''s easy to add nodes
and tag
>> them here... we don''t want people using
>> the extended DB to have to click around a WebUI if they don''t
want to, and
>> it would be helpful with batch population).
>
> http://github.com/ohlol/puppet-dashboard/blob/master/lib/tasks/node.rake
>
> The code sucks! I refactored it but need to push.

Yeah that''s the idea ... longer term, I think we need a user CLI that
takes formal options rather than having rake as an interface (puppet
isn''t invoked with rake either, etc) ...

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

My thoughts:

- - I would have it be a stand alone application, not an add-on to the
primary puppet daemon. The reason for this is that many of us use puppet
from cron instead of as a daemon for various reasons.

- - I do have to wonder how this is much different that cobbling together
the results of puppetlast (to get the nodes that have checked in) with
some sort of parallel SSH solution (besides the fact that you''re using
the puppet certs instead of SSH keys).

- - In general, people are going to have hybrid environments for one
reason or another.  The ability to add additional nodes to this list
that can perhaps just use puppetrun for this functionality may be welcome.

A little more below.

On 02/10/2010 12:45 PM, Michael DeHaan wrote:
> 
> Features I''m thinking of:
> 
> Requires no additonal ports, setup, or config files -- use existing
> puppet listening capability and puppetca, just a /usr/bin app
- - I don''t particularly agree with this for the reasons listed above.
> Be able to query dashboard DB to run against tagged nodes or hosts
> that have certain data there (or in storeconfigs???)
- - Sounds good. Should be pluggable so that you can add additional data
sources with little effort. Flat files, LDAP, MySQL, whatever...
> Be able to run against wildcarded nodes based on what certs are
> present on the puppetmaster (we know the hostnames)
- - This makes sense.
> Be able to be used easily from an API perspective from any ruby application
- - I would vote for a combination of this and the ability to do
everything from the command line. This way people can either use Ruby or
their programming language of choice.
> Be able to invoke ralsh remotely for querying things (and for debug,
> and one off tasks)
- - Are there example use cases of this? It sounds useful, but my minimal
ralsh experimentation didn''t really prove to be greatly fruitful.
> Be able to run shell commands for things that are one offs (emergency
> security power down now)
- - This comes back to the common framework. I''m going to want to be
able
to do this regardless of whether or not puppet is actually running. I
may want to do this (or something like it) because puppet is hung for
some reason.

Thanks,

Trevor

- -- 
Trevor Vaughan
 Vice President, Onyx Point, Inc.
 email: tvaughan@onyxpoint.com
 phone: 410-541-ONYX (6699)

- -- This account not approved for unencrypted sensitive information --
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAktzOZQACgkQyWMIJmxwHpTuvgCcDO9JTYioMKt2AcaEk+sumPfj
OjYAnjTzxExPjs1isDy2n2CeU6F6UdLS
=eaQK
-----END PGP SIGNATURE-----

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wed, Feb 10, 2010 at 5:56 PM, Trevor Vaughan <tvaughan@onyxpoint.com>
wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> My thoughts:
>
> - - I would have it be a stand alone application, not an add-on to the
> primary puppet daemon. The reason for this is that many of us use puppet
> from cron instead of as a daemon for various reasons.
Interesting yeah, in that case the transport then becomes SSH though?
Seems to be redundant in that configuration.

We''re attempting to provide a reason to not use cron :)
>
> - - I do have to wonder how this is much different that cobbling together
> the results of puppetlast (to get the nodes that have checked in) with
> some sort of parallel SSH solution (besides the fact that you''re
using
> the puppet certs instead of SSH keys).
It would resemble that... but the idea here is we want to leverage
those that have already checked in
and the existing comm path.   I understand your use case though...  in
which case, I think just using a parallel
SSH (in that case) if you are ok with that, solves the problem... or
continuing to use cron.

Here it is really about replacing puppetrun ... :)

>
> - - In general, people are going to have hybrid environments for one
> reason or another.  The ability to add additional nodes to this list
> that can perhaps just use puppetrun for this functionality may be welcome.
>
> A little more below.
>
> On 02/10/2010 12:45 PM, Michael DeHaan wrote:
>>
>> Features I''m thinking of:
>>
>> Requires no additonal ports, setup, or config files -- use existing
>> puppet listening capability and puppetca, just a /usr/bin app
>
> - - I don''t particularly agree with this for the reasons listed
above.
Don''t follow, can you elaborate?

(I also probably missed one point here... it /might/ require some
config on puppet nodes in order
to enable the new features beyond what it does today... TBD ... we
haven''t talked about this yet)
>
>> Be able to query dashboard DB to run against tagged nodes or hosts
>> that have certain data there (or in storeconfigs???)
>
> - - Sounds good. Should be pluggable so that you can add additional data
> sources with little effort. Flat files, LDAP, MySQL, whatever...
Right.    We''ll start with the host list and dashboard DB to start.
>
>> Be able to run against wildcarded nodes based on what certs are
>> present on the puppetmaster (we know the hostnames)
>
> - - This makes sense.
>
>> Be able to be used easily from an API perspective from any ruby
application
>
> - - I would vote for a combination of this and the ability to do
> everything from the command line. This way people can either use Ruby or
> their programming language of choice.
We''re on the same page :)
>
>> Be able to invoke ralsh remotely for querying things (and for debug,
>> and one off tasks)
>
> - - Are there example use cases of this? It sounds useful, but my minimal
> ralsh experimentation didn''t really prove to be greatly fruitful.
I''m thinking more about querying for states of things than making
changes.
>
>> Be able to run shell commands for things that are one offs (emergency
>> security power down now)
>
> - - This comes back to the common framework. I''m going to want to
be able
> to do this regardless of whether or not puppet is actually running. I
> may want to do this (or something like it) because puppet is hung for
> some reason.
I understand, yeah.  In this case, distributed SSH may be your friend
if you don''t want to run cron and don''t want puppet to listen.
Otherwise it kind of is cross-purposes with the idea of fixing
puppetrun... and extra management of SSH keys is overhead when
puppetca is there for that purpose.

I know what you''re saying though...

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wed, 10 Feb 2010, Michael DeHaan wrote:
> We''re attempting to provide a reason to not use cron :)
I have a requirement that puppet may not change anything on a production
host without change control approval in advance.  It would be nice if a
new version of puppet had better support for this use case.

Of course there are change control procedures for getting the manifests
updated on the puppetmaster, but that''s not enough; it''s also
necessary
to run the puppet client only when specifically authorised.  For
example, the manifest update and a --noop mode client puppet run might
happen during working hours, but the --no-noop client puppet run might
happen during a maintenance window after hours.

As far as I am aware, the existing combination of "puppetd --listen"
on each client, and "puppetrun" on a central server, does not handle
this use case, so I run "puppetd --onetime --noop" or "puppetd
--onetime
--no-noop" via ssh.

--apb (Alan Barrett)

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Alan Barrett wrote:
> On Wed, 10 Feb 2010, Michael DeHaan wrote:
>> We''re attempting to provide a reason to not use cron :)
> 
> I have a requirement that puppet may not change anything on a production
> host without change control approval in advance.  It would be nice if a
> new version of puppet had better support for this use case.
> 
> Of course there are change control procedures for getting the manifests
> updated on the puppetmaster, but that''s not enough; it''s
also necessary
> to run the puppet client only when specifically authorised.  For
> example, the manifest update and a --noop mode client puppet run might
> happen during working hours, but the --no-noop client puppet run might
> happen during a maintenance window after hours.
> 
> As far as I am aware, the existing combination of "puppetd
--listen"
> on each client, and "puppetrun" on a central server, does not
handle
> this use case, so I run "puppetd --onetime --noop" or
"puppetd --onetime
> --no-noop" via ssh.
> 
> --apb (Alan Barrett)
> 
So let me get this straight: You run --noop throughout the day, 
aggregate the changes that need to be made, and then have a EOD/EOW 
"change control" meeting to go over them and determine if you need to 
run puppet without --noop ?

-scott

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi,

May I recommend that you have a look at the ext directory for
puppetlisten/puppetrun[1], this two scripts I wrote a while ago reuse puppet
certificate infrastructure to trigger remote runs.

additionally, I''ve created a query interface in foreman[2], which could
probably give you some ideas of how integration can be done.

by combining this two solutions, I think that you will have a very powerful
solution.

one thing to keep in mind is that many puppet users are using it via cron,
because of the high memory footprint.

thanks,
Ohad

*1 http://github.com/ohadlevy/puppet/tree/puppetlisten/ext/puppetlisten/
*2 http://theforeman.org/wiki/foreman/Query_Interface




On Thu, Feb 11, 2010 at 1:45 AM, Michael DeHaan
<michael@reductivelabs.com>wrote:
> Teyo, Bruce, and I were bouncing around some ideas resently for an
> simple but enhanced puppetrun.
>
> Basically the idea is merging the ideas behind Func and Puppetrun.
> Obviously other tools like mcollective have various other advantaged
> features so this will be fairly primative by comparison, though it
> won''t require a message bus.  If you want something more advanced
> obviously try out those tools, this is covering a much smaller use case.
>
> This is something I am going to take a crack at this in the coming
> weeks.    This would be something pretty simple and lightweight, and
> could
> probably fix a lot of the use cases around making puppetrun (or
> staggering large sets of hosts) a lot easier.
>
> Features I''m thinking of:
>
> Requires no additonal ports, setup, or config files -- use existing
> puppet listening capability and puppetca, just a /usr/bin app
> Be able to query dashboard DB to run against tagged nodes or hosts
> that have certain data there (or in storeconfigs???)
> Be able to run against wildcarded nodes based on what certs are
> present on the puppetmaster (we know the hostnames)
> Be able to be used easily from an API perspective from any ruby application
> Be able to invoke ralsh remotely for querying things (and for debug,
> and one off tasks)
> Be able to run shell commands for things that are one offs (emergency
> security power down now)
>
> Example syntax:
>
> punc --hosts *.example.org --puppetize  # get new catalog and run
> punc --hosts *.example.org --ralsh "service name=foo
ensure=running"
> # perform an action through ralsh
> punc --hosts *.example.org --shell "/bin/emergency_script"   #
run a
> shell script... for the one-off cases
> punc --hosts foo.*.example.org --ralsh "service name=foo"
> --format=json  # query something with ralsh and generate a report
> punc --hosts foo.*.example.org --facter fact --format=json # similarly
> generate a facter report
> punc --tags webservers [...ditto...]
> punc --critiera "fact==foo" [..ditto...]
> punc --critiera "fact==foo" [some operation to run only if fact
> matches] [...ditto...]
>
> So for example we could choose to reboot all the servers that match a
> given fact, etc.
>
> It should also allow easier staged deployments and environment usage
> from apps that want to use the API.
>
> Additional ideas for stuff you would like to see?
>
> --Michael
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
>
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, 11 Feb 2010, Scott Smith wrote:
> >Of course there are change control procedures for getting the manifests
> >updated on the puppetmaster, but that''s not enough;
it''s also necessary
> >to run the puppet client only when specifically authorised.  For
> >example, the manifest update and a --noop mode client puppet run might
> >happen during working hours, but the --no-noop client puppet run might
> >happen during a maintenance window after hours.
> 
> So let me get this straight: You run --noop throughout the day,
> aggregate the changes that need to be made, and then have a EOD/EOW
> "change control" meeting to go over them and determine if you
need
> to run puppet without --noop ?
Not exactly, but that would probably be acceptable.  What actually
happens today is that puppetd is not run at all on the client unless an
authorised change is known to be ready for deployment; then puppetd is
run in --noop mode to verify that the changes it wants to make are as
expected; finallly puppetd is run in --no-noop mode.

--apb (Alan Barrett)

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 2/12/10 2:10 AM, Alan Barrett wrote:
> Not exactly, but that would probably be acceptable.  What actually
> happens today is that puppetd is not run at all on the client unless an
> authorised change is known to be ready for deployment; then puppetd is
> run in --noop mode to verify that the changes it wants to make are as
> expected; finallly puppetd is run in --no-noop mode.
>
What happens if a manual change is made outside of the scope of your scheduled
Puppet changes? For
example, someone tweaks /etc/syslog.conf. Puppet manages it but because you
don''t run regular noops,
how will you know it needs to be changed back? Do plan around the organization
(Ops or IT or w/e)
not knowing it was changed? Since you don''t have regular reports, how
do you find out?

-scott

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Inline....
>> - - I would have it be a stand alone application, not an add-on to the
>> primary puppet daemon. The reason for this is that many of us use
puppet
>> from cron instead of as a daemon for various reasons.
> 
> Interesting yeah, in that case the transport then becomes SSH though?
> Seems to be redundant in that configuration.
- -THV- Agreed, but SSH seems to be a bit more reliable than puppet as a
transport layer due to user error.  I''ve had cases where I implemented
something that worked fine on 20 nodes but then, on node 21, hung for
some reason and could not recover. Puppetrun, in this case, would be
completely useless.

This is a simplified version of reality, but it happens as not all
variables can be successfully tested, or known, in any given environment.
> 
> We''re attempting to provide a reason to not use cron :)
- -THV- Why? Puppet currently potentially uses quite a bit of memory on a
large manifest set and seems to work great out of cron with no adverse
side effects.
> 
>>
>> - - I do have to wonder how this is much different that cobbling
together
>> the results of puppetlast (to get the nodes that have checked in) with
>> some sort of parallel SSH solution (besides the fact that
you''re using
>> the puppet certs instead of SSH keys).
> 
> It would resemble that... but the idea here is we want to leverage
> those that have already checked in
> and the existing comm path.   I understand your use case though...  in
> which case, I think just using a parallel
> SSH (in that case) if you are ok with that, solves the problem... or
> continuing to use cron.
> 
> Here it is really about replacing puppetrun ... :)
- -THV- I definitely understand this, but I suppose that I''m
questioning
the existence of puppetrun altogether. It''s relatively useful, but
I''m
guessing that one of the main things that most people do with puppet is
to set up SSH. As such, you''re re-inventing the wheel with a
questionable gain for the effort.
>>>
>>> Requires no additonal ports, setup, or config files -- use existing
>>> puppet listening capability and puppetca, just a /usr/bin app
>>
>> - - I don''t particularly agree with this for the reasons
listed above.
> 
> Don''t follow, can you elaborate?
- -THV- No additional ports means an additional listening thread inside of
puppet itself. If we were using Ruby 1.9 with native threads, then this
wouldn''t be a problem but the way it is currently implemented I
don''t
have much faith that a hang in one part of Puppet wouldn''t cause the
listener to hang as well. I could be wrong :-P.

The reduction of setup and config (as it is now) makes sense.
>>
>> - - Are there example use cases of this? It sounds useful, but my
minimal
>> ralsh experimentation didn''t really prove to be greatly
fruitful.
> 
> I''m thinking more about querying for states of things than making
changes.
- -THV- Ah. This makes sense. I have had some problems with ralsh in the
past not picking up my complete manifest. I''ll try to come up with some
concrete examples if possible.
> 
>>
>>> Be able to run shell commands for things that are one offs
(emergency
>>> security power down now)
>>
>> - - This comes back to the common framework. I''m going to want
to be able
>> to do this regardless of whether or not puppet is actually running. I
>> may want to do this (or something like it) because puppet is hung for
>> some reason.
> 
> I understand, yeah.  In this case, distributed SSH may be your friend
> if you don''t want to run cron and don''t want puppet to
listen.
> Otherwise it kind of is cross-purposes with the idea of fixing
> puppetrun... and extra management of SSH keys is overhead when
> puppetca is there for that purpose.
> 
> I know what you''re saying though...
- -THV- I think that the point I''m so blatantly failing to make is that
puppet isn''t a tool for one-offs and isn''t advertised as a
command and
control infrastructure. I suppose that my *NIX roots are showing in that
I shy away from adding features to a tool that I can easily do with
something else.

As I mentioned in this response, one of the first things people are
probably going to do is to get a working SSH infrastructure going. Once
this is done, you have everything you need to do one-offs.

Thinking of some common use cases:

1) I screwed up resolv.conf
  - Puppet''s not going to work unless you pre-mangled /etc/hosts, SSH
it is.

2) I screwed up /etc/hosts
  - Puppet''s not going to work. Back to SSH.

3) DNS is broken
  - See #1

4) I hosed my PKI certs
  - See #2

In these four cases, you *have* to have a working SSH infrastructure
that spans your entire puppet base.

So, after all of this rambling, perhaps you may want to have the new
tool be transport pluggable.  Support both SSH and Puppet in the present
and perhaps grow to support things like mcollective under the same
abstracted administrative interface.

Thanks,

Trevor

- -- 
Trevor Vaughan
 Vice President, Onyx Point, Inc.
 email: tvaughan@onyxpoint.com
 phone: 410-541-ONYX (6699)

- -- This account not approved for unencrypted sensitive information --
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkt2cOMACgkQyWMIJmxwHpSKhACgh6QXNkaQMyh7+tdL33QTIqZm
66AAn0g379T1RQxA0vrJLGXtdoG8AWhj
=tAPZ
-----END PGP SIGNATURE-----

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Sat, Feb 13, 2010 at 4:29 AM, Trevor Vaughan <tvaughan@onyxpoint.com>
wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Inline....
>
>>> - - I would have it be a stand alone application, not an add-on to
the
>>> primary puppet daemon. The reason for this is that many of us use
puppet
>>> from cron instead of as a daemon for various reasons.
>>
>> Interesting yeah, in that case the transport then becomes SSH though?
>> Seems to be redundant in that configuration.
>
> - -THV- Agreed, but SSH seems to be a bit more reliable than puppet as a
> transport layer due to user error.  I''ve had cases where I
implemented
> something that worked fine on 20 nodes but then, on node 21, hung for
> some reason and could not recover. Puppetrun, in this case, would be
> completely useless.
>
> This is a simplified version of reality, but it happens as not all
> variables can be successfully tested, or known, in any given environment.
>
>>
>> We''re attempting to provide a reason to not use cron :)
>
> - -THV- Why? Puppet currently potentially uses quite a bit of memory on a
> large manifest set and seems to work great out of cron with no adverse
> side effects.
We need to work on that.   I don''t want to constrain our conceptual
ideas too much
by current limitations.

Anyway, if you are able to stagger node rollouts according to custom
workflow you gain two
capabilities:

one -- you can not role out your infrastructure all at once, for
safety reasons (even if you
do have a good ''stage'' infrastructure).   For instance, update
just
some of these hosts,
and then update the rest.

two -- you have a nicer scaling puppetmaster as you have more control
on when they check
in and put load on the server, rather than having the spike happen at
intervals (and without
you having to manage these intervals with cron).    While we''re
working to speed up puppetmaster
and also make it scale out easier, just being able to stage rollouts
centrally combined with that
can offer some nice performance opportunity.

Finally three (amongst our capabilities </spanish inquisition
sketch>), you don''t have to manage
your crontab for puppet in your image/kickstart/etc, because it can
all be centrally managed and not
up to the client.

Is that going to work for all use cases?   Probably not, but if you
have something you like, you can definitely
continue to use it.   I''d like to have better control over when things
run from a central server perspective myself,
so I could do things faster at times, have more fined grained control, etc.


>
> - -THV- I definitely understand this, but I suppose that I''m
questioning
> the existence of puppetrun altogether. It''s relatively useful, but
I''m
> guessing that one of the main things that most people do with puppet is
> to set up SSH. As such, you''re re-inventing the wheel with a
> questionable gain for the effort.
Perhaps... I agree with your pluggable transport things below.

It also seems that (at least from what I gather) that not everyone
/does/ distribute
SSH keys.   (They might rely on serial consoles, for instance).
>
> - -THV- No additional ports means an additional listening thread inside of
> puppet itself. If we were using Ruby 1.9 with native threads, then this
> wouldn''t be a problem but the way it is currently implemented I
don''t
> have much faith that a hang in one part of Puppet wouldn''t cause
the
> listener to hang as well. I could be wrong :-P.
The real idea is not have hangs in that part of Puppet, isn''t it?   :)

I''ll keep that in mind as I''m doing it.    I''m sure
there will be some
corner cases here and there that we''ll have
to straighten out.
>
>>>
>>> - - This comes back to the common framework. I''m going to
want to be able
>>> to do this regardless of whether or not puppet is actually running.
I
>>> may want to do this (or something like it) because puppet is hung
for
>>> some reason.

*nod*

That would be just one of many small features.   If you wanted to use
SSH or serial consoles for that, you still could, of course.

For those that don''t have that, having a distributed system out of the
box that can do that is pretty darn useful.... and we can
get that more or less for free.

Best practices, as you say, should probably imply having a backup plan.

> So, after all of this rambling, perhaps you may want to have the new
> tool be transport pluggable.  Support both SSH and Puppet in the present
> and perhaps grow to support things like mcollective under the same
> abstracted administrative interface.
Agreed.   Might not get that in initially, but we should keep it in
mind and do the very first one as a plugin, to make room
for the rest.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

A common object model for orchestrating deployment is the subject of
my MS thesis.  Last weekend I met Dan at SCALE, who suggested I look
up this thread.  Going to your earlier example:
> punc --hosts *.example.org --shell "/bin/emergency_script"
How about extending this to use puppet''s RAL or yaml:

punc --yamlfile maintenance.yaml --process emergency_script

Where yamlfile could resemble:

<snip>
processes:
  emergency_script:
    execute:
      - /bin/sh /bin/emergency_script
    locations:
      hosts:
        - singlehost.example.org
</snip>

Using YAML anchors could be useful for specifying workflows:

<snip>
processes:
  install_and_sign:
    execute:
      - *install
      - *sign
      - *start
  install: &install
    execute:
      - yum install puppet
      - puppetd --waitforcert 60
    locations:
      hosts:
        - $newhost
  sign: &sign
    execute:
      - puppetca --sign $newhost
    locations:
      hosts:
        - puppetmaster
  start: &start
    execute:
      - /etc/init.d/puppet start
    locations:
      hosts:
        - $newhost
</snip>
This example uses variables like $newhost which can be improved upon,
but for now you could support it from the command-line like:
punc --yamlfile sign.yaml --process install_and_sign --set
newhost=newhost.example.com

Setting variables or separate yaml can separate a process from where
it runs (staging vs. production).  But the main idea is to support
classes with more specific behavior, like so:

<snip>
locations:
  --- !ruby/object:YAML::puppetclasses
    name: example.org
  --- !ruby/object:YAML::ldap
    name: example
  --- !ruby/object:YAML::dsh
    name: example
</snip>

To produce non-ruby-specific YAML can simplify the last snip that
to ...
<snip>
locations:
  --- !puppetclasses
    name: example.org
  --- !ldap
    name: example
  --- !dsh
    name: example
</snip>

... though I don''t know if there''s a way (http://www.yaml.org/
YAML_for_ruby.html#objects) to bind YAML <==> Ruby objects, except by
hand, like you can with other implementations:
http://code.google.com/p/snakeyaml/wiki/Documentation#Constructors,_representers,_resolvers
http://pyyaml.org/wiki/PyYAMLDocumentation#Constructorsrepresentersresolvers

In a world of competing deployment tools, being language-agnostic
could unify deployment processes.  Java people resist writing python,
etc so there are so many competing tools.  I''m trying to distill this
to the simplest thing that can possibly be useful, so:
-  I like YAML because it''s human-readable and all languages support
it.  I like Puppet''s RAL, too.
-  As described, it was simple enough to be implemented in about 30
lines in python implementation with fabric (fabfile.org), so hopefully
that remains so with puppetrun/vlad/capistrano/ace/func.
-  I''ve left out ways you might express a transport (ssh vs puppet vs
web services)
-  Brevity limits how to incorporate concepts inspired by BPEL like
decisions (to support if/then), fork/join (execute tasks in parallel),
tasks (prompt user input), and state (variables).

I hope these ideas contribute to a useful orchestration model for
puppet.  I want as much feedback as I can get, my direct email is dave
(AT) peoplemerge (DOT) com.

Dave Thomas

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.