Puppet users - Jun 2011 - howto trigger action on another client

Hi,

I want to automate installation of GridEngine with puppet. GE comes with some 
prepared script for automatic installation.
Unfortunately this is to be run on the GE master to install the executions host 
(=clients).  (This then will run some commands on the exec host via ssh)
So from puppet''s perspective if an agent on an execution host notices
that GE is
not installed but should it has to tell the agent on the GE master to run the 
install script.
Is there an easy (puppet) way to do it?
My idea is to have the exec host write its name into a file
"hosts_to_install"
on a common NFS and have the master check that file. But maybe there are better 
ways.
Does anybody here has some other GE related recipes?

regards, Andreas

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jun 24, 3:14 am, Andreas Kuntzagk <andreas.kuntz...@mdc-berlin.de>
wrote:
> Hi,
>
> I want to automate installation of GridEngine with puppet. GE comes with
some
> prepared script for automatic installation.
> Unfortunately this is to be run on the GE master to install the executions
host
> (=clients).  (This then will run some commands on the exec host via ssh)
> So from puppet''s perspective if an agent on an execution host
notices that GE is
> not installed but should it has to tell the agent on the GE master to run
the
> install script.
> Is there an easy (puppet) way to do it?
> My idea is to have the exec host write its name into a file
"hosts_to_install"
> on a common NFS and have the master check that file. But maybe there are
better
> ways.
> Does anybody here has some other GE related recipes?

Exported resources are the Puppet means for one node to provide
resources for another.  In this case, the execution host could export
an Exec resource for the master to collect and apply.  The tricky part
might be to come up with an ''unless'' parameter for it to
ensure that
the master does not attempt to install the execution host twice.  This
has the advantage that it does not rely on a shared file or indeed on
NFS at all.  It does entail a delay between when the execution host''s
configuration is applied and when GE is actually installed, however.

Your idea has merit too.  It is certainly possible to use Puppet on
the execution hosts to ensure (or not) that some accessible file has a
specific line.  There is no native resource for this narrow purpose
(augeas notwithstanding), but you can find recipes on the PuppetLabs
wiki.  If you have no other need for exported resources then it may be
to your advantage to avoid them.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

jcbollinger wrote:
> 
> On Jun 24, 3:14 am, Andreas Kuntzagk <andreas.kuntz...@mdc-berlin.de>
> wrote:
>> Hi,
>>
>> I want to automate installation of GridEngine with puppet. GE comes
with some
>> prepared script for automatic installation.
>> Unfortunately this is to be run on the GE master to install the
executions host
>> (=clients).  (This then will run some commands on the exec host via
ssh)
>> So from puppet''s perspective if an agent on an execution host
notices that GE is
>> not installed but should it has to tell the agent on the GE master to
run the
>> install script.
>> Is there an easy (puppet) way to do it?
>> My idea is to have the exec host write its name into a file
"hosts_to_install"
>> on a common NFS and have the master check that file. But maybe there
are better
>> ways.
>> Does anybody here has some other GE related recipes?
> 
> 
> Exported resources are the Puppet means for one node to provide
> resources for another.  In this case, the execution host could export
> an Exec resource for the master to collect and apply.  
Hmm, in the (very short) time I''ve been working with puppet I have not
stumbled
across exported resources. Any pointer where I should start reading?
> The tricky part
> might be to come up with an ''unless'' parameter for it to
ensure that
> the master does not attempt to install the execution host twice.  This
> has the advantage that it does not rely on a shared file or indeed on
> NFS at all.  It does entail a delay between when the execution
host''s
> configuration is applied and when GE is actually installed, however.
I can live with that delay. The solution I proposed will probably have the same 
delay - between the puppet run on the exec host and the puppet run on the
master.
> Your idea has merit too.  It is certainly possible to use Puppet on
> the execution hosts to ensure (or not) that some accessible file has a
> specific line.  There is no native resource for this narrow purpose
> (augeas notwithstanding), but you can find recipes on the PuppetLabs
> wiki.  If you have no other need for exported resources then it may be
> to your advantage to avoid them.
Yeah, I''m aware of "addifnosuchline" exec''s and the
like.

Thanks, Andreas

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 06/24/2011 03:39 PM, Andreas Kuntzagk wrote:
>> Exported resources are the Puppet means for one node to provide
>> resources for another. In this case, the execution host could export
>> an Exec resource for the master to collect and apply.
>
> Hmm, in the (very short) time I''ve been working with puppet I have
not
> stumbled across exported resources. Any pointer where I should start
> reading?
Exported resources are great fun. :)

http://docs.puppetlabs.com/guides/exported_resources.html
http://projects.puppetlabs.com/projects/1/wiki/Using_Stored_Configuration


-- 
Daniel Maher
« makin'' plans now to live on Mars ''cuz I got Earth on lock. »

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Daniel Maher wrote:
> On 06/24/2011 03:39 PM, Andreas Kuntzagk wrote:
> 
>>> Exported resources are the Puppet means for one node to provide
>>> resources for another. In this case, the execution host could
export
>>> an Exec resource for the master to collect and apply.
>>
>> Hmm, in the (very short) time I''ve been working with puppet I
have not
>> stumbled across exported resources. Any pointer where I should start
>> reading?
> 
> Exported resources are great fun. :)
> 
> http://docs.puppetlabs.com/guides/exported_resources.html
> http://projects.puppetlabs.com/projects/1/wiki/Using_Stored_Configuration
According to these exported resources are somewhat similar to virtual resources.
Problem is until now I cannot wrap my brain arount virtual resources so I
don''t
understand the exported resources examples.

regards, Andreas


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Another mechanism. You can use mcollective and this resource to
trigger mco calls inside puppet:

https://github.com/ripienaar/puppet-mcollective

The resource acts like an exec ... and can be refreshed and notified.
For example:

file {"/tmp/foo1":
#  content => "foo",
  content => inline_template("<%= Time.now %>"),
  notify => Mcollective["test1"],
}
mcollective {"test1":
  agent => "service",
  action => "restart",
  arguments => {
    "service" => "rsyslog",
  },
  identity_filter => ["puppet1"],
  disctimeout => 3,
  timeout => 10,
  refreshonly => true,
}

Just make an mcollective plugin for adding new clients on your master
and trigger it this way.

ken.

On Fri, Jun 24, 2011 at 3:18 PM, Andreas Kuntzagk
<andreas.kuntzagk@mdc-berlin.de> wrote:
> Daniel Maher wrote:
>>
>> On 06/24/2011 03:39 PM, Andreas Kuntzagk wrote:
>>
>>>> Exported resources are the Puppet means for one node to provide
>>>> resources for another. In this case, the execution host could
export
>>>> an Exec resource for the master to collect and apply.
>>>
>>> Hmm, in the (very short) time I''ve been working with
puppet I have not
>>> stumbled across exported resources. Any pointer where I should
start
>>> reading?
>>
>> Exported resources are great fun. :)
>>
>> http://docs.puppetlabs.com/guides/exported_resources.html
>>
http://projects.puppetlabs.com/projects/1/wiki/Using_Stored_Configuration
>
> According to these exported resources are somewhat similar to virtual
> resources. Problem is until now I cannot wrap my brain arount virtual
> resources so I don''t understand the exported resources examples.
>
> regards, Andreas
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jun 24, 9:18 am, Andreas Kuntzagk <andreas.kuntz...@mdc-berlin.de>
wrote:
> Daniel Maher wrote:
> > On 06/24/2011 03:39 PM, Andreas Kuntzagk wrote:
>
> >>> Exported resources are the Puppet means for one node to
provide
> >>> resources for another. In this case, the execution host could
export
> >>> an Exec resource for the master to collect and apply.
>
> >> Hmm, in the (very short) time I''ve been working with
puppet I have not
> >> stumbled across exported resources. Any pointer where I should
start
> >> reading?
>
> > Exported resources are great fun. :)
>
> >http://docs.puppetlabs.com/guides/exported_resources.html
>
>http://projects.puppetlabs.com/projects/1/wiki/Using_Stored_Configura...
>
> According to these exported resources are somewhat similar to virtual
resources.
> Problem is until now I cannot wrap my brain arount virtual resources so I
don''t
> understand the exported resources examples.

Exported resources are an extended version of virtual resources.  It
took me a while to catch on to virtual resources, too, but once I got
it I realized how useful they are.  I''ll attempt to explain:

Normal resource declarations have two key aspects.  First, they
describe a resource, and second, by their presence in a manifest they
indicate that that resource should be managed.  For example,

user { "john":
  ensure => "absent"
}

describes a User resource whose username is "john", and if it is
present in a node''s manifest then that tells Puppet that the matching
user should be managed on that node -- in this case, User john must be
ensured absent from the node.

Virtual resources separate those key aspects.  They allow a resource
to be declared (described) without specifying whether that resource
should be managed.  For example,

@user { "john":
  ensure => "absent"
}

describes a User resource whose username is "john", but by itself it
does not direct the Puppet agent to do anything about that user.
Elsewhere in the manifest, however, that user can be "realized" to
indicate to Puppet that yes, User ["john"] should indeed be managed
(in this example, poor john would then again be ensured absent).
Virtual resources that are not realized for a particular node have no
effect on that node.

There are at least two ways in which this is useful:

1) If there are resources that must be managed only on some nodes,
then it can be clean and convenient to declare them virtually, all in
one place, for all nodes, and then realize just the needed ones on
each node.  This is a good alternative to bracketing each resource
with a complicated conditional statement.

2) Although resources, including virtual ones, can only be *declared*
once, virtual resources can be *realized* any number of times.  For
instance, if you had several different classes that wanted to drop
files in /etc/cron.d/, then they might all want to ensure that that
directory is present.  They can''t all declare it, else the
declarations would collide.  One possibility would be to declare it
virtually, and let all the classes realize it.  (Another would be to
declare it concretely in a separate class, and have the other classes
each include that class.)


Exported resources are much like virtual ones.  The main difference is
that exported resources can be collected by any node, not just the
node that declares them.  For example, suppose you want all your nodes
to be listed in each other''s /etc/hosts file.  Each node knows its own
name and IP address (suppose for simplicity that it has only one), but
it doesn''t a priori know the others''.  You can do this:

# Export a Host resource describing this node:
@@host { "$fqdn":
  ip => "$ipaddress",
  host_aliases => "$hostname",
  ensure => "present"
}

# Collect all nodes'' exported Host resources
# (including this node''s)
Host <<| |>>


One of the major drawbacks of exported resources is that they require
persistent storage.  Puppet uses a database for that, which is a bit
heavy.  That''s one reason why exported resources may not be worth the
trouble if this is your only use for them.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Fri, Jun 24, 2011 at 11:14 AM, Andreas Kuntzagk
<andreas.kuntzagk@mdc-berlin.de> wrote:
> Hi,
>
> I want to automate installation of GridEngine with puppet. GE comes with
> some prepared script for automatic installation.
> Unfortunately this is to be run on the GE master to install the executions
> host (=clients).  (This then will run some commands on the exec host via
> ssh)
> So from puppet''s perspective if an agent on an execution host
notices that
> GE is not installed but should it has to tell the agent on the GE master to
> run the install script.
> Is there an easy (puppet) way to do it?
> My idea is to have the exec host write its name into a file
> "hosts_to_install" on a common NFS and have the master check that
file. But
> maybe there are better ways.
> Does anybody here has some other GE related recipes?
>
> regards, Andreas
>
and another idea, if you are using foreman[1], you can collect all
hosts which are member of the GE group.
You could also ask for their facts, so you could build a meaningful GE
configuration.

see this puppet function [2]

Ohad

[1] - http://theforeman.org
[2] -
https://github.com/ohadlevy/puppet-foreman/blob/master/foreman/lib/puppet/parser/functions/foreman.rb
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Thanks for these infos. I think I got at least a vague idea how this works and 
can already use it for some simple cases.
One more question:
If I define checks for the "exported exec" (like "creates"
or "onlyif" will
these run on the exporting agent or the executing agent? (my guess is the second
but want to make sure)
If this is the case is there a way to run the "exec" depending on some
checks on
the exporting agent?

regards, Andreas

jcbollinger wrote:
> 
> On Jun 24, 9:18 am, Andreas Kuntzagk <andreas.kuntz...@mdc-berlin.de>
> wrote:
>> Daniel Maher wrote:
>>> On 06/24/2011 03:39 PM, Andreas Kuntzagk wrote:
>>>>> Exported resources are the Puppet means for one node to
provide
>>>>> resources for another. In this case, the execution host
could export
>>>>> an Exec resource for the master to collect and apply.
>>>> Hmm, in the (very short) time I''ve been working with
puppet I have not
>>>> stumbled across exported resources. Any pointer where I should
start
>>>> reading?
>>> Exported resources are great fun. :)
>>> http://docs.puppetlabs.com/guides/exported_resources.html
>>>
http://projects.puppetlabs.com/projects/1/wiki/Using_Stored_Configura...
>> According to these exported resources are somewhat similar to virtual
resources.
>> Problem is until now I cannot wrap my brain arount virtual resources so
I don''t
>> understand the exported resources examples.
> 
> 
> Exported resources are an extended version of virtual resources.  It
> took me a while to catch on to virtual resources, too, but once I got
> it I realized how useful they are.  I''ll attempt to explain:
> 
> Normal resource declarations have two key aspects.  First, they
> describe a resource, and second, by their presence in a manifest they
> indicate that that resource should be managed.  For example,
> 
> user { "john":
>   ensure => "absent"
> }
> 
> describes a User resource whose username is "john", and if it is
> present in a node''s manifest then that tells Puppet that the
matching
> user should be managed on that node -- in this case, User john must be
> ensured absent from the node.
> 
> Virtual resources separate those key aspects.  They allow a resource
> to be declared (described) without specifying whether that resource
> should be managed.  For example,
> 
> @user { "john":
>   ensure => "absent"
> }
> 
> describes a User resource whose username is "john", but by itself
it
> does not direct the Puppet agent to do anything about that user.
> Elsewhere in the manifest, however, that user can be "realized"
to
> indicate to Puppet that yes, User ["john"] should indeed be
managed
> (in this example, poor john would then again be ensured absent).
> Virtual resources that are not realized for a particular node have no
> effect on that node.
> 
> There are at least two ways in which this is useful:
> 
> 1) If there are resources that must be managed only on some nodes,
> then it can be clean and convenient to declare them virtually, all in
> one place, for all nodes, and then realize just the needed ones on
> each node.  This is a good alternative to bracketing each resource
> with a complicated conditional statement.
> 
> 2) Although resources, including virtual ones, can only be *declared*
> once, virtual resources can be *realized* any number of times.  For
> instance, if you had several different classes that wanted to drop
> files in /etc/cron.d/, then they might all want to ensure that that
> directory is present.  They can''t all declare it, else the
> declarations would collide.  One possibility would be to declare it
> virtually, and let all the classes realize it.  (Another would be to
> declare it concretely in a separate class, and have the other classes
> each include that class.)
> 
> 
> Exported resources are much like virtual ones.  The main difference is
> that exported resources can be collected by any node, not just the
> node that declares them.  For example, suppose you want all your nodes
> to be listed in each other''s /etc/hosts file.  Each node knows its
own
> name and IP address (suppose for simplicity that it has only one), but
> it doesn''t a priori know the others''.  You can do this:
> 
> # Export a Host resource describing this node:
> @@host { "$fqdn":
>   ip => "$ipaddress",
>   host_aliases => "$hostname",
>   ensure => "present"
> }
> 
> # Collect all nodes'' exported Host resources
> # (including this node''s)
> Host <<| |>>
> 
> 
> One of the major drawbacks of exported resources is that they require
> persistent storage.  Puppet uses a database for that, which is a bit
> heavy.  That''s one reason why exported resources may not be worth
the
> trouble if this is your only use for them.
> 
> 
> John
> 
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jun 27, 9:37 am, Andreas Kuntzagk <andreas.kuntz...@mdc-berlin.de>
wrote:
> Thanks for these infos. I think I got at least a vague idea how this works
and
> can already use it for some simple cases.
> One more question:
> If I define checks for the "exported exec" (like
"creates" or "onlyif" will
> these run on the exporting agent or the executing agent? (my guess is the
second
> but want to make sure)

The ''unless'' and / or ''onlyif'' command will
run on the same nodes that
the main command runs on.  Likewise the criterion implied by the
''creates'' parameter is evaluated on the same nodes that the
main
command runs on.  In all cases, that means these apply to the node(s)
that *collect* the Exec resources.

> If this is the case is there a way to run the "exec" depending on
some checks on
> the exporting agent?

Certainly.  Define one or more custom facts embodying the results of
your tests (it''s pretty easy).  Distribute these via Puppet''s
pluginsync mechanism.  Use conditionals based on the fact values to
control whether the Exec''s are exported, and / or to vary their
parameters (e.g. setting unless => ''/bin/true'' is one way
for the
exporter to disable execution).

In general, facts, including custom facts, are the Puppet means for
providing nodes'' state details to the puppetmaster.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi
> On Jun 27, 9:37 am, Andreas Kuntzagk <andreas.kuntz...@mdc-berlin.de>
> wrote:
>> Thanks for these infos. I think I got at least a vague idea how this
works and
>> can already use it for some simple cases.
>> One more question:
>> If I define checks for the "exported exec" (like
"creates" or "onlyif" will
>> these run on the exporting agent or the executing agent? (my guess is
the second
>> but want to make sure)
> 
> 
> The ''unless'' and / or ''onlyif'' command
will run on the same nodes that
> the main command runs on.  Likewise the criterion implied by the
> ''creates'' parameter is evaluated on the same nodes that
the main
> command runs on.  In all cases, that means these apply to the node(s)
> that *collect* the Exec resources.
> 
> 
>> If this is the case is there a way to run the "exec"
depending on some checks on
>> the exporting agent?
> 
> 
> Certainly.  Define one or more custom facts embodying the results of
> your tests (it''s pretty easy).  Distribute these via
Puppet''s
> pluginsync mechanism.  Use conditionals based on the fact values to
> control whether the Exec''s are exported, and / or to vary their
> parameters (e.g. setting unless => ''/bin/true'' is one
way for the
> exporter to disable execution).
> 
> In general, facts, including custom facts, are the Puppet means for
> providing nodes'' state details to the puppetmaster.
It''s probably the "clean puppet way(tm)" to do it but to
write custom facts you
need to learn some Ruby. Currently I''d like to avoid learning yet
another
programming language. There are other ways the GridEngine master can test for 
the status of the client.

regards, Andreas

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jun 28, 2:36 am, Andreas Kuntzagk <andreas.kuntz...@mdc-berlin.de>
wrote:

[... I wrote:]
> > In general, facts, including custom facts, are the Puppet means for
> > providing nodes'' state details to the puppetmaster.
>
> It''s probably the "clean puppet way(tm)" to do it but to
write custom facts you
> need to learn some Ruby. Currently I''d like to avoid learning yet
another
> programming language. There are other ways the GridEngine master can test
for
> the status of the client.

It may indeed be that Puppet is not the best available mechanism for
enrolling new execution nodes into GE.  Puppet excels at managing
(semi-)permanent state details that individually are confined to a
single node.  It can certainly be applied to more general scenarios,
but as you move away from its core focus, Puppet begins to lose some
of its appeal.  In particular, if you find yourself doing complex
things with Execs then you are probably running against the grain.

As for custom facts, however, whether you would need to learn any Ruby
depends somewhat on the complexity of the fact you need.  If the fact
value can be represented as the standard output of an OS command then
you can just plug it in to the example at
http://docs.puppetlabs.com/guides/custom_facts.html#an-example.  In
that case there is no need to understand the bit of Ruby window
dressing that interfaces the command with Facter.  Indeed, that window
dressing is much more Puppet-specific than Ruby-general anyway.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.