Puppet users - Oct 2011 - Custom providers - instalation and usage of an executable

Hello all,

I am facing a problem while dealing with a custom provider and its
requirements. I have included a custom provider (with its custom
types) into my puppet deployment that is expected to deal with all
tasks related to glassfish. This provider is able to create domains,
deploy applications and other features.

This provider also requires a glassfish executable named
''asadmin''
that is available after glassfish is installed. If the executable is
available at the machine where agent is executed all works fine but if
it is not available a message like ''[default] ←[0;37mdebug:
Puppet::Type::Domain::ProviderAsadmin: file asadmin does not exist←[0m
Could not find a default provider for domain'' is returned.

All that behaviour is normal since there are not alternative providers
for my resources and then the provider is not appliable but the
problem appears when I want to install the application AND THEN work
with the provider. In the theoretical scenario, puppet should install
glassfish, deal with path and finally do some work with the provider
but the real scenario is not that. The real scenario is that puppet
does a prefetch of all resources that deal with providers and tries to
load them. Since ''asadmin'' is not yet available, the execution
fails.

Do anyone knows any way to let puppet skip that (or delay it until
glassfish has been installed) and finish successfully? Would stages do
a trick here? The only workaround that I have found has been perform
the installation in a first provision and then, at the second
provision, deal with resources.

Scenario:
- Install Glassfish
- Configure Glassfish
- Deal with resources (provider)

Sample code:

node default {
  include glassfish

  Domain {
    user         => "${glassfish::user}",
    asadminuser  => "${glassfish::admin_user}",
    passwordfile => "${glassfish::home}/.aspass",
    portbase     => "${glassfish::domain_port}"
  }

  domain { $glassfish::domain :
    ensure     => present,
	require    => File[''create-aspass-file''],
	notify     => Service[''glassfish''],
  }
}

class glassfish {
  include glassfish::download, glassfish::install, glassfish::config

  #Dependencies
  Class[''glassfish::download''] ->
Class[''glassfish::install''] ->
Class[''glassfish::config'']
}

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, Oct 27, 2011 at 08:00:05AM -0700, David Campos
wrote:
> Hello all,
> 
> I am facing a problem while dealing with a custom provider and its
> requirements. I have included a custom provider (with its custom
> types) into my puppet deployment that is expected to deal with all
> tasks related to glassfish. This provider is able to create domains,
> deploy applications and other features.
> 
> This provider also requires a glassfish executable named
''asadmin''
> that is available after glassfish is installed. If the executable is
> available at the machine where agent is executed all works fine but if
> it is not available a message like ''[default] ←[0;37mdebug:
> Puppet::Type::Domain::ProviderAsadmin: file asadmin does not exist←[0m
> Could not find a default provider for domain'' is returned.
> 
> All that behaviour is normal since there are not alternative providers
> for my resources and then the provider is not appliable but the
> problem appears when I want to install the application AND THEN work
> with the provider. In the theoretical scenario, puppet should install
> glassfish, deal with path and finally do some work with the provider
> but the real scenario is not that. The real scenario is that puppet
> does a prefetch of all resources that deal with providers and tries to
> load them. Since ''asadmin'' is not yet available, the
execution fails.
> 
> Do anyone knows any way to let puppet skip that (or delay it until
> glassfish has been installed) and finish successfully? Would stages do
> a trick here? The only workaround that I have found has been perform
> the installation in a first provision and then, at the second
> provision, deal with resources.
> 
> Scenario:
> - Install Glassfish
> - Configure Glassfish
> - Deal with resources (provider)
> 
Stages does not change a thing. At first I would suggest you watch
https://projects.puppetlabs.com/issues/6907

I can see three possibilities:

- write a dummy provider that doesn''t do anything but also
doesn''t need anything.
  During the first puppet run puppet will pick the dummy provider. When
  you run puppet the next time your glassfish binary is already
  installed and puppet can choose between two providers. IIRC puppet
  tries to pick the one that is the most specific and that is the one
  with the most confines (so in your case it will probably work as
  desired)
- use optional_commands :foo => ''/bin/foo''  instead of
commands :foo =>
  ''/bin/foo''. This way the provider is still suitable even if
your binary is
  not installed.  But prefetching will fail on the first run (but puppet
  should apply the catalog so the second run should work ok)
- use a custom fact that tells you if the binary of your provider is
  installed. Wrap each resource in your manifest in an if clause. Now
  you also need two puppet runs but the first run does not throw an
  error.

-Stefan

On Oct 27, 2:35 pm, Stefan Schulte <stefan.schu...@taunusstein.net>
wrote:
> On Thu, Oct 27, 2011 at 08:00:05AM -0700, David Campos wrote:
> > Hello all,
>
> > I am facing a problem while dealing with a custom provider and its
> > requirements. I have included a custom provider (with its custom
> > types) into my puppet deployment that is expected to deal with all
> > tasks related to glassfish. This provider is able to create domains,
> > deploy applications and other features.
>
> > This provider also requires a glassfish executable named
''asadmin''
> > that is available after glassfish is installed. If the executable is
> > available at the machine where agent is executed all works fine but if
> > it is not available a message like ''[default] ←[0;37mdebug:
> > Puppet::Type::Domain::ProviderAsadmin: file asadmin does not exist←[0m
> > Could not find a default provider for domain'' is returned.
>
> > All that behaviour is normal since there are not alternative providers
> > for my resources and then the provider is not appliable but the
> > problem appears when I want to install the application AND THEN work
> > with the provider. In the theoretical scenario, puppet should install
> > glassfish, deal with path and finally do some work with the provider
> > but the real scenario is not that. The real scenario is that puppet
> > does a prefetch of all resources that deal with providers and tries to
> > load them. Since ''asadmin'' is not yet available, the
execution fails.
>
> > Do anyone knows any way to let puppet skip that (or delay it until
> > glassfish has been installed) and finish successfully? Would stages do
> > a trick here? The only workaround that I have found has been perform
> > the installation in a first provision and then, at the second
> > provision, deal with resources.
>
> > Scenario:
> > - Install Glassfish
> > - Configure Glassfish
> > - Deal with resources (provider)
>
> Stages does not change a thing. At first I would suggest you
watchhttps://projects.puppetlabs.com/issues/6907
>
> I can see three possibilities:
>
> - write a dummy provider that doesn''t do anything but also
doesn''t need anything.
>   During the first puppet run puppet will pick the dummy provider. When
>   you run puppet the next time your glassfish binary is already
>   installed and puppet can choose between two providers. IIRC puppet
>   tries to pick the one that is the most specific and that is the one
>   with the most confines (so in your case it will probably work as
>   desired)
> - use optional_commands :foo => ''/bin/foo''  instead of
commands :foo =>
>   ''/bin/foo''. This way the provider is still suitable
even if your binary is
>   not installed.  But prefetching will fail on the first run (but puppet
>   should apply the catalog so the second run should work ok)
> - use a custom fact that tells you if the binary of your provider is
>   installed. Wrap each resource in your manifest in an if clause. Now
>   you also need two puppet runs but the first run does not throw an
>   error.

I think Stefan has covered the field pretty well.  Personally, I don''t
much like the dummy provider, though: it will prevent the error
message, but because it doesn''t actually do anything, it lies to the
Puppet agent whenever it claims to have synchronized a resource.  This
may cause dependent resources to fail, and it results in two runs
being required to achieve the target state, if that''s an issue.

The optional_commands approach is pretty smooth, and the prefetch
problem is only relevant if it''s possible for there to be existing
resources when the needed binary is not already present.  If you can
use some alternative mechanism to prefetch resources then there
doesn''t have to be a prefetching problem at all.  If you make the
provider a do-nothing in the event that asadmin is missing then this
option becomes a superset of the dummy provider option; that would
allow you to avoid multiple providers.  (I still don''t like the dummy
option even when implemented this way, however: better to fail a
resource if asadmin is not available when the time comes to sync it.)

The custom fact approach is pretty easy to add on top of your existing
provider and manifests.  Also, if you arrange your resources suitably
then you should be able simply to conditionally include an entire
class, instead of conditionally declaring each of several separate
resources.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I tested the optional_command solution with success. The virtual
machine could be provisioned but the first time, any reference to the
asadmin provider failed even though the binary was present. I presume
that once the prefetch of providers has been done and failed, then
that type and provider become unusable at that provisioning.

I am quite sure that the response to my next question will be
''no'' but
I''ll try anyway... It is possible to force a new binary check at each
provider or type execution? The first check would fail but
optional_command would handle it and at the real execution stage, the
check would be successful and the command would be executed.

Otherwise, two runs will be necessary at first provision :)

On 28 oct, 14:47, jcbollinger <John.Bollin...@stJude.org>
wrote:
> On Oct 27, 2:35 pm, Stefan Schulte <stefan.schu...@taunusstein.net>
> wrote:
>
> > On Thu, Oct 27, 2011 at 08:00:05AM -0700, David Campos wrote:
> > > Hello all,
>
> > > I am facing a problem while dealing with acustomprovider and its
> > > requirements. I have included acustomprovider (with itscustom
> > > types) into my puppet deployment that is expected to deal with
all
> > > tasks related to glassfish. This provider is able to create
domains,
> > > deploy applications and other features.
>
> > > This provider also requires a glassfish executable named
''asadmin''
> > > that is available after glassfish is installed. If the executable
is
> > > available at the machine where agent is executed all works fine
but if
> > > it is not available a message like ''[default]
←[0;37mdebug:
> > > Puppet::Type::Domain::ProviderAsadmin: file asadmin does not
exist←[0m
> > > Could not find a default provider for domain'' is
returned.
>
> > > All that behaviour is normal since there are not alternative
providers
> > > for my resources and then the provider is not appliable but the
> > > problem appears when I want to install the application AND THEN
work
> > > with the provider. In the theoretical scenario, puppet should
install
> > > glassfish, deal with path and finally do some work with the
provider
> > > but the real scenario is not that. The real scenario is that
puppet
> > > does a prefetch of all resources that deal with providers and
tries to
> > > load them. Since ''asadmin'' is not yet
available, the execution fails.
>
> > > Do anyone knows any way to let puppet skip that (or delay it
until
> > > glassfish has been installed) and finish successfully? Would
stages do
> > > a trick here? The only workaround that I have found has been
perform
> > > the installation in a first provision and then, at the second
> > > provision, deal with resources.
>
> > > Scenario:
> > > - Install Glassfish
> > > - Configure Glassfish
> > > - Deal with resources (provider)
>
> > Stages does not change a thing. At first I would suggest you
watchhttps://projects.puppetlabs.com/issues/6907
>
> > I can see three possibilities:
>
> > - write a dummy provider that doesn''t do anything but also
doesn''t need anything.
> >   During the first puppet run puppet will pick the dummy provider.
When
> >   you run puppet the next time your glassfish binary is already
> >   installed and puppet can choose between two providers. IIRC puppet
> >   tries to pick the one that is the most specific and that is the one
> >   with the most confines (so in your case it will probably work as
> >   desired)
> > - use optional_commands :foo => ''/bin/foo''
 instead of commands :foo =>
> >   ''/bin/foo''. This way the provider is still
suitable even if your binary is
> >   not installed.  But prefetching will fail on the first run (but
puppet
> >   should apply the catalog so the second run should work ok)
> > - use acustomfact that tells you if the binary of your provider is
> >   installed. Wrap each resource in your manifest in an if clause. Now
> >   you also need two puppet runs but the first run does not throw an
> >   error.
>
> I think Stefan has covered the field pretty well.  Personally, I
don''t
> much like the dummy provider, though: it will prevent the error
> message, but because it doesn''t actually do anything, it lies to
the
> Puppet agent whenever it claims to have synchronized a resource.  This
> may cause dependent resources to fail, and it results in two runs
> being required to achieve the target state, if that''s an issue.
>
> The optional_commands approach is pretty smooth, and the prefetch
> problem is only relevant if it''s possible for there to be existing
> resources when the needed binary is not already present.  If you can
> use some alternative mechanism to prefetch resources then there
> doesn''t have to be a prefetching problem at all.  If you make the
> provider a do-nothing in the event that asadmin is missing then this
> option becomes a superset of the dummy provider option; that would
> allow you to avoid multiple providers.  (I still don''t like the
dummy
> option even when implemented this way, however: better to fail a
> resource if asadmin is not available when the time comes to sync it.)
>
> Thecustomfact approach is pretty easy to add on top of your existing
> provider and manifests.  Also, if you arrange your resources suitably
> then you should be able simply to conditionally include an entire
> class, instead of conditionally declaring each of several separate
> resources.
>
> John
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mon, Nov 07, 2011 at 03:25:33AM -0800, David Campos
wrote:
> I tested the optional_command solution with success. The virtual
> machine could be provisioned but the first time, any reference to the
> asadmin provider failed even though the binary was present. I presume
> that once the prefetch of providers has been done and failed, then
> that type and provider become unusable at that provisioning.
> 
> I am quite sure that the response to my next question will be
''no'' but
> I''ll try anyway... It is possible to force a new binary check at
each
> provider or type execution? The first check would fail but
> optional_command would handle it and at the real execution stage, the
> check would be successful and the command would be executed.
> 
Can you please provide some debug output of your first puppetrun?

In general I think that just ignoring prefetching is pretty bad because
all resources are treated as initially absent (even though that might
not be the case) so create is run for every resource. I thought
that this is the behaviour we have right now, so some debug logs would
be helpful.

-Stefan