Puppet users - Oct 2011 - "Could not find a default provider for ..." should not fail the agent run?

Hi all,

I have repeatedly encountered the problem that I would like Puppet to install a
particular piece of software, for example git, and then use a provider like
https://github.com/puppetlabs/puppet-vcsrepo to check out a repository.

This fails with a message like "Could not run Puppet configuration client:
Could not find a default provider for ..." unless the tool (git, to stick
with my example) is already installed. This chicken and egg problem applies to
some other install-stuff-and-do-more-stuff-with-it situations as well.

I understand that it would be way too complicated for puppet to be able to
handle all this in a single run. But isn''t failing due to the lack of a
particular (default) provider too hard?

If Puppet would carry on and just fail on the vcsrepo {} (or whatever type of
resource), things would probably sort out after two or three agent runs.

In IRC they pointed me to either using environments, which I think is too
complicated (having to maintain "bootstrap" and "production"
manifests).

Another tip was to have a look at the way the pip package provider
(https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package/pip.rb)
works, see lazy_pip at the end. But to me it seems as if that would be out of
line with the rest of providers and working against the API provided by Puppet.

Suggestions?

-mp.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Matthias,

I had a similar problem recently with libvirt Ruby bindings being
required for a provider. Due to a few other requirements I opted for
creating a "second pass" approach by using a custom fact to describe
whether my provider was "ready to be used" on a node and wrap my
resources in a conditional based on this fact. The fact tests certain
files exist which are installed by packages managed in other Puppet
modules. It''s pretty hacky but I didn''t have time to invest in
a nicer
solution such as handling the error inside the provider.

-Luke

On Oct 5, 3:25 pm, Matthias Pigulla <m...@webfactory.de>
wrote:
> Hi all,
>
> I have repeatedly encountered the problem that I would like Puppet to
install a particular piece of software, for example git, and then use a provider
likehttps://github.com/puppetlabs/puppet-vcsrepoto check out a repository.
>
> This fails with a message like "Could not run Puppet configuration
client: Could not find a default provider for ..." unless the tool (git, to
stick with my example) is already installed. This chicken and egg problem
applies to some other install-stuff-and-do-more-stuff-with-it situations as
well.
>
> I understand that it would be way too complicated for puppet to be able to
handle all this in a single run. But isn''t failing due to the lack of a
particular (default) provider too hard?
>
> If Puppet would carry on and just fail on the vcsrepo {} (or whatever type
of resource), things would probably sort out after two or three agent runs.
>
> In IRC they pointed me to either using environments, which I think is too
complicated (having to maintain "bootstrap" and "production"
manifests).
>
> Another tip was to have a look at the way the pip package provider
(https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/...)
works, see lazy_pip at the end. But to me it seems as if that would be out of
line with the rest of providers and working against the API provided by Puppet.
>
> Suggestions?
>
> -mp.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Without waving my ignorance around too much... does Matthias'' issue fit
run stages? This sounds like exactly what they were designed to solve: ensure
certain things definitely happen before certain other things.

On Wed, Oct 05, 2011 at 07:33:29AM -0700, Luke Bigum
wrote:
> Matthias,
> 
> I had a similar problem recently with libvirt Ruby bindings being
> required for a provider. Due to a few other requirements I opted for
> creating a "second pass" approach by using a custom fact to
describe
> whether my provider was "ready to be used" on a node and wrap my
> resources in a conditional based on this fact. The fact tests certain
> files exist which are installed by packages managed in other Puppet
> modules. It''s pretty hacky but I didn''t have time to
invest in a nicer
> solution such as handling the error inside the provider.
> 
> -Luke
> 
> On Oct 5, 3:25 pm, Matthias Pigulla <m...@webfactory.de> wrote:
> > Hi all,
> >
> > I have repeatedly encountered the problem that I would like Puppet to
install a particular piece of software, for example git, and then use a provider
likehttps://github.com/puppetlabs/puppet-vcsrepoto check out a repository.
> >
> > This fails with a message like "Could not run Puppet
configuration client: Could not find a default provider for ..." unless the
tool (git, to stick with my example) is already installed. This chicken and egg
problem applies to some other install-stuff-and-do-more-stuff-with-it situations
as well.
> >
> > I understand that it would be way too complicated for puppet to be
able to handle all this in a single run. But isn''t failing due to the
lack of a particular (default) provider too hard?
> >
> > If Puppet would carry on and just fail on the vcsrepo {} (or whatever
type of resource), things would probably sort out after two or three agent runs.
> >
> > In IRC they pointed me to either using environments, which I think is
too complicated (having to maintain "bootstrap" and
"production" manifests).
> >
> > Another tip was to have a look at the way the pip package provider
(https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/...)
works, see lazy_pip at the end. But to me it seems as if that would be out of
line with the rest of providers and working against the API provided by Puppet.
> >
> > Suggestions?
> >
> > -mp.
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
> 
> 
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Not exactly. If we could both get our resources to fail individually
(like a File''s "source" parameter not existing and throwing
an error
for that resource) it would work, but because we''re using custom
types, the Puppet agent loads all those in (and Facts) and dies before
it gets to it''s catalog, so run levels won''t help.
It''s the same error
as if you went to use a native Augeas resource and didn''t have the
Augeas Ruby bindings installed.

On Oct 5, 3:36 pm, Christopher Wood <christopher_w...@pobox.com>
wrote:
> Without waving my ignorance around too much... does Matthias''
issue fit run stages? This sounds like exactly what they were designed to solve:
ensure certain things definitely happen before certain other things.
>
>
>
>
>
>
>
> On Wed, Oct 05, 2011 at 07:33:29AM -0700, Luke Bigum wrote:
> > Matthias,
>
> > I had a similar problem recently with libvirt Ruby bindings being
> > required for a provider. Due to a few other requirements I opted for
> > creating a "second pass" approach by using a custom fact to
describe
> > whether my provider was "ready to be used" on a node and
wrap my
> > resources in a conditional based on this fact. The fact tests certain
> > files exist which are installed by packages managed in other Puppet
> > modules. It''s pretty hacky but I didn''t have time to
invest in a nicer
> > solution such as handling the error inside the provider.
>
> > -Luke
>
> > On Oct 5, 3:25 pm, Matthias Pigulla <m...@webfactory.de> wrote:
> > > Hi all,
>
> > > I have repeatedly encountered the problem that I would like
Puppet to install a particular piece of software, for example git, and then use
a provider likehttps://github.com/puppetlabs/puppet-vcsrepotocheck out a
repository.
>
> > > This fails with a message like "Could not run Puppet
configuration client: Could not find a default provider for ..." unless the
tool (git, to stick with my example) is already installed. This chicken and egg
problem applies to some other install-stuff-and-do-more-stuff-with-it situations
as well.
>
> > > I understand that it would be way too complicated for puppet to
be able to handle all this in a single run. But isn''t failing due to
the lack of a particular (default) provider too hard?
>
> > > If Puppet would carry on and just fail on the vcsrepo {} (or
whatever type of resource), things would probably sort out after two or three
agent runs.
>
> > > In IRC they pointed me to either using environments, which I
think is too complicated (having to maintain "bootstrap" and
"production" manifests).
>
> > > Another tip was to have a look at the way the pip package
provider
(https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/...)
works, see lazy_pip at the end. But to me it seems as if that would be out of
line with the rest of providers and working against the API provided by Puppet.
>
> > > Suggestions?
>
> > > -mp.
>
> > --
> > You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
> > To post to this group, send email to puppet-users@googlegroups.com.
> > To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> > For more options, visit this group
athttp://groups.google.com/group/puppet-users?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

My explanation''s not the best so went looking to see if the Internals
wiki page specifically mentions what I''m talking about, I''m
pretty
sure the problem is in the Parsing phase:

http://docs.puppetlabs.com/guides/puppet_internals.html

On Oct 5, 3:42 pm, Luke Bigum <Luke.Bi...@lmax.com>
wrote:
> Not exactly. If we could both get our resources to fail individually
> (like a File''s "source" parameter not existing and
throwing an error
> for that resource) it would work, but because we''re using custom
> types, the Puppet agent loads all those in (and Facts) and dies before
> it gets to it''s catalog, so run levels won''t help.
It''s the same error
> as if you went to use a native Augeas resource and didn''t have the
> Augeas Ruby bindings installed.
>
> On Oct 5, 3:36 pm, Christopher Wood <christopher_w...@pobox.com>
> wrote:
>
>
>
>
>
>
>
> > Without waving my ignorance around too much... does Matthias''
issue fit run stages? This sounds like exactly what they were designed to solve:
ensure certain things definitely happen before certain other things.
>
> > On Wed, Oct 05, 2011 at 07:33:29AM -0700, Luke Bigum wrote:
> > > Matthias,
>
> > > I had a similar problem recently with libvirt Ruby bindings being
> > > required for a provider. Due to a few other requirements I opted
for
> > > creating a "second pass" approach by using a custom
fact to describe
> > > whether my provider was "ready to be used" on a node
and wrap my
> > > resources in a conditional based on this fact. The fact tests
certain
> > > files exist which are installed by packages managed in other
Puppet
> > > modules. It''s pretty hacky but I didn''t have
time to invest in a nicer
> > > solution such as handling the error inside the provider.
>
> > > -Luke
>
> > > On Oct 5, 3:25 pm, Matthias Pigulla <m...@webfactory.de>
wrote:
> > > > Hi all,
>
> > > > I have repeatedly encountered the problem that I would like
Puppet to install a particular piece of software, for example git, and then use
a provider likehttps://github.com/puppetlabs/puppet-vcsrepotocheckout a
repository.
>
> > > > This fails with a message like "Could not run Puppet
configuration client: Could not find a default provider for ..." unless the
tool (git, to stick with my example) is already installed. This chicken and egg
problem applies to some other install-stuff-and-do-more-stuff-with-it situations
as well.
>
> > > > I understand that it would be way too complicated for puppet
to be able to handle all this in a single run. But isn''t failing due to
the lack of a particular (default) provider too hard?
>
> > > > If Puppet would carry on and just fail on the vcsrepo {} (or
whatever type of resource), things would probably sort out after two or three
agent runs.
>
> > > > In IRC they pointed me to either using environments, which I
think is too complicated (having to maintain "bootstrap" and
"production" manifests).
>
> > > > Another tip was to have a look at the way the pip package
provider
(https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/...)
works, see lazy_pip at the end. But to me it seems as if that would be out of
line with the rest of providers and working against the API provided by Puppet.
>
> > > > Suggestions?
>
> > > > -mp.
>
> > > --
> > > You received this message because you are subscribed to the
Google Groups "Puppet Users" group.
> > > To post to this group, send email to
puppet-users@googlegroups.com.
> > > To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> > > For more options, visit this group
athttp://groups.google.com/group/puppet-users?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wed, Oct 05, 2011 at 07:50:48AM -0700, Luke Bigum
wrote:
> My explanation''s not the best so went looking to see if the
Internals
> wiki page specifically mentions what I''m talking about,
I''m pretty
> sure the problem is in the Parsing phase:
> 
> http://docs.puppetlabs.com/guides/puppet_internals.html
> 
> On Oct 5, 3:42 pm, Luke Bigum <Luke.Bi...@lmax.com> wrote:
> > Not exactly. If we could both get our resources to fail individually
> > (like a File''s "source" parameter not existing and
throwing an error
> > for that resource) it would work, but because we''re using
custom
> > types, the Puppet agent loads all those in (and Facts) and dies before
> > it gets to it''s catalog, so run levels won''t help.
It''s the same error
> > as if you went to use a native Augeas resource and didn''t
have the
> > Augeas Ruby bindings installed.
I decidedly hadn''t thought this through -- custom types/facts
aren''t part of the catalog, and run stages are only for catalog items.
I certainly appreciate the explanation!
> > On Oct 5, 3:36 pm, Christopher Wood <christopher_w...@pobox.com>
> > wrote:
> >
> >
> >
> >
> >
> >
> >
> > > Without waving my ignorance around too much... does
Matthias'' issue fit run stages? This sounds like exactly what they were
designed to solve: ensure certain things definitely happen before certain other
things.
> >
> > > On Wed, Oct 05, 2011 at 07:33:29AM -0700, Luke Bigum wrote:
> > > > Matthias,
> >
> > > > I had a similar problem recently with libvirt Ruby bindings
being
> > > > required for a provider. Due to a few other requirements I
opted for
> > > > creating a "second pass" approach by using a
custom fact to describe
> > > > whether my provider was "ready to be used" on a
node and wrap my
> > > > resources in a conditional based on this fact. The fact
tests certain
> > > > files exist which are installed by packages managed in other
Puppet
> > > > modules. It''s pretty hacky but I didn''t
have time to invest in a nicer
> > > > solution such as handling the error inside the provider.
> >
> > > > -Luke
> >
> > > > On Oct 5, 3:25 pm, Matthias Pigulla
<m...@webfactory.de> wrote:
> > > > > Hi all,
> >
> > > > > I have repeatedly encountered the problem that I would
like Puppet to install a particular piece of software, for example git, and then
use a provider likehttps://github.com/puppetlabs/puppet-vcsrepotocheckout a
repository.
> >
> > > > > This fails with a message like "Could not run
Puppet configuration client: Could not find a default provider for ..."
unless the tool (git, to stick with my example) is already installed. This
chicken and egg problem applies to some other
install-stuff-and-do-more-stuff-with-it situations as well.
> >
> > > > > I understand that it would be way too complicated for
puppet to be able to handle all this in a single run. But isn''t failing
due to the lack of a particular (default) provider too hard?
> >
> > > > > If Puppet would carry on and just fail on the vcsrepo
{} (or whatever type of resource), things would probably sort out after two or
three agent runs.
> >
> > > > > In IRC they pointed me to either using environments,
which I think is too complicated (having to maintain "bootstrap" and
"production" manifests).
> >
> > > > > Another tip was to have a look at the way the pip
package provider
(https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/...)
works, see lazy_pip at the end. But to me it seems as if that would be out of
line with the rest of providers and working against the API provided by Puppet.
> >
> > > > > Suggestions?
> >
> > > > > -mp.
> >
> > > > --
> > > > You received this message because you are subscribed to the
Google Groups "Puppet Users" group.
> > > > To post to this group, send email to
puppet-users@googlegroups.com.
> > > > To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> > > > For more options, visit this group
athttp://groups.google.com/group/puppet-users?hl=en.
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
> 
> 
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Oct 5, 2011, at 7:25 AM, Matthias Pigulla wrote:
> Hi all,
> 
> I have repeatedly encountered the problem that I would like Puppet to
install a particular piece of software, for example git, and then use a provider
like https://github.com/puppetlabs/puppet-vcsrepo to check out a repository.
> 
> This fails with a message like "Could not run Puppet configuration
client: Could not find a default provider for ..." unless the tool (git, to
stick with my example) is already installed. This chicken and egg problem
applies to some other install-stuff-and-do-more-stuff-with-it situations as
well.
> 
> I understand that it would be way too complicated for puppet to be able to
handle all this in a single run. But isn''t failing due to the lack of a
particular (default) provider too hard?
> 
> If Puppet would carry on and just fail on the vcsrepo {} (or whatever type
of resource), things would probably sort out after two or three agent runs.
> 
> In IRC they pointed me to either using environments, which I think is too
complicated (having to maintain "bootstrap" and "production"
manifests).
> 
> Another tip was to have a look at the way the pip package provider
(https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package/pip.rb)
works, see lazy_pip at the end. But to me it seems as if that would be out of
line with the rest of providers and working against the API provided by Puppet.
> 
> Suggestions?
----
I suspect that I must be missing something here but it seems rather obvious to
me that in this specific example...

require => Package["git"] 

would mean that git is installed first and then it should just work.

As for ''fail'' and continue on, that''s clearly not the
puppet way and any tolerance of failures seems inappropriate on just about any
level.

Craig

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wed, Oct 5, 2011 at 8:40 AM, Craig White <craig.white@ttiltd.com>
wrote:
>
> On Oct 5, 2011, at 7:25 AM, Matthias Pigulla wrote:
>
>> Hi all,
>>
>> I have repeatedly encountered the problem that I would like Puppet to
install a particular piece of software, for example git, and then use a provider
like https://github.com/puppetlabs/puppet-vcsrepo to check out a repository.
>>
>> This fails with a message like "Could not run Puppet configuration
client: Could not find a default provider for ..." unless the tool (git, to
stick with my example) is already installed. This chicken and egg problem
applies to some other install-stuff-and-do-more-stuff-with-it situations as
well.
>>
>> I understand that it would be way too complicated for puppet to be able
to handle all this in a single run. But isn''t failing due to the lack
of a particular (default) provider too hard?
>>
>> If Puppet would carry on and just fail on the vcsrepo {} (or whatever
type of resource), things would probably sort out after two or three agent runs.
>>
>> In IRC they pointed me to either using environments, which I think is
too complicated (having to maintain "bootstrap" and
"production" manifests).
>>
>> Another tip was to have a look at the way the pip package provider
(https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package/pip.rb)
works, see lazy_pip at the end. But to me it seems as if that would be out of
line with the rest of providers and working against the API provided by Puppet.
>>
>> Suggestions?
> ----
> I suspect that I must be missing something here but it seems rather obvious
to me that in this specific example...
>
> require => Package["git"]
>
> would mean that git is installed first and then it should just work.
You can''t use a provider if it depends on a package that has yet to be
installed. At least not in a single puppet run. Here''s the issue:

http://projects.puppetlabs.com/issues/6907

Thanks,

Nan

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Oct 5, 4:40 pm, Craig White <craig.wh...@ttiltd.com>
wrote:
> I suspect that I must be missing something here but it seems rather obvious
to me that in this specific example...
>
> require => Package["git"]
That would work if Puppet was attempting to use Git (in Matthias''
example) and LibVirt (in my example) during the evaluation of it''s
catalog, but it''s not. Puppet is essentially failing with a Parse
error. It''s the same as trying to "include" a CPAN module in
Perl when
it''s not installed, or link against a C library that doesn''t
exist.
The Agent has a catalog with containing a native type of Git and
there''s no Git installed on the Agent.

We''ve got providers that use underlying system functionality (Git/
LibVirt) but that functionality gets installed by Puppet - bit of a
chicken and egg. Personally I don''t want Git/LibVirt installed on
every machine in my estate just in case my nodes might use one of
these user defined types.
> would mean that git is installed first and then it should just work.
>
> As for ''fail'' and continue on, that''s clearly
not the puppet way and any tolerance of failures seems inappropriate on just
about any level.
I agree it''s not perfect, for this problem there''s a number of
less
than perfect solutions, but failing in the catalog evaluation would at
least give you a full report on the first Puppet run and simplify your
manifests from strange conditional logic (like mine have at the
moment).

I''d be very interested in other people''s approaches for user
defined
types based on Puppet installed prerequisites.

-Luke

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.