Puppet users - Jun 2012 - facter plugin requires cause catalog failure, prevent setting up new clients

There''s a generic problem that I suspect that many people are bumping
into, but I suspect it will surprise others. RIP''s concat module
includes a facter plugin which is required for the catalog to compile. This
requires pluginsync to be enabled.

Problem is when you have a brand new client, they need to get the puppet config
which enables pluginsync before this is going to work. You''ve got
stages and such set up to do all this before the module which uses concat is
called, but it will fail compilation. Even if you try to limit the catalog with
tags, it still needs to compile.

# puppet agent --test --tags puppet
Could not retrieve selinux_config_policy: undefined method `each_line''
for nil:NilClass
Could not retrieve selinux_config_policy: undefined method `each_line''
for nil:NilClass
Could not retrieve selinux_config_mode: undefined method `each_line''
for nil:NilClass
Could not retrieve selinux_current_mode: undefined method `each_line''
for nil:NilClass
info: Caching catalog for c1-s1.c.company.com
err: Failed to apply catalog: Parameter path failed: File paths must be fully
qualified, not ''undef'' at
/somepath/puppet/etc/modules/concat/manifests/setup.pp:38

Seriously, you can''t do this. If your module requires something from a
facter to compile, then you need to set a default in your module and override it
if the information is available from facter.  Having to edit the site manifest
to remove concat module, then add it back right after the node''s basic
setup is built… so very, very cowboy.

I''ve rewritten the module to not need a facter plugin at all (because
this really isn''t useful). For anyone else slamming into this problem,
the fix is easy

-  $concatdir = $concat_basedir
+  $concatdir = "${settings::vardir}/concat"

For anyone writing modules, consider this!

NOTE: the only way I can think of to avoid this issue is to set an environment
from the command line during puppet initialization, and use the environment to
limit the modules provided in the catalog.

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

----- Original Message -----
> From: "Jo Rhett" <jrhett@netconsonance.com>
> To: puppet-users@googlegroups.com
> Cc: rip@devco.net
> Sent: Wednesday, June 20, 2012 12:05:47 AM
> Subject: facter plugin requires cause catalog failure, prevent setting up
new clients
> 
> There''s a generic problem that I suspect that many people are
bumping
> into, but I suspect it will surprise others. RIP''s concat module
> includes a facter plugin which is required for the catalog to
> compile. This requires pluginsync to be enabled.
> 
> 
> Problem is when you have a brand new client, they need to get the
> puppet config which enables pluginsync before this is going to work.
> You''ve got stages and such set up to do all this before the module
> which uses concat is called, but it will fail compilation. Even if
> you try to limit the catalog with tags, it still needs to compile.
Just enable pluginsync from your bootstrap, its not hard and really everyone
else manages just fine, 

AFAIK its also on by default now. 

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jun 19, 2012, at 4:13 PM, R.I.Pienaar wrote:
> Just enable pluginsync from your bootstrap, its not hard and really
everyone
> else manages just fine, 
Um, the bootstrap won''t run if the catalog doesn''t compile. 
That''s the problem.
> AFAIK its also on by default now. 

Nope.  If it was we wouldn''t have this problem.  Enabling it is obvious
since we see facter plugins provided by other modules come down.

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

----- Original Message -----
> From: "Jo Rhett" <jrhett@netconsonance.com>
> To: puppet-users@googlegroups.com
> Sent: Wednesday, June 20, 2012 12:26:48 AM
> Subject: Re: [Puppet Users] facter plugin requires cause catalog failure,
prevent setting up new clients
> 
> 
> 
> On Jun 19, 2012, at 4:13 PM, R.I.Pienaar wrote:
> 
> 
> Just enable pluginsync from your bootstrap, its not hard and really
> everyone
> else manages just fine,
> 
> 
> Um, the bootstrap won''t run if the catalog doesn''t
compile. That''s
> the problem.
> 
catalog compile happens *after* pluginsync so if you enable it, it will
fetch the plugins then request catalog with the new fact attached.

by bootstrap i mean kickstart or whatever, ie. before you run puppet.
> AFAIK its also on by default now.
> 
> 
> Nope. If it was we wouldn''t have this problem. Enabling it is
obvious
> since we see facter plugins provided by other modules come down.
> 
ah not released yet, looks like that will be only default from 3 onward

Please dont send HTML mail.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jun 19, 2012, at 4:30 PM, R.I.Pienaar wrote:
> catalog compile happens *after* pluginsync so if you enable it, it will
> fetch the plugins then request catalog with the new fact attached.
But not if the bootstrapping is done via puppet.
> by bootstrap i mean kickstart or whatever, ie. before you run puppet.
I should have some other system that configures my system before I run the
software to configure my system? ;-)

I still think that proper programming practice is to fail gracefully when
dependancies aren''t met, not to cause a catalog failure.

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 06/20/2012 01:34 AM, Jo Rhett wrote:
> 
>> by bootstrap i mean kickstart or whatever, ie. before you run puppet.
> 
> I should have some other system that configures my system before I run
> the software to configure my system? ;-)
Humm, I believe it''s fair to assume that "most people" have
some way of
puppet provisioning that will be able to take care of this configuration
detail.

That being said, I concur that it would be sane design to allow a module
to compile in the absence of the custom facts etc. it may deploy to
agents during the first run.

Just my 2c.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

----- Original Message -----
> From: "Felix Frank" <felix.frank@alumni.tu-berlin.de>
> To: puppet-users@googlegroups.com
> Sent: Wednesday, June 20, 2012 9:03:48 AM
> Subject: Re: [Puppet Users] facter plugin requires cause catalog failure,
prevent setting up new clients
> 
> On 06/20/2012 01:34 AM, Jo Rhett wrote:
> > 
> >> by bootstrap i mean kickstart or whatever, ie. before you run
> >> puppet.
> > 
> > I should have some other system that configures my system before I
> > run
> > the software to configure my system? ;-)
> 
> Humm, I believe it''s fair to assume that "most people"
have some way
> of puppet provisioning that will be able to take care of this
> configuration detail.
> 
> That being said, I concur that it would be sane design to allow a
> module to compile in the absence of the custom facts etc. it may deploy to
> agents during the first run.
I agree but its also quite hard, the proposed fix is using the masters
vardir location as default for where servers must put their snippets, this
is ridiculous and not a usable fix.

There is no default really for where nodes put their vardir, we can assume
/var/lib/puppet or something but that just doesn''t hold true always and
will in those cases just fail again during a run.

Pluginsync is well understood and so widely used that I think more or less
everyone has figured out how to enable it either first run or use a first
run environment that bootstraps puppet - though how you would do that without
the ability to edit the config file is a good question.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jun 20, 2012, at 1:03 AM, Felix Frank wrote:
> Humm, I believe it''s fair to assume that "most people"
have some way of
> puppet provisioning that will be able to take care of this configuration
> detail.
Okay, I''ve worked at 4 different shops using puppet over the last 6
years (one is a non-profit I work at part time). In not one of those shops was
bare-metal provisioning something I set up (so the following decision was not
mine and never has been) and in every case, the system kickstarts to install
puppet rpm, then uses puppet to configure the rest of the system based on the
system name.

This goes back to this funny thing, where people keep saying I should be using a
different configuration management system to handle things outside of puppet.
Should we boot to install cfengine and use that to set up puppet? :p

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jun 20, 2012, at 1:11 AM, R.I.Pienaar wrote:
> I agree but its also quite hard, the proposed fix is using the masters
> vardir location as default for where servers must put their snippets, this
> is ridiculous and not a usable fix.
Yeah, I realized that.
> Pluginsync is well understood and so widely used that I think more or less
> everyone has figured out how to enable it either first run or use a first
> run environment that bootstraps puppet - though how you would do that
without
> the ability to edit the config file is a good question.

Since kickstart can run arbitrary commands, we can have it run the bootstrap
with an environment flag. That''s what I''m switching this to.

I do also think that you could observe that the variable isn''t set,
punch out a notice and avoid the compilation failure. It''s easy enough
to do.

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 06/20/2012 06:54 PM, Jo Rhett wrote:
>> Humm, I believe it''s fair to assume that "most
people" have some way of
>> puppet provisioning that will be able to take care of this
configuration
>> detail.
> 
> Okay, I''ve worked at 4 different shops using puppet over the last
6
> years (one is a non-profit I work at part time). In not one of those
> shops was bare-metal provisioning something I set up (so the following
> decision was not mine and never has been) and in every case, the system
> kickstarts to install puppet rpm, then uses puppet to configure the rest
> of the system based on the system name.
> 
> This goes back to this funny thing, where people keep saying I should be
> using a different configuration management system to handle things
> outside of puppet. Should we boot to install cfengine and use that to
> set up puppet? :p
I seem to have your personal red cloth on me ;)

I''m not suggesting an additional config management system. Most (all?)
bare metal provisioning tools offer some way to carbon copy some base
files, so a default puppet.conf that differs from the one in the RPM
would be one idea to go about it.
If the site has a need to maintain its own RPM of puppet agent (I know I
do, but I realize it''s not commonplace), this is a non-issue.

But, again, I''m totally down with using puppet to configure puppet.
This
should definitely be an option, and it''s a bummer that an important
module keeps it from Just Working.

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jun 21, 2012, at 2:10 AM, Felix Frank wrote:
> I seem to have your personal red cloth on me ;)
Sorry, you keep making comments which remind me of general things. Sorry.
I''ll get over my grump of the last few days.
> But, again, I''m totally down with using puppet to configure
puppet. This
> should definitely be an option, and it''s a bummer that an
important
> module keeps it from Just Working.

I don''t mean to pick on RIP or concat in specific. I think that
it''s something that should be handled better in general.  RIP thinks
that puppet ensures the plugin sync happens. Both are valid PoV, however his
will be more valid once pluginsync = true is a default, or shipped in the
RPM''s default config.

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 6/21/2012 11:41 AM, Jo Rhett wrote:
> On Jun 21, 2012, at 2:10 AM, Felix Frank wrote:
>> I seem to have your personal red cloth on me ;)
>
> Sorry, you keep making comments which remind me of general things.
> Sorry. I''ll get over my grump of the last few days.
>
>> But, again, I''m totally down with using puppet to configure
puppet. This
>> should definitely be an option, and it''s a bummer that an
important
>> module keeps it from Just Working.
>
> I don''t mean to pick on RIP or concat in specific. I think that
it''s
> something that should be handled better in general.  RIP thinks that
> puppet ensures the plugin sync happens. Both are valid PoV, however his
> will be more valid once pluginsync = true is a default, or shipped in
> the RPM''s default config.
Surprised no one has mentioned this, but anything you can set in 
puppet.conf can be passed on the command line. Adding --pluginsync to 
whatever Puppet first run command you use solves the problem, no?

Ramin

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jun 21, 2012, at 12:55 PM, Ramin K wrote:
> Surprised no one has mentioned this, but anything you can set in
puppet.conf can be passed on the command line. Adding --pluginsync to whatever
Puppet first run command you use solves the problem, no?
And pluginsource too. Gets a bit unweildy. I''m still trying to figure
out how I did the bootstrap environment to shorten the catalog at a previous
shop. Memory fail.

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.