Puppet users - Jul 2012 - proper usage of global variables / node variables / +=

Hello puppet masters, I am cleaning up some puppet modules, using 
puppet-lint. The warning I am getting is:

top-scope variable being used without an explicit namespace

I can turn this particular check off, but in doing my research I''m 
finding all sorts of messages saying to avoid using +=, to avoid using 
variables in the node scope, and to switch to a parameterized class 
whenever possible. Okay, I''m game.

So, to understand what I''m doing, I''m setting up a yum
repository and
limiting the includepkgs line to the very minimum amount of packages for 
reasons of security / policy / insanity / etc.

Here is the current implementation of this (which works fine).

node standard {
   $epel_includepkgs += ''puppet augeas-libs facter ruby-augeas
ruby-shadow ''
   class { ''repo_epel'': stage => ''pre'' }
   #other stuff
}

node ''my.node1'' inherits standard {
   include denyhosts
}

node ''my.node2'' inherits standard {
   include denyhosts
   include gitlabhq
}

class repo_epel {
   yumrepo { ''epel'':
     enabled         => 1,
     descr           => ''Extra Packages for Enterprise Linux 6 - 
\$basearch'',
     mirrorlist      => 
''https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=\$basearch'',
     failovermethod  => priority,
     gpgcheck        => 1,
     gpgkey          =>
''file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6'',
     require         =>
File[''/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6''],
     includepkgs     => $epel_includepkgs,
   }
   # other stuff
}

class denyhosts {
   $epel_includepkgs += ''denyhosts ''
   # other stuff
}

class gitlabhq {
   $epel_includepkgs += ''libyaml gitolite git-all rvm ''
   # other stuff
}

my.node1, and my.node2 will have different (and correct) includepkgs 
doing it this way (I have like .. 30+ modules for epel and more of other 
repos to give you an idea)

To get rid of the error I tried making the variable $::epel_includepkgs, 
but that generates the error:
err: Could not parse for environment production: Cannot assign to 
variables in other #namespaces

So what is the ''proper'' puppet way to accomplish this?

I tried a parameterized class like this:

class repo_epel ($include_pkgs) {
   notify { "include_pkgs = $include_pkgs":}
}

class denyhosts {
   $repo_epel::include_pkgs += "denyhosts "
}

but from what I understand that only changes the include_pkgs variable 
inside the denyhosts scope (or maybe I''m confused).

Any help would be appreciated.

Thanks

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thursday, July 5, 2012 5:34:13 AM UTC-5, fpee wrote:
>
> Hello puppet masters, I am cleaning up some puppet modules, using 
> puppet-lint. The warning I am getting is: 
>
> top-scope variable being used without an explicit namespace 
>
> I can turn this particular check off, but in doing my research I''m
> finding all sorts of messages saying to avoid using +
Good advice.
 
> , to avoid using 
> variables in the node scope,

Good advice, but sometimes difficult to apply.
 
> and to switch to a parameterized class 
> whenever possible.

Bad, *bad* advice.  PuppetLabs did a pretty good job of promoting 
parametrized classes when it first introduced them, but the design was 
deeply flawed.  The idea clashes with Puppet''s model of classes as 
singletons, and the worst of the flaws spring from that mismatch.  Only if 
you''re using an ENC should you even consider class parametrization (a 
manifest set designed with an ENC in mind can in principle overcome the 
worst problems attending class parametrization), but think twice even then.

Puppet 3 does better by tying class parameters to hiera, allowing class 
parameters to serve as a structured shortcut to external data, and 
providing a means to wean all the poor folks that invested development time 
in parametrized classes.  Even if you''re aiming for Puppet 3, however,
your
effort would be better spent on externalizing data for access via hiera 
than on parametrizing your classes.

 
> Okay, I''m game. 
>
> So, to understand what I''m doing, I''m setting up a yum
repository and
> limiting the includepkgs line to the very minimum amount of packages for 
> reasons of security / policy / insanity / etc. 
>
> Here is the current implementation of this (which works fine). 
>
> node standard { 
>    $epel_includepkgs += ''puppet augeas-libs facter ruby-augeas
ruby-shadow
> '' 
>    class { ''repo_epel'': stage =>
''pre'' }
>    #other stuff 
> } 
>
The worst thing about run stages in Puppet 2 is that you can only use them 
via the parametrized class syntax, which brings the worst problems of 
parametrized classes even to non-parametrized classes.  Of course, run 
stages are purely syntactic sugar anyway.
 
> node ''my.node1'' inherits standard { 
>    include denyhosts 
> } 
>
> node ''my.node2'' inherits standard { 
>    include denyhosts 
>    include gitlabhq 
> } 
>
> class repo_epel { 
>    yumrepo { ''epel'': 
>      enabled         => 1, 
>      descr           => ''Extra Packages for Enterprise Linux 6
-
> \$basearch'', 
>      mirrorlist      => 
>
''https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=\$basearch'',
>      failovermethod  => priority, 
>      gpgcheck        => 1, 
>      gpgkey          =>
''file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6'',
>      require         =>
File[''/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6''],
>      includepkgs     => $epel_includepkgs, 
>    } 
>    # other stuff 
> } 
>
> class denyhosts { 
>    $epel_includepkgs += ''denyhosts '' 
>    # other stuff 
> } 
>
> class gitlabhq { 
>    $epel_includepkgs += ''libyaml gitolite git-all rvm '' 
>    # other stuff 
> } 
>
> my.node1, and my.node2 will have different (and correct) includepkgs 
> doing it this way (I have like .. 30+ modules for epel and more of other 
> repos to give you an idea) 
>

If that''s so then you are relying on a Puppet bug, or perhaps you have 
oversimplified your example.  The language guide specifically states that 
the += operator affects the observed value of the affected variable only in 
the scope where the plussignment is performed.  You definitely should not 
see the modified value in classes that are not declared in the scope of the 
plussignment.

 
> To get rid of the error I tried making the variable $::epel_includepkgs, 
> but that generates the error: 
> err: Could not parse for environment production: Cannot assign to 
> variables in other #namespaces

Right, because plussignment doesn''t change the values of existing 
variables.  It merely shadows those values with supplemented ones, and only 
within the scope where the plussignment appears.  Dynamic scoping allows 
classes and resources to see that local value when their declarations 
appear in the same scope or a deeper-nested one, but that''s a problem 
because classes can be included more than once, from different scopes.  If 
those scopes provide different values for the same, needed variable then 
which one does such a class use?  For all intents and purposes, it''s 
unpredictable.
 
> So what is the ''proper'' puppet way to accomplish this? 
>
Puppet DSL does not provide a mechanism for constructing values 
incrementally.  Even what you''re currently doing probably
doesn''t work
quite the way you think it does.  There are ways to approach it if you can 
assume strict nesting of all the scopes that are expected to contribute 
values, but that''s not your case.

One messy, but valid way to do this might be to manage the inclusion of 
packages in a repo''s includepkgs via instances of a defined type that 
relies on an exec.  Each instance of that type would ensure that or more 
included packages were listed for the specified repo, and they could all be 
declared independently, wherever the package itself is declared.  Then 
don''t manage the ''includepkgs'' property of the
Yumrepo resource so that you
don''t have clashes.

Alternatively, you could probably write a couple of custom functions to 
manage a named, updatable data cubbyhole to use the way you are currently 
using plussignment.  That would silence the warnings, but you would still 
have to deal with the issue of ensuring that all the needed values had been 
appended before the result was read out.  I think you could work out 
something in that regard via class inheritance.

Or you could switch to Ruby DSL, at least in strategic places.  That would 
give you more freedom, but you would still need to handle the problem of 
accumulating all the needed values before reading them out.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/J2DtcXbtDFMJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 07/05/2012 08:30 AM, jcbollinger wrote:

Thank you very much for the response, I am going over it in detail and 
trying out all of the options you have presented (this may take me a 
couple days). I will reply again once I have gone over your message in 
more detail.
> If that''s so then you are relying on a Puppet bug, or perhaps you
have
> oversimplified your example.  The language guide specifically states
> that the += operator affects the observed value of the affected variable
> only in the scope where the plussignment is performed.  You definitely
> should not see the modified value in classes that are not declared in
> the scope of the plussignment.
Cool! Let''s find out. Here is a working tiny example:

node my_standard {
   $pkg_list += ''one two three ''
}

node ''testnode'' inherits my_standard {
   include my_test1
   include my_test2
}

class my_test1 {
   $pkg_list += ''four five ''
   notify {"pkg_list=$pkg_list":}
   notice("pkg_list=$pkg_list")
}

class my_test2 {
   $pkg_list += ''six seven ''
   notify {"pkg_list=$pkg_list":}
   notice("pkg_list=$pkg_list")
}

The output when run on testnode:

notice: /Stage[main]/My_test1/Notify[pkg_list=one two three four five 
]/message: defined ''message'' as ''pkg_list=one two
three four five ''

notice: /Stage[main]/My_test2/Notify[pkg_list=one two three four five 
six seven ]/message: defined ''message'' as
''pkg_list=one two three four
five six seven ''

So, the my_test2 scope does have the changes from the my_test1 scope. 
Using puppet-server 2.7.9-1 (epel version).

As the guy who has many modules that use this bug, I like it''s 
functionality :)

Should I file a bug report?

Again thank you for the reply.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thursday, July 5, 2012 2:59:07 PM UTC-5, fpee wrote:
>
> On 07/05/2012 08:30 AM, jcbollinger wrote: 
>
> > If that''s so then you are relying on a Puppet bug, or perhaps
you have
> > oversimplified your example.  The language guide specifically states 
> > that the += operator affects the observed value of the affected
variable
> > only in the scope where the plussignment is performed.  You definitely
> > should not see the modified value in classes that are not declared in 
> > the scope of the plussignment. 
>
> Cool! Let''s find out. Here is a working tiny example: 
>
> node my_standard { 
>    $pkg_list += ''one two three '' 
> } 
>
> node ''testnode'' inherits my_standard { 
>    include my_test1 
>    include my_test2 
> } 
>
> class my_test1 { 
>    $pkg_list += ''four five '' 
>    notify {"pkg_list=$pkg_list":} 
>    notice("pkg_list=$pkg_list") 
> } 
>
> class my_test2 { 
>    $pkg_list += ''six seven '' 
>    notify {"pkg_list=$pkg_list":} 
>    notice("pkg_list=$pkg_list") 
> } 
>
> The output when run on testnode: 
>
> notice: /Stage[main]/My_test1/Notify[pkg_list=one two three four five 
> ]/message: defined ''message'' as ''pkg_list=one
two three four five ''
>
> notice: /Stage[main]/My_test2/Notify[pkg_list=one two three four five 
> six seven ]/message: defined ''message'' as
''pkg_list=one two three four
> five six seven '' 
>
> So, the my_test2 scope does have the changes from the my_test1 scope. 
> Using puppet-server 2.7.9-1 (epel version). 
>
> As the guy who has many modules that use this bug, I like it''s 
> functionality :) 
>
> Should I file a bug report? 
>
Not if you like the buggy behavior :-)

Personally, I would be very concerned that it might stop working between 
one point release and another, with no warning.  The behavior is not merely 
undocumented, but *contrary* to the documentation as far as I can tell.

At this point, however, I''m hoping that one of the PuppetLabs guys will
jump in to either explain why I''m wrong or confirm that it''s a
bug.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/s8-GASWdRkkJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, Jul 5, 2012 at 2:41 PM, jcbollinger <John.Bollinger@stjude.org>
wrote:
>
>
> On Thursday, July 5, 2012 2:59:07 PM UTC-5, fpee wrote:
>>
>> On 07/05/2012 08:30 AM, jcbollinger wrote:
>>
>> > If that''s so then you are relying on a Puppet bug, or
perhaps you have
>> > oversimplified your example.  The language guide specifically
states
>> > that the += operator affects the observed value of the affected
variable
>> > only in the scope where the plussignment is performed.  You
definitely
>> > should not see the modified value in classes that are not declared
in
>> > the scope of the plussignment.
>>
>> Cool! Let''s find out. Here is a working tiny example:
>>
>> node my_standard {
>>    $pkg_list += ''one two three ''
>> }
>>
>> node ''testnode'' inherits my_standard {
>>    include my_test1
>>    include my_test2
>> }
>>
>> class my_test1 {
>>    $pkg_list += ''four five ''
>>    notify {"pkg_list=$pkg_list":}
>>    notice("pkg_list=$pkg_list")
>> }
>>
>> class my_test2 {
>>    $pkg_list += ''six seven ''
>>    notify {"pkg_list=$pkg_list":}
>>    notice("pkg_list=$pkg_list")
>> }
>>
>> The output when run on testnode:
>>
>> notice: /Stage[main]/My_test1/Notify[pkg_list=one two three four five
>> ]/message: defined ''message'' as
''pkg_list=one two three four five ''
>>
>> notice: /Stage[main]/My_test2/Notify[pkg_list=one two three four five
>> six seven ]/message: defined ''message'' as
''pkg_list=one two three four
>> five six seven ''
>>
>> So, the my_test2 scope does have the changes from the my_test1 scope.
>> Using puppet-server 2.7.9-1 (epel version).
>>
>> As the guy who has many modules that use this bug, I like it''s
>> functionality :)
>>
>> Should I file a bug report?
>
>
> Not if you like the buggy behavior :-)
>
> Personally, I would be very concerned that it might stop working between
one
> point release and another, with no warning.  The behavior is not merely
> undocumented, but contrary to the documentation as far as I can tell.
>
> At this point, however, I''m hoping that one of the PuppetLabs guys
will jump
> in to either explain why I''m wrong or confirm that it''s a
bug.
This is not an official Puppet Labs opinion. This seems to be
compilation order dependent, so you will have different notify for:

include my_test1
include my_test2

v.s.

include my_test2
include my_test1

Also at a quick glance I can''t see how you would use this in Puppet
3.0 with dynamic scope removed.

Nan

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

> At this point, however, I''m hoping that one of the PuppetLabs guys
will
> jump in to either explain why I''m wrong or confirm that
it''s a bug.
Any PuppetLabs guys reading? I realize they are all busy people, and it 
may take a while. It''s all good.


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Nan is a Puppet Labs guy. One of our best :)

His warning about dynamic scoping is something to take seriously.
That''s not a gratuitous error message - dynamic scoping causes all
kinds of problems and has been scheduled for deprecation for a while now, and
that''s coming in the next major release.

Details:

http://docs.puppetlabs.com/guides/scope_and_puppet.html

-Eric 

-- 

Eric Shamow
Professional Services
http://puppetlabs.com/
(c)631.871.6441


On Thursday, July 12, 2012 at 1:27 AM, fp wrote:
> > At this point, however, I''m hoping that one of the PuppetLabs
guys will
> > jump in to either explain why I''m wrong or confirm that
it''s a bug.
> 
> 
> 
> Any PuppetLabs guys reading? I realize they are all busy people, and it 
> may take a while. It''s all good.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com
(mailto:puppet-users@googlegroups.com).
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
(mailto:puppet-users+unsubscribe@googlegroups.com).
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thursday, July 12, 2012 10:52:23 AM UTC-5, Eric Shamow
wrote:
>
> Nan is a Puppet Labs guy. One of our best :) 
>
> His warning about dynamic scoping is something to take seriously.
That''s
> not a gratuitous error message - dynamic scoping causes all kinds of 
> problems and has been scheduled for deprecation for a while now, and
that''s
> coming in the next major release. 
>
> Details: 
>
> http://docs.puppetlabs.com/guides/scope_and_puppet.html 
>
>
Thanks, Eric, but none of that is responsive to the question presently at 
hand, which is whether the behavior the OP observed with += is buggy (as I 
have claimed) or intentional.  Are you or Nan, either one, willing to opine 
on that one?


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/X4j1dAM9TpIJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.