thr3ads.net - Puppet users - [Puppet Users] Problem with variable scope in Defined Types [Aug 2013]

If this information is useful, please help other people find it:
Share via:

Dan White

2013-Aug-14 02:59 UTC

[Puppet Users] Problem with variable scope in Defined Types

I have a problem with variables in defined types.

I have dug a hole consisting of three levels of nested defines as in 
-------------------------------------------
class users {
...
}
-------------------------------------------
define users::useraccount (
    $username            = $name,
    $uid,
    $info,
    $ingroups            = [],
    $dotfiles            = [],
    $userhome            = "/home/${name}",
    $ensure              = absent,
    $is_role             = false,
    $role_users          = [],
    $delete_role_users   = [],
    $user_options        = [],
) { 

...
            if $is_role {
...
                if ! empty ( $role_users ) {
                    $local_role_users = suffix ( $role_users,
"_${username}" )

                    users::role_user_keys { $local_role_users:
                        ensure => present,
                        key_options => $user_options,
                        is_role => $is_role,
                    }
                }
            }

...
}
-------------------------------------------
define users::role_user_keys (
    $ensure   = absent,
    $key_options  = [],
    $is_role  = true,
) { 
    include ''stdlib''

    $foo = split ( $title, ''_'' )
    $the_user = $foo[0]
    $the_role = $foo[1]

    users::restricted_ssh_user { $title:
        username    => $the_user,
        ensure      => $ensure,
        ssh_options => $key_options,
        role_name   => $the_role,
        is_role     => $is_role,
    } 
}
-------------------------------------------
define users::restricted_ssh_user (
    $username,
    $ensure   = absent,
    $ssh_options  = [],
    $is_role  = true,
    $role_name,
) { 
    include stdlib 

    case $is_role {
        ‘roleA''   : {
            $command_string = “/usr/local/${role_name}/local/bin/commandA
--user=${username}"
            $canned_options = [
                ''no-port-forwarding'',
                ''no-agent-forwarding'',
                ''no-X11-forwarding'',
                ''no-pty'',
            ]

            $option_string = "command=\"${command_string}\""
            $local_options = concat ( $canned_options, [$option_string] )
        }
        ‘roleB''   : {
            $command_string = “/usr/local/${role_name}/bin/commandB
--user=${username}"
            $canned_options = [
                ''no-port-forwarding'',
                ''no-X11-forwarding'',
                ''no-pty'',
            ]

            $option_string = "command=\"${command_string}\""
            $local_options = concat ( $canned_options, [$option_string] )

        }
        ''admin'' : {
            $local_options = $ssh_options
        }
        default : {
            $linkit = link
            $command_string = "${homedir}/bin/ssh_${role_name}_${username}”

            $option_string = "command=\"${command_string}\""
            $local_options = concat ( $ssh_options, [$option_string] )
        }
    }

    ssh_authorized_key {
"${username}_rsa_key_for_${role_name}_${is_role}":
         ensure => $ensure,
         key => file ( "/etc/puppet/config/keys/${username}-rsa" ),
         type => ''ssh-rsa'',
         user => $role_name,
         options => $local_options, 
    }

    ssh_authorized_key {
"${username}_dss_key_for_${role_name}_${is_role}":
         ensure => $ensure,
         key => file ( "/etc/puppet/config/keys/${username}-dss" ),
         type => ''ssh-dss'',
         user => $role_name,
         options => $local_options, 
    }

}

The problem is in how I manipulate the array of option parameters as I go down
the levels ( user_options , key_options , ssh_options ) and the contents of the
array created with the “concat” function persists and accumulates (snowballs!!)
at the bottom level as the "users::restricted_ssh_user" defined type
in instantiated many times.  The parameters were all called “options” at one
point, and I made them different, hoping that would work.

Is there a way to isolate the variables declared in a defined type instance ?
Or do I need to trash this design and start over ?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

jcbollinger

2013-Aug-14 18:13 UTC

head link

[Puppet Users] Re: Problem with variable scope in Defined Types

On Tuesday, August 13, 2013 9:59:22 PM UTC-5, Ygor
wrote:>
> I have a problem with variables in defined types. 
>
> I have dug a hole consisting of three levels of nested defines as in 
> ------------------------------------------- 
> class users { 
> ... 
> } 
> ------------------------------------------- 
> define users::useraccount ( 
>     $username            = $name, 
>     $uid, 
>     $info, 
>     $ingroups            = [], 
>     $dotfiles            = [], 
>     $userhome            = "/home/${name}", 
>     $ensure              = absent, 
>     $is_role             = false, 
>     $role_users          = [], 
>     $delete_role_users   = [], 
>     $user_options        = [], 
> ) { 
>
> ... 
>             if $is_role { 
> ... 
>                 if ! empty ( $role_users ) { 
>                     $local_role_users = suffix ( $role_users, 
> "_${username}" ) 
>
>                     users::role_user_keys { $local_role_users: 
>                         ensure => present, 
>                         key_options => $user_options, 
>                         is_role => $is_role, 
>                     } 
>                 } 
>             } 
>
> ... 
> } 
> ------------------------------------------- 
> define users::role_user_keys ( 
>     $ensure   = absent, 
>     $key_options  = [], 
>     $is_role  = true, 
> ) { 
>     include ''stdlib'' 
>
>     $foo = split ( $title, ''_'' ) 
>     $the_user = $foo[0] 
>     $the_role = $foo[1] 
>
>     users::restricted_ssh_user { $title: 
>         username    => $the_user, 
>         ensure      => $ensure, 
>         ssh_options => $key_options, 
>         role_name   => $the_role, 
>         is_role     => $is_role, 
>     } 
> } 
> ------------------------------------------- 
> define users::restricted_ssh_user ( 
>     $username, 
>     $ensure   = absent, 
>     $ssh_options  = [], 
>     $is_role  = true, 
>     $role_name, 
> ) { 
>     include stdlib 
>
>     case $is_role { 
>         ‘roleA''   : { 
>             $command_string = “/usr/local/${role_name}/local/bin/commandA 
> --user=${username}" 
>             $canned_options = [ 
>                 ''no-port-forwarding'', 
>                 ''no-agent-forwarding'', 
>                 ''no-X11-forwarding'', 
>                 ''no-pty'', 
>             ] 
>
>             $option_string =
"command=\"${command_string}\""
>             $local_options = concat ( $canned_options, [$option_string] ) 
>         } 
>         ‘roleB''   : { 
>             $command_string = “/usr/local/${role_name}/bin/commandB 
> --user=${username}" 
>             $canned_options = [ 
>                 ''no-port-forwarding'', 
>                 ''no-X11-forwarding'', 
>                 ''no-pty'', 
>             ] 
>
>             $option_string =
"command=\"${command_string}\""
>             $local_options = concat ( $canned_options, [$option_string] ) 
>
>         } 
>         ''admin'' : { 
>             $local_options = $ssh_options 
>         } 
>         default : { 
>             $linkit = link 
>             $command_string = 
> "${homedir}/bin/ssh_${role_name}_${username}” 
>
>             $option_string =
"command=\"${command_string}\""
>             $local_options = concat ( $ssh_options, [$option_string] ) 
>         } 
>     } 
>
>     ssh_authorized_key { 
> "${username}_rsa_key_for_${role_name}_${is_role}": 
>          ensure => $ensure, 
>          key => file (
"/etc/puppet/config/keys/${username}-rsa" ),
>          type => ''ssh-rsa'', 
>          user => $role_name, 
>          options => $local_options, 
>     } 
>
>     ssh_authorized_key { 
> "${username}_dss_key_for_${role_name}_${is_role}": 
>          ensure => $ensure, 
>          key => file (
"/etc/puppet/config/keys/${username}-dss" ),
>          type => ''ssh-dss'', 
>          user => $role_name, 
>          options => $local_options, 
>     } 
>
> } 
>
> The problem is in how I manipulate the array of option parameters as I go 
> down the levels ( user_options , key_options , ssh_options ) and the 
> contents of the array created with the “concat” function persists and 
> accumulates (snowballs!!) at the bottom level as the 
> "users::restricted_ssh_user" defined type in instantiated many
times.  The
> parameters were all called “options” at one point, and I made them 
> different, hoping that would work. 
>
> Is there a way to isolate the variables declared in a defined type 
> instance ? 
> Or do I need to trash this design and start over ?


I''m not sure I understand the problem(s).

With respect to the $local_options arrays generated via concat(), these are 
each assigned as resource parameters, so Puppet must retain them all until 
it finishes compiling the catalog.  The point appears to be that the 
options may vary from user to user, so if you have many users then many of 
these option strings must accumulate during catalog compilation.  How do 
you imagine it could be different?

However, it does look like in some cases you are creating temporary arrays 
that you do not need.  It could be that Puppet retains these for the 
duration of catalog compilation, even though it shouldn''t need to do. 
If
that''s the case, then you might be able to lighten the load a bit by 
dropping the temporaries.  For instance, you could avoid the 
$canned_options, $command_string, and $option_string variables in the 
''roleA'' case by writing the option array like this:

$local_options = [ 
  ''no-port-forwarding'', 
  ''no-agent-forwarding'', 
  ''no-X11-forwarding'', 
  ''no-pty'', 
  "command=\"/usr/local/${role_name}/local/bin/commandA 
--user=${username}\""
] 

I don''t know whether that will really gain you anything, though,
especially
inasmuch as you cannot do the same for the default case.

You might also achieve a bit of savings by merging 
users::restricted_ssh_user back into users::role_user_keys.  As you present 
those, I don''t see what you gain by separating them.

Ultimately, though, I don''t think your option arrays grossly outweigh 
everything else declared per-user.  If you are experiencing a capacity 
problem, then you may need to take an altogether different approach, such 
as managing  ~/.ssh/authorized_keys as a whole, via a template, instead of 
declaring thousands of Ssh_authorized_key resources.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Dan White

2013-Aug-14 23:39 UTC

head link

Re: [Puppet Users] Re: Problem with variable scope in Defined Types

On Aug 14, 2013, at 2:13 PM, jcbollinger wrote:> On Tuesday, August 13, 2013 9:59:22 PM UTC-5, Ygor wrote:
> I have a problem with variables in defined types. 
> 
> I have dug a hole consisting of three levels of nested defines as in 
> 
> ------- snip ------------------------------------ 
> 
> The problem is in how I manipulate the array of option parameters as I go
down the levels ( user_options , key_options , ssh_options ) and the contents of
the array created with the “concat” function persists and accumulates
(snowballs!!) at the bottom level as the "users::restricted_ssh_user"
defined type in instantiated many times.  The parameters were all called
“options” at one point, and I made them different, hoping that would work.
> 
> Is there a way to isolate the variables declared in a defined type instance
?
> Or do I need to trash this design and start over ?
> 
> 
> I''m not sure I understand the problem(s).
> 
> With respect to the $local_options arrays generated via concat(), these are
each assigned as resource parameters, so Puppet must retain them all until it
finishes compiling the catalog.  The point appears to be that the options may
vary from user to user, so if you have many users then many of these option
strings must accumulate during catalog compilation.  How do you imagine it could
be different?
> 
> However, it does look like in some cases you are creating temporary arrays
that you do not need.  It could be that Puppet retains these for the duration of
catalog compilation, even though it shouldn''t need to do.  If
that''s the case, then you might be able to lighten the load a bit by
dropping the temporaries.  For instance, you could avoid the $canned_options,
$command_string, and $option_string variables in the ''roleA''
case by writing the option array like this:
> 
> $local_options = [ 
>   ''no-port-forwarding'', 
>   ''no-agent-forwarding'', 
>   ''no-X11-forwarding'', 
>   ''no-pty'', 
>   "command=\"/usr/local/${role_name}/local/bin/commandA
--user=${username}\""
> ] 
> 
> I don''t know whether that will really gain you anything, though,
especially inasmuch as you cannot do the same for the default case.
> 
> You might also achieve a bit of savings by merging
users::restricted_ssh_user back into users::role_user_keys.  As you present
those, I don''t see what you gain by separating them.
> 
> Ultimately, though, I don''t think your option arrays grossly
outweigh everything else declared per-user.  If you are experiencing a capacity
problem, then you may need to take an altogether different approach, such as
managing  ~/.ssh/authorized_keys as a whole, via a template, instead of
declaring thousands of Ssh_authorized_key resources.
> 
> 
> John
Thanks for staring your opinion and thoughts, John.

I think part of the problem is feature creep.

It started with just creating user accounts
Then I added ssh keys
Then I added role accounts 
Then I added ssh keys for role accounts

The whole mess needs refactoring.

I will try your idea of eliminating the temporaries and see how far I get.

One thing I found about the ssh_authorized_key type is that is does not work
with :
	resource { ‘ssh_authorized_key’: purge => true, }
so I have to explicitly un-manage the keys

When you say "managing  ~/.ssh/authorized_keys as a whole, via a template”,
do you mean as a separate class rather than as a facet of my “users” class ?


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

jcbollinger

2013-Aug-15 14:26 UTC

head link

Re: [Puppet Users] Re: Problem with variable scope in Defined Types

On Wednesday, August 14, 2013 6:39:05 PM UTC-5, Ygor wrote:
[...]
> One thing I found about the ssh_authorized_key type is that is does not 
> work with :
> resource { ‘ssh_authorized_key’: purge => true, }
> so I have to explicitly un-manage the keys
>
>
Yes, purging via the Resources meta-resource works only for types that can 
enumerate all their instances on the target node.  Ssh_authorized_key is 
not one of those.  Neither is File, though that type provides its own, more 
narrowly-scoped purging mechanism.

 
> When you say "managing  ~/.ssh/authorized_keys as a whole, via a 
> template”, do you mean as a separate class rather than as a facet of my 
> “users” class ?
>
>
>Not necessarily. I am merely suggesting replacing the (2 * number_of_roles) 
Ssh_authorized_key resources for each user with a single File resource per 
user, with content driven by a template.  Perhaps you could also drop the 
intermediate and bottom defined-type instances for each user.  If your 
problems really arise from the memory consumed by all those 
Users::Restricted_ssh_user instances and the Ssh_authorized_keys they 
declare, then I think it will be a big win to replace them with a single 
File per user, even if the file content is comparatively large.  As an 
added bonus, you would no longer need to worry about explicitly removing 
authorized keys that should no longer be present.

I don''t care where you hang the File resource(s), though it looks like
it
might fit nicely into your users::useraccount definition.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Puppet users - Aug 2013 - Problem with variable scope in Defined Types

[Puppet Users] Problem with variable scope in Defined Types

[Puppet Users] Re: Problem with variable scope in Defined Types

Re: [Puppet Users] Re: Problem with variable scope in Defined Types

Re: [Puppet Users] Re: Problem with variable scope in Defined Types