Puppet users - Apr 2013 - How to pass puppet/hiera veriable to external script ? Do I need to ?

Hi,

I have a problem with running puppet with hiera based data. I am trying to 
cleanup
user accounts on a range of servers (to have the same uid/gid). When I 
create existing account
but with different uid/gid it doesn''t change files group ownership so I
need to
run a script which cleanups after cleanup ;-)

I use the following code:
---
:backends:
  - yaml

:hierarchy:
  - nodes/%{clientcert}
  - virtual/%{virtual}
  - common

:yaml:
# datadir is empty here, so hiera uses its defaults:
# - /var/lib/hiera on *nix
# - %CommonAppData%\PuppetLabs\hiera\var on Windows
# When specifying a datadir, make sure the directory exists.
  #:datadir: /etc/puppet/hieradata
  :datadir: /etc/puppet/environments/%{environment}/hieradata
:merge_behavior: deeper

/etc/puppet/environments/env1/manifests/site.pp
node default {
        hiera_include ( "classes", [] )
}

/etc/puppet/environments/env1/hieradata/common.yaml
classes:
  - accounts

/etc/puppet/environments/env1/modules/accounts/manifests/init.pp
class accounts::users (
        $accounts,
        $groups,
)
{
        create_resources(group, $groups)
        create_resources(user, $accounts, { notify => 
Exec[''clean_up_accounts''] })
}

class accounts::cleanup
{
        exec { "clean_up_accounts":
                command => "/script/to/cleanup",
                refreshonly => true,
        }
}

class accounts {
        include accounts::users, accounts::cleanup
}

Is it possible to run this script ("/script/to/cleanup") for each
created
user (so the script doesn''t have to cope with whole /home
tree structure) with its home (taken from puppet/hiera) as a parameter ? 
How can I pass puppet/hiera parameters to external script ?
Or maybe there is another, more puppet-native way to do that ?


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On 04/19/2013 03:56 PM, ForumUser wrote:
> Is it possible to run this script ("/script/to/cleanup") for each
> created user (so the script doesn''t have to cope with whole /home
> tree structure) with its home (taken from puppet/hiera) as a parameter ?
Yes, assuming you use a defined type such as this:

define repair_user($uid, $gid) {
  include repair_script # <- deploys the cleanup script

  user { $name: uid => $uid, gid => $gid, ... }

  exec { "/script/to/cleanup $name":
    refreshonly => true,
    subscribe => User[$name],
}

This passes the username to the script, and runs only after changing uid
and/or gid.

Note that it''s paramount to include the type instance''s $name
in the
title of the exec resource, so that each user''s respective invocation
ends up with a unique exec title. Otherwise you will end up with
duplicate declarations.

Finally, note that refreshonly/subscribe is a slightly unclean design.
If possible, define an appropriate "onlyif" constraint and replace
refreshonly+subscribe with onlyif+require.
> How can I pass puppet/hiera parameters to external script ?
See above, just use a variable in the exec command.
> Or maybe there is another, more puppet-native way to do that ?
I believe that your general approach is quite appropriate.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Felix,

thank you for your ideas. I am not sure if I understand correctly but
in my code there are two lines which creates all the groups and users:

        create_resources(group, $groups)
        create_resources(user, $accounts, { notify => 
Exec[''clean_up_accounts''] })

Given your code where in my puppet class I should include the call of the 
following line:
define repair_user($uid, $gid) { 
?

And what is "repair_script" in the following line:
include repair_script
?
What should the class/package "repair_script" should include ?

Thanks


On Sunday, 21 April 2013 18:12:25 UTC+1, Felix.Frank
wrote:
>
> On 04/19/2013 03:56 PM, ForumUser wrote: 
> > Is it possible to run this script ("/script/to/cleanup") for
each
> > created user (so the script doesn''t have to cope with whole
/home
> > tree structure) with its home (taken from puppet/hiera) as a parameter
?
>
> Yes, assuming you use a defined type such as this: 
>
> define repair_user($uid, $gid) { 
>   include repair_script # <- deploys the cleanup script 
>
>   user { $name: uid => $uid, gid => $gid, ... } 
>
>   exec { "/script/to/cleanup $name": 
>     refreshonly => true, 
>     subscribe => User[$name], 
> } 
>
> This passes the username to the script, and runs only after changing uid 
> and/or gid. 
>
> Note that it''s paramount to include the type instance''s
$name in the
> title of the exec resource, so that each user''s respective
invocation
> ends up with a unique exec title. Otherwise you will end up with 
> duplicate declarations. 
>
> Finally, note that refreshonly/subscribe is a slightly unclean design. 
> If possible, define an appropriate "onlyif" constraint and
replace
> refreshonly+subscribe with onlyif+require. 
>
> > How can I pass puppet/hiera parameters to external script ? 
>
> See above, just use a variable in the exec command. 
>
> > Or maybe there is another, more puppet-native way to do that ? 
>
> I believe that your general approach is quite appropriate. 
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

I have just realised - should it look like this ?

        create_resources(repair_user, $accounts, { notify =>
Exec[''...''] })

?

On Monday, 22 April 2013 09:54:07 UTC+1, ForumUser
wrote:
>
> Hi Felix,
>
> thank you for your ideas. I am not sure if I understand correctly but
> in my code there are two lines which creates all the groups and users:
>
>         create_resources(group, $groups)
>         create_resources(user, $accounts, { notify => 
> Exec[''clean_up_accounts''] })
>
> Given your code where in my puppet class I should include the call of the 
> following line:
> define repair_user($uid, $gid) { 
> ?
>
> And what is "repair_script" in the following line:
> include repair_script
> ?
> What should the class/package "repair_script" should include ?
>
> Thanks
>
>
> On Sunday, 21 April 2013 18:12:25 UTC+1, Felix.Frank wrote:
>>
>> On 04/19/2013 03:56 PM, ForumUser wrote: 
>> > Is it possible to run this script ("/script/to/cleanup")
for each
>> > created user (so the script doesn''t have to cope with
whole /home
>> > tree structure) with its home (taken from puppet/hiera) as a
parameter
>> ? 
>>
>> Yes, assuming you use a defined type such as this: 
>>
>> define repair_user($uid, $gid) { 
>>   include repair_script # <- deploys the cleanup script 
>>
>>   user { $name: uid => $uid, gid => $gid, ... } 
>>
>>   exec { "/script/to/cleanup $name": 
>>     refreshonly => true, 
>>     subscribe => User[$name], 
>> } 
>>
>> This passes the username to the script, and runs only after changing
uid
>> and/or gid. 
>>
>> Note that it''s paramount to include the type
instance''s $name in the
>> title of the exec resource, so that each user''s respective
invocation
>> ends up with a unique exec title. Otherwise you will end up with 
>> duplicate declarations. 
>>
>> Finally, note that refreshonly/subscribe is a slightly unclean design. 
>> If possible, define an appropriate "onlyif" constraint and
replace
>> refreshonly+subscribe with onlyif+require. 
>>
>> > How can I pass puppet/hiera parameters to external script ? 
>>
>> See above, just use a variable in the exec command. 
>>
>> > Or maybe there is another, more puppet-native way to do that ? 
>>
>> I believe that your general approach is quite appropriate. 
>>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

I am trying to implement this idea but cannot find what is wrong.
Given the following code (I would like to get all possible data from hiera 
files) :

class accounts::users ( $accounts, $groups)
{
        create_resources(group, $groups)
        create_resources(''repair_user'', $accounts)
}

define repair_user ($uid, $gid)
{
        user { $name:
                uid => $uid,
                home => $home,
                gid => $gid
        }

        exec { "/usr/local/bin/cleanup.sh $name":
                refreshonly => true,
                subscribe => User[$name],
        }
}

class accounts {
        include accounts::users
}

and hiera file common.yaml:

accounts::users::accounts:
  user_1:
    ensure: present
    home: /home/user_1
    shell: /bin/bash
    uid: 4018
    gid: sysadmin

accounts::users::groups:
  sysadmin:
    ensure: present
    gid: 4000


I get the following message:
puppet agent --test --server pmaster --environment sas
Info: Retrieving plugin
Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: 
Invalid parameter ensure on node node10
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

What is wrong with it ?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Hi,

On 04/22/2013 01:17 PM, ForumUser wrote:
> What is wrong with it ?
Very good approach!

The last remaining problem is that your repair_user type has no
parameters for ensure, home and shell. create_resources will want to
pass those parameters.

Just add them to your type and pass them on to the user resource. You
even already prepared this for the $home parameter:

define repair_user ($ensure = present, $uid, $gid, $home
"/home/$name", shell = "/bin/bash") {
  ...
}

Notice the defaults I added so the type is more flexible and won''t
require each parameter. Feel free to change or omit those as you see fit.

HTH,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Monday, 22 April 2013 14:22:10 UTC+1, Felix.Frank
wrote:
>
> Hi, 
>
> On 04/22/2013 01:17 PM, ForumUser wrote: 
> > What is wrong with it ? 
>
> Very good approach! 
>
> The last remaining problem is that your repair_user type has no 
> parameters for ensure, home and shell. create_resources will want to 
> pass those parameters. 
>
> Just add them to your type and pass them on to the user resource. You 
> even already prepared this for the $home parameter: 
>
> define repair_user ($ensure = present, $uid, $gid, $home = 
> "/home/$name", shell = "/bin/bash") { 
>   ... 
> } 
>
> Notice the defaults I added so the type is more flexible and won''t
> require each parameter. Feel free to change or omit those as you see fit. 
>
> HTH, 
> Felix 
>
Hi Felix,

now it works - thanks a lot !
However it creates account in the /etc/passwd but doesn''t create users
home
directory
despite the fact I have "managehome" set to ''true''.
Any clue ?

And how I can change the syntax to get all the defaults from yaml file ?
For example I would like to have sth like below:

accounts::users::defaults:
    ensure: present
    shell: /bin/bash
    managehome: true

accounts::users::accounts:
  user_1:
    home: /home/user_1
    uid: 4018
    gid: group1


Is it possible to have something like that ?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On 04/22/2013 03:55 PM, ForumUser wrote:
> However it creates account in the /etc/passwd but doesn''t create
users
> home directory
> despite the fact I have "managehome" set to
''true''. Any clue ?
No, this shouldn''t happen. There must be some sort of error.

You may get lucky and find what''s wrong by examining your catalog in
YAML form. On the agent node, look in /var/lib/puppet/client_yaml/.
> accounts::users::defaults:
>     ensure: present
>     shell: /bin/bash
>     managehome: true
> 
> accounts::users::accounts:
>   user_1:
>     home: /home/user_1
>     uid: 4018
>     gid: group1
> 
> 
> Is it possible to have something like that ?
Not implicitly, I think, but with a sleight of hand you should be able to:

define repair_user (
   $ensure = hiera("accounts::users::defaults")["ensure"],
   $uid, $gid,
   $shell = hiera("accounts::users::defaults")["shell"],
   ...)

Untested, but you get the idea.

It may also be possible to do this on DSL level instead, in the scope of
the call to create_resources:

Repair_user {
  ensure => present,
  shell => "/bin/bash",
  ...
}

The keyword being "resource defaults". I''m not sure whether
those apply
to "create"d resources.

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Monday, 22 April 2013 15:04:02 UTC+1, Felix.Frank
wrote:
>
> On 04/22/2013 03:55 PM, ForumUser wrote: 
> > However it creates account in the /etc/passwd but doesn''t
create users
> > home directory 
> > despite the fact I have "managehome" set to
''true''. Any clue ?
>
> No, this shouldn''t happen. There must be some sort of error. 
>
> You may get lucky and find what''s wrong by examining your catalog
in
> YAML form. On the agent node, look in /var/lib/puppet/client_yaml/. 
>
There is nothing in it:
# ls -al /var/lib/puppet/client_yaml/
total 8
drwxr-x--- 2 root   root   4096 Apr 11 15:31 .
drwxr-x--- 8 puppet puppet 4096 Apr 11 15:32 ..

Also:
# hiera accounts::users::accounts environment=env1
{"user_1"=>{"home"=>"/home/user_1",
"uid"=>4018, "ensure"=>"present",
"managehome"=>true, "shell"=>"/bin/bash",
"gid"=>"group1"}}

And the logs:
# puppet agent --test --server pmaster --environment env1
Info: Retrieving plugin
Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb
Info: Caching catalog for node
Info: Applying configuration version ''1366637930''
Notice: 
/Stage[main]/accounts::Users/Repair_user[user_1]/User[user_1]/ensure: 
created
Info: /Stage[main]/accounts::Users/Repair_user[user_1]/User[user_1]: 
Scheduling refresh of Exec[/usr/local/bin/cleanup.sh user_1]
Notice: 
/Stage[main]/accounts::Users/Repair_user[user_1]/Exec[/usr/local/bin/cleanup.sh 
user_1]: Triggered ''refresh'' from 1 events
Notice: Finished catalog run in 3.00 seconds


 

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On 04/22/2013 04:45 PM, ForumUser wrote:
> 
> 
> On Monday, 22 April 2013 15:04:02 UTC+1, Felix.Frank wrote:
> 
>     On 04/22/2013 03:55 PM, ForumUser wrote:
>     > However it creates account in the /etc/passwd but doesn''t
create
>     users
>     > home directory
>     > despite the fact I have "managehome" set to
''true''. Any clue ?
> 
>     No, this shouldn''t happen. There must be some sort of error.
> 
>     You may get lucky and find what''s wrong by examining your
catalog in
>     YAML form. On the agent node, look in /var/lib/puppet/client_yaml/.
> 
> 
> There is nothing in it:
> # ls -al /var/lib/puppet/client_yaml/
> total 8
> drwxr-x--- 2 root   root   4096 Apr 11 15:31 .
> drwxr-x--- 8 puppet puppet 4096 Apr 11 15:32 ..
Huh, bummer.

How are you using puppet? Puppet agent --test?
> Also:
> # hiera accounts::users::accounts environment=env1
> {"user_1"=>{"home"=>"/home/user_1",
"uid"=>4018, "ensure"=>"present",
> "managehome"=>true,
"shell"=>"/bin/bash",
"gid"=>"group1"}}
Which reminds me, do have a look at the thread ''Puppet 3.1, Hiera and a
class parameter called "service"''.

You may be afflicted by a bug concerning puppet/hiera and boolean values.

HTH,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Monday, 22 April 2013 15:48:57 UTC+1, Felix.Frank
wrote:
>
> On 04/22/2013 04:45 PM, ForumUser wrote: 
> > 
> > 
> > On Monday, 22 April 2013 15:04:02 UTC+1, Felix.Frank wrote: 
> > 
> >     On 04/22/2013 03:55 PM, ForumUser wrote: 
> >     > However it creates account in the /etc/passwd but
doesn''t create
> >     users 
> >     > home directory 
> >     > despite the fact I have "managehome" set to
''true''. Any clue ?
> > 
> >     No, this shouldn''t happen. There must be some sort of
error.
> > 
> >     You may get lucky and find what''s wrong by examining your
catalog in
> >     YAML form. On the agent node, look in
/var/lib/puppet/client_yaml/.
> > 
> > 
> > There is nothing in it: 
> > # ls -al /var/lib/puppet/client_yaml/ 
> > total 8 
> > drwxr-x--- 2 root   root   4096 Apr 11 15:31 . 
> > drwxr-x--- 8 puppet puppet 4096 Apr 11 15:32 .. 
>
> Huh, bummer. 
>
> How are you using puppet? Puppet agent --test? 
>
On the node: 
puppet agent --test --server pmaster --environment env1

> > Also: 
> > # hiera accounts::users::accounts environment=env1 
> >
{"user_1"=>{"home"=>"/home/user_1",
"uid"=>4018, "ensure"=>"present",
> > "managehome"=>true,
"shell"=>"/bin/bash",
"gid"=>"group1"}}
>
> Which reminds me, do have a look at the thread ''Puppet 3.1, Hiera
and a
> class parameter called "service"''. 
>
> You may be afflicted by a bug concerning puppet/hiera and boolean values. 
>
>
Perhaps this is the bug :-(
I am trying to workaround it:

$mh = str2bool ("$accounts::users::accounts::managehome")
define repair_user ($uid, $gid, $ensure = present, $home =
"/home/$name",
$shell = "/bin/bash", $managehome = $mh )
{
        user { $name:
                uid => $uid,
                gid => $gid,
        }
...

but in my case it perhaps won''t work since:
# hiera accounts::users::accounts::managehome environment=env1
nil


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Monday, April 22, 2013 3:04:02 PM UTC+1, Felix.Frank
wrote:
>
> On 04/22/2013 03:55 PM, ForumUser wrote: 
> > However it creates account in the /etc/passwd but doesn''t
create users
> > home directory 
> > despite the fact I have "managehome" set to
''true''. Any clue ?
>
> No, this shouldn''t happen. There must be some sort of error. 
>
> You may get lucky and find what''s wrong by examining your catalog
in
> YAML form. On the agent node, look in /var/lib/puppet/client_yaml/. 
>
> > accounts::users::defaults: 
> >     ensure: present 
> >     shell: /bin/bash 
> >     managehome: true 
> > 
> > accounts::users::accounts: 
> >   user_1: 
> >     home: /home/user_1 
> >     uid: 4018 
> >     gid: group1 
> > 
> > 
> > Is it possible to have something like that ? 
>
> Not implicitly, I think, but with a sleight of hand you should be able to: 
>
> define repair_user ( 
>    $ensure =
hiera("accounts::users::defaults")["ensure"],
>    $uid, $gid, 
>    $shell =
hiera("accounts::users::defaults")["shell"],
>    ...) 
>
> Untested, but you get the idea. 
>
>
Felix,

But how this syntax would work in case when I want non-default shell for 
particular user ?
Since $shell is got from something like 
hiera("accounts::users::defaults")["shell"]
it would never get the shell (/bin/ksh) from the following hierarchy:

accounts::users::defaults: 
    ensure: present 
    shell: /bin/bash 
    managehome: true 

accounts::users::accounts: 
  user_1: 
    home: /home/user_1 
    shell: /bin/ksh
    uid: 4018 
    gid: group1 

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On 04/23/2013 11:22 AM, ForumUser wrote:
>     define repair_user (
>        $ensure =
hiera("accounts::users::defaults")["ensure"],
>        $uid, $gid,
>        $shell =
hiera("accounts::users::defaults")["shell"],
>        ...)
> 
>     Untested, but you get the idea.
> 
> 
> Felix,
> 
> But how this syntax would work in case when I want non-default shell for
> particular user ?
> Since $shell is got from something like
> hiera("accounts::users::defaults")["shell"]
> it would never get the shell (/bin/ksh) from the following hierarchy:
Honestly, I don''t see why not ;-)
> accounts::users::defaults:
>     ensure: present
>     shell: /bin/bash
>     managehome: true
> 
> accounts::users::accounts:
>   user_1:
>     home: /home/user_1
>     shell: /bin/ksh
>     uid: 4018
>     gid: group1
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.