Puppet users - Apr 2009 - How to use definitions with common resources?

Hi all,

I recently came across the thread titled "A simple function to load
external class attributes". The idea sounds good.

I''m bumping into a similar but I think different problem.

I need to install (via custom scripts) a software package using various 
parameters such as install location, software version, and environment.

The initial setup required the following tasks to be executed.
1. Create local directory to download install script.
2. Download install script from puppet master.
3. Run install script providing various parameters unless software is installed.

This worked fine.

It''s now necessary to multiple instances of this software package at
once using different parameters.

My initial idea was to create a defintion and provide the appropriate parameters
for each install.

This doesn''t work as the install script which is part of the definition
and needs
to be downloaded is common within differently defined instances. Thus puppet
complains about a duplicate class definition.

The current code (sanitised for this list) looks as follows:

define software_installation (
         $software_install_dir     =
''/path/to/software/installation'',
         $software_version = ''X.Y.Z'',
         $software_env     = ''prod''
) {
         case $software_env {
                 ''prod'': {}
                 ''dev'':  {}
                 default: { fail( "software_env not defined. Should be
''dev'' or ''prod''" ) }
         }

         $download_dir      = ''/root/tmp''
         $install_script = ''install_script_name''

         file { "$download_dir":
                 owner   => root,
                 group   => root,
                 mode    => 700,
                 ensure  => directory,
         }

         file { "$download_dir/$install_script":
                 backup  => false,
                 owner   => root,
                 group   => root,
                 mode    => 700,
                 source  =>
"puppet:///source_location/$install_script",
                 require => File["$download_dir"]
         }

         exec { "$install_script/$software_env":
                 command => "$download_dir/$install_script ...
appropriate parameters ...",
                 onlyif  => "... test if software not installed
...",
                 require => File["$download_dir/$install_script"]
         }
 }

and is called as follows:

software_installation { ''/path/to/install/prod'':
	software_install_dir     => ''/path/to/install/prod'',
	software_version => ''1.2.3'',
	software_env     => ''prod''
}

software_installation { ''/path/to/install/dev'':
	software_install_dir     => ''/path/to/install/dev'',
	software_version => ''1.3.1'',
	software_env     => ''dev''
}

The error I get is

err: Could not retrieve catalog: Puppet::Parser::AST::Resource failed with error
ArgumentError: Duplicate definition: File[/root/tmp] is already defined in file
//etc/puppet/services/xxxx/manifests/init.pp at line 83; cannot redefine at
//etc/puppet/services/xxxx/manifests/init.pp:83 on node dev01.example.com

I''d rather NOT download the install script unless I want to install
the software and of course the install script is designed to install
multiple versions (if necessary) at once.  Additionally removing the
2 file definitions outside of the defintion will also have issues with
variable scoping, thus making the solution less clean.

So how should I best approach this in puppet?

Thanks for any thoughts.

Simon

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Replace this line:
>          $download_dir      = ''/root/tmp''   
with:

        #NOTE: generate is executed on the puppetmaster.
         $download_dir =
generate("/usr/bin/env","bash",''-c'',"/bin/mktemp")


With this every time your definition get used it will choose an random 
folder in your /tmp.


I hope this works?

Philipp

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi,
> I''d rather NOT download the install script unless I want to
install
> the software and of course the install script is designed to install
> multiple versions (if necessary) at once.  Additionally removing the
> 2 file definitions outside of the defintion will also have issues with
> variable scoping, thus making the solution less clean.
I can guess 2 different approaches:
1- Moving the 2 file definitions in a class, included by the define.
It''s exactly whsat you don''t like, but since the variables
used in
these 2 file types can be considered "local" and are not used in the
define, you should be able to do that without problems
2- Use the software_install_dir variable in the name of the file
types, and add the fixed path, something like:
         file { "$download_dir-$software_install_dir":
                 owner   => root,
                 group   => root,
                 mode    => 700,
                 ensure  => directory,
                 path => $download_dir,
         }

         file { "$download_dir/$install_script-$software_install_dir":
                 backup  => false,
                 owner   => root,
                 group   => root,
                 mode    => 700,
                 source  => "puppet:///source_location/
$install_script",
                 require => File["$download_dir"],
                 path => $download_dir/$install_script,
         }

I personally find the first choice much better.

My 2 cents

Alessandro
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

lab42.it@gmail.com (Macno) writes:
> > I''d rather NOT download the install script unless I want to
install
> > the software and of course the install script is designed to install
> > multiple versions (if necessary) at once.  Additionally removing the
> > 2 file definitions outside of the defintion will also have issues with
> > variable scoping, thus making the solution less clean.
> 
> I can guess 2 different approaches:
> 1- Moving the 2 file definitions in a class, included by the define.
> It''s exactly whsat you don''t like, but since the
variables used in
> these 2 file types can be considered "local" and are not used in
the
> define, you should be able to do that without problems
Well using a separate include would work. I''ll try that. The variable
scoping does bother me as doing what you suggest means I have to be
consistent with the naming in 2 different classes. This may be
unavoidable at the moment, but I don''t like it.
> 2- Use the software_install_dir variable in the name of the file
> types, and add the fixed path, something like:
>          file { "$download_dir-$software_install_dir":
>                  owner   => root,
>                  group   => root,
>                  mode    => 700,
>                  ensure  => directory,
>                  path => $download_dir,
>          }
> 
>          file {
"$download_dir/$install_script-$software_install_dir":
>                  backup  => false,
>                  owner   => root,
>                  group   => root,
>                  mode    => 700,
>                  source  => "puppet:///source_location/
> $install_script",
>                  require => File["$download_dir"],
>                  path => $download_dir/$install_script,
>          }
> 
> I personally find the first choice much better.
Yes. This is just a workarond.

I don''t really see why with a definition that has a
"redefine" which
is _identical_ that it should worry so much. Having a common file on the system
defined exactly the same way shouldn''t be a problem. Even if I need to
add a new attribute like:

                accept_duplicate_resource => true,

would be fine. I confirm to puppet that this specific definition may
lead to identical duplicates and if they occur the duplicate
definitions can be ignored.

Simon

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Replace this line:
>          $download_dir      = ''/root/tmp''
with:

       #NOTE: generate is executed on the puppetmaster.
        $download_dir =
generate("/usr/bin/env","bash",''-c'',"/bin/
mktemp")


With this every time your definition get used it will choose an random
folder in your /tmp.


I hope this works?

Philipp

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Replace this line:
>          $download_dir      = ''/root/tmp''
with:

       #NOTE: generate is executed on the puppetmaster.
        $download_dir =
generate("/usr/bin/env","bash",''-c'',"/bin/
mktemp")


With this every time your definition get used it will choose an random
folder in your /tmp.


I hope this works?

Philipp

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

philipp.hanselmann@gmail.com (puppet) writes:
> Replace this line:
> >          $download_dir      = ''/root/tmp''
> with:
> 
>        #NOTE: generate is executed on the puppetmaster.
>         $download_dir =
generate("/usr/bin/env","bash",''-c'',"/bin/
> mktemp")
> 
> With this every time your definition get used it will choose an random
> folder in your /tmp.
Thanks. The idea is good, but it doesn''t quite work.

1. The script unfortunately contains account information
   user/passwords for the application''s initial installation. Hence
   /root/tmp is better as only root users can access this directory.

2. As this is a "package" and the version may need to change the
   installer is actually called each time puppet is invoked to check
   if the version installed is the right version and to perform the
   upgrade if needed. (The script knows how to install/upgrade and
   check the current installed version.) So downloading into a random
   temp directory doesn''t help. In fact I deliberately don''t
remove
   the script after installation as it will be called by puppet 30
   minutes later in the next run. I could split the "check version"
   and "install script" parts into 2 but then I have 2 things to
   manage rather than 1, so I don''t see this as a win. Normally the
   package manager is generic so it doesn''t matter if it''s
code is
   kept on the client all the time. It''s used for multiple packages so
   that''s fine. With custom installers of course this is not true.

I''ve not seen others mention this sort of issue or show how they solve
it on this list so perhaps I''m not doing things the puppet way?

In any case thanks for the suggestion. If the downloaded install
package is removed after installation then it''s better to use a random
location. Adjusting the umask appropriately will also ensure that
other users are not able to access the file''s contents.

Simon

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Simon J Mudd schrieb:
> philipp.hanselmann@gmail.com (puppet) writes:
>
>   
>> Replace this line:
>>     
>>>          $download_dir      = ''/root/tmp''
>>>       
>> with:
>>
>>        #NOTE: generate is executed on the puppetmaster.
>>         $download_dir =
generate("/usr/bin/env","bash",''-c'',"/bin/
>> mktemp")
>>
>> With this every time your definition get used it will choose an random
>> folder in your /tmp.
>>     
>
> Thanks. The idea is good, but it doesn''t quite work.
>
> 1. The script unfortunately contains account information
>    user/passwords for the application''s initial installation.
Hence
>    /root/tmp is better as only root users can access this directory.
>
>   
1.) With
    mktemp -p /root

this will choose a random folder name begining with /root


2. ) permissions?
Your puppet code:

file { "$download_dir":
                 owner   => root,
                 group   => root,
                 mode    => 700,
                 ensure  => directory,
         }

takes already care that only root has access of new generated files in 
the folder $download_dir ?
At least my tests confirmed  that ...


3.) Clean-up of the random folder?

may something like this helps?

exec { "$install_script/$software_env":
                 command => "$download_dir/$install_script ...
appropriate parameters ...  && rm Rf $download_dir",
                 onlyif  => "... test if software not installed
...",
                 require => File["$download_dir/$install_script"]
         }



Philipp

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

philipp.hanselmann@gmail.com (philipp Hanselmann) writes:
> Simon J Mudd schrieb:
...
> >>        #NOTE: generate is executed on the puppetmaster.
> >>         $download_dir =
generate("/usr/bin/env","bash",''-c'',"/bin/
> >> mktemp")
> >>
> >> With this every time your definition get used it will choose an
random
> >> folder in your /tmp.
> >>     
> >
> > Thanks. The idea is good, but it doesn''t quite work.
> >
> > 1. The script unfortunately contains account information
> >    user/passwords for the application''s initial installation.
Hence
> >    /root/tmp is better as only root users can access this directory.
> >
> 1.) With
>     mktemp -p /root
Sure. If I''d wanted a random directory I would have done something like
that.
The point is that puppet is SLOW at copying files from the master to the client.
Anything that''s not a small file takes a while and syncing a directory
like
this  (with say 50-100 scripts) gets very tedious. So I''m reluctant to:

copy from master, run, delete    every 30 minutes on several hundred boxes.
Hence the optimisation of leaving the install script in one place and using
it to decide if any work needs doing. This is just a wrapper around the
vendor provided install binary which is NOT copied with puppet as puppet is
too slow (and the binary is about 20MB).
> 3.) Clean-up of the random folder?
> 
> may something like this helps?
> 
> exec { "$install_script/$software_env":
>                  command => "$download_dir/$install_script ...
appropriate parameters ...  && rm Rf $download_dir",
>                  onlyif  => "... test if software not installed
...",
>                  require =>
File["$download_dir/$install_script"]
>          }
Not quite. You''d need to have a separate Exec run dependent on the
File["$download_dir/$install_script"] as the command shown is not
always run (see onlyif) and thus the rm would not be triggered.

In many ways a post_exec or cleanup => ....  would be handy which _is_
always run even if command isn''t.

And remember part of the problem of this original post was that the
setup of the temporary directory fails if you have 2 similar
installations requiring the same base directory and installation
script. That''s what bothers me (and I''d like to solve) more
than the
details of the single install problem which works fine for me.

Your suggestions are valid but my implementation is different for the
reasons stated.

Simon

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---