Puppet users - Mar 2013 - Staging content from modules for temporary use by resources

Hello,

I would like to find a few to use files available in a module for the 
execution of a particular resource, without having to stage the file 
explicitly onto the client and possibly having to clean it up afterwards. 
(This could also apply to content coming from templates.)

The use-cases I have in mind are: running a SQL file (to set up some 
content in a database), extracting an archive and running a self-executable 
installer (as generated with makeself, for example).

Currently, one way to do this is to stage the file first and to make the 
resource (e.g. exec) using it depend on that file being set up:

    file { ''/tmp/setupdb.sql'' :
        source => "puppet:///modules/${module_name}/setupdb.sql",
        mode => 644,
        backup => false,
    }

    exec { ''Setup the database'':
        command => ''psql ${databasename} -f
/tmp/setupdb.sql'',
        onlyif => ''psql ... | grep ...'', # Some condition
to check whether
a table exists, for example.
        requires => File[''/tmp/setupdb.sql''],
    }

The problem with this is that it looks procedural, and that it also leaves 
a useless temporary file behind.

One way around this particular use case would be to have support for stdin 
input in exec (See: <http://projects.puppetlabs.com/issues/653>).
Something
like this would feel somewhat cleaner:

    exec { ''Setup the database'':
        command => ''psql ${databasename}'',
        stdin => "puppet:///modules/${module_name}/setupdb.sql",
        onlyif => ''psql ... | grep ...'', # Some condition
to check whether
a table exists, for example.
    }

Nevertheless, this would only address part of the issue of handling files 
temporary files, since this would only work for commands that can read 
their main content from stdin (e.g. mysql, psql, tar).


Another use case would be unzipping an archive into a specific directory:

   file { ''/tmp/myfavouriteblogengine.zip'':
        source => 
"puppet:///modules/${module_name}/myfavouriteblogengine.zip",
        mode => 644,
        backup => false,
    }

    exec { ''Setup the engine for vhost1'':
        command => ''cd /var/lib/www/vhost1 && unzip 
/tmp/myfavouriteblogengine.zip '',
        creates => ''/var/lib/www/vhost1/index.php'',
        requires => File[''/tmp/myfavouriteblogengine.zip''],
    }

The problem remains: there''s still a temporary file left over, and it 
doesn''t feel very declarative.

What are the typical patterns to deal with this kind of situation?

Packaging for the distribution isn''t necessarily viable: it requires a
fair
amount of additional setup, and not everything can be packaged easily 
(whatever the tools), since it would be difficult to install the same 
package multiple times for vhost1, vhost2, ... in cases similar to the 
example above.

Best wishes,

Bruno.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

I have been using the file, exec approach for my modules but transferring
big zip or tar files is a pain because they take up space, as you say, and
sometimes the transfer times out when the files are really big.

Today I started using Jordan Sissel''s fpm [1] to build multiple rpms
for
each tomcat instance with a different prefix - something like:

$ sudo yum install ruby-devel rpmdevtools
$ sudo gem install fpm
$ tar -zxf apache-tomcat-7.0.37.tar.gz
$ for USER in tomcat1 tomcat2 tomcat3; do fpm -t rpm -s dir --prefix
/apps/$USER -C apache-tomcat-7.0.37 -n $USER -v 7.0.37 --rpm-user $USER
--rpm-group tomcat $(ls apache-tomcat-7.0.37); done
Created rpm {:path=>"tomcat1-7.0.37-1.x86_64.rpm"}
Created rpm {:path=>"tomcat2-7.0.37-1.x86_64.rpm"}
Created rpm {:path=>"tomcat3-7.0.37-1.x86_64.rpm"}

This works nicely with a local yum repository and Puppet package resources
though I have had to get Puppet to fix file ownership after installation as
some directories still have root ownership.

  - Keith



On 28 March 2013 13:52, Bruno Harbulot <bruno@distributedmatter.net>
wrote:
> Hello,
>
> I would like to find a few to use files available in a module for the
> execution of a particular resource, without having to stage the file
> explicitly onto the client and possibly having to clean it up afterwards.
> (This could also apply to content coming from templates.)
>
> The use-cases I have in mind are: running a SQL file (to set up some
> content in a database), extracting an archive and running a self-executable
> installer (as generated with makeself, for example).
>
> Currently, one way to do this is to stage the file first and to make the
> resource (e.g. exec) using it depend on that file being set up:
>
>     file { ''/tmp/setupdb.sql'' :
>         source =>
"puppet:///modules/${module_name}/setupdb.sql",
>         mode => 644,
>         backup => false,
>     }
>
>     exec { ''Setup the database'':
>         command => ''psql ${databasename} -f
/tmp/setupdb.sql'',
>         onlyif => ''psql ... | grep ...'', # Some
condition to check whether
> a table exists, for example.
>         requires => File[''/tmp/setupdb.sql''],
>     }
>
> The problem with this is that it looks procedural, and that it also leaves
> a useless temporary file behind.
>
> One way around this particular use case would be to have support for stdin
> input in exec (See: <http://projects.puppetlabs.com/issues/653>).
> Something like this would feel somewhat cleaner:
>
>     exec { ''Setup the database'':
>         command => ''psql ${databasename}'',
>         stdin =>
"puppet:///modules/${module_name}/setupdb.sql",
>         onlyif => ''psql ... | grep ...'', # Some
condition to check whether
> a table exists, for example.
>     }
>
> Nevertheless, this would only address part of the issue of handling files
> temporary files, since this would only work for commands that can read
> their main content from stdin (e.g. mysql, psql, tar).
>
>
> Another use case would be unzipping an archive into a specific directory:
>
>    file { ''/tmp/myfavouriteblogengine.zip'':
>         source =>
> "puppet:///modules/${module_name}/myfavouriteblogengine.zip",
>         mode => 644,
>         backup => false,
>     }
>
>     exec { ''Setup the engine for vhost1'':
>         command => ''cd /var/lib/www/vhost1 && unzip
> /tmp/myfavouriteblogengine.zip '',
>         creates => ''/var/lib/www/vhost1/index.php'',
>         requires =>
File[''/tmp/myfavouriteblogengine.zip''],
>     }
>
> The problem remains: there''s still a temporary file left over, and
it
> doesn''t feel very declarative.
>
> What are the typical patterns to deal with this kind of situation?
>
> Packaging for the distribution isn''t necessarily viable: it
requires a
> fair amount of additional setup, and not everything can be packaged easily
> (whatever the tools), since it would be difficult to install the same
> package multiple times for vhost1, vhost2, ... in cases similar to the
> example above.
>
> Best wishes,
>
> Bruno.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscribe@googlegroups.com.
> To post to this group, send email to puppet-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Mar 28, 2013, at 10:56 AM, Keith Burdis <keith@burdis.org> wrote:
> Today I started using Jordan Sissel''s fpm [1] to build multiple
rpms for each tomcat instance with a different prefix - something like:
This seems like a good opportunity to use CATALINA_BASE for multiple Tomcat
instances per install.  I''ve got a pattern for this with Puppet that
I''ll share if anyone''s interested.

--
Brian Lalor
blalor@bravo5.org


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Indeed, but other than saving disk space, I have found running multiple
instances from the same installation more trouble than it is worth.  Please
do share your pattern though as I am sure there are others who disagree and
I may change my mind :-)


On 30 March 2013 10:20, Brian Lalor <blalor@bravo5.org> wrote:
> On Mar 28, 2013, at 10:56 AM, Keith Burdis <keith@burdis.org> wrote:
>
>  Today I started using Jordan Sissel''s fpm [1] to build multiple
rpms for
> each tomcat instance with a different prefix - something like:
>
>
> This seems like a good opportunity to use CATALINA_BASE for multiple
> Tomcat instances per install.  I''ve got a pattern for this with
Puppet that
> I''ll share if anyone''s interested.
>
> --
> Brian Lalor
> blalor@bravo5.org
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscribe@googlegroups.com.
> To post to this group, send email to puppet-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.