Puppet users - May 2010 - New user: How to deploy new version of a custom RPM package?

Hi there-
sorry if this is an obvious question, but I''m trying to evaluate
puppet on paper before we commit to an experimental deployment.
Aside: We''re using Centos 5.4, so this is RPM/yum specific.

How are updated RPMs in a manifest detected on the puppetmaster side?

What we are trying to achieve:
Set up a puppetmaster with multiple environments <- documentation is
clear on this/can find examples.
Each environment will specify a base set of required applications,
varied by machine type <- mostly understood/can find examples.
Configuration files will be pushed from the puppetmaster to puppet
clients, when a difference in md5 is detected <- understood.

We also want to be able to push updates to our custom (application
executable code) RPMs in a controlled way.
How does the puppetmaster determine the version of an rpm source
file?  Does it base it on the rpm name (eg assuming the standard
naming convention of name-version-release.architecture.rpm)?
What is the format of the ''ensure'' stanza to specify package
versions
for RPM?
Will ''ensure'' also allow/force package downgrades?

We also want to completely separate configuration from application
code (config -> solely deployed via puppet, application code deployed
via RPMs, pushed via puppet.)

We would like config files to be redeployed on package install/
upgrade.
We need services to be started/restarted and added to runlevels on
package install/upgrade.

I''ve tried to create a concrete snippet, below; comments and
suggestions from you experts would be appreciated!

Thanks in advance,

Julian

Guess at a puppet module:

package { our_app_part1:
    source =>
http://our_file_repo/env_xx/release_yy/our_app_part1_ver##_rel##.noarch.rpm,
    ensure => ver##_rel##.noarch,
    require => [Package[list, of, needed, packages],
Files[our_app1_config, ...]]
}

# Is the subscribe line below needed to ensure the config file is
(re)deployed if the package is (re)installed?
file { "/etc/our_app1.config":
    source => "puppet://server/module/our_app1.config",
    require => [Package[our_app_part1_ver##_rel##.noarch.rpm]],
    subscribe => [Package[our_app_part1]]
}

exec { "/etc/init.d/our_app restart":
    path => ["/usr/bin", "/usr/sbin"],
    subscribe => File["/etc/our_app1.config"],
    refreshonly => true
}

# what do we do here to have this execute only when the package is
(re)deployed and after the config file is pushed by puppet?
# If that is really difficult, it could be triggered on every update
to the config file, similar to update.  The overhead is acceptable.
 exec { "chkconfig our_app on":
    path => ["/usr/bin", "/usr/sbin"],
    subscribe => ?????,
    refreshonly => true
}

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

JRendell <juliangrendell@gmail.com> writes:
> sorry if this is an obvious question, but I''m trying to evaluate
> puppet on paper before we commit to an experimental deployment.
> Aside: We''re using Centos 5.4, so this is RPM/yum specific.
>
> How are updated RPMs in a manifest detected on the puppetmaster side?
They are not: the puppetmaster compiles a configuration that includes
instructions (that you write) like "install version N of package M",
or
"install the latest version of package M".

The client then runs, and uses yum to carry out those instructions.
> What we are trying to achieve:
> Set up a puppetmaster with multiple environments <- documentation is
> clear on this/can find examples.
> Each environment will specify a base set of required applications,
> varied by machine type <- mostly understood/can find examples.
> Configuration files will be pushed from the puppetmaster to puppet
> clients, when a difference in md5 is detected <- understood.
Not quite: the client *pulls* the configuration from puppetmaster, rather than
the puppetmaster pushing.  The practical difference is small, but since you
are not the first person to have that confusion...
> We also want to be able to push updates to our custom (application
> executable code) RPMs in a controlled way.  How does the puppetmaster
> determine the version of an rpm source file?
It doesn''t; the client does, and it does that by passing the version
you
specify to yum, or asking yum to define "latest" appropriately.

This is, not coincidentally, exactly what would happen if you entered the same
yum commands on the client by hand...

[...]
> Guess at a puppet module:
>
> package { our_app_part1:
>     source =>
http://our_file_repo/env_xx/release_yy/our_app_part1_ver##_rel##.noarch.rpm,
>     ensure => ver##_rel##.noarch,
>     require => [Package[list, of, needed, packages],
> Files[our_app1_config, ...]]
> }
# If you use the YUM provider...
package { "our_app_part_1": ensure =>
''specific-version-number'' }

# ...and whatever configuration is needed for yum to find the repository that
# contains your RPM better be in puppet too. :)
> # Is the subscribe line below needed to ensure the config file is
> (re)deployed if the package is (re)installed?
> file { "/etc/our_app1.config":
>     source => "puppet://server/module/our_app1.config",
>     require => [Package[our_app_part1_ver##_rel##.noarch.rpm]],
>     subscribe => [Package[our_app_part1]]
> }
No: puppet doesn''t "act once, then ignore changes", it acts
every time it
runs.  If you say "the source of this file is XXX", every time puppet
runs it
will check if the configuration file *is* identical to XXX, and replace it if
not.
> exec { "/etc/init.d/our_app restart":
>     path => ["/usr/bin", "/usr/sbin"],
>     subscribe => File["/etc/our_app1.config"],
>     refreshonly => true
> }
service { "our_app": ensure => running, enable => true ... }

That knows about starting applications the platform way (eg: service foo
start), which is what you want to use.


[...]
>  exec { "chkconfig our_app on":
>     path => ["/usr/bin", "/usr/sbin"],
>     subscribe => ?????,
>     refreshonly => true
> }
It does this too. :)

        Daniel

Hint: if you find yourself using ''exec'' something may well
have gone wrong. :)

-- 
✣ Daniel Pittman            ✉ daniel@rimspace.net            ☎ +61 401 155 707
               ♽ made with 100 percent post-consumer electrons

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Thanks for the info re the service command- makes life easier ;-)
Also
http://serverfault.com/questions/40156/puppet-force-service-restart-after-configuration-file-was-modified
was useful-- how to use ''subscribe'' with
''service'' to ensure a service
is restarted when a change to a configuration file is detected.

First time I read your answers I thought puppet was not going to be
usable for what we want, but I think we had some misunderstandings.
And sorry, yes, well aware it''s pull not push- my bad.

I specifically want to use the RPM provider, not the yum provider.
Otherwise (as I understand it) I will have to set up a yum repository
for each machine (or at least environment), and updating/rolling back
our app will be more complicated than I want to deal with.  ie I don''t
want an inadvertent ''yum update'' changing the version of our
application.  Pulling explicit RPMs via puppet seems easier to control
right now.  Eg we can store the manifests in svn and have an explicit
record of application versions installed at a given point in time/for
a given environment.  If we use yum, I''ll have to manage several yum
repos, including checking in the generated indices etc to be able to
work out what a particular environment had at a given point in time.

Your answers generated some more questions:
> > How are updated RPMs in a manifest detected on the puppetmaster side?
>
> They are not: the puppetmaster compiles a configuration that includes
> instructions (that you write) like "install version N of package
M", or
> "install the latest version of package M".
Ok, and the puppet client determines if it has the appropriate version
already in place, nothing happens- right?
And the puppet client regenerates it''s state, including installed
packages (or at least the ones specified in the manifest), every time
it pulls a configuration from the puppetmaster?

The puppetmaster does nothing with the ''source'' line in a
package
definition?

Thus if we update the ''ensure'' and ''source''
parameters appropriately
(ie make sure we match versions), the puppet client will pull the new
configuration, test the version of the package installed, note that
it''s different, and install the rpm file specified by the source line-
correct?  Fully realizing that if the file specified in the source
!what''s in the ensure line, we''ll start a package-install
loop, that
is, every time the client pulls a configuration down from the
puppetmaster it would see the wrong version of this package is
installed and re-install it.

Question: will puppet handle downgrades?  ie use ''rpm -f'' to
force the
installation?
Or will it only work for upgrades?

Finally, we''d need to make sure the service is restarted, and possibly
make sure the config files are redeployed as well, when the package is
updated.
In the first case, would it work to ''subscribe'' the
''service'' to both
the package and config file definitions? (ie restart if either the
package is re-installed, or the config file changes.)
In the second case, would it work to subscribe the config file to the
package, and the service to the config file?
In both cases, the service would be restarted on either config file
changes or a new version of the application RPM being installed,
correct?

Thanks again-

Julian

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Answering one of my own questions:
>
> Question: will puppet handle downgrades?  ie use ''rpm -f''
to force the
> installation?
> Or will it only work for upgrades?
>
From looking at puppet/provider/package/rpm.rb:
Specifying ''latest'' for the rpm provider will cause the puppet
client
to do an rpm -qf on the rpm specified in the source parameter.
And downgrades are handled, I think...
it looks like it does rpm -i UNLESS the package is listed as installed
(and not absent), in which case it uses rpm -U.
In both cases it adds ''--oldpackage'', which is a synonym for
force.

So it will handle adding, upgrading, and downgrading a package.

Still appreciate any comments re how to best chain subscribe/notify,
and also if I''m wildly wrong about using the RPM provider in
preference to the YUM provider for our use case.

Thanks

Julian

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On May 3, 2010, at 11:10 PM, JRendell wrote:
> Answering one of my own questions:
> 
>> 
>> Question: will puppet handle downgrades?  ie use ''rpm
-f'' to force the
>> installation?
>> Or will it only work for upgrades?
>> 
> 
> From looking at puppet/provider/package/rpm.rb:
> Specifying ''latest'' for the rpm provider will cause the
puppet client
> to do an rpm -qf on the rpm specified in the source parameter.
> And downgrades are handled, I think...
> it looks like it does rpm -i UNLESS the package is listed as installed
> (and not absent), in which case it uses rpm -U.
> In both cases it adds ''--oldpackage'', which is a synonym
for force.
> 
> So it will handle adding, upgrading, and downgrading a package.
> 
> Still appreciate any comments re how to best chain subscribe/notify,
> and also if I''m wildly wrong about using the RPM provider in
> preference to the YUM provider for our use case.
I always use apt, but it looks like creating a yum repository is pretty easy.
http://www.phy.duke.edu/~rgb/General/yum_article/yum_article/node14.html

Why would you need to set up more than one repository?  Testing and production,
or do you have different computers that need different versions?  If
it''s just testing and production, setting up two repositories is
probably easier.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
> I specifically want to use the RPM provider, not the yum provider.
> Otherwise (as I understand it) I will have to set up a yum repository
> for each machine (or at least environment), and updating/rolling back
> our app will be more complicated than I want to deal with.  ie I
don''t
> want an inadvertent ''yum update'' changing the version of
our
> application.  Pulling explicit RPMs via puppet seems easier to control
> right now.  Eg we can store the manifests in svn and have an explicit
> record of application versions installed at a given point in time/for
> a given environment.  If we use yum, I''ll have to manage several
yum
> repos, including checking in the generated indices etc to be able to
> work out what a particular environment had at a given point in time.
if you need this kind of control over your installed packages I would
recommend that you look for patch-/package-version-management into
spacewalk/satellite (if this is possible for your distro). It is imho
the better tool for that job.

Not that it is not possible to do that with puppet, but I see a certain
complexity that will sooner or later come up with controlling exact
package version over plain rpms with puppet.

What I imagine is that you control package installation (ensure =>
installed), config file management and service restarts etc. with puppet
and control the exact patch-level of applications with spacewalk/satellite.

As a sidenote: you might want to use defines to for example wrap up rpm
downloads and package installation. Also be warned that for larger file
downloads you should at least use 0.25.x and that (hopefully) the next
major release (codename rowfl) will finally clean up the remaining
issues with sourcing large files.

cheers pete
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvfx+MACgkQbwltcAfKi38TEgCgsEDkWo1slmSnGqx9A0x5b/zs
yYIAnA9wb76mLGA2BYsP/jrD2oYEOXda
=YcDl
-----END PGP SIGNATURE-----

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

JRendell <juliangrendell@gmail.com> writes:

[...]
> I specifically want to use the RPM provider, not the yum provider.
Ah, sorry.  I missed that, somehow, and have never used the RPM provider.
I think other people have answered your questions there, though, which I hope
is helpful to you.

Thanks for not hating me for my failure to read for content. ;)
        Daniel

-- 
✣ Daniel Pittman            ✉ daniel@rimspace.net            ☎ +61 401 155 707
               ♽ made with 100 percent post-consumer electrons

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.