Puppet users - Jan 2009 - Automating removal of an application

Hello,

   I''m currently prototyping the use of Puppet to manage a bunch of
home-made applications on a lot of servers.
   As our applications evolves frequently and multiples versions can
be used at the same time, we use classes that include version in their
name, like "myApplication_1-2-0"

 A typical application class like "myApplication_1-2-0" contains for
example a package to install, some configuration files as templates, a
crontab, a service definition and a mount point

 When I want to deploy version 1-3-0 of my application on a node, to
replace version 1-2-0, the standard way to do with puppet, as I
understand it, would be :
  - node initially contains "include myApplication_1-2-0"
  - I edit node to contains "include myApplication_1-2-0::remove"
where myApplication_1-2-0::remove is a class that removes everything
myApplication_1-2-0 installed
  - I apply the config to node
  - I edit the node to contains "include myApplication_1-3-0"
  - I apply the config to node

But on node, there is the localconfig.yaml file (usually in /var/lib/
puppet/) that contains everything needed to know what should be
removed when myApplication_1-2-0 is installed.
I was thinking of automating this step like this :
  - retrieve config from master
  - retrieve local config (in localconfig.yaml)
  - if an application has not the same version on the master and
locally then
   a)  change everything to absent for version which is locally
installed
   b) apply this, so local version is removed
   c) then apply config from master, so new version is installed

Do you think it''s a good approach ?
If not, do you know an alternate ways to manage applications that have
several versions ?

thanks for your answers
nicolas

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Mon, Jan 26, 2009 at 10:18 AM, nicolas <ncapponi@yahoo.fr>
wrote:
>
> Hello,
>
>   I''m currently prototyping the use of Puppet to manage a bunch of
> home-made applications on a lot of servers.
>   As our applications evolves frequently and multiples versions can
> be used at the same time, we use classes that include version in their
> name, like "myApplication_1-2-0"
>
>  A typical application class like "myApplication_1-2-0" contains
for
> example a package to install, some configuration files as templates, a
> crontab, a service definition and a mount point
>
>  When I want to deploy version 1-3-0 of my application on a node, to
> replace version 1-2-0, the standard way to do with puppet, as I
> understand it, would be :
>  - node initially contains "include myApplication_1-2-0"
>  - I edit node to contains "include myApplication_1-2-0::remove"
> where myApplication_1-2-0::remove is a class that removes everything
> myApplication_1-2-0 installed
>  - I apply the config to node
>  - I edit the node to contains "include myApplication_1-3-0"
>  - I apply the config to node
>
> But on node, there is the localconfig.yaml file (usually in /var/lib/
> puppet/) that contains everything needed to know what should be
> removed when myApplication_1-2-0 is installed.
> I was thinking of automating this step like this :
>  - retrieve config from master
>  - retrieve local config (in localconfig.yaml)
>  - if an application has not the same version on the master and
> locally then
>   a)  change everything to absent for version which is locally
> installed
>   b) apply this, so local version is removed
>   c) then apply config from master, so new version is installed
>
> Do you think it''s a good approach ?
> If not, do you know an alternate ways to manage applications that have
> several versions ?
>
> thanks for your answers
> nicolas
>
I can see atleast one big got-ya with this approach. It would work
okay with packages using ensure => present and that don''t get update
by the local update utility. But anything using ensure => latest or
that gets updated by the local update utility is going to get hosed. I
find the best approach with pacakges that need a lot of configs that
change version to version is either wrap it all up in an RPM or use
somethinglike the generate function to create config files on the
file. In most cases, though all that really needs to be done is update
the template and config files, they will replace the old ones and tell
puppet to ensure => present with the version number you want.

About the only time you need to do a wholesale application removal is
when you want to remove it from a system. If you have lots of
applications that are just ugly to install an dremove you may want to
take a look at Stow.

Evan

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Evan Hisey wrote:
> About the only time you need to do a wholesale application removal is
> when you want to remove it from a system. If you have lots of 
> applications that are just ugly to install an dremove you may want to
> take a look at Stow.
If you use stow, you may want my stowedpackage definition. You may find 
improvements can be made, too: http://tinyurl.com/3n5npl

-- 
Mike Renfro  / R&D Engineer, Center for Manufacturing Research,
931 372-3601 / Tennessee Technological University -- renfro@tntech.edu

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Thanks for your answer Evan.

For these applications, we need to have a very good control. So we''ll
never update them via local update utility nor use an ensure => latest
We always use ensure => x.y.z
We also need to be able to do rollback, so it''s simpler to remove
everything from say, version 1-3-2, and then apply version 1-3-1

I understand that our needs are specific, but there is something that
surprise me with Puppet : if you don''t explicitly remove something
that was deployed previously with Puppet, then it stays
For example, say you have a mount point A that you manage with Puppet
For some reason, you need to change this mount point, now call it B.
We could expect that if your manifest first contains A, then contains
B, your intent is to remove A and creates B
Puppet will just create B and leave A. It forces you to do it in two
steps, first remove B, then apply and second add A then apply

Any reasons why Puppet forces to explicitely remove everything that it
previously installed (and that is known, thanks to localconfig.yaml
file on nodes) ?

nicolas
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Wed, Jan 28, 2009 at 3:00 AM, nicolas <ncapponi@yahoo.fr>
wrote:
>
>
> Thanks for your answer Evan.
>
> For these applications, we need to have a very good control. So
we''ll
> never update them via local update utility nor use an ensure => latest
> We always use ensure => x.y.z
> We also need to be able to do rollback, so it''s simpler to remove
> everything from say, version 1-3-2, and then apply version 1-3-1
>
> I understand that our needs are specific, but there is something that
> surprise me with Puppet : if you don''t explicitly remove something
> that was deployed previously with Puppet, then it stays
> For example, say you have a mount point A that you manage with Puppet
> For some reason, you need to change this mount point, now call it B.
> We could expect that if your manifest first contains A, then contains
> B, your intent is to remove A and creates B
> Puppet will just create B and leave A. It forces you to do it in two
> steps, first remove B, then apply and second add A then apply
>
> Any reasons why Puppet forces to explicitely remove everything that it
> previously installed (and that is known, thanks to localconfig.yaml
> file on nodes) ?
>
I find that this method is lot more admin friendly. If you have to
explicitly remove things you can not accidentally remove large chunks
of the system. Let say you have apache and php installed on a server.
You want to remove php but have forgotten that you replaced the apache
non-phped config with a phped config using the php module. Under
current operational mode nothing to bad happens except maybe a
complaint about the missing php. Under an automatic associated roll
back all of a sudden apache quits working because it removed the
config file, leaving you having to figure out that happened. A lot of
things in puppet error on the side of least surprise and least
dangerous.

  I occasionally have need to do one time pushes to my machines for
license files. Once all the machines have updated to the newest
license, I disable the license update as it only happens once a year
and only has to be done to existing installs as all new installs will
get the latest license. The current mode of operation makes this work.

Of course Luke may tell me I am totally wrong.

Evan

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Ok, I agree that my request is targeted to a special case that can be
dangerous if applied to general case.
My main concern was that our admins may find tedious to do the two-
steps process for each release of an application. Maybe I should tell
them it''s more secure !

Anyway, thanks for sharing your ideas

nicolas




--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

I''m not entirely certain of your scenario, but unless you are going to
purposely have two versions of the same application running in production
for extended periods, this seems like the wrong way to do this.

Do you have some way to roll through Dev, Test, Staging, Production?

Can you just have one class for myApplication that gets versioned that goes
through the process to production?

We have some plans to handle more generic state transitions using Puppet, so
you could do something like apply this manifest, then this one, then this
other one, which would make your problem trivial.






On Mon, Jan 26, 2009 at 9:18 AM, nicolas <ncapponi@yahoo.fr> wrote:
>
> Hello,
>
>   I''m currently prototyping the use of Puppet to manage a bunch of
> home-made applications on a lot of servers.
>   As our applications evolves frequently and multiples versions can
> be used at the same time, we use classes that include version in their
> name, like "myApplication_1-2-0"
>
>  A typical application class like "myApplication_1-2-0" contains
for
> example a package to install, some configuration files as templates, a
> crontab, a service definition and a mount point
>
>  When I want to deploy version 1-3-0 of my application on a node, to
> replace version 1-2-0, the standard way to do with puppet, as I
> understand it, would be :
>  - node initially contains "include myApplication_1-2-0"
>  - I edit node to contains "include myApplication_1-2-0::remove"
> where myApplication_1-2-0::remove is a class that removes everything
> myApplication_1-2-0 installed
>  - I apply the config to node
>  - I edit the node to contains "include myApplication_1-3-0"
>  - I apply the config to node
>
> But on node, there is the localconfig.yaml file (usually in /var/lib/
> puppet/) that contains everything needed to know what should be
> removed when myApplication_1-2-0 is installed.
> I was thinking of automating this step like this :
>  - retrieve config from master
>  - retrieve local config (in localconfig.yaml)
>  - if an application has not the same version on the master and
> locally then
>   a)  change everything to absent for version which is locally
> installed
>   b) apply this, so local version is removed
>   c) then apply config from master, so new version is installed
>
> Do you think it''s a good approach ?
> If not, do you know an alternate ways to manage applications that have
> several versions ?
>
> thanks for your answers
> nicolas
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

nicolas <ncapponi@yahoo.fr> writes:
> For these applications, we need to have a very good control. So
we''ll
> never update them via local update utility nor use an ensure => latest
> We always use ensure => x.y.z
> We also need to be able to do rollback, so it''s simpler to remove
> everything from say, version 1-3-2, and then apply version 1-3-1
>
> I understand that our needs are specific, but there is something that
> surprise me with Puppet : if you don''t explicitly remove something
> that was deployed previously with Puppet, then it stays
There are other software configuration tools, such as bcfg2[1], that take
an approach similar to what you describe here: they treat anything not
configured as an aberration and remove it.

They are generally more painful to use than puppet by an order of
magnitude, and practically impossible to nicely introduce to existing
infrastructure, at least in my experience.

You can have much the same effect with puppet using this, by the way:

    Package { ensure => absent }
    # repeat for whatever you want strictly managed...
> For example, say you have a mount point A that you manage with Puppet
> For some reason, you need to change this mount point, now call it B.
> We could expect that if your manifest first contains A, then contains
> B, your intent is to remove A and creates B
This describes a situation where puppet records changes in state and
applies the delta between two configurations.

What, instead, puppet actually does is apply the configuration defined
at any point in time, without state or reference to history.


In general, the only tools that work that way are the big graphical
centralized management tools, in my experience.

[...]
> Any reasons why Puppet forces to explicitely remove everything that it
> previously installed (and that is known, thanks to localconfig.yaml
> file on nodes) ?
Because it can''t derive your intention automatically: you /could/ want
it removed, or you could want it unmanaged, and assuming you want it
removed is (generally) more destructive.

Regards,
        Daniel

Footnotes: 
[1]  They may have changed this in the last year or two since I reviewed
     the tool, but this more illustrative than a genuine pointer.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Ok, I think I understand better how we should do now.

However, the use of version in class name is still mandatory for us
because we may have two versions of an application in production at
same time (on different nodes) and need to have very strict control on
installed versions.
Also, we generate classes semi-automatically for each version, and it
would be very difficult to maintain an unique definition of a class
that contains all differences between versions.
I can''t see how we could do without it.

Thanks for your help and advices.

nicolas

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Fri, Jan 30, 2009 at 02:12:03PM +1100, Daniel Pittman
wrote:
> 
> nicolas <ncapponi@yahoo.fr> writes:
> 
> > For these applications, we need to have a very good control. So
we''ll
> > never update them via local update utility nor use an ensure =>
latest
> > We always use ensure => x.y.z
> > We also need to be able to do rollback, so it''s simpler to
remove
> > everything from say, version 1-3-2, and then apply version 1-3-1
> >
> > I understand that our needs are specific, but there is something that
> > surprise me with Puppet : if you don''t explicitly remove
something
> > that was deployed previously with Puppet, then it stays
> 
> There are other software configuration tools, such as bcfg2[1], that take
> an approach similar to what you describe here: they treat anything not
> configured as an aberration and remove it.
> [...] 
> Footnotes: 
> [1]  They may have changed this in the last year or two since I reviewed
>      the tool, but this more illustrative than a genuine pointer.
> 
In my experience (which is admittedly limited), bcfg2 doesn''t remove
anything
without explicitly being told to do so.  What it does do is to tell you that
"Hey!  You''ve got unconfigured stuff here!  Is that what you
want?"

In my opinion, granting a similar capacity to puppet (as an option) would be a
huge gain.  I would *love* to know if things are being configured on my systems
without being done through puppet.  For example, if a new user is added, that
could be an admin doing something stupid, or it could be a rootkit.  But unless
something tells me about it, I might never know.

We chose puppet over bcfg2 because of the notation (bcfg2 is XML!  Gah!) and
the user community, but it was tough because I actually like bcfg2''s
comprehensive model a lot more.

Cheers,

-- 
Eric Gerlach, Network Administrator
Federation of Students
University of Waterloo
p: (519) 888-4567 x36329
e: egerlach@feds.uwaterloo.ca

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---