Puppet users - Oct 2010 - best practice for removing a module & maintained resources from a system ?

Hello,

I am curious to know what is the best practice for removing a module 
(and the resources it maintains) from a system ?

Consider :

node ''webserver'' {
   include webservice
   ...
}

class ''webservice'' {
   file { ''/etc/httpd/conf.d/webservice.conf'':
     ...
   }
}

At some point i no longer want webservice on webserver, and therefore 
remove the module from the node declaration ; however, the configuration 
file remains there, and thus HTTPd continues to load it.

I have occasionally written sub-modules to do the work of cleaning the 
resources away (ex. « webservice::no »), but this seems ridiculous, and 
is an administrative pain besides.  What sorts of approaches might there 
be to make this for manageable ?

Thank you all.


-- 
Daniel Maher <dma AT witbe DOT net>
"The Internet is completely over." -- Prince

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Oct 18, 5:19 am, Daniel Maher <d...@witbe.net>
wrote:
> I am curious to know what is the best practice for removing a module
> (and the resources it maintains) from a system ?
[...]
> I have occasionally written sub-modules to do the work of cleaning the
> resources away (ex. « webservice::no »), but this seems ridiculous, and
> is an administrative pain besides.  What sorts of approaches might there
> be to make this for manageable ?
I''m guessing you mean you have written sub-*classes* to do that job.
That is indeed the Puppet way to do it, and I don''t find it at all
ridiculous.

Perhaps you already understand this, but you do not grok Puppet until
you know in your bones that Puppet is about achieving and maintaining
*state*, and only incidentally about doing particular work.  In this
case, the state detail you achieve _and maintain_ is that the web
service is not installed.  If it is important to you that the service
be absent from certain nodes, then why is it ridiculous for Puppet to
maintain that?

If you find it more palatable, you can create a custom fact that
reports whether the service is installed, and conditionally include
the "webservice::no" class based on its value.

As a practical matter, an advantage of the subclass approach is that,
if done right, including both the base class and the ::no class works,
and results in the web service being absent (as if just the ::no class
were included).  That means you don''t have to worry about finding and
changing all the places in a complex configuration that the base class
may be included.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 10/18/2010 03:13 PM, jcbollinger wrote:
> I''m guessing you mean you have written sub-*classes* to do that
job.
> That is indeed the Puppet way to do it, and I don''t find it at all
> ridiculous.
Classes, yes - sorry, i should know better than to post before the first 
coffee of the day. :P
> Perhaps you already understand this, but you do not grok Puppet until
> you know in your bones that Puppet is about achieving and maintaining
> *state*, and only incidentally about doing particular work.  In this
That is an excellent point.
> As a practical matter, an advantage of the subclass approach is that,
> if done right, including both the base class and the ::no class works,
> and results in the web service being absent (as if just the ::no class
> were included).  That means you don''t have to worry about finding
and
> changing all the places in a complex configuration that the base class
> may be included.
Very good, thank you for your commentary ; assuming no other 
contradictory advice, i will continue writing ::no-style classes across 
the board.


-- 
Daniel Maher <dma AT witbe DOT net>
"The Internet is completely over." -- Prince

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 10/18/2010 3:13 PM, jcbollinger wrote:
>
> On Oct 18, 5:19 am, Daniel Maher<d...@witbe.net>  wrote:
>> I am curious to know what is the best practice for removing a module
>> (and the resources it maintains) from a system ?
>
> [...]
>
>> I have occasionally written sub-modules to do the work of cleaning the
>> resources away (ex. « webservice::no »), but this seems ridiculous, and
>> is an administrative pain besides.  What sorts of approaches might
there
>> be to make this for manageable ?
>
> I''m guessing you mean you have written sub-*classes* to do that
job.
> That is indeed the Puppet way to do it, and I don''t find it at all
> ridiculous.
>
> Perhaps you already understand this, but you do not grok Puppet until
> you know in your bones that Puppet is about achieving and maintaining
> *state*, and only incidentally about doing particular work.  In this
> case, the state detail you achieve _and maintain_ is that the web
> service is not installed.  If it is important to you that the service
> be absent from certain nodes, then why is it ridiculous for Puppet to
> maintain that?
Exactly. If done right, this can be achieved by using purging on the 
proper directories (like /etc/http/conf.d) to keep away unmanaged 
contents. For the most things this obviates the need for a ::no class 
altogether.


Best Regards, David
-- 
dasz.at OG              Tel: +43 (0)664 2602670     Web: http://dasz.at
Klosterneuburg                                         UID: ATU64260999

        FB-Nr.: FN 309285 g          FB-Gericht: LG Korneuburg

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Oct 18, 9:03 am, David Schmitt <da...@dasz.at>
wrote:
> Exactly. If done right, this can be achieved by using purging on the
> proper directories (like /etc/http/conf.d) to keep away unmanaged
> contents. For the most things this obviates the need for a ::no class
> altogether.
But purging makes my skin crawl :)

Seriously, I worry that purging will end up removing something that I
didn''t intend.  If that doesn''t bother you, and if ensuring a
config
file is absent sufficiently addresses the issue, then purging indeed
provides an even simpler solution.

Via the "resources" meta-resource, you can also configure purging of
resources such as users and packages.  That would be unwise unless you
have *ALL* resources of the chosen types under management, but for a
rigorously Puppetized system it probably makes sense.  Mine are not so
thoroughly controlled.


Regards,

John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 10/19/2010 3:29 PM, jcbollinger wrote:
>
> On Oct 18, 9:03 am, David Schmitt<da...@dasz.at>  wrote:
>> Exactly. If done right, this can be achieved by using purging on the
>> proper directories (like /etc/http/conf.d) to keep away unmanaged
>> contents. For the most things this obviates the need for a ::no class
>> altogether.
>
> But purging makes my skin crawl :)
>
> Seriously, I worry that purging will end up removing something that I
> didn''t intend.  If that doesn''t bother you, and if
ensuring a config
> file is absent sufficiently addresses the issue, then purging indeed
> provides an even simpler solution.
>
> Via the "resources" meta-resource, you can also configure purging
of
> resources such as users and packages.  That would be unwise unless you
> have *ALL* resources of the chosen types under management, but for a
> rigorously Puppetized system it probably makes sense.  Mine are not so
> thoroughly controlled.
>
Just FTR: I use purging almost exclusively on .d style directories, most 
of them again managed by volcane''s concat module.


Best Regards, David
-- 
dasz.at OG              Tel: +43 (0)664 2602670     Web: http://dasz.at
Klosterneuburg                                         UID: ATU64260999

        FB-Nr.: FN 309285 g          FB-Gericht: LG Korneuburg

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mon, Oct 18, 2010 at 9:13 AM, jcbollinger <John.Bollinger@stjude.org>
wrote:
> I''m guessing you mean you have written sub-*classes* to do that
job.
> That is indeed the Puppet way to do it, and I don''t find it at all
> ridiculous.
As a puppet newcomer, that is a bit surprising, and IMO unreasonable.

Imagine you are joining a puppet-using company. To "remove a class"
from a server you now need to know all the files ever installed or
managed by that class, since puppet started to be used there.

There is no place (in puppet) to query that info! Hopefully some SCM
was used, and used consistently. And you have to review by hand each
revision. Joy!

Compare/contrast this to what package mgmt tools do: they keep a small
DB of what files they are tracking for each package on a given
machine.

A new version of the package needs only declare what files it has. The
pkg manager will helpfully remove any stale files. From the PoV of the
packager... I don''t need to know what files any and all releases of
this package installed -- it''d be impossible to know anyway. I
don''t
need to remove any "possible" file ever installed by this package.

Puppet manages many resource types, so this isn''t trivial. For some
resource types "what to do" won''t be super clear.

Puppet (currently doesn''t, but ) should track what it manages, and
help us remove it when appropriate.

cheers,



m
-- 
 martin.langhoff@gmail.com
 martin@laptop.org -- School Server Architect
 - ask interesting questions
 - don''t get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Oct 20, 2010, at 8:00 AM, Martin Langhoff wrote:
> On Mon, Oct 18, 2010 at 9:13 AM, jcbollinger
<John.Bollinger@stjude.org> wrote:
>> I''m guessing you mean you have written sub-*classes* to do
that job.
>> That is indeed the Puppet way to do it, and I don''t find it at
all
>> ridiculous.
> 
> As a puppet newcomer, that is a bit surprising, and IMO unreasonable.
I wouldn''t call it unreasonable.  I would call it "lack of a
really cool feature".
> Imagine you are joining a puppet-using company. To "remove a
class"
> from a server you now need to know all the files ever installed or
> managed by that class, since puppet started to be used there.
Except if you look through the definitions in the class it shouldn''t be
too manually write rules that do the opposite.
Second, you don''t need to do this for config files in directories
managed with "purge".
Third, if you''re in a hurry, you don''t actually need to
reverse every step.  You only need to disable anything that''s active. 
This isn''t a best practice, but it does work if you''re
careful.
> There is no place (in puppet) to query that info! Hopefully some SCM
> was used, and used consistently. And you have to review by hand each
> revision. Joy!
> 
> Compare/contrast this to what package mgmt tools do: they keep a small
> DB of what files they are tracking for each package on a given
> machine.
> 
> A new version of the package needs only declare what files it has. The
> pkg manager will helpfully remove any stale files. From the PoV of the
> packager... I don''t need to know what files any and all releases
of
> this package installed -- it''d be impossible to know anyway. I
don''t
> need to remove any "possible" file ever installed by this
package.
> 
> Puppet manages many resource types, so this isn''t trivial. For
some
> resource types "what to do" won''t be super clear.
I would go farther and say that it could be very very unclear.  Exec is a
trivial example and is trivial to fix by adding another parameter.

A much nastier example is File.  I''ve been managing a config file.  The
file existed before and File replaces it.  Later the file is replaced by
something other than Puppet.  File dutifully replaces the file again.  Which
file should puppet restore?

> Puppet (currently doesn''t, but ) should track what it manages, and
> help us remove it when appropriate.
> 
> cheers,
> 
> 
> 
> m
> -- 
>  martin.langhoff@gmail.com
>  martin@laptop.org -- School Server Architect
>  - ask interesting questions
>  - don''t get distracted with shiny stuff  - working code first
>  - http://wiki.laptop.org/go/User:Martinlanghoff
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
> 
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wed, Oct 20, 2010 at 12:03 PM, Patrick <kc7zzv@gmail.com>
wrote:
> I wouldn''t call it unreasonable.  I would call it "lack of a
really cool feature".
Pretty fundamental feature :-)

I am not saying Puppet needs to magically know what I''d like to happen
with the file. However, it should at least give us enough tools/info
to operate.
>> Imagine you are joining a puppet-using company. To "remove a
class"
>> from a server you now need to know all the files ever installed or
>> managed by that class, since puppet started to be used there.
>
> Except if you look through the definitions in the class it
shouldn''t be too manually write rules that do the opposite.
Except that some definitions may be gone. That''s what worries me. Sure
I can read the pp files as they are today.

 - John had a webmail server class assigned to server Frodo. Service
apache must be running, an /etc/apache/conf.d/webmail.conf must be in
place.

 - Frodo is no longer a webmail server. John added a rule to disable
apache service.

 - Here I come new job! First task: set Frodo to be a moodle server.
New class has moodle dpkg, /etc/apache/conf.d/moodle.conf -- and
enables apache.

Hell-O!
> Second, you don''t need to do this for config files in directories
managed with "purge".
How would you tackle the example above with ''purge''?  Note
that
/etc/apache/conf.d/ has other files too, provided by the apache dpkg
(and similar in /etc/httpd/conf.d on RH/Fedora machines).
> A much nastier example is File.  I''ve been managing a config file.
 The file existed before and File replaces it.  Later the file is replaced by
something other than Puppet.  File dutifully replaces the file again.  Which
file should puppet restore?
I think we should at least be able to say "remove it when we no longer
mention it". Something like ''purgewhengone'' =>
YesPlease

Don''t know if Puppet as it stands today keeps enough state for this.

Any kind of ''restoreoriginal'' would get into the problems you
outline,
and then some more. Which "original"?

cheers,


m
-- 
 martin.langhoff@gmail.com
 martin@laptop.org -- School Server Architect
 - ask interesting questions
 - don''t get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wed, Oct 20, 2010 at 10:26 PM, Martin Langhoff
<martin.langhoff@gmail.com
> wrote:
> Pretty fundamental feature :-)
>
You don''t have random scripts doing random business. If the puppet
modules
have been written nicely, I don''t see any reason to be unable to go
through
them once, understand the structure, and work from that point. Even if they
are not written nicely, you still have all the information in there, a bit
of cleanup and you''re on your way. You can have test runs, parse tests
and
all, just to know the state of the modules.


Except that some definitions may be gone. That''s what worries me.
Sure
> I can read the pp files as they are today.
>
That''s probably true for any tool or method. Unless it was in version
control.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wed, Oct 20, 2010 at 3:00 PM, Mohit Chawla
<mohit.chawla.binary@gmail.com> wrote:
>> Except that some definitions may be gone. That''s what worries
me. Sure
>> I can read the pp files as they are today.
>
> That''s probably true for any tool or method. Unless it was in
version
> control.
Not true of packages under any modern packaging system, as per the
example given.



m
-- 
 martin.langhoff@gmail.com
 martin@laptop.org -- School Server Architect
 - ask interesting questions
 - don''t get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 10/20/2010 06:56 PM, Martin Langhoff wrote:
> Except that some definitions may be gone. That''s what worries me.
Sure
> I can read the pp files as they are today.
> 
>  - John had a webmail server class assigned to server Frodo. Service
> apache must be running, an /etc/apache/conf.d/webmail.conf must be in
> place.
> 
>  - Frodo is no longer a webmail server. John added a rule to disable
> apache service.
> 
>  - Here I come new job! First task: set Frodo to be a moodle server.
> New class has moodle dpkg, /etc/apache/conf.d/moodle.conf -- and
> enables apache.
> 
> Hell-O!
Adding the rule to disable apache service is not necessarily the right
thing to do.

Sure, the machine doesn''t *need* to run apache anymore. Tell it so by
removing the resource from the manifest (no longer include the webmail
class).

*Must* the machine no longer run a web server? Then, include a class
that specifically disables apache. But then, that is what you (or the
John from your example) has told puppet to maintain, so you need to
think about it.

I guess what you''re getting at is this: No, puppet is not exactly good
at "uninstall this now and from then on, don''t care about it
anymore".
This is not what puppet has been conceived for, though.

You''re looking at a real problem, but not one puppet is (or should be)
necessarily equipped to handle well (IMO).
>> Second, you don''t need to do this for config files in
directories managed with "purge".
> 
> How would you tackle the example above with ''purge''? 
Note that
> /etc/apache/conf.d/ has other files too, provided by the apache dpkg
> (and similar in /etc/httpd/conf.d on RH/Fedora machines).
If you''re on the purge train, you won''t want your package
manager to
interfere with your conf.d directories. Instead, puppet will need the
whole picture of what should be in the conf.d.

Most puppet providers (package, host, mount, cron etc.) strive to do the
opposite, and amend to a given state. Purge is a way to switch
paradigms. If you choose to do that, be prepared to deal with the
consequences.
>> A much nastier example is File.  I''ve been managing a config
file.  The file existed before and File replaces it.  Later the file is replaced
by something other than Puppet.  File dutifully replaces the file again.  Which
file should puppet restore?
> 
> I think we should at least be able to say "remove it when we no longer
> mention it". Something like ''purgewhengone'' =>
YesPlease
> 
> Don''t know if Puppet as it stands today keeps enough state for
this.
> 
> Any kind of ''restoreoriginal'' would get into the problems
you outline,
> and then some more. Which "original"?
It''s an interesting train of thought. Filebucket may or may not be
extended in some way that would allow such features. Note that this
would be a solution for the file provider only.

I''d be pleased to see your general idea extended in this or the devel
forum.

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, Oct 21, 2010 at 4:56 AM, Felix Frank
<felix.frank@alumni.tu-berlin.de> wrote:
> I guess what you''re getting at is this: No, puppet is not exactly
good
> at "uninstall this now and from then on, don''t care about it
anymore".
> This is not what puppet has been conceived for, though.
OK - but putting (config) files into place is most of what we do with it right?

We need to be able to manage them after putting them there, and that
includes removing them.
> If you''re on the purge train, you won''t want your package
manager to
> interfere with your conf.d directories. Instead, puppet will need the
> whole picture of what should be in the conf.d.
>
> Most puppet providers (package, host, mount, cron etc.) strive to do the
> opposite, and amend to a given state. Purge is a way to switch
> paradigms. If you choose to do that, be prepared to deal with the
> consequences.
Can you flesh out what other consequences you see?



m
-- 
 martin.langhoff@gmail.com
 martin@laptop.org -- School Server Architect
 - ask interesting questions
 - don''t get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 10/21/2010 05:33 PM, Martin Langhoff wrote:
> On Thu, Oct 21, 2010 at 4:56 AM, Felix Frank
> <felix.frank@alumni.tu-berlin.de> wrote:
>> I guess what you''re getting at is this: No, puppet is not
exactly good
>> at "uninstall this now and from then on, don''t care about
it anymore".
>> This is not what puppet has been conceived for, though.
> 
> OK - but putting (config) files into place is most of what we do with it
right?
> 
> We need to be able to manage them after putting them there, and that
> includes removing them.
Managing whole config files is not puppet''s strong suit at all. Puppet
strives to implement as many providers as possible in order to keep you
from the need to copy files to your nodes. Copying is actually a crude
workaround from a designer''s perspective.

Theory aside, it''s what most people use it for. But puppet is not
supposed to be the ultimate file roller. In some cases, you may want to
use a packaging system instead, or even a source control mechanism.
>> If you''re on the purge train, you won''t want your
package manager to
>> interfere with your conf.d directories. Instead, puppet will need the
>> whole picture of what should be in the conf.d.
>>
>> Most puppet providers (package, host, mount, cron etc.) strive to do
the
>> opposite, and amend to a given state. Purge is a way to switch
>> paradigms. If you choose to do that, be prepared to deal with the
>> consequences.
> 
> Can you flesh out what other consequences you see?
None, *you* were the one complaining that purge and dpkg-controlled
conf.d contents won''t cooperate ;-)

Seriously though, this is a matter of both taste and pragmatic choice
you have to take based on the requirements at hand.

Regards,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wed, Oct 20, 2010 at 12:09 PM, Martin Langhoff
<martin.langhoff@gmail.com> wrote:
> On Wed, Oct 20, 2010 at 3:00 PM, Mohit Chawla
> <mohit.chawla.binary@gmail.com> wrote:
>>> Except that some definitions may be gone. That''s what
worries me. Sure
>>> I can read the pp files as they are today.
>>
>> That''s probably true for any tool or method. Unless it was in
version
>> control.
>
> Not true of packages under any modern packaging system, as per the
> example given.
Which is an excellent reason for using packages with Puppet.

There are other issues here, but I''m quite firmly of the opinion that
if you need to manage a complex set of files related to a single
function, and you want the features that a packaging system provides,
you should build packages yourself and use Puppet to manage the state
of that package.

This doesn''t solve every negation problem, but it significantly
reduces the problem space.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.