Puppet users - May 2012 - Resource namespace issues with modules

Hello,
this is somewhat related to

https://groups.google.com/forum/?hl=it&fromgroups#!topic/puppet-users/uqZKLikYbJs

but I wanted to ask since it seems a more general issue.

with puppet 2.7.14, puppet-module-tool has been pulled into core puppet,
hence I think pre-cooked modules will become more and more widespread
and used.

But there''s a namespace issue with resources; while classes got their
own namespace system, and dynamic variable lookup is being deprecated,
this doesn''t hold true for resources.

Consider the following example:

Module one contains this in init.pp

class one {
   file { ''something'':
        path => ''/tmp/that''
        }
}

module two contains this in init.pp

class two {
   file { ''something'':
        path => ''/tmp/this''
        }
}


As a result, somebody using both modules would get an error like:

err: Could not retrieve catalog from remote server: Error 400 on SERVER:
Duplicate declaration: File[something] is already declared in file
/etc/puppet/modules/one/manifests/init.pp at line 17; cannot redeclare
at /etc/puppet/modules/two/manifests/init.pp:19 on node xxx.yyy.zzz

Even though there''s no real issue here, and the user shouldn''t
be forced
to change the code of external modules, IMHO.

It seems that resource types work this way:

- resources are pulled in a global namespace and must be unique with a
"primary key" like (resource type, resource name). This includes
aliases
- you can''t create an alias name that''s not unique (that makes
sense).
- if the namevar parameter is not assigned explicitly, use title as
namevar. The resource name will be called just like the title.
- if the namevar parameter is assigned, the resource will be *aliased*
to the namevar value.
- if an alias parameter is assigned, such alias will be *added* to the
ones the resource may already have.

IMHO again:

* such behaviour may be puzzling. If I give a title to a resource, I
expect conflicts to be checked against that name, not against the
namevar that - very often - is highly semantic, e.g. I must use path
with files, otherwise I can''t tell the system where to put the file;
so,
a useful metadata check ("don''t reuse the same name") becomes
tied to a
semantic behaviour I can''t change.

I think the name checks should be performed after title and aliases
only, not after the namevar, which is something internal.

There may be even cases where I want to use multiple resources with the
same target (e.g. I might want to use two file{} with the same path but
one for content, the other for setting permissions, depending on the
situation, by setting the proper Require =>), and this would prevent me
from doing that.

* non-scoped resources can generate name conflicts everywhere, even
though they''re not useful.

I think resource scoping is highly needed if modularization is the way
to go for puppet.

-- 
Latest blog post: Deploying WAR on Heroku http://t.co/M4krW2Mu
contact me at public@[mysurname].eu

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On May 4, 10:10 am, Alan Franzoni <mail...@franzoni.eu>
wrote:
> Hello,
> this is somewhat related to
>
>
https://groups.google.com/forum/?hl=it&fromgroups#!topic/puppet-users...
>
> but I wanted to ask since it seems a more general issue.
>
> with puppet 2.7.14, puppet-module-tool has been pulled into core puppet,
> hence I think pre-cooked modules will become more and more widespread
> and used.
>
> But there''s a namespace issue with resources; while classes got
their
> own namespace system, and dynamic variable lookup is being deprecated,
> this doesn''t hold true for resources.
>
> Consider the following example:
>
> Module one contains this in init.pp
>
> class one {
>    file { ''something'':
>         path => ''/tmp/that''
>         }
>
> }
>
> module two contains this in init.pp
>
> class two {
>    file { ''something'':
>         path => ''/tmp/this''
>         }
>
> }
>
> As a result, somebody using both modules would get an error like:
>
> err: Could not retrieve catalog from remote server: Error 400 on SERVER:
> Duplicate declaration: File[something] is already declared in file
> /etc/puppet/modules/one/manifests/init.pp at line 17; cannot redeclare
> at /etc/puppet/modules/two/manifests/init.pp:19 on node xxx.yyy.zzz
>
> Even though there''s no real issue here, and the user
shouldn''t be forced
> to change the code of external modules, IMHO.

Fair enough, but I rate that a quality of (module) implementation
issue.  Where a module assigns logical names to its resources, it can
apply its own namespace to those names if the usage warrants it.

Where a module uses physical names (e.g. file paths), on the other
hand, no namespacing is required because the resource is already
uniquely identified.

Ideally, every module works correctly out of the box, regardless of
what other modules are in use.  In practice, that''s an exceedingly
difficult problem, not within Puppet''s power to fully mitigate, even
in theory.

> It seems that resource types work this way:
>
> - resources are pulled in a global namespace and must be unique with a
> "primary key" like (resource type, resource name). This includes
aliases
yes.
> - you can''t create an alias name that''s not unique (that
makes sense).
yes.
> - if the namevar parameter is not assigned explicitly, use title as
> namevar. The resource name will be called just like the title.
yes, that''s what "namevar" means.
> - if the namevar parameter is assigned, the resource will be *aliased*
> to the namevar value.
close enough.
> - if an alias parameter is assigned, such alias will be *added* to the
> ones the resource may already have.
close enough.
> IMHO again:
>
> * such behaviour may be puzzling. If I give a title to a resource, I
> expect conflicts to be checked against that name, not against the
> namevar that - very often - is highly semantic, e.g. I must use path
> with files, otherwise I can''t tell the system where to put the
file; so,
> a useful metadata check ("don''t reuse the same name")
becomes tied to a
> semantic behaviour I can''t change.

Your expectation seems based on a flawed understanding of Puppet''s
data model.  Distinguishing a resource''s title from its namevar
provides for self-documentation and limited indirection, but it does
not and never was intended to alter Puppet''s rules about resource
uniqueness.  Puppet requires resource uniqueness to ensure internal
consistency of the catalogs it compiles.

Whenever it detects multiple Puppet resources referring to the same
physical resource, Puppet assumes it cannot reliably determine the
intended result, and errors out to signal admins that they need to
sort out the situation.  This is a Good Thing, because Puppet''s
assumption is exactly right except perhaps -- perhaps -- in a few
special cases.  You can find discussions of this topic among the
archives of this group; look for terms such as "module compatibility"
and "resource merging".

> I think the name checks should be performed after title and aliases
> only, not after the namevar, which is something internal.

On the contrary, a namevar is typically something very EXternal: the
target node''s idea of the unique identifier of the physical resource
under management.

You seem to misunderstand the purpose of Puppet''s constraints and
checks: it is not really to validate manifests, but rather to reduce
the likelihood of misconfiguring or inconsistently configuring
clients.  Puppet''s insistence on consistent manifests is an
implementation choice, not to be mistaken as the purpose of the
exercise in itself .   The change you suggest would completely
undermine all that.

> There may be even cases where I want to use multiple resources with the
> same target (e.g. I might want to use two file{} with the same path but
> one for content, the other for setting permissions, depending on the
> situation, by setting the proper Require =>), and this would prevent me
> from doing that.

You are right.  Puppet does not allow you to specify different
properties for the same physical resource via different, independent
resource declarations.  This is a longstanding and fundamental
consequence of Puppet''s architecture.  We have had discussions on that
topic before, and even turned up some possible ideas for future
development in that area.  For the time being, however, the tools to
address these areas are these:

1) Virtual resources, allowing a single, central, optional resource
declaration to be shared among multiple classes or modules.

2) External data (e.g. from hiera), to allow admins to define or
customize the properties of declared resources without modifying
manifest files.

3) Resource property overrides via collections or class inheritance.

Those can be used independently or in various combinations.

> * non-scoped resources can generate name conflicts everywhere, even
> though they''re not useful.
>
> I think resource scoping is highly needed if modularization is the way
> to go for puppet.

This again seems to be a data model problem.  It is important to
understand that however nicely organized, modularized, and namespaced
your Puppet manifests may be, they all refer to a physical machine
configuration that recognizes few, if any, of those organizational
distinctions.  This is one of the reasons why module compatibility is
such a hard problem.

Generally speaking, any physical resource can see all other physical
resources on the same node.  Puppet''s resource model reflects that by
putting all resources in the same namespace.  Splitting up the
namespace would make it easier to mask conflicts, but it wouldn''t
resolve them, and it *certainly* would not make them harmless


John
.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.