Puppet users - Mar 2011 - RFC: Splitting up the file{} type functionality.

The file{} type can do all of the following:

* manage single files
* manage directories
* manage symlinks
* manage recursive file copies

The intersection of all these bits of functionality makes it difficult
to understand exactly what is going on when you''re new to Puppet, and
even experienced users often don''t know how combining symlinks/content
management is going to work.

How would people feel about at least splitting out these into their own types?

* symlinks
* recursive file copies

The intersection of files and directories isn''t that big a deal, but
we could split out directories too if we wanted.

Thoughts?


-- 
Nigel Kersten
Product, Puppet Labs
@nigelkersten

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I thing that recursive copies are the type that fits worst into the current
code; I suggest we move that out and then wait and see if symlinks should
follow or not.

Regards,
    Daniel
-- 
Puppet Labs Developer –http://puppetlabs.com
Daniel Pittman <daniel@puppetlabs.com>
Contact me via gtalk, email, or phone: +1 (877) 575-9775
Sent from a mobile device. Please forgive me if this is briefer than usual.
On Mar 21, 2011 6:53 PM, "Nigel Kersten" <nigel@puppetlabs.com>
wrote:
> The file{} type can do all of the following:
>
> * manage single files
> * manage directories
> * manage symlinks
> * manage recursive file copies
>
> The intersection of all these bits of functionality makes it difficult
> to understand exactly what is going on when you''re new to Puppet,
and
> even experienced users often don''t know how combining
symlinks/content
> management is going to work.
>
> How would people feel about at least splitting out these into their own
types?
>
> * symlinks
> * recursive file copies
>
> The intersection of files and directories isn''t that big a deal,
but
> we could split out directories too if we wanted.
>
> Thoughts?
>
>
> --
> Nigel Kersten
> Product, Puppet Labs
> @nigelkersten
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
> To post to this group, send email to puppet-dev@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-dev+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-dev?hl=en.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

> I thing that recursive copies are the type that fits worst into the current
> code; I suggest we move that out and then wait and see if symlinks should
> follow or not.
>
The match with symlinks qua symlinks is pretty hideous as well.  It just
looks a little less lumpy because it''s been hammered on so much, but it
exhibits the stress fracture "if things are going to break anywhere,
they''ll
break here" pattern.  I''m with Nigel; let''s do them both.

-- M
-----------------------------------------------------------
When in trouble or in doubt, run in circles,
scream and shout. -- 1920''s parody of the
maritime general prudential rule
------------------------------------------------------------

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To post to this group, send email to puppet-dev@googlegroups.com.
To unsubscribe from this group, send email to
puppet-dev+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-dev?hl=en.

Nigel Kersten wrote:
> The file{} type can do all of the following:
>
> * manage single files
> * manage directories
> * manage symlinks
> * manage recursive file copies
>
> The intersection of all these bits of functionality makes it difficult
> to understand exactly what is going on when you''re new to Puppet,
and
> even experienced users often don''t know how combining
symlinks/content
> management is going to work.
>
> How would people feel about at least splitting out these into their own
types?
>
> * symlinks
> * recursive file copies
>
+1. The value add of cleaning up that type/provider and the godawful 
mess and edge cases will also be awesome.  We can also revamp the 
documentation at the same time - it provokes a lot of questions 
especially around recursion and symlinks.

On a final note ... can we not overload "ensure" in our new types? :)

James

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mon, Mar 21, 2011 at 09:06:51PM -0700, James Turnbull
wrote:
> Nigel Kersten wrote:
>> The file{} type can do all of the following:
>> * manage symlinks
>> How would people feel about at least splitting out these into their own
types?
>>
>> * symlinks
> On a final note ... can we not overload "ensure" in our new
types? :)
I''m a +1 on splitting out symlinks. It''s been one of the
biggest catches
for people when I''ve been teaching them puppet. Once that''s
split out
removing the overloading of ensure for the file type and replacing it with
some stricter validation would be excellent.

  Dean
-- 
Dean Wilson
http://www.unixdaemon.net       @unixdaemon
http://www.puppetcookbook.com   @puppetcookbook

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Nigel Kersten:
> The file{} type can do all of the following:
> 
> * manage single files
> * manage directories
> * manage symlinks
> * manage recursive file copies
I tend to use different defaults for the different types, so I would
love to be able to do:

	File { mode => 0444 }
	Directory { mode => 0555 }

(I use read-only status as a final safety net for sysadmins who run
"sudo vi <file>" on something managed by puppet)

...and leave Symlink types alone (since that all-bits-high mode is one
of the ways I pick them out in an ls -l lineup sometimes, and is
probably functionally significant somehow).

And as was mentioned elsewhere, the recursive copy off of the file
server is probably the quirkiest use of File resources.

-- 
"Forget the damned motor car
and build cities for lovers and friends."
       -- Lewis Mumford

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mar 21, 8:53 pm, Nigel Kersten <ni...@puppetlabs.com>
wrote:
> The file{} type can do all of the following:
>
> * manage single files
> * manage directories
> * manage symlinks
> * manage recursive file copies
>
> The intersection of all these bits of functionality makes it difficult
> to understand exactly what is going on when you''re new to Puppet,
and
> even experienced users often don''t know how combining
symlinks/content
> management is going to work.
>
> How would people feel about at least splitting out these into their own
types?
>
> * symlinks
> * recursive file copies
>
> The intersection of files and directories isn''t that big a deal,
but
> we could split out directories too if we wanted.
>
> Thoughts?
I agree that File is a mishmash, but I don''t think symlinks and
recursive copying are the key concepts that would be good to split
out.  Instead, I think splitting directories into their own type would
be the way to go.

Consider what would happen if symlinks were made their own type.  What
about dependencies?  Right now, I can have

service { "my_service": require =>
File["/etc/my_service.conf"] }

without caring whether File["/etc/my_service.conf"] represents an
actual file or a symlink.  I can even change that in the declaration
of the file without having to touch anything that depends on it.  If
symlinks were modeled via a separate type, however, then I would need
everywhere to account for which files were plain and which were
symlinks.

Or look at it from a modelling angle: a symlink to a regular file is
much more like a regular file than a directory is like a regular file,
so why does it make sense to split out symlinks but not directories?

Parallel arguments can be made about directories and symlinks to
directories.

As for recursive copying, that''s an action, not an observable,
manageable artifact, so why would it make sense to create a resource
type around it?  It could be recast as something like "directory
hierarchy", but that begs the question of why it should be separate
from ordinary directories.  If you want to think out of the box, then
consider re-implementing recursive directory management via a new
(type of) function that dynamically adds all the appropriate Directory
and File resources to the catalog.  That''s anyway what Puppet already
does, right?


This, then, is the direction that makes the most sense to me:

1) Split out (only) directories into their own type.  Among other
things, recursive-tree management would go into the new Directory
type.
2) Give File and Directory each a "link_to" property by which these
types can be made to manage symbolic links instead of the underlying
regular file or directory.
3) Once (1) and (2) are done, it will be possible and appropriate to
limit the allowed values of both types'' "ensure" properties
to
"absent" and "present".

I recommend seeing how (1) works out before trying to move recursive
directory management into its own entity.  If that feature is indeed
moved out, however, then I truly don''t see how it would make sense to
make a resource type out of it.  Making a function out of it instead
would be a better fit.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tue, Mar 22, 2011 at 7:20 AM, jcbollinger <John.Bollinger@stjude.org>
wrote:
>
>
> On Mar 21, 8:53 pm, Nigel Kersten <ni...@puppetlabs.com> wrote:
>> The file{} type can do all of the following:
>>
>> * manage single files
>> * manage directories
>> * manage symlinks
>> * manage recursive file copies
>>
>> The intersection of all these bits of functionality makes it difficult
>> to understand exactly what is going on when you''re new to
Puppet, and
>> even experienced users often don''t know how combining
symlinks/content
>> management is going to work.
>>
>> How would people feel about at least splitting out these into their own
types?
>>
>> * symlinks
>> * recursive file copies
>>
>> The intersection of files and directories isn''t that big a
deal, but
>> we could split out directories too if we wanted.
>>
>> Thoughts?
>
> I agree that File is a mishmash, but I don''t think symlinks and
> recursive copying are the key concepts that would be good to split
> out.  Instead, I think splitting directories into their own type would
> be the way to go.
>
> Consider what would happen if symlinks were made their own type.  What
> about dependencies?  Right now, I can have
>
> service { "my_service": require =>
File["/etc/my_service.conf"] }
>
> without caring whether File["/etc/my_service.conf"] represents an
> actual file or a symlink.  I can even change that in the declaration
> of the file without having to touch anything that depends on it.  If
> symlinks were modeled via a separate type, however, then I would need
> everywhere to account for which files were plain and which were
> symlinks.
That''s a really good point. One workaround would be to encapsulate
such configs into a class and require that.

class foo::service {
  service { "my_service": require => Class["foo::config"]
}
}

class foo::config { ... }

Another would be to flip this around and instead use before instead of
require, so the service resource wouldn''t need to know what kind of
object is required.
>
> Or look at it from a modelling angle: a symlink to a regular file is
> much more like a regular file than a directory is like a regular file,
> so why does it make sense to split out symlinks but not directories?
Because of the clash between defining a symlink and specifying the
content of a file.

We have edge cases like this:

file { "/tmp/someobject":
  ensure => present,
  content => "foo",
}

Now if /tmp/someobject is a symlink (or even a directory), we need to
special case the code so that we log that the content attribute isn''t
being used.

If it''s a file, it will be used.

It gets worse with the "links" parameter.

file { "/tmp/foo":
  ensure => present,
  links => follow,
  recurse => true,
  source => "....",
}

This does all sorts of weird things depending upon whether the object
is a symlink, directory or file.

We''ve had requests to support sockets in the file type too, which
complicate things further.
>
> Parallel arguments can be made about directories and symlinks to
> directories.
>
> As for recursive copying, that''s an action, not an observable,
> manageable artifact, so why would it make sense to create a resource
> type around it?  It could be recast as something like "directory
> hierarchy", but that begs the question of why it should be separate
> from ordinary directories.  If you want to think out of the box, then
> consider re-implementing recursive directory management via a new
> (type of) function that dynamically adds all the appropriate Directory
> and File resources to the catalog.  That''s anyway what Puppet
already
> does, right?
We have fundamentally different kinds of parameters on a recursive
file source than we do on a normal directory.

Think about the clash between source and content. links. purge.
recurse. recurselimit. ignore.

All those things *only* make sense with a recursive tree, not with a
single file or a single directory.

>
>
> This, then, is the direction that makes the most sense to me:
>
> 1) Split out (only) directories into their own type.  Among other
> things, recursive-tree management would go into the new Directory
> type.
> 2) Give File and Directory each a "link_to" property by which
these
> types can be made to manage symbolic links instead of the underlying
> regular file or directory.
like our existing "target" property? How does it make sense to manage
a symlink in a Directory type? I''m not seeing it....
> 3) Once (1) and (2) are done, it will be possible and appropriate to
> limit the allowed values of both types'' "ensure"
properties to
> "absent" and "present".
>
> I recommend seeing how (1) works out before trying to move recursive
> directory management into its own entity.  If that feature is indeed
> moved out, however, then I truly don''t see how it would make sense
to
> make a resource type out of it.  Making a function out of it instead
> would be a better fit.
I think you''re overlooking the configurable properties for recursive
copies. A function isn''t going to give you all the options that are
widely used now.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tue, Mar 22, 2011 at 08:10:40AM -0700, Nigel Kersten
wrote:
> On Tue, Mar 22, 2011 at 7:20 AM, jcbollinger
<John.Bollinger@stjude.org> wrote:
> >
> >
> > On Mar 21, 8:53 pm, Nigel Kersten <ni...@puppetlabs.com> wrote:
[...]
> >> Thoughts?
> >
> > I agree that File is a mishmash, but I don''t think symlinks
and
> > recursive copying are the key concepts that would be good to split
> > out.  Instead, I think splitting directories into their own type would
> > be the way to go.
> >
> > Consider what would happen if symlinks were made their own type.  What
> > about dependencies?  Right now, I can have
> >
> > service { "my_service": require =>
File["/etc/my_service.conf"] }
> >
> > without caring whether File["/etc/my_service.conf"]
represents an
> > actual file or a symlink.  I can even change that in the declaration
> > of the file without having to touch anything that depends on it.  If
> > symlinks were modeled via a separate type, however, then I would need
> > everywhere to account for which files were plain and which were
> > symlinks.
> 
> That''s a really good point. One workaround would be to encapsulate
> such configs into a class and require that.
> 
[...]
> > This, then, is the direction that makes the most sense to me:
> >
> > 1) Split out (only) directories into their own type.  Among other
> > things, recursive-tree management would go into the new Directory
> > type.
> > 2) Give File and Directory each a "link_to" property by
which these
> > types can be made to manage symbolic links instead of the underlying
> > regular file or directory.
> 
> like our existing "target" property? How does it make sense to
manage
> a symlink in a Directory type? I''m not seeing it....
> 
Same reason like above. If you''re doing something like

  case $operatingsystem {
    ''Solaris'': {
      # Filetype or even new link type
      file { ''/foo'': target =>
''/linktarget'', ensure => link }
    }
    default {
      directory { ''/foo'': ensure => present}
    }
  }

You cannot setup a dependency like require =>
Directory[''/foo'']. If the
directory type has a target attribute and ensure => link, this would be
possible.

While I think that this is a good point I still prefer splitting into
file, directory and directorytree (or similar).

One note about recursive copies. I have one case where I want to
controll owner/group/mode on a toplevel directory and just owner/group
for all the children in that directory. Thats not possible at the moment
(or not that I''m aware of). If I now can define
Directorytree[''foo''] and
Directory[''foo''] I would be able to manage different aspects
for parent
directory and children.

So +1 for Nigel''s initial post from me.

-Stefan

On Monday, March 21, 2011 9:53:26 PM UTC-4, Nigel Kersten
wrote:
>
> The file{} type can do all of the following:
>
> * manage single files
> * manage directories
> * manage symlinks
> * manage recursive file copies
>
> The intersection of all these bits of functionality makes it difficult
> to understand exactly what is going on when you''re new to Puppet,
and
> even experienced users often don''t know how combining
symlinks/content
> management is going to work.
>
> How would people feel about at least splitting out these into their own 
> types?
>
> * symlinks
> * recursive file copies
>
> The intersection of files and directories isn''t that big a deal,
but
> we could split out directories too if we wanted.
>
> Thoughts?
>
>
> -- 
> Nigel Kersten
> Product, Puppet Labs
> @nigelkersten
>
I''m not hugely up-to-date on Puppet internals when it comes to resource
name
resolution, but since they both manage the same underlying object (a path on 
the filesystem), doesn''t this introduce the potential issue that
someone can
define a File and a Symlink on the same path, and Puppet will try to manage 
both in no particular order?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tue, Mar 22, 2011 at 12:39 PM, Jeffrey Goldschrafe
<jeffrey_goldschrafe@timeinc.com> wrote:
> On Monday, March 21, 2011 9:53:26 PM UTC-4, Nigel Kersten wrote:
>>
>> The file{} type can do all of the following:
>>
>> * manage single files
>> * manage directories
>> * manage symlinks
>> * manage recursive file copies
>>
>> The intersection of all these bits of functionality makes it difficult
>> to understand exactly what is going on when you''re new to
Puppet, and
>> even experienced users often don''t know how combining
symlinks/content
>> management is going to work.
>>
>> How would people feel about at least splitting out these into their own
>> types?
>>
>> * symlinks
>> * recursive file copies
>>
>> The intersection of files and directories isn''t that big a
deal, but
>> we could split out directories too if we wanted.
>>
>> Thoughts?
>>
>> --
>> Nigel Kersten
>> Product, Puppet Labs
>> @nigelkersten
>
> I''m not hugely up-to-date on Puppet internals when it comes to
resource name
> resolution, but since they both manage the same underlying object (a path
on
> the filesystem), doesn''t this introduce the potential issue that
someone can
> define a File and a Symlink on the same path, and Puppet will try to manage
> both in no particular order?
If we implemented this badly, yes :)

Seriously though, this is why we have all this functionality in the
one type, but I''ve come around to the opinion that the user-facing
confusion and pain isn''t worth it. We''ve ended up with
difficult code
to maintain and document and spend way too much effort triaging bugs
around it.

Once we''ve worked out how we want it to behave, we can work out how to
achieve that.





-- 
Nigel Kersten
Product, Puppet Labs
@nigelkersten

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mar 22, 10:10 am, Nigel Kersten <ni...@puppetlabs.com>
wrote:
> On Tue, Mar 22, 2011 at 7:20 AM, jcbollinger
<John.Bollin...@stjude.org> wrote:
>
> > On Mar 21, 8:53 pm, Nigel Kersten <ni...@puppetlabs.com> wrote:
> >> The file{} type can do all of the following:
>
> >> * manage single files
> >> * manage directories
> >> * manage symlinks
> >> * manage recursive file copies
>
> >> The intersection of all these bits of functionality makes it
difficult
> >> to understand exactly what is going on when you''re new to
Puppet, and
> >> even experienced users often don''t know how combining
symlinks/content
> >> management is going to work.
>
> >> How would people feel about at least splitting out these into
their own types?
>
> >> * symlinks
> >> * recursive file copies
>
> >> The intersection of files and directories isn''t that big
a deal, but
> >> we could split out directories too if we wanted.
>
> >> Thoughts?
>
> > I agree that File is a mishmash, but I don''t think symlinks
and
> > recursive copying are the key concepts that would be good to split
> > out.  Instead, I think splitting directories into their own type would
> > be the way to go.
>
> > Consider what would happen if symlinks were made their own type.  What
> > about dependencies?  Right now, I can have
>
> > service { "my_service": require =>
File["/etc/my_service.conf"] }
>
> > without caring whether File["/etc/my_service.conf"]
represents an
> > actual file or a symlink.  I can even change that in the declaration
> > of the file without having to touch anything that depends on it.  If
> > symlinks were modeled via a separate type, however, then I would need
> > everywhere to account for which files were plain and which were
> > symlinks.
>
> That''s a really good point. One workaround would be to encapsulate
> such configs into a class and require that.
>
> class foo::service {
>   service { "my_service": require =>
Class["foo::config"] }
>
> }
>
> class foo::config { ... }
>
> Another would be to flip this around and instead use before instead of
> require, so the service resource wouldn''t need to know what kind
of
> object is required.

Ok, but the existence of workarounds to the problem doesn''t mean there
isn''t a problem.  The systems under management draw little distinction
between regular files and symbolic links to files, so at best it would
be quirky for Puppet to draw a substantially larger distinction.  It
is likely that situations would be discovered for which none of the
workarounds is very good.  Instead of solving the problem, you would
merely exchange one set of problems for a different, perhaps smaller,
set.

> > Or look at it from a modelling angle: a symlink to a regular file is
> > much more like a regular file than a directory is like a regular file,
> > so why does it make sense to split out symlinks but not directories?
>
> Because of the clash between defining a symlink and specifying the
> content of a file.
>
> We have edge cases like this:
>
> file { "/tmp/someobject":
>   ensure => present,
>   content => "foo",
>
> }
>
> Now if /tmp/someobject is a symlink (or even a directory), we need to
> special case the code so that we log that the content attribute
isn''t
> being used.
>
> If it''s a file, it will be used.

I''m suggesting that directories get their own type, so they''re
not
germane here.

I''m also suggesting that the new File type know whether the target is
supposed to be a symlink (which would also trivially be the case if
symlinks were split out), so it would be reasonable for a File
managing a symlink and specifying content to manage the content of the
link target.  That would be pretty natural, in fact.

> It gets worse with the "links" parameter.
>
> file { "/tmp/foo":
>   ensure => present,
>   links => follow,
>   recurse => true,
>   source => "....",
>
> }

No it doesn''t, because the File type I''m proposing (like the
one you
started with) does not manage directories or, therefore, support
recursion.

Moreover, with no ambiguity about whether the managed object is
expected to be a symlink, the "links" property would be unnecessary.

> This does all sorts of weird things depending upon whether the object
> is a symlink, directory or file.
>
> We''ve had requests to support sockets in the file type too, which
> complicate things further.

More on that below...

> > Parallel arguments can be made about directories and symlinks to
> > directories.
>
> > As for recursive copying, that''s an action, not an
observable,
> > manageable artifact, so why would it make sense to create a resource
> > type around it?  It could be recast as something like "directory
> > hierarchy", but that begs the question of why it should be
separate
> > from ordinary directories.  If you want to think out of the box, then
> > consider re-implementing recursive directory management via a new
> > (type of) function that dynamically adds all the appropriate Directory
> > and File resources to the catalog.  That''s anyway what Puppet
already
> > does, right?
>
> We have fundamentally different kinds of parameters on a recursive
> file source than we do on a normal directory.
>
> Think about the clash between source and content. links. purge.
> recurse. recurselimit. ignore.
>
> All those things *only* make sense with a recursive tree, not with a
> single file or a single directory.

I recognize that, but it doesn''t change the fact that a directory with
recursively copied contents is still a directory, nor the fact that
the copy *operation* is not a manageable artifact.  As I understand it
-- which is imperfectly -- Puppet currently handles recursive
directories basically by creating resource objects for the whole
hierarchy of files and subdirectories.  I''m suggesting that the Puppet
DSL could provide direct access to that resource generator as an
alternative to having distinct Directory and RecursiveDirectory
types.  That need not be done via a function per se; perhaps there
would be a more general use for such dynamic resource generators,
sufficient to justify adding support for such things to the framework.

> > This, then, is the direction that makes the most sense to me:
>
> > 1) Split out (only) directories into their own type.  Among other
> > things, recursive-tree management would go into the new Directory
> > type.
> > 2) Give File and Directory each a "link_to" property by
which these
> > types can be made to manage symbolic links instead of the underlying
> > regular file or directory.
>
> like our existing "target" property?

Umm... yeah.  Sorry, brain fart there.  Still, that property might
admit a more descriptive name if it were the only thing indicating
that the resource represented a symlink.

>   How does it make sense to manage
> a symlink in a Directory type? I''m not seeing it....

It makes sense if the target of the symlink is expected to be a
directory.  (And the target of a File-managed symlink is expected to
be a regular file.)

> > 3) Once (1) and (2) are done, it will be possible and appropriate to
> > limit the allowed values of both types'' "ensure"
properties to
> > "absent" and "present".
>
> > I recommend seeing how (1) works out before trying to move recursive
> > directory management into its own entity.  If that feature is indeed
> > moved out, however, then I truly don''t see how it would make
sense to
> > make a resource type out of it.  Making a function out of it instead
> > would be a better fit.
>
> I think you''re overlooking the configurable properties for
recursive
> copies. A function isn''t going to give you all the options that
are
> widely used now.

No, I''m not.  The expressive capability of a function call is no less
than that of a resource declaration.  A function might need a dozen
parameters, and might therefore be messy to use, but one *could*
provide all the same facilities.  Maybe the potential mess is good
argument for looking for something new, such as the resource
generators I speculated about above.

On the other hand, perhaps the new thing needed is something
different: what about resource sub-types?  At some levels on which we
might want to manage system configuration, we might not care whether a
file is regular or a directory or a device node, or a symbolic link to
one of the above.  At some other levels, however, those distinctions
can be very important.  So suppose, for example, we have the
alternative of declaring either

plainfile { "/etc/profile.d/foo.sh": source => "..." }

or

filelink { "/etc/profile.d/foo.sh": target =>
"/opt/foo/foo_env.sh" }

but they are mutually exclusive, and either one satisfies the
requirement in

service { "foo": require => File["/etc/profile.d/foo.sh"]
}


Such a framework could be flexible enough to support various sorts of
special files (e.g. device nodes) without making an unmaintainable
shambles out of the File type.  Most importantly, it models the
problem domain better than would addressing plain files and symlinks
via separate, unrelated types.


Best,

John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

What initially attracted me to puppet wast that puppet described state and
not (in general) actions to achieve that state. Yet, I, too, was initially
confued that to manage symlinks I needed to manage "files" which had
something to "ensure" called "symlink". To make sure a
directory is absent I
have to use this same "file" approach.

So, +1

Please, let us move to managing symlinks and recursive file copies, but do
look closely at splitting out directories and files as well.

~Charles~

On Mon, Mar 21, 2011 at 8:53 PM, Nigel Kersten <nigel@puppetlabs.com>
wrote:
> The file{} type can do all of the following:
>
> * manage single files
> * manage directories
> * manage symlinks
> * manage recursive file copies
>
> The intersection of all these bits of functionality makes it difficult
> to understand exactly what is going on when you''re new to Puppet,
and
> even experienced users often don''t know how combining
symlinks/content
> management is going to work.
>
> How would people feel about at least splitting out these into their own
> types?
>
> * symlinks
> * recursive file copies
>
> The intersection of files and directories isn''t that big a deal,
but
> we could split out directories too if we wanted.
>
> Thoughts?
>
>
> --
> Nigel Kersten
> Product, Puppet Labs
> @nigelkersten
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Am 22.03.2011 02:53, schrieb Nigel Kersten:
> The intersection of files and directories isn''t that big a deal,
but
> we could split out directories too if we wanted.
 From the user''s perspective, it''s more like the other way
round. We should
not let implementation issues guide the design of the manifest language. Files 
and directories are different concepts, but symlinks to either files or 
directories are handled mostly transparently.

I would rather suggest a File and a Directory type. Both types should share 
basic attributes like owner/mode etc and both should support symlinks. Each 
one has unique attributes: files should support source/content, while 
directories should support the recursive copy feature.

Regards

Christian

-- 
Dipl.-Inf. Christian Kauhaus <>< · kc@gocept.com · systems
administration
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 11 · fax +49 345 1229889 1
Zope and Plone consulting and development

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tue, Mar 22, 2011 at 3:06 PM, Christian Kauhaus <kc@gocept.com>
wrote:
> Am 22.03.2011 02:53, schrieb Nigel Kersten:
>>
>> The intersection of files and directories isn''t that big a
deal, but
>> we could split out directories too if we wanted.
>
> From the user''s perspective, it''s more like the other way
round. We should
> not let implementation issues guide the design of the manifest language.
> Files and directories are different concepts, but symlinks to either files
> or directories are handled mostly transparently.
>
> I would rather suggest a File and a Directory type. Both types should share
> basic attributes like owner/mode etc and both should support symlinks. Each
> one has unique attributes: files should support source/content, while
> directories should support the recursive copy feature.
If you''re enabling recursive copies for Directories, then
you''re also
supporting the ''source'' property, and you''re also
supporting the
"links => {follow, manage, ignore}" parameter and recurse and
recurselimit....

I don''t think this gets us to a significantly better position for the
user.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Am 22.03.2011 23:52, schrieb Nigel Kersten:
> If you''re enabling recursive copies for Directories, then
you''re also
> supporting the ''source'' property, and you''re
also supporting the
> "links => {follow, manage, ignore}" parameter and recurse and
> recurselimit....
I see your point.

As an alternative, extracting the recursive directory copy facility is
definitely a step in the right direction. I''m not sure if the symlink
feature should be extracted too. It should at least get a different
parameter name, though. The overloaded ''ensure'' is confusing.

Regards

Christian

-- 
Dipl.-Inf. Christian Kauhaus <>< · kc@gocept.com · systems
administration
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 11 · fax +49 345 1229889 1
Zope and Plone consulting and development

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

jcbollinger:
> On the other hand, perhaps the new thing needed is something
> different: what about resource sub-types? [...]
> plainfile { "/etc/profile.d/foo.sh": source => "..."
}
> filelink { "/etc/profile.d/foo.sh": target =>
"/opt/foo/foo_env.sh" }
> service { "foo": require =>
File["/etc/profile.d/foo.sh"] }
...
> it models the problem domain better
This is an excellent idea. It breaks the exact correspondence between
file/File, service/Service, etc., but that''s easier to explain than
symlinks and recursion (if not easier to implement).

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Nigel Kersten wrote:
> How would people feel about at least splitting out these into their own
types?
> 
> * symlinks
> * recursive file copies
> 
> The intersection of files and directories isn''t that big a deal,
but
> we could split out directories too if we wanted.
A tentative -1 from me.  I haven''t had any problems myself with the
file type, and I haven''t heard anyone of my colleagues complaining
either.  In Unix, files are files are files, whether they happen to
be plain files, directories, symlinks, device nodes or named pipes.

But mostly I think we need a full specification to judge on.  There
are many cases and interactions that needs to be considered, and it
isn''t obvious that either way is really better without a detailed
description of how they will work and interact with each other.

My gut feeling, though, is that you will find just as many edge
cases and complexities if you split it into multiple types, just
that they will then be spread over several *interacting* types.


	/Bellman

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I''d really prefer if the name of this resource didn''t change.
I
understand there are problems but can''t you just split the code and
have
different behavior based on something like filetype =>?

On 03/22/11 11:10, Nigel Kersten wrote:
> On Tue, Mar 22, 2011 at 7:20 AM,
jcbollinger<John.Bollinger@stjude.org>  wrote:
>>
>> On Mar 21, 8:53 pm, Nigel Kersten<ni...@puppetlabs.com>  wrote:
>>> The file{} type can do all of the following:
>>>
>>> * manage single files
>>> * manage directories
>>> * manage symlinks
>>> * manage recursive file copies
>>>
>>> The intersection of all these bits of functionality makes it
difficult
>>> to understand exactly what is going on when you''re new to
Puppet, and
>>> even experienced users often don''t know how combining
symlinks/content
>>> management is going to work.
>>>
>>> How would people feel about at least splitting out these into their
own types?
>>>
>>> * symlinks
>>> * recursive file copies
>>>
>>> The intersection of files and directories isn''t that big a
deal, but
>>> we could split out directories too if we wanted.
>>>
>>> Thoughts?
>> I agree that File is a mishmash, but I don''t think symlinks
and
>> recursive copying are the key concepts that would be good to split
>> out.  Instead, I think splitting directories into their own type would
>> be the way to go.
>>
>> Consider what would happen if symlinks were made their own type.  What
>> about dependencies?  Right now, I can have
>>
>> service { "my_service": require => 
File["/etc/my_service.conf"] }
>>
>> without caring whether File["/etc/my_service.conf"]
represents an
>> actual file or a symlink.  I can even change that in the declaration
>> of the file without having to touch anything that depends on it.  If
>> symlinks were modeled via a separate type, however, then I would need
>> everywhere to account for which files were plain and which were
>> symlinks.
> That''s a really good point. One workaround would be to encapsulate
> such configs into a class and require that.
>
> class foo::service {
>    service { "my_service": require => 
Class["foo::config"] }
> }
>
> class foo::config { ... }
>
> Another would be to flip this around and instead use before instead of
> require, so the service resource wouldn''t need to know what kind
of
> object is required.
>
>> Or look at it from a modelling angle: a symlink to a regular file is
>> much more like a regular file than a directory is like a regular file,
>> so why does it make sense to split out symlinks but not directories?
> Because of the clash between defining a symlink and specifying the
> content of a file.
>
> We have edge cases like this:
>
> file { "/tmp/someobject":
>    ensure =>  present,
>    content =>  "foo",
> }
>
> Now if /tmp/someobject is a symlink (or even a directory), we need to
> special case the code so that we log that the content attribute
isn''t
> being used.
>
> If it''s a file, it will be used.
>
> It gets worse with the "links" parameter.
>
> file { "/tmp/foo":
>    ensure =>  present,
>    links =>  follow,
>    recurse =>  true,
>    source =>  "....",
> }
>
> This does all sorts of weird things depending upon whether the object
> is a symlink, directory or file.
>
> We''ve had requests to support sockets in the file type too, which
> complicate things further.
>
>> Parallel arguments can be made about directories and symlinks to
>> directories.
>>
>> As for recursive copying, that''s an action, not an observable,
>> manageable artifact, so why would it make sense to create a resource
>> type around it?  It could be recast as something like "directory
>> hierarchy", but that begs the question of why it should be
separate
>> from ordinary directories.  If you want to think out of the box, then
>> consider re-implementing recursive directory management via a new
>> (type of) function that dynamically adds all the appropriate Directory
>> and File resources to the catalog.  That''s anyway what Puppet
already
>> does, right?
> We have fundamentally different kinds of parameters on a recursive
> file source than we do on a normal directory.
>
> Think about the clash between source and content. links. purge.
> recurse. recurselimit. ignore.
>
> All those things *only* make sense with a recursive tree, not with a
> single file or a single directory.
>
>
>>
>> This, then, is the direction that makes the most sense to me:
>>
>> 1) Split out (only) directories into their own type.  Among other
>> things, recursive-tree management would go into the new Directory
>> type.
>> 2) Give File and Directory each a "link_to" property by which
these
>> types can be made to manage symbolic links instead of the underlying
>> regular file or directory.
> like our existing "target" property? How does it make sense to
manage
> a symlink in a Directory type? I''m not seeing it....
>
>> 3) Once (1) and (2) are done, it will be possible and appropriate to
>> limit the allowed values of both types'' "ensure"
properties to
>> "absent" and "present".
>>
>> I recommend seeing how (1) works out before trying to move recursive
>> directory management into its own entity.  If that feature is indeed
>> moved out, however, then I truly don''t see how it would make
sense to
>> make a resource type out of it.  Making a function out of it instead
>> would be a better fit.
> I think you''re overlooking the configurable properties for
recursive
> copies. A function isn''t going to give you all the options that
are
> widely used now.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tue, Mar 29, 2011 at 8:16 AM, Joe McDonagh
<joseph.e.mcdonagh@gmail.com> wrote:
>  I''d really prefer if the name of this resource didn''t
change. I understand
> there are problems but can''t you just split the code and have
different
> behavior based on something like filetype =>?
We''re not going to rush into this, and we''ll try to maintain
backwards
compatibility as much as possible.


>
> On 03/22/11 11:10, Nigel Kersten wrote:
>>
>> On Tue, Mar 22, 2011 at 7:20 AM,
jcbollinger<John.Bollinger@stjude.org>
>>  wrote:
>>>
>>> On Mar 21, 8:53 pm, Nigel Kersten<ni...@puppetlabs.com>
 wrote:
>>>>
>>>> The file{} type can do all of the following:
>>>>
>>>> * manage single files
>>>> * manage directories
>>>> * manage symlinks
>>>> * manage recursive file copies
>>>>
>>>> The intersection of all these bits of functionality makes it
difficult
>>>> to understand exactly what is going on when you''re new
to Puppet, and
>>>> even experienced users often don''t know how combining
symlinks/content
>>>> management is going to work.
>>>>
>>>> How would people feel about at least splitting out these into
their own
>>>> types?
>>>>
>>>> * symlinks
>>>> * recursive file copies
>>>>
>>>> The intersection of files and directories isn''t that
big a deal, but
>>>> we could split out directories too if we wanted.
>>>>
>>>> Thoughts?
>>>
>>> I agree that File is a mishmash, but I don''t think
symlinks and
>>> recursive copying are the key concepts that would be good to split
>>> out.  Instead, I think splitting directories into their own type
would
>>> be the way to go.
>>>
>>> Consider what would happen if symlinks were made their own type.
 What
>>> about dependencies?  Right now, I can have
>>>
>>> service { "my_service": require =>
 File["/etc/my_service.conf"] }
>>>
>>> without caring whether File["/etc/my_service.conf"]
represents an
>>> actual file or a symlink.  I can even change that in the
declaration
>>> of the file without having to touch anything that depends on it.
 If
>>> symlinks were modeled via a separate type, however, then I would
need
>>> everywhere to account for which files were plain and which were
>>> symlinks.
>>
>> That''s a really good point. One workaround would be to
encapsulate
>> such configs into a class and require that.
>>
>> class foo::service {
>>   service { "my_service": require =>
 Class["foo::config"] }
>> }
>>
>> class foo::config { ... }
>>
>> Another would be to flip this around and instead use before instead of
>> require, so the service resource wouldn''t need to know what
kind of
>> object is required.
>>
>>> Or look at it from a modelling angle: a symlink to a regular file
is
>>> much more like a regular file than a directory is like a regular
file,
>>> so why does it make sense to split out symlinks but not
directories?
>>
>> Because of the clash between defining a symlink and specifying the
>> content of a file.
>>
>> We have edge cases like this:
>>
>> file { "/tmp/someobject":
>>   ensure =>  present,
>>   content =>  "foo",
>> }
>>
>> Now if /tmp/someobject is a symlink (or even a directory), we need to
>> special case the code so that we log that the content attribute
isn''t
>> being used.
>>
>> If it''s a file, it will be used.
>>
>> It gets worse with the "links" parameter.
>>
>> file { "/tmp/foo":
>>   ensure =>  present,
>>   links =>  follow,
>>   recurse =>  true,
>>   source =>  "....",
>> }
>>
>> This does all sorts of weird things depending upon whether the object
>> is a symlink, directory or file.
>>
>> We''ve had requests to support sockets in the file type too,
which
>> complicate things further.
>>
>>> Parallel arguments can be made about directories and symlinks to
>>> directories.
>>>
>>> As for recursive copying, that''s an action, not an
observable,
>>> manageable artifact, so why would it make sense to create a
resource
>>> type around it?  It could be recast as something like
"directory
>>> hierarchy", but that begs the question of why it should be
separate
>>> from ordinary directories.  If you want to think out of the box,
then
>>> consider re-implementing recursive directory management via a new
>>> (type of) function that dynamically adds all the appropriate
Directory
>>> and File resources to the catalog.  That''s anyway what
Puppet already
>>> does, right?
>>
>> We have fundamentally different kinds of parameters on a recursive
>> file source than we do on a normal directory.
>>
>> Think about the clash between source and content. links. purge.
>> recurse. recurselimit. ignore.
>>
>> All those things *only* make sense with a recursive tree, not with a
>> single file or a single directory.
>>
>>
>>>
>>> This, then, is the direction that makes the most sense to me:
>>>
>>> 1) Split out (only) directories into their own type.  Among other
>>> things, recursive-tree management would go into the new Directory
>>> type.
>>> 2) Give File and Directory each a "link_to" property by
which these
>>> types can be made to manage symbolic links instead of the
underlying
>>> regular file or directory.
>>
>> like our existing "target" property? How does it make sense
to manage
>> a symlink in a Directory type? I''m not seeing it....
>>
>>> 3) Once (1) and (2) are done, it will be possible and appropriate
to
>>> limit the allowed values of both types'' "ensure"
properties to
>>> "absent" and "present".
>>>
>>> I recommend seeing how (1) works out before trying to move
recursive
>>> directory management into its own entity.  If that feature is
indeed
>>> moved out, however, then I truly don''t see how it would
make sense to
>>> make a resource type out of it.  Making a function out of it
instead
>>> would be a better fit.
>>
>> I think you''re overlooking the configurable properties for
recursive
>> copies. A function isn''t going to give you all the options
that are
>> widely used now.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>


-- 
Nigel Kersten
Product, Puppet Labs
@nigelkersten

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

> I tend to use different defaults for the different types, so I would
> love to be able to do:
>
> 	File { mode =>  0444 }
> 	Directory { mode =>  0555 }
This is not necessary at all. If you assign "mode => 444" to a 
directory, Puppet will automatically set the "execution" permission so
it will actually be "555".

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I am unsure that necessity is the point.

The discussion as I understand it is whether or not splitting up the file
type would be helpful especially for those who are just coming to puppet.
Would it help reduce confusion and lower the learning curve. While my
background sees "everything as a file", nonetheless I do think it
would help
make the learning curve less steep. After all, this is a business venture
for PuppetLabs and anything that is well thought through which simplifies
access to their product seems like a good idea.

+1

~Charles~

2011/3/30 Carles Amigó <carles.amigo@softonic.com>
> I tend to use different defaults for the different types, so I would
>> love to be able to do:
>>
>>        File { mode =>  0444 }
>>        Directory { mode =>  0555 }
>>
>
> This is not necessary at all. If you assign "mode => 444" to a
directory,
> Puppet will automatically set the "execution" permission so it
will actually
> be "555".
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Am 29.03.2011 18:02, schrieb Nigel Kersten:
> On Tue, Mar 29, 2011 at 8:16 AM, Joe McDonagh
> <joseph.e.mcdonagh@gmail.com>  wrote:
>>   I''d really prefer if the name of this resource
didn''t change. I understand
>> there are problems but can''t you just split the code and have
different
>> behavior based on something like filetype =>?
>
> We''re not going to rush into this, and we''ll try to
maintain backwards
> compatibility as much as possible.
I''m probably totally wrong and didn''t follow the puppet
development tree
for a while, but I guess we should do the following:

1. Directories
We could split out directory-management and that would totally make 
sense, because you handle files and directories differently.
However, from the admin''s POV a symlink to a directory is a directory.
Having said that, it should be possible to do the following:

Directory { "/u":
	ensure	=> link, (or maybe just ensure => present)
	target	=> "/home",
	...
}

To ensure a symlink /u that points to a directory /home. As we now know 
this should be a directory the provider can fail if $target is not a 
directory and we can also generate a dependency on Directory["/home"].

This would split out directory handling, allow for symlinks to 
directories and would even make the symlinking more deterministic, 
because you can check wether the target really is a directory.

2. Files
I''d suggest not to extend the possibilities of file any more. It should
be able to ensure wether a file should be present or not and what 
content should be inside.
As for directories the provider should check the target for the link 
really *is* a plain file.

3. Sockets, FIFOs, devices, whatnot
When handling stuff like this you should usually know what exactly 
you''re doing and you mean what you say (i.e. if you create a socket you
need a socket and not a fifo, dev or plain file).
Thus every new filetype could be done as a new resource (hopefully 
sharing a lot of code with files and directories) with its own specific 
parameters. E.g. a Device needs block/char, major and minor just like a 
file needs its content.

4. Recursive Tree Copy
The recursive tree copy has always been a bit weird. Don''t get me
wrong,
but in puppet you''re usually explicit: To ensure the presence of a set 
of files you''d explicitly list each file in your manifest.
Obviously "the right thing" for a recusive tree copy would be to do 
exactly that: explicitly list every file and directory in your manifest.
If that could be done at compile-time all was well.
What we need for that is:
- function to list all files/directories/sockets/whatever inside a tree
- a method to loop over that list and generate the corresponding set of 
resources

Maybe general approach would help here. As far as I understood it is now 
possible to return nested hashes from functions. Thus one could easily 
write a function that for a given tree returns a hash that looks like this:
{
"/path/to/topdir" => {
			type => directory,
			mode => 0755,
			[...]
		}
"/path/to/topdir/file" => {
			type => file,
			source => "puppet:///modules/xyz/topdir/file",
			mode => 644,
			[...]
		}
}

That can then be fed into a magic loop to generate
Directory { "/path/to/topdir":
	mode	=> 0755,
	[...]
}
File { "/path/to/topdir/file":
	source	=> "puppet:///modules/xyz/topdir/file",
	mode	=> 0644,
}

What do you think?

Regards,
Andreas

-- 
Solvention Ltd. & Co. KG
Egermannstr. 6-8
53359 Rheinbach

Tel: +49 2226 158179-0
Fax: +49 2226 158179-9

http://www.solvention.de
mailto:info@solvention.de

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mar 30, 11:05 am, Andreas Rogge <a.ro...@solvention.de>
wrote:
> Having said that, it should be possible to do the following:
>
> Directory { "/u":
>         ensure  => link, (or maybe just ensure => present)
>         target  => "/home",
>         ...
>
> }
>
> To ensure a symlink /u that points to a directory /home. As we now know
> this should be a directory the provider can fail if $target is not a
> directory and we can also generate a dependency on
Directory["/home"].
>
> This would split out directory handling, allow for symlinks to
> directories and would even make the symlinking more deterministic,
> because you can check wether the target really is a directory.

Yes, indeed.

> 2. Files
> I''d suggest not to extend the possibilities of file any more. It
should
> be able to ensure wether a file should be present or not and what
> content should be inside.
> As for directories the provider should check the target for the link
> really *is* a plain file.

Right on.

> 3. Sockets, FIFOs, devices, whatnot
> When handling stuff like this you should usually know what exactly
> you''re doing and you mean what you say (i.e. if you create a
socket you
> need a socket and not a fifo, dev or plain file).
> Thus every new filetype could be done as a new resource (hopefully
> sharing a lot of code with files and directories) with its own specific
> parameters. E.g. a Device needs block/char, major and minor just like a
> file needs its content.

Indeed.  And here, too, you might want to work in symlinks to these
objects, especially if symlinks managed by the regular file type (2
above) indeed require their target to be a regular file..

> 4. Recursive Tree Copy
> The recursive tree copy has always been a bit weird. Don''t get me
wrong,
> but in puppet you''re usually explicit: To ensure the presence of a
set
> of files you''d explicitly list each file in your manifest.
> Obviously "the right thing" for a recusive tree copy would be to
do
> exactly that: explicitly list every file and directory in your manifest.
> If that could be done at compile-time all was well.
> What we need for that is:
> - function to list all files/directories/sockets/whatever inside a tree
> - a method to loop over that list and generate the corresponding set of
> resources

And that''s similar to my "resource generator" idea, so +1
overall.

People seem to agree that recursive tree copying is a different sort
of beast from the file and directory management that the File type
otherwise does.  More open to debate, however, is whether that feature
should be split out into its own *type*, distinct from whatever type
manages directories without recursion.

I think not.  Either it should be implemented as some other kind of
thing (not resource type) as suggested here, or it should be a sub-
type of a Directory type (if resource sub-typing were introduced).
"Some other kind of thing" might leverage the existing Puppet
framework (i.e. functions, hashes, etc.) to a greater or lesser
extent, but it would be nice to avoid making the feature significantly
harder to describe in a manifest than it is now.

Idea: instead of functions producing hashes, new iterative statements
in Puppet DSL, etc., what about a function that *accepts* a hash as an
argument, where the keys match the recursion-related properties of the
current File type, and which introduces the appropriate resources into
the target node''s catalog?  For example:

recursive_tree({
  root => "/path/to/base/dir",
  source_root => "path/relative/to/modules/file/dir",
  recurse_limit => 2,
  owner => "some_user",
  group => "some_group",
  mode => "0644",
  links => "manage",
  ensure => present,
  purge => false
})

Taking a hash as an argument would provide for syntax similar to the
that of the File type, would more or less provide keyword arguments,
would support omitting properties.  In other words, it would yield
most of the syntax and usage advantages of doing recursive copying via
the File type.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Am 31.03.2011 16:56, schrieb jcbollinger:
>> 3. Sockets, FIFOs, devices, whatnot
> [...]
>
> Indeed.  And here, too, you might want to work in symlinks to these
> objects, especially if symlinks managed by the regular file type (2
> above) indeed require their target to be a regular file..
Yes, of course.
>
>
>> 4. Recursive Tree Copy
>> The recursive tree copy has always been a bit weird. Don''t get
me wrong,
>> but in puppet you''re usually explicit: To ensure the presence
of a set
>> of files you''d explicitly list each file in your manifest.
>> Obviously "the right thing" for a recusive tree copy would be
to do
>> exactly that: explicitly list every file and directory in your
manifest.
>> If that could be done at compile-time all was well.
>> What we need for that is:
>> - function to list all files/directories/sockets/whatever inside a tree
>> - a method to loop over that list and generate the corresponding set of
>> resources
>
>
> And that''s similar to my "resource generator" idea, so
+1 overall.
>
> People seem to agree that recursive tree copying is a different sort
> of beast from the file and directory management that the File type
> otherwise does.  More open to debate, however, is whether that feature
> should be split out into its own *type*, distinct from whatever type
> manages directories without recursion.
>
> I think not.  Either it should be implemented as some other kind of
> thing (not resource type) as suggested here, or it should be a sub-
> type of a Directory type (if resource sub-typing were introduced).
> "Some other kind of thing" might leverage the existing Puppet
> framework (i.e. functions, hashes, etc.) to a greater or lesser
> extent, but it would be nice to avoid making the feature significantly
> harder to describe in a manifest than it is now.
Yep.
>
> Idea: instead of functions producing hashes, new iterative statements
> in Puppet DSL, etc., what about a function that *accepts* a hash as an
> argument, where the keys match the recursion-related properties of the
> current File type, and which introduces the appropriate resources into
> the target node''s catalog?  For example:
>
> recursive_tree({
>    root =>  "/path/to/base/dir",
>    source_root =>  "path/relative/to/modules/file/dir",
>    recurse_limit =>  2,
>    owner =>  "some_user",
>    group =>  "some_group",
>    mode =>  "0644",
>    links =>  "manage",
>    ensure =>  present,
>    purge =>  false
> })
>
> Taking a hash as an argument would provide for syntax similar to the
> that of the File type, would more or less provide keyword arguments,
> would support omitting properties.  In other words, it would yield
> most of the syntax and usage advantages of doing recursive copying via
> the File type.
>
I was thinking about something like "foreach" in the DSL, so you could
simply write something like the following:

foreach($path in recursive_file_list(
		''puppet:///path/in/my/module'',
		''/path/to/base/dir'',
		2)
	) {
	if $path[type] == ''File'' {
		/* explicit setting of parameters */
		file { "$path[title]":
			source => "$path[source]",
			owner	=> "$path[owner]",
			mode	=> 0664,
		}			
	}
	else if $path[type] == ''Directory'' {
		/* implicitly taking all parameters from $path */
		directory { $path }
	}
	else if $path[type] == ''Device'' {
		/* combining both */
		device { $path }
		Device[$path[title]] {
			requires	=> File["/sbin/MAKEDEV"]
		}
	}
}

where recursive_file_list($puppet_uri, $client_path, $recursion) yields 
an array of hashes.
This allows for all kind of neat tricks as you can now generate 
resources based on something that''s returned from a function. 
Additionally this can be done *before* we start to interpret any types 
and resources defined as it is essentially doable by some kind of 
preprocessor.

The Problem I see with your variant is that the resources are created by 
your function. This unneccisarily constrains flexibility. My variant 
allows you to do something like the following:

foreach(dir in only_dirs(recursive_file_list(...))) {
	...
}
Using recursive_file_list() yields an array of hashes which only_dirs() 
can then filter to only contain directories.
Or maybe a function force_mode() that takes the minimum permission bits 
which is then simply ANDed with the bits read from the filesystem.

last but not least the "not harder to understand than what we do right 
now"-example:

$tree = recursive_file_list(...)
foreach(dir in only_dirs($tree)) {
	directory { $dir }
}
foreach(file in only_files($tree)) {
	file { $file }
}

I know this still looks complicated and is the *one* thing that would be 
simpler with the old variant where the File-type does everything:

foreach(file in recursive_file_list(...)) {
	file { $file }
}

However, the upper one definitely wins once you start to customize 
parameters depending on wether it is a file or a directory (i.e. setting 
the mode to 0755 instead of 0644). The main advantage is that 
everything still works just like you had put the whole resource-list 
into your manifest.

I think with that approach you''ll be able to write much simpler 
functions that are focused on a specific job plus you don''t need to
know
how to generate resources from inside a function.
Pushing information down to a function that generates the resources, as 
you suggest, will make people write a special function for every 
use-case instead of expressing what they want in the DSL using existing 
functions.
And by the way: it is much simpler to debug a side-effect-free function 
than debugging one with side-effects.
If you return an array of hashes you can always just print it out. If 
your function generates a set of resources you''ll need debugging-code 
inside the resource-generating function just to find out what resources 
are generated.

Regards,
Andreas

-- 
Solvention Ltd. & Co. KG
Egermannstr. 6-8
53359 Rheinbach

Tel: +49 2226 158179-0
Fax: +49 2226 158179-9

http://www.solvention.de
mailto:info@solvention.de

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Apr 4, 4:28 pm, Andreas Rogge <a.ro...@solvention.de>
wrote:
> I was thinking about something like "foreach" in the DSL, so you
could
> simply write something like the following:
Resource declarations already provide iteration comparable to
''foreach'' when an array is specified as the resource title. 
In
combination with Puppet defined types, that could support for the
lower-level approach of manual iteration over the results of query
functions without DSL changes (though the specific details might need
to be different from your example).  That''s a potential advantage of
that route.

On the other hand, I am well aware that programmers (among whom I
account myself) have a tendency to write software that works well for
*them*, but not necessarily as well for lesser mortals.  I don''t know
where the best balance of power vs. ease of use would be in this area,
but power and flexibility might not be compelling advantages.


Best regards,

John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.