Puppet users - Jun 2007 - Example module for central git repository via http (sort of like a subversion repo) [a bit RFC]

Hi all,

I''ve been looking for a good example for how one would build a module,
but I couldn''t really find something that I liked, so I went ahead and
tried to do it myself.

The result is this: a git module (see attachment).

Please have a look and comment on it, before I add it to the wiki. I
especially want comments on the setup of the module itself, not so
much on the git specific stuff. I think it would be a good idea to
settle on a default template for these things, and make that readily
available on the site.

How to use:
- First of all read the included README for some information about dependencies
- make sure you import the module (in future this will be automatic I''m
told):
   import "git"
- in the node you want to host the repository, do:

    include git::server

    git::repos { "main":
    }

(and see the header for the repos definition in definitions/repos.pp)
- in a node where you want to easily use the above repo, do:

    include git::client

    git::clientsetup { "main":
        gitserver       => "<servername>",
        gitusername     => "<username>",
        passwd          => "<password>",
    }

(and again see the header in definitions/clientsetup.pp)

As you''ll see, it lacks some documentation at the moment, and it is
very definitely debian/ubuntu-centric (as that is what I have).

Thanks for your comments, gr,

Thijs


_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

And already a small update. Found out my handling of the apache2
server types was less than ideal. This seems to work for now...

gr,
Thijs

On 09/06/07, Thijs Oppermann <thijso+puppet@gmail.com>
wrote:
> Hi all,
>
> I''ve been looking for a good example for how one would build a
module,
> but I couldn''t really find something that I liked, so I went ahead
and
> tried to do it myself.
>
> The result is this: a git module (see attachment).
>
> Please have a look and comment on it, before I add it to the wiki. I
> especially want comments on the setup of the module itself, not so
> much on the git specific stuff. I think it would be a good idea to
> settle on a default template for these things, and make that readily
> available on the site.
>
> How to use:
> - First of all read the included README for some information about
dependencies
> - make sure you import the module (in future this will be automatic
I''m told):
>    import "git"
> - in the node you want to host the repository, do:
>
>     include git::server
>
>     git::repos { "main":
>     }
>
> (and see the header for the repos definition in definitions/repos.pp)
> - in a node where you want to easily use the above repo, do:
>
>     include git::client
>
>     git::clientsetup { "main":
>         gitserver       => "<servername>",
>         gitusername     => "<username>",
>         passwd          => "<password>",
>     }
>
> (and again see the header in definitions/clientsetup.pp)
>
> As you''ll see, it lacks some documentation at the moment, and it
is
> very definitely debian/ubuntu-centric (as that is what I have).
>
> Thanks for your comments, gr,
>
> Thijs
>
>

_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

* Thijs Oppermann:
> Hi all,
> 
> I''ve been looking for a good example for how one would build a
module,
> but I couldn''t really find something that I liked, so I went ahead
and
> tried to do it myself.
> 
> The result is this: a git module (see attachment).
I didn''t have time to try, but does this work when you have multiple
repositories for a user?
In my experience repeatedly mentioning a file via different "define"
calls always creates a conflict (which is a pain).

    file { "netrc.d-$gitfile":
	path	=> "/home/${user}/.netrc.d",
	ensure	=> directory,
	mode	=> "0700",
	owner	=> "${user}",
    }

Yeah, ran into that myself actually just after posting this. Couldn''t
be bothered then to try to track it down and fix it (except for
throwing out a short rant on irc).

Not sure, but I think puppet should actually just ''merge''
calls to
resources that are *exactly* the same such as in cases as this.
Because this is something that you''ll want to do on occasion, and
currently I don''t see an easy way to get this kind of thing in puppet
(probably so simple I overlooked it, but... ;).

gr,
Thijs

On 11/06/07, tobutaz at gmail <tobutaz@gmail.com>
wrote:
> * Thijs Oppermann:
> > Hi all,
> >
> > I''ve been looking for a good example for how one would build
a module,
> > but I couldn''t really find something that I liked, so I went
ahead and
> > tried to do it myself.
> >
> > The result is this: a git module (see attachment).
>
> I didn''t have time to try, but does this work when you have
multiple
> repositories for a user?
> In my experience repeatedly mentioning a file via different
"define"
> calls always creates a conflict (which is a pain).
>
>     file { "netrc.d-$gitfile":
>         path    => "/home/${user}/.netrc.d",
>         ensure  => directory,
>         mode    => "0700",
>         owner   => "${user}",
>     }
>
>

On Jun 10, 2007, at 11:01 PM, tobutaz at gmail wrote:
>
> I didn''t have time to try, but does this work when you have
multiple
> repositories for a user?
> In my experience repeatedly mentioning a file via different
"define"
> calls always creates a conflict (which is a pain).
>
>     file { "netrc.d-$gitfile":
> 	path	=> "/home/${user}/.netrc.d",
> 	ensure	=> directory,
> 	mode	=> "0700",
> 	owner	=> "${user}",
>     }
The solution here is to abstract out the common bits into a class and  
include that class in the definition.

Classes are singletons and can be included repeatedly; definitions  
are meant only for those bits that change on every call.

  --
  Fallacies do not cease to be fallacies because they become fashions.
          --G. K. Chesterton
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

* Luke Kanies:
> On Jun 10, 2007, at 11:01 PM, tobutaz at gmail wrote:
>> I didn''t have time to try, but does this work when you have
multiple
>> repositories for a user?
>> In my experience repeatedly mentioning a file via different
"define"
>> calls always creates a conflict (which is a pain).
>>
>>     file { "netrc.d-$gitfile":
>> 	path	=> "/home/${user}/.netrc.d",
>> 	ensure	=> directory,
>> 	mode	=> "0700",
>> 	owner	=> "${user}",
>>     }
> 
> The solution here is to abstract out the common bits into a class and  
> include that class in the definition.
> 
> Classes are singletons and can be included repeatedly; definitions  
> are meant only for those bits that change on every call.
A class would work for something global like installing a package, but
they have their limits.

Say I want a class home-netrc-dir and have it depend on the variable $user.

If I call it like this:
  $user = "foo"
  include home-netrc-dir
in one scope and
  $user = "bar"
  include home-netrc-dir
in another; since it''s singleton only one user will really have the
directory created.

Short of writing a new resource type in ruby:
  home-netrc-dir { "foo": }
  home-netrc-dir { "bar": }

I don''t see how to express it.

On the other hand, I really like the idea of having identical calls to
the same resource (eg file { ... } ) being considered the same resource
with no conflict.

tobutaz at gmail <tobutaz@gmail.com> writes:
> Say I want a class home-netrc-dir and have it depend on the variable $user.
> If I call it like this:
>   $user = "foo"
>   include home-netrc-dir
> in one scope and
>   $user = "bar"
>   include home-netrc-dir
> in another; since it''s singleton only one user will really have
the
> directory created.
> Short of writing a new resource type in ruby:
>   home-netrc-dir { "foo": }
>   home-netrc-dir { "bar": }
> I don''t see how to express it.
Write a definition.  It''s the gateway drug to new resource types.  :)
Using variables to change the meaning of included classes is pretty ugly;
definitions are a cleaner way of expressing the same concept.

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

On 11/06/07, Russ Allbery <rra@stanford.edu>
wrote:
>
> tobutaz at gmail <tobutaz@gmail.com> writes:
>
> > Say I want a class home-netrc-dir and have it depend on the variable
> $user.
>
> > If I call it like this:
> >   $user = "foo"
> >   include home-netrc-dir
> > in one scope and
> >   $user = "bar"
> >   include home-netrc-dir
> > in another; since it''s singleton only one user will really
have the
> > directory created.
>
> > Short of writing a new resource type in ruby:
> >   home-netrc-dir { "foo": }
> >   home-netrc-dir { "bar": }
>
> > I don''t see how to express it.
>
> Write a definition.  It''s the gateway drug to new resource types. 
:)
> Using variables to change the meaning of included classes is pretty ugly;
> definitions are a cleaner way of expressing the same concept.


But they don''t always work. For example, in the example (a bit like)
above
(which was what I had in my git module definition for clientsetup):

     file { "/home/${user}/.netrc.d":
      ensure  => directory,
      mode    => "0700",
      owner   => "${user}",
     }

This was included in a defintion that gets called with a $user var. However,
it possibly gets called multiple times (as I want more than one clientsetup
for my user). This fails miserably, because this directory will already be
defined for puppet when the definition gets called a second time. However:
it is *exactly* the same definition. The only thing I want to do here is
make sure the directory exists before putting stuff in it.

Extracting it out into a class doesn''t work, because it depends on the
$user
var in the definition.

I hacked the puppet code to allow this by allowing completely identical file
resources to be merged into one. Not sure if that''s the way to go, but
it''s
working for me for now...


Thijs




--
> Russ Allbery (rra@stanford.edu)            
<http://www.eyrie.org/~eagle/>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
>

_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

Thijs Oppermann <thijso+puppet@gmail.com> writes:
> But they don''t always work. For example, in the example (a bit
like)
> above (which was what I had in my git module definition for
> clientsetup):
>     file { "/home/${user}/.netrc.d":
>      ensure  => directory,
>      mode    => "0700",
>      owner   => "${user}",
>     }
> This was included in a defintion that gets called with a $user
> var. However, it possibly gets called multiple times (as I want more
> than one clientsetup for my user). This fails miserably, because this
> directory will already be defined for puppet when the definition gets
> called a second time. However: it is *exactly* the same definition. The
> only thing I want to do here is make sure the directory exists before
> putting stuff in it.
Well, from the Puppet perspective, you have a resource structuring problem
here.  You have a definition that defines multiple types, some of which
you may want to define more than once, and some of which you want to
define only once.

If you separated those things out so that you had one definition that did
general user setup and another definition that defined the types that you
may want to do more than once for a given user, Puppet would be happy.

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

On 11/06/07, Russ Allbery <rra@stanford.edu>
wrote:
>
> Thijs Oppermann <thijso+puppet@gmail.com> writes:
>
> > But they don''t always work. For example, in the example (a
bit like)
> > above (which was what I had in my git module definition for
> > clientsetup):
>
> >     file { "/home/${user}/.netrc.d":
> >      ensure  => directory,
> >      mode    => "0700",
> >      owner   => "${user}",
> >     }
>
> > This was included in a defintion that gets called with a $user
> > var. However, it possibly gets called multiple times (as I want more
> > than one clientsetup for my user). This fails miserably, because this
> > directory will already be defined for puppet when the definition gets
> > called a second time. However: it is *exactly* the same definition.
The
> > only thing I want to do here is make sure the directory exists before
> > putting stuff in it.
>
> Well, from the Puppet perspective, you have a resource structuring problem
> here.  You have a definition that defines multiple types, some of which
> you may want to define more than once, and some of which you want to
> define only once.
>
> If you separated those things out so that you had one definition that did
> general user setup and another definition that defined the types that you
> may want to do more than once for a given user, Puppet would be happy.


Yeah, but  let''s step back a bit and look at *why* that resource
''file {
"/home/${user}/.netrc.d": '' is there in the first place: we
want to make
sure that a necessary part of a definition is available for the rest of that
definition. So extracting that out into a separate definition just puts the
burden of making sure of that on the user (developer, admin, whatever). I
can just as easily call the define as follows then, and be done with it:

     file { "/home/xxx/.netrc.d":
      ensure  => directory,
      mode    => "0700",
      owner   => "xxx",
     }
     git:repos { "bla":
        user  => "xxx",
    }

But where is the sense in that? Puppet should make stuff easier, not
introduce extra headaches for us.


--
> Russ Allbery (rra@stanford.edu)            
<http://www.eyrie.org/~eagle/>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
>

_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

Thijs Oppermann <thijso+puppet@gmail.com> writes:
> Yeah, but let''s step back a bit and look at *why* that resource
''file {
> "/home/${user}/.netrc.d": '' is there in the first place:
we want to make
> sure that a necessary part of a definition is available for the rest of
> that definition. So extracting that out into a separate definition just
> puts the burden of making sure of that on the user (developer, admin,
> whatever). I can just as easily call the define as follows then, and be
> done with it:
>     file { "/home/xxx/.netrc.d":
>      ensure  => directory,
>      mode    => "0700",
>      owner   => "xxx",
>     }
>     git:repos { "bla":
>        user  => "xxx",
>    }
> But where is the sense in that? Puppet should make stuff easier, not
> introduce extra headaches for us.
The part that makes this hard is that the file you want to create depends
on a directory.  If it weren''t for that, it would be a fairly
straightforward problem to fix with normal class inclusion.  Hm.  And
that''s because class inclusion already has the behavior that you want;
I
can include the same class from multiple other classes and Puppet realizes
that it should only apply that class once.

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

Russ Allbery <rra@stanford.edu> writes:
> The part that makes this hard is that the file you want to create
> depends on a directory.
Er, depends on a *variable*.  Sorry for the confusion.
> If it weren''t for that, it would be a fairly straightforward
problem to
> fix with normal class inclusion.  Hm.  And that''s because class
> inclusion already has the behavior that you want; I can include the same
> class from multiple other classes and Puppet realizes that it should
> only apply that class once.
-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

On Jun 11, 2007, at 12:01 PM, Thijs Oppermann wrote:
> But they don''t always work. For example, in the example (a bit  
> like) above (which was what I had in my git module definition for  
> clientsetup):
>
>      file { "/home/${user}/.netrc.d":
>       ensure  => directory,
>       mode    => "0700",
>       owner   => "${user}",
>      }
>
> This was included in a defintion that gets called with a $user var.  
> However, it possibly gets called multiple times (as I want more  
> than one clientsetup for my user). This fails miserably, because  
> this directory will already be defined for puppet when the  
> definition gets called a second time. However: it is *exactly* the  
> same definition. The only thing I want to do here is make sure the  
> directory exists before putting stuff in it.
>
> Extracting it out into a class doesn''t work, because it depends on
> the $user var in the definition.
>
> I hacked the puppet code to allow this by allowing completely  
> identical file resources to be merged into one. Not sure if that''s
> the way to go, but it''s working for me for now...
If you can provide a patch that behaves appropriately under all  
conditions (esp. when handling overrides), I''ll accept it.

  --
  A cult is a religion with no political power.   -- Tom Wolfe
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com