Puppet users - Sep 2006 - Using multiple values for source, for platform/arch/host configs

Hi, in an attempt to simplify my configuration, and from a suggestion  
on IRC, I was thinking of using multiple values for file { source =>  
[] }.  Was just currious if anyone had any comments on this.

Basically, I setup my file-server to use:

<snip>
[private]
     path /var/lib/puppet/files/private/%H
     allow *

[shared]
     path /var/lib/puppet/files/shared
     allow *

[groups]
     path /var/lib/puppet/files/groups
     allow *
</snip>

And then in site.pp:

<snip>
$puppetserver = "puppet.my.domain"

define testremotefile($owner = root, $group = root, $mode, $source,  
$backup = false, $recurse = false) {
     file {
         $name:
             mode => $mode,
             owner => $owner,
             group => $group,
             backup => $backup,
             recurse => $recurse,
             source => [
                 "puppet://$puppetserver/private/$source",

                 "puppet://$puppetserver/groups/$groupname/ 
$operatingsystem/$hardwaremodel/$source",
                 "puppet://$puppetserver/groups/$groupname/ 
$operatingsystem/any/$source",
                 "puppet://$puppetserver/groups/$groupname/any/any/ 
$source",

                 "puppet://$puppetserver/shared/$operatingsystem/ 
$hardwaremodel/$source",
                 "puppet://$puppetserver/shared/$operatingsystem/any/ 
$source",
                 "puppet://$puppetserver/shared/any/any/$source",
             ]
     }
}

class testfile {
	$groupname = "default"

	remotefile {
		"/tmp/test":
			mode => 444,
			source => "testing/testfile"
	}
}

node ''host1'' {
	$groupname = "mailserver"
	
	include testfile
}

node ''host2'' {
	$groupname = "vmware"
	
	include testfile
}

node ''host3'' {
	include testfile
}

node ''fedora'' {
	include testfile
}

node ''fedorax8664'' {
	include testfile
}
</snip>

And setup some files...

This should get used for host1:
     /var/lib/puppet/files/groups/mailserver/any/any/testing/testfile

This should get used for host2:
     /var/lib/puppet/files/shared/any/any/testing/testfile

This for host3:
     /var/lib/puppet/files/private/host3/testing/testfile

This for fedora:
     /var/lib/puppet/files/shared/Fedora/any/testing/testfile

This for fedorax8664:
     /var/lib/puppet/files/shared/Fedora/x86_64/testing/testfile

* * *
I did some testing, not using define() though, and for a much simpler  
case this worked fine, the $groupname was picked up correctly,  
private files were served first.  I wanted to take this a step  
further and provide platform support, so that I can use the same  
simple set of class defs to allow a wide variety of configurations to  
be used with out needing to make new classes or add custom code to  
nodes.

This seems kinda useful in general... almost to the point that it  
would be kinda cool if puppet supported this type of platform/arch/ 
version-based selection of files.

Not sure this works with templates or not though... and I imagine  
that this will also result in a bunch of notices in the  
puppetmaster''s log about missing files too :-(

Anyways, any comments about how well this might work?  As I  
mentioned, I did some initial tests... but have not yet gone  
completely down this path to implementing yet, but I need to find a  
better way to serve up files for specific hosts, platforms and custom  
groups soon, else my manifest config is going to get out of control.

Thanks,

--jason

Jason Dillon wrote:
> Hi, in an attempt to simplify my configuration, and from a suggestion  
> on IRC, I was thinking of using multiple values for file { source =>  
> [] }.  Was just currious if anyone had any comments on this.
I didn''t so much *suggest* it as point out its existence.

I personally don''t like this style of distribution, for at least two 
reasons:  First, I just don''t like distributing more than one version 
complete files, and I would normally rather write a type to manage that 
file than distribute more than one version of a file.  The sudoers file 
is fine because you''ll only ever have one version, but syslog.conf 
varies too much.

Second, and more important, though, is that you''re now storing what 
amounts to important configuration information in your filesystem and it 
has to mesh perfectly with your configurations.  Anyone wanting to 
understand how your configuration works will need to study both 
locations -- is a file mentioned, does it exist, and in what order is it 
mentioned?

The other available approaches -- writing a custom type for every file 
type, or having case statements or selectors to choose the appropriate 
file in your configuration -- are a little less automatic, but I think 
they''re much more maintainable.  I''m a bit jaded on complex 
configurations, though, and a lot depends on who else has to maintain 
your configs.

This feature exists to ease people''s transition from cfengine to
Puppet,
and like a few others, I''d probably get rid of it if I could.

That being said...  This system should basically work, albeit relatively 
slowly.  I will note that I haven''t tested this feature all that much, 
so if you drop a new file into place you may need to force a recompile 
on the client (e.g., touch your site.pp file, the server will reparse, 
and the client will get a newly compiled version).  I just checked the 
code, and it looks like once Puppet discovers a valid source for a file, 
it keeps that source until the next recompile.

[SNIP]

This all looks like it should work just fine.  It''d be great if you 
posted this on the Cookbook.
> * * *
> I did some testing, not using define() though, and for a much simpler  
> case this worked fine, the $groupname was picked up correctly,  
> private files were served first.  I wanted to take this a step  
> further and provide platform support, so that I can use the same  
> simple set of class defs to allow a wide variety of configurations to  
> be used with out needing to make new classes or add custom code to  
> nodes.
> 
> This seems kinda useful in general... almost to the point that it  
> would be kinda cool if puppet supported this type of platform/arch/ 
> version-based selection of files.
It does support it -- you''re doing it, aren''t you? :)

I''ve thought about adding automatic support, but there''s no
way for me
to implement that doesn''t encode what I think is right.  It''s
better to
let you choose how you want it to work, even though it''s a little more 
setup.

It actually might not be a bad idea to set up a server-side function 
that did something akin to this -- you give it a search path, and it 
returns the first file that exists, either just the path or the whole 
file.  Hmmm.  That''s a *much* better solution than the array-of-sources
solution.
> Not sure this works with templates or not though... and I imagine  
> that this will also result in a bunch of notices in the  
> puppetmaster''s log about missing files too :-(
Yes, you''ll get a lot of notices about missing files, and if
I''m wrong
about Puppet caching the source, it might be considerably slower (e.g., 
six calls for every file copy instead of one).

I''m not recommending this, either, but you could also just fake the 
whole thing in templates -- your template could have all of the logic 
that looked for the correct file and then returned the correct file as 
the value of the template.

-- 
Don''t tell me how hard you work. Tell me how much you get done.
                 -- James Ling
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com

Okay, fair enough :-)

Perhaps you could explain a bit more how you would use custom types?   
Or how to best manage those case statements?

My environment is not that complicated yet, though as I start to use  
puppet for more things I find that it is a burden to keep adding new  
classes to support different configurations.

Right now, I am using puppet to control these:

  * sudoers
  * motd
  * resolve.conf
  * sshd config
  * yum
  * postfix & aliases
  * iptables

The last three are where things start to become more complicated.   
Yum config needs to be done differently on Fedora and RHEL hosts, and  
I guess its easy enough to include $operatingsystem in the content =>  
path to fix that one (though that leans more towards the file layout  
I was talking about before).

Postfix is more of a pain, since I have a master relay config and  
then a set of client configs.  This seems easiest solved by the / 
private/%H bits... I am not sure how else to do this... I guess a  
conditional to select the right source based on the hostname?

And iptables config provides the most complicated, as I have three  
classes of machines that need a different firewall configuration.

Granted, in the big picture this is not all that complicated compared  
to larger deployments, but even so as I added more and more bits to  
be controlled by puppet I found myself making my configuration more  
complex to hack around not having a good way to pull host specific or  
platform specific configuration files... in a scalable manner, such  
that it would be easy to add new services controlled by puppet and  
add new hosts in different machine classes with out much work.

What I am really looking for is a complete example... best practices  
and all for a scalable setup for a med to large environment.  So far  
all I have found were bits and pieces, that and some chatting in IRC  
to build up the knowledge (still limited) of puppet that I have now.

I would really appreciate any examples, or further advise anyone has.

Thanks,

--jason


On Sep 21, 2006, at 10:47 AM, Luke Kanies wrote:
> Jason Dillon wrote:
>> Hi, in an attempt to simplify my configuration, and from a suggestion
>> on IRC, I was thinking of using multiple values for file { source =>
>> [] }.  Was just currious if anyone had any comments on this.
>
> I didn''t so much *suggest* it as point out its existence.
>
> I personally don''t like this style of distribution, for at least
two
> reasons:  First, I just don''t like distributing more than one
version
> complete files, and I would normally rather write a type to manage  
> that
> file than distribute more than one version of a file.  The sudoers  
> file
> is fine because you''ll only ever have one version, but syslog.conf
> varies too much.
>
> Second, and more important, though, is that you''re now storing
what
> amounts to important configuration information in your filesystem  
> and it
> has to mesh perfectly with your configurations.  Anyone wanting to
> understand how your configuration works will need to study both
> locations -- is a file mentioned, does it exist, and in what order  
> is it
> mentioned?
>
> The other available approaches -- writing a custom type for every file
> type, or having case statements or selectors to choose the appropriate
> file in your configuration -- are a little less automatic, but I think
> they''re much more maintainable.  I''m a bit jaded on
complex
> configurations, though, and a lot depends on who else has to maintain
> your configs.
>
> This feature exists to ease people''s transition from cfengine to  
> Puppet,
> and like a few others, I''d probably get rid of it if I could.
>
> That being said...  This system should basically work, albeit  
> relatively
> slowly.  I will note that I haven''t tested this feature all that
much,
> so if you drop a new file into place you may need to force a recompile
> on the client (e.g., touch your site.pp file, the server will reparse,
> and the client will get a newly compiled version).  I just checked the
> code, and it looks like once Puppet discovers a valid source for a  
> file,
> it keeps that source until the next recompile.
>
> [SNIP]
>
> This all looks like it should work just fine.  It''d be great if
you
> posted this on the Cookbook.
>
>> * * *
>> I did some testing, not using define() though, and for a much simpler
>> case this worked fine, the $groupname was picked up correctly,
>> private files were served first.  I wanted to take this a step
>> further and provide platform support, so that I can use the same
>> simple set of class defs to allow a wide variety of configurations to
>> be used with out needing to make new classes or add custom code to
>> nodes.
>>
>> This seems kinda useful in general... almost to the point that it
>> would be kinda cool if puppet supported this type of platform/arch/
>> version-based selection of files.
>
> It does support it -- you''re doing it, aren''t you? :)
>
> I''ve thought about adding automatic support, but there''s
no way for me
> to implement that doesn''t encode what I think is right. 
It''s
> better to
> let you choose how you want it to work, even though it''s a little
more
> setup.
>
> It actually might not be a bad idea to set up a server-side function
> that did something akin to this -- you give it a search path, and it
> returns the first file that exists, either just the path or the whole
> file.  Hmmm.  That''s a *much* better solution than the array-of- 
> sources
> solution.
>
>> Not sure this works with templates or not though... and I imagine
>> that this will also result in a bunch of notices in the
>> puppetmaster''s log about missing files too :-(
>
> Yes, you''ll get a lot of notices about missing files, and if
I''m wrong
> about Puppet caching the source, it might be considerably slower  
> (e.g.,
> six calls for every file copy instead of one).
>
> I''m not recommending this, either, but you could also just fake
the
> whole thing in templates -- your template could have all of the logic
> that looked for the correct file and then returned the correct file as
> the value of the template.
>
> -- 
> Don''t tell me how hard you work. Tell me how much you get done.
>                  -- James Ling
> ---------------------------------------------------------------------
> Luke Kanies | http://reductivelabs.com | http://madstop.com
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users

Jason Dillon wrote:
> Okay, fair enough :-)
> 
> Perhaps you could explain a bit more how you would use custom types?   
> Or how to best manage those case statements?
> 
> My environment is not that complicated yet, though as I start to use  
> puppet for more things I find that it is a burden to keep adding new  
> classes to support different configurations.
> 
> Right now, I am using puppet to control these:
> 
>   * sudoers
>   * motd
>   * resolve.conf
These are all pretty easy, since the sudoers file can always be the same 
everywhere and the motd and resolve.conf are short enough to generate or 
whatever.
>   * sshd config
>   * yum
>   * postfix & aliases
>   * iptables
These definitely get more complicated.  However, Puppet already includes 
a yumrepo type:

http://reductivelabs.com/projects/puppet/documentation/typedocs.html#yumrepo

You''re forgiven for not knowing that, though, since while writing this 
email I realized it wasn''t being included in the autogenerated docs (I 
swear I thought I fixed that already).

Postfix and aliases are both pretty easy to manage; I''ve written a 
simple puppet type for managing aliases (although I''ve not yet
published
it), and postfix''s config files are pretty sane.

Again, I''m not saying this is what I expect everyone to do, but this is
what I would do.
> What I am really looking for is a complete example... best practices  
> and all for a scalable setup for a med to large environment.  So far  
> all I have found were bits and pieces, that and some chatting in IRC  
> to build up the knowledge (still limited) of puppet that I have now.
I hope to have the time and opportunity to begin publishing some of my 
client configurations, but that''s generally going to be unlikely, and 
the client usually fleshes the configurations out all the way.

We''re working on ways of publishing code (PRM[1] and the cookbook[2])

1 - http://prmweb.hezmatt.org/
2 - http://reductivelabs.com/cookbook

-- 
The easiest way for your children to learn about money is for you
not to have any.    -- Katharine Whitehorn
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com

On Sep 21, 2006, at 12:31 PM, Luke Kanies wrote:
>>   * sshd config
>>   * yum
>>   * postfix & aliases
>>   * iptables
>
> These definitely get more complicated.  However, Puppet already  
> includes
> a yumrepo type:
>
> http://reductivelabs.com/projects/puppet/documentation/ 
> typedocs.html#yumrepo
>
> You''re forgiven for not knowing that, though, since while writing
this
> email I realized it wasn''t being included in the autogenerated
docs (I
> swear I thought I fixed that already).
Ic... so you would rather define a type that would make a config  
file, rather than use the fileserver to pull down those configs?  I  
am guessing that the yumrepo type actually makes a file in /etc/ 
yum.repos.d ?
> Postfix and aliases are both pretty easy to manage; I''ve written a
> simple puppet type for managing aliases (although I''ve not yet  
> published
> it), and postfix''s config files are pretty sane.
And again here use a custom puppet type to generate the main.cf and  
aliases?

Do you want to share what your custom alias type looks like?  Code I  
mean... looks like its finally time for me to learn Ruby.

> Again, I''m not saying this is what I expect everyone to do, but  
> this is
> what I would do.
Of course, but I am looking for as much input as possible :-)

>> What I am really looking for is a complete example... best practices
>> and all for a scalable setup for a med to large environment.  So far
>> all I have found were bits and pieces, that and some chatting in IRC
>> to build up the knowledge (still limited) of puppet that I have now.
>
> I hope to have the time and opportunity to begin publishing some of my
> client configurations, but that''s generally going to be unlikely,
and
> the client usually fleshes the configurations out all the way.
>
> We''re working on ways of publishing code (PRM[1] and the
cookbook[2])
>
> 1 - http://prmweb.hezmatt.org/
> 2 - http://reductivelabs.com/cookbook
While these do help some, I still feel they lack some of the bigger  
picture for how to best organize manifests and fileserver files for a  
med-large host base.

Thanks for all your help.

--jason

On Sep 21, 2006, at 10:47 AM, Luke Kanies wrote:
> It actually might not be a bad idea to set up a server-side function
> that did something akin to this -- you give it a search path, and it
> returns the first file that exists, either just the path or the whole
> file.  Hmmm.  That''s a *much* better solution than the array-of- 
> sources
> solution.
Okay, then use a http[s]:// url for the source?  Will this work with  
recurse?  I can easily whip up a php script that can perform the  
selection.

--jason

On Thu, Sep 21, 2006 at 11:21:02AM -0700, Jason Dillon wrote:
> And iptables config provides the most complicated, as I have three  
> classes of machines that need a different firewall configuration.
> 
> Granted, in the big picture this is not all that complicated compared  
> to larger deployments, but even so as I added more and more bits to  
> be controlled by puppet I found myself making my configuration more  
> complex to hack around not having a good way to pull host specific or  
> platform specific configuration files... in a scalable manner, such  
> that it would be easy to add new services controlled by puppet and  
> add new hosts in different machine classes with out much work.
I was thinking something like this for iptables

define iptables($opentcpports=[], $openudpports=[]) {
  file {
    path => "/etc/sysconfig/iptables",
    mode => 600,
    owner => root,
    group => root,
    ensure => file,
    content => template("iptables.erb")
  }
}

node foo {
  iptables {
    opentcpports => [ 22, 100 ],
    openudpports => [ 53, 100 ]
  }
}

and in the template something like
...
<% opentcpports.each do |v| -%>
-A PUPPET-INPUT -p tcp -m tcp --dport <%= v %> --syn -j ACCEPT
<% end -%>
...

But now I am playing with the idea that I can just check the classes/tags
for the node and open ports accordingly
...
<% if scope.classlist.include?("ssh_server") then -%>
-A PUPPET-INPUT -p tcp -m tcp --dport 22 --syn -j ACCEPT
<% end -%>
<% if scope.classlist.include?("web_server") then -%>
-A PUPPET-INPUT -p tcp -m tcp --dport 80 --syn -j ACCEPT
<% end -%>
...

Cheers,
Kostas

What I meant by "it would be kinda cool if puppet supported this type  
of platform/arch/..." was that if puppet:// urls did this for me...  
configurable path on the server-side to avoid needing a source array  
or external script.  That way the puppet ssl config can be reused to  
auth/secure the xfr of files.

--jason


On Sep 21, 2006, at 1:19 PM, Jason Dillon wrote:
> On Sep 21, 2006, at 10:47 AM, Luke Kanies wrote:
>> It actually might not be a bad idea to set up a server-side function
>> that did something akin to this -- you give it a search path, and it
>> returns the first file that exists, either just the path or the whole
>> file.  Hmmm.  That''s a *much* better solution than the
array-of-
>> sources
>> solution.
>
> Okay, then use a http[s]:// url for the source?  Will this work  
> with recurse?  I can easily whip up a php script that can perform  
> the selection.
>
> --jason
>

Jason Dillon wrote:
> 
> Ic... so you would rather define a type that would make a config  
> file, rather than use the fileserver to pull down those configs?  I  
> am guessing that the yumrepo type actually makes a file in /etc/ 
> yum.repos.d ?
I definitely prefer types over file distribution.  As to how yumrepo 
works, you''ll have to hit the docs, since I didn''t write it
and don''t
know how to use it -- David Lutterkort wrote it.
> And again here use a custom puppet type to generate the main.cf and  
> aliases?
Yep.
> Do you want to share what your custom alias type looks like?  Code I  
> mean... looks like its finally time for me to learn Ruby.
I''ve asked my client to post it.

And actually, the alias type is written in Puppet, using exec.
>> Again, I''m not saying this is what I expect everyone to do,
but
>> this is
>> what I would do.
> 
> Of course, but I am looking for as much input as possible :-)
I understand, which is why I''m providing that feedback, but I might be 
presenting an unrealistic starting point.
> While these do help some, I still feel they lack some of the bigger  
> picture for how to best organize manifests and fileserver files for a  
> med-large host base.
I agree with that.

We should probably make more of an effort to publish that information, 
but, on the other hand, that''s pretty much how I make my consulting 
money -- show up at a client site and work with them to get everything 
set up quickly and smoothly.  I could probably write most of it down, 
but I don''t think I can afford to go quite that open source at this
point.

-- 
Do not think of knocking out another person''s brains because he differs
in opinion from you. It would be as rational to knock yourself on the
head because you differ from yourself ten years ago.    -- Horace Mann
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com

Cool.  I guess you could probably do both...

--jason


On Sep 21, 2006, at 1:20 PM, Kostas Georgiou wrote:
> On Thu, Sep 21, 2006 at 11:21:02AM -0700, Jason Dillon wrote:
>
>> And iptables config provides the most complicated, as I have three
>> classes of machines that need a different firewall configuration.
>>
>> Granted, in the big picture this is not all that complicated compared
>> to larger deployments, but even so as I added more and more bits to
>> be controlled by puppet I found myself making my configuration more
>> complex to hack around not having a good way to pull host specific or
>> platform specific configuration files... in a scalable manner, such
>> that it would be easy to add new services controlled by puppet and
>> add new hosts in different machine classes with out much work.
>
> I was thinking something like this for iptables
>
> define iptables($opentcpports=[], $openudpports=[]) {
>   file {
>     path => "/etc/sysconfig/iptables",
>     mode => 600,
>     owner => root,
>     group => root,
>     ensure => file,
>     content => template("iptables.erb")
>   }
> }
>
> node foo {
>   iptables {
>     opentcpports => [ 22, 100 ],
>     openudpports => [ 53, 100 ]
>   }
> }
>
> and in the template something like
> ...
> <% opentcpports.each do |v| -%>
> -A PUPPET-INPUT -p tcp -m tcp --dport <%= v %> --syn -j ACCEPT
> <% end -%>
> ...
>
> But now I am playing with the idea that I can just check the  
> classes/tags
> for the node and open ports accordingly
> ...
> <% if scope.classlist.include?("ssh_server") then -%>
> -A PUPPET-INPUT -p tcp -m tcp --dport 22 --syn -j ACCEPT
> <% end -%>
> <% if scope.classlist.include?("web_server") then -%>
> -A PUPPET-INPUT -p tcp -m tcp --dport 80 --syn -j ACCEPT
> <% end -%>
> ...
>
> Cheers,
> Kostas
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users

Jason Dillon wrote:
> 
> Okay, then use a http[s]:// url for the source?  Will this work with  
> recurse?  I can easily whip up a php script that can perform the  
> selection.
Actually, I was just thinking of using local files on the server.  E.g., 
your code might look like this:

   file { "/etc/postfix/main.cf":
     content => filesearch("/dist/apps/postfix/main-$hostname.cf", 
"/dist/apps/postfix/main-$operatingsystem.cf",
"/dist/apps/postfix/main.cf")
   }

This would all be interpreted on the server.

It''s not significantly different than just having multiple file
sources,
but for some reason I like it better.

-- 
Risk! Risk anything! Care no more for the opinions of others, for those
voices.  Do the hardest thing on earth for you. Act for yourself. Face
the truth.  -- Katherine Mansfield
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com

Does filesearch() exist already?

--jason


On Sep 21, 2006, at 1:41 PM, Luke Kanies wrote:
> Jason Dillon wrote:
>>
>> Okay, then use a http[s]:// url for the source?  Will this work with
>> recurse?  I can easily whip up a php script that can perform the
>> selection.
>
> Actually, I was just thinking of using local files on the server.   
> E.g.,
> your code might look like this:
>
>    file { "/etc/postfix/main.cf":
>      content =>
filesearch("/dist/apps/postfix/main-$hostname.cf",
> "/dist/apps/postfix/main-$operatingsystem.cf",
"/dist/apps/postfix/
> main.cf")
>    }
>
> This would all be interpreted on the server.
>
> It''s not significantly different than just having multiple file  
> sources,
> but for some reason I like it better.
>
> -- 
> Risk! Risk anything! Care no more for the opinions of others, for  
> those
> voices.  Do the hardest thing on earth for you. Act for yourself. Face
> the truth.  -- Katherine Mansfield
> ---------------------------------------------------------------------
> Luke Kanies | http://reductivelabs.com | http://madstop.com
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users

On Sep 21, 2006, at 1:39 PM, Luke Kanies wrote:
> I understand, which is why I''m providing that feedback, but I
might be
> presenting an unrealistic starting point.
No worries, anything and everything is good input :-)

I think I may start with my crude fileserver bits, then as I learn  
more, and as the recipes grow I will refactor...  I''m not normally a  
sysadmin, and I need to get back to coding ;-)

>> While these do help some, I still feel they lack some of the bigger
>> picture for how to best organize manifests and fileserver files for a
>> med-large host base.
>
> I agree with that.
>
> We should probably make more of an effort to publish that information,
> but, on the other hand, that''s pretty much how I make my
consulting
> money -- show up at a client site and work with them to get everything
> set up quickly and smoothly.  I could probably write most of it down,
> but I don''t think I can afford to go quite that open source at
this
> point.
Well, if I was working for a company I would recommend them to get  
you on to make it all super-sexy... but I''m doing this for Apache  
Geronimo... and we don''t have a budget to bring on consultants ;-)

--jason

Jason Dillon wrote:
> Does filesearch() exist already?
No, I was proposing it as an alternative to multiple file sources.

-- 
The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.  -- George Bernard Shaw
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com

Jason Dillon wrote:
> 
> No worries, anything and everything is good input :-)
> 
> I think I may start with my crude fileserver bits, then as I learn  
> more, and as the recipes grow I will refactor...  I''m not normally
a
> sysadmin, and I need to get back to coding ;-)
I''m moving the opposite direction, sysadmin to developer, but I know
the
feeling.
> Well, if I was working for a company I would recommend them to get  
> you on to make it all super-sexy... but I''m doing this for Apache
> Geronimo... and we don''t have a budget to bring on consultants ;-)
I understand, and I don''t expect to make money on very many Puppet 
users.  I''m just not going to work very hard at writing all of my 
experience in a freely downloadable document, other than Puppet itself. :)

-- 
Should I say "I believe in physics", or "I know that physics is
true"?
                 -- Ludwig Wittgenstein, On Certainty, 602.
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com

Jason Dillon wrote:
> What I meant by "it would be kinda cool if puppet supported this type
> of platform/arch/..." was that if puppet:// urls did this for me...  
> configurable path on the server-side to avoid needing a source array  
> or external script.  That way the puppet ssl config can be reused to  
> auth/secure the xfr of files.
Ah, I see what you mean; I hadn''t thought of that.

-- 
Criminal: A person with predatory instincts who has not sufficient
capital to form a corporation.       --Howard Scott
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com

Sell the docs... it worked well for JBoss.

--jason


On Sep 21, 2006, at 2:11 PM, Luke Kanies wrote:
> Jason Dillon wrote:
>>
>> No worries, anything and everything is good input :-)
>>
>> I think I may start with my crude fileserver bits, then as I learn
>> more, and as the recipes grow I will refactor...  I''m not
normally a
>> sysadmin, and I need to get back to coding ;-)
>
> I''m moving the opposite direction, sysadmin to developer, but I  
> know the
> feeling.
>
>> Well, if I was working for a company I would recommend them to get
>> you on to make it all super-sexy... but I''m doing this for
Apache
>> Geronimo... and we don''t have a budget to bring on consultants
;-)
>
> I understand, and I don''t expect to make money on very many Puppet
> users.  I''m just not going to work very hard at writing all of my
> experience in a freely downloadable document, other than Puppet  
> itself. :)
>
> -- 
> Should I say "I believe in physics", or "I know that physics
is true"?
>                  -- Ludwig Wittgenstein, On Certainty, 602.
> ---------------------------------------------------------------------
> Luke Kanies | http://reductivelabs.com | http://madstop.com
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users

Jason Dillon wrote:
> Sell the docs... it worked well for JBoss.
Heh, I would, if I had time to write the docs. :)

-- 
                         ACHTUNG!!!

Das machine is nicht fur gefingerpoken und mittengrabben.  Ist easy
schnappen der springenwerk, blowenfusen und corkenpoppen mit
spitzensparken.  Ist nicht fur gewerken by das dummkopfen.  Das
rubbernecken sightseeren keepen hands in das pockets.  Relaxen und
vatch das blinkenlights!!!
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com

On Sep 21, 2006, at 10:47 AM, Luke Kanies wrote:
> This all looks like it should work just fine.  It''d be great if
you
> posted this on the Cookbook.
So this "almost" works... there is a problem with the $groupname var  
value getting included when set in a class. Consider the following:

<snip>
define remotefile($owner = root, $group = root, $mode, $source,  
$backup = false, $recurse = false) {
     file {
         $name:
             mode => $mode,
             owner => $owner,
             group => $group,
             backup => $backup,
             recurse => $recurse,
             source => [
                 "puppet://$puppetserver/hosts/$source",
                 "puppet://$puppetserver/groups/$groupname/ 
$operatingsystem/$hardwaremodel/$source",
                 "puppet://$puppetserver/groups/$groupname/ 
$operatingsystem/any/$source",
                 "puppet://$puppetserver/groups/$groupname/any/any/ 
$source"
             ]
     }
}

class any-host {
     $groupname = "default"

     remotefile {
         "/tmp/test":
             mode => 444,
             source => "testing/testfile"
     }
}

class vmware-server inherits any-host {
     $groupname = "vmware-server"
}

node ''vmware1'' {
     include vmware-server
}

node ''vmware2'' {
     $groupname = "vmware-server"

     include vmware-server
}
</snip>

First, since the any-host class defines ''$groupname =
"default"'' no
sub-class or node can override.  If I remove it from class any-host,  
then the vmware1 node''s $groupname evals to "" and the paths
requested are ".../groups/any/any/testing/testfile", but
vmware2''s
definition sticks, and the files eval to "groups/vmware-server/any/ 
any/testing/testfile".

Maybe there is a better way to get a groupname?  Say from the top- 
level class that I am extending from?  I guess it will work if I  
define $groupname for each node... or maybe make a vmware-server- 
node, with that defined, then extend it... not sure.

And also there is setting the default... I had dropped the "shared"  
paths in favor of a "default" group, so that I only need 4 lines, and
if $groupname is not set then it should use "default", but I have yet
to figure out how to make that work.  I was thinking a contitional to  
set $groupname, but I am still confused by that syntax.

--jason

On Thu, 2006-09-21 at 11:21 -0700, Jason Dillon wrote:
> Yum config needs to be done differently on Fedora and RHEL hosts, and  
> I guess its easy enough to include $operatingsystem in the content =>  
> path to fix that one (though that leans more towards the file layout  
> I was talking about before).
As a complete aside, there''s a yumrepo type that manages the files
in /etc/yum.repos.d/ - for RHEL, I assume you are talking about
manipulating /etc/sysconfig/rhn/sources, which would be a bit of a pain
to add, but not insurmountable (except that a lot of the options for
yumrepo just become useless at that point)

David

Unfortunately the yumrepo type does not handle multiple repo  
definitions in one file... which is how FC5 comes by default.  Most  
of the default repos, are defined in a fedora-<something>.repo, which  
contains <something>, <something>-debuginfo and
<something>-sources
repo definitions.

I had hopped to use the yumrepo type to disable most of these, then  
replace the URL for the repos I want enabled to my local YAM repo...  
but I was not able to get this working with yumrepo, so I am just  
serving up a list of repos as files.

--jason


On Sep 25, 2006, at 9:00 AM, David Lutterkort wrote:
> On Thu, 2006-09-21 at 11:21 -0700, Jason Dillon wrote:
>> Yum config needs to be done differently on Fedora and RHEL hosts, and
>> I guess its easy enough to include $operatingsystem in the content
=>
>> path to fix that one (though that leans more towards the file layout
>> I was talking about before).
>
> As a complete aside, there''s a yumrepo type that manages the files
> in /etc/yum.repos.d/ - for RHEL, I assume you are talking about
> manipulating /etc/sysconfig/rhn/sources, which would be a bit of a  
> pain
> to add, but not insurmountable (except that a lot of the options for
> yumrepo just become useless at that point)
>
> David
>
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users

On Mon, 2006-09-25 at 14:59 -0700, Jason Dillon wrote:
> Unfortunately the yumrepo type does not handle multiple repo  
> definitions in one file... which is how FC5 comes by default.  Most  
> of the default repos, are defined in a fedora-<something>.repo, which
> contains <something>, <something>-debuginfo and
<something>-sources
> repo definitions.
This should not be the case anymore (since 2006-06-15); if the yumrepos
your manifest talks about already exist, the yumrepo type will edit
those repo definitions in place; if its a new repo, it will be created
in its own .repo file.

Check
http://hg.et.redhat.com/hg/emd/recipes/fedora-yum?f=8ece33067275;file=manifests/fedora-yum.pp
for an example, I use that to manage the yum config on my FC machines, and it
seems to work - if you run into trouble, you''ve probably found a bug
somewhere ;)

David

It does not work, I''ve tried both:

<snip>
     yumrepo {
         fedora-core:
             name => "core",
             enabled => 0
     }
     yumrepo {
         fedora-core:
             name => "core-debuginfo",
             enabled => 0
     }
     yumrepo {
         fedora-core:
             name => "core-sources",
             enabled => 0
     }
</snip>

and:

<snip>
     yumrepo {
         fedora-core:
             name => "core",
             enabled => 0;

         fedora-core:
             name => "core-debuginfo",
             enabled => 0;

         fedora-core:
             name => "core-sources",
             enabled => 0
     }
</snip>

The file /etc/yum.repos.d/fedora-core.repo contains:

<snip>
[core]
name=Fedora Core $releasever - $basearch
#baseurl=http://download.fedora.redhat.com/pub/fedora/linux/core/ 
$releasever/$basearch/os/
mirrorlist=http://fedora.redhat.com/download/mirrors/fedora-core- 
$releasever
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora file:///etc/pki/rpm- 
gpg/RPM-GPG-KEY

[core-debuginfo]
name=Fedora Core $releasever - $basearch - Debug
baseurl=http://download.fedora.redhat.com/pub/fedora/linux/core/ 
$releasever/$basearch/debug/
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora file:///etc/pki/rpm- 
gpg/RPM-GPG-KEY

[core-source]
name=Fedora Core $releasever - Source
baseurl=http://download.fedora.redhat.com/pub/fedora/linux/core/ 
$releasever/source/SRPMS/
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora file:///etc/pki/rpm- 
gpg/RPM-GPG-KEY
</snip>

Puppet complains about redefining something... sorry I don''t have the  
exact error, but it does not like to have the same value in yumrepo  
{ same-value: ...; same-value: ... }.

--jason


On Sep 26, 2006, at 3:46 AM, David Lutterkort wrote:
> On Mon, 2006-09-25 at 14:59 -0700, Jason Dillon wrote:
>> Unfortunately the yumrepo type does not handle multiple repo
>> definitions in one file... which is how FC5 comes by default.  Most
>> of the default repos, are defined in a fedora-<something>.repo,
which
>> contains <something>, <something>-debuginfo and
<something>-sources
>> repo definitions.
>
> This should not be the case anymore (since 2006-06-15); if the  
> yumrepos
> your manifest talks about already exist, the yumrepo type will edit
> those repo definitions in place; if its a new repo, it will be created
> in its own .repo file.
>
> Check
> http://hg.et.redhat.com/hg/emd/recipes/fedora-yum? 
> f=8ece33067275;file=manifests/fedora-yum.pp for an example, I use  
> that to manage the yum config on my FC machines, and it seems to  
> work - if you run into trouble, you''ve probably found a bug  
> somewhere ;)
>
> David
>
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users

--On Tuesday, September 26, 2006 1:23 PM -0700 Jason Dillon 
<jason@planet57.com> wrote:
> It does not work, I''ve tried both:
>
> <snip>
>      yumrepo {
>          fedora-core:
>              name => "core",
>              enabled => 0
>      }
>      yumrepo {
>          fedora-core:
>              name => "core-debuginfo",
>              enabled => 0
>      }
>      yumrepo {
>          fedora-core:
>              name => "core-sources",
>              enabled => 0
>      }
> </snip>
> Puppet complains about redefining something... sorry I don''t have
the
> exact error, but it does not like to have the same value in yumrepo
> { same-value: ...; same-value: ... }.

I don''t use yumrepo, but if I''m understanding this syntax
correctly, its
interpreting "fedora-core" as the referential name for that definition
and
its seeing "fedora-core" being defined 3 times, which is a violation. 
Try
this:

      yumrepo {
          fedora-core:
              name => "core",
              enabled => 0
      }
      yumrepo {
          fedora-core-debuginfo:
              name => "core-debuginfo",
              enabled => 0
      }
      yumrepo {
          fedora-core-sources:
              name => "core-sources",
              enabled => 0
      }

It uses the <name>: for the name of the file in /etc/yum.repos.d, I  
did not see any way to specify the filename anywhere else.

--jason


On Sep 26, 2006, at 11:04 PM, Digant C Kasundra wrote:
>
>
> --On Tuesday, September 26, 2006 1:23 PM -0700 Jason Dillon
> <jason@planet57.com> wrote:
>
>> It does not work, I''ve tried both:
>>
>> <snip>
>>      yumrepo {
>>          fedora-core:
>>              name => "core",
>>              enabled => 0
>>      }
>>      yumrepo {
>>          fedora-core:
>>              name => "core-debuginfo",
>>              enabled => 0
>>      }
>>      yumrepo {
>>          fedora-core:
>>              name => "core-sources",
>>              enabled => 0
>>      }
>> </snip>
>> Puppet complains about redefining something... sorry I don''t
have the
>> exact error, but it does not like to have the same value in yumrepo
>> { same-value: ...; same-value: ... }.
>
>
> I don''t use yumrepo, but if I''m understanding this syntax
> correctly, its
> interpreting "fedora-core" as the referential name for that  
> definition and
> its seeing "fedora-core" being defined 3 times, which is a  
> violation.  Try
> this:
>
>       yumrepo {
>           fedora-core:
>               name => "core",
>               enabled => 0
>       }
>       yumrepo {
>           fedora-core-debuginfo:
>               name => "core-debuginfo",
>               enabled => 0
>       }
>       yumrepo {
>           fedora-core-sources:
>               name => "core-sources",
>               enabled => 0
>       }
>
>
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users

On Tue, 2006-09-26 at 23:21 -0700, Jason Dillon wrote:
> It uses the <name>: for the name of the file in /etc/yum.repos.d, I  
> did not see any way to specify the filename anywhere else.
It uses the name for the name of the section in the /etc/yum/repos.d/*
files - if such a section already exists in any file, regardless of
name, it changes that section according to your manifest; if not, it
creates a file with name "$name.repo" with a single section
"[$name]"

Have a look at
http://hg.et.redhat.com/hg/emd/recipes/fedora-yum?f=8ece33067275;file=manifests/fedora-yum.pp
for an example.

David

This means you remove all of the .repo files that comes with fedora  
and replace them, which is one way to fix it.  I was trying to leave  
the files as they were and only enable/disable as needed.

--jason


On Oct 3, 2006, at 3:54 AM, David Lutterkort wrote:
> On Tue, 2006-09-26 at 23:21 -0700, Jason Dillon wrote:
>> It uses the <name>: for the name of the file in /etc/yum.repos.d,
I
>> did not see any way to specify the filename anywhere else.
>
> It uses the name for the name of the section in the /etc/yum/repos.d/*
> files - if such a section already exists in any file, regardless of
> name, it changes that section according to your manifest; if not, it
> creates a file with name "$name.repo" with a single section
"[$name]"
>
> Have a look at
> http://hg.et.redhat.com/hg/emd/recipes/fedora-yum? 
> f=8ece33067275;file=manifests/fedora-yum.pp for an example.
>
> David
>
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users

On Tue, 2006-10-03 at 08:53 -0700, Jason Dillon wrote:
> This means you remove all of the .repo files that comes with fedora  
> and replace them, which is one way to fix it.  I was trying to leave  
> the files as they were and only enable/disable as needed.
That''s what the yumrepo type does: if you specify a repo that is
already
defined in one of the yum config files (say ''core''), it will
just change
the already existing definition. So no, it does not replace the .repo
files that come with fedora, it edits them. Only when your yumrepo type
mentions a repo that is not defined yet is a new .repo file created.

David

Um, you are not getting my point.

yumrepo can only work on a single repo defined in one file.

Fedora Core 5 by default comes with many repos defined in one file,  
which yumrepo does not support.

So you can not use yumrepo to simply enable/disable the default repo  
config that comes with FC5.

--jason


On Oct 3, 2006, at 8:56 AM, David Lutterkort wrote:
> On Tue, 2006-10-03 at 08:53 -0700, Jason Dillon wrote:
>> This means you remove all of the .repo files that comes with fedora
>> and replace them, which is one way to fix it.  I was trying to leave
>> the files as they were and only enable/disable as needed.
>
> That''s what the yumrepo type does: if you specify a repo that is  
> already
> defined in one of the yum config files (say ''core''), it
will just
> change
> the already existing definition. So no, it does not replace the .repo
> files that come with fedora, it edits them. Only when your yumrepo  
> type
> mentions a repo that is not defined yet is a new .repo file created.
>
> David
>
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users

Fedora Core 2 (yes, I know laugh at me as I defend myself with febel words
like "but the clients ...")  comes with all the repos in one file, and
I am
using the example provided by David with much success (BTW, thank you David
for everything you''re doing).

In FC5 the repos are distributed throughout many files, as you know,
fedora-core.repo contains 3 in the default.  Again using the example I was
able to configure them just fine.  I have FC[2,4,5] in production right now
all using the same yumrepos.

More-over, you can just nuke everything in /etc/yum.repos.d and let yumrepo
create them.  Effectively I''ve done this for systems that were not
managed
by yum by using puppet to install yum and creating the repo data with
yumrepo, at which point you can edit the files by hand or what have you.

-r''


On 10/3/06, Jason Dillon <jason@planet57.com>
wrote:
>
> Um, you are not getting my point.
>
> yumrepo can only work on a single repo defined in one file.
>
> Fedora Core 5 by default comes with many repos defined in one file,
> which yumrepo does not support.
>
> So you can not use yumrepo to simply enable/disable the default repo
> config that comes with FC5.
>
> --jason
>
>
> On Oct 3, 2006, at 8:56 AM, David Lutterkort wrote:
>
> > On Tue, 2006-10-03 at 08:53 -0700, Jason Dillon wrote:
> >> This means you remove all of the .repo files that comes with
fedora
> >> and replace them, which is one way to fix it.  I was trying to
leave
> >> the files as they were and only enable/disable as needed.
> >
> > That''s what the yumrepo type does: if you specify a repo that
is
> > already
> > defined in one of the yum config files (say ''core''),
it will just
> > change
> > the already existing definition. So no, it does not replace the .repo
> > files that come with fedora, it edits them. Only when your yumrepo
> > type
> > mentions a repo that is not defined yet is a new .repo file created.
> >
> > David
> >
> >
> > _______________________________________________
> > Puppet-users mailing list
> > Puppet-users@madstop.com
> > https://mail.madstop.com/mailman/listinfo/puppet-users
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
>

_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

On Tue, 2006-10-03 at 09:06 -0700, Jason Dillon wrote:
> Um, you are not getting my point.
I am exactly getting your point, and I have spent quite a bit of time
making the yumrepo type work with the FC5 yum config files the way you
have been describing. 

I use that type to manage FC5 and rawhide machines, and it does edit the
files, not replace them, delete them, separate them into one file per
repo or anything else.
> yumrepo can only work on a single repo defined in one file.
> 
> Fedora Core 5 by default comes with many repos defined in one file,  
> which yumrepo does not support.
> 
> So you can not use yumrepo to simply enable/disable the default repo  
> config that comes with FC5.
> 
> --jason
> 
> 
> On Oct 3, 2006, at 8:56 AM, David Lutterkort wrote:
> 
> > On Tue, 2006-10-03 at 08:53 -0700, Jason Dillon wrote:
> >> This means you remove all of the .repo files that comes with
fedora
> >> and replace them, which is one way to fix it.  I was trying to
leave
> >> the files as they were and only enable/disable as needed.
> >
> > That''s what the yumrepo type does: if you specify a repo that
is
> > already
> > defined in one of the yum config files (say ''core''),
it will just
> > change
> > the already existing definition. So no, it does not replace the .repo
> > files that come with fedora, it edits them. Only when your yumrepo  
> > type
> > mentions a repo that is not defined yet is a new .repo file created.
> >
> > David
> >
> >
> > _______________________________________________
> > Puppet-users mailing list
> > Puppet-users@madstop.com
> > https://mail.madstop.com/mailman/listinfo/puppet-users
> 
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users