Hi,
I''m in the process of evaluating puppet and cfengine for a rollout on
a mix of in-house servers and managed customer servers. I''ve always
been a pro-cfengine man, and it''s already running on a small setup of
5 machines. But even though I have pretty clear idea of how I should
proceed building the system, I''m intimidated by the vast amount of
redundant lines of code I need to write to get the system to the
state where I want it.
In enters puppet. I noticed it''s existence when Luke announced it and
immediately thought: "Oh no, not another tool to solve the same
problem!" Sorry Luke! After looking more thouroughly on it, and
trying it out, I''m convinced that you have done a really good job of
addressing the shortcomings of cfengine. Right now I''m trying it out
on a single host, moving to multihost testing soon.
But before I take that step, I want to see that puppet can accomplish
some more of the tasks that I need this tool to do:
1. Security updates on a specific time of the day. I think I can do
this with:
schedule { maintenance:
range => "6:30 - 8:30"
}
and set this schedule on every package definition I make:
package { foo:
ensure => latest,
schedule => maintenance
}
But I also want to be able to do initial host installs any time of
the day, and have puppet install all the packages required for the
host to fulfill it''s purpose. Can I do this with tagging, or...?
2. Do puppet have any feature that resembles cfengines SplayTime
function?
3. I''m using shorewall on my servers, and want to be able to make a
rules file out of several files. For that I''m using puppet to
distribute the files, and make to assemble them into a rules file.
After copying the separate files with puppet, I''m using exec to run
the make part - but I can''t get it to work. Here''s the .pp
file:
<snip>
class shorewall {
package { shorewall:
ensure => latest
}
file { "/etc/shorewall/rulefiles":
ensure => directory,
mode => 750
}
configfile { "/etc/shorewall/shorewall.conf":
mode => 0640,
source => "config/apps/shorewall/shorewall.conf",
require => package["shorewall"]
}
configfile { "/etc/shorewall/Makefile":
source => "config/apps/shorewall/Makefile",
require => package["shorewall"]
}
configfile { "/etc/shorewall/rulefiles/":
mode => 0640,
source => "config/apps/shorewall/rulefiles/",
recurse => inf,
require => package["shorewall"]
}
configfile { "/etc/shorewall/infaces":
mode => 0640,
source => [
"config/apps/shorewall/infaces.$host",
"config/apps/shorewall/infaces"
],
require => package["shorewall"]
}
configfile { "/etc/shorewall/zones":
mode => 0640,
source => [
"config/apps/shorewall/zones.$host",
"config/apps/shorewall/zones"
],
require => package["shorewall"]
}
configfile { "/etc/shorewall/policy":
mode => 0640,
source => [
"config/apps/shorewall/policy.$host",
"config/apps/shorewall/policy"
],
require => package["shorewall"]
}
exec { make:
path => "/usr/bin",
cwd => "/etc/shorewall/",
subscribe => file["/etc/shorewall/rulefiles/"],
require => file["/etc/shorewall/Makefile"],
refreshonly => true
}
}
<snap>
And the Makefile:
<snip>
RULEFILES := $(shell ls /etc/shorewall/rulefiles/*.rules)
all: rules
rules: $(RULEFILES)
@echo "Updating rules file..."
@cat $(CONFDIR)/rulefiles/*.rules > rules
<snap>
Apparently the exec part runs as it should, but it results in an
empty rules file. And I can''t see why. If I run make manually in /etc/
shorewall, the rules file is created correctly. I hope someone can
shed some light on where I''ve done wrong.
And once again, Luke, you''ve done a great job with puppet!
--
Med venlig hilsen
Juri Rischel Jensen
Fab:IT ApS
Vesterbrogade 50
DK-1620 København
Tlf: 70 202 407 / Fax: 33 313 640
www.fab-it.dk / juri@fab-it.dk
Juri Rischel Jensen wrote:> Hi, > > I''m in the process of evaluating puppet and cfengine for a rollout on > a mix of in-house servers and managed customer servers. I''ve always > been a pro-cfengine man, and it''s already running on a small setup of > 5 machines. But even though I have pretty clear idea of how I should > proceed building the system, I''m intimidated by the vast amount of > redundant lines of code I need to write to get the system to the > state where I want it. > > In enters puppet. I noticed it''s existence when Luke announced it and > immediately thought: "Oh no, not another tool to solve the same > problem!" Sorry Luke! After looking more thouroughly on it, and > trying it out, I''m convinced that you have done a really good job of > addressing the shortcomings of cfengine. Right now I''m trying it out > on a single host, moving to multihost testing soon.Great, glad to hear it.> But before I take that step, I want to see that puppet can accomplish > some more of the tasks that I need this tool to do: > > 1. Security updates on a specific time of the day. I think I can do > this with: > > schedule { maintenance: > range => "6:30 - 8:30" > } > > and set this schedule on every package definition I make: > > package { foo: > ensure => latest, > schedule => maintenance > }Yes, that is correct.> But I also want to be able to do initial host installs any time of > the day, and have puppet install all the packages required for the > host to fulfill it''s purpose. Can I do this with tagging, or...?During installs, just start puppetd with --ignoreschedules. This will cause it to, well, ignore schedules. In other words, all objects will run, whether they match their schedule or not.> 2. Do puppet have any feature that resembles cfengines SplayTime > function?It does not, although it could be added relatively easily. I decided against implementing it because it has such a negative impact on usability -- e.g., every time you need to run cfagent, you have to disable splaytime, and it affects even smaller sites that won''t have performance problems. Instead, Puppet relies on the fact that puppetd is a long-running daemon, and each of your daemons will tend to start at slightly different times, and they will thus tend to spread their load over the run interval. I''ve thought about other mechanisms for providing something load balancing, such as a simple method to check the load on the server and have clients automatically throttle back on high load, but until I start hearing performance complaints I''m not going to bother.> 3. I''m using shorewall on my servers, and want to be able to make a > rules file out of several files. For that I''m using puppet to > distribute the files, and make to assemble them into a rules file. > After copying the separate files with puppet, I''m using exec to run > the make part - but I can''t get it to work. Here''s the .pp file: > > <snip> > class shorewall { > package { shorewall: > ensure => latest > } > file { "/etc/shorewall/rulefiles": > ensure => directory, > mode => 750 > } > configfile { "/etc/shorewall/shorewall.conf": > mode => 0640, > source => "config/apps/shorewall/shorewall.conf", > require => package["shorewall"] > } > configfile { "/etc/shorewall/Makefile": > source => "config/apps/shorewall/Makefile", > require => package["shorewall"] > } > configfile { "/etc/shorewall/rulefiles/": > mode => 0640, > source => "config/apps/shorewall/rulefiles/", > recurse => inf, > require => package["shorewall"] > } > configfile { "/etc/shorewall/infaces": > mode => 0640, > source => [ > "config/apps/shorewall/infaces.$host", > "config/apps/shorewall/infaces" > ], > require => package["shorewall"] > }I recommend creating a definition for this, since it''s a repeating pattern: define swconfig { configfile { "/etc/shorewall/$name": mode => 0640, source => [ "config/apps/shorewall/$name.$host", "config/apps/shorewall/$name" ] } } Note I got rid of the require; you''ll need that before you install the rules, but not to copy the files down. Then you can just use that: # Call the swconfig with no attributes swconfig { [infaces, zones, policy]: } Instead of all the different calls to ''configfile'' you have.> exec { make: > path => "/usr/bin", > cwd => "/etc/shorewall/", > subscribe => file["/etc/shorewall/rulefiles/"], > require => file["/etc/shorewall/Makefile"], > refreshonly => true > } > } > > <snap> > > And the Makefile: > > <snip> > RULEFILES := $(shell ls /etc/shorewall/rulefiles/*.rules) > > all: rules > > rules: $(RULEFILES) > @echo "Updating rules file..." > @cat $(CONFDIR)/rulefiles/*.rules > rules > > <snap>Where is CONFDIR set?> Apparently the exec part runs as it should, but it results in an > empty rules file. And I can''t see why. If I run make manually in /etc/ > shorewall, the rules file is created correctly. I hope someone can > shed some light on where I''ve done wrong.Is it possible that CONFDIR is set in your shell environment, but make doesn''t know what that directory is when it is run out of Puppet?> And once again, Luke, you''ve done a great job with puppet!Thanks. :) -- I do not feel obliged to believe that the same God who has endowed us with sense, reason, and intellect has intended us to forgo their use. -- Galileo Galilei --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
On Oct 9, 2006, at 22:45, Luke Kanies wrote:> Juri Rischel Jensen wrote:<snip>> During installs, just start puppetd with --ignoreschedules. This will > cause it to, well, ignore schedules. In other words, all objects will > run, whether they match their schedule or not.This got me thinking: What about general security updates, eg. for all the libraries, dependency programs etc? Can puppet handle them as well...? Or do I have to make an exec statement of some sort to handle that...? One more question that comes to mind: Does puppet run "apt-get update" before every run with "package" statements...?>> 2. Do puppet have any feature that resembles cfengines SplayTime >> function? > > It does not, although it could be added relatively easily. I decided > against implementing it because it has such a negative impact on > usability -- e.g., every time you need to run cfagent, you have to > disable splaytime, and it affects even smaller sites that won''t have > performance problems. > > Instead, Puppet relies on the fact that puppetd is a long-running > daemon, and each of your daemons will tend to start at slightly > different times, and they will thus tend to spread their load over the > run interval.Ok. This seems reasonable to me, and should be Good Enough For Me (tm)... ;-) <snip>> I recommend creating a definition for this, since it''s a repeating > pattern: > > define swconfig { > configfile { "/etc/shorewall/$name": > mode => 0640, > source => [ > "config/apps/shorewall/$name.$host", > "config/apps/shorewall/$name" > ] > } > } > > Note I got rid of the require; you''ll need that before you install the > rules, but not to copy the files down. > > Then you can just use that: > > # Call the swconfig with no attributes > swconfig { [infaces, zones, policy]: } > > Instead of all the different calls to ''configfile'' you have.Wauv, that really cut down on the size of that file! Just what I want: To be able to type less to do more... ;-) Thanks! <snip>>> And the Makefile: >> >> <snip> >> RULEFILES := $(shell ls /etc/shorewall/rulefiles/*.rules) >> >> all: rules >> >> rules: $(RULEFILES) >> @echo "Updating rules file..." >> @cat $(CONFDIR)/rulefiles/*.rules > rules >> >> <snap> > > Where is CONFDIR set?Actually the line should read "@cat /etc/shorewall/rulefiles/*.rules > rules" as I suspected that exact variable to be causing the problem; but it weren''t. It still doesn''t work.>> Apparently the exec part runs as it should, but it results in an >> empty rules file. And I can''t see why. If I run make manually in / >> etc/ >> shorewall, the rules file is created correctly. I hope someone can >> shed some light on where I''ve done wrong. > > Is it possible that CONFDIR is set in your shell environment, but make > doesn''t know what that directory is when it is run out of Puppet?No, unfortunately not. -- Med venlig hilsen Juri Rischel Jensen Fab:IT ApS Vesterbrogade 50 DK-1620 København Tlf: 70 202 407 / Fax: 33 313 640 www.fab-it.dk / juri@fab-it.dk
Juri Rischel Jensen wrote:> On Oct 9, 2006, at 22:45, Luke Kanies wrote: > >> Juri Rischel Jensen wrote: > > <snip> >> During installs, just start puppetd with --ignoreschedules. This will >> cause it to, well, ignore schedules. In other words, all objects will >> run, whether they match their schedule or not. > > This got me thinking: What about general security updates, eg. for > all the libraries, dependency programs etc? Can puppet handle them as > well...? Or do I have to make an exec statement of some sort to > handle that...?It mostly just depends on how you want to manage them. I''d expect you''ll want to just do an apt-get upgrade periodically, in which case it makes more sense to just have an exec statement handle that. If you want to specifically manage the state of a given package, or of all packages, then you use Puppet''s support for packages.> One more question that comes to mind: Does puppet run "apt-get > update" before every run with "package" statements...?No -- it''s up to you to determine how often you want to update.> Wauv, that really cut down on the size of that file! Just what I > want: To be able to type less to do more... ;-) Thanks!Glad to help.> Actually the line should read "@cat /etc/shorewall/rulefiles/*.rules > > rules" as I suspected that exact variable to be causing the > problem; but it weren''t. It still doesn''t work.Ok. Have you tried running the exec with ''logoutput => true'' set? That should at least tell you what make is saying. Are you getting a failure from make? -- What happens to the hole when the cheese is gone? -- Bertolt Brecht --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
Hi Luke On Oct 10, 2006, at 17:21, Luke Kanies wrote:> Juri Rischel Jensen wrote: >> On Oct 9, 2006, at 22:45, Luke Kanies wrote: >> >>> Juri Rischel Jensen wrote: >> >> <snip> >>> During installs, just start puppetd with --ignoreschedules. This >>> will >>> cause it to, well, ignore schedules. In other words, all objects >>> will >>> run, whether they match their schedule or not. >> >> This got me thinking: What about general security updates, eg. for >> all the libraries, dependency programs etc? Can puppet handle them as >> well...? Or do I have to make an exec statement of some sort to >> handle that...? > > It mostly just depends on how you want to manage them. I''d expect > you''ll want to just do an apt-get upgrade periodically, in which > case it > makes more sense to just have an exec statement handle that. If you > want to specifically manage the state of a given package, or of all > packages, then you use Puppet''s support for packages.Ok. I like it that way. Is there any way to set a default on a type, eg. define a general schedule for packages, without specifying "schedule => maintance" on all package statements, or subclassing like "class remotefile { file {..." etc. ??>> One more question that comes to mind: Does puppet run "apt-get >> update" before every run with "package" statements...? > > No -- it''s up to you to determine how often you want to update.Ok.>> Wauv, that really cut down on the size of that file! Just what I >> want: To be able to type less to do more... ;-) Thanks! > > Glad to help. > >> Actually the line should read "@cat /etc/shorewall/rulefiles/*.rules >>> rules" as I suspected that exact variable to be causing the >> problem; but it weren''t. It still doesn''t work. > > Ok. Have you tried running the exec with ''logoutput => true'' set? > That should at least tell you what make is saying. Are you getting a > failure from make?Yes. I hadn''t specified /bin in the path, and make couldn''t find ls... doh! ;-) It''s now fixed. It''s a cool feature that you can turn on debug for a specific part of the code... -- Med venlig hilsen Juri Rischel Jensen Fab:IT ApS Vesterbrogade 50 DK-1620 København Tlf: 70 202 407 / Fax: 33 313 640 www.fab-it.dk / juri@fab-it.dk
Juri Rischel Jensen wrote:> > Ok. I like it that way. Is there any way to set a default on a type, > eg. define a general schedule for packages, without specifying > "schedule => maintance" on all package statements, or subclassing > like "class remotefile { file {..." etc. ??Yeah: Package { schedule => maintenance } The capitalized form with no name affects all instances of that type. These defaults inherit via the scopes, so if you want them to affect your whole configuration you should put them in the top-level scope in the site.pp file or something similar.>> Ok. Have you tried running the exec with ''logoutput => true'' set? >> That should at least tell you what make is saying. Are you getting a >> failure from make? > > Yes. I hadn''t specified /bin in the path, and make couldn''t find > ls... doh! ;-) > > It''s now fixed. It''s a cool feature that you can turn on debug for a > specific part of the code...Okay, glad to hear it works. -- This space intentionally has nothing but text explaining why this space has nothing but text explaining that this space would otherwise have been left blank, and would otherwise have been left blank. --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com